2024-04-24T06:15:49.859 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T06:15:49.859 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} -- /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/orig.config.yaml 2024-04-24T06:15:49.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T06:15:49.993 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164 branch: wip-rishabh-testing-20240416.193735-5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: null first_in_suite: false job_id: '7671164' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-5 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCem8jKKgS1SP/G8/wY/oFvc/9wAUb1j7OEHKfIficKsbklLvdU+qnp/nZZiqAI3qiwxNqLNR18LL0xOOMGmgTA= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeJ6fIitlL1bfgWDiHLCLXODFveauWBlpHqh8YlI3t2I0rjmRWb1wrpdsPz2n6Dkv1JwMPEIugOab5eN+zmTkc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_05:22:11 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T06:15:49.993 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa; will attempt to use it 2024-04-24T06:15:49.994 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks 2024-04-24T06:15:49.994 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T06:15:49.995 INFO:teuthology.task.internal:Checking packages... 2024-04-24T06:15:50.021 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411' 2024-04-24T06:15:50.021 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T06:15:50.021 INFO:teuthology.packaging:ref: None 2024-04-24T06:15:50.021 INFO:teuthology.packaging:tag: None 2024-04-24T06:15:50.021 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-5 2024-04-24T06:15:50.021 INFO:teuthology.packaging:sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:15:50.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-5 2024-04-24T06:15:50.238 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3124.ga3c8a22b 2024-04-24T06:15:50.240 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T06:15:50.246 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T06:15:50.246 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T06:15:50.362 INFO:teuthology.task.internal:Saving configuration 2024-04-24T06:15:50.379 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T06:15:50.385 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:15:50.405 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:09:55.922245', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCem8jKKgS1SP/G8/wY/oFvc/9wAUb1j7OEHKfIficKsbklLvdU+qnp/nZZiqAI3qiwxNqLNR18LL0xOOMGmgTA='} 2024-04-24T06:15:50.425 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:09:55.923985', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeJ6fIitlL1bfgWDiHLCLXODFveauWBlpHqh8YlI3t2I0rjmRWb1wrpdsPz2n6Dkv1JwMPEIugOab5eN+zmTkc='} 2024-04-24T06:15:50.425 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T06:15:50.431 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T06:15:50.431 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T06:15:50.431 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T06:15:50.496 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9fcffd7160>, signals=[15]) 2024-04-24T06:15:50.496 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T06:15:50.503 INFO:teuthology.task.internal:Opening connections... 2024-04-24T06:15:50.503 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:15:50.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:15:50.584 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:15:50.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:15:50.661 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T06:15:50.668 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-04-24T06:15:50.687 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-04-24T06:15:50.687 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-04-24T06:15:50.746 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T06:15:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T06:15:50.748 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-04-24T06:15:50.782 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-04-24T06:15:50.800 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-04-24T06:15:50.801 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-04-24T06:15:50.858 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-04-24T06:15:50.858 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-04-24T06:15:50.858 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T06:15:50.859 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T06:15:50.860 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-04-24T06:15:50.882 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T06:15:50.892 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T06:15:50.898 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T06:15:50.898 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-04-24T06:15:50.901 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-04-24T06:15:50.917 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T06:15:50.923 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T06:15:50.923 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-04-24T06:15:50.958 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-04-24T06:15:50.983 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T06:15:51.036 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T06:15:51.051 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T06:15:51.052 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-24T06:15:51.052 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T06:15:51.052 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T06:15:51.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:15:51.070 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.070 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T06:15:51.070 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-24T06:15:51.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:15:51.071 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.071 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T06:15:51.071 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-04-24T06:15:51.128 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-437.el9.x86_64 2024-04-24T06:15:51.128 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.128 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T06:15:51.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-437.el9.x86_64 2024-04-24T06:15:51.132 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.132 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T06:15:51.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:15:51.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum upgrade -y linux-firmware 2024-04-24T06:15:51.348 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum upgrade -y linux-firmware 2024-04-24T06:15:53.593 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T06:15:54.265 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-04-24T06:15:55.369 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-24T06:16:00.116 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-04-24T06:16:04.408 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T06:16:06.878 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 815 kB/s | 6.2 MB 00:07 2024-04-24T06:16:07.085 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-24T06:16:08.102 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-24T06:16:08.512 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-24T06:16:09.438 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-24T06:16:13.485 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-24T06:16:14.688 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-24T06:16:15.521 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-24T06:16:15.522 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-04-24T06:16:15.522 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-24T06:16:15.616 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T06:16:15.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:16:15.760 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T06:16:15.761 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T06:16:15.763 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi059.front.sepia.ceph.com to complete... 2024-04-24T06:16:16.791 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-24T06:16:16.792 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-04-24T06:16:16.792 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-24T06:16:16.901 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T06:16:16.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:16:17.139 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T06:16:17.139 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T06:16:17.141 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi066.front.sepia.ceph.com to complete... 2024-04-24T06:16:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24 06:16:31 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T06:16:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:FINISHED --2024-04-24 06:16:31-- 2024-04-24T06:16:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:Total wall clock time: 16s 2024-04-24T06:16:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:Downloaded: 1 files, 341M in 16s (21.8 MB/s) 2024-04-24T06:16:31.696 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T06:16:31.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T06:16:33.891 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying... ######################################## 2024-04-24T06:16:33.995 INFO:teuthology.orchestra.run.smithi059.stdout:Preparing... ######################################## 2024-04-24T06:16:34.116 INFO:teuthology.orchestra.run.smithi059.stdout:Updating / installing... 2024-04-24T06:16:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24 06:16:39 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T06:16:39.609 INFO:teuthology.orchestra.run.smithi066.stderr:FINISHED --2024-04-24 06:16:39-- 2024-04-24T06:16:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:Total wall clock time: 22s 2024-04-24T06:16:39.610 INFO:teuthology.orchestra.run.smithi066.stderr:Downloaded: 1 files, 341M in 22s (15.5 MB/s) 2024-04-24T06:16:39.612 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T06:16:39.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T06:16:41.801 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying... ######################################## 2024-04-24T06:16:41.934 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing... ######################################## 2024-04-24T06:16:42.057 INFO:teuthology.orchestra.run.smithi066.stdout:Updating / installing... 2024-04-24T06:16:42.493 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T06:16:50.341 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T06:17:22.124 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T06:17:22.124 DEBUG:teuthology.orchestra.run.smithi059:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T06:17:22.220 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.220 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.220 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T06:17:22.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T06:17:22.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T06:17:22.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T06:17:22.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T06:17:22.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T06:17:22.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T06:17:22.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T06:17:22.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T06:17:22.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T06:17:22.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T06:17:22.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T06:17:22.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T06:17:22.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T06:17:22.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T06:17:22.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T06:17:22.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T06:17:22.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T06:17:22.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T06:17:22.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T06:17:22.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T06:17:22.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T06:17:22.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T06:17:22.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T06:17:22.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T06:17:22.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T06:17:22.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T06:17:22.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T06:17:22.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T06:17:22.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T06:17:22.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T06:17:22.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T06:17:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T06:17:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T06:17:22.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T06:17:22.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T06:17:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T06:17:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T06:17:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T06:17:22.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T06:17:22.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T06:17:22.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T06:17:22.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T06:17:22.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T06:17:22.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T06:17:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T06:17:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T06:17:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T06:17:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T06:17:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T06:17:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T06:17:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T06:17:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T06:17:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T06:17:22.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T06:17:22.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T06:17:22.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T06:17:22.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T06:17:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T06:17:22.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T06:17:22.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T06:17:22.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T06:17:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T06:17:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T06:17:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T06:17:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T06:17:22.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T06:17:22.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T06:17:22.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T06:17:22.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T06:17:22.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T06:17:22.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T06:17:22.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T06:17:22.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T06:17:22.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T06:17:22.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T06:17:22.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T06:17:22.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T06:17:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T06:17:22.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T06:17:22.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T06:17:22.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T06:17:22.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T06:17:22.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T06:17:22.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T06:17:22.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T06:17:22.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T06:17:22.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T06:17:22.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T06:17:22.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T06:17:22.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T06:17:22.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T06:17:22.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T06:17:22.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T06:17:22.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T06:17:22.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T06:17:22.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T06:17:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T06:17:22.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T06:17:22.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T06:17:22.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T06:17:22.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T06:17:22.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T06:17:22.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T06:17:22.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T06:17:22.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T06:17:22.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T06:17:22.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T06:17:22.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T06:17:22.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T06:17:22.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T06:17:22.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T06:17:22.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T06:17:22.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T06:17:22.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T06:17:22.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T06:17:22.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T06:17:22.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T06:17:22.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T06:17:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T06:17:22.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T06:17:22.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T06:17:22.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T06:17:22.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T06:17:22.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T06:17:22.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T06:17:22.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T06:17:22.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T06:17:22.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T06:17:22.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T06:17:22.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T06:17:22.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T06:17:22.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T06:17:22.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T06:17:22.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T06:17:22.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T06:17:22.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T06:17:22.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T06:17:22.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T06:17:22.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T06:17:22.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:17:22.365 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.366 DEBUG:teuthology.orchestra.run.smithi059:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout:for file in vmlinuz System.map config; do 2024-04-24T06:17:22.390 INFO:teuthology.orchestra.run.smithi059.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: fi 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-24T06:17:22.391 INFO:teuthology.orchestra.run.smithi059.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T06:17:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:17:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.392 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-24T06:17:22.392 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-04-24T06:17:22.473 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 68.el9 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8264254 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T06:17:22.474 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:17:22.475 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T06:17:22.476 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T06:17:22.476 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:22.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T06:17:23.088 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-04-24T06:17:24.525 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T06:17:24.562 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-04-24T06:17:24.565 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-04-24T06:17:24.603 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.XJpwDZ3tCZ 2024-04-24T06:17:24.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XJpwDZ3tCZ 2024-04-24T06:17:24.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.XJpwDZ3tCZ 2024-04-24T06:17:24.803 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.XJpwDZ3tCZ is 6KB 2024-04-24T06:17:24.851 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.XJpwDZ3tCZ 2024-04-24T06:17:24.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T06:17:24.927 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T06:17:24.927 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T06:17:24.927 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T06:17:24.927 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T06:17:24.929 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:25.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-04-24T06:17:27.908 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T06:17:27.908 DEBUG:teuthology.orchestra.run.smithi066:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T06:17:27.988 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T06:17:27.989 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T06:17:27.990 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T06:17:27.991 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T06:17:27.992 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T06:17:27.993 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T06:17:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T06:17:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T06:17:27.996 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T06:17:27.997 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T06:17:27.998 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T06:17:27.999 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T06:17:28.000 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T06:17:28.001 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T06:17:28.002 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T06:17:28.003 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T06:17:28.004 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T06:17:28.005 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T06:17:28.006 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T06:17:28.007 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T06:17:28.008 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T06:17:28.009 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T06:17:28.010 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T06:17:28.011 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T06:17:28.012 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T06:17:28.013 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T06:17:28.014 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T06:17:28.015 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T06:17:28.016 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T06:17:28.017 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T06:17:28.018 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T06:17:28.018 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T06:17:28.018 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T06:17:28.018 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T06:17:28.018 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T06:17:28.019 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T06:17:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T06:17:28.021 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T06:17:28.022 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T06:17:28.023 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T06:17:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T06:17:28.025 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T06:17:28.026 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T06:17:28.027 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T06:17:28.028 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T06:17:28.029 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T06:17:28.030 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T06:17:28.031 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T06:17:28.032 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T06:17:28.033 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T06:17:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T06:17:28.035 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T06:17:28.036 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T06:17:28.037 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T06:17:28.038 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T06:17:28.039 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T06:17:28.040 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T06:17:28.041 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T06:17:28.042 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T06:17:28.043 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T06:17:28.044 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T06:17:28.045 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T06:17:28.046 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T06:17:28.047 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T06:17:28.048 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T06:17:28.049 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T06:17:28.050 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T06:17:28.051 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T06:17:28.052 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T06:17:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T06:17:28.054 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T06:17:28.055 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T06:17:28.056 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T06:17:28.057 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T06:17:28.058 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T06:17:28.059 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T06:17:28.060 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T06:17:28.061 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T06:17:28.062 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T06:17:28.063 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T06:17:28.064 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T06:17:28.065 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T06:17:28.066 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T06:17:28.067 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T06:17:28.068 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T06:17:28.069 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T06:17:28.070 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T06:17:28.071 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T06:17:28.072 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T06:17:28.073 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T06:17:28.074 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T06:17:28.075 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T06:17:28.076 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T06:17:28.077 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T06:17:28.078 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T06:17:28.079 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T06:17:28.080 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T06:17:28.081 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T06:17:28.082 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T06:17:28.083 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T06:17:28.084 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T06:17:28.085 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T06:17:28.086 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T06:17:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T06:17:28.088 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T06:17:28.089 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T06:17:28.090 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T06:17:28.091 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T06:17:28.092 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T06:17:28.093 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T06:17:28.094 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T06:17:28.095 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T06:17:28.096 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T06:17:28.097 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T06:17:28.098 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T06:17:28.099 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T06:17:28.100 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T06:17:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T06:17:28.102 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T06:17:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T06:17:28.104 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T06:17:28.105 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T06:17:28.106 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T06:17:28.107 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T06:17:28.108 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T06:17:28.109 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T06:17:28.110 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T06:17:28.111 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T06:17:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T06:17:28.113 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T06:17:28.114 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T06:17:28.115 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T06:17:28.116 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T06:17:28.117 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T06:17:28.118 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T06:17:28.119 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T06:17:28.120 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T06:17:28.121 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T06:17:28.122 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T06:17:28.123 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T06:17:28.124 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T06:17:28.125 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T06:17:28.126 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T06:17:28.127 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T06:17:28.128 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T06:17:28.129 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T06:17:28.130 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T06:17:28.130 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T06:17:28.130 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T06:17:28.130 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T06:17:28.130 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:17:28.133 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:28.133 DEBUG:teuthology.orchestra.run.smithi066:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout:for file in vmlinuz System.map config; do 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T06:17:28.157 INFO:teuthology.orchestra.run.smithi066.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout: fi 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:done 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:17:28.158 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:28.159 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2024-04-24T06:17:28.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2024-04-24T06:17:28.239 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2024-04-24T06:17:28.239 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2024-04-24T06:17:28.239 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.06 2024-04-24T06:17:28.239 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 68.el9 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Group : Unspecified 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 8264254 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : builder@centos.org 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : CentOS 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2024-04-24T06:17:28.240 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:17:28.241 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T06:17:28.242 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T06:17:28.242 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:28.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T06:17:28.756 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-04-24T06:17:30.129 INFO:teuthology.orchestra.run.smithi066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T06:17:30.157 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-04-24T06:17:30.159 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-04-24T06:17:30.174 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.X8HAUOg9it 2024-04-24T06:17:30.175 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.X8HAUOg9it 2024-04-24T06:17:30.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.X8HAUOg9it 2024-04-24T06:17:30.375 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.X8HAUOg9it is 6KB 2024-04-24T06:17:30.423 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.X8HAUOg9it 2024-04-24T06:17:30.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T06:17:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T06:17:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T06:17:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T06:17:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T06:17:30.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T06:17:30.649 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2024-04-24T06:17:55.125 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-24T06:17:55.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:17:55.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:18:00.658 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-24T06:18:00.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:18:00.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:18:13.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-04-24T06:18:19.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-04-24T06:18:22.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:18:22.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:18:28.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:18:28.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:18:31.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-04-24T06:18:43.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:18:43.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:18:46.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-04-24T06:19:01.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:19:01.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:19:19.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-04-24T06:19:22.672 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T06:19:34.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:19:34.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:19:34.998 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-24T06:19:35.631 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:19:35.631 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T06:19:35.632 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-24T06:19:35.687 INFO:teuthology.orchestra.run.smithi059.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T06:19:35.687 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T06:19:35.687 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T06:19:35.687 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T06:19:36.688 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T06:19:36.688 DEBUG:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T06:19:36.826 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2024-04-24T06:19:36.881 DEBUG:teuthology.parallel:result is None 2024-04-24T06:19:37.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:19:37.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:19:40.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-04-24T06:20:01.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:20:01.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:20:02.354 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-04-24T06:20:03.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:20:03.047 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T06:20:03.047 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-04-24T06:20:03.100 INFO:teuthology.orchestra.run.smithi066.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T06:20:03.100 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T06:20:03.100 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T06:20:03.100 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T06:20:04.102 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T06:20:04.102 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T06:20:04.233 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-04-24T06:20:04.319 DEBUG:teuthology.parallel:result is None 2024-04-24T06:20:04.320 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T06:20:04.330 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T06:20:04.330 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T06:20:04.338 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T06:20:04.399 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T06:20:04.406 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T06:20:04.413 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T06:20:04.414 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T06:20:04.421 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T06:20:04.498 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T06:20:04.505 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T06:20:04.506 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T06:20:04.513 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T06:20:04.611 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:20:04.611 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:20:04.645 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:20:04.648 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:20:04.652 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T06:20:04.660 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T06:20:04.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T06:20:04.667 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T06:20:04.759 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T06:20:04.769 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T06:20:04.770 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T06:20:04.777 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T06:20:04.834 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:20:04.911 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:20:05.029 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:20:05.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:20:05.211 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:20:05.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T06:20:05.300 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:20:05.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:20:05.507 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:20:05.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:20:05.682 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:20:05.683 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T06:20:05.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-04-24T06:20:05.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-04-24T06:20:05.913 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:20:05.931 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:20:06.234 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T06:20:06.242 INFO:teuthology.task.internal:Starting timer... 2024-04-24T06:20:06.242 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T06:20:06.262 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T06:20:06.271 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-24T06:20:06.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-04-24T06:20:06.393 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-04-24T06:20:06.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-04-24T06:20:06.530 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-04-24T06:20:06.535 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T06:20:06.535 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-04-24T06:20:06.592 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-04-24T06:20:06.592 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-04-24T06:20:06.649 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-04-24T06:20:06.649 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive'} 2024-04-24T06:20:06.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:20:06.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:20:06.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:20:06.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:20:06.810 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T06:20:06.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-04-24T06:20:06.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-04-24T06:20:06.955 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T06:20:06.964 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T06:20:06.970 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T06:20:07.227 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T06:20:07.243 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T06:20:07.244 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi066.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T06:25:27.278 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com')] 2024-04-24T06:25:27.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:25:27.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:25:27.402 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-24T06:25:27.569 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-24T06:25:27.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:25:27.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:25:27.687 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-04-24T06:25:27.853 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-24T06:25:27.853 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T06:25:27.864 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T06:25:27.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T06:25:27.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:25:27.873 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T06:25:27.873 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:25:27.962 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T06:25:27.976 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T06:25:28.005 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T06:25:28.019 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T06:25:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-04-24T06:25:28.096 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-04-24T06:25:28.116 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-04-24T06:25:28.132 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-04-24T06:25:28.159 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T06:25:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T06:25:28.203 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T06:25:28.219 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T06:25:28.350 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.386 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:25:28.388 INFO:teuthology.run_tasks:Running task install... 2024-04-24T06:25:28.397 DEBUG:teuthology.task.install:project ceph 2024-04-24T06:25:28.397 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}} 2024-04-24T06:25:28.397 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'} 2024-04-24T06:25:28.397 INFO:teuthology.task.install:Using flavor: default 2024-04-24T06:25:28.403 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T06:25:28.403 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T06:25:28.403 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T06:25:28.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:25:28.405 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T06:25:28.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:25:28.563 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T06:25:28.563 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T06:25:28.635 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T06:25:28.636 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T06:25:28.676 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T06:25:28.676 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:25:28.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T06:25:28.743 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T06:25:28.743 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:25:28.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T06:25:28.771 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T06:25:28.771 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T06:25:28.841 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T06:25:28.841 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T06:25:28.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T06:25:28.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T06:25:29.028 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T06:25:29.103 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T06:25:29.111 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-04-24T06:25:29.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-04-24T06:25:29.191 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-04-24T06:25:29.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-04-24T06:25:29.442 INFO:teuthology.orchestra.run.smithi059.stdout:48 files removed 2024-04-24T06:25:29.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T06:25:29.532 INFO:teuthology.orchestra.run.smithi066.stdout:48 files removed 2024-04-24T06:25:29.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T06:25:30.418 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 172 kB/s | 83 kB 00:00 2024-04-24T06:25:30.458 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-04-24T06:25:30.861 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-24T06:25:31.033 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-24T06:25:31.444 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-24T06:25:31.484 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-24T06:25:32.230 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T06:25:32.918 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-04-24T06:25:35.302 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-24T06:25:36.092 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-04-24T06:25:39.961 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-24T06:25:41.934 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-24T06:25:42.619 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-24T06:25:43.589 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-24T06:25:49.764 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T06:25:50.766 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 2.2 kB/s | 16 kB 00:07 2024-04-24T06:25:52.250 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:25:52.252 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:25:52.273 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T06:25:52.275 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T06:25:52.276 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T06:25:52.277 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T06:25:52.280 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T06:25:52.283 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T06:25:52.285 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T06:25:52.361 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-24T06:25:52.378 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-24T06:25:52.378 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-04-24T06:25:52.378 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-24T06:25:52.378 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T06:25:52.379 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T06:25:52.380 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T06:25:52.381 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T06:25:52.382 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T06:25:52.383 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T06:25:52.384 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T06:25:52.385 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T06:25:52.386 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T06:25:52.387 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T06:25:52.388 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:25:52.389 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 4 Packages 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 222 M 2024-04-24T06:25:52.390 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-04-24T06:25:52.676 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T06:25:53.649 INFO:teuthology.orchestra.run.smithi059.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-24T06:25:53.841 INFO:teuthology.orchestra.run.smithi059.stdout:(2/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 4.6 MB/s | 890 kB 00:00 2024-04-24T06:25:54.124 INFO:teuthology.orchestra.run.smithi059.stdout:(3/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-04-24T06:25:54.158 INFO:teuthology.orchestra.run.smithi059.stdout:(4/152): ceph-immutable-object-cache-19.0.0-312 459 kB/s | 145 kB 00:00 2024-04-24T06:25:54.350 INFO:teuthology.orchestra.run.smithi059.stdout:(5/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 11 MB/s | 2.4 MB 00:00 2024-04-24T06:25:54.433 INFO:teuthology.orchestra.run.smithi059.stdout:(6/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-04-24T06:25:54.950 INFO:teuthology.orchestra.run.smithi059.stdout:(7/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 7.8 MB/s | 4.7 MB 00:00 2024-04-24T06:25:55.992 INFO:teuthology.orchestra.run.smithi059.stdout:(8/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x86 11 MB/s | 17 MB 00:01 2024-04-24T06:25:56.042 INFO:teuthology.orchestra.run.smithi059.stdout:(9/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el9 516 kB/s | 25 kB 00:00 2024-04-24T06:25:56.234 INFO:teuthology.orchestra.run.smithi059.stdout:(10/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el 8.3 MB/s | 11 MB 00:01 2024-04-24T06:25:56.309 INFO:teuthology.orchestra.run.smithi059.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 3.0 MB/s | 224 kB 00:00 2024-04-24T06:25:56.376 INFO:teuthology.orchestra.run.smithi059.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 412 kB/s | 27 kB 00:00 2024-04-24T06:25:56.444 INFO:teuthology.orchestra.run.smithi059.stdout:(13/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 480 kB/s | 32 kB 00:00 2024-04-24T06:25:56.594 INFO:teuthology.orchestra.run.smithi059.stdout:(14/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 4.9 MB/s | 747 kB 00:00 2024-04-24T06:25:56.677 INFO:teuthology.orchestra.run.smithi059.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 1.9 MB/s | 163 kB 00:00 2024-04-24T06:25:56.769 INFO:teuthology.orchestra.run.smithi059.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.4 MB/s | 127 kB 00:00 2024-04-24T06:25:56.870 INFO:teuthology.orchestra.run.smithi059.stdout:(17/152): libradosstriper1-19.0.0-3124.ga3c8a22 4.9 MB/s | 501 kB 00:00 2024-04-24T06:25:57.553 INFO:teuthology.orchestra.run.smithi059.stdout:(18/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 7.9 MB/s | 5.4 MB 00:00 2024-04-24T06:25:57.662 INFO:teuthology.orchestra.run.smithi059.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 421 kB/s | 45 kB 00:00 2024-04-24T06:25:57.754 INFO:teuthology.orchestra.run.smithi059.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 1.4 MB/s | 135 kB 00:00 2024-04-24T06:25:57.871 INFO:teuthology.orchestra.run.smithi059.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 1.4 MB/s | 162 kB 00:00 2024-04-24T06:25:57.971 INFO:teuthology.orchestra.run.smithi059.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 3.2 MB/s | 322 kB 00:00 2024-04-24T06:25:58.230 INFO:teuthology.orchestra.run.smithi059.stdout:(23/152): ceph-common-19.0.0-3124.ga3c8a22b.el9 4.4 MB/s | 21 MB 00:04 2024-04-24T06:25:58.271 INFO:teuthology.orchestra.run.smithi059.stdout:(24/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 995 kB/s | 297 kB 00:00 2024-04-24T06:25:58.313 INFO:teuthology.orchestra.run.smithi059.stdout:(25/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T06:25:58.338 INFO:teuthology.orchestra.run.smithi059.stdout:(26/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-24T06:25:58.389 INFO:teuthology.orchestra.run.smithi059.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 3.4 MB/s | 170 kB 00:00 2024-04-24T06:25:58.456 INFO:teuthology.orchestra.run.smithi059.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3124.g 519 kB/s | 34 kB 00:00 2024-04-24T06:25:58.515 INFO:teuthology.orchestra.run.smithi059.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 2.4 MB/s | 142 kB 00:00 2024-04-24T06:25:58.844 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T06:25:59.357 INFO:teuthology.orchestra.run.smithi059.stdout:(30/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 15 MB/s | 49 MB 00:03 2024-04-24T06:25:59.440 INFO:teuthology.orchestra.run.smithi059.stdout:(31/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 2.7 MB/s | 3.1 MB 00:01 2024-04-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi059.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 3.4 MB/s | 3.5 MB 00:01 2024-04-24T06:25:59.582 INFO:teuthology.orchestra.run.smithi059.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3124.ga3 1.7 MB/s | 249 kB 00:00 2024-04-24T06:25:59.633 INFO:teuthology.orchestra.run.smithi059.stdout:(34/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 598 kB/s | 49 kB 00:00 2024-04-24T06:25:59.683 INFO:teuthology.orchestra.run.smithi059.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3124.ga 166 kB/s | 16 kB 00:00 2024-04-24T06:25:59.717 INFO:teuthology.orchestra.run.smithi059.stdout:(36/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 3.2 MB/s | 273 kB 00:00 2024-04-24T06:25:59.792 INFO:teuthology.orchestra.run.smithi059.stdout:(37/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 246 kB/s | 18 kB 00:00 2024-04-24T06:25:59.867 INFO:teuthology.orchestra.run.smithi059.stdout:(38/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-24T06:26:00.026 INFO:teuthology.orchestra.run.smithi059.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-24T06:26:00.218 INFO:teuthology.orchestra.run.smithi059.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-04-24T06:26:00.568 INFO:teuthology.orchestra.run.smithi059.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 725 kB/s | 253 kB 00:00 2024-04-24T06:26:00.610 INFO:teuthology.orchestra.run.smithi059.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 335 kB/s | 195 kB 00:00 2024-04-24T06:26:00.702 INFO:teuthology.orchestra.run.smithi059.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 964 kB/s | 803 kB 00:00 2024-04-24T06:26:00.727 INFO:teuthology.orchestra.run.smithi059.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 916 kB/s | 106 kB 00:00 2024-04-24T06:26:00.811 INFO:teuthology.orchestra.run.smithi059.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-24T06:26:00.844 INFO:teuthology.orchestra.run.smithi059.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-24T06:26:00.894 INFO:teuthology.orchestra.run.smithi059.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2024-04-24T06:26:00.928 INFO:teuthology.orchestra.run.smithi059.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-24T06:26:01.028 INFO:teuthology.orchestra.run.smithi059.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-24T06:26:01.079 INFO:teuthology.orchestra.run.smithi059.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-04-24T06:26:01.121 INFO:teuthology.orchestra.run.smithi059.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-24T06:26:01.186 INFO:teuthology.orchestra.run.smithi059.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 137 kB/s | 21 kB 00:00 2024-04-24T06:26:01.221 INFO:teuthology.orchestra.run.smithi059.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-24T06:26:01.263 INFO:teuthology.orchestra.run.smithi059.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-04-24T06:26:01.329 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:26:01.331 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:26:01.347 INFO:teuthology.orchestra.run.smithi059.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-04-24T06:26:01.352 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T06:26:01.353 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T06:26:01.355 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T06:26:01.356 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T06:26:01.359 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T06:26:01.362 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T06:26:01.363 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T06:26:01.389 INFO:teuthology.orchestra.run.smithi059.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-24T06:26:01.440 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-24T06:26:01.456 INFO:teuthology.orchestra.run.smithi059.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-24T06:26:01.462 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-24T06:26:01.462 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T06:26:01.463 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T06:26:01.464 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-04-24T06:26:01.465 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T06:26:01.466 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T06:26:01.467 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T06:26:01.468 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T06:26:01.469 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T06:26:01.470 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T06:26:01.471 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T06:26:01.472 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T06:26:01.473 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-24T06:26:01.474 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-24T06:26:01.475 INFO:teuthology.orchestra.run.smithi066.stdout:Install 148 Packages 2024-04-24T06:26:01.475 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 4 Packages 2024-04-24T06:26:01.475 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:26:01.475 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 222 M 2024-04-24T06:26:01.475 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-24T06:26:01.523 INFO:teuthology.orchestra.run.smithi059.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2024-04-24T06:26:01.607 INFO:teuthology.orchestra.run.smithi059.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-24T06:26:01.648 INFO:teuthology.orchestra.run.smithi059.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-04-24T06:26:01.715 INFO:teuthology.orchestra.run.smithi059.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-04-24T06:26:01.757 INFO:teuthology.orchestra.run.smithi059.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 170 kB/s | 25 kB 00:00 2024-04-24T06:26:01.799 INFO:teuthology.orchestra.run.smithi059.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 317 kB/s | 47 kB 00:00 2024-04-24T06:26:01.833 INFO:teuthology.orchestra.run.smithi059.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-24T06:26:01.866 INFO:teuthology.orchestra.run.smithi059.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T06:26:01.925 INFO:teuthology.orchestra.run.smithi059.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 282 kB/s | 35 kB 00:00 2024-04-24T06:26:02.175 INFO:teuthology.orchestra.run.smithi059.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.4 MB/s | 1.6 MB 00:00 2024-04-24T06:26:02.325 INFO:teuthology.orchestra.run.smithi059.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-04-24T06:26:02.442 INFO:teuthology.orchestra.run.smithi059.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-24T06:26:02.721 INFO:teuthology.orchestra.run.smithi066.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-24T06:26:03.213 INFO:teuthology.orchestra.run.smithi066.stdout:(2/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 6.4 MB/s | 5.4 MB 00:00 2024-04-24T06:26:03.330 INFO:teuthology.orchestra.run.smithi066.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 1.2 MB/s | 145 kB 00:00 2024-04-24T06:26:03.564 INFO:teuthology.orchestra.run.smithi066.stdout:(4/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 10 MB/s | 2.4 MB 00:00 2024-04-24T06:26:03.731 INFO:teuthology.orchestra.run.smithi066.stdout:(5/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 883 kB/s | 890 kB 00:01 2024-04-24T06:26:03.781 INFO:teuthology.orchestra.run.smithi066.stdout:(6/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-24T06:26:04.018 INFO:teuthology.orchestra.run.smithi059.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 3.7 MB/s | 5.8 MB 00:01 2024-04-24T06:26:04.148 INFO:teuthology.orchestra.run.smithi066.stdout:(7/152): ceph-common-19.0.0-3124.ga3c8a22b.el9. 12 MB/s | 21 MB 00:01 2024-04-24T06:26:04.159 INFO:teuthology.orchestra.run.smithi059.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-04-24T06:26:04.268 INFO:teuthology.orchestra.run.smithi059.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 815 kB/s | 87 kB 00:00 2024-04-24T06:26:04.977 INFO:teuthology.orchestra.run.smithi059.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.4 MB/s | 6.0 MB 00:00 2024-04-24T06:26:05.032 INFO:teuthology.orchestra.run.smithi066.stdout:(8/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el9 12 MB/s | 11 MB 00:00 2024-04-24T06:26:05.082 INFO:teuthology.orchestra.run.smithi066.stdout:(9/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el9 514 kB/s | 25 kB 00:00 2024-04-24T06:26:05.093 INFO:teuthology.orchestra.run.smithi059.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-04-24T06:26:05.210 INFO:teuthology.orchestra.run.smithi059.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-24T06:26:05.332 INFO:teuthology.orchestra.run.smithi066.stdout:(10/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x8 11 MB/s | 17 MB 00:01 2024-04-24T06:26:05.369 INFO:teuthology.orchestra.run.smithi059.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 163 kB/s | 26 kB 00:00 2024-04-24T06:26:05.390 INFO:teuthology.orchestra.run.smithi066.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 3.9 MB/s | 224 kB 00:00 2024-04-24T06:26:05.441 INFO:teuthology.orchestra.run.smithi066.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 550 kB/s | 27 kB 00:00 2024-04-24T06:26:05.491 INFO:teuthology.orchestra.run.smithi066.stdout:(13/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 653 kB/s | 32 kB 00:00 2024-04-24T06:26:05.569 INFO:teuthology.orchestra.run.smithi059.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 96 kB/s | 19 kB 00:00 2024-04-24T06:26:05.599 INFO:teuthology.orchestra.run.smithi066.stdout:(14/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 6.8 MB/s | 747 kB 00:00 2024-04-24T06:26:05.687 INFO:teuthology.orchestra.run.smithi059.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-24T06:26:05.705 INFO:teuthology.orchestra.run.smithi066.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 1.5 MB/s | 163 kB 00:00 2024-04-24T06:26:05.783 INFO:teuthology.orchestra.run.smithi066.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.6 MB/s | 127 kB 00:00 2024-04-24T06:26:05.820 INFO:teuthology.orchestra.run.smithi059.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 262 kB/s | 35 kB 00:00 2024-04-24T06:26:05.884 INFO:teuthology.orchestra.run.smithi066.stdout:(17/152): libradosstriper1-19.0.0-3124.ga3c8a22 4.9 MB/s | 501 kB 00:00 2024-04-24T06:26:05.912 INFO:teuthology.orchestra.run.smithi059.stdout:(80/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:04 2024-04-24T06:26:06.204 INFO:teuthology.orchestra.run.smithi059.stdout:(81/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-04-24T06:26:06.304 INFO:teuthology.orchestra.run.smithi059.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 641 kB/s | 250 kB 00:00 2024-04-24T06:26:06.392 INFO:teuthology.orchestra.run.smithi066.stdout:(18/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-24T06:26:06.429 INFO:teuthology.orchestra.run.smithi059.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.4 MB/s | 5.1 MB 00:00 2024-04-24T06:26:06.459 INFO:teuthology.orchestra.run.smithi066.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 686 kB/s | 45 kB 00:00 2024-04-24T06:26:06.534 INFO:teuthology.orchestra.run.smithi066.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 1.8 MB/s | 135 kB 00:00 2024-04-24T06:26:06.538 INFO:teuthology.orchestra.run.smithi059.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 331 kB/s | 77 kB 00:00 2024-04-24T06:26:06.580 INFO:teuthology.orchestra.run.smithi059.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 283 kB/s | 42 kB 00:00 2024-04-24T06:26:06.605 INFO:teuthology.orchestra.run.smithi059.stdout:(86/152): python3-oauthlib-3.1.1-5.el9.noarch.r 555 kB/s | 222 kB 00:00 2024-04-24T06:26:06.643 INFO:teuthology.orchestra.run.smithi066.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 1.5 MB/s | 162 kB 00:00 2024-04-24T06:26:06.689 INFO:teuthology.orchestra.run.smithi059.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-24T06:26:06.735 INFO:teuthology.orchestra.run.smithi066.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 3.5 MB/s | 322 kB 00:00 2024-04-24T06:26:06.819 INFO:teuthology.orchestra.run.smithi066.stdout:(23/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 3.5 MB/s | 297 kB 00:00 2024-04-24T06:26:06.822 INFO:teuthology.orchestra.run.smithi059.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 385 kB/s | 51 kB 00:00 2024-04-24T06:26:06.894 INFO:teuthology.orchestra.run.smithi066.stdout:(24/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T06:26:06.965 INFO:teuthology.orchestra.run.smithi059.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 381 kB/s | 54 kB 00:00 2024-04-24T06:26:06.969 INFO:teuthology.orchestra.run.smithi066.stdout:(25/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T06:26:07.032 INFO:teuthology.orchestra.run.smithi059.stdout:(90/152): python3-protobuf-3.14.0-13.el9.noarch 547 kB/s | 269 kB 00:00 2024-04-24T06:26:07.073 INFO:teuthology.orchestra.run.smithi059.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 597 kB/s | 279 kB 00:00 2024-04-24T06:26:07.157 INFO:teuthology.orchestra.run.smithi059.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-24T06:26:07.162 INFO:teuthology.orchestra.run.smithi066.stdout:(26/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x8 1.4 MB/s | 4.7 MB 00:03 2024-04-24T06:26:07.207 INFO:teuthology.orchestra.run.smithi059.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 333 kB/s | 44 kB 00:00 2024-04-24T06:26:07.311 INFO:teuthology.orchestra.run.smithi066.stdout:(27/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 9.1 MB/s | 3.1 MB 00:00 2024-04-24T06:26:07.333 INFO:teuthology.orchestra.run.smithi059.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 288 kB/s | 36 kB 00:00 2024-04-24T06:26:07.362 INFO:teuthology.orchestra.run.smithi066.stdout:(28/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 856 kB/s | 170 kB 00:00 2024-04-24T06:26:07.378 INFO:teuthology.orchestra.run.smithi066.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3124.g 520 kB/s | 34 kB 00:00 2024-04-24T06:26:07.466 INFO:teuthology.orchestra.run.smithi059.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 358 kB/s | 47 kB 00:00 2024-04-24T06:26:07.526 INFO:teuthology.orchestra.run.smithi059.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 602 kB/s | 221 kB 00:00 2024-04-24T06:26:07.650 INFO:teuthology.orchestra.run.smithi059.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 465 kB/s | 58 kB 00:00 2024-04-24T06:26:07.850 INFO:teuthology.orchestra.run.smithi059.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 471 kB/s | 94 kB 00:00 2024-04-24T06:26:07.934 INFO:teuthology.orchestra.run.smithi059.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 653 kB/s | 305 kB 00:00 2024-04-24T06:26:07.954 INFO:teuthology.orchestra.run.smithi066.stdout:(30/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 17 MB/s | 49 MB 00:02 2024-04-24T06:26:07.979 INFO:teuthology.orchestra.run.smithi066.stdout:(31/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 231 kB/s | 142 kB 00:00 2024-04-24T06:26:08.059 INFO:teuthology.orchestra.run.smithi059.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-04-24T06:26:08.101 INFO:teuthology.orchestra.run.smithi059.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 255 kB/s | 64 kB 00:00 2024-04-24T06:26:08.113 INFO:teuthology.orchestra.run.smithi066.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 4.8 MB/s | 3.5 MB 00:00 2024-04-24T06:26:08.197 INFO:teuthology.orchestra.run.smithi066.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 600 kB/s | 49 kB 00:00 2024-04-24T06:26:08.230 INFO:teuthology.orchestra.run.smithi066.stdout:(34/152): ceph-mgr-modules-core-19.0.0-3124.ga3 998 kB/s | 249 kB 00:00 2024-04-24T06:26:08.264 INFO:teuthology.orchestra.run.smithi066.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3124.ga 249 kB/s | 16 kB 00:00 2024-04-24T06:26:08.352 INFO:teuthology.orchestra.run.smithi059.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-24T06:26:08.356 INFO:teuthology.orchestra.run.smithi066.stdout:(36/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 8.2 MB/s | 762 kB 00:00 2024-04-24T06:26:08.393 INFO:teuthology.orchestra.run.smithi059.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-04-24T06:26:08.494 INFO:teuthology.orchestra.run.smithi059.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-24T06:26:08.508 INFO:teuthology.orchestra.run.smithi066.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-24T06:26:08.540 INFO:teuthology.orchestra.run.smithi066.stdout:(38/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 886 kB/s | 273 kB 00:00 2024-04-24T06:26:08.544 INFO:teuthology.orchestra.run.smithi059.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-24T06:26:08.565 INFO:teuthology.orchestra.run.smithi066.stdout:(39/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 88 kB/s | 18 kB 00:00 2024-04-24T06:26:08.586 INFO:teuthology.orchestra.run.smithi059.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 213 kB/s | 19 kB 00:00 2024-04-24T06:26:08.628 INFO:teuthology.orchestra.run.smithi059.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-24T06:26:08.695 INFO:teuthology.orchestra.run.smithi059.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-24T06:26:08.723 INFO:teuthology.orchestra.run.smithi066.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 375 kB/s | 80 kB 00:00 2024-04-24T06:26:08.737 INFO:teuthology.orchestra.run.smithi059.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 451 kB/s | 49 kB 00:00 2024-04-24T06:26:08.754 INFO:teuthology.orchestra.run.smithi059.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T06:26:08.765 INFO:teuthology.orchestra.run.smithi066.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 984 kB/s | 195 kB 00:00 2024-04-24T06:26:08.788 INFO:teuthology.orchestra.run.smithi059.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-24T06:26:08.855 INFO:teuthology.orchestra.run.smithi059.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-24T06:26:08.897 INFO:teuthology.orchestra.run.smithi059.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-24T06:26:08.899 INFO:teuthology.orchestra.run.smithi066.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-24T06:26:08.947 INFO:teuthology.orchestra.run.smithi059.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-04-24T06:26:08.982 INFO:teuthology.orchestra.run.smithi066.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-24T06:26:08.989 INFO:teuthology.orchestra.run.smithi059.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 353 kB/s | 32 kB 00:00 2024-04-24T06:26:09.031 INFO:teuthology.orchestra.run.smithi059.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-04-24T06:26:09.049 INFO:teuthology.orchestra.run.smithi066.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-24T06:26:09.064 INFO:teuthology.orchestra.run.smithi059.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-24T06:26:09.106 INFO:teuthology.orchestra.run.smithi059.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-24T06:26:09.108 INFO:teuthology.orchestra.run.smithi066.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-04-24T06:26:09.140 INFO:teuthology.orchestra.run.smithi059.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-24T06:26:09.141 INFO:teuthology.orchestra.run.smithi066.stdout:(46/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-24T06:26:09.167 INFO:teuthology.orchestra.run.smithi066.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-24T06:26:09.182 INFO:teuthology.orchestra.run.smithi059.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 475 kB/s | 35 kB 00:00 2024-04-24T06:26:09.200 INFO:teuthology.orchestra.run.smithi066.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-24T06:26:09.216 INFO:teuthology.orchestra.run.smithi059.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T06:26:09.359 INFO:teuthology.orchestra.run.smithi066.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-24T06:26:09.384 INFO:teuthology.orchestra.run.smithi066.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 585 kB/s | 107 kB 00:00 2024-04-24T06:26:09.430 INFO:teuthology.orchestra.run.smithi066.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 309 kB/s | 21 kB 00:00 2024-04-24T06:26:09.432 INFO:teuthology.orchestra.run.smithi059.stdout:(122/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-04-24T06:26:09.466 INFO:teuthology.orchestra.run.smithi059.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 580 kB/s | 144 kB 00:00 2024-04-24T06:26:09.493 INFO:teuthology.orchestra.run.smithi066.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-24T06:26:09.525 INFO:teuthology.orchestra.run.smithi059.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 118 kB/s | 11 kB 00:00 2024-04-24T06:26:09.535 INFO:teuthology.orchestra.run.smithi066.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-24T06:26:09.560 INFO:teuthology.orchestra.run.smithi066.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 253 kB/s | 33 kB 00:00 2024-04-24T06:26:09.619 INFO:teuthology.orchestra.run.smithi066.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-24T06:26:09.644 INFO:teuthology.orchestra.run.smithi066.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-24T06:26:09.659 INFO:teuthology.orchestra.run.smithi059.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 92 kB/s | 18 kB 00:00 2024-04-24T06:26:09.709 INFO:teuthology.orchestra.run.smithi059.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 127 kB/s | 23 kB 00:00 2024-04-24T06:26:09.727 INFO:teuthology.orchestra.run.smithi066.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-24T06:26:09.751 INFO:teuthology.orchestra.run.smithi059.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2024-04-24T06:26:09.753 INFO:teuthology.orchestra.run.smithi066.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-04-24T06:26:09.778 INFO:teuthology.orchestra.run.smithi066.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-24T06:26:09.801 INFO:teuthology.orchestra.run.smithi059.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 216 kB/s | 20 kB 00:00 2024-04-24T06:26:09.828 INFO:teuthology.orchestra.run.smithi066.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-24T06:26:09.862 INFO:teuthology.orchestra.run.smithi066.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-24T06:26:09.876 INFO:teuthology.orchestra.run.smithi059.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 626 kB/s | 46 kB 00:00 2024-04-24T06:26:09.927 INFO:teuthology.orchestra.run.smithi059.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.9 MB/s | 1.0 MB 00:00 2024-04-24T06:26:09.985 INFO:teuthology.orchestra.run.smithi059.stdout:(131/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-24T06:26:10.010 INFO:teuthology.orchestra.run.smithi059.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 593 kB/s | 79 kB 00:00 2024-04-24T06:26:10.061 INFO:teuthology.orchestra.run.smithi059.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 434 kB/s | 58 kB 00:00 2024-04-24T06:26:10.071 INFO:teuthology.orchestra.run.smithi066.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 87 kB/s | 25 kB 00:00 2024-04-24T06:26:10.096 INFO:teuthology.orchestra.run.smithi066.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 177 kB/s | 47 kB 00:00 2024-04-24T06:26:10.111 INFO:teuthology.orchestra.run.smithi059.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-24T06:26:10.130 INFO:teuthology.orchestra.run.smithi066.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-04-24T06:26:10.136 INFO:teuthology.orchestra.run.smithi059.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T06:26:10.186 INFO:teuthology.orchestra.run.smithi059.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 380 kB/s | 28 kB 00:00 2024-04-24T06:26:10.188 INFO:teuthology.orchestra.run.smithi066.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-24T06:26:10.214 INFO:teuthology.orchestra.run.smithi066.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 302 kB/s | 35 kB 00:00 2024-04-24T06:26:10.237 INFO:teuthology.orchestra.run.smithi059.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-24T06:26:10.270 INFO:teuthology.orchestra.run.smithi059.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 957 kB/s | 272 kB 00:00 2024-04-24T06:26:10.322 INFO:teuthology.orchestra.run.smithi066.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T06:26:10.342 INFO:teuthology.orchestra.run.smithi059.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 566 kB/s | 59 kB 00:00 2024-04-24T06:26:10.396 INFO:teuthology.orchestra.run.smithi059.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 288 kB/s | 36 kB 00:00 2024-04-24T06:26:10.421 INFO:teuthology.orchestra.run.smithi059.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 808 kB/s | 188 kB 00:00 2024-04-24T06:26:10.448 INFO:teuthology.orchestra.run.smithi066.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-24T06:26:10.471 INFO:teuthology.orchestra.run.smithi059.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 399 kB/s | 51 kB 00:00 2024-04-24T06:26:10.497 INFO:teuthology.orchestra.run.smithi059.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T06:26:10.498 INFO:teuthology.orchestra.run.smithi066.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-24T06:26:10.532 INFO:teuthology.orchestra.run.smithi059.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-24T06:26:10.540 INFO:teuthology.orchestra.run.smithi066.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-24T06:26:10.575 INFO:teuthology.orchestra.run.smithi059.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 260 kB/s | 20 kB 00:00 2024-04-24T06:26:10.599 INFO:teuthology.orchestra.run.smithi059.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-24T06:26:10.633 INFO:teuthology.orchestra.run.smithi059.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-24T06:26:10.665 INFO:teuthology.orchestra.run.smithi066.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-24T06:26:10.725 INFO:teuthology.orchestra.run.smithi059.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T06:26:10.732 INFO:teuthology.orchestra.run.smithi066.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-24T06:26:10.757 INFO:teuthology.orchestra.run.smithi066.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 962 kB/s | 87 kB 00:00 2024-04-24T06:26:10.841 INFO:teuthology.orchestra.run.smithi066.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-24T06:26:10.924 INFO:teuthology.orchestra.run.smithi066.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-24T06:26:10.934 INFO:teuthology.orchestra.run.smithi059.stdout:(149/152): librados2-19.0.0-3124.ga3c8a22b.el9. 10 MB/s | 3.4 MB 00:00 2024-04-24T06:26:11.024 INFO:teuthology.orchestra.run.smithi066.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-24T06:26:11.025 INFO:teuthology.orchestra.run.smithi059.stdout:(150/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-24T06:26:11.050 INFO:teuthology.orchestra.run.smithi066.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-04-24T06:26:11.092 INFO:teuthology.orchestra.run.smithi066.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 289 kB/s | 19 kB 00:00 2024-04-24T06:26:11.125 INFO:teuthology.orchestra.run.smithi066.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-24T06:26:11.151 INFO:teuthology.orchestra.run.smithi066.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-24T06:26:11.318 INFO:teuthology.orchestra.run.smithi066.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-24T06:26:11.351 INFO:teuthology.orchestra.run.smithi066.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-04-24T06:26:11.401 INFO:teuthology.orchestra.run.smithi066.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-24T06:26:11.427 INFO:teuthology.orchestra.run.smithi066.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-24T06:26:11.469 INFO:teuthology.orchestra.run.smithi066.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 637 kB/s | 42 kB 00:00 2024-04-24T06:26:11.569 INFO:teuthology.orchestra.run.smithi066.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T06:26:11.576 INFO:teuthology.orchestra.run.smithi059.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-24T06:26:11.602 INFO:teuthology.orchestra.run.smithi066.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-24T06:26:11.636 INFO:teuthology.orchestra.run.smithi066.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-04-24T06:26:11.669 INFO:teuthology.orchestra.run.smithi066.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-24T06:26:11.695 INFO:teuthology.orchestra.run.smithi066.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 557 kB/s | 51 kB 00:00 2024-04-24T06:26:11.778 INFO:teuthology.orchestra.run.smithi066.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 378 kB/s | 54 kB 00:00 2024-04-24T06:26:11.804 INFO:teuthology.orchestra.run.smithi066.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-04-24T06:26:11.809 INFO:teuthology.orchestra.run.smithi059.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 2.4 MB 00:00 2024-04-24T06:26:11.815 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:26:11.816 INFO:teuthology.orchestra.run.smithi059.stdout:Total 11 MB/s | 222 MB 00:19 2024-04-24T06:26:11.854 INFO:teuthology.orchestra.run.smithi066.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 592 kB/s | 44 kB 00:00 2024-04-24T06:26:11.888 INFO:teuthology.orchestra.run.smithi066.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-24T06:26:11.930 INFO:teuthology.orchestra.run.smithi066.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 482 kB/s | 36 kB 00:00 2024-04-24T06:26:11.955 INFO:teuthology.orchestra.run.smithi066.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 716 kB/s | 47 kB 00:00 2024-04-24T06:26:12.005 INFO:teuthology.orchestra.run.smithi066.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-24T06:26:12.030 INFO:teuthology.orchestra.run.smithi066.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 775 kB/s | 58 kB 00:00 2024-04-24T06:26:12.056 INFO:teuthology.orchestra.run.smithi066.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-24T06:26:12.081 INFO:teuthology.orchestra.run.smithi066.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-24T06:26:12.148 INFO:teuthology.orchestra.run.smithi066.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 490 kB/s | 45 kB 00:00 2024-04-24T06:26:12.174 INFO:teuthology.orchestra.run.smithi066.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-04-24T06:26:12.249 INFO:teuthology.orchestra.run.smithi066.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.5 MB/s | 863 kB 00:00 2024-04-24T06:26:12.516 INFO:teuthology.orchestra.run.smithi066.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-24T06:26:12.758 INFO:teuthology.orchestra.run.smithi066.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-24T06:26:12.817 INFO:teuthology.orchestra.run.smithi066.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 854 kB/s | 548 kB 00:00 2024-04-24T06:26:12.850 INFO:teuthology.orchestra.run.smithi066.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 515 kB/s | 309 kB 00:00 2024-04-24T06:26:12.917 INFO:teuthology.orchestra.run.smithi066.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-24T06:26:12.959 INFO:teuthology.orchestra.run.smithi066.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 450 kB/s | 49 kB 00:00 2024-04-24T06:26:13.018 INFO:teuthology.orchestra.run.smithi066.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2024-04-24T06:26:13.109 INFO:teuthology.orchestra.run.smithi066.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-24T06:26:13.235 INFO:teuthology.orchestra.run.smithi066.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-04-24T06:26:13.360 INFO:teuthology.orchestra.run.smithi066.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-24T06:26:13.385 INFO:teuthology.orchestra.run.smithi066.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 291 kB/s | 43 kB 00:00 2024-04-24T06:26:13.478 INFO:teuthology.orchestra.run.smithi066.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-04-24T06:26:13.569 INFO:teuthology.orchestra.run.smithi066.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-24T06:26:13.620 INFO:teuthology.orchestra.run.smithi066.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 64 kB/s | 15 kB 00:00 2024-04-24T06:26:13.653 INFO:teuthology.orchestra.run.smithi066.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 983 kB/s | 172 kB 00:00 2024-04-24T06:26:13.686 INFO:teuthology.orchestra.run.smithi066.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-24T06:26:13.745 INFO:teuthology.orchestra.run.smithi066.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-24T06:26:13.770 INFO:teuthology.orchestra.run.smithi066.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 303 kB/s | 35 kB 00:00 2024-04-24T06:26:13.812 INFO:teuthology.orchestra.run.smithi066.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-24T06:26:13.905 INFO:teuthology.orchestra.run.smithi066.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-24T06:26:13.930 INFO:teuthology.orchestra.run.smithi066.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 911 kB/s | 144 kB 00:00 2024-04-24T06:26:13.955 INFO:teuthology.orchestra.run.smithi066.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-24T06:26:14.013 INFO:teuthology.orchestra.run.smithi066.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-24T06:26:14.047 INFO:teuthology.orchestra.run.smithi066.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2024-04-24T06:26:14.097 INFO:teuthology.orchestra.run.smithi066.stdout:(128/152): python3-jaraco-collections-3.0.0-8.e 165 kB/s | 23 kB 00:00 2024-04-24T06:26:14.131 INFO:teuthology.orchestra.run.smithi066.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-24T06:26:14.198 INFO:teuthology.orchestra.run.smithi066.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-04-24T06:26:14.240 INFO:teuthology.orchestra.run.smithi066.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 733 kB/s | 79 kB 00:00 2024-04-24T06:26:14.290 INFO:teuthology.orchestra.run.smithi066.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-24T06:26:14.315 INFO:teuthology.orchestra.run.smithi066.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 495 kB/s | 58 kB 00:00 2024-04-24T06:26:14.349 INFO:teuthology.orchestra.run.smithi066.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-24T06:26:14.391 INFO:teuthology.orchestra.run.smithi066.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-24T06:26:14.416 INFO:teuthology.orchestra.run.smithi066.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 902 kB/s | 90 kB 00:00 2024-04-24T06:26:14.466 INFO:teuthology.orchestra.run.smithi066.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-04-24T06:26:14.492 INFO:teuthology.orchestra.run.smithi066.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-04-24T06:26:14.525 INFO:teuthology.orchestra.run.smithi066.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-24T06:26:14.592 INFO:teuthology.orchestra.run.smithi066.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-04-24T06:26:14.617 INFO:teuthology.orchestra.run.smithi066.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2024-04-24T06:26:14.659 INFO:teuthology.orchestra.run.smithi066.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2024-04-24T06:26:14.718 INFO:teuthology.orchestra.run.smithi066.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T06:26:14.743 INFO:teuthology.orchestra.run.smithi066.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2024-04-24T06:26:14.785 INFO:teuthology.orchestra.run.smithi066.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-24T06:26:14.810 INFO:teuthology.orchestra.run.smithi066.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-24T06:26:14.828 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-04-24T06:26:14.869 INFO:teuthology.orchestra.run.smithi066.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-24T06:26:14.944 INFO:teuthology.orchestra.run.smithi066.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-24T06:26:15.096 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-04-24T06:26:15.096 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-04-24T06:26:15.161 INFO:teuthology.orchestra.run.smithi066.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-24T06:26:16.086 INFO:teuthology.orchestra.run.smithi066.stdout:(150/152): librados2-19.0.0-3124.ga3c8a22b.el9. 2.7 MB/s | 3.4 MB 00:01 2024-04-24T06:26:16.803 INFO:teuthology.orchestra.run.smithi066.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 744 kB/s | 1.3 MB 00:01 2024-04-24T06:26:17.228 INFO:teuthology.orchestra.run.smithi066.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 2.4 MB 00:02 2024-04-24T06:26:17.234 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:26:17.235 INFO:teuthology.orchestra.run.smithi066.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-24T06:26:17.847 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-04-24T06:26:17.847 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-04-24T06:26:20.205 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-24T06:26:20.473 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-24T06:26:20.473 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-24T06:26:23.242 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-24T06:26:23.243 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-24T06:26:24.452 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-04-24T06:26:24.609 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T06:26:24.711 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T06:26:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T06:26:24.898 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T06:26:24.935 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T06:26:25.063 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T06:26:25.508 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T06:26:25.607 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:26:25.782 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:26:25.816 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:26:25.948 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:26:25.982 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:26:26.342 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T06:26:26.397 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T06:26:26.420 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:26:26.579 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:26:26.613 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:26:26.700 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:26:26.730 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:26:26.794 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:26:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:26:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:26:27.401 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T06:26:27.543 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T06:26:27.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T06:26:27.632 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T06:26:27.738 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T06:26:27.822 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T06:26:27.897 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T06:26:27.955 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T06:26:27.998 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T06:26:28.124 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T06:26:28.199 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T06:26:28.241 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T06:26:28.279 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T06:26:28.386 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T06:26:28.426 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T06:26:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T06:26:28.515 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T06:26:28.633 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T06:26:28.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T06:26:28.797 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T06:26:28.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T06:26:29.039 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T06:26:29.740 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T06:26:29.856 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T06:26:29.898 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T06:26:29.942 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T06:26:29.975 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T06:26:30.041 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T06:26:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-24T06:26:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T06:26:30.274 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T06:26:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T06:26:30.392 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T06:26:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T06:26:30.473 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T06:26:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T06:26:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T06:26:30.570 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T06:26:30.610 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T06:26:30.649 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T06:26:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T06:26:30.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T06:26:30.766 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T06:26:30.829 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T06:26:30.894 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T06:26:31.096 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T06:26:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T06:26:31.171 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T06:26:31.209 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T06:26:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:26:31.268 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T06:26:31.411 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T06:26:31.424 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:26:31.450 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T06:26:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:26:31.500 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T06:26:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T06:26:31.564 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T06:26:31.648 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:26:31.694 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:26:31.747 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T06:26:31.925 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T06:26:31.996 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T06:26:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T06:26:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T06:26:32.114 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T06:26:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T06:26:32.147 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:26:32.191 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T06:26:32.255 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T06:26:32.306 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T06:26:32.317 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:26:32.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:26:32.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T06:26:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T06:26:32.489 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:26:32.491 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T06:26:32.516 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:26:32.537 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T06:26:32.597 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:26:32.686 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T06:26:32.748 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T06:26:32.823 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:26:32.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T06:26:32.987 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:26:33.063 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T06:26:33.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T06:26:33.189 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T06:26:33.303 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T06:26:33.351 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T06:26:33.401 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T06:26:33.527 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T06:26:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T06:26:33.646 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T06:26:33.693 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T06:26:33.732 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T06:26:33.854 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T06:26:33.941 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T06:26:33.977 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T06:26:34.038 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T06:26:34.118 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T06:26:34.155 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T06:26:34.180 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T06:26:34.196 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T06:26:34.213 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T06:26:34.250 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T06:26:34.286 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T06:26:34.341 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T06:26:34.408 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T06:26:34.411 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T06:26:34.475 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T06:26:34.537 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T06:26:34.581 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T06:26:34.588 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T06:26:34.788 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T06:26:34.828 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T06:26:34.861 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T06:26:34.867 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T06:26:34.967 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T06:26:35.586 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T06:26:35.726 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T06:26:35.786 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T06:26:35.827 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T06:26:35.835 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T06:26:35.883 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T06:26:35.955 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T06:26:35.987 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T06:26:36.264 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T06:26:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T06:26:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T06:26:36.517 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T06:26:36.571 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T06:26:36.611 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T06:26:36.683 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T06:26:36.736 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T06:26:36.800 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T06:26:36.868 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T06:26:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T06:26:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T06:26:37.204 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T06:26:37.264 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T06:26:37.405 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T06:26:37.453 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T06:26:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T06:26:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T06:26:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T06:26:37.671 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T06:26:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T06:26:37.812 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T06:26:37.827 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T06:26:37.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T06:26:38.005 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T06:26:38.059 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T06:26:38.124 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T06:26:38.182 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T06:26:38.251 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T06:26:38.276 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T06:26:38.308 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T06:26:38.321 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:26:38.355 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T06:26:38.409 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T06:26:38.445 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:26:38.460 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T06:26:38.478 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T06:26:38.534 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T06:26:38.536 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T06:26:38.597 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T06:26:38.814 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T06:26:38.855 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T06:26:38.928 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T06:26:39.088 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T06:26:39.128 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T06:26:39.159 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T06:26:39.165 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:26:39.267 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:26:39.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T06:26:40.166 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T06:26:40.225 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T06:26:40.258 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T06:26:40.386 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T06:26:40.471 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T06:26:40.581 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T06:26:40.626 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T06:26:40.869 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T06:26:40.906 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T06:26:41.015 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T06:26:41.875 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T06:26:42.025 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T06:26:42.122 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:42.172 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:42.917 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:26:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:26:43.109 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T06:26:43.110 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T06:26:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:26:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T06:26:43.804 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T06:26:43.846 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T06:26:43.889 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T06:26:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T06:26:44.358 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:26:44.465 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:26:44.498 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T06:26:44.556 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T06:26:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T06:26:45.194 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:26:45.294 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:26:45.321 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T06:26:48.253 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:48.318 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:49.086 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:26:49.129 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:26:49.309 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:26:49.309 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T06:26:49.310 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T06:26:49.310 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:26:49.346 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-04-24T06:26:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:26:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:26:56.760 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:26:56.760 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T06:26:56.760 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:58.441 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:26:58.525 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:26:58.526 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:26:58.526 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T06:26:58.526 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:26:58.526 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:26:58.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:58.800 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:26:58.869 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:59.375 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:26:59.446 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:26:59.446 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:26:59.446 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T06:26:59.446 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:26:59.447 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:26:59.447 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:26:59.593 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T06:26:59.658 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T06:26:59.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T06:26:59.987 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T06:27:00.047 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T06:27:00.136 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T06:27:00.253 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:27:00.423 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:27:00.472 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T06:27:00.512 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T06:27:00.545 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T06:27:01.214 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:27:01.274 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:27:01.334 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T06:27:01.467 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T06:27:01.499 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:27:02.872 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:27:02.918 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-04-24T06:27:03.071 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:03.310 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:27:03.359 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:27:03.359 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:03.360 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T06:27:03.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:27:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:27:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:27:04.450 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T06:27:04.487 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:27:04.572 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:27:04.573 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:04.573 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T06:27:04.573 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:27:04.573 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:27:04.573 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:04.629 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:27:04.669 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:27:04.784 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T06:27:04.847 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T06:27:05.070 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:27:05.148 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:05.420 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:27:05.495 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:27:05.495 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:05.495 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T06:27:05.496 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:27:05.496 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:27:05.496 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:05.908 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T06:27:05.940 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:27:06.019 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:27:06.019 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:06.019 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T06:27:06.019 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:27:06.020 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:27:06.020 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:06.020 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:27:06.066 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:27:06.122 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:27:06.123 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:06.123 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T06:27:06.123 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:27:06.123 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:27:06.123 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:06.139 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:27:06.139 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:06.139 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T06:27:06.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:27:06.263 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T06:27:06.320 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:27:06.320 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:06.321 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T06:27:06.321 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:27:06.321 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:27:06.321 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T06:27:06.404 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T06:27:06.677 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T06:27:06.692 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:27:06.744 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:27:06.764 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:06.836 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T06:27:06.978 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:27:07.171 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:27:07.218 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T06:27:07.266 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T06:27:07.298 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T06:27:07.991 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:27:08.044 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:27:08.103 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T06:27:08.215 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T06:27:08.244 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:27:09.608 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:27:09.646 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:27:10.812 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:27:10.846 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:27:11.001 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:27:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T06:27:11.226 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:27:11.311 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:27:11.311 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:11.311 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T06:27:11.311 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:27:11.312 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:27:11.312 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:11.374 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:27:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:27:11.562 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T06:27:11.643 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T06:27:12.082 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T06:27:12.134 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T06:27:12.205 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T06:27:12.252 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T06:27:12.301 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T06:27:12.650 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T06:27:12.703 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T06:27:12.731 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T06:27:12.743 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T06:27:12.777 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:27:12.875 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:27:12.876 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:12.876 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T06:27:12.876 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:27:12.876 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:27:12.876 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:12.961 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:27:13.040 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:27:13.040 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:13.040 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T06:27:13.040 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:13.120 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:27:13.229 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:27:13.229 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:13.229 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T06:27:13.230 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:27:13.230 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:27:13.230 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:13.645 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:27:13.659 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:13.720 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:13.722 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:27:13.722 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:27:13.722 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T06:27:13.722 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:27:13.722 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:27:13.723 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:13.779 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:15.241 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T06:27:15.375 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T06:27:15.466 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T06:27:15.636 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T06:27:15.683 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T06:27:15.683 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:27:15.801 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:27:15.857 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:27:15.857 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:27:15.997 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:27:16.132 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:16.448 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:16.448 INFO:teuthology.orchestra.run.smithi059.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T06:27:16.448 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:19.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T06:27:19.073 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T06:27:19.153 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T06:27:19.196 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T06:27:19.244 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T06:27:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T06:27:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T06:27:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T06:27:20.597 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:20.640 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:20.712 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:27:22.163 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T06:27:22.246 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T06:27:22.246 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T06:27:22.246 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T06:27:22.247 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T06:27:22.248 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T06:27:22.260 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T06:27:22.261 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T06:27:22.262 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T06:27:22.263 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T06:27:22.264 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T06:27:22.265 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T06:27:22.266 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T06:27:22.267 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T06:27:22.268 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:27:22.269 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T06:27:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T06:27:22.375 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T06:27:22.552 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T06:27:22.594 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T06:27:22.595 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:27:22.735 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:27:22.776 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:27:22.777 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:27:22.900 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:27:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:23.380 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:23.380 INFO:teuthology.orchestra.run.smithi066.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T06:27:23.380 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:24.828 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.829 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.830 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T06:27:24.831 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T06:27:24.832 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T06:27:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.834 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T06:27:24.835 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T06:27:24.836 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T06:27:24.837 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T06:27:24.838 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:24.839 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:24.840 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-24T06:27:25.645 DEBUG:teuthology.parallel:result is None 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:27:28.867 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T06:27:28.868 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T06:27:28.869 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T06:27:28.869 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T06:27:28.869 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T06:27:28.869 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T06:27:28.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T06:27:28.880 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T06:27:28.881 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T06:27:28.883 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T06:27:28.884 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T06:27:28.885 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T06:27:28.886 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T06:27:28.887 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T06:27:28.888 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:27:28.889 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:31.353 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.354 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.355 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T06:27:31.356 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T06:27:31.357 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T06:27:31.358 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.359 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T06:27:31.360 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T06:27:31.361 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T06:27:31.362 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T06:27:31.363 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:27:31.364 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:31.365 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-24T06:27:31.960 DEBUG:teuthology.parallel:result is None 2024-04-24T06:27:31.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:27:32.209 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T06:27:32.275 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:27:32.276 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:27:32.276 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T06:27:32.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:27:32.503 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T06:27:32.556 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:27:32.581 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:27:32.581 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T06:27:32.583 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T06:27:32.583 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:32.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T06:27:32.665 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:32.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T06:27:32.748 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T06:27:32.749 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:32.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T06:27:32.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T06:27:32.893 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:32.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T06:27:32.965 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T06:27:33.072 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T06:27:33.072 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:33.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T06:27:33.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T06:27:33.246 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:33.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T06:27:33.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T06:27:33.416 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T06:27:33.416 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:33.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T06:27:33.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T06:27:33.587 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:33.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T06:27:33.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T06:27:33.759 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T06:27:33.849 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T06:27:33.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2024-04-24T06:27:33.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-04-24T06:27:33.921 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T06:27:33.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T06:27:33.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T06:27:33.993 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T06:27:33.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T06:27:33.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T06:27:34.066 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T06:27:34.066 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T06:27:34.066 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T06:27:34.066 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:34.067 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T06:27:34.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T06:27:34.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T06:27:34.391 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:34.391 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T06:27:34.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T06:27:34.617 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T06:27:34.683 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T06:27:34.683 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T06:27:34.683 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164', 'branch': 'wip-rishabh-testing-20240416.193735-5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7671164', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735-5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735-5', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735-5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCem8jKKgS1SP/G8/wY/oFvc/9wAUb1j7OEHKfIficKsbklLvdU+qnp/nZZiqAI3qiwxNqLNR18LL0xOOMGmgTA=', 'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeJ6fIitlL1bfgWDiHLCLXODFveauWBlpHqh8YlI3t2I0rjmRWb1wrpdsPz2n6Dkv1JwMPEIugOab5eN+zmTkc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_05:22:11', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T06:27:34.684 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T06:27:34.692 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T06:27:34.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T06:27:34.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T06:27:34.838 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:34.838 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T06:27:34.882 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T06:27:34.883 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-24 06:27:16.966390069 +0000 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-24 06:24:58.078813388 +0000 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-24 06:24:58.078813388 +0000 2024-04-24T06:27:34.972 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-24 06:24:58.078813388 +0000 2024-04-24T06:27:34.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T06:27:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-24T06:27:35.052 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-24T06:27:35.052 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000424331 s, 1.2 MB/s 2024-04-24T06:27:35.056 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T06:27:35.109 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-04-24T06:27:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T06:27:35.156 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:35.156 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-24 06:27:16.967390056 +0000 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-24 06:24:58.698805533 +0000 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-24 06:24:58.698805533 +0000 2024-04-24T06:27:35.157 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-24 06:24:58.698805533 +0000 2024-04-24T06:27:35.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T06:27:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-24T06:27:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-24T06:27:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000409443 s, 1.3 MB/s 2024-04-24T06:27:35.279 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T06:27:35.329 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-04-24T06:27:35.421 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T06:27:35.421 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-24 06:27:16.968390043 +0000 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-24 06:24:59.331797514 +0000 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-24 06:24:59.331797514 +0000 2024-04-24T06:27:35.422 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-24 06:24:59.331797514 +0000 2024-04-24T06:27:35.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T06:27:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-24T06:27:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-24T06:27:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000308286 s, 1.7 MB/s 2024-04-24T06:27:35.490 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T06:27:35.536 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-24 06:27:16.968390043 +0000 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-24 06:24:59.929789938 +0000 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-24 06:24:59.929789938 +0000 2024-04-24T06:27:35.583 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-24 06:24:59.929789938 +0000 2024-04-24T06:27:35.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T06:27:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-24T06:27:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-24T06:27:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000379745 s, 1.3 MB/s 2024-04-24T06:27:35.716 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T06:27:35.772 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:35.772 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:35.772 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T06:27:35.818 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T06:27:35.818 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-04-24T06:27:35.896 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-24 06:27:23.859029619 +0000 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-24 06:24:58.084891878 +0000 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-24 06:24:58.084891878 +0000 2024-04-24T06:27:35.897 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-24 06:24:58.084891878 +0000 2024-04-24T06:27:35.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T06:27:35.996 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-24T06:27:35.996 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-24T06:27:35.996 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000346695 s, 1.5 MB/s 2024-04-24T06:27:35.999 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T06:27:36.040 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-24 06:27:23.860029610 +0000 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-24 06:24:58.690886381 +0000 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-24 06:24:58.690886381 +0000 2024-04-24T06:27:36.120 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-24 06:24:58.690886381 +0000 2024-04-24T06:27:36.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T06:27:36.222 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-24T06:27:36.223 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-24T06:27:36.223 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000296997 s, 1.7 MB/s 2024-04-24T06:27:36.226 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T06:27:36.267 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-04-24T06:27:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T06:27:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:36.345 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T06:27:36.345 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:36.345 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:36.346 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-24 06:27:23.860029610 +0000 2024-04-24T06:27:36.346 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-24 06:24:59.338880503 +0000 2024-04-24T06:27:36.346 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-24 06:24:59.338880503 +0000 2024-04-24T06:27:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-24 06:24:59.338880503 +0000 2024-04-24T06:27:36.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T06:27:36.444 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-24T06:27:36.444 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-24T06:27:36.444 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000320758 s, 1.6 MB/s 2024-04-24T06:27:36.448 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T06:27:36.488 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-04-24T06:27:36.571 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T06:27:36.571 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:27:36.571 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T06:27:36.571 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:27:36.571 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:27:36.572 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-24 06:27:23.861029601 +0000 2024-04-24T06:27:36.572 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-24 06:24:59.956874897 +0000 2024-04-24T06:27:36.572 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-24 06:24:59.956874897 +0000 2024-04-24T06:27:36.572 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-24 06:24:59.956874897 +0000 2024-04-24T06:27:36.572 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T06:27:36.673 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-24T06:27:36.673 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-24T06:27:36.673 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000305118 s, 1.7 MB/s 2024-04-24T06:27:36.676 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T06:27:36.716 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:36.716 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T06:27:36.716 INFO:tasks.ceph:Generating config... 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] pid file = 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T06:27:36.718 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T06:27:36.718 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T06:27:36.718 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T06:27:36.719 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T06:27:36.720 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T06:27:36.721 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T06:27:36.721 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T06:27:36.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T06:27:36.862 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T06:27:36.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T06:27:37.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T06:27:37.099 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.59'), ('mon.b', '172.21.15.66'), ('mon.c', '[v2:172.21.15.66:3301,v1:172.21.15.66:6790]')] 2024-04-24T06:27:37.100 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T06:27:37.101 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:37.101 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T06:27:37.158 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.59 --add b 172.21.15.66 --addv c '[v2:172.21.15.66:3301,v1:172.21.15.66:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:37.317 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:37.317 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: generated fsid fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:27:37.317 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 0 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:fsid fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:last_changed 2024-04-24T06:27:37.316915+0000 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-04-24T06:27:37.316915+0000 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:min_mon_release 17 (quincy) 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:election_strategy: 1 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:1: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:2: [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] mon.c 2024-04-24T06:27:37.318 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T06:27:37.321 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T06:27:37.375 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fce72cba-ae3c-4ba2-abdc-e927f443d3c7... 2024-04-24T06:27:37.376 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:27:37.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-24T06:27:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-24T06:27:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-24T06:27:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-24T06:27:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-04-24T06:27:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.497 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-24T06:27:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:27:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-24T06:27:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:27:37.501 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:27:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-24T06:27:37.521 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-24T06:27:37.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:27:37.523 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-24T06:27:37.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.525 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-24T06:27:37.526 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.527 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:27:37.528 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:27:37.529 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:27:37.530 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-24T06:27:37.531 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:27:37.532 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:27:37.557 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T06:27:37.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T06:27:37.697 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T06:27:37.697 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:37.697 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T06:27:37.753 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:37.753 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T06:27:37.809 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:27:37.809 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:37.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T06:27:37.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T06:27:37.944 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:27:37.944 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:38.042 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:27:38.042 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:38.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T06:27:38.043 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T06:27:38.152 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:27:38.152 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:38.237 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T06:27:38.237 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T06:27:38.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T06:27:38.420 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T06:27:38.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T06:27:38.589 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T06:27:38.598 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T06:27:38.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T06:27:38.764 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T06:27:38.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T06:27:38.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T06:27:39.025 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T06:27:39.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T06:27:39.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T06:27:39.247 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T06:27:39.254 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T06:27:39.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T06:27:39.448 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T06:27:39.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T06:27:39.545 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T06:27:39.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T06:27:39.676 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T06:27:39.719 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-24T06:27:39.839 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-24T06:27:39.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T06:27:40.018 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T06:27:40.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-24T06:27:40.197 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-24T06:27:40.254 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T06:27:40.255 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T06:27:40.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T06:27:40.348 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:40.348 INFO:tasks.ceph:role: osd.0 2024-04-24T06:27:40.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:27:40.349 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T06:27:40.441 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:40.442 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-24T06:27:40.862 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-24T06:27:40.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T06:27:41.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T06:27:41.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T06:27:41.187 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:41.187 INFO:tasks.ceph:role: osd.1 2024-04-24T06:27:41.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:27:41.187 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:41.274 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:41.275 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:41.275 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:41.629 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-24T06:27:41.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-24T06:27:41.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T06:27:41.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T06:27:41.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T06:27:41.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:41.924 INFO:tasks.ceph:role: osd.2 2024-04-24T06:27:41.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:27:41.924 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T06:27:42.008 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:42.008 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:42.009 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:42.400 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-24T06:27:42.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-24T06:27:42.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T06:27:42.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T06:27:42.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T06:27:42.756 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:42.756 INFO:tasks.ceph:role: osd.3 2024-04-24T06:27:42.756 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:27:42.757 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:42.853 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:43.263 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-24T06:27:43.269 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-24T06:27:43.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T06:27:43.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T06:27:43.462 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.609+0000 7fb1075c0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T06:27:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.609+0000 7fb1075c0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T06:27:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.609+0000 7fb1075c0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T06:27:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.610+0000 7fb1075c0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T06:27:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.610+0000 7fb1075c0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T06:27:43.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:43.614+0000 7fb1075c0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T06:27:46.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T06:27:46.700 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:46.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.836+0000 7f53fea2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T06:27:46.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.837+0000 7f53fea2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T06:27:46.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.837+0000 7f53fea2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T06:27:46.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.837+0000 7f53fea2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T06:27:46.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.837+0000 7f53fea2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T06:27:46.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:46.839+0000 7f53fea2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T06:27:50.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T06:27:50.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:50.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.289+0000 7f70f8816640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T06:27:50.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.289+0000 7f70f8816640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T06:27:50.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.290+0000 7f70f8816640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T06:27:50.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.290+0000 7f70f8816640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T06:27:50.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.290+0000 7f70f8816640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T06:27:50.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:50.292+0000 7f70f8816640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T06:27:53.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T06:27:53.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:27:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.750+0000 7f4127e0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T06:27:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.750+0000 7f4127e0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T06:27:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.750+0000 7f4127e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T06:27:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.750+0000 7f4127e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T06:27:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.750+0000 7f4127e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T06:27:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:27:53.753+0000 7f4127e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T06:27:56.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T06:27:57.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T06:27:57.146 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:57.146 INFO:tasks.ceph:role: osd.4 2024-04-24T06:27:57.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:27:57.147 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:57.235 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:57.653 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-24T06:27:57.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-24T06:27:57.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T06:27:57.817 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T06:27:57.912 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T06:27:57.992 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:57.992 INFO:tasks.ceph:role: osd.5 2024-04-24T06:27:57.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:27:57.992 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T06:27:58.084 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:58.084 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:58.084 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:58.084 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:58.084 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:58.085 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:58.085 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:58.085 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:58.085 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:58.085 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:58.442 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-24T06:27:58.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-24T06:27:58.447 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T06:27:58.566 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T06:27:58.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T06:27:58.731 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:58.731 INFO:tasks.ceph:role: osd.6 2024-04-24T06:27:58.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:27:58.732 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:58.821 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:58.822 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:58.822 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:27:59.208 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-24T06:27:59.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-24T06:27:59.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T06:27:59.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T06:27:59.461 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T06:27:59.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T06:27:59.542 INFO:tasks.ceph:role: osd.7 2024-04-24T06:27:59.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:27:59.542 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T06:27:59.636 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T06:27:59.637 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-04-24T06:28:00.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-04-24T06:28:00.053 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T06:28:00.177 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T06:28:00.262 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:00.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.401+0000 7f0fd5a20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T06:28:00.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.402+0000 7f0fd5a20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T06:28:00.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.402+0000 7f0fd5a20640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T06:28:00.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.402+0000 7f0fd5a20640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T06:28:00.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.402+0000 7f0fd5a20640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T06:28:00.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:00.405+0000 7f0fd5a20640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T06:28:03.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T06:28:03.703 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:03.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.837+0000 7f2a9ccf0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T06:28:03.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.838+0000 7f2a9ccf0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T06:28:03.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.838+0000 7f2a9ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T06:28:03.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.838+0000 7f2a9ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T06:28:03.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.838+0000 7f2a9ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T06:28:03.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:03.840+0000 7f2a9ccf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T06:28:04.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:28:04.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:28:07.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T06:28:07.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:07.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.276+0000 7f772182c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T06:28:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.277+0000 7f772182c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T06:28:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.277+0000 7f772182c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T06:28:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.277+0000 7f772182c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T06:28:07.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.277+0000 7f772182c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T06:28:07.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:07.280+0000 7f772182c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T06:28:10.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T06:28:10.606 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:10.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.749+0000 7f4538629640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T06:28:10.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.750+0000 7f4538629640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T06:28:10.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.750+0000 7f4538629640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T06:28:10.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.750+0000 7f4538629640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T06:28:10.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.750+0000 7f4538629640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T06:28:10.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:10.753+0000 7f4538629640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T06:28:13.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T06:28:14.064 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T06:28:14.064 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T06:28:14.166 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T06:28:14.258 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T06:28:14.346 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T06:28:14.436 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T06:28:14.524 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T06:28:14.613 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:14.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T06:28:14.702 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:14.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T06:28:14.797 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:14.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T06:28:14.882 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:14.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T06:28:14.973 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:14.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T06:28:15.061 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:15.062 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T06:28:15.143 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:15.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T06:28:15.235 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:15.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T06:28:15.327 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:15.327 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T06:28:15.388 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:15.388 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-24T06:28:15.444 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:15.445 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T06:28:15.506 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:15.506 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-24T06:28:15.560 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T06:28:15.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T06:28:15.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA== 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng== 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ== 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2024-04-24T06:28:15.649 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ== 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2024-04-24T06:28:15.650 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[client.2] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ== 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout:[client.3] 2024-04-24T06:28:15.651 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g== 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA== 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng== 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ== 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA== 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-04-24T06:28:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg== 2024-04-24T06:28:15.653 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[client.2] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[client.1] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ== 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout:[client.3] 2024-04-24T06:28:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g== 2024-04-24T06:28:15.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T06:28:15.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T06:28:15.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:15.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:16.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:16.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:16.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.309 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.615 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:16.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T06:28:16.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T06:28:16.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:16.909 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:17.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:17.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T06:28:17.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T06:28:17.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:17.811 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:17.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:17.963 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:18.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:18.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:18.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:18.268 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T06:28:18.424 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T06:28:18.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T06:28:18.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T06:28:18.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T06:28:18.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T06:28:18.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T06:28:19.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T06:28:19.093 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T06:28:19.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T06:28:19.348 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T06:28:19.433 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:19.441 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T06:28:19.501 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T06:28:19.501 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T06:28:19.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T06:28:19.506 INFO:tasks.ceph.mon.a:Started 2024-04-24T06:28:19.507 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T06:28:19.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T06:28:19.514 INFO:tasks.ceph.mon.b:Started 2024-04-24T06:28:19.514 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T06:28:19.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T06:28:19.522 INFO:tasks.ceph.mon.c:Started 2024-04-24T06:28:19.522 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T06:28:19.522 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T06:28:19.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T06:28:19.530 INFO:tasks.ceph.mgr.y:Started 2024-04-24T06:28:19.530 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T06:28:19.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T06:28:19.535 INFO:tasks.ceph.mgr.x:Started 2024-04-24T06:28:19.535 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T06:28:19.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T06:28:19.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.711+0000 7f7923e00640 1 Processor -- start 2024-04-24T06:28:19.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.711+0000 7f7923e00640 1 -- start start 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79240b5da0 con 0x7f79240b4f20 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79240d0a20 con 0x7f792415ed20 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f792415f100 con 0x7f79240d15b0 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f79240b4f20 0x7f79240cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35398/0 (socket says 172.21.15.59:35398) 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f7923400640 1 -- 172.21.15.59:0/2610066990 learned_addr learned my addr 172.21.15.59:0/2610066990 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T06:28:19.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T06:28:19.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T06:28:19.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.713+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:28:19.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.914+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.915+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T06:28:19.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.915+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79240cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T06:28:19.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.915+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T06:28:19.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.915+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:28:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.956+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982822180 0 0) 0x7f79240b5da0 con 0x7f79240b4f20 2024-04-24T06:28:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.956+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7908003520 con 0x7f79240b4f20 2024-04-24T06:28:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.957+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747259681 0 0) 0x7f79240d0a20 con 0x7f792415ed20 2024-04-24T06:28:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.958+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79240b5da0 con 0x7f792415ed20 2024-04-24T06:28:19.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.958+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 103210971 0 0) 0x7f7908003520 con 0x7f79240b4f20 2024-04-24T06:28:19.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.958+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79240cf280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.958+0000 7f7922000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79240cf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.958+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 >> v1:172.21.15.66:6789/0 conn(0x7f792415ed20 legacy=0x7f79240d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79240d2ed0 con 0x7f79240b4f20 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f791c001670 con 0x7f79240b4f20 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 >> v1:172.21.15.59:6789/0 conn(0x7f79240b4f20 legacy=0x7f79240cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7908003b70 0x7f7908003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f7908004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.959+0000 7f7922000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7908004db0 0x7f7908005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.960+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f7908004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.960+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f790800bc30 con 0x7f7908004db0 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.960+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f790800bde0 con 0x7f7908003b70 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.960+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f790800bfe0 con 0x7f79240d15b0 2024-04-24T06:28:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.960+0000 7f7922a00640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7908003b70 0x7f7908003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f7923400640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7908004db0 0x7f7908005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7908003b70 msgr2=0x7f7908003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7908003b70 0x7f7908003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7908004db0 msgr2=0x7f7908005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7908004db0 0x7f7908005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.961+0000 7f791a000640 1 -- 172.21.15.59:0/2610066990 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7908003520 con 0x7f79240d15b0 2024-04-24T06:28:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.964+0000 7f791a000640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f7908004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f790c00da90 tx=0x7f7908005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8602efbc412d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:19.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.967+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c038070 con 0x7f79240d15b0 2024-04-24T06:28:19.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.967+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c033410 con 0x7f79240d15b0 2024-04-24T06:28:19.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.967+0000 7f7922000640 1 -- 172.21.15.59:0/2610066990 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c00d690 con 0x7f79240d15b0 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.968+0000 7f7923e00640 1 -- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f7908004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.968+0000 7f7923e00640 1 --2- 172.21.15.59:0/2610066990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f7908004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f790c00da90 tx=0x7f7908005830 comp rx=0 tx=0).stop 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.968+0000 7f7923400640 1 -- 172.21.15.59:0/2610066990 reap_dead start 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.968+0000 7f7923e00640 1 -- 172.21.15.59:0/2610066990 shutdown_connections 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.968+0000 7f7923e00640 1 -- 172.21.15.59:0/2610066990 >> 172.21.15.59:0/2610066990 conn(0x7f792408bf90 msgr2=0x7f79240d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.969+0000 7f7923e00640 1 -- 172.21.15.59:0/2610066990 shutdown_connections 2024-04-24T06:28:19.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.969+0000 7f7923e00640 1 -- 172.21.15.59:0/2610066990 wait complete. 2024-04-24T06:28:19.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.973+0000 7f7923e00640 1 Processor -- start 2024-04-24T06:28:19.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.973+0000 7f7923e00640 1 -- start start 2024-04-24T06:28:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79241f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79240b9660 0x7f79241619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79240b3600 con 0x7f79240b91e0 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79240d3490 con 0x7f79240b9660 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f7923e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f79240b3780 con 0x7f79240d15b0 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f791a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f791a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49812/0 (socket says 172.21.15.59:49812) 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.975+0000 7f791a000640 1 -- 172.21.15.59:0/3474439625 learned_addr learned my addr 172.21.15.59:0/3474439625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f7922a00640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79241f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f7923400640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79240b9660 0x7f79241619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f791a000640 1 -- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 msgr2=0x7f79241f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f791a000640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79241f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f791a000640 1 -- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79240b9660 msgr2=0x7f79241619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f791a000640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79240b9660 0x7f79241619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.976+0000 7f791a000640 1 -- 172.21.15.59:0/3474439625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241f57e0 con 0x7f79240b91e0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.977+0000 7f791a000640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f790c00dbb0 tx=0x7f790c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.978+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c042070 con 0x7f79240b91e0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.978+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241609d0 con 0x7f79240b91e0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.978+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c03d660 con 0x7f79240b91e0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.978+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c038040 con 0x7f79240b91e0 2024-04-24T06:28:19.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.978+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7924160c70 con 0x7f79240b91e0 2024-04-24T06:28:19.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.980+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f790c045070 con 0x7f79240b91e0 2024-04-24T06:28:19.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.980+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f790c047580 con 0x7f79240b91e0 2024-04-24T06:28:19.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.980+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79240d13d0 con 0x7f79240b91e0 2024-04-24T06:28:19.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:19.984+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f79240d13d0 con 0x7f79240b91e0 2024-04-24T06:28:20.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.122+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f792404fd90 con 0x7f79240b91e0 2024-04-24T06:28:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.126+0000 7f791b400640 1 -- 172.21.15.59:0/3474439625 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f79241f57e0 con 0x7f79240b91e0 2024-04-24T06:28:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 msgr2=0x7f7924161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f790c00dbb0 tx=0x7f790c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 shutdown_connections 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79240b9660 0x7f79241619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79240b91e0 0x7f7924161470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 --2- 172.21.15.59:0/3474439625 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f79240d15b0 0x7f79241f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:20.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 >> 172.21.15.59:0/3474439625 conn(0x7f792408bf90 msgr2=0x7f79240b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 shutdown_connections 2024-04-24T06:28:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:20.128+0000 7f7923e00640 1 -- 172.21.15.59:0/3474439625 wait complete. 2024-04-24T06:28:20.134 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:20.132+0000 7f269251c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:28:20.148 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T06:28:20.148 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T06:28:20.149 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:20.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T06:28:20.181 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:20.179+0000 7f8b45932040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:28:20.220 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:20.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T06:28:20.329 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:20.327+0000 7f269251c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:28:20.341 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:20.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T06:28:20.367 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:20.365+0000 7f8b45932040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:28:20.398 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:20.396+0000 7f269251c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:28:20.419 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:28:20.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T06:28:20.435 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:20.433+0000 7f8b45932040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:28:20.508 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:20.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T06:28:20.587 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:20.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T06:28:20.706 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:20.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T06:28:20.713 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:20.712+0000 7f269251c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:28:20.743 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:20.741+0000 7f8b45932040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:28:20.774 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:20.772+0000 7f269251c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:28:20.788 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:28:20.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T06:28:20.798 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:20.796+0000 7f8b45932040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:28:20.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new a9dfda38-de77-4e28-9c07-9d8651a35103 0 2024-04-24T06:28:21.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.135+0000 7f3b86800640 1 Processor -- start 2024-04-24T06:28:21.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.136+0000 7f3b86800640 1 -- start start 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b8015cb20 0x7f3b800cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 0x7f3b800d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3b80160180 0x7f3b800d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8015cf00 con 0x7f3b80160180 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3b800d01c0 con 0x7f3b8015cb20 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b86800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8015f6a0 con 0x7f3b80163770 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b7f400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b8015cb20 0x7f3b800cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.137+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 0x7f3b800d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 0x7f3b800d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41306/0 (socket says 172.21.15.66:41306) 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b77e00640 1 -- 172.21.15.66:0/479420299 learned_addr learned my addr 172.21.15.66:0/479420299 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7fe00640 1 --2- 172.21.15.66:0/479420299 >> v2:172.21.15.59:3300/0 conn(0x7f3b80160180 0x7f3b800d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7f400640 1 -- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 msgr2=0x7f3b800d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7f400640 1 --2- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 0x7f3b800d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7f400640 1 -- 172.21.15.66:0/479420299 >> v2:172.21.15.59:3300/0 conn(0x7f3b80160180 msgr2=0x7f3b800d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7f400640 1 --2- 172.21.15.66:0/479420299 >> v2:172.21.15.59:3300/0 conn(0x7f3b80160180 0x7f3b800d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.138+0000 7f3b7f400640 1 -- 172.21.15.66:0/479420299 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8015f980 con 0x7f3b8015cb20 2024-04-24T06:28:21.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.140+0000 7f3b7f400640 1 --2- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b8015cb20 0x7f3b800cf840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b8008cb40 tx=0x7f3b7002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a11e78df2e8604 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.140+0000 7f3b7ea00640 1 -- 172.21.15.66:0/479420299 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70037070 con 0x7f3b8015cb20 2024-04-24T06:28:21.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.140+0000 7f3b7ea00640 1 -- 172.21.15.66:0/479420299 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7002add0 con 0x7f3b8015cb20 2024-04-24T06:28:21.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.140+0000 7f3b7ea00640 1 -- 172.21.15.66:0/479420299 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70032660 con 0x7f3b8015cb20 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 -- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b8015cb20 msgr2=0x7f3b800cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 --2- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b8015cb20 0x7f3b800cf840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b8008cb40 tx=0x7f3b7002a080 comp rx=0 tx=0).stop 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 -- 172.21.15.66:0/479420299 shutdown_connections 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 --2- 172.21.15.66:0/479420299 >> v2:172.21.15.59:3300/0 conn(0x7f3b80160180 0x7f3b800d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 --2- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b80163770 0x7f3b800d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 --2- 172.21.15.66:0/479420299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b8015cb20 0x7f3b800cf840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 -- 172.21.15.66:0/479420299 >> 172.21.15.66:0/479420299 conn(0x7f3b8006cc70 msgr2=0x7f3b8006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 -- 172.21.15.66:0/479420299 shutdown_connections 2024-04-24T06:28:21.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.141+0000 7f3b86800640 1 -- 172.21.15.66:0/479420299 wait complete. 2024-04-24T06:28:21.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.143+0000 7f3b86800640 1 Processor -- start 2024-04-24T06:28:21.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.143+0000 7f3b86800640 1 -- start start 2024-04-24T06:28:21.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 0x7f3b801ec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b80163770 0x7f3b801568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800b62b0 con 0x7f3b80160180 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800b6430 con 0x7f3b80163770 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b86800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3b800b65b0 con 0x7f3b8015cb20 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41314/0 (socket says 172.21.15.66:41314) 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 -- 172.21.15.66:0/3682306444 learned_addr learned my addr 172.21.15.66:0/3682306444 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7fe00640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b80163770 0x7f3b801568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b77e00640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 0x7f3b801ec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 -- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b80163770 msgr2=0x7f3b801568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b80163770 0x7f3b801568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 -- 172.21.15.66:0/3682306444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 msgr2=0x7f3b801ec1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.144+0000 7f3b7f400640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 0x7f3b801ec1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b7f400640 1 -- 172.21.15.66:0/3682306444 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b800c89f0 con 0x7f3b8015cb20 2024-04-24T06:28:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b77e00640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 0x7f3b801ec1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:21.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b7f400640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009890 tx=0x7f3b700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70041070 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7003c460 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70037040 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b800b8340 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.145+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b800b8900 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.146+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b70044070 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.146+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f3b70046920 con 0x7f3b8015cb20 2024-04-24T06:28:21.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.147+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b80068a80 con 0x7f3b8015cb20 2024-04-24T06:28:21.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.149+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3b7003c830 con 0x7f3b8015cb20 2024-04-24T06:28:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.230+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9dfda38-de77-4e28-9c07-9d8651a35103", "id": 0} v 0) -- 0x7f3b800cb6a0 con 0x7f3b8015cb20 2024-04-24T06:28:21.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.236+0000 7f3b77400640 1 -- 172.21.15.66:0/3682306444 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9dfda38-de77-4e28-9c07-9d8651a35103", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f3b70046b20 con 0x7f3b8015cb20 2024-04-24T06:28:21.238 INFO:teuthology.orchestra.run.smithi066.stdout:0 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 msgr2=0x7f3b800c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009890 tx=0x7f3b700046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 shutdown_connections 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b80163770 0x7f3b801568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b80160180 0x7f3b801ec1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 --2- 172.21.15.66:0/3682306444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b8015cb20 0x7f3b800c8360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.240 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 >> 172.21.15.66:0/3682306444 conn(0x7f3b8006cc70 msgr2=0x7f3b8015e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 shutdown_connections 2024-04-24T06:28:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.238+0000 7f3b86800640 1 -- 172.21.15.66:0/3682306444 wait complete. 2024-04-24T06:28:21.256 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new 2cdfd9f0-875c-4896-8b64-f84e435ed544 1 2024-04-24T06:28:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.428+0000 7fe628600640 1 Processor -- start 2024-04-24T06:28:21.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.428+0000 7fe628600640 1 -- start start 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe628600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe62015e9d0 0x7fe62015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe628600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6200d42c0 0x7fe6200d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe628600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62015edb0 con 0x7fe6200ba0e0 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe628600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200d3e30 con 0x7fe6200d42c0 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe628600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe62015ff20 con 0x7fe62015e9d0 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe625c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe62015e9d0 0x7fe62015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe625c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe62015e9d0 0x7fe62015f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55230/0 (socket says 172.21.15.66:55230) 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.431+0000 7fe625c00640 1 -- 172.21.15.66:0/615434988 learned_addr learned my addr 172.21.15.66:0/615434988 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.432+0000 7fe625200640 1 --2- 172.21.15.66:0/615434988 >> v2:172.21.15.59:3300/0 conn(0x7fe6200d42c0 0x7fe6200d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.432+0000 7fe625c00640 1 -- 172.21.15.66:0/615434988 >> v2:172.21.15.59:3300/0 conn(0x7fe6200d42c0 msgr2=0x7fe6200d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.432+0000 7fe625c00640 1 --2- 172.21.15.66:0/615434988 >> v2:172.21.15.59:3300/0 conn(0x7fe6200d42c0 0x7fe6200d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.432+0000 7fe625c00640 1 -- 172.21.15.66:0/615434988 >> v1:172.21.15.59:6789/0 conn(0x7fe6200ba0e0 legacy=0x7fe6200b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.432+0000 7fe625c00640 1 -- 172.21.15.66:0/615434988 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200ba630 con 0x7fe62015e9d0 2024-04-24T06:28:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.433+0000 7fe625c00640 1 --2- 172.21.15.66:0/615434988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe62015e9d0 0x7fe62015f9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe6200d3fb0 tx=0x7fe61402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fd3ae8f25a529c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.433+0000 7fe61fe00640 1 -- 172.21.15.66:0/615434988 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614037070 con 0x7fe62015e9d0 2024-04-24T06:28:21.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.433+0000 7fe61fe00640 1 -- 172.21.15.66:0/615434988 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614032440 con 0x7fe62015e9d0 2024-04-24T06:28:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.433+0000 7fe61fe00640 1 -- 172.21.15.66:0/615434988 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614032810 con 0x7fe62015e9d0 2024-04-24T06:28:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 -- 172.21.15.66:0/615434988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe62015e9d0 msgr2=0x7fe62015f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 --2- 172.21.15.66:0/615434988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe62015e9d0 0x7fe62015f9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe6200d3fb0 tx=0x7fe61402a180 comp rx=0 tx=0).stop 2024-04-24T06:28:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 -- 172.21.15.66:0/615434988 shutdown_connections 2024-04-24T06:28:21.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 --2- 172.21.15.66:0/615434988 >> v2:172.21.15.59:3300/0 conn(0x7fe6200d42c0 0x7fe6200d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 --2- 172.21.15.66:0/615434988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe62015e9d0 0x7fe62015f9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 -- 172.21.15.66:0/615434988 >> 172.21.15.66:0/615434988 conn(0x7fe62006cbd0 msgr2=0x7fe62006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 -- 172.21.15.66:0/615434988 shutdown_connections 2024-04-24T06:28:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.434+0000 7fe628600640 1 -- 172.21.15.66:0/615434988 wait complete. 2024-04-24T06:28:21.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.436+0000 7fe628600640 1 Processor -- start 2024-04-24T06:28:21.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.436+0000 7fe628600640 1 -- start start 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6200d42c0 0x7fe6200c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 0x7fe6200cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cfd70 con 0x7fe62015e9d0 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cfa90 con 0x7fe6200ba0e0 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe628600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200d14a0 con 0x7fe6200d42c0 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe625c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe625c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55232/0 (socket says 172.21.15.66:55232) 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe625c00640 1 -- 172.21.15.66:0/1361893019 learned_addr learned my addr 172.21.15.66:0/1361893019 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe625200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6200d42c0 0x7fe6200c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.437+0000 7fe626600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 0x7fe6200cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 -- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6200d42c0 msgr2=0x7fe6200c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6200d42c0 0x7fe6200c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 -- 172.21.15.66:0/1361893019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 msgr2=0x7fe6200cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 0x7fe6200cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 -- 172.21.15.66:0/1361893019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200c8d40 con 0x7fe6200ba0e0 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe626600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 0x7fe6200cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe625c00640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe6140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614041020 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140306c0 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6200c9020 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.438+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614037040 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.439+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6200b4340 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.439+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe614044020 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.439+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fe61403dcc0 con 0x7fe6200ba0e0 2024-04-24T06:28:21.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.440+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6200b3dd0 con 0x7fe6200ba0e0 2024-04-24T06:28:21.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.443+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe6200b3dd0 con 0x7fe6200ba0e0 2024-04-24T06:28:21.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.522+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2cdfd9f0-875c-4896-8b64-f84e435ed544", "id": 1} v 0) -- 0x7fe620069190 con 0x7fe6200ba0e0 2024-04-24T06:28:21.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.530+0000 7fe61d600640 1 -- 172.21.15.66:0/1361893019 <== mon.1 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2cdfd9f0-875c-4896-8b64-f84e435ed544", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fe61404b030 con 0x7fe6200ba0e0 2024-04-24T06:28:21.532 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.531+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 msgr2=0x7fe62014bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.531+0000 7fe628600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe6140046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 shutdown_connections 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e9d0 0x7fe6200cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6200d42c0 0x7fe6200c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 --2- 172.21.15.66:0/1361893019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6200ba0e0 0x7fe62014bcf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 >> 172.21.15.66:0/1361893019 conn(0x7fe62006cbd0 msgr2=0x7fe6200d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 shutdown_connections 2024-04-24T06:28:21.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.532+0000 7fe628600640 1 -- 172.21.15.66:0/1361893019 wait complete. 2024-04-24T06:28:21.551 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new 35f4ea9b-5f4e-456f-b819-b9121469d316 2 2024-04-24T06:28:21.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.736+0000 7f54fdc00640 1 Processor -- start 2024-04-24T06:28:21.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.737+0000 7f54fdc00640 1 -- start start 2024-04-24T06:28:21.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.738+0000 7f54fdc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54fdc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f54f80b3600 0x7f54f80d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54fdc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f80b5da0 con 0x7f54f80b3f10 2024-04-24T06:28:21.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54fdc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f54f815f100 con 0x7f54f80b3600 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54fdc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f54f80d0a20 con 0x7f54f815ed20 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41338/0 (socket says 172.21.15.66:41338) 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 -- 172.21.15.66:0/562423779 learned_addr learned my addr 172.21.15.66:0/562423779 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7400640 1 --2- 172.21.15.66:0/562423779 >> v2:172.21.15.66:3300/0 conn(0x7f54f80b3600 0x7f54f80d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 -- 172.21.15.66:0/562423779 >> v2:172.21.15.66:3300/0 conn(0x7f54f80b3600 msgr2=0x7f54f80d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 --2- 172.21.15.66:0/562423779 >> v2:172.21.15.66:3300/0 conn(0x7f54f80b3600 0x7f54f80d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 -- 172.21.15.66:0/562423779 >> v1:172.21.15.59:6789/0 conn(0x7f54f80b3f10 legacy=0x7f54f80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:21.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.739+0000 7f54f7e00640 1 -- 172.21.15.66:0/562423779 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f80d2ed0 con 0x7f54f815ed20 2024-04-24T06:28:21.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.741+0000 7f54f7e00640 1 --2- 172.21.15.66:0/562423779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009870 tx=0x7f54e802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1ff33838b4548a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.741+0000 7f54f6a00640 1 -- 172.21.15.66:0/562423779 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037070 con 0x7f54f815ed20 2024-04-24T06:28:21.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.741+0000 7f54f6a00640 1 -- 172.21.15.66:0/562423779 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e802ae00 con 0x7f54f815ed20 2024-04-24T06:28:21.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.741+0000 7f54f6a00640 1 -- 172.21.15.66:0/562423779 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8032660 con 0x7f54f815ed20 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 -- 172.21.15.66:0/562423779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 msgr2=0x7f54f80d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 --2- 172.21.15.66:0/562423779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009870 tx=0x7f54e802a0f0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 -- 172.21.15.66:0/562423779 shutdown_connections 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 --2- 172.21.15.66:0/562423779 >> v2:172.21.15.66:3300/0 conn(0x7f54f80b3600 0x7f54f80d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 --2- 172.21.15.66:0/562423779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f815ed20 0x7f54f80d0ea0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 -- 172.21.15.66:0/562423779 >> 172.21.15.66:0/562423779 conn(0x7f54f808bf90 msgr2=0x7f54f80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 -- 172.21.15.66:0/562423779 shutdown_connections 2024-04-24T06:28:21.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.742+0000 7f54fdc00640 1 -- 172.21.15.66:0/562423779 wait complete. 2024-04-24T06:28:21.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.744+0000 7f54fdc00640 1 Processor -- start 2024-04-24T06:28:21.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.744+0000 7f54fdc00640 1 -- start start 2024-04-24T06:28:21.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54f815ed20 0x7f54f80b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54f80d0290 con 0x7f54f815ed20 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54f80d3490 con 0x7f54f80b3600 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fdc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f54f80d0410 con 0x7f54f80b3f10 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41354/0 (socket says 172.21.15.66:41354) 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55244/0 (socket says 172.21.15.66:55244) 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 -- 172.21.15.66:0/2431601711 learned_addr learned my addr 172.21.15.66:0/2431601711 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54fd200640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54f815ed20 0x7f54f80b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 -- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 msgr2=0x7f54f80cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 -- 172.21.15.66:0/2431601711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54f815ed20 msgr2=0x7f54f80b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54f815ed20 0x7f54f80b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7400640 1 -- 172.21.15.66:0/2431601711 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f81f4b40 con 0x7f54f80b3f10 2024-04-24T06:28:21.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.745+0000 7f54f7e00640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:21.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54f7400640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f54e400e9b0 tx=0x7f54e400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4017070 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e400c9a0 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f81f4e00 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e400cd90 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f81f53c0 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54e401a070 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.746+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f54e40066f0 con 0x7f54f80b3f10 2024-04-24T06:28:21.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.747+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f804fd90 con 0x7f54f80b3f10 2024-04-24T06:28:21.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.750+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f54e4006020 con 0x7f54f80b3f10 2024-04-24T06:28:21.753 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:21.751+0000 7f269251c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:28:21.801 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:21.799+0000 7f8b45932040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:28:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.830+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd new", "uuid": "35f4ea9b-5f4e-456f-b819-b9121469d316", "id": 2} v 0) -- 0x7f54f80d2500 con 0x7f54f80b3f10 2024-04-24T06:28:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.837+0000 7f54efe00640 1 -- 172.21.15.66:0/2431601711 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35f4ea9b-5f4e-456f-b819-b9121469d316", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f54e4005990 con 0x7f54f80b3f10 2024-04-24T06:28:21.839 INFO:teuthology.orchestra.run.smithi066.stdout:2 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 msgr2=0x7f54f80b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f54e400e9b0 tx=0x7f54e400ee70 comp rx=0 tx=0).stop 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 shutdown_connections 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54f815ed20 0x7f54f80b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54f80b3f10 0x7f54f80b4b70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 --2- 172.21.15.66:0/2431601711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f54f80b3600 0x7f54f80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.838+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 >> 172.21.15.66:0/2431601711 conn(0x7f54f808bf90 msgr2=0x7f54f80d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.839+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 shutdown_connections 2024-04-24T06:28:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:21.839+0000 7f54fdc00640 1 -- 172.21.15.66:0/2431601711 wait complete. 2024-04-24T06:28:21.858 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new e444ebd4-f529-486c-803d-a536f20d5b73 3 2024-04-24T06:28:21.873 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:21.871+0000 7f269251c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:28:21.910 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:21.908+0000 7f8b45932040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:28:21.933 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:21.931+0000 7f269251c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:28:21.963 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:21.961+0000 7f8b45932040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:28:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.043+0000 7f1e94a00640 1 Processor -- start 2024-04-24T06:28:22.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.043+0000 7f1e94a00640 1 -- start start 2024-04-24T06:28:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.045+0000 7f1e94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e900b4f20 0x7f1e900cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.045+0000 7f1e94a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900b5da0 con 0x7f1e900d15b0 2024-04-24T06:28:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e94a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900d0a20 con 0x7f1e9015ed20 2024-04-24T06:28:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e94a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9015f100 con 0x7f1e900b4f20 2024-04-24T06:28:22.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f1e9015ed20 0x7f1e900d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33770/0 (socket says 172.21.15.66:33770) 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8f400640 1 -- 172.21.15.66:0/4198228297 learned_addr learned my addr 172.21.15.66:0/4198228297 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449930195 0 0) 0x7f1e900d0a20 con 0x7f1e9015ed20 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e74003520 con 0x7f1e9015ed20 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8fe00640 1 --2- 172.21.15.66:0/4198228297 >> v2:172.21.15.59:3300/0 conn(0x7f1e900b4f20 0x7f1e900cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3578039228 0 0) 0x7f1e74003520 con 0x7f1e9015ed20 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 >> v2:172.21.15.59:3300/0 conn(0x7f1e900b4f20 msgr2=0x7f1e900cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 --2- 172.21.15.66:0/4198228297 >> v2:172.21.15.59:3300/0 conn(0x7f1e900b4f20 0x7f1e900cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.046+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 >> v1:172.21.15.59:6789/0 conn(0x7f1e900d15b0 legacy=0x7f1e900cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900d2ed0 con 0x7f1e9015ed20 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1e80001670 con 0x7f1e9015ed20 2024-04-24T06:28:22.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 >> v1:172.21.15.66:6789/0 conn(0x7f1e9015ed20 legacy=0x7f1e900d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 0x7f1e74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e740053c0 0x7f1e7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7400c270 con 0x7f1e74004800 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7400c420 con 0x7f1e74003a00 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7400c620 con 0x7f1e740053c0 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8f400640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 0x7f1e74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e8fe00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e740053c0 0x7f1e7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.047+0000 7f1e86a00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8f400640 1 -- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e740053c0 msgr2=0x7f1e7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8f400640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e740053c0 0x7f1e7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8f400640 1 -- 172.21.15.66:0/4198228297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74004800 msgr2=0x7f1e74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8f400640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8f400640 1 -- 172.21.15.66:0/4198228297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e74003520 con 0x7f1e74003a00 2024-04-24T06:28:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e86a00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74004800 0x7f1e74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:22.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.048+0000 7f1e8fe00640 1 -- 172.21.15.66:0/4198228297 reap_dead start 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e8f400640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 0x7f1e74003e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c190 tx=0x7f1e74005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d333353e411945b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80007c60 con 0x7f1e74003a00 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80005af0 con 0x7f1e74003a00 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e8ea00640 1 -- 172.21.15.66:0/4198228297 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80007070 con 0x7f1e74003a00 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e94a00640 1 -- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 msgr2=0x7f1e74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.049+0000 7f1e94a00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 0x7f1e74003e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c190 tx=0x7f1e74005830 comp rx=0 tx=0).stop 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.050+0000 7f1e94a00640 1 -- 172.21.15.66:0/4198228297 shutdown_connections 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.050+0000 7f1e94a00640 1 --2- 172.21.15.66:0/4198228297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e74003a00 0x7f1e74003e20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.050+0000 7f1e94a00640 1 -- 172.21.15.66:0/4198228297 >> 172.21.15.66:0/4198228297 conn(0x7f1e9008bf90 msgr2=0x7f1e900d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.057 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.050+0000 7f1e94a00640 1 -- 172.21.15.66:0/4198228297 shutdown_connections 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.050+0000 7f1e94a00640 1 -- 172.21.15.66:0/4198228297 wait complete. 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.051+0000 7f1e94a00640 1 Processor -- start 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 -- start start 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74003a00 0x7f1e900c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 0x7f1e90160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e900b3600 con 0x7f1e74003a00 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.052+0000 7f1e94a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e900d3490 con 0x7f1e90160b20 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e94a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1e900b3780 con 0x7f1e900caa90 2024-04-24T06:28:22.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 0x7f1e90160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41378/0 (socket says 172.21.15.66:41378) 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 -- 172.21.15.66:0/3680257400 learned_addr learned my addr 172.21.15.66:0/3680257400 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 0x7f1e90160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55274/0 (socket says 172.21.15.66:55274) 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e8f400640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74003a00 0x7f1e900c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 -- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 msgr2=0x7f1e90160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 0x7f1e90160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 -- 172.21.15.66:0/3680257400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74003a00 msgr2=0x7f1e900c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74003a00 0x7f1e900c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.053+0000 7f1e86a00640 1 -- 172.21.15.66:0/3680257400 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900caf10 con 0x7f1e900caa90 2024-04-24T06:28:22.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e86a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e440 tx=0x7f1e7800e900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78019070 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e780053a0 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78003810 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e900c9200 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e900c97c0 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e7801c070 con 0x7f1e900caa90 2024-04-24T06:28:22.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.054+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1e78014480 con 0x7f1e900caa90 2024-04-24T06:28:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.055+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e900d0700 con 0x7f1e900caa90 2024-04-24T06:28:22.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.057+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1e900d0700 con 0x7f1e900caa90 2024-04-24T06:28:22.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.135+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e444ebd4-f529-486c-803d-a536f20d5b73", "id": 3} v 0) -- 0x7f1e90069190 con 0x7f1e900caa90 2024-04-24T06:28:22.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.140+0000 7f1e87e00640 1 -- 172.21.15.66:0/3680257400 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e444ebd4-f529-486c-803d-a536f20d5b73", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1e90069190 con 0x7f1e900caa90 2024-04-24T06:28:22.143 INFO:teuthology.orchestra.run.smithi066.stdout:3 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 msgr2=0x7f1e90159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e440 tx=0x7f1e7800e900 comp rx=0 tx=0).stop 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 shutdown_connections 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e90160b20 0x7f1e90160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e900caa90 0x7f1e90159b20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 --2- 172.21.15.66:0/3680257400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e74003a00 0x7f1e900c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 >> 172.21.15.66:0/3680257400 conn(0x7f1e9008bf90 msgr2=0x7f1e9006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 shutdown_connections 2024-04-24T06:28:22.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.142+0000 7f1e94a00640 1 -- 172.21.15.66:0/3680257400 wait complete. 2024-04-24T06:28:22.153 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:22.151+0000 7f269251c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:28:22.162 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:22.161+0000 7f8b45932040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:28:22.166 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new 8d9d7e5e-8655-4873-845e-49975cfd2ce3 4 2024-04-24T06:28:22.220 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:22.218+0000 7f8b45932040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:28:22.221 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:22.219+0000 7f269251c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:28:22.284 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:22.282+0000 7f8b45932040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:28:22.288 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:22.286+0000 7f269251c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:28:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.394+0000 7ff382a00640 1 Processor -- start 2024-04-24T06:28:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.394+0000 7ff382a00640 1 -- start start 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff382a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff37c0d0de0 0x7ff37c0d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff382a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37c0d22e0 con 0x7ff37c161c50 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff382a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37c0d2550 con 0x7ff37c0d5240 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff382a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff37c0d2ba0 con 0x7ff37c0d0de0 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff380c00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7ff37c0d5240 0x7ff37c0d3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33772/0 (socket says 172.21.15.66:33772) 2024-04-24T06:28:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff380c00640 1 -- 172.21.15.66:0/1389478776 learned_addr learned my addr 172.21.15.66:0/1389478776 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff382000640 1 --2- 172.21.15.66:0/1389478776 >> v2:172.21.15.59:3300/0 conn(0x7ff37c0d0de0 0x7ff37c0d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217228933 0 0) 0x7ff37c0d2550 con 0x7ff37c0d5240 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.396+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff364003520 con 0x7ff37c0d5240 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133398082 0 0) 0x7ff364003520 con 0x7ff37c0d5240 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 >> v2:172.21.15.59:3300/0 conn(0x7ff37c0d0de0 msgr2=0x7ff37c0d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 --2- 172.21.15.66:0/1389478776 >> v2:172.21.15.59:3300/0 conn(0x7ff37c0d0de0 0x7ff37c0d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 >> v1:172.21.15.59:6789/0 conn(0x7ff37c161c50 legacy=0x7ff37c0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0d2830 con 0x7ff37c0d5240 2024-04-24T06:28:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff368001670 con 0x7ff37c0d5240 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 >> v1:172.21.15.66:6789/0 conn(0x7ff37c0d5240 legacy=0x7ff37c0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 0x7ff364003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff364004800 0x7ff364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640051c0 0x7ff36400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0d2ba0 con 0x7ff3640051c0 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff36400c0c0 con 0x7ff3640039b0 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff36400c2c0 con 0x7ff364004800 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff381600640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 0x7ff364003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.397+0000 7ff380c00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff364004800 0x7ff364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff382000640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640051c0 0x7ff36400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff381600640 1 -- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff364004800 msgr2=0x7ff364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff381600640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff364004800 0x7ff364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff381600640 1 -- 172.21.15.66:0/1389478776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640051c0 msgr2=0x7ff36400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff381600640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640051c0 0x7ff36400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff381600640 1 -- 172.21.15.66:0/1389478776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff364003520 con 0x7ff3640039b0 2024-04-24T06:28:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.398+0000 7ff382000640 1 -- 172.21.15.66:0/1389478776 reap_dead start 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.399+0000 7ff381600640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 0x7ff364003db0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff370001780 tx=0x7ff364005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a4c35ab39b3037 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.399+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370001350 con 0x7ff3640039b0 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.399+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3700052d0 con 0x7ff3640039b0 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.399+0000 7ff37be00640 1 -- 172.21.15.66:0/1389478776 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37002ab40 con 0x7ff3640039b0 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 -- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 msgr2=0x7ff364003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 0x7ff364003db0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff370001780 tx=0x7ff364005830 comp rx=0 tx=0).stop 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 -- 172.21.15.66:0/1389478776 shutdown_connections 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 --2- 172.21.15.66:0/1389478776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3640039b0 0x7ff364003db0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 -- 172.21.15.66:0/1389478776 >> 172.21.15.66:0/1389478776 conn(0x7ff37c08bf90 msgr2=0x7ff37c0d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 -- 172.21.15.66:0/1389478776 shutdown_connections 2024-04-24T06:28:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.400+0000 7ff382a00640 1 -- 172.21.15.66:0/1389478776 wait complete. 2024-04-24T06:28:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.402+0000 7ff382a00640 1 Processor -- start 2024-04-24T06:28:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.402+0000 7ff382a00640 1 -- start start 2024-04-24T06:28:22.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640039b0 0x7ff37c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff37c15db30 0x7ff37c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0d1270 con 0x7ff3640039b0 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0d0460 con 0x7ff37c15db30 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c0d13f0 con 0x7ff37c0c9900 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff382000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff37c15db30 0x7ff37c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41396/0 (socket says 172.21.15.66:41396) 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 -- 172.21.15.66:0/858502152 learned_addr learned my addr 172.21.15.66:0/858502152 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff381600640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640039b0 0x7ff37c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 -- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff37c15db30 msgr2=0x7ff37c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff37c15db30 0x7ff37c0cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 -- 172.21.15.66:0/858502152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640039b0 msgr2=0x7ff37c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.403+0000 7ff380c00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640039b0 0x7ff37c14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff380c00640 1 -- 172.21.15.66:0/858502152 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c15c7d0 con 0x7ff37c0c9900 2024-04-24T06:28:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff380c00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff368007320 tx=0x7ff3680077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36801a070 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36800f940 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c15ca90 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36800fd10 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.404+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c15d050 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.405+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff36801d070 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.405+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7ff368014960 con 0x7ff37c0c9900 2024-04-24T06:28:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.405+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c04fd90 con 0x7ff37c0c9900 2024-04-24T06:28:22.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.408+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff368014450 con 0x7ff37c0c9900 2024-04-24T06:28:22.417 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:22.415+0000 7f8b45932040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:28:22.428 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:22.426+0000 7f269251c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:28:22.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.488+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8d9d7e5e-8655-4873-845e-49975cfd2ce3", "id": 4} v 0) -- 0x7ff37c0d1cf0 con 0x7ff37c0c9900 2024-04-24T06:28:22.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.495+0000 7ff379600640 1 -- 172.21.15.66:0/858502152 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d9d7e5e-8655-4873-845e-49975cfd2ce3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7ff368014630 con 0x7ff37c0c9900 2024-04-24T06:28:22.497 INFO:teuthology.orchestra.run.smithi066.stdout:4 2024-04-24T06:28:22.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 msgr2=0x7ff37c15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff368007320 tx=0x7ff3680077e0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 shutdown_connections 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff37c15db30 0x7ff37c0cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff37c0c9900 0x7ff37c15c290 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 --2- 172.21.15.66:0/858502152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3640039b0 0x7ff37c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 >> 172.21.15.66:0/858502152 conn(0x7ff37c08bf90 msgr2=0x7ff37c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.496+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 shutdown_connections 2024-04-24T06:28:22.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.497+0000 7ff382a00640 1 -- 172.21.15.66:0/858502152 wait complete. 2024-04-24T06:28:22.515 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a 5 2024-04-24T06:28:22.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.742+0000 7fd6eb200640 1 Processor -- start 2024-04-24T06:28:22.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.742+0000 7fd6eb200640 1 -- start start 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.743+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.743+0000 7fd6eb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6ec0d0de0 0x7fd6ec0d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6eb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ec0d22e0 con 0x7fd6ec161c50 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6eb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ec0d2ba0 con 0x7fd6ec0d0de0 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6eb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec0d2550 con 0x7fd6ec0d5240 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41404/0 (socket says 172.21.15.66:41404) 2024-04-24T06:28:22.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 -- 172.21.15.66:0/1358634621 learned_addr learned my addr 172.21.15.66:0/1358634621 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:22.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6ea800640 1 --2- 172.21.15.66:0/1358634621 >> v2:172.21.15.59:3300/0 conn(0x7fd6ec0d0de0 0x7fd6ec0d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 -- 172.21.15.66:0/1358634621 >> v2:172.21.15.59:3300/0 conn(0x7fd6ec0d0de0 msgr2=0x7fd6ec0d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 --2- 172.21.15.66:0/1358634621 >> v2:172.21.15.59:3300/0 conn(0x7fd6ec0d0de0 0x7fd6ec0d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 -- 172.21.15.66:0/1358634621 >> v1:172.21.15.59:6789/0 conn(0x7fd6ec161c50 legacy=0x7fd6ec0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:22.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.744+0000 7fd6e9400640 1 -- 172.21.15.66:0/1358634621 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0d2830 con 0x7fd6ec0d5240 2024-04-24T06:28:22.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.746+0000 7fd6e9400640 1 --2- 172.21.15.66:0/1358634621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327f34acb0ba3675 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.746+0000 7fd6e8a00640 1 -- 172.21.15.66:0/1358634621 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4036070 con 0x7fd6ec0d5240 2024-04-24T06:28:22.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.746+0000 7fd6e8a00640 1 -- 172.21.15.66:0/1358634621 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d4031600 con 0x7fd6ec0d5240 2024-04-24T06:28:22.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.746+0000 7fd6e8a00640 1 -- 172.21.15.66:0/1358634621 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40319d0 con 0x7fd6ec0d5240 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 -- 172.21.15.66:0/1358634621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 msgr2=0x7fd6ec0d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 --2- 172.21.15.66:0/1358634621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d402a6c0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 -- 172.21.15.66:0/1358634621 shutdown_connections 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 --2- 172.21.15.66:0/1358634621 >> v2:172.21.15.59:3300/0 conn(0x7fd6ec0d0de0 0x7fd6ec0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 --2- 172.21.15.66:0/1358634621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec0d5240 0x7fd6ec0d3030 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 -- 172.21.15.66:0/1358634621 >> 172.21.15.66:0/1358634621 conn(0x7fd6ec08bf90 msgr2=0x7fd6ec0d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 -- 172.21.15.66:0/1358634621 shutdown_connections 2024-04-24T06:28:22.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.747+0000 7fd6eb200640 1 -- 172.21.15.66:0/1358634621 wait complete. 2024-04-24T06:28:22.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.749+0000 7fd6eb200640 1 Processor -- start 2024-04-24T06:28:22.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.749+0000 7fd6eb200640 1 -- start start 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.750+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ec0d0de0 0x7fd6ec0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.750+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 0x7fd6ec0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.750+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6eb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec0d3570 con 0x7fd6ec0d0de0 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6eb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec0d0460 con 0x7fd6ec0d5240 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6eb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec06cb70 con 0x7fd6ec161c50 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41414/0 (socket says 172.21.15.66:41414) 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6e9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 0x7fd6ec0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 -- 172.21.15.66:0/183471842 learned_addr learned my addr 172.21.15.66:0/183471842 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6e9e00640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ec0d0de0 0x7fd6ec0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 -- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 msgr2=0x7fd6ec0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 0x7fd6ec0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 -- 172.21.15.66:0/183471842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ec0d0de0 msgr2=0x7fd6ec0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ec0d0de0 0x7fd6ec0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6ea800640 1 -- 172.21.15.66:0/183471842 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0b8e20 con 0x7fd6ec161c50 2024-04-24T06:28:22.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.751+0000 7fd6e9400640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 0x7fd6ec0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6ea800640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c7f0 tx=0x7fd6e400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4015070 con 0x7fd6ec161c50 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e4004220 con 0x7fd6ec161c50 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4007620 con 0x7fd6ec161c50 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec0b9100 con 0x7fd6ec161c50 2024-04-24T06:28:22.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec1f7a00 con 0x7fd6ec161c50 2024-04-24T06:28:22.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6e40079d0 con 0x7fd6ec161c50 2024-04-24T06:28:22.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.752+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fd6e4003b90 con 0x7fd6ec161c50 2024-04-24T06:28:22.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.753+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec1f7c90 con 0x7fd6ec161c50 2024-04-24T06:28:22.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.755+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd6ec1f7c90 con 0x7fd6ec161c50 2024-04-24T06:28:22.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.835+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a", "id": 5} v 0) -- 0x7fd6ec04fd90 con 0x7fd6ec161c50 2024-04-24T06:28:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.840+0000 7fd6e2000640 1 -- 172.21.15.66:0/183471842 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fd6e4007d90 con 0x7fd6ec161c50 2024-04-24T06:28:22.842 INFO:teuthology.orchestra.run.smithi066.stdout:5 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 msgr2=0x7fd6ec1f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c7f0 tx=0x7fd6e400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 shutdown_connections 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6ec161c50 0x7fd6ec1f74c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6ec0d5240 0x7fd6ec0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 --2- 172.21.15.66:0/183471842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ec0d0de0 0x7fd6ec0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 >> 172.21.15.66:0/183471842 conn(0x7fd6ec08bf90 msgr2=0x7fd6ec0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 shutdown_connections 2024-04-24T06:28:22.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:22.842+0000 7fd6eb200640 1 -- 172.21.15.66:0/183471842 wait complete. 2024-04-24T06:28:22.861 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new 33f963ab-a2df-4fda-a7e4-34d0116e6efa 6 2024-04-24T06:28:22.893 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:22.891+0000 7f8b45932040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:28:22.932 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:22.930+0000 7f269251c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:28:23.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.048+0000 7f0e6a800640 1 Processor -- start 2024-04-24T06:28:23.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.048+0000 7f0e6a800640 1 -- start start 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.049+0000 7f0e6a800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.049+0000 7f0e6a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e640d0de0 0x7f0e640d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.049+0000 7f0e6a800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e640d22e0 con 0x7f0e64161c50 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.049+0000 7f0e6a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0e640d2ba0 con 0x7f0e640d0de0 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.049+0000 7f0e6a800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e640d2550 con 0x7f0e640d5240 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.66:41426/0 (socket says 172.21.15.66:41426) 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 -- 172.21.15.66:0/3765615416 learned_addr learned my addr 172.21.15.66:0/3765615416 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e69e00640 1 --2- 172.21.15.66:0/3765615416 >> v2:172.21.15.66:3300/0 conn(0x7f0e640d0de0 0x7f0e640d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 -- 172.21.15.66:0/3765615416 >> v2:172.21.15.66:3300/0 conn(0x7f0e640d0de0 msgr2=0x7f0e640d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 --2- 172.21.15.66:0/3765615416 >> v2:172.21.15.66:3300/0 conn(0x7f0e640d0de0 0x7f0e640d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 -- 172.21.15.66:0/3765615416 >> v1:172.21.15.59:6789/0 conn(0x7f0e64161c50 legacy=0x7f0e640cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.050+0000 7f0e68a00640 1 -- 172.21.15.66:0/3765615416 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e640d2830 con 0x7f0e640d5240 2024-04-24T06:28:23.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.051+0000 7f0e68a00640 1 --2- 172.21.15.66:0/3765615416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f31a4b923a499b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e63e00640 1 -- 172.21.15.66:0/3765615416 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50037070 con 0x7f0e640d5240 2024-04-24T06:28:23.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e63e00640 1 -- 172.21.15.66:0/3765615416 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50032440 con 0x7f0e640d5240 2024-04-24T06:28:23.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e63e00640 1 -- 172.21.15.66:0/3765615416 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50032810 con 0x7f0e640d5240 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 -- 172.21.15.66:0/3765615416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 msgr2=0x7f0e640d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3765615416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002a250 comp rx=0 tx=0).stop 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 -- 172.21.15.66:0/3765615416 shutdown_connections 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3765615416 >> v2:172.21.15.66:3300/0 conn(0x7f0e640d0de0 0x7f0e640d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3765615416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e640d5240 0x7f0e640d3030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.052+0000 7f0e6a800640 1 -- 172.21.15.66:0/3765615416 >> 172.21.15.66:0/3765615416 conn(0x7f0e6408bf90 msgr2=0x7f0e640d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.053+0000 7f0e6a800640 1 -- 172.21.15.66:0/3765615416 shutdown_connections 2024-04-24T06:28:23.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.053+0000 7f0e6a800640 1 -- 172.21.15.66:0/3765615416 wait complete. 2024-04-24T06:28:23.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.054+0000 7f0e6a800640 1 Processor -- start 2024-04-24T06:28:23.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.054+0000 7f0e6a800640 1 -- start start 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e640d5240 0x7f0e6415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e64161c50 0x7f0e640b9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e640d3570 con 0x7f0e640d5240 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e640d0460 con 0x7f0e640d0de0 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e6a800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6406cb70 con 0x7f0e64161c50 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.055+0000 7f0e69400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55310/0 (socket says 172.21.15.66:55310) 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 -- 172.21.15.66:0/3175176460 learned_addr learned my addr 172.21.15.66:0/3175176460 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69e00640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e64161c50 0x7f0e640b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e68a00640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e640d5240 0x7f0e6415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 -- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e64161c50 msgr2=0x7f0e640b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e64161c50 0x7f0e640b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 -- 172.21.15.66:0/3175176460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e640d5240 msgr2=0x7f0e6415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e640d5240 0x7f0e6415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 -- 172.21.15.66:0/3175176460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e640ca0b0 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e69400640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c730 tx=0x7f0e5400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.056+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54003b70 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e54003d30 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400aa90 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e640ca390 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e640ca950 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e5400ac30 con 0x7f0e640d0de0 2024-04-24T06:28:23.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.057+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f0e540056c0 con 0x7f0e640d0de0 2024-04-24T06:28:23.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.058+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6404fd90 con 0x7f0e640d0de0 2024-04-24T06:28:23.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.060+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0e540050d0 con 0x7f0e640d0de0 2024-04-24T06:28:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.139+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33f963ab-a2df-4fda-a7e4-34d0116e6efa", "id": 6} v 0) -- 0x7f0e640b5320 con 0x7f0e640d0de0 2024-04-24T06:28:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.148+0000 7f0e61600640 1 -- 172.21.15.66:0/3175176460 <== mon.1 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33f963ab-a2df-4fda-a7e4-34d0116e6efa", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0e540052b0 con 0x7f0e640d0de0 2024-04-24T06:28:23.151 INFO:teuthology.orchestra.run.smithi066.stdout:6 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 msgr2=0x7f0e641f8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c730 tx=0x7f0e5400cbf0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 shutdown_connections 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e64161c50 0x7f0e640b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e640d5240 0x7f0e6415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 --2- 172.21.15.66:0/3175176460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e640d0de0 0x7f0e641f8640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 >> 172.21.15.66:0/3175176460 conn(0x7f0e6408bf90 msgr2=0x7f0e640b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 shutdown_connections 2024-04-24T06:28:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.150+0000 7f0e6a800640 1 -- 172.21.15.66:0/3175176460 wait complete. 2024-04-24T06:28:23.168 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd new 2c312b36-a2ce-426d-a404-5c717fd2bd5b 7 2024-04-24T06:28:23.201 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:23.199+0000 7f8b45932040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:28:23.257 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:23.254+0000 7f269251c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:28:23.265 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:23.262+0000 7f8b45932040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:28:23.321 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:23.319+0000 7f269251c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:28:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.348+0000 7f0bdf000640 1 Processor -- start 2024-04-24T06:28:23.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.348+0000 7f0bdf000640 1 -- start start 2024-04-24T06:28:23.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.349+0000 7f0bdf000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.349+0000 7f0bdf000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 0x7f0bd80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.349+0000 7f0bdf000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd80b2ca0 con 0x7f0bd815c940 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.349+0000 7f0bdf000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd80cdaa0 con 0x7f0bd806bf10 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.349+0000 7f0bdf000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd80cdd80 con 0x7f0bd80ce860 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bdd200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bddc00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0bd815c940 0x7f0bd80b21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.66:33782/0 (socket says 172.21.15.66:33782) 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bddc00640 1 -- 172.21.15.66:0/1046354285 learned_addr learned my addr 172.21.15.66:0/1046354285 (peer_addr_for_me v1:172.21.15.66:0/0) 2024-04-24T06:28:23.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bde600640 1 --2- 172.21.15.66:0/1046354285 >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 0x7f0bd80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bdd200640 1 -- 172.21.15.66:0/1046354285 >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 msgr2=0x7f0bd80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bd7e00640 1 -- 172.21.15.66:0/1046354285 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119019869 0 0) 0x7f0bd80b2ca0 con 0x7f0bd815c940 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bdd200640 1 --2- 172.21.15.66:0/1046354285 >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 0x7f0bd80b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bdd200640 1 -- 172.21.15.66:0/1046354285 >> v1:172.21.15.66:6789/0 conn(0x7f0bd815c940 legacy=0x7f0bd80b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bdd200640 1 -- 172.21.15.66:0/1046354285 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80ce060 con 0x7f0bd80ce860 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.350+0000 7f0bde600640 1 --2- 172.21.15.66:0/1046354285 >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 0x7f0bd80b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:23.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.351+0000 7f0bdd200640 1 --2- 172.21.15.66:0/1046354285 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd806b9d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40097a0 tx=0x7f0bc402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f71b4c84696110 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bd7e00640 1 -- 172.21.15.66:0/1046354285 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4037070 con 0x7f0bd80ce860 2024-04-24T06:28:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bd7e00640 1 -- 172.21.15.66:0/1046354285 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc4032440 con 0x7f0bd80ce860 2024-04-24T06:28:23.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bd7e00640 1 -- 172.21.15.66:0/1046354285 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4032810 con 0x7f0bd80ce860 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 -- 172.21.15.66:0/1046354285 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 msgr2=0x7f0bd806b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1046354285 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd806b9d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40097a0 tx=0x7f0bc402a180 comp rx=0 tx=0).stop 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 -- 172.21.15.66:0/1046354285 shutdown_connections 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1046354285 >> v2:172.21.15.66:3300/0 conn(0x7f0bd806bf10 0x7f0bd80b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1046354285 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd806b9d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 -- 172.21.15.66:0/1046354285 >> 172.21.15.66:0/1046354285 conn(0x7f0bd80b2f10 msgr2=0x7f0bd80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.352+0000 7f0bdf000640 1 -- 172.21.15.66:0/1046354285 shutdown_connections 2024-04-24T06:28:23.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.353+0000 7f0bdf000640 1 -- 172.21.15.66:0/1046354285 wait complete. 2024-04-24T06:28:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.354+0000 7f0bdf000640 1 Processor -- start 2024-04-24T06:28:23.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.354+0000 7f0bdf000640 1 -- start start 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 0x7f0bd81f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd80ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80b28f0 con 0x7f0bd806bf10 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80c4ba0 con 0x7f0bd815c940 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdf000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd80ae730 con 0x7f0bd80ce860 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bde600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bdd200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd80ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bde600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55318/0 (socket says 172.21.15.66:55318) 2024-04-24T06:28:23.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.355+0000 7f0bde600640 1 -- 172.21.15.66:0/1663063619 learned_addr learned my addr 172.21.15.66:0/1663063619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bddc00640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 0x7f0bd81f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 -- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 msgr2=0x7f0bd80ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd80ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 -- 172.21.15.66:0/1663063619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 msgr2=0x7f0bd81f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 0x7f0bd81f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 -- 172.21.15.66:0/1663063619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80ae9b0 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bddc00640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 0x7f0bd81f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bde600640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000efc0 tx=0x7f0bd000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.356+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0018030 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd000cd90 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd80aec90 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0003bb0 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd80af250 con 0x7f0bd815c940 2024-04-24T06:28:23.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0bd001b070 con 0x7f0bd815c940 2024-04-24T06:28:23.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.357+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0bd0014ae0 con 0x7f0bd815c940 2024-04-24T06:28:23.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.358+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd804fd90 con 0x7f0bd815c940 2024-04-24T06:28:23.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.360+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0bd0010040 con 0x7f0bd815c940 2024-04-24T06:28:23.402 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:23.400+0000 7f8b45932040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:28:23.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.440+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2c312b36-a2ce-426d-a404-5c717fd2bd5b", "id": 7} v 0) -- 0x7f0bd806c610 con 0x7f0bd815c940 2024-04-24T06:28:23.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.449+0000 7f0bd5600640 1 -- 172.21.15.66:0/1663063619 <== mon.1 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c312b36-a2ce-426d-a404-5c717fd2bd5b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0bd0014480 con 0x7f0bd815c940 2024-04-24T06:28:23.451 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.450+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 msgr2=0x7f0bd80c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.450+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000efc0 tx=0x7f0bd000c490 comp rx=0 tx=0).stop 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 shutdown_connections 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bd815c940 0x7f0bd80c6a90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bd80ce860 0x7f0bd80ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 --2- 172.21.15.66:0/1663063619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bd806bf10 0x7f0bd81f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 >> 172.21.15.66:0/1663063619 conn(0x7f0bd80b2f10 msgr2=0x7f0bd806f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 shutdown_connections 2024-04-24T06:28:23.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:28:23.451+0000 7f0bdf000640 1 -- 172.21.15.66:0/1663063619 wait complete. 2024-04-24T06:28:23.466 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:23.464+0000 7f269251c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:28:23.469 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T06:28:23.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T06:28:23.477 INFO:tasks.ceph.osd.0:Started 2024-04-24T06:28:23.477 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T06:28:23.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T06:28:23.485 INFO:tasks.ceph.osd.1:Started 2024-04-24T06:28:23.485 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T06:28:23.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T06:28:23.491 INFO:tasks.ceph.osd.2:Started 2024-04-24T06:28:23.491 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T06:28:23.491 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T06:28:23.495 INFO:tasks.ceph.osd.3:Started 2024-04-24T06:28:23.495 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T06:28:23.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T06:28:23.501 INFO:tasks.ceph.osd.4:Started 2024-04-24T06:28:23.501 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T06:28:23.501 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T06:28:23.506 INFO:tasks.ceph.osd.5:Started 2024-04-24T06:28:23.507 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T06:28:23.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T06:28:23.511 INFO:tasks.ceph.osd.6:Started 2024-04-24T06:28:23.513 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T06:28:23.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T06:28:23.519 INFO:tasks.ceph.osd.7:Started 2024-04-24T06:28:23.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:23.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.729+0000 7f7b17600640 1 Processor -- start 2024-04-24T06:28:23.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.730+0000 7f7b17600640 1 -- start start 2024-04-24T06:28:23.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b17600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7b1015eb60 0x7f7b100d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b17600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7b100b99a0 0x7f7b100b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b17600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b100ba1e0 con 0x7f7b100d2d60 2024-04-24T06:28:23.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b17600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7b100ba7a0 con 0x7f7b1015eb60 2024-04-24T06:28:23.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b17600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7b100ba4c0 con 0x7f7b100b99a0 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b0fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7b100d2d60 0x7f7b100b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35432/0 (socket says 172.21.15.59:35432) 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b14c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7b1015eb60 0x7f7b100d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b0fe00640 1 -- 172.21.15.59:0/3760217258 learned_addr learned my addr 172.21.15.59:0/3760217258 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b15600640 1 --2- 172.21.15.59:0/3760217258 >> v2:172.21.15.66:3300/0 conn(0x7f7b100b99a0 0x7f7b100b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b0f400640 1 -- 172.21.15.59:0/3760217258 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297539837 0 0) 0x7f7b100ba1e0 con 0x7f7b100d2d60 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b0f400640 1 -- 172.21.15.59:0/3760217258 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8003520 con 0x7f7b100d2d60 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b14c00640 1 -- 172.21.15.59:0/3760217258 >> v2:172.21.15.66:3300/0 conn(0x7f7b100b99a0 msgr2=0x7f7b100b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b14c00640 1 --2- 172.21.15.59:0/3760217258 >> v2:172.21.15.66:3300/0 conn(0x7f7b100b99a0 0x7f7b100b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.731+0000 7f7b14c00640 1 -- 172.21.15.59:0/3760217258 >> v1:172.21.15.59:6789/0 conn(0x7f7b100d2d60 legacy=0x7f7b100b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.732+0000 7f7b14c00640 1 -- 172.21.15.59:0/3760217258 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100b8a90 con 0x7f7b1015eb60 2024-04-24T06:28:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b14c00640 1 --2- 172.21.15.59:0/3760217258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1015eb60 0x7f7b100d2820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b7390 tx=0x7f7b0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b99d77613d6f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b0f400640 1 -- 172.21.15.59:0/3760217258 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04037070 con 0x7f7b1015eb60 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b0f400640 1 -- 172.21.15.59:0/3760217258 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04032440 con 0x7f7b1015eb60 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b0f400640 1 -- 172.21.15.59:0/3760217258 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04032810 con 0x7f7b1015eb60 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b17600640 1 -- 172.21.15.59:0/3760217258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1015eb60 msgr2=0x7f7b100d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.733+0000 7f7b17600640 1 --2- 172.21.15.59:0/3760217258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1015eb60 0x7f7b100d2820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b7390 tx=0x7f7b0402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 -- 172.21.15.59:0/3760217258 shutdown_connections 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 --2- 172.21.15.59:0/3760217258 >> v2:172.21.15.66:3300/0 conn(0x7f7b100b99a0 0x7f7b100b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 --2- 172.21.15.59:0/3760217258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1015eb60 0x7f7b100d2820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 -- 172.21.15.59:0/3760217258 >> 172.21.15.59:0/3760217258 conn(0x7f7b10067b50 msgr2=0x7f7b10067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 -- 172.21.15.59:0/3760217258 shutdown_connections 2024-04-24T06:28:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.734+0000 7f7b17600640 1 -- 172.21.15.59:0/3760217258 wait complete. 2024-04-24T06:28:23.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.735+0000 7f7b17600640 1 Processor -- start 2024-04-24T06:28:23.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.735+0000 7f7b17600640 1 -- start start 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1015eb60 0x7f7b1014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1014af20 0x7f7b100ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100b3490 con 0x7f7b100b99a0 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100b3c80 con 0x7f7b1014af20 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.736+0000 7f7b17600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7b100b3610 con 0x7f7b1015eb60 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49842/0 (socket says 172.21.15.59:49842) 2024-04-24T06:28:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 -- 172.21.15.59:0/2621610362 learned_addr learned my addr 172.21.15.59:0/2621610362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b0fe00640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1015eb60 0x7f7b1014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b15600640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1014af20 0x7f7b100ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 -- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1015eb60 msgr2=0x7f7b1014d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1015eb60 0x7f7b1014d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 -- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1014af20 msgr2=0x7f7b100ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1014af20 0x7f7b100ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 -- 172.21.15.59:0/2621610362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100c9dd0 con 0x7f7b100b99a0 2024-04-24T06:28:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.737+0000 7f7b14c00640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04041020 con 0x7f7b100b99a0 2024-04-24T06:28:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0403d700 con 0x7f7b100b99a0 2024-04-24T06:28:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04037040 con 0x7f7b100b99a0 2024-04-24T06:28:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b100ca0b0 con 0x7f7b100b99a0 2024-04-24T06:28:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b100ca670 con 0x7f7b100b99a0 2024-04-24T06:28:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b04044020 con 0x7f7b100b99a0 2024-04-24T06:28:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.738+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7b04046bd0 con 0x7f7b100b99a0 2024-04-24T06:28:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.739+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b100b7420 con 0x7f7b100b99a0 2024-04-24T06:28:23.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.742+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7b100b7420 con 0x7f7b100b99a0 2024-04-24T06:28:23.823 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:23.821+0000 7f8b45932040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:28:23.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.826+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b10068a80 con 0x7f7b100b99a0 2024-04-24T06:28:23.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.827+0000 7f7b0cc00640 1 -- 172.21.15.59:0/2621610362 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7b0404b030 con 0x7f7b100b99a0 2024-04-24T06:28:23.829 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:23.830 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:23.443041+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 msgr2=0x7f7b1014cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b040046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 shutdown_connections 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1014af20 0x7f7b100ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1015eb60 0x7f7b1014d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 --2- 172.21.15.59:0/2621610362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b100b99a0 0x7f7b1014cc60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 >> 172.21.15.59:0/2621610362 conn(0x7f7b10067b50 msgr2=0x7f7b100d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 shutdown_connections 2024-04-24T06:28:23.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:23.829+0000 7f7b17600640 1 -- 172.21.15.59:0/2621610362 wait complete. 2024-04-24T06:28:23.846 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T06:28:23.847 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T06:28:23.859 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:23.857+0000 7f269251c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:28:23.883 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:23.881+0000 7f8b45932040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:28:23.919 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:23.917+0000 7f269251c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:28:24.073 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:24.071+0000 7f8b45932040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:28:24.113 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:24.110+0000 7f269251c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:28:24.176 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-24T06:28:24.174+0000 7f5a47cb5640 -1 Falling back to public interface 2024-04-24T06:28:24.183 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-04-24T06:28:24.181+0000 7f5a2a62e640 -1 Falling back to public interface 2024-04-24T06:28:24.202 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-24T06:28:24.200+0000 7fc41b7b6640 -1 Falling back to public interface 2024-04-24T06:28:24.204 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-24T06:28:24.202+0000 7f716cc86640 -1 Falling back to public interface 2024-04-24T06:28:24.204 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-24T06:28:24.202+0000 7f53d8e14640 -1 Falling back to public interface 2024-04-24T06:28:24.223 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-24T06:28:24.221+0000 7f09e34f0640 -1 Falling back to public interface 2024-04-24T06:28:24.227 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-04-24T06:28:24.225+0000 7f16fe20c640 -1 Falling back to public interface 2024-04-24T06:28:24.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-04-24T06:28:24.237+0000 7fb2e6dc0640 -1 Falling back to public interface 2024-04-24T06:28:24.248 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:24.247+0000 7f8b45932040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:28:24.290 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:24.288+0000 7f269251c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:28:24.309 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:24.307+0000 7f8b45932040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:28:24.352 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:24.350+0000 7f269251c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:28:24.394 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:24.391+0000 7f8b45932040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:28:24.437 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:24.435+0000 7f269251c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:28:24.448 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:24.446+0000 7f8b45932040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:28:24.494 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:24.492+0000 7f269251c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:28:24.651 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:24.811 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.809+0000 7fd937600640 1 Processor -- start 2024-04-24T06:28:24.811 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.809+0000 7fd937600640 1 -- start start 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.810+0000 7fd937600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9300d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.810+0000 7fd937600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9300d3d20 0x7fd9300cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.810+0000 7fd937600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9300cff60 con 0x7fd9300d14c0 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.810+0000 7fd937600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd930162d80 con 0x7fd9300d3d20 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.810+0000 7fd937600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9300d0240 con 0x7fd9300d18a0 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9300d3d20 0x7fd9300cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9300d3d20 0x7fd9300cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49846/0 (socket says 172.21.15.59:49846) 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 -- 172.21.15.59:0/2090688194 learned_addr learned my addr 172.21.15.59:0/2090688194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd934c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd9300d14c0 0x7fd9300b7f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35438/0 (socket says 172.21.15.59:35438) 2024-04-24T06:28:24.813 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd92fe00640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9300d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:24.814 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 -- 172.21.15.59:0/2090688194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 msgr2=0x7fd9300d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:24.814 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9300d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.814 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 -- 172.21.15.59:0/2090688194 >> v1:172.21.15.59:6789/0 conn(0x7fd9300d14c0 legacy=0x7fd9300b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:24.814 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.811+0000 7fd935600640 1 -- 172.21.15.59:0/2090688194 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9300d2fe0 con 0x7fd9300d3d20 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.812+0000 7fd935600640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d3d20 0x7fd9300cf3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd928009e50 tx=0x7fd92802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f385f9e9f5ff43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.812+0000 7fd92f400640 1 -- 172.21.15.59:0/2090688194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928036070 con 0x7fd9300d3d20 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.812+0000 7fd92f400640 1 -- 172.21.15.59:0/2090688194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd928031600 con 0x7fd9300d3d20 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.812+0000 7fd92f400640 1 -- 172.21.15.59:0/2090688194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280319d0 con 0x7fd9300d3d20 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 -- 172.21.15.59:0/2090688194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d3d20 msgr2=0x7fd9300cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d3d20 0x7fd9300cf3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd928009e50 tx=0x7fd92802a730 comp rx=0 tx=0).stop 2024-04-24T06:28:24.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 -- 172.21.15.59:0/2090688194 shutdown_connections 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d3d20 0x7fd9300cf3c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 --2- 172.21.15.59:0/2090688194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9300d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 -- 172.21.15.59:0/2090688194 >> 172.21.15.59:0/2090688194 conn(0x7fd93006cc80 msgr2=0x7fd93006d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 -- 172.21.15.59:0/2090688194 shutdown_connections 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.813+0000 7fd937600640 1 -- 172.21.15.59:0/2090688194 wait complete. 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.815+0000 7fd937600640 1 Processor -- start 2024-04-24T06:28:24.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.815+0000 7fd937600640 1 -- start start 2024-04-24T06:28:24.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:24.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9301f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:24.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9300d3d20 0x7fd930160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd9300b87c0 con 0x7fd9300d14c0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd9300b8640 con 0x7fd9300d3d20 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd937600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9300cf900 con 0x7fd9300d18a0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49850/0 (socket says 172.21.15.59:49850) 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 -- 172.21.15.59:0/4135004443 learned_addr learned my addr 172.21.15.59:0/4135004443 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd935600640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9300d3d20 0x7fd930160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd92fe00640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9301f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 -- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 msgr2=0x7fd9301f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9301f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 -- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9300d3d20 msgr2=0x7fd930160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9300d3d20 0x7fd930160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.816+0000 7fd934c00640 1 -- 172.21.15.59:0/4135004443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9300ab2d0 con 0x7fd9300d14c0 2024-04-24T06:28:24.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd934c00640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c400 tx=0x7fd92400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:24.820 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240074e0 con 0x7fd9300d14c0 2024-04-24T06:28:24.820 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd924007680 con 0x7fd9300d14c0 2024-04-24T06:28:24.820 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924004a00 con 0x7fd9300d14c0 2024-04-24T06:28:24.821 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9300ab570 con 0x7fd9300d14c0 2024-04-24T06:28:24.821 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd930160550 con 0x7fd9300d14c0 2024-04-24T06:28:24.821 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd92401a020 con 0x7fd9300d14c0 2024-04-24T06:28:24.821 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.817+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd924013ad0 con 0x7fd9300d14c0 2024-04-24T06:28:24.821 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.818+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd930074d00 con 0x7fd9300d14c0 2024-04-24T06:28:24.822 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.821+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd930074d00 con 0x7fd9300d14c0 2024-04-24T06:28:24.890 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.888+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd93004fd90 con 0x7fd9300d14c0 2024-04-24T06:28:24.892 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.890+0000 7fd92cc00640 1 -- 172.21.15.59:0/4135004443 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd924021070 con 0x7fd9300d14c0 2024-04-24T06:28:24.892 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-24T06:28:24.892 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":10,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:23.443041+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:24.893 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 msgr2=0x7fd93015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:24.893 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c400 tx=0x7fd92400c8c0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.893 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 shutdown_connections 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9300d3d20 0x7fd930160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9300d18a0 0x7fd9301f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 --2- 172.21.15.59:0/4135004443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9300d14c0 0x7fd93015d3a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 >> 172.21.15.59:0/4135004443 conn(0x7fd93006cc80 msgr2=0x7fd9300c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 shutdown_connections 2024-04-24T06:28:24.894 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:24.891+0000 7fd937600640 1 -- 172.21.15.59:0/4135004443 wait complete. 2024-04-24T06:28:24.906 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T06:28:25.019 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:25.017+0000 7f8b45932040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:28:25.067 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:25.065+0000 7f269251c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:28:25.157 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:25.155+0000 7f8b45932040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:28:25.213 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-04-24T06:28:25.211+0000 7f269251c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:28:26.201 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:26.199+0000 7f8ae6000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T06:28:26.201 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-24T06:28:26.199+0000 7f8ae6000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T06:28:27.348 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-24T06:28:27.346+0000 7f53d8e14640 -1 osd.2 0 log_to_monitors true 2024-04-24T06:28:27.810 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-24T06:28:27.808+0000 7f716cc86640 -1 osd.0 0 log_to_monitors true 2024-04-24T06:28:27.857 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-04-24T06:28:27.855+0000 7fb2e6dc0640 -1 osd.7 0 log_to_monitors true 2024-04-24T06:28:27.859 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-04-24T06:28:27.857+0000 7f16fe20c640 -1 osd.5 0 log_to_monitors true 2024-04-24T06:28:27.860 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-24T06:28:27.858+0000 7fc41b7b6640 -1 osd.6 0 log_to_monitors true 2024-04-24T06:28:27.873 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-24T06:28:27.871+0000 7f09e34f0640 -1 osd.3 0 log_to_monitors true 2024-04-24T06:28:27.975 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-24T06:28:27.974+0000 7f5a47cb5640 -1 osd.1 0 log_to_monitors true 2024-04-24T06:28:28.022 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-04-24T06:28:28.020+0000 7f5a2a62e640 -1 osd.4 0 log_to_monitors true 2024-04-24T06:28:31.557 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-24T06:28:31.556+0000 7f7167c00640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T06:28:31.560 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-24T06:28:31.560+0000 7f5a43600640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T06:28:31.564 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-24T06:28:31.563+0000 7f715dc00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.568 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-24T06:28:31.568+0000 7f5a38c00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.575 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-24T06:28:31.574+0000 7f09dee00640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T06:28:31.576 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-04-24T06:28:31.576+0000 7f5a24a00640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T06:28:31.583 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-04-24T06:28:31.582+0000 7f5a1aa00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.583 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-24T06:28:31.583+0000 7f53d3600640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T06:28:31.584 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-24T06:28:31.584+0000 7f09d4400640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.590 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-24T06:28:31.589+0000 7f53c9600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.600 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-24T06:28:31.600+0000 7fc417000640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T06:28:31.606 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-04-24T06:28:31.605+0000 7fc40c600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-04-24T06:28:31.607+0000 7f16f8a00640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T06:28:31.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-04-24T06:28:31.612+0000 7f16eea00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.616 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-04-24T06:28:31.616+0000 7fb2e1c00640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T06:28:31.626 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-04-24T06:28:31.625+0000 7fb2d7c00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:28:31.712 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:31.896 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.895+0000 7fc0f3800640 1 Processor -- start 2024-04-24T06:28:31.896 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.895+0000 7fc0f3800640 1 -- start start 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.897+0000 7fc0f3800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0ec0b5ff0 0x7fc0ec15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.897+0000 7fc0f3800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec071f30 con 0x7fc0ec145530 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0f3800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec072210 con 0x7fc0ec145bd0 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0f3800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ec145910 con 0x7fc0ec0b5ff0 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0f0e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc0ec145530 0x7fc0ec072fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54426/0 (socket says 172.21.15.59:54426) 2024-04-24T06:28:31.899 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1046090410 learned_addr learned my addr 172.21.15.59:0/1046090410 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:31.901 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186995144 0 0) 0x7fc0ec071f30 con 0x7fc0ec145530 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0d0003520 con 0x7fc0ec145530 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.898+0000 7fc0f1800640 1 --2- 172.21.15.59:0/1046090410 >> v2:172.21.15.66:3300/0 conn(0x7fc0ec0b5ff0 0x7fc0ec15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259506688 0 0) 0x7fc0d0003520 con 0x7fc0ec145530 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 >> v2:172.21.15.66:3300/0 conn(0x7fc0ec0b5ff0 msgr2=0x7fc0ec15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 --2- 172.21.15.59:0/1046090410 >> v2:172.21.15.66:3300/0 conn(0x7fc0ec0b5ff0 0x7fc0ec15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 >> v1:172.21.15.66:6789/0 conn(0x7fc0ec145bd0 legacy=0x7fc0ec0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0724f0 con 0x7fc0ec145530 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc0dc0019f0 con 0x7fc0ec145530 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 >> v1:172.21.15.59:6789/0 conn(0x7fc0ec145530 legacy=0x7fc0ec072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0d0004800 0x7fc0d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.902 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0d00052e0 0x7fc0d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec145910 con 0x7fc0d0003a00 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d000c200 con 0x7fc0d00052e0 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d000c400 con 0x7fc0d0004800 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0d0004800 msgr2=0x7fc0d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0d0004800 0x7fc0d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0d00052e0 msgr2=0x7fc0d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.899+0000 7fc0f1800640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0d00052e0 0x7fc0d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.900+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0d00052e0 0x7fc0d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.900+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1046090410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d0003520 con 0x7fc0d0003a00 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.900+0000 7fc0f1800640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0d00052e0 0x7fc0d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.900+0000 7fc0f1800640 1 -- 172.21.15.59:0/1046090410 reap_dead start 2024-04-24T06:28:31.903 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a770 tx=0x7fc0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd349c9be114f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:31.904 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00a390 con 0x7fc0d0003a00 2024-04-24T06:28:31.904 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc02f070 con 0x7fc0d0003a00 2024-04-24T06:28:31.904 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0eb400640 1 -- 172.21.15.59:0/1046090410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc02bc20 con 0x7fc0d0003a00 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f3800640 1 -- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 msgr2=0x7fc0d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a770 tx=0x7fc0d0005830 comp rx=0 tx=0).stop 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f3800640 1 -- 172.21.15.59:0/1046090410 shutdown_connections 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1046090410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0d0003e20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.901+0000 7fc0f3800640 1 -- 172.21.15.59:0/1046090410 >> 172.21.15.59:0/1046090410 conn(0x7fc0ec06b9e0 msgr2=0x7fc0ec06bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.902+0000 7fc0f3800640 1 -- 172.21.15.59:0/1046090410 shutdown_connections 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.902+0000 7fc0f3800640 1 -- 172.21.15.59:0/1046090410 wait complete. 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.903+0000 7fc0f3800640 1 Processor -- start 2024-04-24T06:28:31.905 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.903+0000 7fc0f3800640 1 -- start start 2024-04-24T06:28:31.907 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.907 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0ec15ebe0 0x7fc0ec15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.907 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0ec0a94d0 0x7fc0ec0a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.907 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec06f0a0 con 0x7fc0d0003a00 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec072ab0 con 0x7fc0ec0a94d0 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.904+0000 7fc0f3800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec06f220 con 0x7fc0ec15ebe0 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43348/0 (socket says 172.21.15.59:43348) 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1679275886 learned_addr learned my addr 172.21.15.59:0/1679275886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0ebe00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0ec15ebe0 0x7fc0ec15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0ec15ebe0 msgr2=0x7fc0ec15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0ec15ebe0 0x7fc0ec15f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0ec0a94d0 msgr2=0x7fc0ec0a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:31.908 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0ec0a94d0 0x7fc0ec0a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 -- 172.21.15.59:0/1679275886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec15db40 con 0x7fc0d0003a00 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.905+0000 7fc0f0e00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a770 tx=0x7fc0ec072100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.906+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc001250 con 0x7fc0d0003a00 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.906+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc034470 con 0x7fc0d0003a00 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.906+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc034840 con 0x7fc0d0003a00 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.906+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec15de20 con 0x7fc0d0003a00 2024-04-24T06:28:31.909 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.906+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec15e3e0 con 0x7fc0d0003a00 2024-04-24T06:28:31.910 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.907+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec06c9f0 con 0x7fc0d0003a00 2024-04-24T06:28:31.910 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.908+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fc0dc0349e0 con 0x7fc0d0003a00 2024-04-24T06:28:31.910 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.909+0000 7fc0e8c00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 0x7fc0cc06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:31.910 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.909+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fc0dc0ab050 con 0x7fc0d0003a00 2024-04-24T06:28:31.910 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.909+0000 7fc0ebe00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 0x7fc0cc06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:31.911 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.910+0000 7fc0ebe00640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 0x7fc0cc06ef40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0b1600 tx=0x7fc0d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:31.915 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:31.915+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0dc07a360 con 0x7fc0d0003a00 2024-04-24T06:28:32.110 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.109+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0ec04fd90 con 0x7fc0d0003a00 2024-04-24T06:28:32.111 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.110+0000 7fc0e8c00640 1 -- 172.21.15.59:0/1679275886 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fc0dc079c60 con 0x7fc0d0003a00 2024-04-24T06:28:32.112 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-24T06:28:32.112 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":12,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:29.200987+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 msgr2=0x7fc0cc06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 0x7fc0cc06ef40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0b1600 tx=0x7fc0d8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 msgr2=0x7fc0ec13eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00a770 tx=0x7fc0ec072100 comp rx=0 tx=0).stop 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 shutdown_connections 2024-04-24T06:28:32.114 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06ca80 0x7fc0cc06ef40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0ec0a94d0 0x7fc0ec0a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0ec15ebe0 0x7fc0ec15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 --2- 172.21.15.59:0/1679275886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0d0003a00 0x7fc0ec13eb50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.113+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 >> 172.21.15.59:0/1679275886 conn(0x7fc0ec06b9e0 msgr2=0x7fc0ec0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.114+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 shutdown_connections 2024-04-24T06:28:32.115 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:32.114+0000 7fc0f3800640 1 -- 172.21.15.59:0/1679275886 wait complete. 2024-04-24T06:28:32.128 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T06:28:34.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:28:34.865 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:28:38.937 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:39.127 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.126+0000 7f5ccb200640 1 Processor -- start 2024-04-24T06:28:39.127 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.126+0000 7f5ccb200640 1 -- start start 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5ccb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ccc0b2ca0 0x7f5ccc0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5ccb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc146510 con 0x7f5ccc146130 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5ccb200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc0afe00 con 0x7f5ccc06d670 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5ccb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5ccc0b1f90 con 0x7f5ccc0b2ca0 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5cc9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ccc146130 0x7f5ccc0b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49256/0 (socket says 172.21.15.59:49256) 2024-04-24T06:28:39.129 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.128+0000 7f5cc9e00640 1 -- 172.21.15.59:0/286431640 learned_addr learned my addr 172.21.15.59:0/286431640 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:39.131 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152419945 0 0) 0x7f5ccc146510 con 0x7f5ccc146130 2024-04-24T06:28:39.131 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb0003520 con 0x7f5ccc146130 2024-04-24T06:28:39.131 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cca800640 1 --2- 172.21.15.59:0/286431640 >> v2:172.21.15.66:3300/0 conn(0x7f5ccc0b2ca0 0x7f5ccc0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.131 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1946656406 0 0) 0x7f5cb0003520 con 0x7f5ccc146130 2024-04-24T06:28:39.131 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 >> v2:172.21.15.66:3300/0 conn(0x7f5ccc0b2ca0 msgr2=0x7f5ccc0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 --2- 172.21.15.59:0/286431640 >> v2:172.21.15.66:3300/0 conn(0x7f5ccc0b2ca0 0x7f5ccc0b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 >> v1:172.21.15.66:6789/0 conn(0x7f5ccc06d670 legacy=0x7f5ccc15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc0b00e0 con 0x7f5ccc146130 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5cbc001670 con 0x7f5ccc146130 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 >> v1:172.21.15.59:6789/0 conn(0x7f5ccc146130 legacy=0x7f5ccc0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5cb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb0004800 0x7f5cb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5cb00052e0 0x7f5cb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0b1f90 con 0x7f5cb00039b0 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb000c1b0 con 0x7f5cb0004800 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.129+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb000c3b0 con 0x7f5cb00052e0 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5cb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9400640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb0004800 0x7f5cb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.132 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 -- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5cb00052e0 msgr2=0x7f5cb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5cb00052e0 0x7f5cb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 -- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb0004800 msgr2=0x7f5cb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb0004800 0x7f5cb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9e00640 1 -- 172.21.15.59:0/286431640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0003520 con 0x7f5cb00039b0 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cca800640 1 -- 172.21.15.59:0/286431640 reap_dead start 2024-04-24T06:28:39.133 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.130+0000 7f5cc9400640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cb0004800 0x7f5cb0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.131+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5cb0003dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001eb0 tx=0x7f5cb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fb9982d04b0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.132+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc007d30 con 0x7f5cb00039b0 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.132+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc005af0 con 0x7f5cb00039b0 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.132+0000 7f5cc8a00640 1 -- 172.21.15.59:0/286431640 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc007070 con 0x7f5cb00039b0 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.132+0000 7f5ccb200640 1 -- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 msgr2=0x7f5cb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.134 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.132+0000 7f5ccb200640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5cb0003dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001eb0 tx=0x7f5cb0005830 comp rx=0 tx=0).stop 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.133+0000 7f5ccb200640 1 -- 172.21.15.59:0/286431640 shutdown_connections 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.133+0000 7f5ccb200640 1 --2- 172.21.15.59:0/286431640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5cb0003dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.133+0000 7f5ccb200640 1 -- 172.21.15.59:0/286431640 >> 172.21.15.59:0/286431640 conn(0x7f5ccc0b6c10 msgr2=0x7f5ccc09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.133+0000 7f5ccb200640 1 -- 172.21.15.59:0/286431640 shutdown_connections 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.133+0000 7f5ccb200640 1 -- 172.21.15.59:0/286431640 wait complete. 2024-04-24T06:28:39.135 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.135+0000 7f5ccb200640 1 Processor -- start 2024-04-24T06:28:39.136 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.135+0000 7f5ccb200640 1 -- start start 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ccc09d710 0x7f5ccc09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ccc0b0bc0 0x7f5ccc0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0b0980 con 0x7f5cb00039b0 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0b06a0 con 0x7f5ccc0b0bc0 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5ccb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0afa20 con 0x7f5ccc09d710 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5cc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.138 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5cc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58360/0 (socket says 172.21.15.59:58360) 2024-04-24T06:28:39.139 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.136+0000 7f5cc9e00640 1 -- 172.21.15.59:0/3975046264 learned_addr learned my addr 172.21.15.59:0/3975046264 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 -- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ccc09d710 msgr2=0x7f5ccc09db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ccc09d710 0x7f5ccc09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 -- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ccc0b0bc0 msgr2=0x7f5ccc0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ccc0b0bc0 0x7f5ccc0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 -- 172.21.15.59:0/3975046264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc09e1c0 con 0x7f5cb00039b0 2024-04-24T06:28:39.140 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc9e00640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c300 tx=0x7f5cbc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.141 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc037040 con 0x7f5cb00039b0 2024-04-24T06:28:39.141 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc041070 con 0x7f5cb00039b0 2024-04-24T06:28:39.141 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0047e0 con 0x7f5cb00039b0 2024-04-24T06:28:39.141 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc09e4a0 con 0x7f5cb00039b0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.137+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc09ea60 con 0x7f5cb00039b0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.139+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc0b1580 con 0x7f5cb00039b0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.140+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cbc004980 con 0x7f5cb00039b0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.140+0000 7f5cc2000640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 0x7f5ca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.141+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5cbc0ab5a0 con 0x7f5cb00039b0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.141+0000 7f5cc9400640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 0x7f5ca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.142 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.141+0000 7f5cc9400640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 0x7f5ca806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc09f0d0 tx=0x7f5cb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.146 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.146+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5cbc005040 con 0x7f5cb00039b0 2024-04-24T06:28:39.351 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.350+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ccc04fd90 con 0x7f5cb00039b0 2024-04-24T06:28:39.353 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.352+0000 7f5cc2000640 1 -- 172.21.15.59:0/3975046264 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f5cbc079c80 con 0x7f5cb00039b0 2024-04-24T06:28:39.353 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-24T06:28:39.353 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":16,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:35.616935+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:39.356 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 msgr2=0x7f5ca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.356 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 0x7f5ca806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc09f0d0 tx=0x7f5cb4009040 comp rx=0 tx=0).stop 2024-04-24T06:28:39.356 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 msgr2=0x7f5ccc13f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.356 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c300 tx=0x7f5cbc001a40 comp rx=0 tx=0).stop 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 shutdown_connections 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ca806c8f0 0x7f5ca806edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ccc0b0bc0 0x7f5ccc0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ccc09d710 0x7f5ccc09db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 --2- 172.21.15.59:0/3975046264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cb00039b0 0x7f5ccc13f1c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 >> 172.21.15.59:0/3975046264 conn(0x7f5ccc0b6c10 msgr2=0x7f5ccc143c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 shutdown_connections 2024-04-24T06:28:39.357 INFO:teuthology.misc.health.smithi059.stderr:2024-04-24T06:28:39.355+0000 7f5ccb200640 1 -- 172.21.15.59:0/3975046264 wait complete. 2024-04-24T06:28:39.371 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T06:28:39.371 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T06:28:39.372 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:28:39.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:28:39.376 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:28:39.376 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:28:39.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:28:39.382 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:28:39.382 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:28:39.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:28:39.390 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:28:39.390 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:28:39.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:28:39.397 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:28:39.398 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T06:28:39.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:39.525 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:28:39.526 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:28:39.529 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:28:39.528+0000 7f4921e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-04-24T06:28:39.554 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:28:39.563 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:28:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.581+0000 7fb6dbe00640 1 Processor -- start 2024-04-24T06:28:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.581+0000 7fb6dbe00640 1 -- start start 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.582+0000 7fb6dbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6d415b960 0x7fb6d40cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.582+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.582+0000 7fb6dbe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d40d0480 con 0x7fb6d40cffe0 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.582+0000 7fb6dbe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d415bd40 con 0x7fb6d415b960 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.582+0000 7fb6dbe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d40d1d60 con 0x7fb6d40ce490 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6d415b960 0x7fb6d40cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:60186/0 (socket says 172.21.15.59:60186) 2024-04-24T06:28:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9e00640 1 -- 172.21.15.59:0/1995098125 learned_addr learned my addr 172.21.15.59:0/1995098125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:39.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9400640 1 -- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 msgr2=0x7fb6d40ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9400640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9400640 1 -- 172.21.15.59:0/1995098125 >> v1:172.21.15.66:6789/0 conn(0x7fb6d40cffe0 legacy=0x7fb6d40cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9400640 1 -- 172.21.15.59:0/1995098125 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d40d2040 con 0x7fb6d415b960 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d3e00640 1 -- 172.21.15.59:0/1995098125 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196278559 0 0) 0x7fb6d40d0480 con 0x7fb6d40cffe0 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.583+0000 7fb6d9e00640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.584+0000 7fb6d9400640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415b960 0x7fb6d40cf080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0bc7009840dcf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6d3e00640 1 -- 172.21.15.59:0/1995098125 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8007740 con 0x7fb6d415b960 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6d3e00640 1 -- 172.21.15.59:0/1995098125 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c80078e0 con 0x7fb6d415b960 2024-04-24T06:28:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6d3e00640 1 -- 172.21.15.59:0/1995098125 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8007cb0 con 0x7fb6d415b960 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415b960 msgr2=0x7fb6d40cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415b960 0x7fb6d40cf080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802a660 comp rx=0 tx=0).stop 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1995098125 shutdown_connections 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d40ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/1995098125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d415b960 0x7fb6d40cf080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1995098125 >> 172.21.15.59:0/1995098125 conn(0x7fb6d409aa00 msgr2=0x7fb6d40d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:39.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1995098125 shutdown_connections 2024-04-24T06:28:39.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.585+0000 7fb6dbe00640 1 -- 172.21.15.59:0/1995098125 wait complete. 2024-04-24T06:28:39.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.587+0000 7fb6dbe00640 1 Processor -- start 2024-04-24T06:28:39.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.587+0000 7fb6dbe00640 1 -- start start 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d41ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40cffe0 0x7fb6d415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40cd1f0 con 0x7fb6d415b960 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40ae300 con 0x7fb6d40cffe0 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6dbe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d40cd370 con 0x7fb6d40ce490 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58392/0 (socket says 172.21.15.59:58392) 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.588+0000 7fb6d9e00640 1 -- 172.21.15.59:0/2614028438 learned_addr learned my addr 172.21.15.59:0/2614028438 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9400640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d41ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d8a00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40cffe0 0x7fb6d415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 -- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 msgr2=0x7fb6d41ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d41ee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 -- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40cffe0 msgr2=0x7fb6d415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40cffe0 0x7fb6d415c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 -- 172.21.15.59:0/2614028438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41f4c40 con 0x7fb6d415b960 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d9e00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006e00 tx=0x7fb6cc00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.589+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc004030 con 0x7fb6d415b960 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.590+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6cc0041d0 con 0x7fb6d415b960 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.590+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00c9a0 con 0x7fb6d415b960 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.590+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41f4ec0 con 0x7fb6d415b960 2024-04-24T06:28:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.590+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41f5480 con 0x7fb6d415b960 2024-04-24T06:28:39.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.591+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4069190 con 0x7fb6d415b960 2024-04-24T06:28:39.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.592+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6cc00cb40 con 0x7fb6d415b960 2024-04-24T06:28:39.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.594+0000 7fb6d1600640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 0x7fb6b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:39.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.594+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb6cc08b730 con 0x7fb6d415b960 2024-04-24T06:28:39.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.594+0000 7fb6d9400640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 0x7fb6b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:39.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.595+0000 7fb6d9400640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 0x7fb6b406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:39.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.599+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6d41f4c40 con 0x7fb6d415b960 2024-04-24T06:28:39.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.806+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6d406f120 con 0x7fb6d415b960 2024-04-24T06:28:39.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.807+0000 7fb6d1600640 1 -- 172.21.15.59:0/2614028438 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7fb6cc017020 con 0x7fb6d415b960 2024-04-24T06:28:39.809 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:39.809 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:35.616935+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 msgr2=0x7fb6b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 0x7fb6b406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802f000 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 msgr2=0x7fb6d41f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006e00 tx=0x7fb6cc00ee10 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 shutdown_connections 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb6b406c8f0 0x7fb6b406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6d415b960 0x7fb6d41f4700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6d40cffe0 0x7fb6d415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 --2- 172.21.15.59:0/2614028438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6d40ce490 0x7fb6d41ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 >> 172.21.15.59:0/2614028438 conn(0x7fb6d409aa00 msgr2=0x7fb6d40c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.810+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 shutdown_connections 2024-04-24T06:28:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:39.811+0000 7fb6dbe00640 1 -- 172.21.15.59:0/2614028438 wait complete. 2024-04-24T06:28:39.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:28:39.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:28:40.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.030+0000 7f6739200640 1 Processor -- start 2024-04-24T06:28:40.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.031+0000 7f6739200640 1 -- start start 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6739200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 0x7f67340d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6739200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67340b3600 0x7f67340d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6739200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67340b5da0 con 0x7f67340b3f10 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6739200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f673415f100 con 0x7f67340b3600 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6739200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f67340d0a20 con 0x7f673415ed20 2024-04-24T06:28:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.032+0000 7f6732a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67340b3600 0x7f67340d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67340b3600 0x7f67340d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58394/0 (socket says 172.21.15.59:58394) 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 -- 172.21.15.59:0/3406948114 learned_addr learned my addr 172.21.15.59:0/3406948114 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6733400640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 0x7f67340d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 -- 172.21.15.59:0/3406948114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 msgr2=0x7f67340d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 0x7f67340d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 -- 172.21.15.59:0/3406948114 >> v1:172.21.15.66:6789/0 conn(0x7f67340b3f10 legacy=0x7f67340b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6732a00640 1 -- 172.21.15.59:0/3406948114 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340d2ed0 con 0x7f67340b3600 2024-04-24T06:28:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.033+0000 7f6733400640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 0x7f67340d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.034+0000 7f6732a00640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67340b3600 0x7f67340d13e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f671c009870 tx=0x7f671c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc77dade4fd1fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.034+0000 7f6732000640 1 -- 172.21.15.59:0/3406948114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c037070 con 0x7f67340b3600 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.034+0000 7f6732000640 1 -- 172.21.15.59:0/3406948114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c02ae10 con 0x7f67340b3600 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.034+0000 7f6732000640 1 -- 172.21.15.59:0/3406948114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c032660 con 0x7f67340b3600 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 -- 172.21.15.59:0/3406948114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67340b3600 msgr2=0x7f67340d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67340b3600 0x7f67340d13e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f671c009870 tx=0x7f671c02a100 comp rx=0 tx=0).stop 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 -- 172.21.15.59:0/3406948114 shutdown_connections 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67340b3600 0x7f67340d13e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 --2- 172.21.15.59:0/3406948114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f673415ed20 0x7f67340d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 -- 172.21.15.59:0/3406948114 >> 172.21.15.59:0/3406948114 conn(0x7f673408bf90 msgr2=0x7f67340d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 -- 172.21.15.59:0/3406948114 shutdown_connections 2024-04-24T06:28:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.035+0000 7f6739200640 1 -- 172.21.15.59:0/3406948114 wait complete. 2024-04-24T06:28:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.037+0000 7f6739200640 1 Processor -- start 2024-04-24T06:28:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.037+0000 7f6739200640 1 -- start start 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 0x7f673415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67340b3f10 0x7f673415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67340d0290 con 0x7f673415ed20 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67340d3490 con 0x7f67340b3f10 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6739200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f67340d0410 con 0x7f67340b3600 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.038+0000 7f6733e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58408/0 (socket says 172.21.15.59:58408) 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 -- 172.21.15.59:0/625015756 learned_addr learned my addr 172.21.15.59:0/625015756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733400640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 0x7f673415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6732a00640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67340b3f10 0x7f673415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 -- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 msgr2=0x7f673415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 0x7f673415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 -- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67340b3f10 msgr2=0x7f673415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67340b3f10 0x7f673415d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 -- 172.21.15.59:0/625015756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340ce500 con 0x7f673415ed20 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733400640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 0x7f673415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.039+0000 7f6733e00640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c440 tx=0x7f672c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.040+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c011790 con 0x7f673415ed20 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.040+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c017070 con 0x7f673415ed20 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.040+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c011b70 con 0x7f673415ed20 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.040+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673409e440 con 0x7f673415ed20 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.040+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673409ea00 con 0x7f673415ed20 2024-04-24T06:28:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.041+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673404fd90 con 0x7f673415ed20 2024-04-24T06:28:40.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.042+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f672c011d10 con 0x7f673415ed20 2024-04-24T06:28:40.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.043+0000 7f672b400640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 0x7f671006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.043+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f672c08b5f0 con 0x7f673415ed20 2024-04-24T06:28:40.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.043+0000 7f6733400640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 0x7f671006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.044+0000 7f6733400640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 0x7f671006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6724007f40 tx=0x7f6724007490 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.048+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f672c059d60 con 0x7f673415ed20 2024-04-24T06:28:40.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.250+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67340b4ea0 con 0x7f673415ed20 2024-04-24T06:28:40.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.251+0000 7f672b400640 1 -- 172.21.15.59:0/625015756 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f672c059660 con 0x7f673415ed20 2024-04-24T06:28:40.252 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 msgr2=0x7f671006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 0x7f671006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6724007f40 tx=0x7f6724007490 comp rx=0 tx=0).stop 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 msgr2=0x7f673415ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c440 tx=0x7f672c00c900 comp rx=0 tx=0).stop 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 shutdown_connections 2024-04-24T06:28:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f671006c8f0 0x7f671006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673415ed20 0x7f673415ddb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67340b3f10 0x7f673415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 --2- 172.21.15.59:0/625015756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67340b3600 0x7f673415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.253+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 >> 172.21.15.59:0/625015756 conn(0x7f673408bf90 msgr2=0x7f673406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.254+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 shutdown_connections 2024-04-24T06:28:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.254+0000 7f6739200640 1 -- 172.21.15.59:0/625015756 wait complete. 2024-04-24T06:28:40.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:28:40.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.484+0000 7f27d6000640 1 Processor -- start 2024-04-24T06:28:40.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.484+0000 7f27d6000640 1 -- start start 2024-04-24T06:28:40.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.485+0000 7f27d6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d009b410 0x7f27d00af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.485+0000 7f27d6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27d00d09e0 0x7f27d00d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.485+0000 7f27d6000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d00b35f0 con 0x7f27d015c9d0 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.485+0000 7f27d6000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f27d00cd2d0 con 0x7f27d00d09e0 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.485+0000 7f27d6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27d00d13b0 con 0x7f27d009b410 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27d5600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27d00d09e0 0x7f27d00d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27d5600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27d00d09e0 0x7f27d00d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58416/0 (socket says 172.21.15.59:58416) 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27d5600640 1 -- 172.21.15.59:0/3892159266 learned_addr learned my addr 172.21.15.59:0/3892159266 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27cfe00640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d009b410 0x7f27d00af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27d5600640 1 -- 172.21.15.59:0/3892159266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d009b410 msgr2=0x7f27d00af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.486+0000 7f27d5600640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d009b410 0x7f27d00af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.487+0000 7f27d5600640 1 -- 172.21.15.59:0/3892159266 >> v1:172.21.15.66:6789/0 conn(0x7f27d015c9d0 legacy=0x7f27d00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.487+0000 7f27d5600640 1 -- 172.21.15.59:0/3892159266 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d00b3890 con 0x7f27d00d09e0 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27d5600640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d00d0e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009870 tx=0x7f27c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114def2f61ecc5a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27cf400640 1 -- 172.21.15.59:0/3892159266 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8037070 con 0x7f27d00d09e0 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27cf400640 1 -- 172.21.15.59:0/3892159266 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c8032440 con 0x7f27d00d09e0 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27cf400640 1 -- 172.21.15.59:0/3892159266 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8032810 con 0x7f27d00d09e0 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27d6000640 1 -- 172.21.15.59:0/3892159266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 msgr2=0x7f27d00d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.488+0000 7f27d6000640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d00d0e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009870 tx=0x7f27c802a270 comp rx=0 tx=0).stop 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 -- 172.21.15.59:0/3892159266 shutdown_connections 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d009b410 0x7f27d00af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 --2- 172.21.15.59:0/3892159266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d00d0e70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 -- 172.21.15.59:0/3892159266 >> 172.21.15.59:0/3892159266 conn(0x7f27d00b3ef0 msgr2=0x7f27d00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 -- 172.21.15.59:0/3892159266 shutdown_connections 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.489+0000 7f27d6000640 1 -- 172.21.15.59:0/3892159266 wait complete. 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.490+0000 7f27d6000640 1 Processor -- start 2024-04-24T06:28:40.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 -- start start 2024-04-24T06:28:40.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d009b410 0x7f27d00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 0x7f27d0152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27d00ccef0 con 0x7f27d00d09e0 2024-04-24T06:28:40.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27d00af5d0 con 0x7f27d009b410 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.491+0000 7f27d6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27d015cdb0 con 0x7f27d015c9d0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58426/0 (socket says 172.21.15.59:58426) 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 -- 172.21.15.59:0/3701161728 learned_addr learned my addr 172.21.15.59:0/3701161728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27d4c00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d009b410 0x7f27d00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27d5600640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 0x7f27d0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 -- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 msgr2=0x7f27d0152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 0x7f27d0152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 -- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d009b410 msgr2=0x7f27d00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d009b410 0x7f27d00c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27cfe00640 1 -- 172.21.15.59:0/3701161728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d0070840 con 0x7f27d00d09e0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.492+0000 7f27d5600640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 0x7f27d0152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27cfe00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27bc007e40 tx=0x7f27bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc014620 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27bc019070 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc014a00 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d0070b20 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.493+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d00710e0 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.494+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d004fd90 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.495+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27bc014ba0 con 0x7f27d00d09e0 2024-04-24T06:28:40.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.496+0000 7f27ccc00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 0x7f27b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.496+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f27bc08c240 con 0x7f27d00d09e0 2024-04-24T06:28:40.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.496+0000 7f27d4c00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 0x7f27b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.497+0000 7f27d4c00640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 0x7f27b006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27c0001640 tx=0x7f27c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.501+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27bc05a890 con 0x7f27d00d09e0 2024-04-24T06:28:40.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.752+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d00d07c0 con 0x7f27d00d09e0 2024-04-24T06:28:40.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.754+0000 7f27ccc00640 1 -- 172.21.15.59:0/3701161728 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7f27bc05a190 con 0x7f27d00d09e0 2024-04-24T06:28:40.755 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:40.755 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T06:28:40.755 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.756+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 msgr2=0x7f27b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.756+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 0x7f27b006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27c0001640 tx=0x7f27c0009040 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.756+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 msgr2=0x7f27d006fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.756+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27bc007e40 tx=0x7f27bc00e4b0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 shutdown_connections 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27b006c8f0 0x7f27b006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27d015c9d0 0x7f27d0152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27d00d09e0 0x7f27d006fdf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 --2- 172.21.15.59:0/3701161728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d009b410 0x7f27d00c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 >> 172.21.15.59:0/3701161728 conn(0x7f27d00b3ef0 msgr2=0x7f27d015f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:40.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 shutdown_connections 2024-04-24T06:28:40.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.757+0000 7f27d6000640 1 -- 172.21.15.59:0/3701161728 wait complete. 2024-04-24T06:28:40.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:40.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.981+0000 7f29bce00640 1 Processor -- start 2024-04-24T06:28:40.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.981+0000 7f29bce00640 1 -- start start 2024-04-24T06:28:40.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.982+0000 7f29bce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29b80cc7e0 0x7f29b8167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.982+0000 7f29bce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b8160210 con 0x7f29b815c3d0 2024-04-24T06:28:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.982+0000 7f29bce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b81604f0 con 0x7f29b80cd760 2024-04-24T06:28:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.982+0000 7f29bce00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f29b81607d0 con 0x7f29b80cc7e0 2024-04-24T06:28:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29b80cc7e0 0x7f29b8167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29b80cc7e0 0x7f29b8167400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58440/0 (socket says 172.21.15.59:58440) 2024-04-24T06:28:40.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 -- 172.21.15.59:0/3908206220 learned_addr learned my addr 172.21.15.59:0/3908206220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 -- 172.21.15.59:0/3908206220 >> v1:172.21.15.66:6789/0 conn(0x7f29b80cd760 legacy=0x7f29b80b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 -- 172.21.15.59:0/3908206220 >> v1:172.21.15.59:6789/0 conn(0x7f29b815c3d0 legacy=0x7f29b80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.983+0000 7f29b7e00640 1 -- 172.21.15.59:0/3908206220 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b8160ab0 con 0x7f29b80cc7e0 2024-04-24T06:28:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.984+0000 7f29b7e00640 1 --2- 172.21.15.59:0/3908206220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cc7e0 0x7f29b8167400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33369b9dbfa26e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29b6000640 1 -- 172.21.15.59:0/3908206220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0037070 con 0x7f29b80cc7e0 2024-04-24T06:28:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29b6000640 1 -- 172.21.15.59:0/3908206220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b0032440 con 0x7f29b80cc7e0 2024-04-24T06:28:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29b6000640 1 -- 172.21.15.59:0/3908206220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0032810 con 0x7f29b80cc7e0 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29bce00640 1 -- 172.21.15.59:0/3908206220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cc7e0 msgr2=0x7f29b8167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29bce00640 1 --2- 172.21.15.59:0/3908206220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cc7e0 0x7f29b8167400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009870 tx=0x7f29b002a2a0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29bce00640 1 -- 172.21.15.59:0/3908206220 shutdown_connections 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29bce00640 1 --2- 172.21.15.59:0/3908206220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cc7e0 0x7f29b8167400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.985+0000 7f29bce00640 1 -- 172.21.15.59:0/3908206220 >> 172.21.15.59:0/3908206220 conn(0x7f29b80b21a0 msgr2=0x7f29b80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.986+0000 7f29bce00640 1 -- 172.21.15.59:0/3908206220 shutdown_connections 2024-04-24T06:28:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.986+0000 7f29bce00640 1 -- 172.21.15.59:0/3908206220 wait complete. 2024-04-24T06:28:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.987+0000 7f29bce00640 1 Processor -- start 2024-04-24T06:28:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.987+0000 7f29bce00640 1 -- start start 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 0x7f29b80ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b815c3d0 0x7f29b80c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29b815c930 con 0x7f29b80cd760 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29b815c7b0 con 0x7f29b815c3d0 2024-04-24T06:28:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29bce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29b80b4860 con 0x7f29b80cc7e0 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.988+0000 7f29b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58448/0 (socket says 172.21.15.59:58448) 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 -- 172.21.15.59:0/49156619 learned_addr learned my addr 172.21.15.59:0/49156619 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b7e00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b815c3d0 0x7f29b80c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b7400640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 0x7f29b80ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 -- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 msgr2=0x7f29b80ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 0x7f29b80ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 -- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b815c3d0 msgr2=0x7f29b80c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b815c3d0 0x7f29b80c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 -- 172.21.15.59:0/49156619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b80ae630 con 0x7f29b80cd760 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b7400640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 0x7f29b80ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.989+0000 7f29b6a00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c460 tx=0x7f29a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.990+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4016020 con 0x7f29b80cd760 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.990+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4012870 con 0x7f29b80cd760 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.990+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4012c60 con 0x7f29b80cd760 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.990+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b8073300 con 0x7f29b80cd760 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.990+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b80c4e90 con 0x7f29b80cd760 2024-04-24T06:28:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.991+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b804fd90 con 0x7f29b80cd760 2024-04-24T06:28:40.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.992+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29a4019020 con 0x7f29b80cd760 2024-04-24T06:28:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.993+0000 7f29af400640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 0x7f299806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.993+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f29a408b5e0 con 0x7f29b80cd760 2024-04-24T06:28:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.993+0000 7f29b7400640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 0x7f299806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.994+0000 7f29b7400640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 0x7f299806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29b8154570 tx=0x7f29a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:40.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:40.999+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29a4020030 con 0x7f29b80cd760 2024-04-24T06:28:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.205+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b806e400 con 0x7f29b80cd760 2024-04-24T06:28:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.207+0000 7f29af400640 1 -- 172.21.15.59:0/49156619 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f29a4059cc0 con 0x7f29b80cd760 2024-04-24T06:28:41.208 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:41.208 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:35.616935+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 msgr2=0x7f299806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 0x7f299806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29b8154570 tx=0x7f29a8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 msgr2=0x7f29b80c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c460 tx=0x7f29a400c920 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 shutdown_connections 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f299806c8f0 0x7f299806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b815c3d0 0x7f29b80c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b80cd760 0x7f29b80c9c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 --2- 172.21.15.59:0/49156619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29b80cc7e0 0x7f29b80ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.209+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 >> 172.21.15.59:0/49156619 conn(0x7f29b80b21a0 msgr2=0x7f29b806ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.210+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 shutdown_connections 2024-04-24T06:28:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.210+0000 7f29bce00640 1 -- 172.21.15.59:0/49156619 wait complete. 2024-04-24T06:28:41.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:28:41.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:28:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.440+0000 7f2acfe00640 1 Processor -- start 2024-04-24T06:28:41.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.440+0000 7f2acfe00640 1 -- start start 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2ad015d810 0x7f2ad0160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad015cab0 con 0x7f2ad00d0d10 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acfe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad0160e40 con 0x7f2ad01567e0 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acfe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad015cd30 con 0x7f2ad015d810 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2ace000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2ad00d0d10 0x7f2ad0160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49278/0 (socket says 172.21.15.59:49278) 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2ace000640 1 -- 172.21.15.59:0/3799101886 learned_addr learned my addr 172.21.15.59:0/3799101886 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:41.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acf400640 1 --2- 172.21.15.59:0/3799101886 >> v2:172.21.15.66:3300/0 conn(0x7f2ad015d810 0x7f2ad0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.442+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832186165 0 0) 0x7f2abc001110 con 0x7f2ad00d0d10 2024-04-24T06:28:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab4003520 con 0x7f2ad00d0d10 2024-04-24T06:28:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1103507201 0 0) 0x7f2ab4003520 con 0x7f2ad00d0d10 2024-04-24T06:28:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 >> v2:172.21.15.66:3300/0 conn(0x7f2ad015d810 msgr2=0x7f2ad0160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:41.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 --2- 172.21.15.59:0/3799101886 >> v2:172.21.15.66:3300/0 conn(0x7f2ad015d810 0x7f2ad0160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 >> v1:172.21.15.66:6789/0 conn(0x7f2ad01567e0 legacy=0x7f2ad00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad015d010 con 0x7f2ad00d0d10 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2abc001840 con 0x7f2ad00d0d10 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 >> v1:172.21.15.59:6789/0 conn(0x7f2ad00d0d10 legacy=0x7f2ad0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ab400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad015cd30 con 0x7f2ab40051c0 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab400c0c0 con 0x7f2ab4003a00 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab400c2c0 con 0x7f2ab4004800 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.443+0000 7f2acf400640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ab400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acea00640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2ace000640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acf400640 1 -- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ab4004800 msgr2=0x7f2ab4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acf400640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acf400640 1 -- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab4003a00 msgr2=0x7f2ab4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acf400640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acf400640 1 -- 172.21.15.59:0/3799101886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab4003520 con 0x7f2ab40051c0 2024-04-24T06:28:41.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.444+0000 7f2acea00640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.445+0000 7f2acf400640 1 -- 172.21.15.59:0/3799101886 reap_dead start 2024-04-24T06:28:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.445+0000 7f2acf400640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ab400b9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009f90 tx=0x7f2ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d8890180b62393 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.445+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8035020 con 0x7f2ab40051c0 2024-04-24T06:28:41.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.445+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac8003b90 con 0x7f2ab40051c0 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.445+0000 7f2acd600640 1 -- 172.21.15.59:0/3799101886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8009c10 con 0x7f2ab40051c0 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 -- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 msgr2=0x7f2ab400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ab400b9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009f90 tx=0x7f2ab4005830 comp rx=0 tx=0).stop 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 -- 172.21.15.59:0/3799101886 shutdown_connections 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 --2- 172.21.15.59:0/3799101886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ab400b9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 -- 172.21.15.59:0/3799101886 >> 172.21.15.59:0/3799101886 conn(0x7f2ad006bdf0 msgr2=0x7f2ad006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 -- 172.21.15.59:0/3799101886 shutdown_connections 2024-04-24T06:28:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.446+0000 7f2acfe00640 1 -- 172.21.15.59:0/3799101886 wait complete. 2024-04-24T06:28:41.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.448+0000 7f2acfe00640 1 Processor -- start 2024-04-24T06:28:41.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.448+0000 7f2acfe00640 1 -- start start 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 0x7f2ad0153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 0x7f2ad009de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0156d40 con 0x7f2ab40051c0 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0156bc0 con 0x7f2ad01532b0 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad0156ec0 con 0x7f2ad009d980 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58478/0 (socket says 172.21.15.59:58478) 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.449+0000 7f2acea00640 1 -- 172.21.15.59:0/95163352 learned_addr learned my addr 172.21.15.59:0/95163352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acf400640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 0x7f2ad009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 -- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 msgr2=0x7f2ad009de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2ace000640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 0x7f2ad0153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 0x7f2ad009de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 -- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 msgr2=0x7f2ad0153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 0x7f2ad0153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 -- 172.21.15.59:0/95163352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad01f8b10 con 0x7f2ab40051c0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2ace000640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 0x7f2ad0153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acf400640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 0x7f2ad009de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2acea00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e530 tx=0x7f2ac000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.450+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00052e0 con 0x7f2ab40051c0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.451+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0018070 con 0x7f2ab40051c0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.451+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003810 con 0x7f2ab40051c0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.451+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad009cbc0 con 0x7f2ab40051c0 2024-04-24T06:28:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.451+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad009d180 con 0x7f2ab40051c0 2024-04-24T06:28:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.452+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad004fd90 con 0x7f2ab40051c0 2024-04-24T06:28:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.453+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ac00039b0 con 0x7f2ab40051c0 2024-04-24T06:28:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.454+0000 7f2ac6a00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 0x7f2ab006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.454+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2ac008b3f0 con 0x7f2ab40051c0 2024-04-24T06:28:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.454+0000 7f2ace000640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 0x7f2ab006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.455+0000 7f2ace000640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 0x7f2ab006ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2abc001eb0 tx=0x7f2abc000f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.459+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ac0059b60 con 0x7f2ab40051c0 2024-04-24T06:28:41.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.662+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ad00b5150 con 0x7f2ab40051c0 2024-04-24T06:28:41.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.663+0000 7f2ac6a00640 1 -- 172.21.15.59:0/95163352 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2ac0059460 con 0x7f2ab40051c0 2024-04-24T06:28:41.664 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 msgr2=0x7f2ab006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 0x7f2ab006ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2abc001eb0 tx=0x7f2abc000f90 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 msgr2=0x7f2ad01f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e530 tx=0x7f2ac000e9f0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 shutdown_connections 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ab006c920 0x7f2ab006ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ad009d980 0x7f2ad009de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ad01532b0 0x7f2ad0153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.666+0000 7f2acfe00640 1 --2- 172.21.15.59:0/95163352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab40051c0 0x7f2ad01f7540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.667+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 >> 172.21.15.59:0/95163352 conn(0x7f2ad006bdf0 msgr2=0x7f2ad015b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.667+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 shutdown_connections 2024-04-24T06:28:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.667+0000 7f2acfe00640 1 -- 172.21.15.59:0/95163352 wait complete. 2024-04-24T06:28:41.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:28:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.893+0000 7f38fa000640 1 Processor -- start 2024-04-24T06:28:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.893+0000 7f38fa000640 1 -- start start 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38fa000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f38f406f3f0 0x7f38f40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38fa000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38f40b3080 0x7f38f40cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38fa000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f40b3870 con 0x7f38f415b450 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38fa000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40b41a0 con 0x7f38f40b3080 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38fa000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40b3b50 con 0x7f38f406f3f0 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38f9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38f40b3080 0x7f38f40cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38f9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38f40b3080 0x7f38f40cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58492/0 (socket says 172.21.15.59:58492) 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38f9600640 1 -- 172.21.15.59:0/1886393447 learned_addr learned my addr 172.21.15.59:0/1886393447 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38f3e00640 1 --2- 172.21.15.59:0/1886393447 >> v2:172.21.15.66:3300/0 conn(0x7f38f406f3f0 0x7f38f40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.895+0000 7f38f9600640 1 -- 172.21.15.59:0/1886393447 >> v2:172.21.15.66:3300/0 conn(0x7f38f406f3f0 msgr2=0x7f38f40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.896+0000 7f38f9600640 1 --2- 172.21.15.59:0/1886393447 >> v2:172.21.15.66:3300/0 conn(0x7f38f406f3f0 0x7f38f40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.896+0000 7f38f9600640 1 -- 172.21.15.59:0/1886393447 >> v1:172.21.15.66:6789/0 conn(0x7f38f415b450 legacy=0x7f38f40b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.896+0000 7f38f9600640 1 -- 172.21.15.59:0/1886393447 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40b3e30 con 0x7f38f40b3080 2024-04-24T06:28:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38f9600640 1 --2- 172.21.15.59:0/1886393447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40cd590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f40af4a0 tx=0x7f38ec02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f32a29c2dd9dadd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38f3400640 1 -- 172.21.15.59:0/1886393447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec036070 con 0x7f38f40b3080 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38f3400640 1 -- 172.21.15.59:0/1886393447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec031600 con 0x7f38f40b3080 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38f3400640 1 -- 172.21.15.59:0/1886393447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0319f0 con 0x7f38f40b3080 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38fa000640 1 -- 172.21.15.59:0/1886393447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 msgr2=0x7f38f40cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.897+0000 7f38fa000640 1 --2- 172.21.15.59:0/1886393447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40cd590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f40af4a0 tx=0x7f38ec02a7c0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 -- 172.21.15.59:0/1886393447 shutdown_connections 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 --2- 172.21.15.59:0/1886393447 >> v2:172.21.15.66:3300/0 conn(0x7f38f406f3f0 0x7f38f40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 --2- 172.21.15.59:0/1886393447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40cd590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 -- 172.21.15.59:0/1886393447 >> 172.21.15.59:0/1886393447 conn(0x7f38f40b21a0 msgr2=0x7f38f40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 -- 172.21.15.59:0/1886393447 shutdown_connections 2024-04-24T06:28:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.898+0000 7f38fa000640 1 -- 172.21.15.59:0/1886393447 wait complete. 2024-04-24T06:28:41.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.899+0000 7f38fa000640 1 Processor -- start 2024-04-24T06:28:41.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.900+0000 7f38fa000640 1 -- start start 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.900+0000 7f38fa000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 0x7f38f40c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.900+0000 7f38fa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.900+0000 7f38fa000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f38f415b450 0x7f38f4072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38fa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40ad060 con 0x7f38f40b3080 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38fa000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40b4ff0 con 0x7f38f406f3f0 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38fa000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f38f40ad1e0 con 0x7f38f415b450 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58508/0 (socket says 172.21.15.59:58508) 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 -- 172.21.15.59:0/3856502018 learned_addr learned my addr 172.21.15.59:0/3856502018 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f8c00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 0x7f38f40c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f9600640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f38f415b450 0x7f38f4072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 -- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f38f415b450 msgr2=0x7f38f4072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f38f415b450 0x7f38f4072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 -- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 msgr2=0x7f38f40c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 0x7f38f40c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f3e00640 1 -- 172.21.15.59:0/3856502018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f4073220 con 0x7f38f40b3080 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.901+0000 7f38f8c00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 0x7f38f40c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38f3e00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d830 tx=0x7f38e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e000b660 con 0x7f38f40b3080 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e0019070 con 0x7f38f40b3080 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e000ba60 con 0x7f38f40b3080 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f4073500 con 0x7f38f40b3080 2024-04-24T06:28:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.902+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f4073ac0 con 0x7f38f40b3080 2024-04-24T06:28:41.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.903+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f40cf7f0 con 0x7f38f40b3080 2024-04-24T06:28:41.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.904+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38e001e460 con 0x7f38f40b3080 2024-04-24T06:28:41.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.905+0000 7f38f0c00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:41.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.906+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f38e008bf50 con 0x7f38f40b3080 2024-04-24T06:28:41.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.906+0000 7f38f8c00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:41.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.907+0000 7f38f8c00640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 0x7f38d406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f60 tx=0x7f38e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:41.910+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38f40cf7f0 con 0x7f38f40b3080 2024-04-24T06:28:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.122+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38f404fd90 con 0x7f38f40b3080 2024-04-24T06:28:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.122+0000 7f38f0c00640 1 -- 172.21.15.59:0/3856502018 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f38e005a630 con 0x7f38f40b3080 2024-04-24T06:28:42.123 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:28:42.124 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 msgr2=0x7f38d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 0x7f38d406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f60 tx=0x7f38e8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 msgr2=0x7f38f40726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38e000d830 tx=0x7f38e000dcf0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 shutdown_connections 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f38d406c920 0x7f38d406ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f38f415b450 0x7f38f4072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38f40b3080 0x7f38f40726d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 --2- 172.21.15.59:0/3856502018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38f406f3f0 0x7f38f40c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 >> 172.21.15.59:0/3856502018 conn(0x7f38f40b21a0 msgr2=0x7f38f40cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 shutdown_connections 2024-04-24T06:28:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.125+0000 7f38fa000640 1 -- 172.21.15.59:0/3856502018 wait complete. 2024-04-24T06:28:42.145 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:28:42.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:28:42.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.344+0000 7f025b000640 1 Processor -- start 2024-04-24T06:28:42.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.344+0000 7f025b000640 1 -- start start 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f025b000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0254156c20 0x7f02540aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f025b000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f025406c500 0x7f0254160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f025b000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0254157000 con 0x7f02540af4c0 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f025b000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02540afb00 con 0x7f025406c500 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f025b000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f025415cc20 con 0x7f0254156c20 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f0259200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f02540af4c0 0x7f025406bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49290/0 (socket says 172.21.15.59:49290) 2024-04-24T06:28:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.346+0000 7f0259200640 1 -- 172.21.15.59:0/4113960333 learned_addr learned my addr 172.21.15.59:0/4113960333 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f025a600640 1 --2- 172.21.15.59:0/4113960333 >> v2:172.21.15.59:3300/0 conn(0x7f025406c500 0x7f0254160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f0259c00640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0254156c20 0x7f02540aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f0253e00640 1 -- 172.21.15.59:0/4113960333 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766713406 0 0) 0x7f0254157000 con 0x7f02540af4c0 2024-04-24T06:28:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f0253e00640 1 -- 172.21.15.59:0/4113960333 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023c003520 con 0x7f02540af4c0 2024-04-24T06:28:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f025a600640 1 -- 172.21.15.59:0/4113960333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0254156c20 msgr2=0x7f02540aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f025a600640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0254156c20 0x7f02540aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f025a600640 1 -- 172.21.15.59:0/4113960333 >> v1:172.21.15.59:6789/0 conn(0x7f02540af4c0 legacy=0x7f025406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.347+0000 7f025a600640 1 -- 172.21.15.59:0/4113960333 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025415cf00 con 0x7f025406c500 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.348+0000 7f025a600640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025406c500 0x7f0254160100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ee29734c102823 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.348+0000 7f0253e00640 1 -- 172.21.15.59:0/4113960333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037070 con 0x7f025406c500 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.348+0000 7f0253e00640 1 -- 172.21.15.59:0/4113960333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c032440 con 0x7f025406c500 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.348+0000 7f0253e00640 1 -- 172.21.15.59:0/4113960333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0327f0 con 0x7f025406c500 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 -- 172.21.15.59:0/4113960333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025406c500 msgr2=0x7f0254160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025406c500 0x7f0254160100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02a180 comp rx=0 tx=0).stop 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 -- 172.21.15.59:0/4113960333 shutdown_connections 2024-04-24T06:28:42.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f025406c500 0x7f0254160100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 --2- 172.21.15.59:0/4113960333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0254156c20 0x7f02540aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 -- 172.21.15.59:0/4113960333 >> 172.21.15.59:0/4113960333 conn(0x7f02540b2200 msgr2=0x7f02540b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 -- 172.21.15.59:0/4113960333 shutdown_connections 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.349+0000 7f025b000640 1 -- 172.21.15.59:0/4113960333 wait complete. 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.351+0000 7f025b000640 1 Processor -- start 2024-04-24T06:28:42.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.351+0000 7f025b000640 1 -- start start 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f025406c500 0x7f02540ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0254156c20 0x7f025406f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f025415d9e0 con 0x7f02540af4c0 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f025415d4c0 con 0x7f0254156c20 2024-04-24T06:28:42.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025b000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0254073ba0 con 0x7f025406c500 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58532/0 (socket says 172.21.15.59:58532) 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 -- 172.21.15.59:0/1516739976 learned_addr learned my addr 172.21.15.59:0/1516739976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f025a600640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0254156c20 0x7f025406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 -- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f025406c500 msgr2=0x7f02540ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259c00640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f025406c500 0x7f02540ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f025406c500 0x7f02540ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 -- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0254156c20 msgr2=0x7f025406f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0254156c20 0x7f025406f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.352+0000 7f0259200640 1 -- 172.21.15.59:0/1516739976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025414c6c0 con 0x7f02540af4c0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f0259200640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f024000c460 tx=0x7f024000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240017070 con 0x7f02540af4c0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0240005ce0 con 0x7f02540af4c0 2024-04-24T06:28:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025414c9a0 con 0x7f02540af4c0 2024-04-24T06:28:42.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240007290 con 0x7f02540af4c0 2024-04-24T06:28:42.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.353+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025414cf60 con 0x7f02540af4c0 2024-04-24T06:28:42.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.354+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025404fd90 con 0x7f02540af4c0 2024-04-24T06:28:42.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.355+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f024001a070 con 0x7f02540af4c0 2024-04-24T06:28:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.356+0000 7f0251600640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 0x7f023406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.357+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f024008ab80 con 0x7f02540af4c0 2024-04-24T06:28:42.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.357+0000 7f0259c00640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 0x7f023406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.357+0000 7f0259c00640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 0x7f023406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0254156570 tx=0x7f024800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.361+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02400592f0 con 0x7f02540af4c0 2024-04-24T06:28:42.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.563+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f02540b5090 con 0x7f02540af4c0 2024-04-24T06:28:42.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.683+0000 7f0251600640 1 -- 172.21.15.59:0/1516739976 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f0240058bf0 con 0x7f02540af4c0 2024-04-24T06:28:42.685 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:28:42.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.686+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 msgr2=0x7f023406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.686+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 0x7f023406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0254156570 tx=0x7f024800b040 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.686+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 msgr2=0x7f025406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.686+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f024000c460 tx=0x7f024000c920 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 shutdown_connections 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f023406c8f0 0x7f023406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0254156c20 0x7f025406f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02540af4c0 0x7f025406eef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 --2- 172.21.15.59:0/1516739976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f025406c500 0x7f02540ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 >> 172.21.15.59:0/1516739976 conn(0x7f02540b2200 msgr2=0x7f02540aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 shutdown_connections 2024-04-24T06:28:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.687+0000 7f025b000640 1 -- 172.21.15.59:0/1516739976 wait complete. 2024-04-24T06:28:42.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:28:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.915+0000 7f4b76200640 1 Processor -- start 2024-04-24T06:28:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.915+0000 7f4b76200640 1 -- start start 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.916+0000 7f4b76200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b7006dfa0 0x7f4b70160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.916+0000 7f4b76200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4b700ce880 0x7f4b700b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b76200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b7006dd40 con 0x7f4b700ccb90 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b76200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4b700b3fe0 con 0x7f4b7006dfa0 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b76200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4b701607e0 con 0x7f4b700ce880 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b7006dfa0 0x7f4b70160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b7006dfa0 0x7f4b70160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58534/0 (socket says 172.21.15.59:58534) 2024-04-24T06:28:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 -- 172.21.15.59:0/1641343666 learned_addr learned my addr 172.21.15.59:0/1641343666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b75800640 1 --2- 172.21.15.59:0/1641343666 >> v2:172.21.15.66:3300/0 conn(0x7f4b700ce880 0x7f4b700b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 -- 172.21.15.59:0/1641343666 >> v2:172.21.15.66:3300/0 conn(0x7f4b700ce880 msgr2=0x7f4b700b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 --2- 172.21.15.59:0/1641343666 >> v2:172.21.15.66:3300/0 conn(0x7f4b700ce880 0x7f4b700b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 -- 172.21.15.59:0/1641343666 >> v1:172.21.15.66:6789/0 conn(0x7f4b700ccb90 legacy=0x7f4b700ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.917+0000 7f4b6fe00640 1 -- 172.21.15.59:0/1641343666 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70160ac0 con 0x7f4b7006dfa0 2024-04-24T06:28:42.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.918+0000 7f4b6fe00640 1 --2- 172.21.15.59:0/1641343666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70160200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b700ca780 tx=0x7f4b5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2369908d42c7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b6f400640 1 -- 172.21.15.59:0/1641343666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c037070 con 0x7f4b7006dfa0 2024-04-24T06:28:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b6f400640 1 -- 172.21.15.59:0/1641343666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c032440 con 0x7f4b7006dfa0 2024-04-24T06:28:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b6f400640 1 -- 172.21.15.59:0/1641343666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c032810 con 0x7f4b7006dfa0 2024-04-24T06:28:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 -- 172.21.15.59:0/1641343666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 msgr2=0x7f4b70160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 --2- 172.21.15.59:0/1641343666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70160200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b700ca780 tx=0x7f4b5c02a230 comp rx=0 tx=0).stop 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 -- 172.21.15.59:0/1641343666 shutdown_connections 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 --2- 172.21.15.59:0/1641343666 >> v2:172.21.15.66:3300/0 conn(0x7f4b700ce880 0x7f4b700b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 --2- 172.21.15.59:0/1641343666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70160200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 -- 172.21.15.59:0/1641343666 >> 172.21.15.59:0/1641343666 conn(0x7f4b7006bbb0 msgr2=0x7f4b7006bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 -- 172.21.15.59:0/1641343666 shutdown_connections 2024-04-24T06:28:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.919+0000 7f4b76200640 1 -- 172.21.15.59:0/1641343666 wait complete. 2024-04-24T06:28:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.921+0000 7f4b76200640 1 Processor -- start 2024-04-24T06:28:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.921+0000 7f4b76200640 1 -- start start 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b700ccb90 0x7f4b7015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4b700ce880 0x7f4b7015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7015c910 con 0x7f4b7006dfa0 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70161080 con 0x7f4b700ccb90 2024-04-24T06:28:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.922+0000 7f4b76200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7015ca90 con 0x7f4b700ce880 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58550/0 (socket says 172.21.15.59:58550) 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 -- 172.21.15.59:0/2152277893 learned_addr learned my addr 172.21.15.59:0/2152277893 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b6fe00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b700ccb90 0x7f4b7015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b75800640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4b700ce880 0x7f4b7015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 -- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4b700ce880 msgr2=0x7f4b7015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4b700ce880 0x7f4b7015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 -- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b700ccb90 msgr2=0x7f4b7015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b700ccb90 0x7f4b7015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 -- 172.21.15.59:0/2152277893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7015e850 con 0x7f4b7006dfa0 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.923+0000 7f4b74e00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c460 tx=0x7f4b6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.924+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64016020 con 0x7f4b7006dfa0 2024-04-24T06:28:42.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.924+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64012870 con 0x7f4b7006dfa0 2024-04-24T06:28:42.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.924+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64012c60 con 0x7f4b7006dfa0 2024-04-24T06:28:42.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.924+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7015eb30 con 0x7f4b7006dfa0 2024-04-24T06:28:42.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.924+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7015f0f0 con 0x7f4b7006dfa0 2024-04-24T06:28:42.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.925+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700ce450 con 0x7f4b7006dfa0 2024-04-24T06:28:42.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.926+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b64019020 con 0x7f4b7006dfa0 2024-04-24T06:28:42.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.927+0000 7f4b6cc00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 0x7f4b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:42.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.927+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f4b6408b5c0 con 0x7f4b7006dfa0 2024-04-24T06:28:42.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.927+0000 7f4b6fe00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 0x7f4b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:42.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.928+0000 7f4b6fe00640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 0x7f4b5006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c034000 tx=0x7f4b5c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:42.932+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b700ce450 con 0x7f4b7006dfa0 2024-04-24T06:28:43.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.135+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4b70068290 con 0x7f4b7006dfa0 2024-04-24T06:28:43.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.699+0000 7f4b6cc00640 1 -- 172.21.15.59:0/2152277893 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f4b70068290 con 0x7f4b7006dfa0 2024-04-24T06:28:43.701 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.703+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 msgr2=0x7f4b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.703+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 0x7f4b5006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c034000 tx=0x7f4b5c02e000 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.703+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 msgr2=0x7f4b70099620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.703+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c460 tx=0x7f4b6400c920 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 shutdown_connections 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4b5006c8f0 0x7f4b5006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4b700ce880 0x7f4b7015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b700ccb90 0x7f4b7015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 --2- 172.21.15.59:0/2152277893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b7006dfa0 0x7f4b70099620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 >> 172.21.15.59:0/2152277893 conn(0x7f4b7006bbb0 msgr2=0x7f4b700b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:43.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 shutdown_connections 2024-04-24T06:28:43.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.704+0000 7f4b76200640 1 -- 172.21.15.59:0/2152277893 wait complete. 2024-04-24T06:28:43.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:28:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.922+0000 7f8723000640 1 Processor -- start 2024-04-24T06:28:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.922+0000 7f8723000640 1 -- start start 2024-04-24T06:28:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8723000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f871c15e790 0x7f871c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8723000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 0x7f871c0721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8723000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0adc80 con 0x7f871c06f6f0 2024-04-24T06:28:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8723000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f871c0aef40 con 0x7f871c15e790 2024-04-24T06:28:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8723000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f871c072720 con 0x7f871c071d70 2024-04-24T06:28:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.924+0000 7f8721000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 0x7f871c0721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f871be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f871c15e790 0x7f871c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f8721000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 0x7f871c0721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:60292/0 (socket says 172.21.15.59:60292) 2024-04-24T06:28:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f8721000640 1 -- 172.21.15.59:0/3346751249 learned_addr learned my addr 172.21.15.59:0/3346751249 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f871be00640 1 -- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 msgr2=0x7f871c0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f871be00640 1 --2- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 0x7f871c0721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f871be00640 1 -- 172.21.15.59:0/3346751249 >> v1:172.21.15.66:6789/0 conn(0x7f871c06f6f0 legacy=0x7f871c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.925+0000 7f871be00640 1 -- 172.21.15.59:0/3346751249 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c0adf20 con 0x7f871c15e790 2024-04-24T06:28:43.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.927+0000 7f871be00640 1 --2- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c15e790 0x7f871c0aea00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f870c0097a0 tx=0x7f870c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b7c967f1344152 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.927+0000 7f871aa00640 1 -- 172.21.15.59:0/3346751249 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c037070 con 0x7f871c15e790 2024-04-24T06:28:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.927+0000 7f871aa00640 1 -- 172.21.15.59:0/3346751249 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c032440 con 0x7f871c15e790 2024-04-24T06:28:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.927+0000 7f871aa00640 1 -- 172.21.15.59:0/3346751249 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c032810 con 0x7f871c15e790 2024-04-24T06:28:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 -- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c15e790 msgr2=0x7f871c0aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 --2- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c15e790 0x7f871c0aea00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f870c0097a0 tx=0x7f870c02a180 comp rx=0 tx=0).stop 2024-04-24T06:28:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 -- 172.21.15.59:0/3346751249 shutdown_connections 2024-04-24T06:28:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 --2- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c071d70 0x7f871c0721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 --2- 172.21.15.59:0/3346751249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c15e790 0x7f871c0aea00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 -- 172.21.15.59:0/3346751249 >> 172.21.15.59:0/3346751249 conn(0x7f871c067d90 msgr2=0x7f871c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 -- 172.21.15.59:0/3346751249 shutdown_connections 2024-04-24T06:28:43.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.928+0000 7f8723000640 1 -- 172.21.15.59:0/3346751249 wait complete. 2024-04-24T06:28:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.930+0000 7f8723000640 1 Processor -- start 2024-04-24T06:28:43.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.930+0000 7f8723000640 1 -- start start 2024-04-24T06:28:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 0x7f871c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c15e790 0x7f871c1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0ae4e0 con 0x7f871c071d70 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f871c06fbb0 con 0x7f871c06f6f0 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f8723000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f871c15eb70 con 0x7f871c15e790 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f871b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58556/0 (socket says 172.21.15.59:58556) 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.931+0000 7f871b400640 1 -- 172.21.15.59:0/2686978046 learned_addr learned my addr 172.21.15.59:0/2686978046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871be00640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 0x7f871c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f8721000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c15e790 0x7f871c1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 -- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c15e790 msgr2=0x7f871c1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c15e790 0x7f871c1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 -- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 msgr2=0x7f871c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 0x7f871c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 -- 172.21.15.59:0/2686978046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c15bd10 con 0x7f871c071d70 2024-04-24T06:28:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871be00640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 0x7f871c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f871b400640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f870800c400 tx=0x7f870800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080052d0 con 0x7f871c071d70 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708005490 con 0x7f871c071d70 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.932+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c160e20 con 0x7f871c071d70 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.933+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708007260 con 0x7f871c071d70 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.933+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c15bfe0 con 0x7f871c071d70 2024-04-24T06:28:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.934+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c068a80 con 0x7f871c071d70 2024-04-24T06:28:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.935+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8708007400 con 0x7f871c071d70 2024-04-24T06:28:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.936+0000 7f8713e00640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 0x7f86fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.936+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f870808b120 con 0x7f871c071d70 2024-04-24T06:28:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.936+0000 7f871be00640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 0x7f86fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.937+0000 7f871be00640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 0x7f86fc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:43.941+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f871c15bd10 con 0x7f871c071d70 2024-04-24T06:28:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.182+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f871c162190 con 0x7f871c071d70 2024-04-24T06:28:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.704+0000 7f8713e00640 1 -- 172.21.15.59:0/2686978046 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8708059510 con 0x7f871c071d70 2024-04-24T06:28:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T06:28:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 msgr2=0x7f86fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 0x7f86fc06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c02e000 comp rx=0 tx=0).stop 2024-04-24T06:28:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 msgr2=0x7f871c1608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f870800c400 tx=0x7f870800c8c0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 shutdown_connections 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f86fc06c8f0 0x7f86fc06edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f871c15e790 0x7f871c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f871c071d70 0x7f871c1608e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 --2- 172.21.15.59:0/2686978046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f871c06f6f0 0x7f871c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.708+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 >> 172.21.15.59:0/2686978046 conn(0x7f871c067d90 msgr2=0x7f871c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.709+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 shutdown_connections 2024-04-24T06:28:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.709+0000 7f8723000640 1 -- 172.21.15.59:0/2686978046 wait complete. 2024-04-24T06:28:44.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.922+0000 7fd7aa400640 1 Processor -- start 2024-04-24T06:28:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.922+0000 7fd7aa400640 1 -- start start 2024-04-24T06:28:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.924+0000 7fd7aa400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd7a41454e0 0x7fd7a4145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.924+0000 7fd7aa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a4160910 0x7fd7a4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.924+0000 7fd7aa400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a40cd520 con 0x7fd7a40cd020 2024-04-24T06:28:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.924+0000 7fd7aa400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a40cddc0 con 0x7fd7a41454e0 2024-04-24T06:28:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.924+0000 7fd7aa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a40cd800 con 0x7fd7a4160910 2024-04-24T06:28:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd7a41454e0 0x7fd7a4145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd79be00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd7a40cd020 0x7fd7a40b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:38126/0 (socket says 172.21.15.59:38126) 2024-04-24T06:28:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd79be00640 1 -- 172.21.15.59:0/981024813 learned_addr learned my addr 172.21.15.59:0/981024813 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3e00640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a4160910 0x7fd7a4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3400640 1 -- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a4160910 msgr2=0x7fd7a4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3400640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a4160910 0x7fd7a4160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3400640 1 -- 172.21.15.59:0/981024813 >> v1:172.21.15.66:6789/0 conn(0x7fd7a40cd020 legacy=0x7fd7a40b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.925+0000 7fd7a3400640 1 -- 172.21.15.59:0/981024813 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a40cdae0 con 0x7fd7a41454e0 2024-04-24T06:28:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.927+0000 7fd7a3400640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a41454e0 0x7fd7a4145bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7940097a0 tx=0x7fd79402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7d8a3dd0081e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.927+0000 7fd7a2a00640 1 -- 172.21.15.59:0/981024813 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794037070 con 0x7fd7a41454e0 2024-04-24T06:28:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.927+0000 7fd7a2a00640 1 -- 172.21.15.59:0/981024813 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79402ae50 con 0x7fd7a41454e0 2024-04-24T06:28:44.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.927+0000 7fd7a2a00640 1 -- 172.21.15.59:0/981024813 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794032660 con 0x7fd7a41454e0 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 -- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a41454e0 msgr2=0x7fd7a4145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a41454e0 0x7fd7a4145bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7940097a0 tx=0x7fd79402a100 comp rx=0 tx=0).stop 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 -- 172.21.15.59:0/981024813 shutdown_connections 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a4160910 0x7fd7a4160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 --2- 172.21.15.59:0/981024813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a41454e0 0x7fd7a4145bb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 -- 172.21.15.59:0/981024813 >> 172.21.15.59:0/981024813 conn(0x7fd7a4067ea0 msgr2=0x7fd7a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 -- 172.21.15.59:0/981024813 shutdown_connections 2024-04-24T06:28:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.928+0000 7fd7aa400640 1 -- 172.21.15.59:0/981024813 wait complete. 2024-04-24T06:28:44.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.930+0000 7fd7aa400640 1 Processor -- start 2024-04-24T06:28:44.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.930+0000 7fd7aa400640 1 -- start start 2024-04-24T06:28:44.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a40cd020 0x7fd7a4142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a41454e0 0x7fd7a41f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40b4010 con 0x7fd7a4160910 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41460f0 con 0x7fd7a40cd020 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7aa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a41f9110 con 0x7fd7a41454e0 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58572/0 (socket says 172.21.15.59:58572) 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7a3e00640 1 -- 172.21.15.59:0/1281035584 learned_addr learned my addr 172.21.15.59:0/1281035584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.931+0000 7fd7a3400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a40cd020 0x7fd7a4142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd79be00640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a41454e0 0x7fd7a41f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 -- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a41454e0 msgr2=0x7fd7a41f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a41454e0 0x7fd7a41f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 -- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a40cd020 msgr2=0x7fd7a4142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:44.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a40cd020 0x7fd7a4142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:44.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 -- 172.21.15.59:0/1281035584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a41f25e0 con 0x7fd7a4160910 2024-04-24T06:28:44.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7a3e00640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009d30 tx=0x7fd79c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c018020 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c014480 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a41f4de0 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c014870 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.932+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41f53a0 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.934+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a4068a80 con 0x7fd7a4160910 2024-04-24T06:28:44.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.934+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd79c005d10 con 0x7fd7a4160910 2024-04-24T06:28:44.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.935+0000 7fd79b400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:44.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.936+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fd79c08c500 con 0x7fd7a4160910 2024-04-24T06:28:44.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.936+0000 7fd7a3400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:44.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.936+0000 7fd7a3400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 0x7fd78006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd794034000 tx=0x7fd79402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:44.940+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd79c022070 con 0x7fd7a4160910 2024-04-24T06:28:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.139+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7a4160030 con 0x7fd7a4160910 2024-04-24T06:28:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.140+0000 7fd79b400640 1 -- 172.21.15.59:0/1281035584 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16534 (secure 0 0 0) 0x7fd79c010040 con 0x7fd7a4160910 2024-04-24T06:28:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:45.142 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:44.183651+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 msgr2=0x7fd78006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 0x7fd78006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd794034000 tx=0x7fd79402e000 comp rx=0 tx=0).stop 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 msgr2=0x7fd7a41f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009d30 tx=0x7fd79c00c490 comp rx=0 tx=0).stop 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 shutdown_connections 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd78006c8f0 0x7fd78006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7a4160910 0x7fd7a41f20a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7a41454e0 0x7fd7a41f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 --2- 172.21.15.59:0/1281035584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7a40cd020 0x7fd7a4142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 >> 172.21.15.59:0/1281035584 conn(0x7fd7a4067ea0 msgr2=0x7fd7a40b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:45.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 shutdown_connections 2024-04-24T06:28:45.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.143+0000 7fd7aa400640 1 -- 172.21.15.59:0/1281035584 wait complete. 2024-04-24T06:28:45.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.367+0000 7fe4de600640 1 Processor -- start 2024-04-24T06:28:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.368+0000 7fe4de600640 1 -- start start 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4de600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 0x7fe4d8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4de600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80cd520 con 0x7fe4d81454e0 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4de600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80cd800 con 0x7fe4d80cd020 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4de600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80cddc0 con 0x7fe4d8160910 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4d7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe4d81454e0 0x7fe4d8145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49294/0 (socket says 172.21.15.59:49294) 2024-04-24T06:28:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.369+0000 7fe4d7400640 1 -- 172.21.15.59:0/1592397333 learned_addr learned my addr 172.21.15.59:0/1592397333 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169600964 0 0) 0x7fe4d80cd520 con 0x7fe4d81454e0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4bc003520 con 0x7fe4d81454e0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d7e00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 0x7fe4d8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3714308443 0 0) 0x7fe4bc003520 con 0x7fe4d81454e0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 msgr2=0x7fe4d8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 0x7fe4d8160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 >> v1:172.21.15.66:6789/0 conn(0x7fe4d80cd020 legacy=0x7fe4d80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d80cdae0 con 0x7fe4d81454e0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe4c8001670 con 0x7fe4d81454e0 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 >> v1:172.21.15.59:6789/0 conn(0x7fe4d81454e0 legacy=0x7fe4d8145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4bc0039e0 0x7fe4bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4bc004800 0x7fe4bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 0x7fe4bc005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80cddc0 con 0x7fe4bc004800 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc00bb80 con 0x7fe4bc0039e0 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc00bd80 con 0x7fe4d8160910 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.370+0000 7fe4cfe00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4bc004800 0x7fe4bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4d7400640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4bc0039e0 0x7fe4bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4cfe00640 1 -- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 msgr2=0x7fe4bc005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4cfe00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4d8160910 0x7fe4bc005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4cfe00640 1 -- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4bc0039e0 msgr2=0x7fe4bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4cfe00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4bc0039e0 0x7fe4bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4cfe00640 1 -- 172.21.15.59:0/1592397333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc003520 con 0x7fe4bc004800 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.371+0000 7fe4d7400640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4bc0039e0 0x7fe4bc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.372+0000 7fe4cfe00640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4bc004800 0x7fe4bc004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0003db0 tx=0x7fe4bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe221cae6ab632e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.372+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0037070 con 0x7fe4bc004800 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.372+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c002aa00 con 0x7fe4bc004800 2024-04-24T06:28:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.372+0000 7fe4d6a00640 1 -- 172.21.15.59:0/1592397333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0003980 con 0x7fe4bc004800 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 -- 172.21.15.59:0/1592397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4bc004800 msgr2=0x7fe4bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 --2- 172.21.15.59:0/1592397333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4bc004800 0x7fe4bc004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0003db0 tx=0x7fe4bc005830 comp rx=0 tx=0).stop 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4d7e00640 1 -- 172.21.15.59:0/1592397333 reap_dead start 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 -- 172.21.15.59:0/1592397333 shutdown_connections 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 -- 172.21.15.59:0/1592397333 >> 172.21.15.59:0/1592397333 conn(0x7fe4d8067ea0 msgr2=0x7fe4d806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 -- 172.21.15.59:0/1592397333 shutdown_connections 2024-04-24T06:28:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.373+0000 7fe4de600640 1 -- 172.21.15.59:0/1592397333 wait complete. 2024-04-24T06:28:45.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.375+0000 7fe4de600640 1 Processor -- start 2024-04-24T06:28:45.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.375+0000 7fe4de600640 1 -- start start 2024-04-24T06:28:45.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4bc004800 0x7fe4d806f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d81f5570 0x7fe4d815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80b4010 con 0x7fe4d81437a0 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81462c0 con 0x7fe4d81f5570 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4de600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d815ed20 con 0x7fe4bc004800 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58590/0 (socket says 172.21.15.59:58590) 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 -- 172.21.15.59:0/2380209237 learned_addr learned my addr 172.21.15.59:0/2380209237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7400640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4bc004800 0x7fe4d806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4cfe00640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d81f5570 0x7fe4d815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 -- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4bc004800 msgr2=0x7fe4d806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4bc004800 0x7fe4d806f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 -- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d81f5570 msgr2=0x7fe4d815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d81f5570 0x7fe4d815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.376+0000 7fe4d7e00640 1 -- 172.21.15.59:0/2380209237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d815de90 con 0x7fe4d81437a0 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4d7e00640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80702d0 tx=0x7fe4d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000eb30 con 0x7fe4d81437a0 2024-04-24T06:28:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d000ecf0 con 0x7fe4d81437a0 2024-04-24T06:28:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000eea0 con 0x7fe4d81437a0 2024-04-24T06:28:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d815e110 con 0x7fe4d81437a0 2024-04-24T06:28:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.377+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d815e6d0 con 0x7fe4d81437a0 2024-04-24T06:28:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.378+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d8068a80 con 0x7fe4d81437a0 2024-04-24T06:28:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.379+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4d001b020 con 0x7fe4d81437a0 2024-04-24T06:28:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.380+0000 7fe4cf400640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.380+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fe4d008b900 con 0x7fe4d81437a0 2024-04-24T06:28:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.380+0000 7fe4d7400640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.381+0000 7fe4d7400640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 0x7fe4b406ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80017e0 tx=0x7fe4c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.385+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4d815de90 con 0x7fe4d81437a0 2024-04-24T06:28:45.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.595+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d8160030 con 0x7fe4d81437a0 2024-04-24T06:28:45.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.598+0000 7fe4cf400640 1 -- 172.21.15.59:0/2380209237 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16534 (secure 0 0 0) 0x7fe4d008b090 con 0x7fe4d81437a0 2024-04-24T06:28:45.600 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:45.600 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:44.183651+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 msgr2=0x7fe4b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 0x7fe4b406ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80017e0 tx=0x7fe4c8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 msgr2=0x7fe4d8143c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80702d0 tx=0x7fe4d000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 shutdown_connections 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4d81437a0 0x7fe4d8143c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d81f5570 0x7fe4d815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 --2- 172.21.15.59:0/2380209237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4bc004800 0x7fe4d806f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 >> 172.21.15.59:0/2380209237 conn(0x7fe4d8067ea0 msgr2=0x7fe4d8140c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 shutdown_connections 2024-04-24T06:28:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.601+0000 7fe4de600640 1 -- 172.21.15.59:0/2380209237 wait complete. 2024-04-24T06:28:45.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:28:45.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.839+0000 7f46c9a00640 1 Processor -- start 2024-04-24T06:28:45.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.840+0000 7f46c9a00640 1 -- start start 2024-04-24T06:28:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.841+0000 7f46c9a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46c4145e10 0x7f46c406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.841+0000 7f46c9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c40d0cc0 0x7f46c40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.841+0000 7f46c9a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c40cfc20 con 0x7f46c4071d70 2024-04-24T06:28:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.841+0000 7f46c9a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f46c40cff00 con 0x7f46c4145e10 2024-04-24T06:28:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.841+0000 7f46c9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46c40d01e0 con 0x7f46c40d0cc0 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46c4145e10 0x7f46c406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46c4145e10 0x7f46c406f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:48294/0 (socket says 172.21.15.59:48294) 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 -- 172.21.15.59:0/943118976 learned_addr learned my addr 172.21.15.59:0/943118976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2000640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c40d0cc0 0x7f46c40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 -- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c40d0cc0 msgr2=0x7f46c40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c40d0cc0 0x7f46c40cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 -- 172.21.15.59:0/943118976 >> v1:172.21.15.66:6789/0 conn(0x7f46c4071d70 legacy=0x7f46c415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.842+0000 7f46c2a00640 1 -- 172.21.15.59:0/943118976 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c40d04c0 con 0x7f46c4145e10 2024-04-24T06:28:45.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.843+0000 7f46c2a00640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4145e10 0x7f46c406f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46b40098a0 tx=0x7f46b402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c666cba1f4042ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.844+0000 7f46c1600640 1 -- 172.21.15.59:0/943118976 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4037070 con 0x7f46c4145e10 2024-04-24T06:28:45.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.844+0000 7f46c1600640 1 -- 172.21.15.59:0/943118976 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b4032440 con 0x7f46c4145e10 2024-04-24T06:28:45.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.844+0000 7f46c1600640 1 -- 172.21.15.59:0/943118976 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4032810 con 0x7f46c4145e10 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 -- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4145e10 msgr2=0x7f46c406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4145e10 0x7f46c406f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46b40098a0 tx=0x7f46b402a250 comp rx=0 tx=0).stop 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 -- 172.21.15.59:0/943118976 shutdown_connections 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c40d0cc0 0x7f46c40cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 --2- 172.21.15.59:0/943118976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4145e10 0x7f46c406f130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 -- 172.21.15.59:0/943118976 >> 172.21.15.59:0/943118976 conn(0x7f46c4067d90 msgr2=0x7f46c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 -- 172.21.15.59:0/943118976 shutdown_connections 2024-04-24T06:28:45.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.845+0000 7f46c9a00640 1 -- 172.21.15.59:0/943118976 wait complete. 2024-04-24T06:28:45.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.847+0000 7f46c9a00640 1 Processor -- start 2024-04-24T06:28:45.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.847+0000 7f46c9a00640 1 -- start start 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4071d70 0x7f46c415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c4145e10 0x7f46c40b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40cfa00 con 0x7f46c40d0cc0 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40cf880 con 0x7f46c4071d70 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46c41461f0 con 0x7f46c4145e10 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58602/0 (socket says 172.21.15.59:58602) 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.848+0000 7f46c2000640 1 -- 172.21.15.59:0/4140953880 learned_addr learned my addr 172.21.15.59:0/4140953880 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4071d70 0x7f46c415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c3400640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c4145e10 0x7f46c40b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 -- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c4145e10 msgr2=0x7f46c40b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c4145e10 0x7f46c40b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 -- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4071d70 msgr2=0x7f46c415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:45.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4071d70 0x7f46c415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:45.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 -- 172.21.15.59:0/4140953880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c40b0820 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46c2000640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9e0 tx=0x7f46ac00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac017070 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.849+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac00c9d0 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.850+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c40b0b00 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.850+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00cdc0 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.850+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c40b10c0 con 0x7f46c40d0cc0 2024-04-24T06:28:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.851+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c4068a80 con 0x7f46c40d0cc0 2024-04-24T06:28:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.852+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46ac01a070 con 0x7f46c40d0cc0 2024-04-24T06:28:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.853+0000 7f46baa00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 0x7f46a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.853+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46ac08beb0 con 0x7f46c40d0cc0 2024-04-24T06:28:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.853+0000 7f46c2a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 0x7f46a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.854+0000 7f46c2a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 0x7f46a006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:45.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:45.857+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46ac05a2e0 con 0x7f46c40d0cc0 2024-04-24T06:28:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.109+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f46c40d1190 con 0x7f46c40d0cc0 2024-04-24T06:28:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.738+0000 7f46baa00640 1 -- 172.21.15.59:0/4140953880 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f46ac059be0 con 0x7f46c40d0cc0 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 msgr2=0x7f46a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 0x7f46a006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b402e000 comp rx=0 tx=0).stop 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 msgr2=0x7f46c40b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00e9e0 tx=0x7f46ac00eea0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 shutdown_connections 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a006c8f0 0x7f46a006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46c4145e10 0x7f46c40b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40d0cc0 0x7f46c40b4b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 --2- 172.21.15.59:0/4140953880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46c4071d70 0x7f46c415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.742+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 >> 172.21.15.59:0/4140953880 conn(0x7f46c4067d90 msgr2=0x7f46c415e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.743+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 shutdown_connections 2024-04-24T06:28:46.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.743+0000 7f46c9a00640 1 -- 172.21.15.59:0/4140953880 wait complete. 2024-04-24T06:28:46.766 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:28:46.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:28:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.981+0000 7fe54f200640 1 Processor -- start 2024-04-24T06:28:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.982+0000 7fe54f200640 1 -- start start 2024-04-24T06:28:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5480b1c00 0x7fe54815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 0x7fe548070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe548070300 con 0x7fe54809bd40 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe548070910 con 0x7fe5480b1c00 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe54815d800 con 0x7fe54809b110 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe547e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5480b1c00 0x7fe54815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe547e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5480b1c00 0x7fe54815cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58604/0 (socket says 172.21.15.59:58604) 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe547e00640 1 -- 172.21.15.59:0/1984446899 learned_addr learned my addr 172.21.15.59:0/1984446899 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.984+0000 7fe54d200640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 0x7fe548070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.985+0000 7fe547e00640 1 -- 172.21.15.59:0/1984446899 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 msgr2=0x7fe548070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.985+0000 7fe547e00640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 0x7fe548070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.985+0000 7fe547e00640 1 -- 172.21.15.59:0/1984446899 >> v1:172.21.15.66:6789/0 conn(0x7fe54809bd40 legacy=0x7fe54809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:46.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.985+0000 7fe547e00640 1 -- 172.21.15.59:0/1984446899 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54815dae0 con 0x7fe5480b1c00 2024-04-24T06:28:46.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.985+0000 7fe54d200640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 0x7fe548070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.986+0000 7fe547e00640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480b1c00 0x7fe54815cdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe548073060 tx=0x7fe53802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ae88a91265bca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.986+0000 7fe546a00640 1 -- 172.21.15.59:0/1984446899 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538036070 con 0x7fe5480b1c00 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.986+0000 7fe546a00640 1 -- 172.21.15.59:0/1984446899 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538031600 con 0x7fe5480b1c00 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.986+0000 7fe546a00640 1 -- 172.21.15.59:0/1984446899 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380319d0 con 0x7fe5480b1c00 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 -- 172.21.15.59:0/1984446899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480b1c00 msgr2=0x7fe54815cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480b1c00 0x7fe54815cdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe548073060 tx=0x7fe53802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 -- 172.21.15.59:0/1984446899 shutdown_connections 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe54809b110 0x7fe548070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 --2- 172.21.15.59:0/1984446899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480b1c00 0x7fe54815cdb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 -- 172.21.15.59:0/1984446899 >> 172.21.15.59:0/1984446899 conn(0x7fe5480b40d0 msgr2=0x7fe5480b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 -- 172.21.15.59:0/1984446899 shutdown_connections 2024-04-24T06:28:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.987+0000 7fe54f200640 1 -- 172.21.15.59:0/1984446899 wait complete. 2024-04-24T06:28:46.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.990+0000 7fe54f200640 1 Processor -- start 2024-04-24T06:28:46.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.990+0000 7fe54f200640 1 -- start start 2024-04-24T06:28:46.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe54809b110 0x7fe54814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 0x7fe5480c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe54815d470 con 0x7fe5480c6600 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe54815d2f0 con 0x7fe54809b110 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.991+0000 7fe54f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe5480712e0 con 0x7fe5480b1c00 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58618/0 (socket says 172.21.15.59:58618) 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 -- 172.21.15.59:0/2595432226 learned_addr learned my addr 172.21.15.59:0/2595432226 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe547400640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 0x7fe5480c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe547e00640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe54809b110 0x7fe54814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 -- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 msgr2=0x7fe5480c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 0x7fe5480c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 -- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe54809b110 msgr2=0x7fe54814f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe54809b110 0x7fe54814f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 -- 172.21.15.59:0/2595432226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5480c9750 con 0x7fe5480c6600 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe547400640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 0x7fe5480c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.992+0000 7fe54d200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c7f0 tx=0x7fe54000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.993+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540015070 con 0x7fe5480c6600 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.993+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5400077c0 con 0x7fe5480c6600 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.993+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540007bf0 con 0x7fe5480c6600 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.993+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5480c9a30 con 0x7fe5480c6600 2024-04-24T06:28:46.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.993+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5480c9ff0 con 0x7fe5480c6600 2024-04-24T06:28:46.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.994+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5480ca1f0 con 0x7fe5480c6600 2024-04-24T06:28:46.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.995+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe540018070 con 0x7fe5480c6600 2024-04-24T06:28:46.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.996+0000 7fe53fe00640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 0x7fe52406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:46.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.996+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe54008bcc0 con 0x7fe5480c6600 2024-04-24T06:28:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.996+0000 7fe547e00640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 0x7fe52406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:46.997+0000 7fe547e00640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 0x7fe52406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe538033000 tx=0x7fe538009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.001+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5480ca1f0 con 0x7fe5480c6600 2024-04-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.251+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe548069190 con 0x7fe5480c6600 2024-04-24T06:28:47.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.748+0000 7fe53fe00640 1 -- 172.21.15.59:0/2595432226 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fe540092080 con 0x7fe5480c6600 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 msgr2=0x7fe52406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 0x7fe52406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe538033000 tx=0x7fe538009000 comp rx=0 tx=0).stop 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 msgr2=0x7fe5480c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe54000c7f0 tx=0x7fe54000ccb0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 shutdown_connections 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe52406c920 0x7fe52406ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5480c6600 0x7fe5480c8350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5480b1c00 0x7fe5480c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 --2- 172.21.15.59:0/2595432226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe54809b110 0x7fe54814f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 >> 172.21.15.59:0/2595432226 conn(0x7fe5480b40d0 msgr2=0x7fe548074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.752+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 shutdown_connections 2024-04-24T06:28:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.753+0000 7fe54f200640 1 -- 172.21.15.59:0/2595432226 wait complete. 2024-04-24T06:28:47.776 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:28:47.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:28:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.989+0000 7f9f9ee00640 1 Processor -- start 2024-04-24T06:28:47.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.989+0000 7f9f9ee00640 1 -- start start 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 0x7f9f980b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f980cf030 con 0x7f9f9815e7c0 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f980b6a40 con 0x7f9f980d0d90 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f980b7000 con 0x7f9f980b5ff0 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9f9815e7c0 0x7f9f980d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49306/0 (socket says 172.21.15.59:49306) 2024-04-24T06:28:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9da00640 1 -- 172.21.15.59:0/2056483100 learned_addr learned my addr 172.21.15.59:0/2056483100 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.991+0000 7f9f9d000640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 0x7f9f980b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142107857 0 0) 0x7f9f8c001490 con 0x7f9f9815e7c0 2024-04-24T06:28:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f80003520 con 0x7f9f9815e7c0 2024-04-24T06:28:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 31024120 0 0) 0x7f9f80003520 con 0x7f9f9815e7c0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 msgr2=0x7f9f980b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 0x7f9f980b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 >> v1:172.21.15.66:6789/0 conn(0x7f9f980d0d90 legacy=0x7f9f980cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f980b6d20 con 0x7f9f9815e7c0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9f8c001bc0 con 0x7f9f9815e7c0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 >> v1:172.21.15.59:6789/0 conn(0x7f9f9815e7c0 legacy=0x7f9f980d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f800039e0 0x7f9f80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 0x7f9f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f980b7000 con 0x7f9f80004800 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8000baa0 con 0x7f9f800039e0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.992+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8000bca0 con 0x7f9f980b5ff0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9da00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f800039e0 0x7f9f80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 -- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 msgr2=0x7f9f800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f980b5ff0 0x7f9f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 -- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f800039e0 msgr2=0x7f9f80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f800039e0 0x7f9f80003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:47.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.993+0000 7f9f9d000640 1 -- 172.21.15.59:0/2056483100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80003520 con 0x7f9f80004800 2024-04-24T06:28:47.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.994+0000 7f9f9d000640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f80004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009a90 tx=0x7f9f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d03e608021972 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:47.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.994+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84036070 con 0x7f9f80004800 2024-04-24T06:28:47.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.994+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84031400 con 0x7f9f80004800 2024-04-24T06:28:47.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.994+0000 7f9f97e00640 1 -- 172.21.15.59:0/2056483100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84009690 con 0x7f9f80004800 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 -- 172.21.15.59:0/2056483100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 msgr2=0x7f9f80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/2056483100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f80004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009a90 tx=0x7f9f80005830 comp rx=0 tx=0).stop 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9e400640 1 -- 172.21.15.59:0/2056483100 reap_dead start 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 -- 172.21.15.59:0/2056483100 shutdown_connections 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 -- 172.21.15.59:0/2056483100 >> 172.21.15.59:0/2056483100 conn(0x7f9f9809d920 msgr2=0x7f9f9808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 -- 172.21.15.59:0/2056483100 shutdown_connections 2024-04-24T06:28:47.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.995+0000 7f9f9ee00640 1 -- 172.21.15.59:0/2056483100 wait complete. 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.997+0000 7f9f9ee00640 1 Processor -- start 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.997+0000 7f9f9ee00640 1 -- start start 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 0x7f9f981566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f98156bf0 0x7f9f980b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f980d2800 con 0x7f9f80004800 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f980cf620 con 0x7f9f98156bf0 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f980d1a80 con 0x7f9f9815b050 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58634/0 (socket says 172.21.15.59:58634) 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.998+0000 7f9f9da00640 1 -- 172.21.15.59:0/3979137078 learned_addr learned my addr 172.21.15.59:0/3979137078 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9d000640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 0x7f9f981566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 -- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 msgr2=0x7f9f981566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 0x7f9f981566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 -- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f98156bf0 msgr2=0x7f9f980b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9e400640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f98156bf0 0x7f9f980b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f98156bf0 0x7f9f980b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 -- 172.21.15.59:0/3979137078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981558f0 con 0x7f9f80004800 2024-04-24T06:28:48.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9d000640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 0x7f9f981566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9da00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c001c30 tx=0x7f9f8c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c019070 con 0x7f9f80004800 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c004d60 con 0x7f9f80004800 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0072d0 con 0x7f9f80004800 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:47.999+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f98155bd0 con 0x7f9f80004800 2024-04-24T06:28:48.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.000+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f98156190 con 0x7f9f80004800 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.001+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98156390 con 0x7f9f80004800 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.002+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f8c01c070 con 0x7f9f80004800 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.002+0000 7f9f95600640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.003+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9f8c08bce0 con 0x7f9f80004800 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.003+0000 7f9f9d000640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.004+0000 7f9f9d000640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f84033000 tx=0x7f9f84008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.007+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f98156390 con 0x7f9f80004800 2024-04-24T06:28:48.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.252+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f980b1c90 con 0x7f9f80004800 2024-04-24T06:28:48.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.254+0000 7f9f95600640 1 -- 172.21.15.59:0/3979137078 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f9f980cf030 con 0x7f9f80004800 2024-04-24T06:28:48.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:48.255 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:28:48.255 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 msgr2=0x7f9f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 0x7f9f7806ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f84033000 tx=0x7f9f84008030 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 msgr2=0x7f9f9815ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c001c30 tx=0x7f9f8c00b4a0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 shutdown_connections 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f7806c920 0x7f9f7806ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f98156bf0 0x7f9f980b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f9815b050 0x7f9f981566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 --2- 172.21.15.59:0/3979137078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f80004800 0x7f9f9815ab10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.256+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 >> 172.21.15.59:0/3979137078 conn(0x7f9f9809d920 msgr2=0x7f9f9806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:48.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.257+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 shutdown_connections 2024-04-24T06:28:48.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.257+0000 7f9f9ee00640 1 -- 172.21.15.59:0/3979137078 wait complete. 2024-04-24T06:28:48.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:48.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.484+0000 7f3fade00640 1 Processor -- start 2024-04-24T06:28:48.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.484+0000 7f3fade00640 1 -- start start 2024-04-24T06:28:48.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fade00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa815e7c0 0x7f3fa80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fade00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fa80d2630 0x7f3fa80b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fade00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa80cf030 con 0x7f3fa80b6530 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fade00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa80b7000 con 0x7f3fa80d2630 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fade00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa80cf620 con 0x7f3fa815e7c0 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.485+0000 7f3fa7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fa80d2630 0x7f3fa80b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fa80d2630 0x7f3fa80b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58646/0 (socket says 172.21.15.59:58646) 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 -- 172.21.15.59:0/1328349726 learned_addr learned my addr 172.21.15.59:0/1328349726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3faca00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa815e7c0 0x7f3fa80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 -- 172.21.15.59:0/1328349726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa815e7c0 msgr2=0x7f3fa80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa815e7c0 0x7f3fa80d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 -- 172.21.15.59:0/1328349726 >> v1:172.21.15.66:6789/0 conn(0x7f3fa80b6530 legacy=0x7f3fa80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:48.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.486+0000 7f3fa7e00640 1 -- 172.21.15.59:0/1328349726 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa80b6a40 con 0x7f3fa80d2630 2024-04-24T06:28:48.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.487+0000 7f3fa7e00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b5ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a57bed8640a192 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.487+0000 7f3fa7400640 1 -- 172.21.15.59:0/1328349726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90037070 con 0x7f3fa80d2630 2024-04-24T06:28:48.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fa7400640 1 -- 172.21.15.59:0/1328349726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90032440 con 0x7f3fa80d2630 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fa7400640 1 -- 172.21.15.59:0/1328349726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90032810 con 0x7f3fa80d2630 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 -- 172.21.15.59:0/1328349726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 msgr2=0x7f3fa80b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b5ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f9002a210 comp rx=0 tx=0).stop 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 -- 172.21.15.59:0/1328349726 shutdown_connections 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b5ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 --2- 172.21.15.59:0/1328349726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa815e7c0 0x7f3fa80d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 -- 172.21.15.59:0/1328349726 >> 172.21.15.59:0/1328349726 conn(0x7f3fa809d920 msgr2=0x7f3fa808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 -- 172.21.15.59:0/1328349726 shutdown_connections 2024-04-24T06:28:48.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.488+0000 7f3fade00640 1 -- 172.21.15.59:0/1328349726 wait complete. 2024-04-24T06:28:48.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.490+0000 7f3fade00640 1 Processor -- start 2024-04-24T06:28:48.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.490+0000 7f3fade00640 1 -- start start 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa80b6530 0x7f3fa814fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fa815e7c0 0x7f3fa80cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa80d1a80 con 0x7f3fa80d2630 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa80cf2b0 con 0x7f3fa815e7c0 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fade00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa80d1c00 con 0x7f3fa80b6530 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.491+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58648/0 (socket says 172.21.15.59:58648) 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 -- 172.21.15.59:0/2937401735 learned_addr learned my addr 172.21.15.59:0/2937401735 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3faca00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa80b6530 0x7f3fa814fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fad400640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fa815e7c0 0x7f3fa80cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 -- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa80b6530 msgr2=0x7f3fa814fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa80b6530 0x7f3fa814fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 -- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fa815e7c0 msgr2=0x7f3fa80cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fa815e7c0 0x7f3fa80cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 -- 172.21.15.59:0/2937401735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa814ff40 con 0x7f3fa80d2630 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.492+0000 7f3fa7e00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.493+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90041020 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.493+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f900306c0 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.493+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90037040 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.493+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa80b17e0 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.493+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa80b1da0 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.494+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa804fd90 con 0x7f3fa80d2630 2024-04-24T06:28:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.495+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f90044020 con 0x7f3fa80d2630 2024-04-24T06:28:48.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.496+0000 7f3fa4c00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.496+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f900ac3f0 con 0x7f3fa80d2630 2024-04-24T06:28:48.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.496+0000 7f3faca00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.497+0000 7f3faca00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 0x7f3f8406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f980099c0 tx=0x7f3f98003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.501+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f9004b030 con 0x7f3fa80d2630 2024-04-24T06:28:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.707+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fa815cd90 con 0x7f3fa80d2630 2024-04-24T06:28:48.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.710+0000 7f3fa4c00640 1 -- 172.21.15.59:0/2937401735 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f3fa814ff40 con 0x7f3fa80d2630 2024-04-24T06:28:48.712 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:48.712 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:48.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 msgr2=0x7f3f8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 0x7f3f8406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f980099c0 tx=0x7f3f98003990 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 msgr2=0x7f3fa80b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009870 tx=0x7f3f900046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 shutdown_connections 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3f8406c8f0 0x7f3f8406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3fa815e7c0 0x7f3fa80cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fa80d2630 0x7f3fa80b25a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 --2- 172.21.15.59:0/2937401735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3fa80b6530 0x7f3fa814fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.713+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 >> 172.21.15.59:0/2937401735 conn(0x7f3fa809d920 msgr2=0x7f3fa80b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.714+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 shutdown_connections 2024-04-24T06:28:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.714+0000 7f3fade00640 1 -- 172.21.15.59:0/2937401735 wait complete. 2024-04-24T06:28:48.734 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:28:48.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:28:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.939+0000 7f2be7c00640 1 Processor -- start 2024-04-24T06:28:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.940+0000 7f2be7c00640 1 -- start start 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.941+0000 7f2be7c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.941+0000 7f2be7c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2be0163770 0x7f2be00d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.941+0000 7f2be7c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00d01c0 con 0x7f2be0160180 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.941+0000 7f2be7c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2be015cf00 con 0x7f2be0163770 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.941+0000 7f2be7c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2be015f6a0 con 0x7f2be015cb20 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2bd7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2be0163770 0x7f2be00d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2be5c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2be0160180 0x7f2be00b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59870/0 (socket says 172.21.15.59:59870) 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2be5c00640 1 -- 172.21.15.59:0/351595971 learned_addr learned my addr 172.21.15.59:0/351595971 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2be5200640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2bd7e00640 1 -- 172.21.15.59:0/351595971 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 msgr2=0x7f2be00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2bd7e00640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be00cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2bd7e00640 1 -- 172.21.15.59:0/351595971 >> v1:172.21.15.59:6789/0 conn(0x7f2be0160180 legacy=0x7f2be00b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.942+0000 7f2bd7e00640 1 -- 172.21.15.59:0/351595971 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be015f980 con 0x7f2be0163770 2024-04-24T06:28:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.943+0000 7f2bd7e00640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0163770 0x7f2be00d1240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80097a0 tx=0x7f2bc802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2719b56675428e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.943+0000 7f2bdfe00640 1 -- 172.21.15.59:0/351595971 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037070 con 0x7f2be0163770 2024-04-24T06:28:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.943+0000 7f2bdfe00640 1 -- 172.21.15.59:0/351595971 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc802ae40 con 0x7f2be0163770 2024-04-24T06:28:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.943+0000 7f2bdfe00640 1 -- 172.21.15.59:0/351595971 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8032660 con 0x7f2be0163770 2024-04-24T06:28:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 -- 172.21.15.59:0/351595971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0163770 msgr2=0x7f2be00d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0163770 0x7f2be00d1240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80097a0 tx=0x7f2bc802a0f0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 -- 172.21.15.59:0/351595971 shutdown_connections 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0163770 0x7f2be00d1240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 --2- 172.21.15.59:0/351595971 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 -- 172.21.15.59:0/351595971 >> 172.21.15.59:0/351595971 conn(0x7f2be006cc70 msgr2=0x7f2be006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 -- 172.21.15.59:0/351595971 shutdown_connections 2024-04-24T06:28:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.944+0000 7f2be7c00640 1 -- 172.21.15.59:0/351595971 wait complete. 2024-04-24T06:28:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.946+0000 7f2be7c00640 1 Processor -- start 2024-04-24T06:28:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.947+0000 7f2be7c00640 1 -- start start 2024-04-24T06:28:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2be0163770 0x7f2be01e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00d1780 con 0x7f2be0160180 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00d1900 con 0x7f2be0163770 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2be7c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2be00d1a80 con 0x7f2be015cb20 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37758/0 (socket says 172.21.15.59:37758) 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.948+0000 7f2bd7e00640 1 -- 172.21.15.59:0/417310274 learned_addr learned my addr 172.21.15.59:0/417310274 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2be5200640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be014aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 -- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 msgr2=0x7f2be014aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be014aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 -- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2be0163770 msgr2=0x7f2be01e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2be0163770 0x7f2be01e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 -- 172.21.15.59:0/417310274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be00b9ac0 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bd7e00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8035040 tx=0x7f2bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8041070 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc803c670 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037040 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be00c9200 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.949+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be00c97c0 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.951+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0068a80 con 0x7f2be0160180 2024-04-24T06:28:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.952+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bc803a070 con 0x7f2be0160180 2024-04-24T06:28:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.952+0000 7f2bdd600640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.953+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2bc80abeb0 con 0x7f2be0160180 2024-04-24T06:28:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.953+0000 7f2be5200640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.953+0000 7f2be5200640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0006fd0 tx=0x7f2bd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:48.957+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bc807a2e0 con 0x7f2be0160180 2024-04-24T06:28:49.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.157+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2be00d1560 con 0x7f2bbc06c8f0 2024-04-24T06:28:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.178+0000 7f2bdd600640 1 -- 172.21.15.59:0/417310274 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7f2be00d1560 con 0x7f2bbc06c8f0 2024-04-24T06:28:49.183 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:49.184 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 msgr2=0x7f2bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0006fd0 tx=0x7f2bd0008040 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 msgr2=0x7f2be01e90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8035040 tx=0x7f2bc80046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 shutdown_connections 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2bbc06c8f0 0x7f2bbc06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2be0163770 0x7f2be01e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2be0160180 0x7f2be01e90b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 --2- 172.21.15.59:0/417310274 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2be015cb20 0x7f2be014aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 >> 172.21.15.59:0/417310274 conn(0x7f2be006cc70 msgr2=0x7f2be015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 shutdown_connections 2024-04-24T06:28:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.183+0000 7f2be7c00640 1 -- 172.21.15.59:0/417310274 wait complete. 2024-04-24T06:28:49.203 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-24T06:28:47.188046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218600,"kb_used_data":2436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756120,"statfs":{"total":773094113280,"available":772870266880,"internally_reserved":0,"allocated":2494464,"data_stored":1575957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.500020"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.696542+0000","last_change":"2024-04-24T06:28:42.696542+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.696542+0000","last_undegraded":"2024-04-24T06:28:42.696542+0000","last_fullsized":"2024-04-24T06:28:42.696542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.701644+0000","last_change":"2024-04-24T06:28:42.701644+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.701644+0000","last_undegraded":"2024-04-24T06:28:42.701644+0000","last_fullsized":"2024-04-24T06:28:42.701644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.696465+0000","last_change":"2024-04-24T06:28:42.696465+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.696465+0000","last_undegraded":"2024-04-24T06:28:42.696465+0000","last_fullsized":"2024-04-24T06:28:42.696465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.707159+0000","last_change":"2024-04-24T06:28:42.707159+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.707159+0000","last_undegraded":"2024-04-24T06:28:42.707159+0000","last_fullsized":"2024-04-24T06:28:42.707159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.707046+0000","last_change":"2024-04-24T06:28:42.707046+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.707046+0000","last_undegraded":"2024-04-24T06:28:42.707046+0000","last_fullsized":"2024-04-24T06:28:42.707046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.711075+0000","last_change":"2024-04-24T06:28:42.711075+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.711075+0000","last_undegraded":"2024-04-24T06:28:42.711075+0000","last_fullsized":"2024-04-24T06:28:42.711075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.693126+0000","last_change":"2024-04-24T06:28:42.693126+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.693126+0000","last_undegraded":"2024-04-24T06:28:42.693126+0000","last_fullsized":"2024-04-24T06:28:42.693126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.702064+0000","last_change":"2024-04-24T06:28:42.702064+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.702064+0000","last_undegraded":"2024-04-24T06:28:42.702064+0000","last_fullsized":"2024-04-24T06:28:42.702064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.695286+0000","last_change":"2024-04-24T06:28:42.695286+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.695286+0000","last_undegraded":"2024-04-24T06:28:42.695286+0000","last_fullsized":"2024-04-24T06:28:42.695286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.711300+0000","last_change":"2024-04-24T06:28:42.711300+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.711300+0000","last_undegraded":"2024-04-24T06:28:42.711300+0000","last_fullsized":"2024-04-24T06:28:42.711300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.701481+0000","last_change":"2024-04-24T06:28:42.701481+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.701481+0000","last_undegraded":"2024-04-24T06:28:42.701481+0000","last_fullsized":"2024-04-24T06:28:42.701481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.716362+0000","last_change":"2024-04-24T06:28:42.716362+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.716362+0000","last_undegraded":"2024-04-24T06:28:42.716362+0000","last_fullsized":"2024-04-24T06:28:42.716362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.703878+0000","last_change":"2024-04-24T06:28:42.703878+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.703878+0000","last_undegraded":"2024-04-24T06:28:42.703878+0000","last_fullsized":"2024-04-24T06:28:42.703878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.716233+0000","last_change":"2024-04-24T06:28:42.716233+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.716233+0000","last_undegraded":"2024-04-24T06:28:42.716233+0000","last_fullsized":"2024-04-24T06:28:42.716233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.692264+0000","last_change":"2024-04-24T06:28:42.692264+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.692264+0000","last_undegraded":"2024-04-24T06:28:42.692264+0000","last_fullsized":"2024-04-24T06:28:42.692264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.716711+0000","last_change":"2024-04-24T06:28:42.716711+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.716711+0000","last_undegraded":"2024-04-24T06:28:42.716711+0000","last_fullsized":"2024-04-24T06:28:42.716711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T06:28:42.701849+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:28:42.701849+0000","last_peered":"2024-04-24T06:28:42.701849+0000","last_clean":"2024-04-24T06:28:42.701849+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:28:42.701849+0000","last_undegraded":"2024-04-24T06:28:42.701849+0000","last_fullsized":"2024-04-24T06:28:42.701849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.709378+0000","last_change":"2024-04-24T06:28:42.709378+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.709378+0000","last_undegraded":"2024-04-24T06:28:42.709378+0000","last_fullsized":"2024-04-24T06:28:42.709378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.694559+0000","last_change":"2024-04-24T06:28:42.694559+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.694559+0000","last_undegraded":"2024-04-24T06:28:42.694559+0000","last_fullsized":"2024-04-24T06:28:42.694559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.716428+0000","last_change":"2024-04-24T06:28:42.716428+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.716428+0000","last_undegraded":"2024-04-24T06:28:42.716428+0000","last_fullsized":"2024-04-24T06:28:42.716428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.707149+0000","last_change":"2024-04-24T06:28:42.707149+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.707149+0000","last_undegraded":"2024-04-24T06:28:42.707149+0000","last_fullsized":"2024-04-24T06:28:42.707149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.690778+0000","last_change":"2024-04-24T06:28:42.690778+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.690778+0000","last_undegraded":"2024-04-24T06:28:42.690778+0000","last_fullsized":"2024-04-24T06:28:42.690778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.695190+0000","last_change":"2024-04-24T06:28:42.695190+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.695190+0000","last_undegraded":"2024-04-24T06:28:42.695190+0000","last_fullsized":"2024-04-24T06:28:42.695190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.690779+0000","last_change":"2024-04-24T06:28:42.690779+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.690779+0000","last_undegraded":"2024-04-24T06:28:42.690779+0000","last_fullsized":"2024-04-24T06:28:42.690779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.704557+0000","last_change":"2024-04-24T06:28:42.704557+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.704557+0000","last_undegraded":"2024-04-24T06:28:42.704557+0000","last_fullsized":"2024-04-24T06:28:42.704557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.697446+0000","last_change":"2024-04-24T06:28:42.697446+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.697446+0000","last_undegraded":"2024-04-24T06:28:42.697446+0000","last_fullsized":"2024-04-24T06:28:42.697446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.691742+0000","last_change":"2024-04-24T06:28:42.691742+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.691742+0000","last_undegraded":"2024-04-24T06:28:42.691742+0000","last_fullsized":"2024-04-24T06:28:42.691742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.691728+0000","last_change":"2024-04-24T06:28:42.691728+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.691728+0000","last_undegraded":"2024-04-24T06:28:42.691728+0000","last_fullsized":"2024-04-24T06:28:42.691728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.695280+0000","last_change":"2024-04-24T06:28:42.695280+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.695280+0000","last_undegraded":"2024-04-24T06:28:42.695280+0000","last_fullsized":"2024-04-24T06:28:42.695280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.696800+0000","last_change":"2024-04-24T06:28:42.696800+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.696800+0000","last_undegraded":"2024-04-24T06:28:42.696800+0000","last_fullsized":"2024-04-24T06:28:42.696800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.698066+0000","last_change":"2024-04-24T06:28:42.698066+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.698066+0000","last_undegraded":"2024-04-24T06:28:42.698066+0000","last_fullsized":"2024-04-24T06:28:42.698066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.698513+0000","last_change":"2024-04-24T06:28:42.698513+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.698513+0000","last_undegraded":"2024-04-24T06:28:42.698513+0000","last_fullsized":"2024-04-24T06:28:42.698513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:28:43.684851+0000","last_change":"2024-04-24T06:28:43.684851+0000","last_active":"2024-04-24T06:28:43.684851+0000","last_peered":"2024-04-24T06:28:43.684851+0000","last_clean":"2024-04-24T06:28:43.684851+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:43.684851+0000","last_undegraded":"2024-04-24T06:28:43.684851+0000","last_fullsized":"2024-04-24T06:28:43.684851+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-24T06:28:42.695255+0000","last_change":"2024-04-24T06:28:42.695255+0000","last_active":"2024-04-24T06:28:42.667285+0000","last_peered":"2024-04-24T06:28:42.667285+0000","last_clean":"2024-04-24T06:28:42.667285+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:28:42.695255+0000","last_undegraded":"2024-04-24T06:28:42.695255+0000","last_fullsized":"2024-04-24T06:28:42.695255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:28:49.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:28:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.460+0000 7f5b03200640 1 Processor -- start 2024-04-24T06:28:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.460+0000 7f5b03200640 1 -- start start 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5afc15c3d0 0x7f5afc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 0x7f5afc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5afc0b4690 0x7f5afc06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5afc160210 con 0x7f5afc15c3d0 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5afc1604f0 con 0x7f5afc0b4690 2024-04-24T06:28:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b03200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc1607d0 con 0x7f5afc0cd590 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b01e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5afc15c3d0 0x7f5afc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b01e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5afc15c3d0 0x7f5afc0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37760/0 (socket says 172.21.15.59:37760) 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b01e00640 1 -- 172.21.15.59:0/2566784618 learned_addr learned my addr 172.21.15.59:0/2566784618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b01400640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 0x7f5afc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.462+0000 7f5b02800640 1 --2- 172.21.15.59:0/2566784618 >> v2:172.21.15.66:3300/0 conn(0x7f5afc0b4690 0x7f5afc06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01e00640 1 -- 172.21.15.59:0/2566784618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 msgr2=0x7f5afc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01e00640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 0x7f5afc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01e00640 1 -- 172.21.15.59:0/2566784618 >> v2:172.21.15.66:3300/0 conn(0x7f5afc0b4690 msgr2=0x7f5afc06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01e00640 1 --2- 172.21.15.59:0/2566784618 >> v2:172.21.15.66:3300/0 conn(0x7f5afc0b4690 0x7f5afc06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01e00640 1 -- 172.21.15.59:0/2566784618 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc160ab0 con 0x7f5afc15c3d0 2024-04-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.463+0000 7f5b01400640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 0x7f5afc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.464+0000 7f5b01e00640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc15c3d0 0x7f5afc0cd050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5afc161170 tx=0x7f5af002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ca7c260a120c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.465+0000 7f5b00a00640 1 -- 172.21.15.59:0/2566784618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0037070 con 0x7f5afc15c3d0 2024-04-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.465+0000 7f5b00a00640 1 -- 172.21.15.59:0/2566784618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af0032440 con 0x7f5afc15c3d0 2024-04-24T06:28:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.465+0000 7f5b00a00640 1 -- 172.21.15.59:0/2566784618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0032810 con 0x7f5afc15c3d0 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 -- 172.21.15.59:0/2566784618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc15c3d0 msgr2=0x7f5afc0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc15c3d0 0x7f5afc0cd050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5afc161170 tx=0x7f5af002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 -- 172.21.15.59:0/2566784618 shutdown_connections 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 --2- 172.21.15.59:0/2566784618 >> v2:172.21.15.66:3300/0 conn(0x7f5afc0b4690 0x7f5afc06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0cd590 0x7f5afc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 --2- 172.21.15.59:0/2566784618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc15c3d0 0x7f5afc0cd050 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 -- 172.21.15.59:0/2566784618 >> 172.21.15.59:0/2566784618 conn(0x7f5afc0b21a0 msgr2=0x7f5afc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 -- 172.21.15.59:0/2566784618 shutdown_connections 2024-04-24T06:28:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.466+0000 7f5b03200640 1 -- 172.21.15.59:0/2566784618 wait complete. 2024-04-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.468+0000 7f5b03200640 1 Processor -- start 2024-04-24T06:28:49.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.469+0000 7f5b03200640 1 -- start start 2024-04-24T06:28:49.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0b4690 0x7f5afc1f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5afc15c3d0 0x7f5afc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc15c7b0 con 0x7f5afc0cd590 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc0b4b00 con 0x7f5afc15c3d0 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b03200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc15c930 con 0x7f5afc0b4690 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b01400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b01400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37764/0 (socket says 172.21.15.59:37764) 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b01400640 1 -- 172.21.15.59:0/3818085566 learned_addr learned my addr 172.21.15.59:0/3818085566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.470+0000 7f5b01e00640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0b4690 0x7f5afc1f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 -- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0b4690 msgr2=0x7f5afc1f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0b4690 0x7f5afc1f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 -- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5afc15c3d0 msgr2=0x7f5afc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5afc15c3d0 0x7f5afc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 -- 172.21.15.59:0/3818085566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc0c9d90 con 0x7f5afc0cd590 2024-04-24T06:28:49.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b01400640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e730 tx=0x7f5aec00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00f040 con 0x7f5afc0cd590 2024-04-24T06:28:49.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec004230 con 0x7f5afc0cd590 2024-04-24T06:28:49.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc1f7490 con 0x7f5afc0cd590 2024-04-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00ee40 con 0x7f5afc0cd590 2024-04-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.471+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc0c76e0 con 0x7f5afc0cd590 2024-04-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.473+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc04fd90 con 0x7f5afc0cd590 2024-04-24T06:28:49.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.474+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5aec00c6d0 con 0x7f5afc0cd590 2024-04-24T06:28:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.474+0000 7f5af6000640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 0x7f5ae006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.475+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5aec08b850 con 0x7f5afc0cd590 2024-04-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.475+0000 7f5b01e00640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 0x7f5ae006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.475+0000 7f5b01e00640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 0x7f5ae006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5af0034000 tx=0x7f5af002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.479+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5afc1f7490 con 0x7f5afc0cd590 2024-04-24T06:28:49.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.684+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5afc0c69e0 con 0x7f5ae006c8f0 2024-04-24T06:28:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.699+0000 7f5af6000640 1 -- 172.21.15.59:0/3818085566 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f5afc0c69e0 con 0x7f5ae006c8f0 2024-04-24T06:28:49.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:49.704 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:28:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 msgr2=0x7f5ae006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 0x7f5ae006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5af0034000 tx=0x7f5af002e000 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 msgr2=0x7f5afc0c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e730 tx=0x7f5aec00ebf0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 shutdown_connections 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae006c8f0 0x7f5ae006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5afc15c3d0 0x7f5afc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5afc0cd590 0x7f5afc0c81c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 --2- 172.21.15.59:0/3818085566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5afc0b4690 0x7f5afc1f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.704+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 >> 172.21.15.59:0/3818085566 conn(0x7f5afc0b21a0 msgr2=0x7f5afc0b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.705+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 shutdown_connections 2024-04-24T06:28:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.705+0000 7f5b03200640 1 -- 172.21.15.59:0/3818085566 wait complete. 2024-04-24T06:28:49.726 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T06:28:49.188624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994473+0000","last_change":"2024-04-24T06:28:44.737375+0000","last_active":"2024-04-24T06:28:45.994473+0000","last_peered":"2024-04-24T06:28:45.994473+0000","last_clean":"2024-04-24T06:28:45.994473+0000","last_became_active":"2024-04-24T06:28:44.737064+0000","last_became_peered":"2024-04-24T06:28:44.737064+0000","last_unstale":"2024-04-24T06:28:45.994473+0000","last_undegraded":"2024-04-24T06:28:45.994473+0000","last_fullsized":"2024-04-24T06:28:45.994473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991754+0000","last_change":"2024-04-24T06:28:44.748671+0000","last_active":"2024-04-24T06:28:45.991754+0000","last_peered":"2024-04-24T06:28:45.991754+0000","last_clean":"2024-04-24T06:28:45.991754+0000","last_became_active":"2024-04-24T06:28:44.748393+0000","last_became_peered":"2024-04-24T06:28:44.748393+0000","last_unstale":"2024-04-24T06:28:45.991754+0000","last_undegraded":"2024-04-24T06:28:45.991754+0000","last_fullsized":"2024-04-24T06:28:45.991754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990411+0000","last_change":"2024-04-24T06:28:44.738384+0000","last_active":"2024-04-24T06:28:45.990411+0000","last_peered":"2024-04-24T06:28:45.990411+0000","last_clean":"2024-04-24T06:28:45.990411+0000","last_became_active":"2024-04-24T06:28:44.738063+0000","last_became_peered":"2024-04-24T06:28:44.738063+0000","last_unstale":"2024-04-24T06:28:45.990411+0000","last_undegraded":"2024-04-24T06:28:45.990411+0000","last_fullsized":"2024-04-24T06:28:45.990411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991159+0000","last_change":"2024-04-24T06:28:44.754483+0000","last_active":"2024-04-24T06:28:45.991159+0000","last_peered":"2024-04-24T06:28:45.991159+0000","last_clean":"2024-04-24T06:28:45.991159+0000","last_became_active":"2024-04-24T06:28:44.754150+0000","last_became_peered":"2024-04-24T06:28:44.754150+0000","last_unstale":"2024-04-24T06:28:45.991159+0000","last_undegraded":"2024-04-24T06:28:45.991159+0000","last_fullsized":"2024-04-24T06:28:45.991159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993216+0000","last_change":"2024-04-24T06:28:44.741673+0000","last_active":"2024-04-24T06:28:45.993216+0000","last_peered":"2024-04-24T06:28:45.993216+0000","last_clean":"2024-04-24T06:28:45.993216+0000","last_became_active":"2024-04-24T06:28:44.741361+0000","last_became_peered":"2024-04-24T06:28:44.741361+0000","last_unstale":"2024-04-24T06:28:45.993216+0000","last_undegraded":"2024-04-24T06:28:45.993216+0000","last_fullsized":"2024-04-24T06:28:45.993216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990171+0000","last_change":"2024-04-24T06:28:44.745828+0000","last_active":"2024-04-24T06:28:45.990171+0000","last_peered":"2024-04-24T06:28:45.990171+0000","last_clean":"2024-04-24T06:28:45.990171+0000","last_became_active":"2024-04-24T06:28:44.745499+0000","last_became_peered":"2024-04-24T06:28:44.745499+0000","last_unstale":"2024-04-24T06:28:45.990171+0000","last_undegraded":"2024-04-24T06:28:45.990171+0000","last_fullsized":"2024-04-24T06:28:45.990171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992379+0000","last_change":"2024-04-24T06:28:44.752057+0000","last_active":"2024-04-24T06:28:45.992379+0000","last_peered":"2024-04-24T06:28:45.992379+0000","last_clean":"2024-04-24T06:28:45.992379+0000","last_became_active":"2024-04-24T06:28:44.746797+0000","last_became_peered":"2024-04-24T06:28:44.746797+0000","last_unstale":"2024-04-24T06:28:45.992379+0000","last_undegraded":"2024-04-24T06:28:45.992379+0000","last_fullsized":"2024-04-24T06:28:45.992379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741286+0000","last_change":"2024-04-24T06:28:44.734221+0000","last_active":"2024-04-24T06:28:45.741286+0000","last_peered":"2024-04-24T06:28:45.741286+0000","last_clean":"2024-04-24T06:28:45.741286+0000","last_became_active":"2024-04-24T06:28:44.733624+0000","last_became_peered":"2024-04-24T06:28:44.733624+0000","last_unstale":"2024-04-24T06:28:45.741286+0000","last_undegraded":"2024-04-24T06:28:45.741286+0000","last_fullsized":"2024-04-24T06:28:45.741286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994911+0000","last_change":"2024-04-24T06:28:44.744186+0000","last_active":"2024-04-24T06:28:45.994911+0000","last_peered":"2024-04-24T06:28:45.994911+0000","last_clean":"2024-04-24T06:28:45.994911+0000","last_became_active":"2024-04-24T06:28:44.743766+0000","last_became_peered":"2024-04-24T06:28:44.743766+0000","last_unstale":"2024-04-24T06:28:45.994911+0000","last_undegraded":"2024-04-24T06:28:45.994911+0000","last_fullsized":"2024-04-24T06:28:45.994911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:00.455831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989452+0000","last_change":"2024-04-24T06:28:44.742684+0000","last_active":"2024-04-24T06:28:45.989452+0000","last_peered":"2024-04-24T06:28:45.989452+0000","last_clean":"2024-04-24T06:28:45.989452+0000","last_became_active":"2024-04-24T06:28:44.742251+0000","last_became_peered":"2024-04-24T06:28:44.742251+0000","last_unstale":"2024-04-24T06:28:45.989452+0000","last_undegraded":"2024-04-24T06:28:45.989452+0000","last_fullsized":"2024-04-24T06:28:45.989452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989977+0000","last_change":"2024-04-24T06:28:44.741312+0000","last_active":"2024-04-24T06:28:45.989977+0000","last_peered":"2024-04-24T06:28:45.989977+0000","last_clean":"2024-04-24T06:28:45.989977+0000","last_became_active":"2024-04-24T06:28:44.740904+0000","last_became_peered":"2024-04-24T06:28:44.740904+0000","last_unstale":"2024-04-24T06:28:45.989977+0000","last_undegraded":"2024-04-24T06:28:45.989977+0000","last_fullsized":"2024-04-24T06:28:45.989977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741756+0000","last_change":"2024-04-24T06:28:44.748944+0000","last_active":"2024-04-24T06:28:45.741756+0000","last_peered":"2024-04-24T06:28:45.741756+0000","last_clean":"2024-04-24T06:28:45.741756+0000","last_became_active":"2024-04-24T06:28:44.748456+0000","last_became_peered":"2024-04-24T06:28:44.748456+0000","last_unstale":"2024-04-24T06:28:45.741756+0000","last_undegraded":"2024-04-24T06:28:45.741756+0000","last_fullsized":"2024-04-24T06:28:45.741756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741786+0000","last_change":"2024-04-24T06:28:44.742242+0000","last_active":"2024-04-24T06:28:45.741786+0000","last_peered":"2024-04-24T06:28:45.741786+0000","last_clean":"2024-04-24T06:28:45.741786+0000","last_became_active":"2024-04-24T06:28:44.741323+0000","last_became_peered":"2024-04-24T06:28:44.741323+0000","last_unstale":"2024-04-24T06:28:45.741786+0000","last_undegraded":"2024-04-24T06:28:45.741786+0000","last_fullsized":"2024-04-24T06:28:45.741786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990162+0000","last_change":"2024-04-24T06:28:44.733381+0000","last_active":"2024-04-24T06:28:45.990162+0000","last_peered":"2024-04-24T06:28:45.990162+0000","last_clean":"2024-04-24T06:28:45.990162+0000","last_became_active":"2024-04-24T06:28:44.733086+0000","last_became_peered":"2024-04-24T06:28:44.733086+0000","last_unstale":"2024-04-24T06:28:45.990162+0000","last_undegraded":"2024-04-24T06:28:45.990162+0000","last_fullsized":"2024-04-24T06:28:45.990162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:46.231913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989857+0000","last_change":"2024-04-24T06:28:44.743567+0000","last_active":"2024-04-24T06:28:45.989857+0000","last_peered":"2024-04-24T06:28:45.989857+0000","last_clean":"2024-04-24T06:28:45.989857+0000","last_became_active":"2024-04-24T06:28:44.743281+0000","last_became_peered":"2024-04-24T06:28:44.743281+0000","last_unstale":"2024-04-24T06:28:45.989857+0000","last_undegraded":"2024-04-24T06:28:45.989857+0000","last_fullsized":"2024-04-24T06:28:45.989857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990473+0000","last_change":"2024-04-24T06:28:44.734065+0000","last_active":"2024-04-24T06:28:45.990473+0000","last_peered":"2024-04-24T06:28:45.990473+0000","last_clean":"2024-04-24T06:28:45.990473+0000","last_became_active":"2024-04-24T06:28:44.733730+0000","last_became_peered":"2024-04-24T06:28:44.733730+0000","last_unstale":"2024-04-24T06:28:45.990473+0000","last_undegraded":"2024-04-24T06:28:45.990473+0000","last_fullsized":"2024-04-24T06:28:45.990473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992895+0000","last_change":"2024-04-24T06:28:44.749923+0000","last_active":"2024-04-24T06:28:45.992895+0000","last_peered":"2024-04-24T06:28:45.992895+0000","last_clean":"2024-04-24T06:28:45.992895+0000","last_became_active":"2024-04-24T06:28:44.749576+0000","last_became_peered":"2024-04-24T06:28:44.749576+0000","last_unstale":"2024-04-24T06:28:45.992895+0000","last_undegraded":"2024-04-24T06:28:45.992895+0000","last_fullsized":"2024-04-24T06:28:45.992895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987678+0000","last_change":"2024-04-24T06:28:44.737360+0000","last_active":"2024-04-24T06:28:45.987678+0000","last_peered":"2024-04-24T06:28:45.987678+0000","last_clean":"2024-04-24T06:28:45.987678+0000","last_became_active":"2024-04-24T06:28:44.737001+0000","last_became_peered":"2024-04-24T06:28:44.737001+0000","last_unstale":"2024-04-24T06:28:45.987678+0000","last_undegraded":"2024-04-24T06:28:45.987678+0000","last_fullsized":"2024-04-24T06:28:45.987678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994589+0000","last_change":"2024-04-24T06:28:44.748893+0000","last_active":"2024-04-24T06:28:45.994589+0000","last_peered":"2024-04-24T06:28:45.994589+0000","last_clean":"2024-04-24T06:28:45.994589+0000","last_became_active":"2024-04-24T06:28:44.748554+0000","last_became_peered":"2024-04-24T06:28:44.748554+0000","last_unstale":"2024-04-24T06:28:45.994589+0000","last_undegraded":"2024-04-24T06:28:45.994589+0000","last_fullsized":"2024-04-24T06:28:45.994589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989639+0000","last_change":"2024-04-24T06:28:44.759322+0000","last_active":"2024-04-24T06:28:45.989639+0000","last_peered":"2024-04-24T06:28:45.989639+0000","last_clean":"2024-04-24T06:28:45.989639+0000","last_became_active":"2024-04-24T06:28:44.758153+0000","last_became_peered":"2024-04-24T06:28:44.758153+0000","last_unstale":"2024-04-24T06:28:45.989639+0000","last_undegraded":"2024-04-24T06:28:45.989639+0000","last_fullsized":"2024-04-24T06:28:45.989639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:44.745627+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:44.745334+0000","last_became_peered":"2024-04-24T06:28:44.745334+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993161+0000","last_change":"2024-04-24T06:28:44.742546+0000","last_active":"2024-04-24T06:28:45.993161+0000","last_peered":"2024-04-24T06:28:45.993161+0000","last_clean":"2024-04-24T06:28:45.993161+0000","last_became_active":"2024-04-24T06:28:44.740862+0000","last_became_peered":"2024-04-24T06:28:44.740862+0000","last_unstale":"2024-04-24T06:28:45.993161+0000","last_undegraded":"2024-04-24T06:28:45.993161+0000","last_fullsized":"2024-04-24T06:28:45.993161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741404+0000","last_change":"2024-04-24T06:28:44.748223+0000","last_active":"2024-04-24T06:28:45.741404+0000","last_peered":"2024-04-24T06:28:45.741404+0000","last_clean":"2024-04-24T06:28:45.741404+0000","last_became_active":"2024-04-24T06:28:44.745248+0000","last_became_peered":"2024-04-24T06:28:44.745248+0000","last_unstale":"2024-04-24T06:28:45.741404+0000","last_undegraded":"2024-04-24T06:28:45.741404+0000","last_fullsized":"2024-04-24T06:28:45.741404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988946+0000","last_change":"2024-04-24T06:28:44.739176+0000","last_active":"2024-04-24T06:28:45.988946+0000","last_peered":"2024-04-24T06:28:45.988946+0000","last_clean":"2024-04-24T06:28:45.988946+0000","last_became_active":"2024-04-24T06:28:44.738787+0000","last_became_peered":"2024-04-24T06:28:44.738787+0000","last_unstale":"2024-04-24T06:28:45.988946+0000","last_undegraded":"2024-04-24T06:28:45.988946+0000","last_fullsized":"2024-04-24T06:28:45.988946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742676+0000","last_change":"2024-04-24T06:28:44.757966+0000","last_active":"2024-04-24T06:28:45.742676+0000","last_peered":"2024-04-24T06:28:45.742676+0000","last_clean":"2024-04-24T06:28:45.742676+0000","last_became_active":"2024-04-24T06:28:44.757694+0000","last_became_peered":"2024-04-24T06:28:44.757694+0000","last_unstale":"2024-04-24T06:28:45.742676+0000","last_undegraded":"2024-04-24T06:28:45.742676+0000","last_fullsized":"2024-04-24T06:28:45.742676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989747+0000","last_change":"2024-04-24T06:28:44.747390+0000","last_active":"2024-04-24T06:28:45.989747+0000","last_peered":"2024-04-24T06:28:45.989747+0000","last_clean":"2024-04-24T06:28:45.989747+0000","last_became_active":"2024-04-24T06:28:44.746835+0000","last_became_peered":"2024-04-24T06:28:44.746835+0000","last_unstale":"2024-04-24T06:28:45.989747+0000","last_undegraded":"2024-04-24T06:28:45.989747+0000","last_fullsized":"2024-04-24T06:28:45.989747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991286+0000","last_change":"2024-04-24T06:28:44.751607+0000","last_active":"2024-04-24T06:28:45.991286+0000","last_peered":"2024-04-24T06:28:45.991286+0000","last_clean":"2024-04-24T06:28:45.991286+0000","last_became_active":"2024-04-24T06:28:44.750641+0000","last_became_peered":"2024-04-24T06:28:44.750641+0000","last_unstale":"2024-04-24T06:28:45.991286+0000","last_undegraded":"2024-04-24T06:28:45.991286+0000","last_fullsized":"2024-04-24T06:28:45.991286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742952+0000","last_change":"2024-04-24T06:28:44.760798+0000","last_active":"2024-04-24T06:28:45.742952+0000","last_peered":"2024-04-24T06:28:45.742952+0000","last_clean":"2024-04-24T06:28:45.742952+0000","last_became_active":"2024-04-24T06:28:44.760283+0000","last_became_peered":"2024-04-24T06:28:44.760283+0000","last_unstale":"2024-04-24T06:28:45.742952+0000","last_undegraded":"2024-04-24T06:28:45.742952+0000","last_fullsized":"2024-04-24T06:28:45.742952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990053+0000","last_change":"2024-04-24T06:28:44.734598+0000","last_active":"2024-04-24T06:28:45.990053+0000","last_peered":"2024-04-24T06:28:45.990053+0000","last_clean":"2024-04-24T06:28:45.990053+0000","last_became_active":"2024-04-24T06:28:44.733583+0000","last_became_peered":"2024-04-24T06:28:44.733583+0000","last_unstale":"2024-04-24T06:28:45.990053+0000","last_undegraded":"2024-04-24T06:28:45.990053+0000","last_fullsized":"2024-04-24T06:28:45.990053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:51:11.987585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992619+0000","last_change":"2024-04-24T06:28:44.747811+0000","last_active":"2024-04-24T06:28:45.992619+0000","last_peered":"2024-04-24T06:28:45.992619+0000","last_clean":"2024-04-24T06:28:45.992619+0000","last_became_active":"2024-04-24T06:28:44.747378+0000","last_became_peered":"2024-04-24T06:28:44.747378+0000","last_unstale":"2024-04-24T06:28:45.992619+0000","last_undegraded":"2024-04-24T06:28:45.992619+0000","last_fullsized":"2024-04-24T06:28:45.992619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987349+0000","last_change":"2024-04-24T06:28:44.730489+0000","last_active":"2024-04-24T06:28:45.987349+0000","last_peered":"2024-04-24T06:28:45.987349+0000","last_clean":"2024-04-24T06:28:45.987349+0000","last_became_active":"2024-04-24T06:28:44.729738+0000","last_became_peered":"2024-04-24T06:28:44.729738+0000","last_unstale":"2024-04-24T06:28:45.987349+0000","last_undegraded":"2024-04-24T06:28:45.987349+0000","last_fullsized":"2024-04-24T06:28:45.987349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990150+0000","last_change":"2024-04-24T06:28:44.746610+0000","last_active":"2024-04-24T06:28:45.990150+0000","last_peered":"2024-04-24T06:28:45.990150+0000","last_clean":"2024-04-24T06:28:45.990150+0000","last_became_active":"2024-04-24T06:28:44.746300+0000","last_became_peered":"2024-04-24T06:28:44.746300+0000","last_unstale":"2024-04-24T06:28:45.990150+0000","last_undegraded":"2024-04-24T06:28:45.990150+0000","last_fullsized":"2024-04-24T06:28:45.990150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741355+0000","last_change":"2024-04-24T06:28:44.740084+0000","last_active":"2024-04-24T06:28:45.741355+0000","last_peered":"2024-04-24T06:28:45.741355+0000","last_clean":"2024-04-24T06:28:45.741355+0000","last_became_active":"2024-04-24T06:28:44.739785+0000","last_became_peered":"2024-04-24T06:28:44.739785+0000","last_unstale":"2024-04-24T06:28:45.741355+0000","last_undegraded":"2024-04-24T06:28:45.741355+0000","last_fullsized":"2024-04-24T06:28:45.741355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741767+0000","last_change":"2024-04-24T06:28:43.741594+0000","last_active":"2024-04-24T06:28:45.741767+0000","last_peered":"2024-04-24T06:28:45.741767+0000","last_clean":"2024-04-24T06:28:45.741767+0000","last_became_active":"2024-04-24T06:28:43.740453+0000","last_became_peered":"2024-04-24T06:28:43.740453+0000","last_unstale":"2024-04-24T06:28:45.741767+0000","last_undegraded":"2024-04-24T06:28:45.741767+0000","last_fullsized":"2024-04-24T06:28:45.741767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990371+0000","last_change":"2024-04-24T06:28:44.750178+0000","last_active":"2024-04-24T06:28:45.990371+0000","last_peered":"2024-04-24T06:28:45.990371+0000","last_clean":"2024-04-24T06:28:45.990371+0000","last_became_active":"2024-04-24T06:28:44.749081+0000","last_became_peered":"2024-04-24T06:28:44.749081+0000","last_unstale":"2024-04-24T06:28:45.990371+0000","last_undegraded":"2024-04-24T06:28:45.990371+0000","last_fullsized":"2024-04-24T06:28:45.990371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989324+0000","last_change":"2024-04-24T06:28:43.745532+0000","last_active":"2024-04-24T06:28:45.989324+0000","last_peered":"2024-04-24T06:28:45.989324+0000","last_clean":"2024-04-24T06:28:45.989324+0000","last_became_active":"2024-04-24T06:28:43.745191+0000","last_became_peered":"2024-04-24T06:28:43.745191+0000","last_unstale":"2024-04-24T06:28:45.989324+0000","last_undegraded":"2024-04-24T06:28:45.989324+0000","last_fullsized":"2024-04-24T06:28:45.989324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741301+0000","last_change":"2024-04-24T06:28:43.740409+0000","last_active":"2024-04-24T06:28:45.741301+0000","last_peered":"2024-04-24T06:28:45.741301+0000","last_clean":"2024-04-24T06:28:45.741301+0000","last_became_active":"2024-04-24T06:28:43.740113+0000","last_became_peered":"2024-04-24T06:28:43.740113+0000","last_unstale":"2024-04-24T06:28:45.741301+0000","last_undegraded":"2024-04-24T06:28:45.741301+0000","last_fullsized":"2024-04-24T06:28:45.741301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741118+0000","last_change":"2024-04-24T06:28:44.734557+0000","last_active":"2024-04-24T06:28:45.741118+0000","last_peered":"2024-04-24T06:28:45.741118+0000","last_clean":"2024-04-24T06:28:45.741118+0000","last_became_active":"2024-04-24T06:28:44.733956+0000","last_became_peered":"2024-04-24T06:28:44.733956+0000","last_unstale":"2024-04-24T06:28:45.741118+0000","last_undegraded":"2024-04-24T06:28:45.741118+0000","last_fullsized":"2024-04-24T06:28:45.741118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989096+0000","last_change":"2024-04-24T06:28:43.731610+0000","last_active":"2024-04-24T06:28:45.989096+0000","last_peered":"2024-04-24T06:28:45.989096+0000","last_clean":"2024-04-24T06:28:45.989096+0000","last_became_active":"2024-04-24T06:28:43.731214+0000","last_became_peered":"2024-04-24T06:28:43.731214+0000","last_unstale":"2024-04-24T06:28:45.989096+0000","last_undegraded":"2024-04-24T06:28:45.989096+0000","last_fullsized":"2024-04-24T06:28:45.989096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988808+0000","last_change":"2024-04-24T06:28:44.733043+0000","last_active":"2024-04-24T06:28:45.988808+0000","last_peered":"2024-04-24T06:28:45.988808+0000","last_clean":"2024-04-24T06:28:45.988808+0000","last_became_active":"2024-04-24T06:28:44.732788+0000","last_became_peered":"2024-04-24T06:28:44.732788+0000","last_unstale":"2024-04-24T06:28:45.988808+0000","last_undegraded":"2024-04-24T06:28:45.988808+0000","last_fullsized":"2024-04-24T06:28:45.988808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989195+0000","last_change":"2024-04-24T06:28:43.748552+0000","last_active":"2024-04-24T06:28:45.989195+0000","last_peered":"2024-04-24T06:28:45.989195+0000","last_clean":"2024-04-24T06:28:45.989195+0000","last_became_active":"2024-04-24T06:28:43.748142+0000","last_became_peered":"2024-04-24T06:28:43.748142+0000","last_unstale":"2024-04-24T06:28:45.989195+0000","last_undegraded":"2024-04-24T06:28:45.989195+0000","last_fullsized":"2024-04-24T06:28:45.989195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992302+0000","last_change":"2024-04-24T06:28:44.745984+0000","last_active":"2024-04-24T06:28:45.992302+0000","last_peered":"2024-04-24T06:28:45.992302+0000","last_clean":"2024-04-24T06:28:45.992302+0000","last_became_active":"2024-04-24T06:28:44.745648+0000","last_became_peered":"2024-04-24T06:28:44.745648+0000","last_unstale":"2024-04-24T06:28:45.992302+0000","last_undegraded":"2024-04-24T06:28:45.992302+0000","last_fullsized":"2024-04-24T06:28:45.992302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992307+0000","last_change":"2024-04-24T06:28:43.742088+0000","last_active":"2024-04-24T06:28:45.992307+0000","last_peered":"2024-04-24T06:28:45.992307+0000","last_clean":"2024-04-24T06:28:45.992307+0000","last_became_active":"2024-04-24T06:28:43.741051+0000","last_became_peered":"2024-04-24T06:28:43.741051+0000","last_unstale":"2024-04-24T06:28:45.992307+0000","last_undegraded":"2024-04-24T06:28:45.992307+0000","last_fullsized":"2024-04-24T06:28:45.992307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990451+0000","last_change":"2024-04-24T06:28:44.737169+0000","last_active":"2024-04-24T06:28:45.990451+0000","last_peered":"2024-04-24T06:28:45.990451+0000","last_clean":"2024-04-24T06:28:45.990451+0000","last_became_active":"2024-04-24T06:28:44.736490+0000","last_became_peered":"2024-04-24T06:28:44.736490+0000","last_unstale":"2024-04-24T06:28:45.990451+0000","last_undegraded":"2024-04-24T06:28:45.990451+0000","last_fullsized":"2024-04-24T06:28:45.990451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991438+0000","last_change":"2024-04-24T06:28:43.744091+0000","last_active":"2024-04-24T06:28:45.991438+0000","last_peered":"2024-04-24T06:28:45.991438+0000","last_clean":"2024-04-24T06:28:45.991438+0000","last_became_active":"2024-04-24T06:28:43.743603+0000","last_became_peered":"2024-04-24T06:28:43.743603+0000","last_unstale":"2024-04-24T06:28:45.991438+0000","last_undegraded":"2024-04-24T06:28:45.991438+0000","last_fullsized":"2024-04-24T06:28:45.991438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741915+0000","last_change":"2024-04-24T06:28:44.752430+0000","last_active":"2024-04-24T06:28:45.741915+0000","last_peered":"2024-04-24T06:28:45.741915+0000","last_clean":"2024-04-24T06:28:45.741915+0000","last_became_active":"2024-04-24T06:28:44.752129+0000","last_became_peered":"2024-04-24T06:28:44.752129+0000","last_unstale":"2024-04-24T06:28:45.741915+0000","last_undegraded":"2024-04-24T06:28:45.741915+0000","last_fullsized":"2024-04-24T06:28:45.741915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:11:40.427236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:43.746350+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:43.745990+0000","last_became_peered":"2024-04-24T06:28:43.745990+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994027+0000","last_change":"2024-04-24T06:28:44.755036+0000","last_active":"2024-04-24T06:28:45.994027+0000","last_peered":"2024-04-24T06:28:45.994027+0000","last_clean":"2024-04-24T06:28:45.994027+0000","last_became_active":"2024-04-24T06:28:44.754677+0000","last_became_peered":"2024-04-24T06:28:44.754677+0000","last_unstale":"2024-04-24T06:28:45.994027+0000","last_undegraded":"2024-04-24T06:28:45.994027+0000","last_fullsized":"2024-04-24T06:28:45.994027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990120+0000","last_change":"2024-04-24T06:28:43.733886+0000","last_active":"2024-04-24T06:28:45.990120+0000","last_peered":"2024-04-24T06:28:45.990120+0000","last_clean":"2024-04-24T06:28:45.990120+0000","last_became_active":"2024-04-24T06:28:43.733540+0000","last_became_peered":"2024-04-24T06:28:43.733540+0000","last_unstale":"2024-04-24T06:28:45.990120+0000","last_undegraded":"2024-04-24T06:28:45.990120+0000","last_fullsized":"2024-04-24T06:28:45.990120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990757+0000","last_change":"2024-04-24T06:28:44.740692+0000","last_active":"2024-04-24T06:28:45.990757+0000","last_peered":"2024-04-24T06:28:45.990757+0000","last_clean":"2024-04-24T06:28:45.990757+0000","last_became_active":"2024-04-24T06:28:44.740305+0000","last_became_peered":"2024-04-24T06:28:44.740305+0000","last_unstale":"2024-04-24T06:28:45.990757+0000","last_undegraded":"2024-04-24T06:28:45.990757+0000","last_fullsized":"2024-04-24T06:28:45.990757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992763+0000","last_change":"2024-04-24T06:28:43.735350+0000","last_active":"2024-04-24T06:28:45.992763+0000","last_peered":"2024-04-24T06:28:45.992763+0000","last_clean":"2024-04-24T06:28:45.992763+0000","last_became_active":"2024-04-24T06:28:43.735053+0000","last_became_peered":"2024-04-24T06:28:43.735053+0000","last_unstale":"2024-04-24T06:28:45.992763+0000","last_undegraded":"2024-04-24T06:28:45.992763+0000","last_fullsized":"2024-04-24T06:28:45.992763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993602+0000","last_change":"2024-04-24T06:28:44.758074+0000","last_active":"2024-04-24T06:28:45.993602+0000","last_peered":"2024-04-24T06:28:45.993602+0000","last_clean":"2024-04-24T06:28:45.993602+0000","last_became_active":"2024-04-24T06:28:44.757701+0000","last_became_peered":"2024-04-24T06:28:44.757701+0000","last_unstale":"2024-04-24T06:28:45.993602+0000","last_undegraded":"2024-04-24T06:28:45.993602+0000","last_fullsized":"2024-04-24T06:28:45.993602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988739+0000","last_change":"2024-04-24T06:28:43.743727+0000","last_active":"2024-04-24T06:28:45.988739+0000","last_peered":"2024-04-24T06:28:45.988739+0000","last_clean":"2024-04-24T06:28:45.988739+0000","last_became_active":"2024-04-24T06:28:43.743425+0000","last_became_peered":"2024-04-24T06:28:43.743425+0000","last_unstale":"2024-04-24T06:28:45.988739+0000","last_undegraded":"2024-04-24T06:28:45.988739+0000","last_fullsized":"2024-04-24T06:28:45.988739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989586+0000","last_change":"2024-04-24T06:28:44.743620+0000","last_active":"2024-04-24T06:28:45.989586+0000","last_peered":"2024-04-24T06:28:45.989586+0000","last_clean":"2024-04-24T06:28:45.989586+0000","last_became_active":"2024-04-24T06:28:44.743033+0000","last_became_peered":"2024-04-24T06:28:44.743033+0000","last_unstale":"2024-04-24T06:28:45.989586+0000","last_undegraded":"2024-04-24T06:28:45.989586+0000","last_fullsized":"2024-04-24T06:28:45.989586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987173+0000","last_change":"2024-04-24T06:28:43.736607+0000","last_active":"2024-04-24T06:28:45.987173+0000","last_peered":"2024-04-24T06:28:45.987173+0000","last_clean":"2024-04-24T06:28:45.987173+0000","last_became_active":"2024-04-24T06:28:43.735768+0000","last_became_peered":"2024-04-24T06:28:43.735768+0000","last_unstale":"2024-04-24T06:28:45.987173+0000","last_undegraded":"2024-04-24T06:28:45.987173+0000","last_fullsized":"2024-04-24T06:28:45.987173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:44.752144+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:44.751815+0000","last_became_peered":"2024-04-24T06:28:44.751815+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741519+0000","last_change":"2024-04-24T06:28:43.738580+0000","last_active":"2024-04-24T06:28:45.741519+0000","last_peered":"2024-04-24T06:28:45.741519+0000","last_clean":"2024-04-24T06:28:45.741519+0000","last_became_active":"2024-04-24T06:28:43.737916+0000","last_became_peered":"2024-04-24T06:28:43.737916+0000","last_unstale":"2024-04-24T06:28:45.741519+0000","last_undegraded":"2024-04-24T06:28:45.741519+0000","last_fullsized":"2024-04-24T06:28:45.741519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994443+0000","last_change":"2024-04-24T06:28:44.753155+0000","last_active":"2024-04-24T06:28:45.994443+0000","last_peered":"2024-04-24T06:28:45.994443+0000","last_clean":"2024-04-24T06:28:45.994443+0000","last_became_active":"2024-04-24T06:28:44.752638+0000","last_became_peered":"2024-04-24T06:28:44.752638+0000","last_unstale":"2024-04-24T06:28:45.994443+0000","last_undegraded":"2024-04-24T06:28:45.994443+0000","last_fullsized":"2024-04-24T06:28:45.994443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987739+0000","last_change":"2024-04-24T06:28:43.738005+0000","last_active":"2024-04-24T06:28:45.987739+0000","last_peered":"2024-04-24T06:28:45.987739+0000","last_clean":"2024-04-24T06:28:45.987739+0000","last_became_active":"2024-04-24T06:28:43.737449+0000","last_became_peered":"2024-04-24T06:28:43.737449+0000","last_unstale":"2024-04-24T06:28:45.987739+0000","last_undegraded":"2024-04-24T06:28:45.987739+0000","last_fullsized":"2024-04-24T06:28:45.987739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994839+0000","last_change":"2024-04-24T06:28:44.754240+0000","last_active":"2024-04-24T06:28:45.994839+0000","last_peered":"2024-04-24T06:28:45.994839+0000","last_clean":"2024-04-24T06:28:45.994839+0000","last_became_active":"2024-04-24T06:28:44.753698+0000","last_became_peered":"2024-04-24T06:28:44.753698+0000","last_unstale":"2024-04-24T06:28:45.994839+0000","last_undegraded":"2024-04-24T06:28:45.994839+0000","last_fullsized":"2024-04-24T06:28:45.994839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:57:38.145042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991465+0000","last_change":"2024-04-24T06:28:43.744401+0000","last_active":"2024-04-24T06:28:45.991465+0000","last_peered":"2024-04-24T06:28:45.991465+0000","last_clean":"2024-04-24T06:28:45.991465+0000","last_became_active":"2024-04-24T06:28:43.744085+0000","last_became_peered":"2024-04-24T06:28:43.744085+0000","last_unstale":"2024-04-24T06:28:45.991465+0000","last_undegraded":"2024-04-24T06:28:45.991465+0000","last_fullsized":"2024-04-24T06:28:45.991465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992064+0000","last_change":"2024-04-24T06:28:44.747939+0000","last_active":"2024-04-24T06:28:45.992064+0000","last_peered":"2024-04-24T06:28:45.992064+0000","last_clean":"2024-04-24T06:28:45.992064+0000","last_became_active":"2024-04-24T06:28:44.747593+0000","last_became_peered":"2024-04-24T06:28:44.747593+0000","last_unstale":"2024-04-24T06:28:45.992064+0000","last_undegraded":"2024-04-24T06:28:45.992064+0000","last_fullsized":"2024-04-24T06:28:45.992064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987616+0000","last_change":"2024-04-24T06:28:43.736080+0000","last_active":"2024-04-24T06:28:45.987616+0000","last_peered":"2024-04-24T06:28:45.987616+0000","last_clean":"2024-04-24T06:28:45.987616+0000","last_became_active":"2024-04-24T06:28:43.735262+0000","last_became_peered":"2024-04-24T06:28:43.735262+0000","last_unstale":"2024-04-24T06:28:45.987616+0000","last_undegraded":"2024-04-24T06:28:45.987616+0000","last_fullsized":"2024-04-24T06:28:45.987616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994003+0000","last_change":"2024-04-24T06:28:44.748730+0000","last_active":"2024-04-24T06:28:45.994003+0000","last_peered":"2024-04-24T06:28:45.994003+0000","last_clean":"2024-04-24T06:28:45.994003+0000","last_became_active":"2024-04-24T06:28:44.748473+0000","last_became_peered":"2024-04-24T06:28:44.748473+0000","last_unstale":"2024-04-24T06:28:45.994003+0000","last_undegraded":"2024-04-24T06:28:45.994003+0000","last_fullsized":"2024-04-24T06:28:45.994003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989929+0000","last_change":"2024-04-24T06:28:43.732980+0000","last_active":"2024-04-24T06:28:45.989929+0000","last_peered":"2024-04-24T06:28:45.989929+0000","last_clean":"2024-04-24T06:28:45.989929+0000","last_became_active":"2024-04-24T06:28:43.732613+0000","last_became_peered":"2024-04-24T06:28:43.732613+0000","last_unstale":"2024-04-24T06:28:45.989929+0000","last_undegraded":"2024-04-24T06:28:45.989929+0000","last_fullsized":"2024-04-24T06:28:45.989929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741850+0000","last_change":"2024-04-24T06:28:44.741822+0000","last_active":"2024-04-24T06:28:45.741850+0000","last_peered":"2024-04-24T06:28:45.741850+0000","last_clean":"2024-04-24T06:28:45.741850+0000","last_became_active":"2024-04-24T06:28:44.737221+0000","last_became_peered":"2024-04-24T06:28:44.737221+0000","last_unstale":"2024-04-24T06:28:45.741850+0000","last_undegraded":"2024-04-24T06:28:45.741850+0000","last_fullsized":"2024-04-24T06:28:45.741850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992125+0000","last_change":"2024-04-24T06:28:43.742633+0000","last_active":"2024-04-24T06:28:45.992125+0000","last_peered":"2024-04-24T06:28:45.992125+0000","last_clean":"2024-04-24T06:28:45.992125+0000","last_became_active":"2024-04-24T06:28:43.742329+0000","last_became_peered":"2024-04-24T06:28:43.742329+0000","last_unstale":"2024-04-24T06:28:45.992125+0000","last_undegraded":"2024-04-24T06:28:45.992125+0000","last_fullsized":"2024-04-24T06:28:45.992125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991037+0000","last_change":"2024-04-24T06:28:44.729217+0000","last_active":"2024-04-24T06:28:45.991037+0000","last_peered":"2024-04-24T06:28:45.991037+0000","last_clean":"2024-04-24T06:28:45.991037+0000","last_became_active":"2024-04-24T06:28:44.728584+0000","last_became_peered":"2024-04-24T06:28:44.728584+0000","last_unstale":"2024-04-24T06:28:45.991037+0000","last_undegraded":"2024-04-24T06:28:45.991037+0000","last_fullsized":"2024-04-24T06:28:45.991037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987414+0000","last_change":"2024-04-24T06:28:43.737893+0000","last_active":"2024-04-24T06:28:45.987414+0000","last_peered":"2024-04-24T06:28:45.987414+0000","last_clean":"2024-04-24T06:28:45.987414+0000","last_became_active":"2024-04-24T06:28:43.737490+0000","last_became_peered":"2024-04-24T06:28:43.737490+0000","last_unstale":"2024-04-24T06:28:45.987414+0000","last_undegraded":"2024-04-24T06:28:45.987414+0000","last_fullsized":"2024-04-24T06:28:45.987414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988935+0000","last_change":"2024-04-24T06:28:44.727674+0000","last_active":"2024-04-24T06:28:45.988935+0000","last_peered":"2024-04-24T06:28:45.988935+0000","last_clean":"2024-04-24T06:28:45.988935+0000","last_became_active":"2024-04-24T06:28:44.726973+0000","last_became_peered":"2024-04-24T06:28:44.726973+0000","last_unstale":"2024-04-24T06:28:45.988935+0000","last_undegraded":"2024-04-24T06:28:45.988935+0000","last_fullsized":"2024-04-24T06:28:45.988935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992992+0000","last_change":"2024-04-24T06:28:43.734680+0000","last_active":"2024-04-24T06:28:45.992992+0000","last_peered":"2024-04-24T06:28:45.992992+0000","last_clean":"2024-04-24T06:28:45.992992+0000","last_became_active":"2024-04-24T06:28:43.734291+0000","last_became_peered":"2024-04-24T06:28:43.734291+0000","last_unstale":"2024-04-24T06:28:45.992992+0000","last_undegraded":"2024-04-24T06:28:45.992992+0000","last_fullsized":"2024-04-24T06:28:45.992992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741903+0000","last_change":"2024-04-24T06:28:44.747768+0000","last_active":"2024-04-24T06:28:45.741903+0000","last_peered":"2024-04-24T06:28:45.741903+0000","last_clean":"2024-04-24T06:28:45.741903+0000","last_became_active":"2024-04-24T06:28:44.747439+0000","last_became_peered":"2024-04-24T06:28:44.747439+0000","last_unstale":"2024-04-24T06:28:45.741903+0000","last_undegraded":"2024-04-24T06:28:45.741903+0000","last_fullsized":"2024-04-24T06:28:45.741903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:18.635305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991108+0000","last_change":"2024-04-24T06:28:43.736187+0000","last_active":"2024-04-24T06:28:45.991108+0000","last_peered":"2024-04-24T06:28:45.991108+0000","last_clean":"2024-04-24T06:28:45.991108+0000","last_became_active":"2024-04-24T06:28:43.735013+0000","last_became_peered":"2024-04-24T06:28:43.735013+0000","last_unstale":"2024-04-24T06:28:45.991108+0000","last_undegraded":"2024-04-24T06:28:45.991108+0000","last_fullsized":"2024-04-24T06:28:45.991108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991099+0000","last_change":"2024-04-24T06:28:44.748068+0000","last_active":"2024-04-24T06:28:45.991099+0000","last_peered":"2024-04-24T06:28:45.991099+0000","last_clean":"2024-04-24T06:28:45.991099+0000","last_became_active":"2024-04-24T06:28:44.747692+0000","last_became_peered":"2024-04-24T06:28:44.747692+0000","last_unstale":"2024-04-24T06:28:45.991099+0000","last_undegraded":"2024-04-24T06:28:45.991099+0000","last_fullsized":"2024-04-24T06:28:45.991099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994396+0000","last_change":"2024-04-24T06:28:43.746631+0000","last_active":"2024-04-24T06:28:45.994396+0000","last_peered":"2024-04-24T06:28:45.994396+0000","last_clean":"2024-04-24T06:28:45.994396+0000","last_became_active":"2024-04-24T06:28:43.746051+0000","last_became_peered":"2024-04-24T06:28:43.746051+0000","last_unstale":"2024-04-24T06:28:45.994396+0000","last_undegraded":"2024-04-24T06:28:45.994396+0000","last_fullsized":"2024-04-24T06:28:45.994396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:17.617739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991283+0000","last_change":"2024-04-24T06:28:44.730831+0000","last_active":"2024-04-24T06:28:45.991283+0000","last_peered":"2024-04-24T06:28:45.991283+0000","last_clean":"2024-04-24T06:28:45.991283+0000","last_became_active":"2024-04-24T06:28:44.730464+0000","last_became_peered":"2024-04-24T06:28:44.730464+0000","last_unstale":"2024-04-24T06:28:45.991283+0000","last_undegraded":"2024-04-24T06:28:45.991283+0000","last_fullsized":"2024-04-24T06:28:45.991283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991856+0000","last_change":"2024-04-24T06:28:43.746118+0000","last_active":"2024-04-24T06:28:45.991856+0000","last_peered":"2024-04-24T06:28:45.991856+0000","last_clean":"2024-04-24T06:28:45.991856+0000","last_became_active":"2024-04-24T06:28:43.745789+0000","last_became_peered":"2024-04-24T06:28:43.745789+0000","last_unstale":"2024-04-24T06:28:45.991856+0000","last_undegraded":"2024-04-24T06:28:45.991856+0000","last_fullsized":"2024-04-24T06:28:45.991856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742491+0000","last_change":"2024-04-24T06:28:44.760317+0000","last_active":"2024-04-24T06:28:45.742491+0000","last_peered":"2024-04-24T06:28:45.742491+0000","last_clean":"2024-04-24T06:28:45.742491+0000","last_became_active":"2024-04-24T06:28:44.759974+0000","last_became_peered":"2024-04-24T06:28:44.759974+0000","last_unstale":"2024-04-24T06:28:45.742491+0000","last_undegraded":"2024-04-24T06:28:45.742491+0000","last_fullsized":"2024-04-24T06:28:45.742491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989548+0000","last_change":"2024-04-24T06:28:43.751042+0000","last_active":"2024-04-24T06:28:45.989548+0000","last_peered":"2024-04-24T06:28:45.989548+0000","last_clean":"2024-04-24T06:28:45.989548+0000","last_became_active":"2024-04-24T06:28:43.750648+0000","last_became_peered":"2024-04-24T06:28:43.750648+0000","last_unstale":"2024-04-24T06:28:45.989548+0000","last_undegraded":"2024-04-24T06:28:45.989548+0000","last_fullsized":"2024-04-24T06:28:45.989548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991697+0000","last_change":"2024-04-24T06:28:44.735037+0000","last_active":"2024-04-24T06:28:45.991697+0000","last_peered":"2024-04-24T06:28:45.991697+0000","last_clean":"2024-04-24T06:28:45.991697+0000","last_became_active":"2024-04-24T06:28:44.734809+0000","last_became_peered":"2024-04-24T06:28:44.734809+0000","last_unstale":"2024-04-24T06:28:45.991697+0000","last_undegraded":"2024-04-24T06:28:45.991697+0000","last_fullsized":"2024-04-24T06:28:45.991697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991820+0000","last_change":"2024-04-24T06:28:43.744133+0000","last_active":"2024-04-24T06:28:45.991820+0000","last_peered":"2024-04-24T06:28:45.991820+0000","last_clean":"2024-04-24T06:28:45.991820+0000","last_became_active":"2024-04-24T06:28:43.743825+0000","last_became_peered":"2024-04-24T06:28:43.743825+0000","last_unstale":"2024-04-24T06:28:45.991820+0000","last_undegraded":"2024-04-24T06:28:45.991820+0000","last_fullsized":"2024-04-24T06:28:45.991820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742147+0000","last_change":"2024-04-24T06:28:44.739332+0000","last_active":"2024-04-24T06:28:45.742147+0000","last_peered":"2024-04-24T06:28:45.742147+0000","last_clean":"2024-04-24T06:28:45.742147+0000","last_became_active":"2024-04-24T06:28:44.738899+0000","last_became_peered":"2024-04-24T06:28:44.738899+0000","last_unstale":"2024-04-24T06:28:45.742147+0000","last_undegraded":"2024-04-24T06:28:45.742147+0000","last_fullsized":"2024-04-24T06:28:45.742147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989171+0000","last_change":"2024-04-24T06:28:43.742255+0000","last_active":"2024-04-24T06:28:45.989171+0000","last_peered":"2024-04-24T06:28:45.989171+0000","last_clean":"2024-04-24T06:28:45.989171+0000","last_became_active":"2024-04-24T06:28:43.741940+0000","last_became_peered":"2024-04-24T06:28:43.741940+0000","last_unstale":"2024-04-24T06:28:45.989171+0000","last_undegraded":"2024-04-24T06:28:45.989171+0000","last_fullsized":"2024-04-24T06:28:45.989171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741541+0000","last_change":"2024-04-24T06:28:44.739763+0000","last_active":"2024-04-24T06:28:45.741541+0000","last_peered":"2024-04-24T06:28:45.741541+0000","last_clean":"2024-04-24T06:28:45.741541+0000","last_became_active":"2024-04-24T06:28:44.739490+0000","last_became_peered":"2024-04-24T06:28:44.739490+0000","last_unstale":"2024-04-24T06:28:45.741541+0000","last_undegraded":"2024-04-24T06:28:45.741541+0000","last_fullsized":"2024-04-24T06:28:45.741541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990882+0000","last_change":"2024-04-24T06:28:43.735231+0000","last_active":"2024-04-24T06:28:45.990882+0000","last_peered":"2024-04-24T06:28:45.990882+0000","last_clean":"2024-04-24T06:28:45.990882+0000","last_became_active":"2024-04-24T06:28:43.734870+0000","last_became_peered":"2024-04-24T06:28:43.734870+0000","last_unstale":"2024-04-24T06:28:45.990882+0000","last_undegraded":"2024-04-24T06:28:45.990882+0000","last_fullsized":"2024-04-24T06:28:45.990882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741685+0000","last_change":"2024-04-24T06:28:44.746720+0000","last_active":"2024-04-24T06:28:45.741685+0000","last_peered":"2024-04-24T06:28:45.741685+0000","last_clean":"2024-04-24T06:28:45.741685+0000","last_became_active":"2024-04-24T06:28:44.746083+0000","last_became_peered":"2024-04-24T06:28:44.746083+0000","last_unstale":"2024-04-24T06:28:45.741685+0000","last_undegraded":"2024-04-24T06:28:45.741685+0000","last_fullsized":"2024-04-24T06:28:45.741685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993302+0000","last_change":"2024-04-24T06:28:43.748833+0000","last_active":"2024-04-24T06:28:45.993302+0000","last_peered":"2024-04-24T06:28:45.993302+0000","last_clean":"2024-04-24T06:28:45.993302+0000","last_became_active":"2024-04-24T06:28:43.748530+0000","last_became_peered":"2024-04-24T06:28:43.748530+0000","last_unstale":"2024-04-24T06:28:45.993302+0000","last_undegraded":"2024-04-24T06:28:45.993302+0000","last_fullsized":"2024-04-24T06:28:45.993302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989154+0000","last_change":"2024-04-24T06:28:44.755539+0000","last_active":"2024-04-24T06:28:45.989154+0000","last_peered":"2024-04-24T06:28:45.989154+0000","last_clean":"2024-04-24T06:28:45.989154+0000","last_became_active":"2024-04-24T06:28:44.755179+0000","last_became_peered":"2024-04-24T06:28:44.755179+0000","last_unstale":"2024-04-24T06:28:45.989154+0000","last_undegraded":"2024-04-24T06:28:45.989154+0000","last_fullsized":"2024-04-24T06:28:45.989154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991127+0000","last_change":"2024-04-24T06:28:44.738377+0000","last_active":"2024-04-24T06:28:45.991127+0000","last_peered":"2024-04-24T06:28:45.991127+0000","last_clean":"2024-04-24T06:28:45.991127+0000","last_became_active":"2024-04-24T06:28:44.738086+0000","last_became_peered":"2024-04-24T06:28:44.738086+0000","last_unstale":"2024-04-24T06:28:45.991127+0000","last_undegraded":"2024-04-24T06:28:45.991127+0000","last_fullsized":"2024-04-24T06:28:45.991127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993626+0000","last_change":"2024-04-24T06:28:43.747066+0000","last_active":"2024-04-24T06:28:45.993626+0000","last_peered":"2024-04-24T06:28:45.993626+0000","last_clean":"2024-04-24T06:28:45.993626+0000","last_became_active":"2024-04-24T06:28:43.746745+0000","last_became_peered":"2024-04-24T06:28:43.746745+0000","last_unstale":"2024-04-24T06:28:45.993626+0000","last_undegraded":"2024-04-24T06:28:45.993626+0000","last_fullsized":"2024-04-24T06:28:45.993626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991013+0000","last_change":"2024-04-24T06:28:44.754533+0000","last_active":"2024-04-24T06:28:45.991013+0000","last_peered":"2024-04-24T06:28:45.991013+0000","last_clean":"2024-04-24T06:28:45.991013+0000","last_became_active":"2024-04-24T06:28:44.754250+0000","last_became_peered":"2024-04-24T06:28:44.754250+0000","last_unstale":"2024-04-24T06:28:45.991013+0000","last_undegraded":"2024-04-24T06:28:45.991013+0000","last_fullsized":"2024-04-24T06:28:45.991013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994203+0000","last_change":"2024-04-24T06:28:43.749898+0000","last_active":"2024-04-24T06:28:45.994203+0000","last_peered":"2024-04-24T06:28:45.994203+0000","last_clean":"2024-04-24T06:28:45.994203+0000","last_became_active":"2024-04-24T06:28:43.749544+0000","last_became_peered":"2024-04-24T06:28:43.749544+0000","last_unstale":"2024-04-24T06:28:45.994203+0000","last_undegraded":"2024-04-24T06:28:45.994203+0000","last_fullsized":"2024-04-24T06:28:45.994203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991965+0000","last_change":"2024-04-24T06:28:44.736764+0000","last_active":"2024-04-24T06:28:45.991965+0000","last_peered":"2024-04-24T06:28:45.991965+0000","last_clean":"2024-04-24T06:28:45.991965+0000","last_became_active":"2024-04-24T06:28:44.736187+0000","last_became_peered":"2024-04-24T06:28:44.736187+0000","last_unstale":"2024-04-24T06:28:45.991965+0000","last_undegraded":"2024-04-24T06:28:45.991965+0000","last_fullsized":"2024-04-24T06:28:45.991965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742308+0000","last_change":"2024-04-24T06:28:43.739421+0000","last_active":"2024-04-24T06:28:45.742308+0000","last_peered":"2024-04-24T06:28:45.742308+0000","last_clean":"2024-04-24T06:28:45.742308+0000","last_became_active":"2024-04-24T06:28:43.737666+0000","last_became_peered":"2024-04-24T06:28:43.737666+0000","last_unstale":"2024-04-24T06:28:45.742308+0000","last_undegraded":"2024-04-24T06:28:45.742308+0000","last_fullsized":"2024-04-24T06:28:45.742308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742640+0000","last_change":"2024-04-24T06:28:44.741299+0000","last_active":"2024-04-24T06:28:45.742640+0000","last_peered":"2024-04-24T06:28:45.742640+0000","last_clean":"2024-04-24T06:28:45.742640+0000","last_became_active":"2024-04-24T06:28:44.740960+0000","last_became_peered":"2024-04-24T06:28:44.740960+0000","last_unstale":"2024-04-24T06:28:45.742640+0000","last_undegraded":"2024-04-24T06:28:45.742640+0000","last_fullsized":"2024-04-24T06:28:45.742640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990886+0000","last_change":"2024-04-24T06:28:43.734445+0000","last_active":"2024-04-24T06:28:45.990886+0000","last_peered":"2024-04-24T06:28:45.990886+0000","last_clean":"2024-04-24T06:28:45.990886+0000","last_became_active":"2024-04-24T06:28:43.733088+0000","last_became_peered":"2024-04-24T06:28:43.733088+0000","last_unstale":"2024-04-24T06:28:45.990886+0000","last_undegraded":"2024-04-24T06:28:45.990886+0000","last_fullsized":"2024-04-24T06:28:45.990886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:28:49.729 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:28:49.730 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T06:28:49.730 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T06:28:49.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T06:28:49.730 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T06:28:49.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.975+0000 7f89c9400640 1 Processor -- start 2024-04-24T06:28:49.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.975+0000 7f89c9400640 1 -- start start 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89c4160960 0x7f89c4163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c40b4550 0x7f89c406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c9400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c4160d40 con 0x7f89c41644b0 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c9400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f89c4164270 con 0x7f89c4160960 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c9400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f89c4166de0 con 0x7f89c40b4550 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.977+0000 7f89c2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89c4160960 0x7f89c4163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89c4160960 0x7f89c4163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37766/0 (socket says 172.21.15.59:37766) 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 -- 172.21.15.59:0/3572584365 learned_addr learned my addr 172.21.15.59:0/3572584365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c3200640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c40b4550 0x7f89c406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 -- 172.21.15.59:0/3572584365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c40b4550 msgr2=0x7f89c406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c40b4550 0x7f89c406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 -- 172.21.15.59:0/3572584365 >> v1:172.21.15.66:6789/0 conn(0x7f89c41644b0 legacy=0x7f89c40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:49.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.978+0000 7f89c2800640 1 -- 172.21.15.59:0/3572584365 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c41670c0 con 0x7f89c4160960 2024-04-24T06:28:49.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.979+0000 7f89c2800640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c4160960 0x7f89c4163d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009870 tx=0x7f89b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910a7c3e4a9e45e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.980+0000 7f89c1e00640 1 -- 172.21.15.59:0/3572584365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4037070 con 0x7f89c4160960 2024-04-24T06:28:49.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.980+0000 7f89c1e00640 1 -- 172.21.15.59:0/3572584365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b4032440 con 0x7f89c4160960 2024-04-24T06:28:49.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.980+0000 7f89c1e00640 1 -- 172.21.15.59:0/3572584365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4032810 con 0x7f89c4160960 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.980+0000 7f89c9400640 1 -- 172.21.15.59:0/3572584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c4160960 msgr2=0x7f89c4163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.980+0000 7f89c9400640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c4160960 0x7f89c4163d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009870 tx=0x7f89b402a210 comp rx=0 tx=0).stop 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 -- 172.21.15.59:0/3572584365 shutdown_connections 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c40b4550 0x7f89c406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 --2- 172.21.15.59:0/3572584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c4160960 0x7f89c4163d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 -- 172.21.15.59:0/3572584365 >> 172.21.15.59:0/3572584365 conn(0x7f89c4067ea0 msgr2=0x7f89c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 -- 172.21.15.59:0/3572584365 shutdown_connections 2024-04-24T06:28:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.981+0000 7f89c9400640 1 -- 172.21.15.59:0/3572584365 wait complete. 2024-04-24T06:28:49.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.982+0000 7f89c9400640 1 Processor -- start 2024-04-24T06:28:49.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 -- start start 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c4160960 0x7f89c40cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c41644b0 0x7f89c40b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41668e0 con 0x7f89c40b4550 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4167680 con 0x7f89c4160960 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.983+0000 7f89c9400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f89c4166a60 con 0x7f89c41644b0 2024-04-24T06:28:49.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37778/0 (socket says 172.21.15.59:37778) 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 -- 172.21.15.59:0/300477237 learned_addr learned my addr 172.21.15.59:0/300477237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c3200640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c41644b0 0x7f89c40b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89b9e00640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c4160960 0x7f89c40cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 -- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c41644b0 msgr2=0x7f89c40b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c41644b0 0x7f89c40b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 -- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c4160960 msgr2=0x7f89c40cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c4160960 0x7f89c40cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:49.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.984+0000 7f89c2800640 1 -- 172.21.15.59:0/300477237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c40cd1b0 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89c2800640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89b4000c00 tx=0x7f89b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4041020 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b40306c0 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4037040 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c40cd490 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.985+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c40cda50 con 0x7f89c40b4550 2024-04-24T06:28:49.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.986+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c4068a80 con 0x7f89c40b4550 2024-04-24T06:28:49.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.987+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89b4044020 con 0x7f89c40b4550 2024-04-24T06:28:49.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.988+0000 7f89bb400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 0x7f89a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:49.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.988+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89b40ac370 con 0x7f89c40b4550 2024-04-24T06:28:49.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.988+0000 7f89b9e00640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 0x7f89a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:49.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.989+0000 7f89b9e00640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 0x7f89a006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006fd0 tx=0x7f89ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:49.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:49.993+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89b404b030 con 0x7f89c40b4550 2024-04-24T06:28:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.250+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f89c4163850 con 0x7f89c40b4550 2024-04-24T06:28:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.260+0000 7f89bb400640 1 -- 172.21.15.59:0/300477237 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174135 (secure 0 0 0) 0x7f89b407a790 con 0x7f89c40b4550 2024-04-24T06:28:50.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 msgr2=0x7f89a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 0x7f89a006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006fd0 tx=0x7f89ac008040 comp rx=0 tx=0).stop 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 msgr2=0x7f89c415f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89b4000c00 tx=0x7f89b40046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 shutdown_connections 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f89a006c8f0 0x7f89a006edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89c41644b0 0x7f89c40b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89c4160960 0x7f89c40cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 --2- 172.21.15.59:0/300477237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89c40b4550 0x7f89c415f090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 >> 172.21.15.59:0/300477237 conn(0x7f89c4067ea0 msgr2=0x7f89c406f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 shutdown_connections 2024-04-24T06:28:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.264+0000 7f89c9400640 1 -- 172.21.15.59:0/300477237 wait complete. 2024-04-24T06:28:50.285 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4266715280},{"type":"v1","addr":"172.21.15.59:6833","nonce":4266715280}]},"active_addr":"172.21.15.59:6833/4266715280","active_change":"2024-04-24T06:28:25.162526+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2104260659}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1638266324}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":15813732}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3654885365}]}]} 2024-04-24T06:28:50.289 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T06:28:50.289 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T06:28:50.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.490+0000 7fbb7d000640 1 Processor -- start 2024-04-24T06:28:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.491+0000 7fbb7d000640 1 -- start start 2024-04-24T06:28:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.492+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb7806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.492+0000 7fbb7d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb780b3080 0x7fbb780cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.492+0000 7fbb7d000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7815f160 con 0x7fbb7815ece0 2024-04-24T06:28:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.492+0000 7fbb7d000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbb780ced20 con 0x7fbb780b3080 2024-04-24T06:28:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.492+0000 7fbb7d000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7815fcd0 con 0x7fbb780cd050 2024-04-24T06:28:50.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb76a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbb7815ece0 0x7fbb780cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59884/0 (socket says 172.21.15.59:59884) 2024-04-24T06:28:50.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb76a00640 1 -- 172.21.15.59:0/881751708 learned_addr learned my addr 172.21.15.59:0/881751708 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb77400640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb7806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb77e00640 1 --2- 172.21.15.59:0/881751708 >> v2:172.21.15.66:3300/0 conn(0x7fbb780b3080 0x7fbb780cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375825225 0 0) 0x7fbb7815f160 con 0x7fbb7815ece0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.493+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb5c003520 con 0x7fbb7815ece0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1242188272 0 0) 0x7fbb5c003520 con 0x7fbb7815ece0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 msgr2=0x7fbb7806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb7806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 >> v2:172.21.15.66:3300/0 conn(0x7fbb780b3080 msgr2=0x7fbb780cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 --2- 172.21.15.59:0/881751708 >> v2:172.21.15.66:3300/0 conn(0x7fbb780b3080 0x7fbb780cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7815f430 con 0x7fbb7815ece0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbb60001670 con 0x7fbb7815ece0 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 >> v1:172.21.15.59:6789/0 conn(0x7fbb7815ece0 legacy=0x7fbb780cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c0039b0 0x7fbb5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb5c004800 0x7fbb5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7815fcd0 con 0x7fbb5c004800 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c00b9c0 con 0x7fbb5c0039b0 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c00bb90 con 0x7fbb780cd050 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76a00640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb5c004800 0x7fbb5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb77400640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c0039b0 0x7fbb5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb77e00640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.494+0000 7fbb76a00640 1 -- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 msgr2=0x7fbb5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.495+0000 7fbb76a00640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780cd050 0x7fbb5c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.495+0000 7fbb76a00640 1 -- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c0039b0 msgr2=0x7fbb5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.495+0000 7fbb76a00640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c0039b0 0x7fbb5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.495+0000 7fbb76a00640 1 -- 172.21.15.59:0/881751708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c003520 con 0x7fbb5c004800 2024-04-24T06:28:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.495+0000 7fbb76a00640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb5c004800 0x7fbb5c004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c1c0 tx=0x7fbb5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65c333224dcced2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60007c80 con 0x7fbb5c004800 2024-04-24T06:28:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6002e070 con 0x7fbb5c004800 2024-04-24T06:28:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb76000640 1 -- 172.21.15.59:0/881751708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60005d30 con 0x7fbb5c004800 2024-04-24T06:28:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb7d000640 1 -- 172.21.15.59:0/881751708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb5c004800 msgr2=0x7fbb5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb7d000640 1 --2- 172.21.15.59:0/881751708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb5c004800 0x7fbb5c004c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c1c0 tx=0x7fbb5c005830 comp rx=0 tx=0).stop 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb77e00640 1 -- 172.21.15.59:0/881751708 reap_dead start 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb7d000640 1 -- 172.21.15.59:0/881751708 shutdown_connections 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.496+0000 7fbb7d000640 1 -- 172.21.15.59:0/881751708 >> 172.21.15.59:0/881751708 conn(0x7fbb780b21a0 msgr2=0x7fbb780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.497+0000 7fbb7d000640 1 -- 172.21.15.59:0/881751708 shutdown_connections 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.497+0000 7fbb7d000640 1 -- 172.21.15.59:0/881751708 wait complete. 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.498+0000 7fbb7d000640 1 Processor -- start 2024-04-24T06:28:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.499+0000 7fbb7d000640 1 -- start start 2024-04-24T06:28:50.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.499+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c004800 0x7fbb781f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.499+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780c46b0 0x7fbb780cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb7d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb780cef50 con 0x7fbb7809d8a0 2024-04-24T06:28:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb7d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7815f9f0 con 0x7fbb5c004800 2024-04-24T06:28:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb7d000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb78159110 con 0x7fbb780c46b0 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37798/0 (socket says 172.21.15.59:37798) 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 -- 172.21.15.59:0/1631478925 learned_addr learned my addr 172.21.15.59:0/1631478925 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb77e00640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780c46b0 0x7fbb780cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb77400640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c004800 0x7fbb781f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 -- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780c46b0 msgr2=0x7fbb780cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780c46b0 0x7fbb780cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 -- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c004800 msgr2=0x7fbb781f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c004800 0x7fbb781f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.500+0000 7fbb76a00640 1 -- 172.21.15.59:0/1631478925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb780ca960 con 0x7fbb7809d8a0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb76a00640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c160 tx=0x7fbb60001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60007270 con 0x7fbb7809d8a0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb60007430 con 0x7fbb7809d8a0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb780cac40 con 0x7fbb7809d8a0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60033470 con 0x7fbb7809d8a0 2024-04-24T06:28:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.501+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb780cb200 con 0x7fbb7809d8a0 2024-04-24T06:28:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.502+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7804fd90 con 0x7fbb7809d8a0 2024-04-24T06:28:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.503+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb6003e030 con 0x7fbb7809d8a0 2024-04-24T06:28:50.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.504+0000 7fbb6f400640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 0x7fbb5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.504+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb600aba10 con 0x7fbb7809d8a0 2024-04-24T06:28:50.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.505+0000 7fbb77400640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 0x7fbb5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.505+0000 7fbb77400640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 0x7fbb5406ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb680098b0 tx=0x7fbb68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.509+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb6003c040 con 0x7fbb7809d8a0 2024-04-24T06:28:50.556 INFO:tasks.ceph.mds.b.smithi066.stderr:2024-04-24T06:28:50.555+0000 7f8881000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:28:50.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.715+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb780ce3c0 con 0x7fbb7809d8a0 2024-04-24T06:28:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.718+0000 7fbb6f400640 1 -- 172.21.15.59:0/1631478925 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fbb60079da0 con 0x7fbb7809d8a0 2024-04-24T06:28:50.720 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:50.720 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 msgr2=0x7fbb5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 0x7fbb5406ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb680098b0 tx=0x7fbb68008040 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 msgr2=0x7fbb780c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c160 tx=0x7fbb60001970 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 shutdown_connections 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5406c920 0x7fbb5406ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb780c46b0 0x7fbb780cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7809d8a0 0x7fbb780c9960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 --2- 172.21.15.59:0/1631478925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb5c004800 0x7fbb781f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.721+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 >> 172.21.15.59:0/1631478925 conn(0x7fbb780b21a0 msgr2=0x7fbb7806f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.722+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 shutdown_connections 2024-04-24T06:28:50.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.722+0000 7fbb7d000640 1 -- 172.21.15.59:0/1631478925 wait complete. 2024-04-24T06:28:50.741 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T06:28:50.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.954+0000 7f3e4fe00640 1 Processor -- start 2024-04-24T06:28:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.955+0000 7f3e4fe00640 1 -- start start 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e5015e7c0 0x7f3e500d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 0x7f3e500d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500cf030 con 0x7f3e500d2630 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3e500b7000 con 0x7f3e5015e7c0 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3e500cf620 con 0x7f3e500d0d90 2024-04-24T06:28:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e5015e7c0 0x7f3e500d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e500d2630 0x7f3e500b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59890/0 (socket says 172.21.15.59:59890) 2024-04-24T06:28:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.956+0000 7f3e4e000640 1 -- 172.21.15.59:0/544487947 learned_addr learned my addr 172.21.15.59:0/544487947 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4f400640 1 --2- 172.21.15.59:0/544487947 >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 0x7f3e500d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4ea00640 1 -- 172.21.15.59:0/544487947 >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 msgr2=0x7f3e500d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/544487947 >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 0x7f3e500d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4ea00640 1 -- 172.21.15.59:0/544487947 >> v1:172.21.15.59:6789/0 conn(0x7f3e500d2630 legacy=0x7f3e500b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4ea00640 1 -- 172.21.15.59:0/544487947 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500b6a40 con 0x7f3e5015e7c0 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.957+0000 7f3e4f400640 1 --2- 172.21.15.59:0/544487947 >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 0x7f3e500d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.958+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/544487947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500d20f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cb580 tx=0x7f3e4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbe77b030516ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.958+0000 7f3e4d600640 1 -- 172.21.15.59:0/544487947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40037070 con 0x7f3e5015e7c0 2024-04-24T06:28:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.958+0000 7f3e4d600640 1 -- 172.21.15.59:0/544487947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e40032440 con 0x7f3e5015e7c0 2024-04-24T06:28:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.958+0000 7f3e4d600640 1 -- 172.21.15.59:0/544487947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40032810 con 0x7f3e5015e7c0 2024-04-24T06:28:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 -- 172.21.15.59:0/544487947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 msgr2=0x7f3e500d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/544487947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500d20f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cb580 tx=0x7f3e4002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 -- 172.21.15.59:0/544487947 shutdown_connections 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/544487947 >> v2:172.21.15.66:3300/0 conn(0x7f3e500d0d90 0x7f3e500d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/544487947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500d20f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 -- 172.21.15.59:0/544487947 >> 172.21.15.59:0/544487947 conn(0x7f3e5009d920 msgr2=0x7f3e5008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 -- 172.21.15.59:0/544487947 shutdown_connections 2024-04-24T06:28:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.959+0000 7f3e4fe00640 1 -- 172.21.15.59:0/544487947 wait complete. 2024-04-24T06:28:50.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.961+0000 7f3e4fe00640 1 Processor -- start 2024-04-24T06:28:50.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.961+0000 7f3e4fe00640 1 -- start start 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 0x7f3e50155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 0x7f3e500b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500cf2b0 con 0x7f3e5015e7c0 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500d1700 con 0x7f3e500d0d90 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3e500b6700 con 0x7f3e500d2630 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37824/0 (socket says 172.21.15.59:37824) 2024-04-24T06:28:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.962+0000 7f3e4f400640 1 -- 172.21.15.59:0/3572505326 learned_addr learned my addr 172.21.15.59:0/3572505326 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 0x7f3e50155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4e000640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 0x7f3e500b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 -- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 msgr2=0x7f3e500b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 0x7f3e500b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 -- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 msgr2=0x7f3e50155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 0x7f3e50155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 -- 172.21.15.59:0/3572505326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5015ce30 con 0x7f3e5015e7c0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 0x7f3e50155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4e000640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 0x7f3e500b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4f400640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800cd80 tx=0x7f3e4800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800f680 con 0x7f3e5015e7c0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4800f840 con 0x7f3e5015e7c0 2024-04-24T06:28:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800fc30 con 0x7f3e5015e7c0 2024-04-24T06:28:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.963+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5015d0d0 con 0x7f3e5015e7c0 2024-04-24T06:28:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.964+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5015d690 con 0x7f3e5015e7c0 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.965+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5004fd90 con 0x7f3e5015e7c0 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.966+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e4800e3e0 con 0x7f3e5015e7c0 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.967+0000 7f3e47400640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 0x7f3e2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.967+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3e4808c970 con 0x7f3e5015e7c0 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.967+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 0x7f3e2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.968+0000 7f3e4ea00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 0x7f3e2c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e40034000 tx=0x7f3e4002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:50.971+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e48014030 con 0x7f3e5015e7c0 2024-04-24T06:28:51.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.185+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e500b64e0 con 0x7f3e5015e7c0 2024-04-24T06:28:51.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.187+0000 7f3e47400640 1 -- 172.21.15.59:0/3572505326 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f3e4805ad00 con 0x7f3e5015e7c0 2024-04-24T06:28:51.189 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:51.189 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:51.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 msgr2=0x7f3e2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 0x7f3e2c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e40034000 tx=0x7f3e4002e000 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 msgr2=0x7f3e500b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800cd80 tx=0x7f3e4800a650 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 shutdown_connections 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2c06c8f0 0x7f3e2c06edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5015e7c0 0x7f3e500b2900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e500d2630 0x7f3e500b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 --2- 172.21.15.59:0/3572505326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e500d0d90 0x7f3e50155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 >> 172.21.15.59:0/3572505326 conn(0x7f3e5009d920 msgr2=0x7f3e500d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 shutdown_connections 2024-04-24T06:28:51.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.190+0000 7f3e4fe00640 1 -- 172.21.15.59:0/3572505326 wait complete. 2024-04-24T06:28:51.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T06:28:51.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T06:28:51.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T06:28:51.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T06:28:51.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T06:28:51.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T06:28:51.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T06:28:51.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.518+0000 7fdb9d200640 1 Processor -- start 2024-04-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.519+0000 7fdb9d200640 1 -- start start 2024-04-24T06:28:51.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.520+0000 7f8fee800640 1 Processor -- start 2024-04-24T06:28:51.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.520+0000 7fdb9d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb98161de0 0x7fdb980b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.521+0000 7fdb9d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb980b5530 con 0x7fdb980d32b0 2024-04-24T06:28:51.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.521+0000 7fdb9d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb98163010 con 0x7fdb981624a0 2024-04-24T06:28:51.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.521+0000 7fdb9d200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdb980d26a0 con 0x7fdb98161de0 2024-04-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.522+0000 7fdb96a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fdb981624a0 0x7fdb980d2ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44398/0 (socket says 172.21.15.59:44398) 2024-04-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.521+0000 7f8fee800640 1 -- start start 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.522+0000 7fdb97400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb98161de0 0x7fdb980b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.522+0000 7fdb97e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdb980d32b0 0x7fdb980cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59900/0 (socket says 172.21.15.59:59900) 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.522+0000 7fdb97e00640 1 -- 172.21.15.59:0/389487337 learned_addr learned my addr 172.21.15.59:0/389487337 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.523+0000 7f8fee800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fe00061f0 0x7f8fe00066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.523+0000 7f8fee800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8fe0006be0 0x7f8fe003c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.523+0000 7f8fee800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe00e36e0 con 0x7f8fe00e3300 2024-04-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.524+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2480290973 0 0) 0x7fdb980b5530 con 0x7fdb980d32b0 2024-04-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.523+0000 7f8fee800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe0057b40 con 0x7f8fe00061f0 2024-04-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.523+0000 7f8fee800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe0057e20 con 0x7f8fe0006be0 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.524+0000 7f8fed400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8fe00e3300 0x7f8fe00572f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59912/0 (socket says 172.21.15.59:59912) 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.524+0000 7f8fed400640 1 -- 172.21.15.59:0/512254895 learned_addr learned my addr 172.21.15.59:0/512254895 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.524+0000 7f8feca00640 1 --2- 172.21.15.59:0/512254895 >> v2:172.21.15.59:3300/0 conn(0x7f8fe00061f0 0x7f8fe00066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.524+0000 7f8fede00640 1 --2- 172.21.15.59:0/512254895 >> v2:172.21.15.66:3300/0 conn(0x7f8fe0006be0 0x7f8fe003c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb7c003520 con 0x7fdb980d32b0 2024-04-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7f8feca00640 1 -- 172.21.15.59:0/512254895 >> v2:172.21.15.66:3300/0 conn(0x7f8fe0006be0 msgr2=0x7f8fe003c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7f8fe7e00640 1 -- 172.21.15.59:0/512254895 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193003673 0 0) 0x7f8fe00e36e0 con 0x7f8fe00e3300 2024-04-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7f8feca00640 1 --2- 172.21.15.59:0/512254895 >> v2:172.21.15.66:3300/0 conn(0x7f8fe0006be0 0x7f8fe003c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7f8feca00640 1 -- 172.21.15.59:0/512254895 >> v1:172.21.15.59:6789/0 conn(0x7f8fe00e3300 legacy=0x7f8fe00572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7f8feca00640 1 -- 172.21.15.59:0/512254895 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe0058100 con 0x7f8fe00061f0 2024-04-24T06:28:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2603228663 0 0) 0x7fdb98163010 con 0x7fdb981624a0 2024-04-24T06:28:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb980b5530 con 0x7fdb981624a0 2024-04-24T06:28:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb97400640 1 -- 172.21.15.59:0/389487337 >> v1:172.21.15.66:6789/0 conn(0x7fdb981624a0 legacy=0x7fdb980d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb97400640 1 -- 172.21.15.59:0/389487337 >> v1:172.21.15.59:6789/0 conn(0x7fdb980d32b0 legacy=0x7fdb980cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.525+0000 7fdb97400640 1 -- 172.21.15.59:0/389487337 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980b4510 con 0x7fdb98161de0 2024-04-24T06:28:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.527+0000 7f8feca00640 1 --2- 172.21.15.59:0/512254895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe00061f0 0x7f8fe00066a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00387d0 tx=0x7f8fdc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a2ffa451ef6f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.527+0000 7fdb97400640 1 --2- 172.21.15.59:0/389487337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb98161de0 0x7fdb980b4ff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb840097a0 tx=0x7fdb8402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddcf7e094baec0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7f8fe7e00640 1 -- 172.21.15.59:0/512254895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc037070 con 0x7f8fe00061f0 2024-04-24T06:28:51.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84037070 con 0x7fdb98161de0 2024-04-24T06:28:51.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb84032440 con 0x7fdb98161de0 2024-04-24T06:28:51.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7fdb96000640 1 -- 172.21.15.59:0/389487337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84032810 con 0x7fdb98161de0 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7f8fe7e00640 1 -- 172.21.15.59:0/512254895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fdc032440 con 0x7f8fe00061f0 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.528+0000 7f8fe7e00640 1 -- 172.21.15.59:0/512254895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc032810 con 0x7f8fe00061f0 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 -- 172.21.15.59:0/512254895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe00061f0 msgr2=0x7f8fe00066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 --2- 172.21.15.59:0/512254895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe00061f0 0x7f8fe00066a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00387d0 tx=0x7f8fdc02a130 comp rx=0 tx=0).stop 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 -- 172.21.15.59:0/512254895 shutdown_connections 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 --2- 172.21.15.59:0/512254895 >> v2:172.21.15.66:3300/0 conn(0x7f8fe0006be0 0x7f8fe003c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 --2- 172.21.15.59:0/512254895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe00061f0 0x7f8fe00066a0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00387d0 tx=0x7f8fdc02a130 comp rx=0 tx=0).stop 2024-04-24T06:28:51.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.529+0000 7f8fee800640 1 -- 172.21.15.59:0/512254895 >> 172.21.15.59:0/512254895 conn(0x7f8fe0058ef0 msgr2=0x7f8fe0059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 -- 172.21.15.59:0/389487337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb98161de0 msgr2=0x7fdb980b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 --2- 172.21.15.59:0/389487337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb98161de0 0x7fdb980b4ff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb840097a0 tx=0x7fdb8402a210 comp rx=0 tx=0).stop 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 -- 172.21.15.59:0/389487337 shutdown_connections 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 --2- 172.21.15.59:0/389487337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb98161de0 0x7fdb980b4ff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 -- 172.21.15.59:0/389487337 >> 172.21.15.59:0/389487337 conn(0x7fdb9809d930 msgr2=0x7fdb9808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 -- 172.21.15.59:0/389487337 shutdown_connections 2024-04-24T06:28:51.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7f8fee800640 1 -- 172.21.15.59:0/512254895 shutdown_connections 2024-04-24T06:28:51.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7fdb9d200640 1 -- 172.21.15.59:0/389487337 wait complete. 2024-04-24T06:28:51.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.530+0000 7f8fee800640 1 -- 172.21.15.59:0/512254895 wait complete. 2024-04-24T06:28:51.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.531+0000 7f3b9ce00640 1 Processor -- start 2024-04-24T06:28:51.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.531+0000 7f3b9ce00640 1 -- start start 2024-04-24T06:28:51.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7fdb9d200640 1 Processor -- start 2024-04-24T06:28:51.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f3b9ce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3b98159dd0 0x7f3b980d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f8fee800640 1 Processor -- start 2024-04-24T06:28:51.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7fdb9d200640 1 -- start start 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f3b9ce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980d0890 con 0x7f3b980b5710 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f3b9ce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b980b3df0 con 0x7f3b98070640 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f3b9ce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3b980b3ba0 con 0x7f3b98159dd0 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.533+0000 7f3b96800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3b980b5710 0x7f3b9806e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59918/0 (socket says 172.21.15.59:59918) 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.534+0000 7f3b96800640 1 -- 172.21.15.59:0/1713696106 learned_addr learned my addr 172.21.15.59:0/1713696106 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.534+0000 7f3b95e00640 1 --2- 172.21.15.59:0/1713696106 >> v2:172.21.15.66:3300/0 conn(0x7f3b98159dd0 0x7f3b980d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.534+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724458434 0 0) 0x7f3b980d0890 con 0x7f3b980b5710 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b7c003520 con 0x7f3b980b5710 2024-04-24T06:28:51.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.534+0000 7f8fee800640 1 -- start start 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 110441474 0 0) 0x7f3b7c003520 con 0x7f3b980b5710 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 >> v2:172.21.15.66:3300/0 conn(0x7f3b98159dd0 msgr2=0x7f3b980d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 --2- 172.21.15.59:0/1713696106 >> v2:172.21.15.66:3300/0 conn(0x7f3b98159dd0 0x7f3b980d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 >> v1:172.21.15.66:6789/0 conn(0x7f3b98070640 legacy=0x7f3b980b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b980b40d0 con 0x7f3b980b5710 2024-04-24T06:28:51.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.535+0000 7f0418a00640 1 Processor -- start 2024-04-24T06:28:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f3814600640 1 Processor -- start 2024-04-24T06:28:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f8fee800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe0006be0 0x7f8fe00e5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f321e400640 1 Processor -- start 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f3814600640 1 -- start start 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f8fee800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8fe00e3300 0x7f8fe00e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f8fee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 0x7f8fe00e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f8fee800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0058be0 con 0x7f8fe0006be0 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f8fee800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0006fc0 con 0x7f8fe00de550 2024-04-24T06:28:51.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f8fee800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe0052110 con 0x7f8fe00e3300 2024-04-24T06:28:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f321e400640 1 -- start start 2024-04-24T06:28:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7fdb9d200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 0x7fdb98151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7fdb9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb981624a0 0x7fdb9815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.536+0000 7f0418a00640 1 -- start start 2024-04-24T06:28:51.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f0418a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04140d0f80 0x7f04140cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f0418a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f041415ea40 0x7f041415f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f0418a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f0414162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0418a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f041415fdc0 con 0x7f04140d0f80 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0418a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f04140cf930 con 0x7f041415ea40 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0418a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04140cfc10 con 0x7f04140d1360 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7fdb97400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 0x7fdb98151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.537+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f3b900019f0 con 0x7f3b980b5710 2024-04-24T06:28:51.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 >> v1:172.21.15.59:6789/0 conn(0x7f3b980b5710 legacy=0x7f3b9806e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f8fede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 0x7f8fe00e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f8fede00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 0x7f8fe00e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41154/0 (socket says 172.21.15.59:41154) 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f8fede00640 1 -- 172.21.15.59:0/1392381331 learned_addr learned my addr 172.21.15.59:0/1392381331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f8feca00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8fe00e3300 0x7f8fe00e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0412800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04140d0f80 0x7f04140cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0412800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04140d0f80 0x7f04140cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37850/0 (socket says 172.21.15.59:37850) 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0412800640 1 -- 172.21.15.59:0/2073696483 learned_addr learned my addr 172.21.15.59:0/2073696483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3814600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f380c15e9c0 0x7f380c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3814600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3814600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380c0cf3c0 con 0x7f380c0d2d70 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3814600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f380c0cf640 con 0x7f380c15e9c0 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3814600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f380c0cf900 con 0x7f380c0d1800 2024-04-24T06:28:51.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7fdb97400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 0x7fdb98151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55864/0 (socket says 172.21.15.59:55864) 2024-04-24T06:28:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f8fed400640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe0006be0 0x7f8fe00e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0411e00640 1 --2- 172.21.15.59:0/2073696483 >> v2:172.21.15.66:3300/0 conn(0x7f041415ea40 0x7f041415f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb9d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb980ce160 0x7fdb9815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f9daf200640 1 Processor -- start 2024-04-24T06:28:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f0413200640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f0414162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f8fede00640 1 -- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8fe00e3300 msgr2=0x7f8fe00e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb9d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981621c0 con 0x7fdb981624a0 2024-04-24T06:28:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb9d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980b4ad0 con 0x7fdb980ce160 2024-04-24T06:28:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb9d200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdb981632f0 con 0x7fdb980d32b0 2024-04-24T06:28:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb97400640 1 -- 172.21.15.59:0/2643849917 learned_addr learned my addr 172.21.15.59:0/2643849917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 0x7f3b7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f3811c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f380c15e9c0 0x7f380c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f3811c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f380c15e9c0 0x7f380c0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37864/0 (socket says 172.21.15.59:37864) 2024-04-24T06:28:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f8fede00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8fe00e3300 0x7f8fe00e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f8fede00640 1 -- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe0006be0 msgr2=0x7f8fe00e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f3811c00640 1 -- 172.21.15.59:0/1990335362 learned_addr learned my addr 172.21.15.59:0/1990335362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c004360 0x7f3b7c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b7c005020 0x7f3b7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980b3ba0 con 0x7f3b7c0039b0 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00bf60 con 0x7f3b7c004360 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.538+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c00c160 con 0x7f3b7c005020 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f3812600640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7fdb96a00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb981624a0 0x7fdb9815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f8fede00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe0006be0 0x7f8fe00e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f8fede00640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe00523f0 con 0x7f8fe00de550 2024-04-24T06:28:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7fdb97e00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb980ce160 0x7fdb9815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f9daf200640 1 -- start start 2024-04-24T06:28:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7fa0bb800640 1 Processor -- start 2024-04-24T06:28:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f321e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3218071d90 0x7f32180cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7fa0bb800640 1 -- start start 2024-04-24T06:28:51.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3b95e00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 0x7f3b7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f8fed400640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fe0006be0 0x7f8fe00e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:51.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f321e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218072880 0x7f321806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f0412800640 1 -- 172.21.15.59:0/2073696483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 msgr2=0x7f0414162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f0412800640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f0414162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f0412800640 1 -- 172.21.15.59:0/2073696483 >> v2:172.21.15.66:3300/0 conn(0x7f041415ea40 msgr2=0x7f041415f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3b96800640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b7c005020 0x7f3b7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f0412800640 1 --2- 172.21.15.59:0/2073696483 >> v2:172.21.15.66:3300/0 conn(0x7f041415ea40 0x7f041415f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.539+0000 7f0412800640 1 -- 172.21.15.59:0/2073696483 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140cfef0 con 0x7f04140d0f80 2024-04-24T06:28:51.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f321e400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321806e040 con 0x7f32180b3430 2024-04-24T06:28:51.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb97400640 1 -- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb980ce160 msgr2=0x7fdb9815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f321e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f321806ddf0 con 0x7f3218071d90 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f321e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f321806e320 con 0x7f3218072880 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3b95e00640 1 -- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b7c005020 msgr2=0x7f3b7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3b95e00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b7c005020 0x7f3b7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3b95e00640 1 -- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c004360 msgr2=0x7f3b7c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3b95e00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c004360 0x7f3b7c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3b95e00640 1 -- 172.21.15.59:0/1713696106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c003520 con 0x7f3b7c0039b0 2024-04-24T06:28:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb97400640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb980ce160 0x7fdb9815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb97400640 1 -- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb981624a0 msgr2=0x7fdb9815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb97400640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb981624a0 0x7fdb9815c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb97400640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980b71e0 con 0x7fdb980d32b0 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3217400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3218071d90 0x7f32180cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f8fede00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 0x7f8fe00e3da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400e820 tx=0x7f8fd400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3217400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3218071d90 0x7f32180cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41184/0 (socket says 172.21.15.59:41184) 2024-04-24T06:28:51.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7fdb96a00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb981624a0 0x7fdb9815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:51.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3217400640 1 -- 172.21.15.59:0/3590346991 learned_addr learned my addr 172.21.15.59:0/3590346991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.541+0000 7f3217e00640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218072880 0x7f321806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3811c00640 1 -- 172.21.15.59:0/1990335362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 msgr2=0x7f380c0d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3811c00640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3811c00640 1 -- 172.21.15.59:0/1990335362 >> v1:172.21.15.59:6789/0 conn(0x7f380c0d2d70 legacy=0x7f380c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.540+0000 7f3811c00640 1 -- 172.21.15.59:0/1990335362 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0cfbe0 con 0x7f380c15e9c0 2024-04-24T06:28:51.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f9daf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9db015ea50 0x7f9db0162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7fdb97400640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 0x7fdb98151ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb840097a0 tx=0x7fdb840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4017070 con 0x7f8fe00de550 2024-04-24T06:28:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3b96800640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b7c005020 0x7f3b7c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3b96800640 1 -- 172.21.15.59:0/1713696106 reap_dead start 2024-04-24T06:28:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f9daf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00b7410 0x7f9db00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd400ee70 con 0x7f8fe00de550 2024-04-24T06:28:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400cb60 con 0x7f8fe00de550 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f9daf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db0162c20 con 0x7f9db00d44b0 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f9daf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9db00cf260 con 0x7f9db015ea50 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f9daf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9db0162f00 con 0x7f9db00b7410 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7fa0bb800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0b40d1840 0x7fa0b41698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7fa0bb800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40d2560 con 0x7fa0b40d05f0 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7fa0bb800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b40d2840 con 0x7fa0b415e890 2024-04-24T06:28:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7fa0bb800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b40d2310 con 0x7fa0b40d1840 2024-04-24T06:28:51.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe00526d0 con 0x7f8fe00de550 2024-04-24T06:28:51.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe0051c80 con 0x7f8fe00de550 2024-04-24T06:28:51.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3811c00640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c15e9c0 0x7f380c0d03e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3800009850 tx=0x7f380002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3715c5909064c912 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3217400640 1 -- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218072880 msgr2=0x7f321806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3217400640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218072880 0x7f321806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3217400640 1 -- 172.21.15.59:0/3590346991 >> v1:172.21.15.66:6789/0 conn(0x7f32180b3430 legacy=0x7f3218072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.542+0000 7f3217400640 1 -- 172.21.15.59:0/3590346991 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321806e600 con 0x7f3218071d90 2024-04-24T06:28:51.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7f0412800640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04140cf250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04140a2450 tx=0x7f040402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3896634ad048d206 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7fa0b8e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa0b415e890 0x7fa0b40cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44414/0 (socket says 172.21.15.59:44414) 2024-04-24T06:28:51.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7fa0b8e00640 1 -- 172.21.15.59:0/183476348 learned_addr learned my addr 172.21.15.59:0/183476348 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7f380be00640 1 -- 172.21.15.59:0/1990335362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800037070 con 0x7f380c15e9c0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7fa0b9800640 1 --2- 172.21.15.59:0/183476348 >> v2:172.21.15.59:3300/0 conn(0x7fa0b40d1840 0x7fa0b41698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84037040 con 0x7fdb980d32b0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb84004230 con 0x7fdb980d32b0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84006e10 con 0x7fdb980d32b0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb980b74a0 con 0x7fdb980d32b0 2024-04-24T06:28:51.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb980b7a60 con 0x7fdb980d32b0 2024-04-24T06:28:51.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fe016bc10 con 0x7f8fe00de550 2024-04-24T06:28:51.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3b95e00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 0x7f3b7c003d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009ac0 tx=0x7f3b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baa093301de2cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84036070 con 0x7f3b7c0039b0 2024-04-24T06:28:51.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b84031400 con 0x7f3b7c0039b0 2024-04-24T06:28:51.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3b94a00640 1 -- 172.21.15.59:0/1713696106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84009690 con 0x7f3b7c0039b0 2024-04-24T06:28:51.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7f9dae800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9db00d44b0 0x7f9db00d4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59950/0 (socket says 172.21.15.59:59950) 2024-04-24T06:28:51.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.543+0000 7f9dae800640 1 -- 172.21.15.59:0/718626994 learned_addr learned my addr 172.21.15.59:0/718626994 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f9dad400640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00b7410 0x7f9db00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f9dade00640 1 --2- 172.21.15.59:0/718626994 >> v2:172.21.15.59:3300/0 conn(0x7f9db015ea50 0x7f9db0162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f380be00640 1 -- 172.21.15.59:0/1990335362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800032440 con 0x7f380c15e9c0 2024-04-24T06:28:51.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f380be00640 1 -- 172.21.15.59:0/1990335362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000327f0 con 0x7f380c15e9c0 2024-04-24T06:28:51.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fa0b9800640 1 -- 172.21.15.59:0/183476348 >> v1:172.21.15.66:6789/0 conn(0x7fa0b415e890 legacy=0x7fa0b40cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fa0b9800640 1 -- 172.21.15.59:0/183476348 >> v1:172.21.15.59:6789/0 conn(0x7fa0b40d05f0 legacy=0x7fa0b40d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7fa0b9800640 1 -- 172.21.15.59:0/183476348 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b40d2b20 con 0x7fa0b40d1840 2024-04-24T06:28:51.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f9dade00640 1 -- 172.21.15.59:0/718626994 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00b7410 msgr2=0x7f9db00b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f0411400640 1 -- 172.21.15.59:0/2073696483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404037070 con 0x7f04140d0f80 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f0411400640 1 -- 172.21.15.59:0/2073696483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404032440 con 0x7f04140d0f80 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f0411400640 1 -- 172.21.15.59:0/2073696483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404032810 con 0x7f04140d0f80 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 -- 172.21.15.59:0/2073696483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 msgr2=0x7f04140cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04140cf250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04140a2450 tx=0x7f040402a140 comp rx=0 tx=0).stop 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3b9ce00640 1 -- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 msgr2=0x7f3b7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3b9ce00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 0x7f3b7c003d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009ac0 tx=0x7f3b7c005830 comp rx=0 tx=0).stop 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3b9ce00640 1 -- 172.21.15.59:0/1713696106 shutdown_connections 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3b9ce00640 1 --2- 172.21.15.59:0/1713696106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b7c0039b0 0x7f3b7c003d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3b9ce00640 1 -- 172.21.15.59:0/1713696106 >> 172.21.15.59:0/1713696106 conn(0x7f3b98073400 msgr2=0x7f3b9806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3b9ce00640 1 -- 172.21.15.59:0/1713696106 shutdown_connections 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3b9ce00640 1 -- 172.21.15.59:0/1713696106 wait complete. 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f9dade00640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00b7410 0x7f9db00b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f9dade00640 1 -- 172.21.15.59:0/718626994 >> v1:172.21.15.59:6789/0 conn(0x7f9db00d44b0 legacy=0x7f9db00d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f9dade00640 1 -- 172.21.15.59:0/718626994 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db01631e0 con 0x7f9db015ea50 2024-04-24T06:28:51.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3814600640 1 -- 172.21.15.59:0/1990335362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c15e9c0 msgr2=0x7f380c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.544+0000 7f3814600640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c15e9c0 0x7f380c0d03e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3800009850 tx=0x7f380002a180 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3814600640 1 -- 172.21.15.59:0/1990335362 shutdown_connections 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3814600640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3814600640 1 --2- 172.21.15.59:0/1990335362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c15e9c0 0x7f380c0d03e0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3800009850 tx=0x7f380002a180 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f3814600640 1 -- 172.21.15.59:0/1990335362 >> 172.21.15.59:0/1990335362 conn(0x7f380c08bf90 msgr2=0x7f380c0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 -- 172.21.15.59:0/2073696483 shutdown_connections 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f0414162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 --2- 172.21.15.59:0/2073696483 >> v2:172.21.15.66:3300/0 conn(0x7f041415ea40 0x7f041415f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 --2- 172.21.15.59:0/2073696483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04140cf250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 -- 172.21.15.59:0/2073696483 >> 172.21.15.59:0/2073696483 conn(0x7f041408bf90 msgr2=0x7f04140d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.545+0000 7f0418a00640 1 -- 172.21.15.59:0/2073696483 shutdown_connections 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f0418a00640 1 -- 172.21.15.59:0/2073696483 wait complete. 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fd401a070 con 0x7f8fe00de550 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7f3217400640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218071d90 0x7f32180cd1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3208009870 tx=0x7f320802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93e1e655585850b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f3216000640 1 -- 172.21.15.59:0/3590346991 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208037070 con 0x7f3218071d90 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f3216000640 1 -- 172.21.15.59:0/3590346991 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208032440 con 0x7f3218071d90 2024-04-24T06:28:51.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f3216000640 1 -- 172.21.15.59:0/3590346991 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208032810 con 0x7f3218071d90 2024-04-24T06:28:51.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7fa0b9800640 1 --2- 172.21.15.59:0/183476348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 0x7fa0b41698c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097d0 tx=0x7fa0a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3ba4a3227e56f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7fa0b3400640 1 -- 172.21.15.59:0/183476348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0037070 con 0x7fa0b40d1840 2024-04-24T06:28:51.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7fa0b3400640 1 -- 172.21.15.59:0/183476348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0032440 con 0x7fa0b40d1840 2024-04-24T06:28:51.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f3814600640 1 -- 172.21.15.59:0/1990335362 shutdown_connections 2024-04-24T06:28:51.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7f3814600640 1 -- 172.21.15.59:0/1990335362 wait complete. 2024-04-24T06:28:51.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.546+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb70001030 con 0x7fdb980d32b0 2024-04-24T06:28:51.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.547+0000 7fa0b3400640 1 -- 172.21.15.59:0/183476348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0032810 con 0x7fa0b40d1840 2024-04-24T06:28:51.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f9dade00640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db015ea50 0x7f9db0162640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0097a0 tx=0x7f9d9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c12436135e7067 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7fa0bb800640 1 -- 172.21.15.59:0/183476348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 msgr2=0x7fa0b41698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f8fe5600640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fc806c940 0x7f8fc806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fd408bf60 con 0x7f8fe00de550 2024-04-24T06:28:51.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 -- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218071d90 msgr2=0x7f32180cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218071d90 0x7f32180cd1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3208009870 tx=0x7f320802a270 comp rx=0 tx=0).stop 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f9daca00640 1 -- 172.21.15.59:0/718626994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c037070 con 0x7f9db015ea50 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f9daca00640 1 -- 172.21.15.59:0/718626994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c032440 con 0x7f9db015ea50 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f9daca00640 1 -- 172.21.15.59:0/718626994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c032810 con 0x7f9db015ea50 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7fa0bb800640 1 --2- 172.21.15.59:0/183476348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 0x7fa0b41698c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097d0 tx=0x7fa0a002a210 comp rx=0 tx=0).stop 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f8fe5600640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7f8fc8074bc0 0x7f8fc8076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fc8077690 con 0x7f8fc8074bc0 2024-04-24T06:28:51.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8fd408c440 con 0x7f8fe00de550 2024-04-24T06:28:51.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 -- 172.21.15.59:0/3590346991 shutdown_connections 2024-04-24T06:28:51.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218072880 0x7f321806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 --2- 172.21.15.59:0/3590346991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218071d90 0x7f32180cd1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.548+0000 7f321e400640 1 -- 172.21.15.59:0/3590346991 >> 172.21.15.59:0/3590346991 conn(0x7f32180b2130 msgr2=0x7f32180b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f8fed400640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fc806c940 0x7f8fc806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f321e400640 1 -- 172.21.15.59:0/3590346991 shutdown_connections 2024-04-24T06:28:51.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f3814600640 1 Processor -- start 2024-04-24T06:28:51.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f3814600640 1 -- start start 2024-04-24T06:28:51.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f0418a00640 1 Processor -- start 2024-04-24T06:28:51.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f8feca00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7f8fc8074bc0 0x7f8fc8076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f8fed400640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fc806c940 0x7f8fc806ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000b60 tx=0x7f8fe8069f40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f3b9ce00640 1 Processor -- start 2024-04-24T06:28:51.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f0418a00640 1 -- start start 2024-04-24T06:28:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f8feca00640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7f8fc8074bc0 0x7f8fc8076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f9daf200640 1 -- 172.21.15.59:0/718626994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db015ea50 msgr2=0x7f9db0162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f9daf200640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db015ea50 0x7f9db0162640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0097a0 tx=0x7f9d9c02a180 comp rx=0 tx=0).stop 2024-04-24T06:28:51.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f3b9ce00640 1 -- start start 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.549+0000 7f321e400640 1 -- 172.21.15.59:0/3590346991 wait complete. 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f3814600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7f3814600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c0d2d70 0x7f380c0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f3814600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f380c0caaf0 0x7f380c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f3814600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f380c06cd10 con 0x7f380c0d2d70 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f3814600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f380c06cb90 con 0x7f380c0caaf0 2024-04-24T06:28:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f3814600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f380c06ce90 con 0x7f380c0d1800 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f0418a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04141fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f0418a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f041415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f0418a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f041415ea40 0x7f04140cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f0418a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d1810 con 0x7f04140d0f80 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f0412800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04141fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.550+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb8403a070 con 0x7fdb980d32b0 2024-04-24T06:28:51.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.551+0000 7f3811c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 -- 172.21.15.59:0/718626994 shutdown_connections 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00b7410 0x7f9db00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 --2- 172.21.15.59:0/718626994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db015ea50 0x7f9db0162640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 -- 172.21.15.59:0/718626994 >> 172.21.15.59:0/718626994 conn(0x7f9db009d950 msgr2=0x7f9db008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0412800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04141fcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37916/0 (socket says 172.21.15.59:37916) 2024-04-24T06:28:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0412800640 1 -- 172.21.15.59:0/4091184586 learned_addr learned my addr 172.21.15.59:0/4091184586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 -- 172.21.15.59:0/718626994 shutdown_connections 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3811c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55906/0 (socket says 172.21.15.59:55906) 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3811c00640 1 -- 172.21.15.59:0/458191690 learned_addr learned my addr 172.21.15.59:0/458191690 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c0039b0 0x7f3b98144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 0x7f3b98157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b98148da0 0x7f3b98157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980b4970 con 0x7f3b98148da0 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980b4690 con 0x7f3b7c0039b0 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3b9ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9815a1b0 con 0x7f3b98148940 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0411e00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f041415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0413200640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f041415ea40 0x7f04140cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04140d04b0 con 0x7f041415ea40 2024-04-24T06:28:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f3812600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f380c0caaf0 0x7f380c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7fdb8f400640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb7806c9f0 0x7fdb7806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7fa0bb800640 1 -- 172.21.15.59:0/183476348 shutdown_connections 2024-04-24T06:28:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7fa0bb800640 1 --2- 172.21.15.59:0/183476348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 0x7fa0b41698c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7fa0bb800640 1 -- 172.21.15.59:0/183476348 >> 172.21.15.59:0/183476348 conn(0x7fa0b408bf90 msgr2=0x7fa0b40d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04141628b0 con 0x7f04140d1360 2024-04-24T06:28:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb840ac6e0 con 0x7fdb980d32b0 2024-04-24T06:28:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fc8077690 con 0x7f8fc8074bc0 2024-04-24T06:28:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f3b95400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 0x7f3b98157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f3b95400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 0x7f3b98157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55924/0 (socket says 172.21.15.59:55924) 2024-04-24T06:28:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f3b95400640 1 -- 172.21.15.59:0/497438446 learned_addr learned my addr 172.21.15.59:0/497438446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f3811200640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c0d2d70 0x7f380c0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f9daf200640 1 -- 172.21.15.59:0/718626994 wait complete. 2024-04-24T06:28:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fa0bb800640 1 -- 172.21.15.59:0/183476348 shutdown_connections 2024-04-24T06:28:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fdb8f400640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7fdb78074c10 0x7fdb78077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fdb96a00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb7806c9f0 0x7fdb7806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f3b95e00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c0039b0 0x7f3b98144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb840067f0 con 0x7fdb78074c10 2024-04-24T06:28:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb840acad0 con 0x7fdb980d32b0 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3811c00640 1 -- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f380c0caaf0 msgr2=0x7f380c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3811c00640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f380c0caaf0 0x7f380c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3811c00640 1 -- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c0d2d70 msgr2=0x7f380c0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3811c00640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f380c0d2d70 0x7f380c0b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3811c00640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c0cd050 con 0x7f380c0d1800 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3812600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f380c0caaf0 0x7f380c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7fa0bb800640 1 -- 172.21.15.59:0/183476348 wait complete. 2024-04-24T06:28:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7fdb97e00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7fdb78074c10 0x7fdb78077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3b95400640 1 -- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c0039b0 msgr2=0x7f3b98144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3b95400640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b7c0039b0 0x7f3b98144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3b95400640 1 -- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b98148da0 msgr2=0x7f3b98157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3b95400640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b98148da0 0x7f3b98157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3b95400640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9815b170 con 0x7f3b98148940 2024-04-24T06:28:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f321e400640 1 Processor -- start 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.552+0000 7f321e400640 1 -- start start 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218071d90 0x7f3218146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218072880 0x7f3218163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3430 0x7f321814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32180b3a20 con 0x7f32180b3430 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32180b38a0 con 0x7f3218072880 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.553+0000 7f321e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3218166430 con 0x7f3218071d90 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3216a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218072880 0x7f3218163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3216a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218072880 0x7f3218163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41228/0 (socket says 172.21.15.59:41228) 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3216a00640 1 -- 172.21.15.59:0/3564193863 learned_addr learned my addr 172.21.15.59:0/3564193863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.554+0000 7f3217400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218071d90 0x7f3218146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f9daf200640 1 Processor -- start 2024-04-24T06:28:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f9daf200640 1 -- start start 2024-04-24T06:28:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7fdb97e00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7fdb78074c10 0x7fdb78077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f3b95400640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 0x7f3b98157880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c450 tx=0x7f3b8800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0413200640 1 -- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 msgr2=0x7f041415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0413200640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04140d1360 0x7f041415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0413200640 1 -- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 msgr2=0x7f04141fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0413200640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04141fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0413200640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041415b020 con 0x7f041415ea40 2024-04-24T06:28:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f3811c00640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0cca20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3800009970 tx=0x7f38000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.555+0000 7f0412800640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04140d0f80 0x7f04141fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88017070 con 0x7f3b98148940 2024-04-24T06:28:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b880035f0 con 0x7f3b98148940 2024-04-24T06:28:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880039e0 con 0x7f3b98148940 2024-04-24T06:28:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3217400640 1 -- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218072880 msgr2=0x7f3218163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3217400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3218072880 0x7f3218163800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3217400640 1 -- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3430 msgr2=0x7f321814b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3217400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3430 0x7f321814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3217400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218163d40 con 0x7f3218071d90 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f0413200640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f041415ea40 0x7f04140cd9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f040c00efc0 tx=0x7f040c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800037040 con 0x7f380c0d1800 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800005990 con 0x7f380c0d1800 2024-04-24T06:28:51.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800005d80 con 0x7f380c0d1800 2024-04-24T06:28:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7fdb96a00640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb7806c9f0 0x7fdb7806eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb980caf40 tx=0x7fdb88007c80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb840067f0 con 0x7fdb78074c10 2024-04-24T06:28:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9815bba0 con 0x7f3b98148940 2024-04-24T06:28:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9815c160 con 0x7f3b98148940 2024-04-24T06:28:51.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f3217400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218071d90 0x7f3218146710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3208031000 tx=0x7f32080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c0cd330 con 0x7f380c0d1800 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c0cd8f0 con 0x7f380c0d1800 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c018020 con 0x7f041415ea40 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c014480 con 0x7f041415ea40 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c014870 con 0x7f041415ea40 2024-04-24T06:28:51.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140cb660 con 0x7f041415ea40 2024-04-24T06:28:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7fa0bb800640 1 Processor -- start 2024-04-24T06:28:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.558+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f380c068a80 con 0x7f380c0d1800 2024-04-24T06:28:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04140ce1d0 con 0x7f041415ea40 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db00b7410 0x7f9db0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00d44b0 0x7f9db00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015ea50 0x7f9db00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9db015ee30 con 0x7f9db00b7410 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00b5e50 con 0x7f9db015ea50 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.556+0000 7f9daf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9db0161320 con 0x7f9db00d44b0 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f9dad400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00d44b0 0x7f9db00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f9dad400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00d44b0 0x7f9db00cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55942/0 (socket says 172.21.15.59:55942) 2024-04-24T06:28:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.557+0000 7f9dad400640 1 -- 172.21.15.59:0/3781597803 learned_addr learned my addr 172.21.15.59:0/3781597803 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.558+0000 7f9dade00640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db00b7410 0x7f9db0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.558+0000 7f9dae800640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015ea50 0x7f9db00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.558+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b9804fd90 con 0x7f3b98148940 2024-04-24T06:28:51.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 -- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00d44b0 msgr2=0x7f9db00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9db00d44b0 0x7f9db00cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03ec001030 con 0x7f041415ea40 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 -- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015ea50 msgr2=0x7f9db00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015ea50 0x7f9db00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db0161600 con 0x7f9db00b7410 2024-04-24T06:28:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7fa0bb800640 1 -- start start 2024-04-24T06:28:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dae800640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015ea50 0x7f9db00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:51.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f9dade00640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db00b7410 0x7f9db0160d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c031000 tx=0x7f9d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.559+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208037040 con 0x7f3218071d90 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208030500 con 0x7f3218071d90 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080308f0 con 0x7f3218071d90 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321814a500 con 0x7f3218071d90 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f321814aac0 con 0x7f3218071d90 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c040020 con 0x7f9db00b7410 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db01618e0 con 0x7f9db00b7410 2024-04-24T06:28:51.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db00ce440 con 0x7f9db00b7410 2024-04-24T06:28:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.561+0000 7fa0bb800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b40d05f0 0x7fa0b4150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.561+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31f0001030 con 0x7f3218071d90 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.561+0000 7fa0bb800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 0x7fa0b415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.561+0000 7fa0bb800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 0x7fa0b415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.561+0000 7fa0bb800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b415ec70 con 0x7fa0b40d1840 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7fa0bb800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40d30e0 con 0x7fa0b40d05f0 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7fa0bb800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b415edf0 con 0x7fa0b415e890 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c038040 con 0x7f9db00b7410 2024-04-24T06:28:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.560+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c037070 con 0x7f9db00b7410 2024-04-24T06:28:51.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7fa0b9800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 0x7fa0b415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7fa0b9800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 0x7fa0b415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55956/0 (socket says 172.21.15.59:55956) 2024-04-24T06:28:51.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7fa0b9800640 1 -- 172.21.15.59:0/2722325145 learned_addr learned my addr 172.21.15.59:0/2722325145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.562+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d88001030 con 0x7f9db00b7410 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b8e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b40d05f0 0x7fa0b4150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b9800640 1 -- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b40d05f0 msgr2=0x7fa0b4150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b9800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b40d05f0 0x7fa0b4150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b9800640 1 -- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 msgr2=0x7fa0b415cd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b9800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b40d1840 0x7fa0b415cd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b9800640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b4156e30 con 0x7fa0b415e890 2024-04-24T06:28:51.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.563+0000 7fa0b8e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b40d05f0 0x7fa0b4150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:51.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.564+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9c043020 con 0x7f9db00b7410 2024-04-24T06:28:51.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.564+0000 7fa0b9800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 0x7fa0b415d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002a800 tx=0x7fa0a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.564+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3800006030 con 0x7f380c0d1800 2024-04-24T06:28:51.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.564+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f040c005d10 con 0x7f041415ea40 2024-04-24T06:28:51.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0037040 con 0x7fa0b415e890 2024-04-24T06:28:51.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0004230 con 0x7fa0b415e890 2024-04-24T06:28:51.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0030670 con 0x7fa0b415e890 2024-04-24T06:28:51.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b8801a070 con 0x7f3b98148940 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f3809600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37ec06c9c0 0x7f37ec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38000ac510 con 0x7f380c0d1800 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b40ce520 con 0x7fa0b415e890 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b4157380 con 0x7fa0b415e890 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f9da6000640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d9006c8f0 0x7f9d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9c0abbb0 con 0x7f9db00b7410 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f9da6000640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7f9d90074a90 0x7f9d90076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d9c034ec0 con 0x7f9d90074a90 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c0b02a0 con 0x7f9db00b7410 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f9dae800640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7f9d90074a90 0x7f9d90076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.565+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3208044020 con 0x7f3218071d90 2024-04-24T06:28:51.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f3809600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f37ec074be0 0x7f37ec077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f380000af30 con 0x7f37ec074be0 2024-04-24T06:28:51.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.566+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f38000ac900 con 0x7f380c0d1800 2024-04-24T06:28:51.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f9dad400640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d9006c8f0 0x7f9d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f040aa00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f03f406c970 0x7f03f406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f040c08d0d0 con 0x7f041415ea40 2024-04-24T06:28:51.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f3b8e000640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3b7806c9c0 0x7f3b7806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f9dae800640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7f9d90074a90 0x7f9d90076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b8808b420 con 0x7f3b98148940 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3b8e000640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7f3b78074be0 0x7f3b78077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b8801be50 con 0x7f3b78074be0 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f320f400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31f806c8f0 0x7f31f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32080ac6b0 con 0x7f3218071d90 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f320f400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f31f8074b10 0x7f31f8076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f320804fea0 con 0x7f31f8074b10 2024-04-24T06:28:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f32080acaa0 con 0x7f3218071d90 2024-04-24T06:28:51.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3b8808b810 con 0x7f3b98148940 2024-04-24T06:28:51.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f040aa00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f03f4074b90 0x7f03f4076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03f4077660 con 0x7f03f4074b90 2024-04-24T06:28:51.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f0412800640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f03f406c970 0x7f03f406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f040c08d480 con 0x7f041415ea40 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0b4157760 con 0x7fa0b415e890 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3216a00640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31f806c8f0 0x7f31f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.567+0000 7f3812600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f37ec074be0 0x7f37ec077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3811200640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37ec06c9c0 0x7f37ec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3812600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f37ec074be0 0x7f37ec077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.568+0000 7f3b95e00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3b7806c9c0 0x7f3b7806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f0411e00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f03f4074b90 0x7f03f4076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f3b96800640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7f3b78074be0 0x7f3b78077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f3217e00640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f31f8074b10 0x7f31f8076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f9dad400640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d9006c8f0 0x7f9d9006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9db00ca350 tx=0x7f9da800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f3811200640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37ec06c9c0 0x7f37ec06ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f380c155500 tx=0x7f37f8007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.569+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d9c034ec0 con 0x7f9d90074a90 2024-04-24T06:28:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.570+0000 7f3b95e00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3b7806c9c0 0x7f3b7806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3b84033000 tx=0x7f3b84008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.570+0000 7f0411e00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f03f4074b90 0x7f03f4076fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.570+0000 7f0412800640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f03f406c970 0x7f03f406ee30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f040402a180 tx=0x7f040402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.570+0000 7f3b96800640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7f3b78074be0 0x7f3b78077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.570+0000 7f3217e00640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f31f8074b10 0x7f31f8076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.571+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0a0030810 con 0x7fa0b415e890 2024-04-24T06:28:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f380000af30 con 0x7f37ec074be0 2024-04-24T06:28:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7f3216a00640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31f806c8f0 0x7f31f806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3218162f40 tx=0x7f3204009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03f4077660 con 0x7f03f4074b90 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7fa0b0c00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa09406c940 0x7fa09406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0a00ad680 con 0x7fa0b415e890 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7fa0b0c00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fa094074bc0 0x7fa094076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0a0006810 con 0x7fa094074bc0 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.572+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa0a00ada30 con 0x7fa0b415e890 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.573+0000 7fa0b8e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa09406c940 0x7fa09406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.573+0000 7fa0b3e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fa094074bc0 0x7fa094076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.574+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b8801be50 con 0x7f3b78074be0 2024-04-24T06:28:51.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.575+0000 7fa0b8e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa09406c940 0x7fa09406ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005c00 tx=0x7fa0ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.576+0000 7fa0b3e00640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fa094074bc0 0x7fa094076fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.576+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f320804fea0 con 0x7f31f8074b10 2024-04-24T06:28:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.578+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0a0006810 con 0x7fa094074bc0 2024-04-24T06:28:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.586+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fe0005710 con 0x7f8fc8074bc0 2024-04-24T06:28:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.587+0000 7f8fe5600640 1 -- 172.21.15.59:0/1392381331 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8fe0005710 con 0x7f8fc8074bc0 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7f8fc8074bc0 msgr2=0x7f8fc8076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7f8fc8074bc0 0x7f8fc8076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fc806c940 msgr2=0x7f8fc806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fc806c940 0x7f8fc806ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000b60 tx=0x7f8fe8069f40 comp rx=0 tx=0).stop 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 msgr2=0x7f8fe00e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 --2- 172.21.15.59:0/1392381331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fe00de550 0x7f8fe00e3da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400e820 tx=0x7f8fd400ec20 comp rx=0 tx=0).stop 2024-04-24T06:28:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fede00640 1 -- 172.21.15.59:0/1392381331 reap_dead start 2024-04-24T06:28:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb70002dd0 con 0x7fdb78074c10 2024-04-24T06:28:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 shutdown_connections 2024-04-24T06:28:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 >> 172.21.15.59:0/1392381331 conn(0x7f8fe0058ef0 msgr2=0x7f8fe00e25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.594+0000 7fdb8f400640 1 -- 172.21.15.59:0/2643849917 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb70002dd0 con 0x7fdb78074c10 2024-04-24T06:28:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.594+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 shutdown_connections 2024-04-24T06:28:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.594+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7fdb78074c10 msgr2=0x7fdb78077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.594+0000 7fdb9d200640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7fdb78074c10 0x7fdb78077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.594+0000 7f8fee800640 1 -- 172.21.15.59:0/1392381331 wait complete. 2024-04-24T06:28:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.592+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d88002dd0 con 0x7f9d90074a90 2024-04-24T06:28:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.593+0000 7f9da6000640 1 -- 172.21.15.59:0/3781597803 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9d88002dd0 con 0x7f9d90074a90 2024-04-24T06:28:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb7806c9f0 msgr2=0x7fdb7806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb7806c9f0 0x7fdb7806eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb980caf40 tx=0x7fdb88007c80 comp rx=0 tx=0).stop 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 msgr2=0x7fdb98151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 --2- 172.21.15.59:0/2643849917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb980d32b0 0x7fdb98151ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb840097a0 tx=0x7fdb840046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb97e00640 1 -- 172.21.15.59:0/2643849917 reap_dead start 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 shutdown_connections 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7f9d90074a90 msgr2=0x7f9d90076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7f9d90074a90 0x7f9d90076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 >> 172.21.15.59:0/2643849917 conn(0x7fdb9809d930 msgr2=0x7fdb980d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d9006c8f0 msgr2=0x7f9d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d9006c8f0 0x7f9d9006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9db00ca350 tx=0x7f9da800b040 comp rx=0 tx=0).stop 2024-04-24T06:28:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db00b7410 msgr2=0x7f9db0160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.595+0000 7f9daf200640 1 --2- 172.21.15.59:0/3781597803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9db00b7410 0x7f9db0160d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c031000 tx=0x7f9d9c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 shutdown_connections 2024-04-24T06:28:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7f9dae800640 1 -- 172.21.15.59:0/3781597803 reap_dead start 2024-04-24T06:28:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 shutdown_connections 2024-04-24T06:28:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 >> 172.21.15.59:0/3781597803 conn(0x7f9db009d950 msgr2=0x7f9db006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7fdb9d200640 1 -- 172.21.15.59:0/2643849917 wait complete. 2024-04-24T06:28:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 shutdown_connections 2024-04-24T06:28:51.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.596+0000 7f9daf200640 1 -- 172.21.15.59:0/3781597803 wait complete. 2024-04-24T06:28:51.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.606+0000 7f0418a00640 1 -- 172.21.15.59:0/4091184586 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03ec002dd0 con 0x7f03f4074b90 2024-04-24T06:28:51.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.606+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31f0002d50 con 0x7f31f8074b10 2024-04-24T06:28:51.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.607+0000 7f320f400640 1 -- 172.21.15.59:0/3564193863 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f31f0002d50 con 0x7f31f8074b10 2024-04-24T06:28:51.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.607+0000 7f040aa00640 1 -- 172.21.15.59:0/4091184586 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03ec002dd0 con 0x7f03f4074b90 2024-04-24T06:28:51.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.607+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f380c0d2630 con 0x7f37ec074be0 2024-04-24T06:28:51.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f31f8074b10 msgr2=0x7f31f8076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f03f4074b90 msgr2=0x7f03f4076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f03f4074b90 0x7f03f4076fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f03f406c970 msgr2=0x7f03f406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f03f406c970 0x7f03f406ee30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f040402a180 tx=0x7f040402e000 comp rx=0 tx=0).stop 2024-04-24T06:28:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f041415ea40 msgr2=0x7f04140cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f31f8074b10 0x7f31f8076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31f806c8f0 msgr2=0x7f31f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31f806c8f0 0x7f31f806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3218162f40 tx=0x7f3204009040 comp rx=0 tx=0).stop 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218071d90 msgr2=0x7f3218146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f321e400640 1 --2- 172.21.15.59:0/3564193863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3218071d90 0x7f3218146710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3208031000 tx=0x7f32080046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f3217e00640 1 -- 172.21.15.59:0/3564193863 reap_dead start 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 shutdown_connections 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 >> 172.21.15.59:0/3564193863 conn(0x7f32180b2130 msgr2=0x7f3218166720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 shutdown_connections 2024-04-24T06:28:51.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f321e400640 1 -- 172.21.15.59:0/3564193863 wait complete. 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f3809600640 1 -- 172.21.15.59:0/458191690 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f380c0d2630 con 0x7f37ec074be0 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f03ffe00640 1 --2- 172.21.15.59:0/4091184586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f041415ea40 0x7f04140cd9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f040c00efc0 tx=0x7f040c00c490 comp rx=0 tx=0).stop 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.608+0000 7f0413200640 1 -- 172.21.15.59:0/4091184586 reap_dead start 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 shutdown_connections 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 >> 172.21.15.59:0/4091184586 conn(0x7f041408bf90 msgr2=0x7f041415f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 shutdown_connections 2024-04-24T06:28:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f03ffe00640 1 -- 172.21.15.59:0/4091184586 wait complete. 2024-04-24T06:28:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0b4069190 con 0x7fa094074bc0 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f37ec074be0 msgr2=0x7f37ec077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.609+0000 7f3814600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f37ec074be0 0x7f37ec077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37ec06c9c0 msgr2=0x7f37ec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37ec06c9c0 0x7f37ec06ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f380c155500 tx=0x7f37f8007e30 comp rx=0 tx=0).stop 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 msgr2=0x7f380c0cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 --2- 172.21.15.59:0/458191690 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f380c0d1800 0x7f380c0cca20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3800009970 tx=0x7f38000046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3812600640 1 -- 172.21.15.59:0/458191690 reap_dead start 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 shutdown_connections 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.610+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 >> 172.21.15.59:0/458191690 conn(0x7f380c08bf90 msgr2=0x7f380c15f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.611+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 shutdown_connections 2024-04-24T06:28:51.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.611+0000 7f3814600640 1 -- 172.21.15.59:0/458191690 wait complete. 2024-04-24T06:28:51.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.611+0000 7fa0b0c00640 1 -- 172.21.15.59:0/2722325145 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0b4069190 con 0x7fa094074bc0 2024-04-24T06:28:51.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b98067de0 con 0x7f3b78074be0 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fa094074bc0 msgr2=0x7fa094076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fa094074bc0 0x7fa094076fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa09406c940 msgr2=0x7fa09406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa09406c940 0x7fa09406ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005c00 tx=0x7fa0ac009040 comp rx=0 tx=0).stop 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 msgr2=0x7fa0b415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.612+0000 7fa0bb800640 1 --2- 172.21.15.59:0/2722325145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0b415e890 0x7fa0b415d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002a800 tx=0x7fa0a00046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7fa0b9800640 1 -- 172.21.15.59:0/2722325145 reap_dead start 2024-04-24T06:28:51.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 shutdown_connections 2024-04-24T06:28:51.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 >> 172.21.15.59:0/2722325145 conn(0x7fa0b408bf90 msgr2=0x7fa0b40d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 shutdown_connections 2024-04-24T06:28:51.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7fa0bb800640 1 -- 172.21.15.59:0/2722325145 wait complete. 2024-04-24T06:28:51.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.613+0000 7f3b8e000640 1 -- 172.21.15.59:0/497438446 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3b98067de0 con 0x7f3b78074be0 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.614+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7f3b78074be0 msgr2=0x7f3b78077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.614+0000 7f3b9ce00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7f3b78074be0 0x7f3b78077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3b7806c9c0 msgr2=0x7f3b7806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3b7806c9c0 0x7f3b7806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3b84033000 tx=0x7f3b84008030 comp rx=0 tx=0).stop 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 msgr2=0x7f3b98157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 --2- 172.21.15.59:0/497438446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3b98148940 0x7f3b98157880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c450 tx=0x7f3b8800c910 comp rx=0 tx=0).stop 2024-04-24T06:28:51.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b96800640 1 -- 172.21.15.59:0/497438446 reap_dead start 2024-04-24T06:28:51.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 shutdown_connections 2024-04-24T06:28:51.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.615+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 >> 172.21.15.59:0/497438446 conn(0x7f3b98073400 msgr2=0x7f3b9806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.616+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 shutdown_connections 2024-04-24T06:28:51.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.616+0000 7f3b9ce00640 1 -- 172.21.15.59:0/497438446 wait complete. 2024-04-24T06:28:51.643 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:51.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T06:28:51.645 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:51.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T06:28:51.646 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:51.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T06:28:51.647 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:51.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T06:28:51.648 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:51.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T06:28:51.649 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:51.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T06:28:51.650 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:51.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T06:28:51.651 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:51.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T06:28:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.970+0000 7f8ff9a00640 1 Processor -- start 2024-04-24T06:28:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.970+0000 7f8ff9a00640 1 -- start start 2024-04-24T06:28:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.973+0000 7f8ff9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ff40d0c10 0x7f8ff40cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.973+0000 7f8ff9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff40af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.973+0000 7f8ff9a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff41676a0 con 0x7f8ff40d0830 2024-04-24T06:28:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.973+0000 7f8ff9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff4166b20 con 0x7f8ff40d0c10 2024-04-24T06:28:51.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.973+0000 7f8ff9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff4166e00 con 0x7f8ff40af010 2024-04-24T06:28:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.975+0000 7f8ff9000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff40af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.975+0000 7f8ff3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ff40d0c10 0x7f8ff40cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.975+0000 7f8ff3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8ff40d0830 0x7f8ff4163bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59976/0 (socket says 172.21.15.59:59976) 2024-04-24T06:28:51.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.975+0000 7f8ff3e00640 1 -- 172.21.15.59:0/1797061523 learned_addr learned my addr 172.21.15.59:0/1797061523 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.977+0000 7f8ff3400640 1 -- 172.21.15.59:0/1797061523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 msgr2=0x7f8ff40af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.977+0000 7f8ff3400640 1 --2- 172.21.15.59:0/1797061523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff40af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.977+0000 7f8ff3400640 1 -- 172.21.15.59:0/1797061523 >> v1:172.21.15.59:6789/0 conn(0x7f8ff40d0830 legacy=0x7f8ff4163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.977+0000 7f8ff3400640 1 -- 172.21.15.59:0/1797061523 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff41670e0 con 0x7f8ff40d0c10 2024-04-24T06:28:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.979+0000 7f8ff3400640 1 --2- 172.21.15.59:0/1797061523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0c10 0x7f8ff40cefd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009a00 tx=0x7f8fe402a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efaa4dab6d226ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.979+0000 7fb8ea400640 1 Processor -- start 2024-04-24T06:28:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.980+0000 7f8ff2a00640 1 -- 172.21.15.59:0/1797061523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4037070 con 0x7f8ff40d0c10 2024-04-24T06:28:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.980+0000 7f8ff2a00640 1 -- 172.21.15.59:0/1797061523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe4032440 con 0x7f8ff40d0c10 2024-04-24T06:28:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.980+0000 7f8ff2a00640 1 -- 172.21.15.59:0/1797061523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4032850 con 0x7f8ff40d0c10 2024-04-24T06:28:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.980+0000 7fb8ea400640 1 -- start start 2024-04-24T06:28:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.981+0000 7f8ff9a00640 1 -- 172.21.15.59:0/1797061523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0c10 msgr2=0x7f8ff40cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.981+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/1797061523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0c10 0x7f8ff40cefd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009a00 tx=0x7f8fe402a390 comp rx=0 tx=0).stop 2024-04-24T06:28:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 0x7fb8e40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7f8ff9a00640 1 -- 172.21.15.59:0/1797061523 shutdown_connections 2024-04-24T06:28:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/1797061523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff40af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/1797061523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0c10 0x7f8ff40cefd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7f8ff9a00640 1 -- 172.21.15.59:0/1797061523 >> 172.21.15.59:0/1797061523 conn(0x7f8ff406bdf0 msgr2=0x7f8ff406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb8e40af4d0 0x7fb8e40ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb8e40cd0a0 0x7fb8e40cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e40cdd70 con 0x7fb8e40cd0a0 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e40b40b0 con 0x7fb8e40af4d0 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8ea400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e40b4390 con 0x7fb8e4073b10 2024-04-24T06:28:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.982+0000 7fb8e9000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 0x7fb8e40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.983+0000 7f8ff9a00640 1 -- 172.21.15.59:0/1797061523 shutdown_connections 2024-04-24T06:28:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.983+0000 7f8ff9a00640 1 -- 172.21.15.59:0/1797061523 wait complete. 2024-04-24T06:28:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.983+0000 7fb8e9000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 0x7fb8e40b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55978/0 (socket says 172.21.15.59:55978) 2024-04-24T06:28:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.983+0000 7fb8e9000640 1 -- 172.21.15.59:0/2867370047 learned_addr learned my addr 172.21.15.59:0/2867370047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.983+0000 7fb8e9a00640 1 --2- 172.21.15.59:0/2867370047 >> v2:172.21.15.59:3300/0 conn(0x7fb8e40cd0a0 0x7fb8e40cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.984+0000 7fb8e9a00640 1 -- 172.21.15.59:0/2867370047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 msgr2=0x7fb8e40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.984+0000 7fb8e9a00640 1 --2- 172.21.15.59:0/2867370047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 0x7fb8e40b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.984+0000 7fb8e9a00640 1 -- 172.21.15.59:0/2867370047 >> v2:172.21.15.66:3300/0 conn(0x7fb8e40af4d0 msgr2=0x7fb8e40ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.984+0000 7fb8e9a00640 1 --2- 172.21.15.59:0/2867370047 >> v2:172.21.15.66:3300/0 conn(0x7fb8e40af4d0 0x7fb8e40ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.984+0000 7fb8e9a00640 1 -- 172.21.15.59:0/2867370047 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e40b4670 con 0x7fb8e40cd0a0 2024-04-24T06:28:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.986+0000 7fb8e9a00640 1 --2- 172.21.15.59:0/2867370047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e40cd0a0 0x7fb8e40cd530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4075500 tx=0x7fb8dc02a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ff1289203ad543 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.986+0000 7f8ff9a00640 1 Processor -- start 2024-04-24T06:28:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.987+0000 7f8ff9a00640 1 -- start start 2024-04-24T06:28:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.987+0000 7fb8e3400640 1 -- 172.21.15.59:0/2867370047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc031600 con 0x7fb8e40cd0a0 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.987+0000 7fb8e3400640 1 -- 172.21.15.59:0/2867370047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8dc036070 con 0x7fb8e40cd0a0 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.987+0000 7fb8e3400640 1 -- 172.21.15.59:0/2867370047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc031a00 con 0x7fb8e40cd0a0 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 -- 172.21.15.59:0/2867370047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e40cd0a0 msgr2=0x7fb8e40cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 --2- 172.21.15.59:0/2867370047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e40cd0a0 0x7fb8e40cd530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4075500 tx=0x7fb8dc02a870 comp rx=0 tx=0).stop 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 -- 172.21.15.59:0/2867370047 shutdown_connections 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 --2- 172.21.15.59:0/2867370047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e40cd0a0 0x7fb8e40cd530 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 --2- 172.21.15.59:0/2867370047 >> v2:172.21.15.66:3300/0 conn(0x7fb8e40af4d0 0x7fb8e40ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 --2- 172.21.15.59:0/2867370047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e4073b10 0x7fb8e40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 -- 172.21.15.59:0/2867370047 >> 172.21.15.59:0/2867370047 conn(0x7fb8e4072ff0 msgr2=0x7fb8e4073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 -- 172.21.15.59:0/2867370047 shutdown_connections 2024-04-24T06:28:51.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7fb8ea400640 1 -- 172.21.15.59:0/2867370047 wait complete. 2024-04-24T06:28:51.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7f71d6800640 1 Processor -- start 2024-04-24T06:28:51.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.989+0000 7f8ff9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f8ff9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 0x7f8ff41ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7fbc82000640 1 Processor -- start 2024-04-24T06:28:51.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f71d6800640 1 -- start start 2024-04-24T06:28:51.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7fbc82000640 1 -- start start 2024-04-24T06:28:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f8ff3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f8ff3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55994/0 (socket says 172.21.15.59:55994) 2024-04-24T06:28:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f8ff3e00640 1 -- 172.21.15.59:0/2770821305 learned_addr learned my addr 172.21.15.59:0/2770821305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.990+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff40d0c10 0x7f8ff41622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.991+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40b45f0 con 0x7f8ff40d0830 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.991+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40af990 con 0x7f8ff40d0c10 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.991+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff4166740 con 0x7f8ff40af010 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.991+0000 7f8ff9000640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff40d0c10 0x7f8ff41622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.991+0000 7f8ff3400640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 0x7f8ff41ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3e00640 1 -- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff40d0c10 msgr2=0x7f8ff41622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3e00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff40d0c10 0x7f8ff41622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3e00640 1 -- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 msgr2=0x7f8ff41ec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3e00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 0x7f8ff41ec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:51.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3e00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4162830 con 0x7f8ff40af010 2024-04-24T06:28:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7fb8ea400640 1 Processor -- start 2024-04-24T06:28:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7fb8ea400640 1 -- start start 2024-04-24T06:28:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7f8ff3400640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 0x7f8ff41ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7fbc82000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc7c15e9c0 0x7fbc7c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7fbc82000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 0x7fbc7c0d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.992+0000 7fbc82000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc82000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7c0cf3c0 con 0x7fbc7c0d2d70 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc82000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7c0cf640 con 0x7fbc7c15e9c0 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc82000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c0cf900 con 0x7fbc7c0d10f0 2024-04-24T06:28:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:51.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f8ff3e00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c4d0 tx=0x7f8fe000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:51.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56010/0 (socket says 172.21.15.59:56010) 2024-04-24T06:28:51.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71d015e9c0 0x7f71d00d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:51.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f71d00d10f0 0x7f71d00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0017070 con 0x7f8ff40af010 2024-04-24T06:28:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f71d00d3710 con 0x7f71d015e9c0 2024-04-24T06:28:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f71d00cf3c0 con 0x7f71d00d10f0 2024-04-24T06:28:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7f71d6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f71d00cf640 con 0x7f71d00d2d70 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe0005ce0 con 0x7f8ff40af010 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0007270 con 0x7f8ff40af010 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41ecc30 con 0x7f8ff40af010 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41ed1f0 con 0x7f8ff40af010 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 0x7fb8e41488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 0x7fb8e414a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 0x7fb8e4149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e406f340 con 0x7fb8e4073b10 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e40b4c30 con 0x7fb8e40af4d0 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7fb8ea400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e406f4c0 con 0x7fb8e40cd0a0 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc7be00640 1 -- 172.21.15.59:0/4018285052 learned_addr learned my addr 172.21.15.59:0/4018285052 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.993+0000 7fbc72a00640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 0x7fbc7c0d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.994+0000 7f71cf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71d015e9c0 0x7f71d00d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fb8e9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 0x7fb8e41488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fb8e9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 0x7fb8e4149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7f71cea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fb8e9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 0x7fb8e4149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56032/0 (socket says 172.21.15.59:56032) 2024-04-24T06:28:52.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fb8e9a00640 1 -- 172.21.15.59:0/1941677470 learned_addr learned my addr 172.21.15.59:0/1941677470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fbc7be00640 1 -- 172.21.15.59:0/4018285052 >> v2:172.21.15.66:3300/0 conn(0x7fbc7c15e9c0 msgr2=0x7fbc7c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fbc7be00640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.66:3300/0 conn(0x7fbc7c15e9c0 0x7fbc7c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fbc7be00640 1 -- 172.21.15.59:0/4018285052 >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 msgr2=0x7fbc7c0d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7f71cea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56020/0 (socket says 172.21.15.59:56020) 2024-04-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7f71cea00640 1 -- 172.21.15.59:0/1570903709 learned_addr learned my addr 172.21.15.59:0/1570903709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7f71cfe00640 1 --2- 172.21.15.59:0/1570903709 >> v2:172.21.15.66:3300/0 conn(0x7f71d00d10f0 0x7f71d00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.995+0000 7fb8e3e00640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 0x7fb8e414a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7f71cea00640 1 -- 172.21.15.59:0/1570903709 >> v2:172.21.15.66:3300/0 conn(0x7f71d00d10f0 msgr2=0x7f71d00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7f71cea00640 1 --2- 172.21.15.59:0/1570903709 >> v2:172.21.15.66:3300/0 conn(0x7f71d00d10f0 0x7f71d00d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7f71cea00640 1 -- 172.21.15.59:0/1570903709 >> v2:172.21.15.59:3300/0 conn(0x7f71d015e9c0 msgr2=0x7f71d00d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7f71cea00640 1 --2- 172.21.15.59:0/1570903709 >> v2:172.21.15.59:3300/0 conn(0x7f71d015e9c0 0x7f71d00d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7f71cea00640 1 -- 172.21.15.59:0/1570903709 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d00cf900 con 0x7f71d00d2d70 2024-04-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7fbc7be00640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 0x7fbc7c0d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.996+0000 7fbc7be00640 1 -- 172.21.15.59:0/4018285052 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c0cfbe0 con 0x7fbc7c0d10f0 2024-04-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.997+0000 7fbc72a00640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 0x7fbc7c0d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.997+0000 7fb8e9000640 1 -- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 msgr2=0x7fb8e4149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.997+0000 7fb8e9000640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 0x7fb8e4149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.997+0000 7fb8e9000640 1 -- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 msgr2=0x7fb8e414a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.997+0000 7fb8e9000640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 0x7fb8e414a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff404fd90 con 0x7f8ff40af010 2024-04-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7fb8e9000640 1 -- 172.21.15.59:0/1941677470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e414cad0 con 0x7fb8e4073b10 2024-04-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7fb8e3e00640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 0x7fb8e414a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8e9000640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 0x7fb8e41488e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800e8a0 tx=0x7fb8d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7f71cea00640 1 --2- 172.21.15.59:0/1570903709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71d00cb810 tx=0x7f71c802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750e4162a1960e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7f6928400640 1 Processor -- start 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8017070 con 0x7fb8e4073b10 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d800c730 con 0x7fb8e4073b10 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800cb00 con 0x7fb8e4073b10 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e414cdb0 con 0x7fb8e4073b10 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.999+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e414d370 con 0x7fb8e4073b10 2024-04-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.000+0000 7f71ce000640 1 -- 172.21.15.59:0/1570903709 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8036070 con 0x7f71d00d2d70 2024-04-24T06:28:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.000+0000 7f71ce000640 1 -- 172.21.15.59:0/1570903709 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c8031600 con 0x7f71d00d2d70 2024-04-24T06:28:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.000+0000 7f71ce000640 1 -- 172.21.15.59:0/1570903709 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8031a10 con 0x7f71d00d2d70 2024-04-24T06:28:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.000+0000 7fbc7be00640 1 --2- 172.21.15.59:0/4018285052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c08caf0 tx=0x7fbc7402a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c832433d9707886 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:51.998+0000 7f6928400640 1 -- start start 2024-04-24T06:28:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.000+0000 7f6928400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f692015e890 0x7f69200cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f6928400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69200d3600 0x7f69200d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fe001a070 con 0x7f8ff40af010 2024-04-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f6928400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200d2560 con 0x7f69200d1130 2024-04-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f6928400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f69200d2310 con 0x7f692015e890 2024-04-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f6928400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69200d2840 con 0x7f69200d3600 2024-04-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f5f10e00640 1 Processor -- start 2024-04-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 -- 172.21.15.59:0/1570903709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 msgr2=0x7f71d00d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 --2- 172.21.15.59:0/1570903709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71d00cb810 tx=0x7f71c802a6a0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f5f10e00640 1 -- start start 2024-04-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 -- 172.21.15.59:0/1570903709 shutdown_connections 2024-04-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 --2- 172.21.15.59:0/1570903709 >> v2:172.21.15.66:3300/0 conn(0x7f71d00d10f0 0x7f71d00d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 --2- 172.21.15.59:0/1570903709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d00d2d70 0x7f71d00d31d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 --2- 172.21.15.59:0/1570903709 >> v2:172.21.15.59:3300/0 conn(0x7f71d015e9c0 0x7f71d00d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.001+0000 7f71d6800640 1 -- 172.21.15.59:0/1570903709 >> 172.21.15.59:0/1570903709 conn(0x7f71d008bf90 msgr2=0x7f71d00d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f71d6800640 1 -- 172.21.15.59:0/1570903709 shutdown_connections 2024-04-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f71d6800640 1 -- 172.21.15.59:0/1570903709 wait complete. 2024-04-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6925a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f692015e890 0x7f69200cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f8febe00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 0x7f8fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6926400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f69200d1130 0x7f6920162520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59984/0 (socket says 172.21.15.59:59984) 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6926400640 1 -- 172.21.15.59:0/3724536081 learned_addr learned my addr 172.21.15.59:0/3724536081 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6925a00640 1 -- 172.21.15.59:0/3724536081 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69200d3600 msgr2=0x7f69200d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6925a00640 1 --2- 172.21.15.59:0/3724536081 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69200d3600 0x7f69200d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f6925a00640 1 -- 172.21.15.59:0/3724536081 >> v1:172.21.15.59:6789/0 conn(0x7f69200d1130 legacy=0x7f6920162520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e4068290 con 0x7fb8e4073b10 2024-04-24T06:28:52.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f05af200640 1 Processor -- start 2024-04-24T06:28:52.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.002+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe008ae20 con 0x7f8ff40af010 2024-04-24T06:28:52.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f05af200640 1 -- start start 2024-04-24T06:28:52.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.004+0000 7f8ff3400640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 0x7f8fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f5f10e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f0c0c5560 0x7f5f0c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f5f10e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0c1678a0 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f5f10e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0c0c5e80 con 0x7f5f0c0cf7d0 2024-04-24T06:28:52.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f5f10e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5f0c0d3050 con 0x7f5f0c0c5560 2024-04-24T06:28:52.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.003+0000 7f6925a00640 1 -- 172.21.15.59:0/3724536081 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69200d2b20 con 0x7f692015e890 2024-04-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.004+0000 7f5f09e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f0c0c5560 0x7f5f0c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.004+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8d801a070 con 0x7fb8e4073b10 2024-04-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.004+0000 7f5f09e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5f0c0c5560 0x7f5f0c0c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41300/0 (socket says 172.21.15.59:41300) 2024-04-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.004+0000 7f5f09e00640 1 -- 172.21.15.59:0/2889200068 learned_addr learned my addr 172.21.15.59:0/2889200068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f6925a00640 1 --2- 172.21.15.59:0/3724536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f692015e890 0x7f69200cfee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6914009a00 tx=0x7f691402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1897aa74bbeb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f5f09e00640 1 -- 172.21.15.59:0/2889200068 >> v1:172.21.15.66:6789/0 conn(0x7f5f0c0cf7d0 legacy=0x7f5f0c166a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f71d6800640 1 Processor -- start 2024-04-24T06:28:52.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f5f09e00640 1 -- 172.21.15.59:0/2889200068 >> v1:172.21.15.59:6789/0 conn(0x7f5f0c0b37f0 legacy=0x7f5f0c167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f5f09e00640 1 -- 172.21.15.59:0/2889200068 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0d3330 con 0x7f5f0c0c5560 2024-04-24T06:28:52.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.005+0000 7f71d6800640 1 -- start start 2024-04-24T06:28:52.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.006+0000 7f05af200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05b00ba440 0x7f05b00ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.006+0000 7f05af200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b006e680 con 0x7f05b0162cb0 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.006+0000 7f05af200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b006e960 con 0x7f05b00b6250 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.006+0000 7f05af200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f05b00bade0 con 0x7f05b00ba440 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f8ff3400640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 0x7f8fd406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4034000 tx=0x7f8fe402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.006+0000 7fb8e0c00640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 0x7fb8c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d808bbc0 con 0x7fb8e4073b10 2024-04-24T06:28:52.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f05ae800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f05b00b6250 0x7f05b00b66a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44436/0 (socket says 172.21.15.59:44436) 2024-04-24T06:28:52.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f05ae800640 1 -- 172.21.15.59:0/262749785 learned_addr learned my addr 172.21.15.59:0/262749785 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:52.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d00d10f0 0x7f71d015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d015e9c0 0x7f71d00b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f5f09e00640 1 --2- 172.21.15.59:0/2889200068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c5940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009e50 tx=0x7f5f0402a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599a8b217bdfbeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71d00d1ac0 con 0x7f71d00d2d70 2024-04-24T06:28:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71d015eda0 con 0x7f71d00d10f0 2024-04-24T06:28:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71d6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f71d006cb90 con 0x7f71d015e9c0 2024-04-24T06:28:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7fb8e3e00640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 0x7fb8c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f05a5400640 1 --2- 172.21.15.59:0/262749785 >> v2:172.21.15.59:3300/0 conn(0x7f05b00ba440 0x7f05b00ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.007+0000 7f71cea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38004/0 (socket says 172.21.15.59:38004) 2024-04-24T06:28:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 -- 172.21.15.59:0/759024940 learned_addr learned my addr 172.21.15.59:0/759024940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418527518 0 0) 0x7f05b006e960 con 0x7f05b00b6250 2024-04-24T06:28:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0594003520 con 0x7f05b00b6250 2024-04-24T06:28:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f5f08a00640 1 -- 172.21.15.59:0/2889200068 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04036070 con 0x7f5f0c0c5560 2024-04-24T06:28:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f5f08a00640 1 -- 172.21.15.59:0/2889200068 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f04031600 con 0x7f5f0c0c5560 2024-04-24T06:28:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cfe00640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d015e9c0 0x7f71d00b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cf400640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d00d10f0 0x7f71d015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f5f08a00640 1 -- 172.21.15.59:0/2889200068 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040319d0 con 0x7f5f0c0c5560 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 -- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d015e9c0 msgr2=0x7f71d00b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d015e9c0 0x7f71d00b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 -- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d00d10f0 msgr2=0x7f71d015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d00d10f0 0x7f71d015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f71cea00640 1 -- 172.21.15.59:0/759024940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d00b6f90 con 0x7f71d00d2d70 2024-04-24T06:28:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7fb8e3e00640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 0x7fb8c406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40b0520 tx=0x7fb8d0009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.1 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2152236013 0 0) 0x7f0594003520 con 0x7f05b00b6250 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 >> v2:172.21.15.59:3300/0 conn(0x7f05b00ba440 msgr2=0x7f05b00ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 --2- 172.21.15.59:0/262749785 >> v2:172.21.15.59:3300/0 conn(0x7f05b00ba440 0x7f05b00ba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 >> v1:172.21.15.59:6789/0 conn(0x7f05b0162cb0 legacy=0x7f05b006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b00b5770 con 0x7f05b00b6250 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.1 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f05a80019f0 con 0x7f05b00b6250 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.008+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 >> v1:172.21.15.66:6789/0 conn(0x7f05b00b6250 legacy=0x7f05b00b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f0594003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0594004800 0x7f0594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05940051c0 0x7f059400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f059400c030 con 0x7f05940039b0 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f059400c1e0 con 0x7f0594004800 2024-04-24T06:28:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f059400c3e0 con 0x7f05940051c0 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f71cea00640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009f90 tx=0x7f71c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ade00640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f0594003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.009+0000 7f05ae800640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05940051c0 0x7f059400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ade00640 1 -- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05940051c0 msgr2=0x7f059400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ade00640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05940051c0 0x7f059400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ade00640 1 -- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0594004800 msgr2=0x7f0594004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ade00640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0594004800 0x7f0594004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ade00640 1 -- 172.21.15.59:0/262749785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b00b5770 con 0x7f05940039b0 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f05ae800640 1 -- 172.21.15.59:0/262749785 reap_dead start 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 -- 172.21.15.59:0/2889200068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 msgr2=0x7f5f0c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 --2- 172.21.15.59:0/2889200068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c5940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009e50 tx=0x7f5f0402a780 comp rx=0 tx=0).stop 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 -- 172.21.15.59:0/2889200068 shutdown_connections 2024-04-24T06:28:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 --2- 172.21.15.59:0/2889200068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c5940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 -- 172.21.15.59:0/2889200068 >> 172.21.15.59:0/2889200068 conn(0x7f5f0c08bf90 msgr2=0x7f5f0c08c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 -- 172.21.15.59:0/2889200068 shutdown_connections 2024-04-24T06:28:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.010+0000 7f5f10e00640 1 -- 172.21.15.59:0/2889200068 wait complete. 2024-04-24T06:28:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.011+0000 7f05ade00640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f0594003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05a0001730 tx=0x7f0594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32be3e4204da7c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8041020 con 0x7f71d00d2d70 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f691fe00640 1 -- 172.21.15.59:0/3724536081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914036070 con 0x7f692015e890 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f691fe00640 1 -- 172.21.15.59:0/3724536081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914031600 con 0x7f692015e890 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f691fe00640 1 -- 172.21.15.59:0/3724536081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914031a10 con 0x7f692015e890 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0001350 con 0x7f05940039b0 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a00052f0 con 0x7f05940039b0 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f05ad400640 1 -- 172.21.15.59:0/262749785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a002abc0 con 0x7f05940039b0 2024-04-24T06:28:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d00cd2c0 con 0x7f71d00d2d70 2024-04-24T06:28:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f5f10e00640 1 Processor -- start 2024-04-24T06:28:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.012+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d00cd880 con 0x7f71d00d2d70 2024-04-24T06:28:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f5f10e00640 1 -- start start 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f6928400640 1 -- 172.21.15.59:0/3724536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f692015e890 msgr2=0x7f69200cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f6928400640 1 --2- 172.21.15.59:0/3724536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f692015e890 0x7f69200cfee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6914009a00 tx=0x7f691402a420 comp rx=0 tx=0).stop 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f05af200640 1 -- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 msgr2=0x7f0594003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f05af200640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f0594003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05a0001730 tx=0x7f0594005830 comp rx=0 tx=0).stop 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f05af200640 1 -- 172.21.15.59:0/262749785 shutdown_connections 2024-04-24T06:28:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f05af200640 1 --2- 172.21.15.59:0/262749785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f0594003dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7f05af200640 1 -- 172.21.15.59:0/262749785 >> 172.21.15.59:0/262749785 conn(0x7f05b00d5440 msgr2=0x7f05b00d5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f6928400640 1 -- 172.21.15.59:0/3724536081 shutdown_connections 2024-04-24T06:28:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f6928400640 1 --2- 172.21.15.59:0/3724536081 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69200d3600 0x7f69200d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f6928400640 1 --2- 172.21.15.59:0/3724536081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f692015e890 0x7f69200cfee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f6928400640 1 -- 172.21.15.59:0/3724536081 >> 172.21.15.59:0/3724536081 conn(0x7f692008bf90 msgr2=0x7f69200d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c8037040 con 0x7f71d00d2d70 2024-04-24T06:28:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8036070 con 0x7f71d00d2d70 2024-04-24T06:28:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.013+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8e4068290 con 0x7fb8e4073b10 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 0x7f5f0c157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f0c0cf7d0 0x7f5f0c0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0cfe90 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c06ce40 con 0x7f5f0c0c5560 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.014+0000 7f5f10e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c163210 con 0x7f5f0c0cf7d0 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f5f09400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f5f09e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 0x7f5f0c157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d00cd0d0 con 0x7f71d00d2d70 2024-04-24T06:28:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f5f0a800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f0c0cf7d0 0x7f5f0c0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f5f09400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41328/0 (socket says 172.21.15.59:41328) 2024-04-24T06:28:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f5f09400640 1 -- 172.21.15.59:0/3722491058 learned_addr learned my addr 172.21.15.59:0/3722491058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f6928400640 1 -- 172.21.15.59:0/3724536081 shutdown_connections 2024-04-24T06:28:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.015+0000 7f6928400640 1 -- 172.21.15.59:0/3724536081 wait complete. 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 -- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f0c0cf7d0 msgr2=0x7f5f0c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f0c0cf7d0 0x7f5f0c0d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 -- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 msgr2=0x7f5f0c0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 -- 172.21.15.59:0/3722491058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0ca020 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f5f09e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 0x7f5f0c157a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f04030000 tx=0x7f5f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c8044020 con 0x7f71d00d2d70 2024-04-24T06:28:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f05af200640 1 -- 172.21.15.59:0/262749785 shutdown_connections 2024-04-24T06:28:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.016+0000 7f05af200640 1 -- 172.21.15.59:0/262749785 wait complete. 2024-04-24T06:28:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.017+0000 7f71c7400640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 0x7f71b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.017+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71c80ac300 con 0x7f71d00d2d70 2024-04-24T06:28:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.018+0000 7f71cf400640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 0x7f71b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.018+0000 7f6928400640 1 Processor -- start 2024-04-24T06:28:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.018+0000 7f6928400640 1 -- start start 2024-04-24T06:28:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f05af200640 1 Processor -- start 2024-04-24T06:28:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f71cf400640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 0x7f71b006ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71d00b6230 tx=0x7f71c0009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f05af200640 1 -- start start 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69200d3600 0x7f69200cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 0x7f69200c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f692015ec70 con 0x7f69200d1130 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f69200d30e0 con 0x7f69200d3600 2024-04-24T06:28:52.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.019+0000 7f6928400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f692015edf0 con 0x7f692015e890 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7fbc7aa00640 1 -- 172.21.15.59:0/4018285052 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74036070 con 0x7fbc7c0d10f0 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6925a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6926400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 0x7f69200c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6926400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 0x7f69200c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56078/0 (socket says 172.21.15.59:56078) 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6926400640 1 -- 172.21.15.59:0/2575633546 learned_addr learned my addr 172.21.15.59:0/2575633546 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6925a00640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38036/0 (socket says 172.21.15.59:38036) 2024-04-24T06:28:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7fbc7aa00640 1 -- 172.21.15.59:0/4018285052 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc74031600 con 0x7fbc7c0d10f0 2024-04-24T06:28:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7fbc7aa00640 1 -- 172.21.15.59:0/4018285052 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74031a10 con 0x7fbc7c0d10f0 2024-04-24T06:28:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fe0058b60 con 0x7f8ff40af010 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f05b01fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 0x7f05b00d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00b8010 con 0x7f05940039b0 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05b00b5d30 con 0x7f05b00d6e60 2024-04-24T06:28:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.021+0000 7f05af200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05b0163090 con 0x7f05b01f5570 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.020+0000 7f6925000640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69200d3600 0x7f69200cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 -- 172.21.15.59:0/4018285052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 msgr2=0x7fbc7c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 --2- 172.21.15.59:0/4018285052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c08caf0 tx=0x7fbc7402a620 comp rx=0 tx=0).stop 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 -- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 msgr2=0x7f69200c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 0x7f69200c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 -- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69200d3600 msgr2=0x7f69200cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69200d3600 0x7f69200cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f05a5400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f05a5400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56094/0 (socket says 172.21.15.59:56094) 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f05a5400640 1 -- 172.21.15.59:0/2472081847 learned_addr learned my addr 172.21.15.59:0/2472081847 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 -- 172.21.15.59:0/2575633546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69200ccb20 con 0x7f69200d1130 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f05ae800640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 0x7f05b00d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 -- 172.21.15.59:0/4018285052 shutdown_connections 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 --2- 172.21.15.59:0/4018285052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c0d10f0 0x7fbc7c0d1580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.59:3300/0 conn(0x7fbc7c0d2d70 0x7fbc7c0d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 --2- 172.21.15.59:0/4018285052 >> v2:172.21.15.66:3300/0 conn(0x7fbc7c15e9c0 0x7fbc7c0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 -- 172.21.15.59:0/4018285052 >> 172.21.15.59:0/4018285052 conn(0x7fbc7c08bf90 msgr2=0x7fbc7c0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fcdde400640 1 Processor -- start 2024-04-24T06:28:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7f6925a00640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6914031000 tx=0x7f69140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.022+0000 7fbc82000640 1 -- 172.21.15.59:0/4018285052 shutdown_connections 2024-04-24T06:28:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7fbc82000640 1 -- 172.21.15.59:0/4018285052 wait complete. 2024-04-24T06:28:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7fcdde400640 1 -- start start 2024-04-24T06:28:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 -- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 msgr2=0x7f05b00d72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 0x7f05b00d72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 -- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 msgr2=0x7f05b01fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f05b01fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 -- 172.21.15.59:0/2472081847 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b0153cf0 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05ae800640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 0x7f05b00d72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.023+0000 7f05a5400640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f059800e930 tx=0x7f059800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598017070 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059800c980 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800cd70 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b0153fd0 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b0154590 con 0x7f05b01f5570 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7fcdde400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcdd80d3480 0x7fcdd80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdde400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd80cf3c0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdde400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd80cf640 con 0x7fcdd815e9c0 2024-04-24T06:28:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdde400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd80cf900 con 0x7fcdd80d3480 2024-04-24T06:28:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.024+0000 7fbc82000640 1 Processor -- start 2024-04-24T06:28:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04041030 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f04031400 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04036040 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdcfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcdd80d3480 0x7fcdd80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdd7400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcdd815e9c0 0x7fcdd80d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44452/0 (socket says 172.21.15.59:44452) 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdcfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcdd80d3480 0x7fcdd80d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38052/0 (socket says 172.21.15.59:38052) 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fcdcfe00640 1 -- 172.21.15.59:0/524458918 learned_addr learned my addr 172.21.15.59:0/524458918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914041070 con 0x7f69200d1130 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914031e10 con 0x7f69200d1130 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914036040 con 0x7f69200d1130 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69200cce00 con 0x7f69200d1130 2024-04-24T06:28:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69200cd3c0 con 0x7f69200d1130 2024-04-24T06:28:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c0ca300 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71c804b030 con 0x7f71d00d2d70 2024-04-24T06:28:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c0ca8c0 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7fcdcfe00640 1 -- 172.21.15.59:0/524458918 >> v1:172.21.15.66:6789/0 conn(0x7fcdd815e9c0 legacy=0x7fcdd80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7fcdcfe00640 1 -- 172.21.15.59:0/524458918 >> v1:172.21.15.59:6789/0 conn(0x7fcdd80d10f0 legacy=0x7fcdd80d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.026+0000 7fcdcfe00640 1 -- 172.21.15.59:0/524458918 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd80cfbe0 con 0x7fcdd80d3480 2024-04-24T06:28:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b004fd90 con 0x7f05b01f5570 2024-04-24T06:28:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/524458918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d3480 0x7fcdd80d03e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009850 tx=0x7fcdc802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53c0a4f7ca566c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c069190 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7fcdd6a00640 1 -- 172.21.15.59:0/524458918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004030 con 0x7fcdd80d3480 2024-04-24T06:28:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7fcdd6a00640 1 -- 172.21.15.59:0/524458918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80041d0 con 0x7fcdd80d3480 2024-04-24T06:28:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.028+0000 7fcdd6a00640 1 -- 172.21.15.59:0/524458918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8003610 con 0x7fcdd80d3480 2024-04-24T06:28:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.029+0000 7fcdde400640 1 -- 172.21.15.59:0/524458918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d3480 msgr2=0x7fcdd80d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.029+0000 7fcdde400640 1 --2- 172.21.15.59:0/524458918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d3480 0x7fcdd80d03e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009850 tx=0x7fcdc802a200 comp rx=0 tx=0).stop 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.025+0000 7fbc82000640 1 -- start start 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7fbc82000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 0x7fbc7c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7fbc82000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc7c0d2d70 0x7fbc7c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7fbc82000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 0x7fbc7c160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7fbc82000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c06cb90 con 0x7fbc7c0d2d70 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7fbc82000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c0d1ac0 con 0x7fbc7c0d10f0 2024-04-24T06:28:52.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692015e030 con 0x7f69200d1130 2024-04-24T06:28:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.030+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f04044070 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc82000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c06cd10 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.030+0000 7fcdde400640 1 -- 172.21.15.59:0/524458918 shutdown_connections 2024-04-24T06:28:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.030+0000 7fcdde400640 1 --2- 172.21.15.59:0/524458918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d3480 0x7fcdd80d03e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.030+0000 7fcdde400640 1 -- 172.21.15.59:0/524458918 >> 172.21.15.59:0/524458918 conn(0x7fcdd808bf90 msgr2=0x7fcdd80d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 0x7fbc7c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc72a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc7c0d2d70 0x7fbc7c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 0x7fbc7c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41348/0 (socket says 172.21.15.59:41348) 2024-04-24T06:28:52.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc7b400640 1 -- 172.21.15.59:0/2359458732 learned_addr learned my addr 172.21.15.59:0/2359458732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7f5f02000640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 0x7f5eec06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7fbc7be00640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 0x7fbc7c160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.029+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f059801a070 con 0x7f05b01f5570 2024-04-24T06:28:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7f05a6a00640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 0x7f058c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5f040acce0 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.031+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f059808be80 con 0x7f05b01f5570 2024-04-24T06:28:52.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.032+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6914044070 con 0x7f69200d1130 2024-04-24T06:28:52.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7fcdde400640 1 -- 172.21.15.59:0/524458918 shutdown_connections 2024-04-24T06:28:52.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7f5f09400640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 0x7f5eec06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7fbc7be00640 1 -- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 msgr2=0x7fbc7c0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7fbc7be00640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 0x7fbc7c0cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7fbc7be00640 1 -- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc7c0d2d70 msgr2=0x7fbc7c0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7f05ade00640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 0x7f058c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.034+0000 7f5f09400640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 0x7f5eec06eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0cc6d0 tx=0x7f5efc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.034+0000 7f05ade00640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 0x7f058c06ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05a0032000 tx=0x7f05a002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.034+0000 7f691d600640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 0x7f690006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.034+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f69140ac380 con 0x7f69200d1130 2024-04-24T06:28:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.034+0000 7f6925000640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 0x7f690006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7fbc7be00640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc7c0d2d70 0x7fbc7c0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7fbc7be00640 1 -- 172.21.15.59:0/2359458732 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c1f9090 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.033+0000 7fcdde400640 1 -- 172.21.15.59:0/524458918 wait complete. 2024-04-24T06:28:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7fcdde400640 1 Processor -- start 2024-04-24T06:28:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7fcdde400640 1 -- start start 2024-04-24T06:28:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7f6925000640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 0x7f690006ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f692006df80 tx=0x7f690c009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.035+0000 7fbc7be00640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 0x7fbc7c160800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc74030000 tx=0x7fbc74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.036+0000 7fcdde400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.036+0000 7fcdde400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 0x7fcdd81e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdde400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdd815e9c0 0x7fcdd81e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdd7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdd7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38062/0 (socket says 172.21.15.59:38062) 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdd7400640 1 -- 172.21.15.59:0/4156286889 learned_addr learned my addr 172.21.15.59:0/4156286889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 0x7fcdd81e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd806cd10 con 0x7fcdd80d10f0 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd806cb90 con 0x7fcdd80d3480 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.037+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd806ce90 con 0x7fcdd815e9c0 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.038+0000 7fcdd7e00640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdd815e9c0 0x7fcdd81e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.038+0000 7fcdd7400640 1 -- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdd815e9c0 msgr2=0x7fcdd81e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.038+0000 7fcdd7400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdd815e9c0 0x7fcdd81e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f0407abc0 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7fcdd7400640 1 -- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 msgr2=0x7fcdd81e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7fcdd7400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 0x7fcdd81e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7fcdd7400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd80c98c0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 0x7fcdd81e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.039+0000 7fcdd7400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c3c0 tx=0x7fcdc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.045+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74036050 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.045+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0017070 con 0x7fcdd80d10f0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.045+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc00074e0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc00059a0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd80c9ba0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc74042070 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd80ca160 con 0x7fcdd80d10f0 2024-04-24T06:28:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c1f9370 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f692015e030 con 0x7f69200d1130 2024-04-24T06:28:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.047+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f059805a2d0 con 0x7f05b01f5570 2024-04-24T06:28:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1f9930 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.046+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74007c70 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.047+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd8068a80 con 0x7fcdd80d10f0 2024-04-24T06:28:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.048+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc54005290 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.051+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdc001a070 con 0x7fcdd80d10f0 2024-04-24T06:28:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.052+0000 7fcdcf400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 0x7fcdb406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.052+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdc008acb0 con 0x7fcdd80d10f0 2024-04-24T06:28:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.054+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 0x7fcdb406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.056+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 0x7fcdb406ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8033000 tx=0x7fcdc802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.056+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc74047460 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.057+0000 7fbc73e00640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 0x7fbc5806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.057+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc740791e0 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.058+0000 7fbc7b400640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 0x7fbc5806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.059+0000 7fbc7b400640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 0x7fbc5806ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.061+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdc0058b00 con 0x7fcdd80d10f0 2024-04-24T06:28:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.066+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc7407b140 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.457+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb8e404fd90 con 0x7fb8e4073b10 2024-04-24T06:28:52.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.458+0000 7fb8e0c00640 1 -- 172.21.15.59:0/1941677470 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8d805ee20 con 0x7fb8e4073b10 2024-04-24T06:28:52.460 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:52.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.462+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 msgr2=0x7fb8c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.462+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 0x7fb8c406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40b0520 tx=0x7fb8d0009110 comp rx=0 tx=0).stop 2024-04-24T06:28:52.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.462+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 msgr2=0x7fb8e41488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.462+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 0x7fb8e41488e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800e8a0 tx=0x7fb8d800eca0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 shutdown_connections 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8c406c8f0 0x7fb8c406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8e40cd0a0 0x7fb8e4149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8e40af4d0 0x7fb8e414a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 --2- 172.21.15.59:0/1941677470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8e4073b10 0x7fb8e41488e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.463+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 >> 172.21.15.59:0/1941677470 conn(0x7fb8e4072ff0 msgr2=0x7fb8e40ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.464+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 shutdown_connections 2024-04-24T06:28:52.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.464+0000 7fb8ea400640 1 -- 172.21.15.59:0/1941677470 wait complete. 2024-04-24T06:28:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.480+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8ff40cedb0 con 0x7f8ff40af010 2024-04-24T06:28:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.483+0000 7f8febe00640 1 -- 172.21.15.59:0/2770821305 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8fe005ce60 con 0x7f8ff40af010 2024-04-24T06:28:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.484+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f71d0068a80 con 0x7f71d00d2d70 2024-04-24T06:28:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.485+0000 7f71c7400640 1 -- 172.21.15.59:0/759024940 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71c807a820 con 0x7f71d00d2d70 2024-04-24T06:28:52.487 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:52.487 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.488+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f69200d18f0 con 0x7f69200d1130 2024-04-24T06:28:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.489+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 msgr2=0x7f71b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.489+0000 7f691d600640 1 -- 172.21.15.59:0/2575633546 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f691407a240 con 0x7f69200d1130 2024-04-24T06:28:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.489+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 0x7f71b006ede0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71d00b6230 tx=0x7f71c0009110 comp rx=0 tx=0).stop 2024-04-24T06:28:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 msgr2=0x7f8fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.491 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.489+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 msgr2=0x7f71d00b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.489+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009f90 tx=0x7f71c8004a40 comp rx=0 tx=0).stop 2024-04-24T06:28:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 0x7f8fd406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4034000 tx=0x7f8fe402e000 comp rx=0 tx=0).stop 2024-04-24T06:28:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 msgr2=0x7f8ff415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 shutdown_connections 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71b006c920 0x7f71b006ede0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71d015e9c0 0x7f71d00b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d00d2d70 0x7f71d00b41f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 --2- 172.21.15.59:0/759024940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71d00d10f0 0x7f71d015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 >> 172.21.15.59:0/759024940 conn(0x7f71d008bf90 msgr2=0x7f71d015f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.490+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c4d0 tx=0x7f8fe000c990 comp rx=0 tx=0).stop 2024-04-24T06:28:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 shutdown_connections 2024-04-24T06:28:52.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 shutdown_connections 2024-04-24T06:28:52.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f71d6800640 1 -- 172.21.15.59:0/759024940 wait complete. 2024-04-24T06:28:52.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8fd406c8f0 0x7f8fd406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ff40d0c10 0x7f8ff41622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ff40d0830 0x7f8ff41ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 --2- 172.21.15.59:0/2770821305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ff40af010 0x7f8ff415c4d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 >> 172.21.15.59:0/2770821305 conn(0x7f8ff406bdf0 msgr2=0x7f8ff4072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 shutdown_connections 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.492+0000 7f8ff9a00640 1 -- 172.21.15.59:0/2770821305 wait complete. 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 msgr2=0x7f690006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 0x7f690006ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f692006df80 tx=0x7f690c009110 comp rx=0 tx=0).stop 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 msgr2=0x7f69201f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6914031000 tx=0x7f69140046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 shutdown_connections 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f690006c940 0x7f690006ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f692015e890 0x7f69200c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69200d3600 0x7f69200cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 --2- 172.21.15.59:0/2575633546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69200d1130 0x7f69201f7850 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.494+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 >> 172.21.15.59:0/2575633546 conn(0x7f692008bf90 msgr2=0x7f692006d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-24T06:28:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.495+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 shutdown_connections 2024-04-24T06:28:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.495+0000 7f6928400640 1 -- 172.21.15.59:0/2575633546 wait complete. 2024-04-24T06:28:52.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.497+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5f0c0b4b80 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.497+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f05b006f350 con 0x7f05b01f5570 2024-04-24T06:28:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.498+0000 7f05a6a00640 1 -- 172.21.15.59:0/2472081847 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0598059bd0 con 0x7f05b01f5570 2024-04-24T06:28:52.499 INFO:teuthology.orchestra.run.smithi059.stdout:55834574852 2024-04-24T06:28:52.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.498+0000 7f5f02000640 1 -- 172.21.15.59:0/3722491058 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f0407eec0 con 0x7f5f0c0b37f0 2024-04-24T06:28:52.501 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.500+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 msgr2=0x7f058c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 0x7f058c06ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05a0032000 tx=0x7f05a002c000 comp rx=0 tx=0).stop 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 msgr2=0x7f05b01f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f059800e930 tx=0x7f059800edf0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 shutdown_connections 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f058c06c920 0x7f058c06ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b00d6e60 0x7f05b00d72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05b01f5570 0x7f05b01f59f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 --2- 172.21.15.59:0/2472081847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05940039b0 0x7f05b01fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 >> 172.21.15.59:0/2472081847 conn(0x7f05b00d5440 msgr2=0x7f05b00b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 shutdown_connections 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.501+0000 7f05af200640 1 -- 172.21.15.59:0/2472081847 wait complete. 2024-04-24T06:28:52.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 msgr2=0x7f5eec06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 0x7f5eec06eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0cc6d0 tx=0x7f5efc009040 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 msgr2=0x7f5f0c157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 0x7f5f0c157a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f04030000 tx=0x7f5f04004a40 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 shutdown_connections 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eec06c9f0 0x7f5eec06eeb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f0c0cf7d0 0x7f5f0c0d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c0c5560 0x7f5f0c0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 --2- 172.21.15.59:0/3722491058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f0c0b37f0 0x7f5f0c157a10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.502+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 >> 172.21.15.59:0/3722491058 conn(0x7f5f0c08bf90 msgr2=0x7f5f0c0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.503+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 shutdown_connections 2024-04-24T06:28:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.503+0000 7f5f10e00640 1 -- 172.21.15.59:0/3722491058 wait complete. 2024-04-24T06:28:52.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.512+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbc54005850 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.513+0000 7fbc73e00640 1 -- 172.21.15.59:0/2359458732 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc7407f440 con 0x7fbc7c15e9c0 2024-04-24T06:28:52.515 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 msgr2=0x7fbc5806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 0x7fbc5806ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64009110 comp rx=0 tx=0).stop 2024-04-24T06:28:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 msgr2=0x7fbc7c160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 0x7fbc7c160800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc74030000 tx=0x7fbc74004a40 comp rx=0 tx=0).stop 2024-04-24T06:28:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcdd80b3450 con 0x7fcdd80d10f0 2024-04-24T06:28:52.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 shutdown_connections 2024-04-24T06:28:52.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc5806c940 0x7fbc5806ee00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc7c15e9c0 0x7fbc7c160800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc7c0d2d70 0x7fbc7c0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 --2- 172.21.15.59:0/2359458732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc7c0d10f0 0x7fbc7c0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 >> 172.21.15.59:0/2359458732 conn(0x7fbc7c08bf90 msgr2=0x7fbc7c15dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.516+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 shutdown_connections 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.517+0000 7fbc82000640 1 -- 172.21.15.59:0/2359458732 wait complete. 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.517+0000 7fcdcf400640 1 -- 172.21.15.59:0/4156286889 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcdc005ce00 con 0x7fcdd80d10f0 2024-04-24T06:28:52.519 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:52.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-24T06:28:52.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 msgr2=0x7fcdb406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 0x7fcdb406ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8033000 tx=0x7fcdc802e000 comp rx=0 tx=0).stop 2024-04-24T06:28:52.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 msgr2=0x7fcdd81e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:52.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c3c0 tx=0x7fcdc000c880 comp rx=0 tx=0).stop 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 shutdown_connections 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdb406c9c0 0x7fcdb406ee80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdd815e9c0 0x7fcdd81e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdd80d3480 0x7fcdd81e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 --2- 172.21.15.59:0/4156286889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd80d10f0 0x7fcdd81e6fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.521+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 >> 172.21.15.59:0/4156286889 conn(0x7fcdd808bf90 msgr2=0x7fcdd80d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.522+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 shutdown_connections 2024-04-24T06:28:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:52.522+0000 7fcdde400640 1 -- 172.21.15.59:0/4156286889 wait complete. 2024-04-24T06:28:52.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-24T06:28:52.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-04-24T06:28:52.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-24T06:28:52.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-24T06:28:52.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-24T06:28:52.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-24T06:28:53.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T06:28:53.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T06:28:53.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T06:28:53.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T06:28:53.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T06:28:53.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T06:28:53.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T06:28:53.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T06:28:53.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.792+0000 7effafe00640 1 Processor -- start 2024-04-24T06:28:53.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.792+0000 7effafe00640 1 -- start start 2024-04-24T06:28:53.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.794+0000 7effafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7effa815ac40 0x7effa80ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.795+0000 7effafe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815e820 0x7effa80ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.795+0000 7effafe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa8073ea0 con 0x7effa80b2830 2024-04-24T06:28:53.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.797+0000 7effade00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7effa80b2830 0x7effa80ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44456/0 (socket says 172.21.15.59:44456) 2024-04-24T06:28:53.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.797+0000 7effade00640 1 -- 172.21.15.59:0/1726205011 learned_addr learned my addr 172.21.15.59:0/1726205011 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.798+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7effa80adb70 con 0x7effa815ac40 2024-04-24T06:28:53.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.798+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7effa80acff0 con 0x7effa815e820 2024-04-24T06:28:53.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.798+0000 7effad400640 1 --2- 172.21.15.59:0/1726205011 >> v2:172.21.15.59:3300/0 conn(0x7effa815ac40 0x7effa80ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.799+0000 7effad400640 1 -- 172.21.15.59:0/1726205011 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815e820 msgr2=0x7effa80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.799+0000 7effad400640 1 --2- 172.21.15.59:0/1726205011 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815e820 0x7effa80ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.800+0000 7effad400640 1 -- 172.21.15.59:0/1726205011 >> v1:172.21.15.66:6789/0 conn(0x7effa80b2830 legacy=0x7effa80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.800+0000 7effad400640 1 -- 172.21.15.59:0/1726205011 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa80ad240 con 0x7effa815ac40 2024-04-24T06:28:53.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.802+0000 7effad400640 1 --2- 172.21.15.59:0/1726205011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa815ac40 0x7effa80ce0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097a0 tx=0x7eff9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b93d599bf45795 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.803+0000 7effa7e00640 1 -- 172.21.15.59:0/1726205011 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c037070 con 0x7effa815ac40 2024-04-24T06:28:53.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.804+0000 7effa7e00640 1 -- 172.21.15.59:0/1726205011 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9c032440 con 0x7effa815ac40 2024-04-24T06:28:53.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.804+0000 7effa7e00640 1 -- 172.21.15.59:0/1726205011 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c032810 con 0x7effa815ac40 2024-04-24T06:28:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.805+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa815ac40 msgr2=0x7effa80ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.805+0000 7effafe00640 1 --2- 172.21.15.59:0/1726205011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa815ac40 0x7effa80ce0c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097a0 tx=0x7eff9c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.806+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 shutdown_connections 2024-04-24T06:28:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.806+0000 7effafe00640 1 --2- 172.21.15.59:0/1726205011 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815e820 0x7effa80ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.806+0000 7effafe00640 1 --2- 172.21.15.59:0/1726205011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa815ac40 0x7effa80ce0c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.806+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 >> 172.21.15.59:0/1726205011 conn(0x7effa80b1d10 msgr2=0x7effa80b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.807+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 shutdown_connections 2024-04-24T06:28:53.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.807+0000 7effafe00640 1 -- 172.21.15.59:0/1726205011 wait complete. 2024-04-24T06:28:53.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.810+0000 7effafe00640 1 Processor -- start 2024-04-24T06:28:53.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.810+0000 7effafe00640 1 -- start start 2024-04-24T06:28:53.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.812+0000 7effafe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 0x7effa80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.813+0000 7effafe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa80c59f0 0x7effa815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.813+0000 7effafe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effafe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effa815b020 con 0x7effa80b2830 2024-04-24T06:28:53.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effafe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effa80ad800 con 0x7effa80c59f0 2024-04-24T06:28:53.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effafe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7effa815b1a0 con 0x7effa815de40 2024-04-24T06:28:53.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effade00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effade00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56134/0 (socket says 172.21.15.59:56134) 2024-04-24T06:28:53.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.814+0000 7effade00640 1 -- 172.21.15.59:0/4247086879 learned_addr learned my addr 172.21.15.59:0/4247086879 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.815+0000 7effade00640 1 -- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa80c59f0 msgr2=0x7effa815d900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T06:28:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.815+0000 7effade00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa80c59f0 0x7effa815d900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.815+0000 7effade00640 1 -- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 msgr2=0x7effa80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.816+0000 7effad400640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 0x7effa80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.816+0000 7effade00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 0x7effa80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.816+0000 7effade00640 1 -- 172.21.15.59:0/4247086879 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa80b0780 con 0x7effa815de40 2024-04-24T06:28:53.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.816+0000 7effad400640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 0x7effa80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:53.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.817+0000 7effade00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7effa000caf0 tx=0x7effa000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.817+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0017070 con 0x7effa815de40 2024-04-24T06:28:53.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.817+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa000aca0 con 0x7effa815de40 2024-04-24T06:28:53.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.817+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0012600 con 0x7effa815de40 2024-04-24T06:28:53.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.819+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa80b0a60 con 0x7effa815de40 2024-04-24T06:28:53.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.819+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effa80b1020 con 0x7effa815de40 2024-04-24T06:28:53.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.821+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa804fd90 con 0x7effa815de40 2024-04-24T06:28:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.823+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effa001a070 con 0x7effa815de40 2024-04-24T06:28:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.825+0000 7effa5600640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 0x7eff8806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.826+0000 7effad400640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 0x7eff8806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.828+0000 7effad400640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 0x7eff8806ee30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff9c034000 tx=0x7eff9c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.828+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7effa0057b70 con 0x7effa815de40 2024-04-24T06:28:53.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.835+0000 7f0c77000640 1 Processor -- start 2024-04-24T06:28:53.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.836+0000 7f0c77000640 1 -- start start 2024-04-24T06:28:53.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.837+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effa0059ad0 con 0x7effa815de40 2024-04-24T06:28:53.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.837+0000 7fbc47e00640 1 Processor -- start 2024-04-24T06:28:53.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.837+0000 7fbc47e00640 1 -- start start 2024-04-24T06:28:53.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.837+0000 7f0c77000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c7015e9c0 0x7f0c700d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.837+0000 7f0c77000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c700d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7f0c77000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c700cf3c0 con 0x7f0c700d10f0 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7f0c77000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0c700d3710 con 0x7f0c7015e9c0 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7f0c77000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c700cf640 con 0x7f0c700d2d70 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7f0c6f400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c700d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7f0c6fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c7015e9c0 0x7f0c700d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7f0c75000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0c700d10f0 0x7f0c700d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60016/0 (socket says 172.21.15.59:60016) 2024-04-24T06:28:53.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7f0c75000640 1 -- 172.21.15.59:0/596322856 learned_addr learned my addr 172.21.15.59:0/596322856 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc40051690 0x7fbc400e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7fbc47e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc400e3490 0x7fbc400e3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7fbc47e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc40051e80 con 0x7fbc40009da0 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7fbc47e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbc40052160 con 0x7fbc400e3490 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.838+0000 7fbc47e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc400e2c90 con 0x7fbc40051690 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7fbc46a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fbc40009da0 0x7fbc4000a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44460/0 (socket says 172.21.15.59:44460) 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.839+0000 7fbc46a00640 1 -- 172.21.15.59:0/3334257118 learned_addr learned my addr 172.21.15.59:0/3334257118 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762193647 0 0) 0x7fbc40051e80 con 0x7fbc40009da0 2024-04-24T06:28:53.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc2c003520 con 0x7fbc40009da0 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6ea00640 1 -- 172.21.15.59:0/596322856 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084370660 0 0) 0x7f0c68001490 con 0x7f0c700d10f0 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6ea00640 1 -- 172.21.15.59:0/596322856 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c54003520 con 0x7f0c700d10f0 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7fbc46000640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc40051690 0x7fbc400e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.1 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1981419780 0 0) 0x7fbc2c003520 con 0x7fbc40009da0 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc40051690 msgr2=0x7fbc400e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc40051690 0x7fbc400e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 >> v2:172.21.15.59:3300/0 conn(0x7fbc400e3490 msgr2=0x7fbc400e3920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 --2- 172.21.15.59:0/3334257118 >> v2:172.21.15.59:3300/0 conn(0x7fbc400e3490 0x7fbc400e3920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 --> v1:172.21.15.66:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc400e2f70 con 0x7fbc40009da0 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6fe00640 1 -- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 msgr2=0x7f0c700d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c700d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6fe00640 1 -- 172.21.15.59:0/596322856 >> v1:172.21.15.59:6789/0 conn(0x7f0c700d10f0 legacy=0x7f0c700d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.840+0000 7f0c6fe00640 1 -- 172.21.15.59:0/596322856 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c700cf900 con 0x7f0c7015e9c0 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.1 v1:172.21.15.66:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbc38001670 con 0x7fbc40009da0 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 >> v1:172.21.15.66:6789/0 conn(0x7fbc40009da0 legacy=0x7fbc4000a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c0039d0 0x7fbc2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc2c004800 0x7fbc2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0051c0 0x7fbc2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40052160 con 0x7fbc2c0039d0 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc400e2c90 con 0x7fbc2c0051c0 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.841+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c014920 con 0x7fbc2c004800 2024-04-24T06:28:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.842+0000 7fbc47400640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0051c0 0x7fbc2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.842+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7015e9c0 0x7f0c700d03e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009790 tx=0x7f0c6002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930c759be1cb3579 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.842+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c0039d0 0x7fbc2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46000640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc2c004800 0x7fbc2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7f0c6ea00640 1 -- 172.21.15.59:0/596322856 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60037070 con 0x7f0c7015e9c0 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7f0c6ea00640 1 -- 172.21.15.59:0/596322856 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c60032440 con 0x7f0c7015e9c0 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7f0c6ea00640 1 -- 172.21.15.59:0/596322856 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6002ae60 con 0x7f0c7015e9c0 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.844+0000 7f0c77000640 1 -- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7015e9c0 msgr2=0x7f0c700d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.844+0000 7f0c77000640 1 --2- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7015e9c0 0x7f0c700d03e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009790 tx=0x7f0c6002a110 comp rx=0 tx=0).stop 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46a00640 1 -- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc2c004800 msgr2=0x7fbc2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc2c004800 0x7fbc2c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46a00640 1 -- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0051c0 msgr2=0x7fbc2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0051c0 0x7fbc2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.843+0000 7fbc46a00640 1 -- 172.21.15.59:0/3334257118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc40051e80 con 0x7fbc2c0039d0 2024-04-24T06:28:53.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 -- 172.21.15.59:0/596322856 shutdown_connections 2024-04-24T06:28:53.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 --2- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c700d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 --2- 172.21.15.59:0/596322856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7015e9c0 0x7f0c700d03e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 -- 172.21.15.59:0/596322856 >> 172.21.15.59:0/596322856 conn(0x7f0c7008bf90 msgr2=0x7f0c700d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 -- 172.21.15.59:0/596322856 shutdown_connections 2024-04-24T06:28:53.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7f0c77000640 1 -- 172.21.15.59:0/596322856 wait complete. 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.845+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c0039d0 0x7fbc2c003e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c290 tx=0x7fbc2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808726ee4fff5a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.846+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38007d50 con 0x7fbc2c0039d0 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.846+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc38004ac0 con 0x7fbc2c0039d0 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.846+0000 7fbc45600640 1 -- 172.21.15.59:0/3334257118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38007070 con 0x7fbc2c0039d0 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.847+0000 7fbc47e00640 1 -- 172.21.15.59:0/3334257118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c0039d0 msgr2=0x7fbc2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.847+0000 7fbc47e00640 1 --2- 172.21.15.59:0/3334257118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc2c0039d0 0x7fbc2c003e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c290 tx=0x7fbc2c005830 comp rx=0 tx=0).stop 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.847+0000 7fbc47400640 1 -- 172.21.15.59:0/3334257118 reap_dead start 2024-04-24T06:28:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.848+0000 7f0c77000640 1 Processor -- start 2024-04-24T06:28:53.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.848+0000 7f0c77000640 1 -- start start 2024-04-24T06:28:53.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.849+0000 7fbc47e00640 1 -- 172.21.15.59:0/3334257118 shutdown_connections 2024-04-24T06:28:53.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.849+0000 7fbc47e00640 1 -- 172.21.15.59:0/3334257118 >> 172.21.15.59:0/3334257118 conn(0x7fbc400537b0 msgr2=0x7fbc40053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.849+0000 7f0c77000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.850+0000 7fbc47e00640 1 -- 172.21.15.59:0/3334257118 shutdown_connections 2024-04-24T06:28:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.850+0000 7fbc47e00640 1 -- 172.21.15.59:0/3334257118 wait complete. 2024-04-24T06:28:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.850+0000 7f0c77000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c70161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c77000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 0x7f0c700ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c77000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7006cb90 con 0x7f0c7015e9c0 2024-04-24T06:28:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c77000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7015eda0 con 0x7f0c700d10f0 2024-04-24T06:28:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c77000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7006cd10 con 0x7f0c700d2d70 2024-04-24T06:28:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c75000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 0x7f0c700ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41404/0 (socket says 172.21.15.59:41404) 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c6fe00640 1 -- 172.21.15.59:0/968311441 learned_addr learned my addr 172.21.15.59:0/968311441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c6f400640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c70161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.851+0000 7f0c75000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 0x7f0c700ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38098/0 (socket says 172.21.15.59:38098) 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7efc12a00640 1 Processor -- start 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7f0c6fe00640 1 -- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 msgr2=0x7f0c70161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c70161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7f0c6fe00640 1 -- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 msgr2=0x7f0c700ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 0x7f0c700ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7f0c6fe00640 1 -- 172.21.15.59:0/968311441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c70161f60 con 0x7f0c700d10f0 2024-04-24T06:28:53.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.853+0000 7fbc47e00640 1 Processor -- start 2024-04-24T06:28:53.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.852+0000 7efc12a00640 1 -- start start 2024-04-24T06:28:53.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.853+0000 7fa4f0a00640 1 Processor -- start 2024-04-24T06:28:53.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.853+0000 7fbc47e00640 1 -- start start 2024-04-24T06:28:53.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.853+0000 7fa4f0a00640 1 -- start start 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.854+0000 7efc12a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc0c071d90 0x7efc0c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.854+0000 7efc12a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.854+0000 7efc12a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c06e040 con 0x7efc0c0b3430 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.854+0000 7efc12a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c06ddf0 con 0x7efc0c071d90 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.854+0000 7efc12a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c06e320 con 0x7efc0c072880 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc0c071d90 0x7efc0c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc0c071d90 0x7efc0c0cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38112/0 (socket says 172.21.15.59:38112) 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 -- 172.21.15.59:0/646277260 learned_addr learned my addr 172.21.15.59:0/646277260 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0039d0 0x7fbc400d9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 -- 172.21.15.59:0/646277260 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 msgr2=0x7efc0c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 --2- 172.21.15.59:0/646277260 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 -- 172.21.15.59:0/646277260 >> v1:172.21.15.66:6789/0 conn(0x7efc0c0b3430 legacy=0x7efc0c072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.855+0000 7efc0be00640 1 -- 172.21.15.59:0/646277260 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c06e600 con 0x7efc0c071d90 2024-04-24T06:28:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4f0a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4f0a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4ec0d10f0 0x7fa4ec0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc400d6ec0 0x7fbc400d7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4f0a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec0cf3c0 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4f0a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec0cf640 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4f0a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec0cf900 con 0x7fa4ec0d3480 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fbc47e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc400e2380 con 0x7fbc400498d0 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4e9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4ea800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4ec0d10f0 0x7fa4ec0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4e9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56184/0 (socket says 172.21.15.59:56184) 2024-04-24T06:28:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7fa4e9400640 1 -- 172.21.15.59:0/2169570312 learned_addr learned my addr 172.21.15.59:0/2169570312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.856+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009790 tx=0x7f0c600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc46a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0039d0 0x7fbc400d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc46000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc47400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc400d6ec0 0x7fbc400d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc46000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38126/0 (socket says 172.21.15.59:38126) 2024-04-24T06:28:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc46000640 1 -- 172.21.15.59:0/3946619184 learned_addr learned my addr 172.21.15.59:0/3946619184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60037040 con 0x7f0c700d10f0 2024-04-24T06:28:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc400e2200 con 0x7fbc2c0039d0 2024-04-24T06:28:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7efc0be00640 1 --2- 172.21.15.59:0/646277260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cd1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc00009870 tx=0x7efc0002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f9955ca5941e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc400e2500 con 0x7fbc400d6ec0 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fa4ea800640 1 -- 172.21.15.59:0/2169570312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 msgr2=0x7fa4ec0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fa4ea800640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fa4ea800640 1 -- 172.21.15.59:0/2169570312 >> v1:172.21.15.66:6789/0 conn(0x7fa4ec15e9c0 legacy=0x7fa4ec0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.857+0000 7fa4ea800640 1 -- 172.21.15.59:0/2169570312 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0cfbe0 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fa4e9400640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fbc46000640 1 -- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc400d6ec0 msgr2=0x7fbc400d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fbc46000640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc400d6ec0 0x7fbc400d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fbc46000640 1 -- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0039d0 msgr2=0x7fbc400d9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fbc46000640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0039d0 0x7fbc400d9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fbc46000640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc400d9b40 con 0x7fbc400498d0 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7fbc46000640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e4b0 tx=0x7fbc3400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c60042070 con 0x7f0c700d10f0 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60032830 con 0x7f0c700d10f0 2024-04-24T06:28:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c700caf60 con 0x7f0c700d10f0 2024-04-24T06:28:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fd615800640 1 Processor -- start 2024-04-24T06:28:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7efc0aa00640 1 -- 172.21.15.59:0/646277260 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00037070 con 0x7efc0c071d90 2024-04-24T06:28:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7efc0aa00640 1 -- 172.21.15.59:0/646277260 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00032440 con 0x7efc0c071d90 2024-04-24T06:28:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7efc0aa00640 1 -- 172.21.15.59:0/646277260 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00032810 con 0x7efc0c071d90 2024-04-24T06:28:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c700cc970 con 0x7f0c700d10f0 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7efc12a00640 1 -- 172.21.15.59:0/646277260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 msgr2=0x7efc0c0cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7efc12a00640 1 --2- 172.21.15.59:0/646277260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cd1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc00009870 tx=0x7efc0002a250 comp rx=0 tx=0).stop 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34017020 con 0x7fbc400498d0 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34013610 con 0x7fbc400498d0 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.859+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400f040 con 0x7fbc400498d0 2024-04-24T06:28:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc400db2c0 con 0x7fbc400498d0 2024-04-24T06:28:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 -- 172.21.15.59:0/646277260 shutdown_connections 2024-04-24T06:28:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 --2- 172.21.15.59:0/646277260 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 --2- 172.21.15.59:0/646277260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cd1b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 -- 172.21.15.59:0/646277260 >> 172.21.15.59:0/646277260 conn(0x7efc0c0b2130 msgr2=0x7efc0c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 -- 172.21.15.59:0/646277260 shutdown_connections 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.858+0000 7fd615800640 1 -- start start 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fd615800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 0x7fd6100d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fd615800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6100d0af0 con 0x7fd6100b34a0 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fd615800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd61015b860 con 0x7fd61015ab80 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fd615800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd61006fcc0 con 0x7fd6100d01d0 2024-04-24T06:28:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7efc12a00640 1 -- 172.21.15.59:0/646277260 wait complete. 2024-04-24T06:28:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7fd60f400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fd61015ab80 0x7fd61015b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44482/0 (socket says 172.21.15.59:44482) 2024-04-24T06:28:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7fd60f400640 1 -- 172.21.15.59:0/1396808100 learned_addr learned my addr 172.21.15.59:0/1396808100 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7fa4ea800640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0d1580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12baa29aeb5c321 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.860+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc400db880 con 0x7fbc400498d0 2024-04-24T06:28:53.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7fd60ea00640 1 --2- 172.21.15.59:0/1396808100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 0x7fd6100d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323665782 0 0) 0x7fd61015b860 con 0x7fd61015ab80 2024-04-24T06:28:53.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.861+0000 7f657e600640 1 Processor -- start 2024-04-24T06:28:53.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.862+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5f4004630 con 0x7fd61015ab80 2024-04-24T06:28:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.862+0000 7f657e600640 1 -- start start 2024-04-24T06:28:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.863+0000 7fa4e8a00640 1 -- 172.21.15.59:0/2169570312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4007740 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.863+0000 7fa4e8a00640 1 -- 172.21.15.59:0/2169570312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e40078e0 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.863+0000 7fa4e8a00640 1 -- 172.21.15.59:0/2169570312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4007cb0 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7efc12a00640 1 Processor -- start 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c70068a80 con 0x7f0c700d10f0 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f657e600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 0x7f65780cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f657e600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f657815f880 0x7f65780d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f657e600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65780cf930 con 0x7f657815ea40 2024-04-24T06:28:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f657e600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f65780cfc10 con 0x7f657815f880 2024-04-24T06:28:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f657e600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65780cfef0 con 0x7f65780d0f80 2024-04-24T06:28:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 -- 172.21.15.59:0/2169570312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 msgr2=0x7fa4ec0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0d1580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402a570 comp rx=0 tx=0).stop 2024-04-24T06:28:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc20005290 con 0x7fbc400498d0 2024-04-24T06:28:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 -- 172.21.15.59:0/2169570312 shutdown_connections 2024-04-24T06:28:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc3401a020 con 0x7fbc400498d0 2024-04-24T06:28:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/2169570312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0d1580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f6577e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f657815ea40 0x7f65780d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60036/0 (socket says 172.21.15.59:60036) 2024-04-24T06:28:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7f6577e00640 1 -- 172.21.15.59:0/834111135 learned_addr learned my addr 172.21.15.59:0/834111135 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7efc12a00640 1 -- start start 2024-04-24T06:28:53.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.864+0000 7fa4f0a00640 1 -- 172.21.15.59:0/2169570312 >> 172.21.15.59:0/2169570312 conn(0x7fa4ec08bf90 msgr2=0x7fa4ec0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.865+0000 7f657d200640 1 --2- 172.21.15.59:0/834111135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 0x7f65780cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.865+0000 7f657dc00640 1 --2- 172.21.15.59:0/834111135 >> v2:172.21.15.66:3300/0 conn(0x7f657815f880 0x7f65780d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.865+0000 7ff8f7c00640 1 Processor -- start 2024-04-24T06:28:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7efc12a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc12a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc12a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc0c0b3430 0x7efc0c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc12a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0b3a20 con 0x7efc0c071d90 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc12a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0b38a0 con 0x7efc0c0b3430 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc12a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c166430 con 0x7efc0c072880 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7fa4f0a00640 1 -- 172.21.15.59:0/2169570312 shutdown_connections 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7f657d200640 1 -- 172.21.15.59:0/834111135 >> v2:172.21.15.66:3300/0 conn(0x7f657815f880 msgr2=0x7f65780d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7f657d200640 1 --2- 172.21.15.59:0/834111135 >> v2:172.21.15.66:3300/0 conn(0x7f657815f880 0x7f65780d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7f657d200640 1 -- 172.21.15.59:0/834111135 >> v1:172.21.15.59:6789/0 conn(0x7f657815ea40 legacy=0x7f65780d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7f657d200640 1 -- 172.21.15.59:0/834111135 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780d01d0 con 0x7f65780d0f80 2024-04-24T06:28:53.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.862+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 <== mon.1 v1:172.21.15.66:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 849657610 0 0) 0x7fd5f4004630 con 0x7fd61015ab80 2024-04-24T06:28:53.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7fd60ea00640 1 -- 172.21.15.59:0/1396808100 >> v1:172.21.15.66:6789/0 conn(0x7fd61015ab80 legacy=0x7fd61015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7fd60ea00640 1 -- 172.21.15.59:0/1396808100 >> v1:172.21.15.59:6789/0 conn(0x7fd6100b34a0 legacy=0x7fd61015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7fd60ea00640 1 -- 172.21.15.59:0/1396808100 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61015bb40 con 0x7fd6100d01d0 2024-04-24T06:28:53.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7fa4f0a00640 1 -- 172.21.15.59:0/2169570312 wait complete. 2024-04-24T06:28:53.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.866+0000 7ff8f7c00640 1 -- start start 2024-04-24T06:28:53.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38136/0 (socket says 172.21.15.59:38136) 2024-04-24T06:28:53.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc0be00640 1 -- 172.21.15.59:0/1326288217 learned_addr learned my addr 172.21.15.59:0/1326288217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.867+0000 7efc0b400640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8f7c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff8f015e9c0 0x7ff8f00d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8f7c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff8f00d1800 0x7ff8f00d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8f7c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f00cf3c0 con 0x7ff8f00d2d70 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8f7c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f00cf640 con 0x7ff8f00d1800 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8f7c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f00cf900 con 0x7ff8f015e9c0 2024-04-24T06:28:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7fbc3ea00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 0x7fbc2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7f657d200640 1 --2- 172.21.15.59:0/834111135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 0x7f65780cf250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097a0 tx=0x7f656c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d653062f702d8490 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7efc0be00640 1 -- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 msgr2=0x7efc0c165690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7efc0be00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c165690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7efc0be00640 1 -- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc0c0b3430 msgr2=0x7efc0c1541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7efc0be00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc0c0b3430 0x7efc0c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7efc0be00640 1 -- 172.21.15.59:0/1326288217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c0d0160 con 0x7efc0c071d90 2024-04-24T06:28:53.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc3408b320 con 0x7fbc400498d0 2024-04-24T06:28:53.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7fd60ea00640 1 --2- 172.21.15.59:0/1396808100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 0x7fd6100d05b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd600009dc0 tx=0x7fd60002a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93862ebe4ef2dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8efe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff8f00d2d70 0x7ff8f00d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60038/0 (socket says 172.21.15.59:60038) 2024-04-24T06:28:53.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.868+0000 7ff8efe00640 1 -- 172.21.15.59:0/874837250 learned_addr learned my addr 172.21.15.59:0/874837250 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:53.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7ff8f5200640 1 --2- 172.21.15.59:0/874837250 >> v2:172.21.15.66:3300/0 conn(0x7ff8f015e9c0 0x7ff8f00d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7f6577400640 1 -- 172.21.15.59:0/834111135 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c037070 con 0x7f65780d0f80 2024-04-24T06:28:53.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7f6577400640 1 -- 172.21.15.59:0/834111135 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c02ae40 con 0x7f65780d0f80 2024-04-24T06:28:53.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7f6577400640 1 -- 172.21.15.59:0/834111135 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c032660 con 0x7f65780d0f80 2024-04-24T06:28:53.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 0x7fbc2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7ff8f5c00640 1 --2- 172.21.15.59:0/874837250 >> v2:172.21.15.59:3300/0 conn(0x7ff8f00d1800 0x7ff8f00d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.869+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600035020 con 0x7fd6100d01d0 2024-04-24T06:28:53.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7efc0be00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc00009870 tx=0x7efc000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7fa4f0a00640 1 Processor -- start 2024-04-24T06:28:53.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7ff8f5200640 1 -- 172.21.15.59:0/874837250 >> v2:172.21.15.59:3300/0 conn(0x7ff8f00d1800 msgr2=0x7ff8f00d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:53.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7ff8f5200640 1 --2- 172.21.15.59:0/874837250 >> v2:172.21.15.59:3300/0 conn(0x7ff8f00d1800 0x7ff8f00d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7fbc46a00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 0x7fbc2806ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc38033000 tx=0x7fbc38009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd600007540 con 0x7fd6100d01d0 2024-04-24T06:28:53.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7fd60e000640 1 -- 172.21.15.59:0/1396808100 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608004cd0 con 0x7fd6100d01d0 2024-04-24T06:28:53.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7fa4f0a00640 1 -- start start 2024-04-24T06:28:53.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7ff8f5200640 1 -- 172.21.15.59:0/874837250 >> v1:172.21.15.59:6789/0 conn(0x7ff8f00d2d70 legacy=0x7ff8f00d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7f657e600640 1 -- 172.21.15.59:0/834111135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 msgr2=0x7f65780cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7f657e600640 1 --2- 172.21.15.59:0/834111135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 0x7f65780cf250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097a0 tx=0x7f656c02a0f0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc34021030 con 0x7fbc400498d0 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7f657e600640 1 -- 172.21.15.59:0/834111135 shutdown_connections 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7f657e600640 1 --2- 172.21.15.59:0/834111135 >> v2:172.21.15.66:3300/0 conn(0x7f657815f880 0x7f65780d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7f657e600640 1 --2- 172.21.15.59:0/834111135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65780d0f80 0x7f65780cf250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7f657e600640 1 -- 172.21.15.59:0/834111135 >> 172.21.15.59:0/834111135 conn(0x7f657808bf90 msgr2=0x7f65780d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.870+0000 7ff8f5200640 1 -- 172.21.15.59:0/874837250 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00cfbe0 con 0x7ff8f015e9c0 2024-04-24T06:28:53.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7f657e600640 1 -- 172.21.15.59:0/834111135 shutdown_connections 2024-04-24T06:28:53.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 -- 172.21.15.59:0/1396808100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 msgr2=0x7fd6100d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 --2- 172.21.15.59:0/1396808100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 0x7fd6100d05b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd600009dc0 tx=0x7fd60002a580 comp rx=0 tx=0).stop 2024-04-24T06:28:53.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 -- 172.21.15.59:0/1396808100 shutdown_connections 2024-04-24T06:28:53.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.872+0000 7f657e600640 1 -- 172.21.15.59:0/834111135 wait complete. 2024-04-24T06:28:53.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 --2- 172.21.15.59:0/1396808100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6100d01d0 0x7fd6100d05b0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd600009dc0 tx=0x7fd60002a580 comp rx=0 tx=0).stop 2024-04-24T06:28:53.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 -- 172.21.15.59:0/1396808100 >> 172.21.15.59:0/1396808100 conn(0x7fd61006b9d0 msgr2=0x7fd61006bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7fd615800640 1 -- 172.21.15.59:0/1396808100 shutdown_connections 2024-04-24T06:28:53.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00041020 con 0x7efc0c071d90 2024-04-24T06:28:53.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.872+0000 7fd615800640 1 -- 172.21.15.59:0/1396808100 wait complete. 2024-04-24T06:28:53.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.872+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c153100 con 0x7efc0c071d90 2024-04-24T06:28:53.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.872+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1536c0 con 0x7efc0c071d90 2024-04-24T06:28:53.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.872+0000 7ff8f5200640 1 --2- 172.21.15.59:0/874837250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f00d03e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8f008caf0 tx=0x7ff8e402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0ff650fdac179b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.871+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00030800 con 0x7efc0c071d90 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.873+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00037040 con 0x7efc0c071d90 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec200f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 0x7fa4ec0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec06cb90 con 0x7fa4ec0d10f0 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec0d1ac0 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fa4f0a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec06cd10 con 0x7fa4ec0d3480 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7f657e600640 1 Processor -- start 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c600329d0 con 0x7f0c700d10f0 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38144/0 (socket says 172.21.15.59:38144) 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7fa4e9e00640 1 -- 172.21.15.59:0/3511690756 learned_addr learned my addr 172.21.15.59:0/3511690756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7fa4ea800640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 0x7fa4ec0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.875+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c0b1f50 con 0x7efc0c071d90 2024-04-24T06:28:53.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4e9400640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec200f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7f0c67e00640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4ea800640 1 -- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 msgr2=0x7fa4ec200f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c600ac200 con 0x7f0c700d10f0 2024-04-24T06:28:53.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4ea800640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec200f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4ea800640 1 -- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 msgr2=0x7fa4ec0b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4ea800640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4ea800640 1 -- 172.21.15.59:0/3511690756 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0b8dc0 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fa4e9e00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:28:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc00044020 con 0x7efc0c071d90 2024-04-24T06:28:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7f657e600640 1 -- start start 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fd615800640 1 Processor -- start 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.874+0000 7fd615800640 1 -- start start 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 0x7fd6101f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 0x7fd6101fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd61015e880 con 0x7fd6100b34a0 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd61015e5a0 con 0x7fd61015ab80 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.876+0000 7fd615800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd61006c4f0 con 0x7fd610147d40 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38150/0 (socket says 172.21.15.59:38150) 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 -- 172.21.15.59:0/2005581253 learned_addr learned my addr 172.21.15.59:0/2005581253 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 -- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 msgr2=0x7fd6101fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60f400640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 0x7fd6101fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd606000640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 0x7fd6101f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 0x7fd6101fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7fd60ea00640 1 -- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 msgr2=0x7fd6101f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 0x7f65780cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f657815ea40 0x7f65780b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 0x7f65780ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65780d2850 con 0x7f657815f880 2024-04-24T06:28:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f657815fce0 con 0x7f65780d0f80 2024-04-24T06:28:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.877+0000 7f657e600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65780d29d0 con 0x7f657815ea40 2024-04-24T06:28:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7fd60ea00640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 0x7fd6101f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7fd60ea00640 1 -- 172.21.15.59:0/2005581253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61014f330 con 0x7fd6100b34a0 2024-04-24T06:28:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7f0c6f400640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7fd60f400640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 0x7fd6101fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7fd606000640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 0x7fd6101f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7f657dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 0x7f65780ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7f657d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 0x7f65780cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7f657d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 0x7f65780cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:41466/0 (socket says 172.21.15.59:41466) 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7f657d200640 1 -- 172.21.15.59:0/852398997 learned_addr learned my addr 172.21.15.59:0/852398997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7fd60ea00640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ee0 tx=0x7fd600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f0c6f400640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c700b2720 tx=0x7f0c5c009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7efbffe00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 0x7efbec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.878+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efc000ac400 con 0x7efc0c071d90 2024-04-24T06:28:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 -- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f657815ea40 msgr2=0x7f65780b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f657815ea40 0x7f65780b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 -- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 msgr2=0x7f65780cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 0x7f65780cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 -- 172.21.15.59:0/852398997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780b8db0 con 0x7f657815f880 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7fa4ea800640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 0x7fa4ec0cdd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4031000 tx=0x7fa4e4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7f657dc00640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 0x7f65780ba070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f657000ebd0 tx=0x7f657000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.879+0000 7efc0b400640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 0x7efbec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.881+0000 7efc0b400640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 0x7efbec06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0b01a0 tx=0x7efbf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60003f070 con 0x7fd6100b34a0 2024-04-24T06:28:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd600003bb0 con 0x7fd6100b34a0 2024-04-24T06:28:53.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600035070 con 0x7fd6100b34a0 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61014f610 con 0x7fd6100b34a0 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570018020 con 0x7f657815f880 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657000cd20 con 0x7f657815f880 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700146c0 con 0x7f657815f880 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65780b7610 con 0x7f657815f880 2024-04-24T06:28:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.882+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101500f0 con 0x7fd6100b34a0 2024-04-24T06:28:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.883+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65780b9050 con 0x7f657815f880 2024-04-24T06:28:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc0c0b1f50 con 0x7efc0c071d90 2024-04-24T06:28:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd610068a80 con 0x7fd6100b34a0 2024-04-24T06:28:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657804fd90 con 0x7f657815f880 2024-04-24T06:28:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7ff8ef400640 1 -- 172.21.15.59:0/874837250 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8f015e9c0 2024-04-24T06:28:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7ff8ef400640 1 -- 172.21.15.59:0/874837250 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e402ae50 con 0x7ff8f015e9c0 2024-04-24T06:28:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7ff8ef400640 1 -- 172.21.15.59:0/874837250 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8f015e9c0 2024-04-24T06:28:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.885+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c60079fd0 con 0x7f0c700d10f0 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4040070 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.884+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0ba220 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.885+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4031b80 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.885+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4045460 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.886+0000 7ff8f7c00640 1 -- 172.21.15.59:0/874837250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 msgr2=0x7ff8f00d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.886+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/874837250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f00d03e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8f008caf0 tx=0x7ff8e402a100 comp rx=0 tx=0).stop 2024-04-24T06:28:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.886+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0cadc0 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.886+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6570005d10 con 0x7f657815f880 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7f6574c00640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 0x7f655806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7ff8f7c00640 1 -- 172.21.15.59:0/874837250 shutdown_connections 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/874837250 >> v2:172.21.15.59:3300/0 conn(0x7ff8f00d1800 0x7ff8f00d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/874837250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f00d03e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7ff8f7c00640 1 -- 172.21.15.59:0/874837250 >> 172.21.15.59:0/874837250 conn(0x7ff8f008bf90 msgr2=0x7ff8f00d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.887+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f657008c700 con 0x7f657815f880 2024-04-24T06:28:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.888+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec068a80 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.888+0000 7ff8f7c00640 1 -- 172.21.15.59:0/874837250 shutdown_connections 2024-04-24T06:28:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.888+0000 7ff8f7c00640 1 -- 172.21.15.59:0/874837250 wait complete. 2024-04-24T06:28:53.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.890+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd600042070 con 0x7fd6100b34a0 2024-04-24T06:28:53.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.888+0000 7f657d200640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 0x7f655806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.889+0000 7f657d200640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 0x7f655806ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f656c009560 tx=0x7f656c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.891+0000 7fd607400640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 0x7fd5ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.891+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6000acf20 con 0x7fd6100b34a0 2024-04-24T06:28:53.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.890+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4e4039040 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.891+0000 7ff8f7c00640 1 Processor -- start 2024-04-24T06:28:53.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.891+0000 7ff8f7c00640 1 -- start start 2024-04-24T06:28:53.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.892+0000 7fa4e2000640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 0x7fa4cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.892+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e40ad290 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 0x7ff8f00b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f015fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f006cd10 con 0x7ff8f00d1800 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f006cb90 con 0x7ff8f015e9c0 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f7c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f006ce90 con 0x7ff8f00d2d70 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38160/0 (socket says 172.21.15.59:38160) 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7ff8f5200640 1 -- 172.21.15.59:0/3480716452 learned_addr learned my addr 172.21.15.59:0/3480716452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8efe00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 0x7ff8f00b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8f5200640 1 -- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 msgr2=0x7ff8f00b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8f5200640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 0x7ff8f00b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8f5200640 1 -- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 msgr2=0x7ff8f015fbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8f5200640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f015fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8f5200640 1 -- 172.21.15.59:0/3480716452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00b5670 con 0x7ff8f00d1800 2024-04-24T06:28:53.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7ff8efe00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 0x7ff8f00b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:53.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.893+0000 7fd606000640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 0x7fd5ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.894+0000 7fd606000640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 0x7fd5ec06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6100afa70 tx=0x7fd5f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.895+0000 7fa4e9e00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 0x7fa4cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.895+0000 7ff8f5200640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4035040 tx=0x7ff8e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.895+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd60007b4e0 con 0x7fd6100b34a0 2024-04-24T06:28:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.896+0000 7fa4e9e00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 0x7fa4cc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004380 tx=0x7fa4dc007ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.903+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037050 con 0x7ff8f00d1800 2024-04-24T06:28:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.903+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f00b5950 con 0x7ff8f00d1800 2024-04-24T06:28:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.903+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f00b5f10 con 0x7ff8f00d1800 2024-04-24T06:28:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.903+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4043070 con 0x7ff8f00d1800 2024-04-24T06:28:53.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.903+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6570022030 con 0x7f657815f880 2024-04-24T06:28:53.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.904+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4032650 con 0x7ff8f00d1800 2024-04-24T06:28:53.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.905+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc005290 con 0x7ff8f00d1800 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.906+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4ec0cf3c0 con 0x7fa4ec15e9c0 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.908+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8e40327f0 con 0x7ff8f00d1800 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.910+0000 7ff8ecc00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 0x7ff8d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.910+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8e40acf70 con 0x7ff8f00d1800 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.911+0000 7ff8efe00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 0x7ff8d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.914+0000 7ff8efe00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 0x7ff8d006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8f01611e0 tx=0x7ff8dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:53.916+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8e407b390 con 0x7ff8f00d1800 2024-04-24T06:28:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.282+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7effa80b6140 con 0x7effa815de40 2024-04-24T06:28:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.284+0000 7effa5600640 1 -- 172.21.15.59:0/4247086879 <== mon.2 v2:172.21.15.66:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effa005ddd0 con 0x7effa815de40 2024-04-24T06:28:54.285 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:54.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.288+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 msgr2=0x7eff8806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.288+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 0x7eff8806ee30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff9c034000 tx=0x7eff9c02e000 comp rx=0 tx=0).stop 2024-04-24T06:28:54.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.288+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 msgr2=0x7effa815ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.288+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7effa000caf0 tx=0x7effa000a060 comp rx=0 tx=0).stop 2024-04-24T06:28:54.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 shutdown_connections 2024-04-24T06:28:54.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff8806c970 0x7eff8806ee30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effa815de40 0x7effa815ba10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa80c59f0 0x7effa815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 --2- 172.21.15.59:0/4247086879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effa80b2830 0x7effa80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.289+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 >> 172.21.15.59:0/4247086879 conn(0x7effa80b1d10 msgr2=0x7effa806dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.290+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 shutdown_connections 2024-04-24T06:28:54.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.290+0000 7effafe00640 1 -- 172.21.15.59:0/4247086879 wait complete. 2024-04-24T06:28:54.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.311+0000 7fbc47e00640 1 -- 172.21.15.59:0/3946619184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbc20005580 con 0x7fbc400498d0 2024-04-24T06:28:54.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.312+0000 7fbc3ea00640 1 -- 172.21.15.59:0/3946619184 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc340597c0 con 0x7fbc400498d0 2024-04-24T06:28:54.313 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:54.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.316+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 msgr2=0x7fbc2806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.316+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 0x7fbc2806ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc38033000 tx=0x7fbc38009030 comp rx=0 tx=0).stop 2024-04-24T06:28:54.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.316+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 msgr2=0x7fbc400505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.316+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e4b0 tx=0x7fbc3400e970 comp rx=0 tx=0).stop 2024-04-24T06:28:54.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-24T06:28:54.319 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 shutdown_connections 2024-04-24T06:28:54.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c9c0 0x7fbc2806ee80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc400d6ec0 0x7fbc400d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc400498d0 0x7fbc400505b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 --2- 172.21.15.59:0/3946619184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc2c0039d0 0x7fbc400d9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 >> 172.21.15.59:0/3946619184 conn(0x7fbc400537b0 msgr2=0x7fbc40004e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 shutdown_connections 2024-04-24T06:28:54.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.317+0000 7fbc33e00640 1 -- 172.21.15.59:0/3946619184 wait complete. 2024-04-24T06:28:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.322+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efc0c068a80 con 0x7efc0c071d90 2024-04-24T06:28:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.323+0000 7efbffe00640 1 -- 172.21.15.59:0/1326288217 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc0004b030 con 0x7efc0c071d90 2024-04-24T06:28:54.324 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.326+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 msgr2=0x7efbec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.326+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 0x7efbec06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0b01a0 tx=0x7efbf8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.326+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 msgr2=0x7efc0c0cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.326+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc00009870 tx=0x7efc000046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 shutdown_connections 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efbec06c940 0x7efbec06ee00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc0c0b3430 0x7efc0c1541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc0c072880 0x7efc0c165690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 --2- 172.21.15.59:0/1326288217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc0c071d90 0x7efc0c0cfc20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 >> 172.21.15.59:0/1326288217 conn(0x7efc0c0b2130 msgr2=0x7efc0c0b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 shutdown_connections 2024-04-24T06:28:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.327+0000 7efc12a00640 1 -- 172.21.15.59:0/1326288217 wait complete. 2024-04-24T06:28:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.335+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0c700d1f40 con 0x7f0c700d10f0 2024-04-24T06:28:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.336+0000 7f0c67e00640 1 -- 172.21.15.59:0/968311441 <== mon.1 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c6007e2d0 con 0x7f0c700d10f0 2024-04-24T06:28:54.337 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.339+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 msgr2=0x7f0c5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.339+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c700b2720 tx=0x7f0c5c009110 comp rx=0 tx=0).stop 2024-04-24T06:28:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.339+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 msgr2=0x7f0c700c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.339+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009790 tx=0x7f0c600046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 shutdown_connections 2024-04-24T06:28:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c7015e9c0 0x7f0c700ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c700d2d70 0x7f0c70161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 --2- 172.21.15.59:0/968311441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d10f0 0x7f0c700c83b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 >> 172.21.15.59:0/968311441 conn(0x7f0c7008bf90 msgr2=0x7f0c700b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 shutdown_connections 2024-04-24T06:28:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.340+0000 7f0c77000640 1 -- 172.21.15.59:0/968311441 wait complete. 2024-04-24T06:28:54.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-24T06:28:54.343 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.345+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f657806ead0 con 0x7f657815f880 2024-04-24T06:28:54.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.345+0000 7f6574c00640 1 -- 172.21.15.59:0/852398997 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f657008ae20 con 0x7f657815f880 2024-04-24T06:28:54.346 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.348+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 msgr2=0x7f655806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.348+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 0x7f655806ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f656c009560 tx=0x7f656c02e000 comp rx=0 tx=0).stop 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.348+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 msgr2=0x7f65780ba070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.348+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 0x7f65780ba070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f657000ebd0 tx=0x7f657000c490 comp rx=0 tx=0).stop 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 shutdown_connections 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f655806c940 0x7f655806ee00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f657815f880 0x7f65780ba070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f657815ea40 0x7f65780b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 --2- 172.21.15.59:0/852398997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65780d0f80 0x7f65780cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 >> 172.21.15.59:0/852398997 conn(0x7f657808bf90 msgr2=0x7f65780d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 shutdown_connections 2024-04-24T06:28:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.349+0000 7f657e600640 1 -- 172.21.15.59:0/852398997 wait complete. 2024-04-24T06:28:54.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.351+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff8cc005850 con 0x7ff8f00d1800 2024-04-24T06:28:54.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.352+0000 7ff8ecc00640 1 -- 172.21.15.59:0/3480716452 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff8e407ac90 con 0x7ff8f00d1800 2024-04-24T06:28:54.353 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-24T06:28:54.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-24T06:28:54.357 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 msgr2=0x7ff8d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 0x7ff8d006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8f01611e0 tx=0x7ff8dc009040 comp rx=0 tx=0).stop 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 msgr2=0x7ff8f00ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4035040 tx=0x7ff8e40046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 shutdown_connections 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8d006c8f0 0x7ff8d006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8f015e9c0 0x7ff8f015fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8f00d2d70 0x7ff8f00b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 --2- 172.21.15.59:0/3480716452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f00d1800 0x7ff8f00ccc30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.355+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 >> 172.21.15.59:0/3480716452 conn(0x7ff8f008bf90 msgr2=0x7ff8f00b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.356+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 shutdown_connections 2024-04-24T06:28:54.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.356+0000 7ff8f7c00640 1 -- 172.21.15.59:0/3480716452 wait complete. 2024-04-24T06:28:54.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.358+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd6100ccc50 con 0x7fd6100b34a0 2024-04-24T06:28:54.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.358+0000 7fd607400640 1 -- 172.21.15.59:0/2005581253 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd60007ade0 con 0x7fd6100b34a0 2024-04-24T06:28:54.360 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-24T06:28:54.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-24T06:28:54.362 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 msgr2=0x7fd5ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 0x7fd5ec06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6100afa70 tx=0x7fd5f8009040 comp rx=0 tx=0).stop 2024-04-24T06:28:54.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 msgr2=0x7fd610153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ee0 tx=0x7fd600004a40 comp rx=0 tx=0).stop 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 shutdown_connections 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5ec06c8f0 0x7fd5ec06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd610147d40 0x7fd6101fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd61015ab80 0x7fd6101f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 --2- 172.21.15.59:0/2005581253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6100b34a0 0x7fd610153a10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 >> 172.21.15.59:0/2005581253 conn(0x7fd61006b9d0 msgr2=0x7fd6100cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.362+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 shutdown_connections 2024-04-24T06:28:54.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.363+0000 7fd615800640 1 -- 172.21.15.59:0/2005581253 wait complete. 2024-04-24T06:28:54.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.365+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa4ec0d1f40 con 0x7fa4ec15e9c0 2024-04-24T06:28:54.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.366+0000 7fa4e2000640 1 -- 172.21.15.59:0/3511690756 <== mon.1 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4e407b740 con 0x7fa4ec15e9c0 2024-04-24T06:28:54.367 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.368+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 msgr2=0x7fa4cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.368+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 0x7fa4cc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004380 tx=0x7fa4dc007ef0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.368+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 msgr2=0x7fa4ec0cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.368+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 0x7fa4ec0cdd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4031000 tx=0x7fa4e4004a40 comp rx=0 tx=0).stop 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 shutdown_connections 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4cc06c8f0 0x7fa4cc06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4ec15e9c0 0x7fa4ec0cdd10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4ec0d3480 0x7fa4ec200f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 --2- 172.21.15.59:0/3511690756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4ec0d10f0 0x7fa4ec0b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 >> 172.21.15.59:0/3511690756 conn(0x7fa4ec08bf90 msgr2=0x7fa4ec0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 shutdown_connections 2024-04-24T06:28:54.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.369+0000 7fa4f0a00640 1 -- 172.21.15.59:0/3511690756 wait complete. 2024-04-24T06:28:54.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-24T06:28:54.372 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-24T06:28:54.377 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-24T06:28:54.383 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-24T06:28:54.386 DEBUG:teuthology.parallel:result is None 2024-04-24T06:28:54.387 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T06:28:54.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:28:54.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.616+0000 7fe5b2400640 1 Processor -- start 2024-04-24T06:28:54.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.617+0000 7fe5b2400640 1 -- start start 2024-04-24T06:28:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 0x7fe5ac0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b2400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5ac160210 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b2400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5ac1604f0 con 0x7fe5ac0b4690 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b2400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac1607d0 con 0x7fe5ac0cd760 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b1000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe5ac15c3d0 0x7fe5ac0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60052/0 (socket says 172.21.15.59:60052) 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.618+0000 7fe5b1000640 1 -- 172.21.15.59:0/1096732463 learned_addr learned my addr 172.21.15.59:0/1096732463 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704369826 0 0) 0x7fe5ac160210 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe594003520 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5abe00640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 0x7fe5ac0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713190970 0 0) 0x7fe594003520 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 msgr2=0x7fe5ac0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 0x7fe5ac0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 >> v1:172.21.15.66:6789/0 conn(0x7fe5ac0b4690 legacy=0x7fe5ac0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac160ab0 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe5a0001670 con 0x7fe5ac15c3d0 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 >> v1:172.21.15.59:6789/0 conn(0x7fe5ac15c3d0 legacy=0x7fe5ac0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5940039e0 0x7fe594003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe594004800 0x7fe594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 0x7fe594005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac1607d0 con 0x7fe594004800 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe59400bb50 con 0x7fe5940039e0 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.619+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe59400bd50 con 0x7fe5ac0cd760 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe594004800 0x7fe594004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5b1000640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5940039e0 0x7fe594003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 -- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 msgr2=0x7fe594005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0cd760 0x7fe594005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 -- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5940039e0 msgr2=0x7fe594003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5940039e0 0x7fe594003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5abe00640 1 -- 172.21.15.59:0/1096732463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe594003520 con 0x7fe594004800 2024-04-24T06:28:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.620+0000 7fe5b1000640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5940039e0 0x7fe594003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.621+0000 7fe5abe00640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe594004800 0x7fe594004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe59800ba30 tx=0x7fe594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51037ae0331df65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.621+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598030070 con 0x7fe594004800 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.621+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59802ae10 con 0x7fe594004800 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.621+0000 7fe5ab400640 1 -- 172.21.15.59:0/1096732463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800b600 con 0x7fe594004800 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 -- 172.21.15.59:0/1096732463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe594004800 msgr2=0x7fe594004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 --2- 172.21.15.59:0/1096732463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe594004800 0x7fe594004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe59800ba30 tx=0x7fe594005830 comp rx=0 tx=0).stop 2024-04-24T06:28:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b1a00640 1 -- 172.21.15.59:0/1096732463 reap_dead start 2024-04-24T06:28:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 -- 172.21.15.59:0/1096732463 shutdown_connections 2024-04-24T06:28:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 -- 172.21.15.59:0/1096732463 >> 172.21.15.59:0/1096732463 conn(0x7fe5ac0b21a0 msgr2=0x7fe5ac0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 -- 172.21.15.59:0/1096732463 shutdown_connections 2024-04-24T06:28:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.622+0000 7fe5b2400640 1 -- 172.21.15.59:0/1096732463 wait complete. 2024-04-24T06:28:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.624+0000 7fe5b2400640 1 Processor -- start 2024-04-24T06:28:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.624+0000 7fe5b2400640 1 -- start start 2024-04-24T06:28:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe594004800 0x7fe5ac0c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0b13a0 0x7fe5ac09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac15c7b0 con 0x7fe5ac0ad060 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac0b4ac0 con 0x7fe594004800 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b2400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac15c930 con 0x7fe5ac0b13a0 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38174/0 (socket says 172.21.15.59:38174) 2024-04-24T06:28:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.625+0000 7fe5b1a00640 1 -- 172.21.15.59:0/2719615086 learned_addr learned my addr 172.21.15.59:0/2719615086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1000640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe594004800 0x7fe5ac0c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 -- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0b13a0 msgr2=0x7fe5ac09dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0b13a0 0x7fe5ac09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 -- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe594004800 msgr2=0x7fe5ac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe594004800 0x7fe5ac0c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 -- 172.21.15.59:0/2719615086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac0c6af0 con 0x7fe5ac0ad060 2024-04-24T06:28:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b1a00640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06ffe0 tx=0x7fe5a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4005d40 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a400a610 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400aa00 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac0ada20 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.626+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac0adcc0 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.628+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac04fd90 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.629+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5a4011070 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.629+0000 7fe5a8c00640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.630+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe5a408ab90 con 0x7fe5ac0ad060 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.630+0000 7fe5b1000640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.630+0000 7fe5b1000640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 0x7fe58c06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001200 tx=0x7fe5a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.634+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5a4019060 con 0x7fe5ac0ad060 2024-04-24T06:28:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.826+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5ac06e380 con 0x7fe58c06c8f0 2024-04-24T06:28:54.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.840+0000 7fe5a8c00640 1 -- 172.21.15.59:0/2719615086 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fe5ac06e380 con 0x7fe58c06c8f0 2024-04-24T06:28:54.843 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:28:54.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 msgr2=0x7fe58c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 0x7fe58c06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001200 tx=0x7fe5a0009040 comp rx=0 tx=0).stop 2024-04-24T06:28:54.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 msgr2=0x7fe5ac0ad4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06ffe0 tx=0x7fe5a400cb90 comp rx=0 tx=0).stop 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 shutdown_connections 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe58c06c8f0 0x7fe58c06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5ac0ad060 0x7fe5ac0ad4e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe5ac0b13a0 0x7fe5ac09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 --2- 172.21.15.59:0/2719615086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe594004800 0x7fe5ac0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 >> 172.21.15.59:0/2719615086 conn(0x7fe5ac0b21a0 msgr2=0x7fe5ac0c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 shutdown_connections 2024-04-24T06:28:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:54.846+0000 7fe5b2400640 1 -- 172.21.15.59:0/2719615086 wait complete. 2024-04-24T06:28:54.863 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-24T06:28:53.189912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.481560"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994473+0000","last_change":"2024-04-24T06:28:44.737375+0000","last_active":"2024-04-24T06:28:45.994473+0000","last_peered":"2024-04-24T06:28:45.994473+0000","last_clean":"2024-04-24T06:28:45.994473+0000","last_became_active":"2024-04-24T06:28:44.737064+0000","last_became_peered":"2024-04-24T06:28:44.737064+0000","last_unstale":"2024-04-24T06:28:45.994473+0000","last_undegraded":"2024-04-24T06:28:45.994473+0000","last_fullsized":"2024-04-24T06:28:45.994473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991754+0000","last_change":"2024-04-24T06:28:44.748671+0000","last_active":"2024-04-24T06:28:45.991754+0000","last_peered":"2024-04-24T06:28:45.991754+0000","last_clean":"2024-04-24T06:28:45.991754+0000","last_became_active":"2024-04-24T06:28:44.748393+0000","last_became_peered":"2024-04-24T06:28:44.748393+0000","last_unstale":"2024-04-24T06:28:45.991754+0000","last_undegraded":"2024-04-24T06:28:45.991754+0000","last_fullsized":"2024-04-24T06:28:45.991754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990411+0000","last_change":"2024-04-24T06:28:44.738384+0000","last_active":"2024-04-24T06:28:45.990411+0000","last_peered":"2024-04-24T06:28:45.990411+0000","last_clean":"2024-04-24T06:28:45.990411+0000","last_became_active":"2024-04-24T06:28:44.738063+0000","last_became_peered":"2024-04-24T06:28:44.738063+0000","last_unstale":"2024-04-24T06:28:45.990411+0000","last_undegraded":"2024-04-24T06:28:45.990411+0000","last_fullsized":"2024-04-24T06:28:45.990411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991159+0000","last_change":"2024-04-24T06:28:44.754483+0000","last_active":"2024-04-24T06:28:45.991159+0000","last_peered":"2024-04-24T06:28:45.991159+0000","last_clean":"2024-04-24T06:28:45.991159+0000","last_became_active":"2024-04-24T06:28:44.754150+0000","last_became_peered":"2024-04-24T06:28:44.754150+0000","last_unstale":"2024-04-24T06:28:45.991159+0000","last_undegraded":"2024-04-24T06:28:45.991159+0000","last_fullsized":"2024-04-24T06:28:45.991159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993216+0000","last_change":"2024-04-24T06:28:44.741673+0000","last_active":"2024-04-24T06:28:45.993216+0000","last_peered":"2024-04-24T06:28:45.993216+0000","last_clean":"2024-04-24T06:28:45.993216+0000","last_became_active":"2024-04-24T06:28:44.741361+0000","last_became_peered":"2024-04-24T06:28:44.741361+0000","last_unstale":"2024-04-24T06:28:45.993216+0000","last_undegraded":"2024-04-24T06:28:45.993216+0000","last_fullsized":"2024-04-24T06:28:45.993216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990171+0000","last_change":"2024-04-24T06:28:44.745828+0000","last_active":"2024-04-24T06:28:45.990171+0000","last_peered":"2024-04-24T06:28:45.990171+0000","last_clean":"2024-04-24T06:28:45.990171+0000","last_became_active":"2024-04-24T06:28:44.745499+0000","last_became_peered":"2024-04-24T06:28:44.745499+0000","last_unstale":"2024-04-24T06:28:45.990171+0000","last_undegraded":"2024-04-24T06:28:45.990171+0000","last_fullsized":"2024-04-24T06:28:45.990171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992379+0000","last_change":"2024-04-24T06:28:44.752057+0000","last_active":"2024-04-24T06:28:45.992379+0000","last_peered":"2024-04-24T06:28:45.992379+0000","last_clean":"2024-04-24T06:28:45.992379+0000","last_became_active":"2024-04-24T06:28:44.746797+0000","last_became_peered":"2024-04-24T06:28:44.746797+0000","last_unstale":"2024-04-24T06:28:45.992379+0000","last_undegraded":"2024-04-24T06:28:45.992379+0000","last_fullsized":"2024-04-24T06:28:45.992379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741286+0000","last_change":"2024-04-24T06:28:44.734221+0000","last_active":"2024-04-24T06:28:45.741286+0000","last_peered":"2024-04-24T06:28:45.741286+0000","last_clean":"2024-04-24T06:28:45.741286+0000","last_became_active":"2024-04-24T06:28:44.733624+0000","last_became_peered":"2024-04-24T06:28:44.733624+0000","last_unstale":"2024-04-24T06:28:45.741286+0000","last_undegraded":"2024-04-24T06:28:45.741286+0000","last_fullsized":"2024-04-24T06:28:45.741286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994911+0000","last_change":"2024-04-24T06:28:44.744186+0000","last_active":"2024-04-24T06:28:45.994911+0000","last_peered":"2024-04-24T06:28:45.994911+0000","last_clean":"2024-04-24T06:28:45.994911+0000","last_became_active":"2024-04-24T06:28:44.743766+0000","last_became_peered":"2024-04-24T06:28:44.743766+0000","last_unstale":"2024-04-24T06:28:45.994911+0000","last_undegraded":"2024-04-24T06:28:45.994911+0000","last_fullsized":"2024-04-24T06:28:45.994911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:00.455831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989452+0000","last_change":"2024-04-24T06:28:44.742684+0000","last_active":"2024-04-24T06:28:45.989452+0000","last_peered":"2024-04-24T06:28:45.989452+0000","last_clean":"2024-04-24T06:28:45.989452+0000","last_became_active":"2024-04-24T06:28:44.742251+0000","last_became_peered":"2024-04-24T06:28:44.742251+0000","last_unstale":"2024-04-24T06:28:45.989452+0000","last_undegraded":"2024-04-24T06:28:45.989452+0000","last_fullsized":"2024-04-24T06:28:45.989452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989977+0000","last_change":"2024-04-24T06:28:44.741312+0000","last_active":"2024-04-24T06:28:45.989977+0000","last_peered":"2024-04-24T06:28:45.989977+0000","last_clean":"2024-04-24T06:28:45.989977+0000","last_became_active":"2024-04-24T06:28:44.740904+0000","last_became_peered":"2024-04-24T06:28:44.740904+0000","last_unstale":"2024-04-24T06:28:45.989977+0000","last_undegraded":"2024-04-24T06:28:45.989977+0000","last_fullsized":"2024-04-24T06:28:45.989977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741756+0000","last_change":"2024-04-24T06:28:44.748944+0000","last_active":"2024-04-24T06:28:45.741756+0000","last_peered":"2024-04-24T06:28:45.741756+0000","last_clean":"2024-04-24T06:28:45.741756+0000","last_became_active":"2024-04-24T06:28:44.748456+0000","last_became_peered":"2024-04-24T06:28:44.748456+0000","last_unstale":"2024-04-24T06:28:45.741756+0000","last_undegraded":"2024-04-24T06:28:45.741756+0000","last_fullsized":"2024-04-24T06:28:45.741756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741786+0000","last_change":"2024-04-24T06:28:44.742242+0000","last_active":"2024-04-24T06:28:45.741786+0000","last_peered":"2024-04-24T06:28:45.741786+0000","last_clean":"2024-04-24T06:28:45.741786+0000","last_became_active":"2024-04-24T06:28:44.741323+0000","last_became_peered":"2024-04-24T06:28:44.741323+0000","last_unstale":"2024-04-24T06:28:45.741786+0000","last_undegraded":"2024-04-24T06:28:45.741786+0000","last_fullsized":"2024-04-24T06:28:45.741786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990162+0000","last_change":"2024-04-24T06:28:44.733381+0000","last_active":"2024-04-24T06:28:45.990162+0000","last_peered":"2024-04-24T06:28:45.990162+0000","last_clean":"2024-04-24T06:28:45.990162+0000","last_became_active":"2024-04-24T06:28:44.733086+0000","last_became_peered":"2024-04-24T06:28:44.733086+0000","last_unstale":"2024-04-24T06:28:45.990162+0000","last_undegraded":"2024-04-24T06:28:45.990162+0000","last_fullsized":"2024-04-24T06:28:45.990162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:46.231913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989857+0000","last_change":"2024-04-24T06:28:44.743567+0000","last_active":"2024-04-24T06:28:45.989857+0000","last_peered":"2024-04-24T06:28:45.989857+0000","last_clean":"2024-04-24T06:28:45.989857+0000","last_became_active":"2024-04-24T06:28:44.743281+0000","last_became_peered":"2024-04-24T06:28:44.743281+0000","last_unstale":"2024-04-24T06:28:45.989857+0000","last_undegraded":"2024-04-24T06:28:45.989857+0000","last_fullsized":"2024-04-24T06:28:45.989857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990473+0000","last_change":"2024-04-24T06:28:44.734065+0000","last_active":"2024-04-24T06:28:45.990473+0000","last_peered":"2024-04-24T06:28:45.990473+0000","last_clean":"2024-04-24T06:28:45.990473+0000","last_became_active":"2024-04-24T06:28:44.733730+0000","last_became_peered":"2024-04-24T06:28:44.733730+0000","last_unstale":"2024-04-24T06:28:45.990473+0000","last_undegraded":"2024-04-24T06:28:45.990473+0000","last_fullsized":"2024-04-24T06:28:45.990473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992895+0000","last_change":"2024-04-24T06:28:44.749923+0000","last_active":"2024-04-24T06:28:45.992895+0000","last_peered":"2024-04-24T06:28:45.992895+0000","last_clean":"2024-04-24T06:28:45.992895+0000","last_became_active":"2024-04-24T06:28:44.749576+0000","last_became_peered":"2024-04-24T06:28:44.749576+0000","last_unstale":"2024-04-24T06:28:45.992895+0000","last_undegraded":"2024-04-24T06:28:45.992895+0000","last_fullsized":"2024-04-24T06:28:45.992895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987678+0000","last_change":"2024-04-24T06:28:44.737360+0000","last_active":"2024-04-24T06:28:45.987678+0000","last_peered":"2024-04-24T06:28:45.987678+0000","last_clean":"2024-04-24T06:28:45.987678+0000","last_became_active":"2024-04-24T06:28:44.737001+0000","last_became_peered":"2024-04-24T06:28:44.737001+0000","last_unstale":"2024-04-24T06:28:45.987678+0000","last_undegraded":"2024-04-24T06:28:45.987678+0000","last_fullsized":"2024-04-24T06:28:45.987678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994589+0000","last_change":"2024-04-24T06:28:44.748893+0000","last_active":"2024-04-24T06:28:45.994589+0000","last_peered":"2024-04-24T06:28:45.994589+0000","last_clean":"2024-04-24T06:28:45.994589+0000","last_became_active":"2024-04-24T06:28:44.748554+0000","last_became_peered":"2024-04-24T06:28:44.748554+0000","last_unstale":"2024-04-24T06:28:45.994589+0000","last_undegraded":"2024-04-24T06:28:45.994589+0000","last_fullsized":"2024-04-24T06:28:45.994589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989639+0000","last_change":"2024-04-24T06:28:44.759322+0000","last_active":"2024-04-24T06:28:45.989639+0000","last_peered":"2024-04-24T06:28:45.989639+0000","last_clean":"2024-04-24T06:28:45.989639+0000","last_became_active":"2024-04-24T06:28:44.758153+0000","last_became_peered":"2024-04-24T06:28:44.758153+0000","last_unstale":"2024-04-24T06:28:45.989639+0000","last_undegraded":"2024-04-24T06:28:45.989639+0000","last_fullsized":"2024-04-24T06:28:45.989639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:44.745627+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:44.745334+0000","last_became_peered":"2024-04-24T06:28:44.745334+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993161+0000","last_change":"2024-04-24T06:28:44.742546+0000","last_active":"2024-04-24T06:28:45.993161+0000","last_peered":"2024-04-24T06:28:45.993161+0000","last_clean":"2024-04-24T06:28:45.993161+0000","last_became_active":"2024-04-24T06:28:44.740862+0000","last_became_peered":"2024-04-24T06:28:44.740862+0000","last_unstale":"2024-04-24T06:28:45.993161+0000","last_undegraded":"2024-04-24T06:28:45.993161+0000","last_fullsized":"2024-04-24T06:28:45.993161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741404+0000","last_change":"2024-04-24T06:28:44.748223+0000","last_active":"2024-04-24T06:28:45.741404+0000","last_peered":"2024-04-24T06:28:45.741404+0000","last_clean":"2024-04-24T06:28:45.741404+0000","last_became_active":"2024-04-24T06:28:44.745248+0000","last_became_peered":"2024-04-24T06:28:44.745248+0000","last_unstale":"2024-04-24T06:28:45.741404+0000","last_undegraded":"2024-04-24T06:28:45.741404+0000","last_fullsized":"2024-04-24T06:28:45.741404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988946+0000","last_change":"2024-04-24T06:28:44.739176+0000","last_active":"2024-04-24T06:28:45.988946+0000","last_peered":"2024-04-24T06:28:45.988946+0000","last_clean":"2024-04-24T06:28:45.988946+0000","last_became_active":"2024-04-24T06:28:44.738787+0000","last_became_peered":"2024-04-24T06:28:44.738787+0000","last_unstale":"2024-04-24T06:28:45.988946+0000","last_undegraded":"2024-04-24T06:28:45.988946+0000","last_fullsized":"2024-04-24T06:28:45.988946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742676+0000","last_change":"2024-04-24T06:28:44.757966+0000","last_active":"2024-04-24T06:28:45.742676+0000","last_peered":"2024-04-24T06:28:45.742676+0000","last_clean":"2024-04-24T06:28:45.742676+0000","last_became_active":"2024-04-24T06:28:44.757694+0000","last_became_peered":"2024-04-24T06:28:44.757694+0000","last_unstale":"2024-04-24T06:28:45.742676+0000","last_undegraded":"2024-04-24T06:28:45.742676+0000","last_fullsized":"2024-04-24T06:28:45.742676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989747+0000","last_change":"2024-04-24T06:28:44.747390+0000","last_active":"2024-04-24T06:28:45.989747+0000","last_peered":"2024-04-24T06:28:45.989747+0000","last_clean":"2024-04-24T06:28:45.989747+0000","last_became_active":"2024-04-24T06:28:44.746835+0000","last_became_peered":"2024-04-24T06:28:44.746835+0000","last_unstale":"2024-04-24T06:28:45.989747+0000","last_undegraded":"2024-04-24T06:28:45.989747+0000","last_fullsized":"2024-04-24T06:28:45.989747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991286+0000","last_change":"2024-04-24T06:28:44.751607+0000","last_active":"2024-04-24T06:28:45.991286+0000","last_peered":"2024-04-24T06:28:45.991286+0000","last_clean":"2024-04-24T06:28:45.991286+0000","last_became_active":"2024-04-24T06:28:44.750641+0000","last_became_peered":"2024-04-24T06:28:44.750641+0000","last_unstale":"2024-04-24T06:28:45.991286+0000","last_undegraded":"2024-04-24T06:28:45.991286+0000","last_fullsized":"2024-04-24T06:28:45.991286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742952+0000","last_change":"2024-04-24T06:28:44.760798+0000","last_active":"2024-04-24T06:28:45.742952+0000","last_peered":"2024-04-24T06:28:45.742952+0000","last_clean":"2024-04-24T06:28:45.742952+0000","last_became_active":"2024-04-24T06:28:44.760283+0000","last_became_peered":"2024-04-24T06:28:44.760283+0000","last_unstale":"2024-04-24T06:28:45.742952+0000","last_undegraded":"2024-04-24T06:28:45.742952+0000","last_fullsized":"2024-04-24T06:28:45.742952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990053+0000","last_change":"2024-04-24T06:28:44.734598+0000","last_active":"2024-04-24T06:28:45.990053+0000","last_peered":"2024-04-24T06:28:45.990053+0000","last_clean":"2024-04-24T06:28:45.990053+0000","last_became_active":"2024-04-24T06:28:44.733583+0000","last_became_peered":"2024-04-24T06:28:44.733583+0000","last_unstale":"2024-04-24T06:28:45.990053+0000","last_undegraded":"2024-04-24T06:28:45.990053+0000","last_fullsized":"2024-04-24T06:28:45.990053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:51:11.987585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992619+0000","last_change":"2024-04-24T06:28:44.747811+0000","last_active":"2024-04-24T06:28:45.992619+0000","last_peered":"2024-04-24T06:28:45.992619+0000","last_clean":"2024-04-24T06:28:45.992619+0000","last_became_active":"2024-04-24T06:28:44.747378+0000","last_became_peered":"2024-04-24T06:28:44.747378+0000","last_unstale":"2024-04-24T06:28:45.992619+0000","last_undegraded":"2024-04-24T06:28:45.992619+0000","last_fullsized":"2024-04-24T06:28:45.992619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987349+0000","last_change":"2024-04-24T06:28:44.730489+0000","last_active":"2024-04-24T06:28:45.987349+0000","last_peered":"2024-04-24T06:28:45.987349+0000","last_clean":"2024-04-24T06:28:45.987349+0000","last_became_active":"2024-04-24T06:28:44.729738+0000","last_became_peered":"2024-04-24T06:28:44.729738+0000","last_unstale":"2024-04-24T06:28:45.987349+0000","last_undegraded":"2024-04-24T06:28:45.987349+0000","last_fullsized":"2024-04-24T06:28:45.987349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990150+0000","last_change":"2024-04-24T06:28:44.746610+0000","last_active":"2024-04-24T06:28:45.990150+0000","last_peered":"2024-04-24T06:28:45.990150+0000","last_clean":"2024-04-24T06:28:45.990150+0000","last_became_active":"2024-04-24T06:28:44.746300+0000","last_became_peered":"2024-04-24T06:28:44.746300+0000","last_unstale":"2024-04-24T06:28:45.990150+0000","last_undegraded":"2024-04-24T06:28:45.990150+0000","last_fullsized":"2024-04-24T06:28:45.990150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741355+0000","last_change":"2024-04-24T06:28:44.740084+0000","last_active":"2024-04-24T06:28:45.741355+0000","last_peered":"2024-04-24T06:28:45.741355+0000","last_clean":"2024-04-24T06:28:45.741355+0000","last_became_active":"2024-04-24T06:28:44.739785+0000","last_became_peered":"2024-04-24T06:28:44.739785+0000","last_unstale":"2024-04-24T06:28:45.741355+0000","last_undegraded":"2024-04-24T06:28:45.741355+0000","last_fullsized":"2024-04-24T06:28:45.741355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741767+0000","last_change":"2024-04-24T06:28:43.741594+0000","last_active":"2024-04-24T06:28:45.741767+0000","last_peered":"2024-04-24T06:28:45.741767+0000","last_clean":"2024-04-24T06:28:45.741767+0000","last_became_active":"2024-04-24T06:28:43.740453+0000","last_became_peered":"2024-04-24T06:28:43.740453+0000","last_unstale":"2024-04-24T06:28:45.741767+0000","last_undegraded":"2024-04-24T06:28:45.741767+0000","last_fullsized":"2024-04-24T06:28:45.741767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990371+0000","last_change":"2024-04-24T06:28:44.750178+0000","last_active":"2024-04-24T06:28:45.990371+0000","last_peered":"2024-04-24T06:28:45.990371+0000","last_clean":"2024-04-24T06:28:45.990371+0000","last_became_active":"2024-04-24T06:28:44.749081+0000","last_became_peered":"2024-04-24T06:28:44.749081+0000","last_unstale":"2024-04-24T06:28:45.990371+0000","last_undegraded":"2024-04-24T06:28:45.990371+0000","last_fullsized":"2024-04-24T06:28:45.990371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989324+0000","last_change":"2024-04-24T06:28:43.745532+0000","last_active":"2024-04-24T06:28:45.989324+0000","last_peered":"2024-04-24T06:28:45.989324+0000","last_clean":"2024-04-24T06:28:45.989324+0000","last_became_active":"2024-04-24T06:28:43.745191+0000","last_became_peered":"2024-04-24T06:28:43.745191+0000","last_unstale":"2024-04-24T06:28:45.989324+0000","last_undegraded":"2024-04-24T06:28:45.989324+0000","last_fullsized":"2024-04-24T06:28:45.989324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741301+0000","last_change":"2024-04-24T06:28:43.740409+0000","last_active":"2024-04-24T06:28:45.741301+0000","last_peered":"2024-04-24T06:28:45.741301+0000","last_clean":"2024-04-24T06:28:45.741301+0000","last_became_active":"2024-04-24T06:28:43.740113+0000","last_became_peered":"2024-04-24T06:28:43.740113+0000","last_unstale":"2024-04-24T06:28:45.741301+0000","last_undegraded":"2024-04-24T06:28:45.741301+0000","last_fullsized":"2024-04-24T06:28:45.741301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741118+0000","last_change":"2024-04-24T06:28:44.734557+0000","last_active":"2024-04-24T06:28:45.741118+0000","last_peered":"2024-04-24T06:28:45.741118+0000","last_clean":"2024-04-24T06:28:45.741118+0000","last_became_active":"2024-04-24T06:28:44.733956+0000","last_became_peered":"2024-04-24T06:28:44.733956+0000","last_unstale":"2024-04-24T06:28:45.741118+0000","last_undegraded":"2024-04-24T06:28:45.741118+0000","last_fullsized":"2024-04-24T06:28:45.741118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989096+0000","last_change":"2024-04-24T06:28:43.731610+0000","last_active":"2024-04-24T06:28:45.989096+0000","last_peered":"2024-04-24T06:28:45.989096+0000","last_clean":"2024-04-24T06:28:45.989096+0000","last_became_active":"2024-04-24T06:28:43.731214+0000","last_became_peered":"2024-04-24T06:28:43.731214+0000","last_unstale":"2024-04-24T06:28:45.989096+0000","last_undegraded":"2024-04-24T06:28:45.989096+0000","last_fullsized":"2024-04-24T06:28:45.989096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988808+0000","last_change":"2024-04-24T06:28:44.733043+0000","last_active":"2024-04-24T06:28:45.988808+0000","last_peered":"2024-04-24T06:28:45.988808+0000","last_clean":"2024-04-24T06:28:45.988808+0000","last_became_active":"2024-04-24T06:28:44.732788+0000","last_became_peered":"2024-04-24T06:28:44.732788+0000","last_unstale":"2024-04-24T06:28:45.988808+0000","last_undegraded":"2024-04-24T06:28:45.988808+0000","last_fullsized":"2024-04-24T06:28:45.988808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989195+0000","last_change":"2024-04-24T06:28:43.748552+0000","last_active":"2024-04-24T06:28:45.989195+0000","last_peered":"2024-04-24T06:28:45.989195+0000","last_clean":"2024-04-24T06:28:45.989195+0000","last_became_active":"2024-04-24T06:28:43.748142+0000","last_became_peered":"2024-04-24T06:28:43.748142+0000","last_unstale":"2024-04-24T06:28:45.989195+0000","last_undegraded":"2024-04-24T06:28:45.989195+0000","last_fullsized":"2024-04-24T06:28:45.989195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992302+0000","last_change":"2024-04-24T06:28:44.745984+0000","last_active":"2024-04-24T06:28:45.992302+0000","last_peered":"2024-04-24T06:28:45.992302+0000","last_clean":"2024-04-24T06:28:45.992302+0000","last_became_active":"2024-04-24T06:28:44.745648+0000","last_became_peered":"2024-04-24T06:28:44.745648+0000","last_unstale":"2024-04-24T06:28:45.992302+0000","last_undegraded":"2024-04-24T06:28:45.992302+0000","last_fullsized":"2024-04-24T06:28:45.992302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992307+0000","last_change":"2024-04-24T06:28:43.742088+0000","last_active":"2024-04-24T06:28:45.992307+0000","last_peered":"2024-04-24T06:28:45.992307+0000","last_clean":"2024-04-24T06:28:45.992307+0000","last_became_active":"2024-04-24T06:28:43.741051+0000","last_became_peered":"2024-04-24T06:28:43.741051+0000","last_unstale":"2024-04-24T06:28:45.992307+0000","last_undegraded":"2024-04-24T06:28:45.992307+0000","last_fullsized":"2024-04-24T06:28:45.992307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990451+0000","last_change":"2024-04-24T06:28:44.737169+0000","last_active":"2024-04-24T06:28:45.990451+0000","last_peered":"2024-04-24T06:28:45.990451+0000","last_clean":"2024-04-24T06:28:45.990451+0000","last_became_active":"2024-04-24T06:28:44.736490+0000","last_became_peered":"2024-04-24T06:28:44.736490+0000","last_unstale":"2024-04-24T06:28:45.990451+0000","last_undegraded":"2024-04-24T06:28:45.990451+0000","last_fullsized":"2024-04-24T06:28:45.990451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991438+0000","last_change":"2024-04-24T06:28:43.744091+0000","last_active":"2024-04-24T06:28:45.991438+0000","last_peered":"2024-04-24T06:28:45.991438+0000","last_clean":"2024-04-24T06:28:45.991438+0000","last_became_active":"2024-04-24T06:28:43.743603+0000","last_became_peered":"2024-04-24T06:28:43.743603+0000","last_unstale":"2024-04-24T06:28:45.991438+0000","last_undegraded":"2024-04-24T06:28:45.991438+0000","last_fullsized":"2024-04-24T06:28:45.991438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741915+0000","last_change":"2024-04-24T06:28:44.752430+0000","last_active":"2024-04-24T06:28:45.741915+0000","last_peered":"2024-04-24T06:28:45.741915+0000","last_clean":"2024-04-24T06:28:45.741915+0000","last_became_active":"2024-04-24T06:28:44.752129+0000","last_became_peered":"2024-04-24T06:28:44.752129+0000","last_unstale":"2024-04-24T06:28:45.741915+0000","last_undegraded":"2024-04-24T06:28:45.741915+0000","last_fullsized":"2024-04-24T06:28:45.741915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:11:40.427236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:43.746350+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:43.745990+0000","last_became_peered":"2024-04-24T06:28:43.745990+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994027+0000","last_change":"2024-04-24T06:28:44.755036+0000","last_active":"2024-04-24T06:28:45.994027+0000","last_peered":"2024-04-24T06:28:45.994027+0000","last_clean":"2024-04-24T06:28:45.994027+0000","last_became_active":"2024-04-24T06:28:44.754677+0000","last_became_peered":"2024-04-24T06:28:44.754677+0000","last_unstale":"2024-04-24T06:28:45.994027+0000","last_undegraded":"2024-04-24T06:28:45.994027+0000","last_fullsized":"2024-04-24T06:28:45.994027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990120+0000","last_change":"2024-04-24T06:28:43.733886+0000","last_active":"2024-04-24T06:28:45.990120+0000","last_peered":"2024-04-24T06:28:45.990120+0000","last_clean":"2024-04-24T06:28:45.990120+0000","last_became_active":"2024-04-24T06:28:43.733540+0000","last_became_peered":"2024-04-24T06:28:43.733540+0000","last_unstale":"2024-04-24T06:28:45.990120+0000","last_undegraded":"2024-04-24T06:28:45.990120+0000","last_fullsized":"2024-04-24T06:28:45.990120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990757+0000","last_change":"2024-04-24T06:28:44.740692+0000","last_active":"2024-04-24T06:28:45.990757+0000","last_peered":"2024-04-24T06:28:45.990757+0000","last_clean":"2024-04-24T06:28:45.990757+0000","last_became_active":"2024-04-24T06:28:44.740305+0000","last_became_peered":"2024-04-24T06:28:44.740305+0000","last_unstale":"2024-04-24T06:28:45.990757+0000","last_undegraded":"2024-04-24T06:28:45.990757+0000","last_fullsized":"2024-04-24T06:28:45.990757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992763+0000","last_change":"2024-04-24T06:28:43.735350+0000","last_active":"2024-04-24T06:28:45.992763+0000","last_peered":"2024-04-24T06:28:45.992763+0000","last_clean":"2024-04-24T06:28:45.992763+0000","last_became_active":"2024-04-24T06:28:43.735053+0000","last_became_peered":"2024-04-24T06:28:43.735053+0000","last_unstale":"2024-04-24T06:28:45.992763+0000","last_undegraded":"2024-04-24T06:28:45.992763+0000","last_fullsized":"2024-04-24T06:28:45.992763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993602+0000","last_change":"2024-04-24T06:28:44.758074+0000","last_active":"2024-04-24T06:28:45.993602+0000","last_peered":"2024-04-24T06:28:45.993602+0000","last_clean":"2024-04-24T06:28:45.993602+0000","last_became_active":"2024-04-24T06:28:44.757701+0000","last_became_peered":"2024-04-24T06:28:44.757701+0000","last_unstale":"2024-04-24T06:28:45.993602+0000","last_undegraded":"2024-04-24T06:28:45.993602+0000","last_fullsized":"2024-04-24T06:28:45.993602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988739+0000","last_change":"2024-04-24T06:28:43.743727+0000","last_active":"2024-04-24T06:28:45.988739+0000","last_peered":"2024-04-24T06:28:45.988739+0000","last_clean":"2024-04-24T06:28:45.988739+0000","last_became_active":"2024-04-24T06:28:43.743425+0000","last_became_peered":"2024-04-24T06:28:43.743425+0000","last_unstale":"2024-04-24T06:28:45.988739+0000","last_undegraded":"2024-04-24T06:28:45.988739+0000","last_fullsized":"2024-04-24T06:28:45.988739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989586+0000","last_change":"2024-04-24T06:28:44.743620+0000","last_active":"2024-04-24T06:28:45.989586+0000","last_peered":"2024-04-24T06:28:45.989586+0000","last_clean":"2024-04-24T06:28:45.989586+0000","last_became_active":"2024-04-24T06:28:44.743033+0000","last_became_peered":"2024-04-24T06:28:44.743033+0000","last_unstale":"2024-04-24T06:28:45.989586+0000","last_undegraded":"2024-04-24T06:28:45.989586+0000","last_fullsized":"2024-04-24T06:28:45.989586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987173+0000","last_change":"2024-04-24T06:28:43.736607+0000","last_active":"2024-04-24T06:28:45.987173+0000","last_peered":"2024-04-24T06:28:45.987173+0000","last_clean":"2024-04-24T06:28:45.987173+0000","last_became_active":"2024-04-24T06:28:43.735768+0000","last_became_peered":"2024-04-24T06:28:43.735768+0000","last_unstale":"2024-04-24T06:28:45.987173+0000","last_undegraded":"2024-04-24T06:28:45.987173+0000","last_fullsized":"2024-04-24T06:28:45.987173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:44.752144+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:44.751815+0000","last_became_peered":"2024-04-24T06:28:44.751815+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741519+0000","last_change":"2024-04-24T06:28:43.738580+0000","last_active":"2024-04-24T06:28:45.741519+0000","last_peered":"2024-04-24T06:28:45.741519+0000","last_clean":"2024-04-24T06:28:45.741519+0000","last_became_active":"2024-04-24T06:28:43.737916+0000","last_became_peered":"2024-04-24T06:28:43.737916+0000","last_unstale":"2024-04-24T06:28:45.741519+0000","last_undegraded":"2024-04-24T06:28:45.741519+0000","last_fullsized":"2024-04-24T06:28:45.741519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994443+0000","last_change":"2024-04-24T06:28:44.753155+0000","last_active":"2024-04-24T06:28:45.994443+0000","last_peered":"2024-04-24T06:28:45.994443+0000","last_clean":"2024-04-24T06:28:45.994443+0000","last_became_active":"2024-04-24T06:28:44.752638+0000","last_became_peered":"2024-04-24T06:28:44.752638+0000","last_unstale":"2024-04-24T06:28:45.994443+0000","last_undegraded":"2024-04-24T06:28:45.994443+0000","last_fullsized":"2024-04-24T06:28:45.994443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987739+0000","last_change":"2024-04-24T06:28:43.738005+0000","last_active":"2024-04-24T06:28:45.987739+0000","last_peered":"2024-04-24T06:28:45.987739+0000","last_clean":"2024-04-24T06:28:45.987739+0000","last_became_active":"2024-04-24T06:28:43.737449+0000","last_became_peered":"2024-04-24T06:28:43.737449+0000","last_unstale":"2024-04-24T06:28:45.987739+0000","last_undegraded":"2024-04-24T06:28:45.987739+0000","last_fullsized":"2024-04-24T06:28:45.987739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994839+0000","last_change":"2024-04-24T06:28:44.754240+0000","last_active":"2024-04-24T06:28:45.994839+0000","last_peered":"2024-04-24T06:28:45.994839+0000","last_clean":"2024-04-24T06:28:45.994839+0000","last_became_active":"2024-04-24T06:28:44.753698+0000","last_became_peered":"2024-04-24T06:28:44.753698+0000","last_unstale":"2024-04-24T06:28:45.994839+0000","last_undegraded":"2024-04-24T06:28:45.994839+0000","last_fullsized":"2024-04-24T06:28:45.994839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:57:38.145042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991465+0000","last_change":"2024-04-24T06:28:43.744401+0000","last_active":"2024-04-24T06:28:45.991465+0000","last_peered":"2024-04-24T06:28:45.991465+0000","last_clean":"2024-04-24T06:28:45.991465+0000","last_became_active":"2024-04-24T06:28:43.744085+0000","last_became_peered":"2024-04-24T06:28:43.744085+0000","last_unstale":"2024-04-24T06:28:45.991465+0000","last_undegraded":"2024-04-24T06:28:45.991465+0000","last_fullsized":"2024-04-24T06:28:45.991465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992064+0000","last_change":"2024-04-24T06:28:44.747939+0000","last_active":"2024-04-24T06:28:45.992064+0000","last_peered":"2024-04-24T06:28:45.992064+0000","last_clean":"2024-04-24T06:28:45.992064+0000","last_became_active":"2024-04-24T06:28:44.747593+0000","last_became_peered":"2024-04-24T06:28:44.747593+0000","last_unstale":"2024-04-24T06:28:45.992064+0000","last_undegraded":"2024-04-24T06:28:45.992064+0000","last_fullsized":"2024-04-24T06:28:45.992064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987616+0000","last_change":"2024-04-24T06:28:43.736080+0000","last_active":"2024-04-24T06:28:45.987616+0000","last_peered":"2024-04-24T06:28:45.987616+0000","last_clean":"2024-04-24T06:28:45.987616+0000","last_became_active":"2024-04-24T06:28:43.735262+0000","last_became_peered":"2024-04-24T06:28:43.735262+0000","last_unstale":"2024-04-24T06:28:45.987616+0000","last_undegraded":"2024-04-24T06:28:45.987616+0000","last_fullsized":"2024-04-24T06:28:45.987616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994003+0000","last_change":"2024-04-24T06:28:44.748730+0000","last_active":"2024-04-24T06:28:45.994003+0000","last_peered":"2024-04-24T06:28:45.994003+0000","last_clean":"2024-04-24T06:28:45.994003+0000","last_became_active":"2024-04-24T06:28:44.748473+0000","last_became_peered":"2024-04-24T06:28:44.748473+0000","last_unstale":"2024-04-24T06:28:45.994003+0000","last_undegraded":"2024-04-24T06:28:45.994003+0000","last_fullsized":"2024-04-24T06:28:45.994003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989929+0000","last_change":"2024-04-24T06:28:43.732980+0000","last_active":"2024-04-24T06:28:45.989929+0000","last_peered":"2024-04-24T06:28:45.989929+0000","last_clean":"2024-04-24T06:28:45.989929+0000","last_became_active":"2024-04-24T06:28:43.732613+0000","last_became_peered":"2024-04-24T06:28:43.732613+0000","last_unstale":"2024-04-24T06:28:45.989929+0000","last_undegraded":"2024-04-24T06:28:45.989929+0000","last_fullsized":"2024-04-24T06:28:45.989929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741850+0000","last_change":"2024-04-24T06:28:44.741822+0000","last_active":"2024-04-24T06:28:45.741850+0000","last_peered":"2024-04-24T06:28:45.741850+0000","last_clean":"2024-04-24T06:28:45.741850+0000","last_became_active":"2024-04-24T06:28:44.737221+0000","last_became_peered":"2024-04-24T06:28:44.737221+0000","last_unstale":"2024-04-24T06:28:45.741850+0000","last_undegraded":"2024-04-24T06:28:45.741850+0000","last_fullsized":"2024-04-24T06:28:45.741850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992125+0000","last_change":"2024-04-24T06:28:43.742633+0000","last_active":"2024-04-24T06:28:45.992125+0000","last_peered":"2024-04-24T06:28:45.992125+0000","last_clean":"2024-04-24T06:28:45.992125+0000","last_became_active":"2024-04-24T06:28:43.742329+0000","last_became_peered":"2024-04-24T06:28:43.742329+0000","last_unstale":"2024-04-24T06:28:45.992125+0000","last_undegraded":"2024-04-24T06:28:45.992125+0000","last_fullsized":"2024-04-24T06:28:45.992125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991037+0000","last_change":"2024-04-24T06:28:44.729217+0000","last_active":"2024-04-24T06:28:45.991037+0000","last_peered":"2024-04-24T06:28:45.991037+0000","last_clean":"2024-04-24T06:28:45.991037+0000","last_became_active":"2024-04-24T06:28:44.728584+0000","last_became_peered":"2024-04-24T06:28:44.728584+0000","last_unstale":"2024-04-24T06:28:45.991037+0000","last_undegraded":"2024-04-24T06:28:45.991037+0000","last_fullsized":"2024-04-24T06:28:45.991037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987414+0000","last_change":"2024-04-24T06:28:43.737893+0000","last_active":"2024-04-24T06:28:45.987414+0000","last_peered":"2024-04-24T06:28:45.987414+0000","last_clean":"2024-04-24T06:28:45.987414+0000","last_became_active":"2024-04-24T06:28:43.737490+0000","last_became_peered":"2024-04-24T06:28:43.737490+0000","last_unstale":"2024-04-24T06:28:45.987414+0000","last_undegraded":"2024-04-24T06:28:45.987414+0000","last_fullsized":"2024-04-24T06:28:45.987414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988935+0000","last_change":"2024-04-24T06:28:44.727674+0000","last_active":"2024-04-24T06:28:45.988935+0000","last_peered":"2024-04-24T06:28:45.988935+0000","last_clean":"2024-04-24T06:28:45.988935+0000","last_became_active":"2024-04-24T06:28:44.726973+0000","last_became_peered":"2024-04-24T06:28:44.726973+0000","last_unstale":"2024-04-24T06:28:45.988935+0000","last_undegraded":"2024-04-24T06:28:45.988935+0000","last_fullsized":"2024-04-24T06:28:45.988935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992992+0000","last_change":"2024-04-24T06:28:43.734680+0000","last_active":"2024-04-24T06:28:45.992992+0000","last_peered":"2024-04-24T06:28:45.992992+0000","last_clean":"2024-04-24T06:28:45.992992+0000","last_became_active":"2024-04-24T06:28:43.734291+0000","last_became_peered":"2024-04-24T06:28:43.734291+0000","last_unstale":"2024-04-24T06:28:45.992992+0000","last_undegraded":"2024-04-24T06:28:45.992992+0000","last_fullsized":"2024-04-24T06:28:45.992992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741903+0000","last_change":"2024-04-24T06:28:44.747768+0000","last_active":"2024-04-24T06:28:45.741903+0000","last_peered":"2024-04-24T06:28:45.741903+0000","last_clean":"2024-04-24T06:28:45.741903+0000","last_became_active":"2024-04-24T06:28:44.747439+0000","last_became_peered":"2024-04-24T06:28:44.747439+0000","last_unstale":"2024-04-24T06:28:45.741903+0000","last_undegraded":"2024-04-24T06:28:45.741903+0000","last_fullsized":"2024-04-24T06:28:45.741903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:18.635305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991108+0000","last_change":"2024-04-24T06:28:43.736187+0000","last_active":"2024-04-24T06:28:45.991108+0000","last_peered":"2024-04-24T06:28:45.991108+0000","last_clean":"2024-04-24T06:28:45.991108+0000","last_became_active":"2024-04-24T06:28:43.735013+0000","last_became_peered":"2024-04-24T06:28:43.735013+0000","last_unstale":"2024-04-24T06:28:45.991108+0000","last_undegraded":"2024-04-24T06:28:45.991108+0000","last_fullsized":"2024-04-24T06:28:45.991108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991099+0000","last_change":"2024-04-24T06:28:44.748068+0000","last_active":"2024-04-24T06:28:45.991099+0000","last_peered":"2024-04-24T06:28:45.991099+0000","last_clean":"2024-04-24T06:28:45.991099+0000","last_became_active":"2024-04-24T06:28:44.747692+0000","last_became_peered":"2024-04-24T06:28:44.747692+0000","last_unstale":"2024-04-24T06:28:45.991099+0000","last_undegraded":"2024-04-24T06:28:45.991099+0000","last_fullsized":"2024-04-24T06:28:45.991099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994396+0000","last_change":"2024-04-24T06:28:43.746631+0000","last_active":"2024-04-24T06:28:45.994396+0000","last_peered":"2024-04-24T06:28:45.994396+0000","last_clean":"2024-04-24T06:28:45.994396+0000","last_became_active":"2024-04-24T06:28:43.746051+0000","last_became_peered":"2024-04-24T06:28:43.746051+0000","last_unstale":"2024-04-24T06:28:45.994396+0000","last_undegraded":"2024-04-24T06:28:45.994396+0000","last_fullsized":"2024-04-24T06:28:45.994396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:17.617739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991283+0000","last_change":"2024-04-24T06:28:44.730831+0000","last_active":"2024-04-24T06:28:45.991283+0000","last_peered":"2024-04-24T06:28:45.991283+0000","last_clean":"2024-04-24T06:28:45.991283+0000","last_became_active":"2024-04-24T06:28:44.730464+0000","last_became_peered":"2024-04-24T06:28:44.730464+0000","last_unstale":"2024-04-24T06:28:45.991283+0000","last_undegraded":"2024-04-24T06:28:45.991283+0000","last_fullsized":"2024-04-24T06:28:45.991283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991856+0000","last_change":"2024-04-24T06:28:43.746118+0000","last_active":"2024-04-24T06:28:45.991856+0000","last_peered":"2024-04-24T06:28:45.991856+0000","last_clean":"2024-04-24T06:28:45.991856+0000","last_became_active":"2024-04-24T06:28:43.745789+0000","last_became_peered":"2024-04-24T06:28:43.745789+0000","last_unstale":"2024-04-24T06:28:45.991856+0000","last_undegraded":"2024-04-24T06:28:45.991856+0000","last_fullsized":"2024-04-24T06:28:45.991856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742491+0000","last_change":"2024-04-24T06:28:44.760317+0000","last_active":"2024-04-24T06:28:45.742491+0000","last_peered":"2024-04-24T06:28:45.742491+0000","last_clean":"2024-04-24T06:28:45.742491+0000","last_became_active":"2024-04-24T06:28:44.759974+0000","last_became_peered":"2024-04-24T06:28:44.759974+0000","last_unstale":"2024-04-24T06:28:45.742491+0000","last_undegraded":"2024-04-24T06:28:45.742491+0000","last_fullsized":"2024-04-24T06:28:45.742491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989548+0000","last_change":"2024-04-24T06:28:43.751042+0000","last_active":"2024-04-24T06:28:45.989548+0000","last_peered":"2024-04-24T06:28:45.989548+0000","last_clean":"2024-04-24T06:28:45.989548+0000","last_became_active":"2024-04-24T06:28:43.750648+0000","last_became_peered":"2024-04-24T06:28:43.750648+0000","last_unstale":"2024-04-24T06:28:45.989548+0000","last_undegraded":"2024-04-24T06:28:45.989548+0000","last_fullsized":"2024-04-24T06:28:45.989548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991697+0000","last_change":"2024-04-24T06:28:44.735037+0000","last_active":"2024-04-24T06:28:45.991697+0000","last_peered":"2024-04-24T06:28:45.991697+0000","last_clean":"2024-04-24T06:28:45.991697+0000","last_became_active":"2024-04-24T06:28:44.734809+0000","last_became_peered":"2024-04-24T06:28:44.734809+0000","last_unstale":"2024-04-24T06:28:45.991697+0000","last_undegraded":"2024-04-24T06:28:45.991697+0000","last_fullsized":"2024-04-24T06:28:45.991697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991820+0000","last_change":"2024-04-24T06:28:43.744133+0000","last_active":"2024-04-24T06:28:45.991820+0000","last_peered":"2024-04-24T06:28:45.991820+0000","last_clean":"2024-04-24T06:28:45.991820+0000","last_became_active":"2024-04-24T06:28:43.743825+0000","last_became_peered":"2024-04-24T06:28:43.743825+0000","last_unstale":"2024-04-24T06:28:45.991820+0000","last_undegraded":"2024-04-24T06:28:45.991820+0000","last_fullsized":"2024-04-24T06:28:45.991820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742147+0000","last_change":"2024-04-24T06:28:44.739332+0000","last_active":"2024-04-24T06:28:45.742147+0000","last_peered":"2024-04-24T06:28:45.742147+0000","last_clean":"2024-04-24T06:28:45.742147+0000","last_became_active":"2024-04-24T06:28:44.738899+0000","last_became_peered":"2024-04-24T06:28:44.738899+0000","last_unstale":"2024-04-24T06:28:45.742147+0000","last_undegraded":"2024-04-24T06:28:45.742147+0000","last_fullsized":"2024-04-24T06:28:45.742147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989171+0000","last_change":"2024-04-24T06:28:43.742255+0000","last_active":"2024-04-24T06:28:45.989171+0000","last_peered":"2024-04-24T06:28:45.989171+0000","last_clean":"2024-04-24T06:28:45.989171+0000","last_became_active":"2024-04-24T06:28:43.741940+0000","last_became_peered":"2024-04-24T06:28:43.741940+0000","last_unstale":"2024-04-24T06:28:45.989171+0000","last_undegraded":"2024-04-24T06:28:45.989171+0000","last_fullsized":"2024-04-24T06:28:45.989171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741541+0000","last_change":"2024-04-24T06:28:44.739763+0000","last_active":"2024-04-24T06:28:45.741541+0000","last_peered":"2024-04-24T06:28:45.741541+0000","last_clean":"2024-04-24T06:28:45.741541+0000","last_became_active":"2024-04-24T06:28:44.739490+0000","last_became_peered":"2024-04-24T06:28:44.739490+0000","last_unstale":"2024-04-24T06:28:45.741541+0000","last_undegraded":"2024-04-24T06:28:45.741541+0000","last_fullsized":"2024-04-24T06:28:45.741541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990882+0000","last_change":"2024-04-24T06:28:43.735231+0000","last_active":"2024-04-24T06:28:45.990882+0000","last_peered":"2024-04-24T06:28:45.990882+0000","last_clean":"2024-04-24T06:28:45.990882+0000","last_became_active":"2024-04-24T06:28:43.734870+0000","last_became_peered":"2024-04-24T06:28:43.734870+0000","last_unstale":"2024-04-24T06:28:45.990882+0000","last_undegraded":"2024-04-24T06:28:45.990882+0000","last_fullsized":"2024-04-24T06:28:45.990882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741685+0000","last_change":"2024-04-24T06:28:44.746720+0000","last_active":"2024-04-24T06:28:45.741685+0000","last_peered":"2024-04-24T06:28:45.741685+0000","last_clean":"2024-04-24T06:28:45.741685+0000","last_became_active":"2024-04-24T06:28:44.746083+0000","last_became_peered":"2024-04-24T06:28:44.746083+0000","last_unstale":"2024-04-24T06:28:45.741685+0000","last_undegraded":"2024-04-24T06:28:45.741685+0000","last_fullsized":"2024-04-24T06:28:45.741685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993302+0000","last_change":"2024-04-24T06:28:43.748833+0000","last_active":"2024-04-24T06:28:45.993302+0000","last_peered":"2024-04-24T06:28:45.993302+0000","last_clean":"2024-04-24T06:28:45.993302+0000","last_became_active":"2024-04-24T06:28:43.748530+0000","last_became_peered":"2024-04-24T06:28:43.748530+0000","last_unstale":"2024-04-24T06:28:45.993302+0000","last_undegraded":"2024-04-24T06:28:45.993302+0000","last_fullsized":"2024-04-24T06:28:45.993302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989154+0000","last_change":"2024-04-24T06:28:44.755539+0000","last_active":"2024-04-24T06:28:45.989154+0000","last_peered":"2024-04-24T06:28:45.989154+0000","last_clean":"2024-04-24T06:28:45.989154+0000","last_became_active":"2024-04-24T06:28:44.755179+0000","last_became_peered":"2024-04-24T06:28:44.755179+0000","last_unstale":"2024-04-24T06:28:45.989154+0000","last_undegraded":"2024-04-24T06:28:45.989154+0000","last_fullsized":"2024-04-24T06:28:45.989154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991127+0000","last_change":"2024-04-24T06:28:44.738377+0000","last_active":"2024-04-24T06:28:45.991127+0000","last_peered":"2024-04-24T06:28:45.991127+0000","last_clean":"2024-04-24T06:28:45.991127+0000","last_became_active":"2024-04-24T06:28:44.738086+0000","last_became_peered":"2024-04-24T06:28:44.738086+0000","last_unstale":"2024-04-24T06:28:45.991127+0000","last_undegraded":"2024-04-24T06:28:45.991127+0000","last_fullsized":"2024-04-24T06:28:45.991127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993626+0000","last_change":"2024-04-24T06:28:43.747066+0000","last_active":"2024-04-24T06:28:45.993626+0000","last_peered":"2024-04-24T06:28:45.993626+0000","last_clean":"2024-04-24T06:28:45.993626+0000","last_became_active":"2024-04-24T06:28:43.746745+0000","last_became_peered":"2024-04-24T06:28:43.746745+0000","last_unstale":"2024-04-24T06:28:45.993626+0000","last_undegraded":"2024-04-24T06:28:45.993626+0000","last_fullsized":"2024-04-24T06:28:45.993626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991013+0000","last_change":"2024-04-24T06:28:44.754533+0000","last_active":"2024-04-24T06:28:45.991013+0000","last_peered":"2024-04-24T06:28:45.991013+0000","last_clean":"2024-04-24T06:28:45.991013+0000","last_became_active":"2024-04-24T06:28:44.754250+0000","last_became_peered":"2024-04-24T06:28:44.754250+0000","last_unstale":"2024-04-24T06:28:45.991013+0000","last_undegraded":"2024-04-24T06:28:45.991013+0000","last_fullsized":"2024-04-24T06:28:45.991013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994203+0000","last_change":"2024-04-24T06:28:43.749898+0000","last_active":"2024-04-24T06:28:45.994203+0000","last_peered":"2024-04-24T06:28:45.994203+0000","last_clean":"2024-04-24T06:28:45.994203+0000","last_became_active":"2024-04-24T06:28:43.749544+0000","last_became_peered":"2024-04-24T06:28:43.749544+0000","last_unstale":"2024-04-24T06:28:45.994203+0000","last_undegraded":"2024-04-24T06:28:45.994203+0000","last_fullsized":"2024-04-24T06:28:45.994203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991965+0000","last_change":"2024-04-24T06:28:44.736764+0000","last_active":"2024-04-24T06:28:45.991965+0000","last_peered":"2024-04-24T06:28:45.991965+0000","last_clean":"2024-04-24T06:28:45.991965+0000","last_became_active":"2024-04-24T06:28:44.736187+0000","last_became_peered":"2024-04-24T06:28:44.736187+0000","last_unstale":"2024-04-24T06:28:45.991965+0000","last_undegraded":"2024-04-24T06:28:45.991965+0000","last_fullsized":"2024-04-24T06:28:45.991965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742308+0000","last_change":"2024-04-24T06:28:43.739421+0000","last_active":"2024-04-24T06:28:45.742308+0000","last_peered":"2024-04-24T06:28:45.742308+0000","last_clean":"2024-04-24T06:28:45.742308+0000","last_became_active":"2024-04-24T06:28:43.737666+0000","last_became_peered":"2024-04-24T06:28:43.737666+0000","last_unstale":"2024-04-24T06:28:45.742308+0000","last_undegraded":"2024-04-24T06:28:45.742308+0000","last_fullsized":"2024-04-24T06:28:45.742308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742640+0000","last_change":"2024-04-24T06:28:44.741299+0000","last_active":"2024-04-24T06:28:45.742640+0000","last_peered":"2024-04-24T06:28:45.742640+0000","last_clean":"2024-04-24T06:28:45.742640+0000","last_became_active":"2024-04-24T06:28:44.740960+0000","last_became_peered":"2024-04-24T06:28:44.740960+0000","last_unstale":"2024-04-24T06:28:45.742640+0000","last_undegraded":"2024-04-24T06:28:45.742640+0000","last_fullsized":"2024-04-24T06:28:45.742640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990886+0000","last_change":"2024-04-24T06:28:43.734445+0000","last_active":"2024-04-24T06:28:45.990886+0000","last_peered":"2024-04-24T06:28:45.990886+0000","last_clean":"2024-04-24T06:28:45.990886+0000","last_became_active":"2024-04-24T06:28:43.733088+0000","last_became_peered":"2024-04-24T06:28:43.733088+0000","last_unstale":"2024-04-24T06:28:45.990886+0000","last_undegraded":"2024-04-24T06:28:45.990886+0000","last_fullsized":"2024-04-24T06:28:45.990886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:28:54.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:28:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.115+0000 7f3afae00640 1 Processor -- start 2024-04-24T06:28:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.115+0000 7f3afae00640 1 -- start start 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3afae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 0x7f3af41696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3afae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af40cea70 con 0x7f3af40b24b0 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3afae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af406bc00 con 0x7f3af415ed60 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3afae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3af406b9d0 con 0x7f3af40cbb90 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3af3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3af40b24b0 0x7f3af40b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60054/0 (socket says 172.21.15.59:60054) 2024-04-24T06:28:55.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3af3e00640 1 -- 172.21.15.59:0/3032816040 learned_addr learned my addr 172.21.15.59:0/3032816040 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3af8e00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 0x7f3af41696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675846173 0 0) 0x7f3af40cea70 con 0x7f3af40b24b0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.117+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ad8003520 con 0x7f3af40b24b0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3190942522 0 0) 0x7f3ad8003520 con 0x7f3af40b24b0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 msgr2=0x7f3af41696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 0x7f3af41696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 >> v1:172.21.15.66:6789/0 conn(0x7f3af415ed60 legacy=0x7f3af40b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af406bee0 con 0x7f3af40b24b0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f3ae4001670 con 0x7f3af40b24b0 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 >> v1:172.21.15.59:6789/0 conn(0x7f3af40b24b0 legacy=0x7f3af40b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80039e0 0x7f3ad8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 0x7f3ad8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad8004d40 0x7f3ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800baa0 con 0x7f3ad80039e0 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad800bc50 con 0x7f3ad8004d40 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad800be50 con 0x7f3af40cbb90 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af3e00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80039e0 0x7f3ad8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af3e00640 1 -- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 msgr2=0x7f3ad8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af3e00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40cbb90 0x7f3ad8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.118+0000 7f3af3e00640 1 -- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad8004d40 msgr2=0x7f3ad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.119+0000 7f3af3e00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad8004d40 0x7f3ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.119+0000 7f3af3e00640 1 -- 172.21.15.59:0/3032816040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad8003520 con 0x7f3ad80039e0 2024-04-24T06:28:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3af3e00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80039e0 0x7f3ad8003e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c160 tx=0x7f3ad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da77b51c684fbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4007c60 con 0x7f3ad80039e0 2024-04-24T06:28:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4005ad0 con 0x7f3ad80039e0 2024-04-24T06:28:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3af2a00640 1 -- 172.21.15.59:0/3032816040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4007070 con 0x7f3ad80039e0 2024-04-24T06:28:55.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3afae00640 1 -- 172.21.15.59:0/3032816040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80039e0 msgr2=0x7f3ad8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3afae00640 1 --2- 172.21.15.59:0/3032816040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80039e0 0x7f3ad8003e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c160 tx=0x7f3ad8005830 comp rx=0 tx=0).stop 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3af8e00640 1 -- 172.21.15.59:0/3032816040 reap_dead start 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.120+0000 7f3afae00640 1 -- 172.21.15.59:0/3032816040 shutdown_connections 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.121+0000 7f3afae00640 1 -- 172.21.15.59:0/3032816040 >> 172.21.15.59:0/3032816040 conn(0x7f3af4073450 msgr2=0x7f3af409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.121+0000 7f3afae00640 1 -- 172.21.15.59:0/3032816040 shutdown_connections 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.121+0000 7f3afae00640 1 -- 172.21.15.59:0/3032816040 wait complete. 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.122+0000 7f3afae00640 1 Processor -- start 2024-04-24T06:28:55.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.122+0000 7f3afae00640 1 -- start start 2024-04-24T06:28:55.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad80039e0 0x7f3af4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40b0060 0x7f3af40b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3af406c250 con 0x7f3af40cd570 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40ce720 con 0x7f3ad80039e0 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.123+0000 7f3afae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3af41676d0 con 0x7f3af40b0060 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38186/0 (socket says 172.21.15.59:38186) 2024-04-24T06:28:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 -- 172.21.15.59:0/1135659891 learned_addr learned my addr 172.21.15.59:0/1135659891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3e00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad80039e0 0x7f3af4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 -- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40b0060 msgr2=0x7f3af40b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40b0060 0x7f3af40b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 -- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad80039e0 msgr2=0x7f3af4153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad80039e0 0x7f3af4153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 -- 172.21.15.59:0/1135659891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4153990 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.124+0000 7f3af3400640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000c480 tx=0x7f3ae000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.125+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00052d0 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.125+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0005470 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.125+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0012870 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.125+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af40af580 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.125+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af40afb40 con 0x7f3af40cd570 2024-04-24T06:28:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.126+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4069190 con 0x7f3af40cd570 2024-04-24T06:28:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.127+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ae0012a10 con 0x7f3af40cd570 2024-04-24T06:28:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.128+0000 7f3aebe00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 0x7f3ad406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.128+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae008b6b0 con 0x7f3af40cd570 2024-04-24T06:28:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.128+0000 7f3af3e00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 0x7f3ad406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.129+0000 7f3af3e00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 0x7f3ad406ee30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80057e0 tx=0x7f3ae402d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.133+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae0059b50 con 0x7f3af40cd570 2024-04-24T06:28:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.329+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3af406def0 con 0x7f3ad406c970 2024-04-24T06:28:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.347+0000 7f3aebe00640 1 -- 172.21.15.59:0/1135659891 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f3af406def0 con 0x7f3ad406c970 2024-04-24T06:28:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:55.353 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.353+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 msgr2=0x7f3ad406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.353+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 0x7f3ad406ee30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80057e0 tx=0x7f3ae402d000 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.353+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 msgr2=0x7f3af40cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.353+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000c480 tx=0x7f3ae000c940 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.353+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 shutdown_connections 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3ad406c970 0x7f3ad406ee30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3af40b0060 0x7f3af40b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3af40cd570 0x7f3af40cc2e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 --2- 172.21.15.59:0/1135659891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ad80039e0 0x7f3af4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 >> 172.21.15.59:0/1135659891 conn(0x7f3af4073450 msgr2=0x7f3af406f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 shutdown_connections 2024-04-24T06:28:55.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.354+0000 7f3afae00640 1 -- 172.21.15.59:0/1135659891 wait complete. 2024-04-24T06:28:55.373 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-24T06:28:55.190840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003247"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994473+0000","last_change":"2024-04-24T06:28:44.737375+0000","last_active":"2024-04-24T06:28:45.994473+0000","last_peered":"2024-04-24T06:28:45.994473+0000","last_clean":"2024-04-24T06:28:45.994473+0000","last_became_active":"2024-04-24T06:28:44.737064+0000","last_became_peered":"2024-04-24T06:28:44.737064+0000","last_unstale":"2024-04-24T06:28:45.994473+0000","last_undegraded":"2024-04-24T06:28:45.994473+0000","last_fullsized":"2024-04-24T06:28:45.994473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991754+0000","last_change":"2024-04-24T06:28:44.748671+0000","last_active":"2024-04-24T06:28:45.991754+0000","last_peered":"2024-04-24T06:28:45.991754+0000","last_clean":"2024-04-24T06:28:45.991754+0000","last_became_active":"2024-04-24T06:28:44.748393+0000","last_became_peered":"2024-04-24T06:28:44.748393+0000","last_unstale":"2024-04-24T06:28:45.991754+0000","last_undegraded":"2024-04-24T06:28:45.991754+0000","last_fullsized":"2024-04-24T06:28:45.991754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990411+0000","last_change":"2024-04-24T06:28:44.738384+0000","last_active":"2024-04-24T06:28:45.990411+0000","last_peered":"2024-04-24T06:28:45.990411+0000","last_clean":"2024-04-24T06:28:45.990411+0000","last_became_active":"2024-04-24T06:28:44.738063+0000","last_became_peered":"2024-04-24T06:28:44.738063+0000","last_unstale":"2024-04-24T06:28:45.990411+0000","last_undegraded":"2024-04-24T06:28:45.990411+0000","last_fullsized":"2024-04-24T06:28:45.990411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991159+0000","last_change":"2024-04-24T06:28:44.754483+0000","last_active":"2024-04-24T06:28:45.991159+0000","last_peered":"2024-04-24T06:28:45.991159+0000","last_clean":"2024-04-24T06:28:45.991159+0000","last_became_active":"2024-04-24T06:28:44.754150+0000","last_became_peered":"2024-04-24T06:28:44.754150+0000","last_unstale":"2024-04-24T06:28:45.991159+0000","last_undegraded":"2024-04-24T06:28:45.991159+0000","last_fullsized":"2024-04-24T06:28:45.991159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993216+0000","last_change":"2024-04-24T06:28:44.741673+0000","last_active":"2024-04-24T06:28:45.993216+0000","last_peered":"2024-04-24T06:28:45.993216+0000","last_clean":"2024-04-24T06:28:45.993216+0000","last_became_active":"2024-04-24T06:28:44.741361+0000","last_became_peered":"2024-04-24T06:28:44.741361+0000","last_unstale":"2024-04-24T06:28:45.993216+0000","last_undegraded":"2024-04-24T06:28:45.993216+0000","last_fullsized":"2024-04-24T06:28:45.993216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990171+0000","last_change":"2024-04-24T06:28:44.745828+0000","last_active":"2024-04-24T06:28:45.990171+0000","last_peered":"2024-04-24T06:28:45.990171+0000","last_clean":"2024-04-24T06:28:45.990171+0000","last_became_active":"2024-04-24T06:28:44.745499+0000","last_became_peered":"2024-04-24T06:28:44.745499+0000","last_unstale":"2024-04-24T06:28:45.990171+0000","last_undegraded":"2024-04-24T06:28:45.990171+0000","last_fullsized":"2024-04-24T06:28:45.990171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992379+0000","last_change":"2024-04-24T06:28:44.752057+0000","last_active":"2024-04-24T06:28:45.992379+0000","last_peered":"2024-04-24T06:28:45.992379+0000","last_clean":"2024-04-24T06:28:45.992379+0000","last_became_active":"2024-04-24T06:28:44.746797+0000","last_became_peered":"2024-04-24T06:28:44.746797+0000","last_unstale":"2024-04-24T06:28:45.992379+0000","last_undegraded":"2024-04-24T06:28:45.992379+0000","last_fullsized":"2024-04-24T06:28:45.992379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741286+0000","last_change":"2024-04-24T06:28:44.734221+0000","last_active":"2024-04-24T06:28:45.741286+0000","last_peered":"2024-04-24T06:28:45.741286+0000","last_clean":"2024-04-24T06:28:45.741286+0000","last_became_active":"2024-04-24T06:28:44.733624+0000","last_became_peered":"2024-04-24T06:28:44.733624+0000","last_unstale":"2024-04-24T06:28:45.741286+0000","last_undegraded":"2024-04-24T06:28:45.741286+0000","last_fullsized":"2024-04-24T06:28:45.741286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994911+0000","last_change":"2024-04-24T06:28:44.744186+0000","last_active":"2024-04-24T06:28:45.994911+0000","last_peered":"2024-04-24T06:28:45.994911+0000","last_clean":"2024-04-24T06:28:45.994911+0000","last_became_active":"2024-04-24T06:28:44.743766+0000","last_became_peered":"2024-04-24T06:28:44.743766+0000","last_unstale":"2024-04-24T06:28:45.994911+0000","last_undegraded":"2024-04-24T06:28:45.994911+0000","last_fullsized":"2024-04-24T06:28:45.994911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:00.455831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989452+0000","last_change":"2024-04-24T06:28:44.742684+0000","last_active":"2024-04-24T06:28:45.989452+0000","last_peered":"2024-04-24T06:28:45.989452+0000","last_clean":"2024-04-24T06:28:45.989452+0000","last_became_active":"2024-04-24T06:28:44.742251+0000","last_became_peered":"2024-04-24T06:28:44.742251+0000","last_unstale":"2024-04-24T06:28:45.989452+0000","last_undegraded":"2024-04-24T06:28:45.989452+0000","last_fullsized":"2024-04-24T06:28:45.989452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989977+0000","last_change":"2024-04-24T06:28:44.741312+0000","last_active":"2024-04-24T06:28:45.989977+0000","last_peered":"2024-04-24T06:28:45.989977+0000","last_clean":"2024-04-24T06:28:45.989977+0000","last_became_active":"2024-04-24T06:28:44.740904+0000","last_became_peered":"2024-04-24T06:28:44.740904+0000","last_unstale":"2024-04-24T06:28:45.989977+0000","last_undegraded":"2024-04-24T06:28:45.989977+0000","last_fullsized":"2024-04-24T06:28:45.989977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741756+0000","last_change":"2024-04-24T06:28:44.748944+0000","last_active":"2024-04-24T06:28:45.741756+0000","last_peered":"2024-04-24T06:28:45.741756+0000","last_clean":"2024-04-24T06:28:45.741756+0000","last_became_active":"2024-04-24T06:28:44.748456+0000","last_became_peered":"2024-04-24T06:28:44.748456+0000","last_unstale":"2024-04-24T06:28:45.741756+0000","last_undegraded":"2024-04-24T06:28:45.741756+0000","last_fullsized":"2024-04-24T06:28:45.741756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741786+0000","last_change":"2024-04-24T06:28:44.742242+0000","last_active":"2024-04-24T06:28:45.741786+0000","last_peered":"2024-04-24T06:28:45.741786+0000","last_clean":"2024-04-24T06:28:45.741786+0000","last_became_active":"2024-04-24T06:28:44.741323+0000","last_became_peered":"2024-04-24T06:28:44.741323+0000","last_unstale":"2024-04-24T06:28:45.741786+0000","last_undegraded":"2024-04-24T06:28:45.741786+0000","last_fullsized":"2024-04-24T06:28:45.741786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990162+0000","last_change":"2024-04-24T06:28:44.733381+0000","last_active":"2024-04-24T06:28:45.990162+0000","last_peered":"2024-04-24T06:28:45.990162+0000","last_clean":"2024-04-24T06:28:45.990162+0000","last_became_active":"2024-04-24T06:28:44.733086+0000","last_became_peered":"2024-04-24T06:28:44.733086+0000","last_unstale":"2024-04-24T06:28:45.990162+0000","last_undegraded":"2024-04-24T06:28:45.990162+0000","last_fullsized":"2024-04-24T06:28:45.990162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:46.231913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989857+0000","last_change":"2024-04-24T06:28:44.743567+0000","last_active":"2024-04-24T06:28:45.989857+0000","last_peered":"2024-04-24T06:28:45.989857+0000","last_clean":"2024-04-24T06:28:45.989857+0000","last_became_active":"2024-04-24T06:28:44.743281+0000","last_became_peered":"2024-04-24T06:28:44.743281+0000","last_unstale":"2024-04-24T06:28:45.989857+0000","last_undegraded":"2024-04-24T06:28:45.989857+0000","last_fullsized":"2024-04-24T06:28:45.989857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990473+0000","last_change":"2024-04-24T06:28:44.734065+0000","last_active":"2024-04-24T06:28:45.990473+0000","last_peered":"2024-04-24T06:28:45.990473+0000","last_clean":"2024-04-24T06:28:45.990473+0000","last_became_active":"2024-04-24T06:28:44.733730+0000","last_became_peered":"2024-04-24T06:28:44.733730+0000","last_unstale":"2024-04-24T06:28:45.990473+0000","last_undegraded":"2024-04-24T06:28:45.990473+0000","last_fullsized":"2024-04-24T06:28:45.990473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992895+0000","last_change":"2024-04-24T06:28:44.749923+0000","last_active":"2024-04-24T06:28:45.992895+0000","last_peered":"2024-04-24T06:28:45.992895+0000","last_clean":"2024-04-24T06:28:45.992895+0000","last_became_active":"2024-04-24T06:28:44.749576+0000","last_became_peered":"2024-04-24T06:28:44.749576+0000","last_unstale":"2024-04-24T06:28:45.992895+0000","last_undegraded":"2024-04-24T06:28:45.992895+0000","last_fullsized":"2024-04-24T06:28:45.992895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:10.470031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987678+0000","last_change":"2024-04-24T06:28:44.737360+0000","last_active":"2024-04-24T06:28:45.987678+0000","last_peered":"2024-04-24T06:28:45.987678+0000","last_clean":"2024-04-24T06:28:45.987678+0000","last_became_active":"2024-04-24T06:28:44.737001+0000","last_became_peered":"2024-04-24T06:28:44.737001+0000","last_unstale":"2024-04-24T06:28:45.987678+0000","last_undegraded":"2024-04-24T06:28:45.987678+0000","last_fullsized":"2024-04-24T06:28:45.987678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:49.955424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994589+0000","last_change":"2024-04-24T06:28:44.748893+0000","last_active":"2024-04-24T06:28:45.994589+0000","last_peered":"2024-04-24T06:28:45.994589+0000","last_clean":"2024-04-24T06:28:45.994589+0000","last_became_active":"2024-04-24T06:28:44.748554+0000","last_became_peered":"2024-04-24T06:28:44.748554+0000","last_unstale":"2024-04-24T06:28:45.994589+0000","last_undegraded":"2024-04-24T06:28:45.994589+0000","last_fullsized":"2024-04-24T06:28:45.994589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989639+0000","last_change":"2024-04-24T06:28:44.759322+0000","last_active":"2024-04-24T06:28:45.989639+0000","last_peered":"2024-04-24T06:28:45.989639+0000","last_clean":"2024-04-24T06:28:45.989639+0000","last_became_active":"2024-04-24T06:28:44.758153+0000","last_became_peered":"2024-04-24T06:28:44.758153+0000","last_unstale":"2024-04-24T06:28:45.989639+0000","last_undegraded":"2024-04-24T06:28:45.989639+0000","last_fullsized":"2024-04-24T06:28:45.989639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:44.745627+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:44.745334+0000","last_became_peered":"2024-04-24T06:28:44.745334+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993161+0000","last_change":"2024-04-24T06:28:44.742546+0000","last_active":"2024-04-24T06:28:45.993161+0000","last_peered":"2024-04-24T06:28:45.993161+0000","last_clean":"2024-04-24T06:28:45.993161+0000","last_became_active":"2024-04-24T06:28:44.740862+0000","last_became_peered":"2024-04-24T06:28:44.740862+0000","last_unstale":"2024-04-24T06:28:45.993161+0000","last_undegraded":"2024-04-24T06:28:45.993161+0000","last_fullsized":"2024-04-24T06:28:45.993161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:55.063222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741404+0000","last_change":"2024-04-24T06:28:44.748223+0000","last_active":"2024-04-24T06:28:45.741404+0000","last_peered":"2024-04-24T06:28:45.741404+0000","last_clean":"2024-04-24T06:28:45.741404+0000","last_became_active":"2024-04-24T06:28:44.745248+0000","last_became_peered":"2024-04-24T06:28:44.745248+0000","last_unstale":"2024-04-24T06:28:45.741404+0000","last_undegraded":"2024-04-24T06:28:45.741404+0000","last_fullsized":"2024-04-24T06:28:45.741404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988946+0000","last_change":"2024-04-24T06:28:44.739176+0000","last_active":"2024-04-24T06:28:45.988946+0000","last_peered":"2024-04-24T06:28:45.988946+0000","last_clean":"2024-04-24T06:28:45.988946+0000","last_became_active":"2024-04-24T06:28:44.738787+0000","last_became_peered":"2024-04-24T06:28:44.738787+0000","last_unstale":"2024-04-24T06:28:45.988946+0000","last_undegraded":"2024-04-24T06:28:45.988946+0000","last_fullsized":"2024-04-24T06:28:45.988946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742676+0000","last_change":"2024-04-24T06:28:44.757966+0000","last_active":"2024-04-24T06:28:45.742676+0000","last_peered":"2024-04-24T06:28:45.742676+0000","last_clean":"2024-04-24T06:28:45.742676+0000","last_became_active":"2024-04-24T06:28:44.757694+0000","last_became_peered":"2024-04-24T06:28:44.757694+0000","last_unstale":"2024-04-24T06:28:45.742676+0000","last_undegraded":"2024-04-24T06:28:45.742676+0000","last_fullsized":"2024-04-24T06:28:45.742676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989747+0000","last_change":"2024-04-24T06:28:44.747390+0000","last_active":"2024-04-24T06:28:45.989747+0000","last_peered":"2024-04-24T06:28:45.989747+0000","last_clean":"2024-04-24T06:28:45.989747+0000","last_became_active":"2024-04-24T06:28:44.746835+0000","last_became_peered":"2024-04-24T06:28:44.746835+0000","last_unstale":"2024-04-24T06:28:45.989747+0000","last_undegraded":"2024-04-24T06:28:45.989747+0000","last_fullsized":"2024-04-24T06:28:45.989747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991286+0000","last_change":"2024-04-24T06:28:44.751607+0000","last_active":"2024-04-24T06:28:45.991286+0000","last_peered":"2024-04-24T06:28:45.991286+0000","last_clean":"2024-04-24T06:28:45.991286+0000","last_became_active":"2024-04-24T06:28:44.750641+0000","last_became_peered":"2024-04-24T06:28:44.750641+0000","last_unstale":"2024-04-24T06:28:45.991286+0000","last_undegraded":"2024-04-24T06:28:45.991286+0000","last_fullsized":"2024-04-24T06:28:45.991286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742952+0000","last_change":"2024-04-24T06:28:44.760798+0000","last_active":"2024-04-24T06:28:45.742952+0000","last_peered":"2024-04-24T06:28:45.742952+0000","last_clean":"2024-04-24T06:28:45.742952+0000","last_became_active":"2024-04-24T06:28:44.760283+0000","last_became_peered":"2024-04-24T06:28:44.760283+0000","last_unstale":"2024-04-24T06:28:45.742952+0000","last_undegraded":"2024-04-24T06:28:45.742952+0000","last_fullsized":"2024-04-24T06:28:45.742952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990053+0000","last_change":"2024-04-24T06:28:44.734598+0000","last_active":"2024-04-24T06:28:45.990053+0000","last_peered":"2024-04-24T06:28:45.990053+0000","last_clean":"2024-04-24T06:28:45.990053+0000","last_became_active":"2024-04-24T06:28:44.733583+0000","last_became_peered":"2024-04-24T06:28:44.733583+0000","last_unstale":"2024-04-24T06:28:45.990053+0000","last_undegraded":"2024-04-24T06:28:45.990053+0000","last_fullsized":"2024-04-24T06:28:45.990053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:51:11.987585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992619+0000","last_change":"2024-04-24T06:28:44.747811+0000","last_active":"2024-04-24T06:28:45.992619+0000","last_peered":"2024-04-24T06:28:45.992619+0000","last_clean":"2024-04-24T06:28:45.992619+0000","last_became_active":"2024-04-24T06:28:44.747378+0000","last_became_peered":"2024-04-24T06:28:44.747378+0000","last_unstale":"2024-04-24T06:28:45.992619+0000","last_undegraded":"2024-04-24T06:28:45.992619+0000","last_fullsized":"2024-04-24T06:28:45.992619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987349+0000","last_change":"2024-04-24T06:28:44.730489+0000","last_active":"2024-04-24T06:28:45.987349+0000","last_peered":"2024-04-24T06:28:45.987349+0000","last_clean":"2024-04-24T06:28:45.987349+0000","last_became_active":"2024-04-24T06:28:44.729738+0000","last_became_peered":"2024-04-24T06:28:44.729738+0000","last_unstale":"2024-04-24T06:28:45.987349+0000","last_undegraded":"2024-04-24T06:28:45.987349+0000","last_fullsized":"2024-04-24T06:28:45.987349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990150+0000","last_change":"2024-04-24T06:28:44.746610+0000","last_active":"2024-04-24T06:28:45.990150+0000","last_peered":"2024-04-24T06:28:45.990150+0000","last_clean":"2024-04-24T06:28:45.990150+0000","last_became_active":"2024-04-24T06:28:44.746300+0000","last_became_peered":"2024-04-24T06:28:44.746300+0000","last_unstale":"2024-04-24T06:28:45.990150+0000","last_undegraded":"2024-04-24T06:28:45.990150+0000","last_fullsized":"2024-04-24T06:28:45.990150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741355+0000","last_change":"2024-04-24T06:28:44.740084+0000","last_active":"2024-04-24T06:28:45.741355+0000","last_peered":"2024-04-24T06:28:45.741355+0000","last_clean":"2024-04-24T06:28:45.741355+0000","last_became_active":"2024-04-24T06:28:44.739785+0000","last_became_peered":"2024-04-24T06:28:44.739785+0000","last_unstale":"2024-04-24T06:28:45.741355+0000","last_undegraded":"2024-04-24T06:28:45.741355+0000","last_fullsized":"2024-04-24T06:28:45.741355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:43.821449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741767+0000","last_change":"2024-04-24T06:28:43.741594+0000","last_active":"2024-04-24T06:28:45.741767+0000","last_peered":"2024-04-24T06:28:45.741767+0000","last_clean":"2024-04-24T06:28:45.741767+0000","last_became_active":"2024-04-24T06:28:43.740453+0000","last_became_peered":"2024-04-24T06:28:43.740453+0000","last_unstale":"2024-04-24T06:28:45.741767+0000","last_undegraded":"2024-04-24T06:28:45.741767+0000","last_fullsized":"2024-04-24T06:28:45.741767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990371+0000","last_change":"2024-04-24T06:28:44.750178+0000","last_active":"2024-04-24T06:28:45.990371+0000","last_peered":"2024-04-24T06:28:45.990371+0000","last_clean":"2024-04-24T06:28:45.990371+0000","last_became_active":"2024-04-24T06:28:44.749081+0000","last_became_peered":"2024-04-24T06:28:44.749081+0000","last_unstale":"2024-04-24T06:28:45.990371+0000","last_undegraded":"2024-04-24T06:28:45.990371+0000","last_fullsized":"2024-04-24T06:28:45.990371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989324+0000","last_change":"2024-04-24T06:28:43.745532+0000","last_active":"2024-04-24T06:28:45.989324+0000","last_peered":"2024-04-24T06:28:45.989324+0000","last_clean":"2024-04-24T06:28:45.989324+0000","last_became_active":"2024-04-24T06:28:43.745191+0000","last_became_peered":"2024-04-24T06:28:43.745191+0000","last_unstale":"2024-04-24T06:28:45.989324+0000","last_undegraded":"2024-04-24T06:28:45.989324+0000","last_fullsized":"2024-04-24T06:28:45.989324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741301+0000","last_change":"2024-04-24T06:28:43.740409+0000","last_active":"2024-04-24T06:28:45.741301+0000","last_peered":"2024-04-24T06:28:45.741301+0000","last_clean":"2024-04-24T06:28:45.741301+0000","last_became_active":"2024-04-24T06:28:43.740113+0000","last_became_peered":"2024-04-24T06:28:43.740113+0000","last_unstale":"2024-04-24T06:28:45.741301+0000","last_undegraded":"2024-04-24T06:28:45.741301+0000","last_fullsized":"2024-04-24T06:28:45.741301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741118+0000","last_change":"2024-04-24T06:28:44.734557+0000","last_active":"2024-04-24T06:28:45.741118+0000","last_peered":"2024-04-24T06:28:45.741118+0000","last_clean":"2024-04-24T06:28:45.741118+0000","last_became_active":"2024-04-24T06:28:44.733956+0000","last_became_peered":"2024-04-24T06:28:44.733956+0000","last_unstale":"2024-04-24T06:28:45.741118+0000","last_undegraded":"2024-04-24T06:28:45.741118+0000","last_fullsized":"2024-04-24T06:28:45.741118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:26.765022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989096+0000","last_change":"2024-04-24T06:28:43.731610+0000","last_active":"2024-04-24T06:28:45.989096+0000","last_peered":"2024-04-24T06:28:45.989096+0000","last_clean":"2024-04-24T06:28:45.989096+0000","last_became_active":"2024-04-24T06:28:43.731214+0000","last_became_peered":"2024-04-24T06:28:43.731214+0000","last_unstale":"2024-04-24T06:28:45.989096+0000","last_undegraded":"2024-04-24T06:28:45.989096+0000","last_fullsized":"2024-04-24T06:28:45.989096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988808+0000","last_change":"2024-04-24T06:28:44.733043+0000","last_active":"2024-04-24T06:28:45.988808+0000","last_peered":"2024-04-24T06:28:45.988808+0000","last_clean":"2024-04-24T06:28:45.988808+0000","last_became_active":"2024-04-24T06:28:44.732788+0000","last_became_peered":"2024-04-24T06:28:44.732788+0000","last_unstale":"2024-04-24T06:28:45.988808+0000","last_undegraded":"2024-04-24T06:28:45.988808+0000","last_fullsized":"2024-04-24T06:28:45.988808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:29.484715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989195+0000","last_change":"2024-04-24T06:28:43.748552+0000","last_active":"2024-04-24T06:28:45.989195+0000","last_peered":"2024-04-24T06:28:45.989195+0000","last_clean":"2024-04-24T06:28:45.989195+0000","last_became_active":"2024-04-24T06:28:43.748142+0000","last_became_peered":"2024-04-24T06:28:43.748142+0000","last_unstale":"2024-04-24T06:28:45.989195+0000","last_undegraded":"2024-04-24T06:28:45.989195+0000","last_fullsized":"2024-04-24T06:28:45.989195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992302+0000","last_change":"2024-04-24T06:28:44.745984+0000","last_active":"2024-04-24T06:28:45.992302+0000","last_peered":"2024-04-24T06:28:45.992302+0000","last_clean":"2024-04-24T06:28:45.992302+0000","last_became_active":"2024-04-24T06:28:44.745648+0000","last_became_peered":"2024-04-24T06:28:44.745648+0000","last_unstale":"2024-04-24T06:28:45.992302+0000","last_undegraded":"2024-04-24T06:28:45.992302+0000","last_fullsized":"2024-04-24T06:28:45.992302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992307+0000","last_change":"2024-04-24T06:28:43.742088+0000","last_active":"2024-04-24T06:28:45.992307+0000","last_peered":"2024-04-24T06:28:45.992307+0000","last_clean":"2024-04-24T06:28:45.992307+0000","last_became_active":"2024-04-24T06:28:43.741051+0000","last_became_peered":"2024-04-24T06:28:43.741051+0000","last_unstale":"2024-04-24T06:28:45.992307+0000","last_undegraded":"2024-04-24T06:28:45.992307+0000","last_fullsized":"2024-04-24T06:28:45.992307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990451+0000","last_change":"2024-04-24T06:28:44.737169+0000","last_active":"2024-04-24T06:28:45.990451+0000","last_peered":"2024-04-24T06:28:45.990451+0000","last_clean":"2024-04-24T06:28:45.990451+0000","last_became_active":"2024-04-24T06:28:44.736490+0000","last_became_peered":"2024-04-24T06:28:44.736490+0000","last_unstale":"2024-04-24T06:28:45.990451+0000","last_undegraded":"2024-04-24T06:28:45.990451+0000","last_fullsized":"2024-04-24T06:28:45.990451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991438+0000","last_change":"2024-04-24T06:28:43.744091+0000","last_active":"2024-04-24T06:28:45.991438+0000","last_peered":"2024-04-24T06:28:45.991438+0000","last_clean":"2024-04-24T06:28:45.991438+0000","last_became_active":"2024-04-24T06:28:43.743603+0000","last_became_peered":"2024-04-24T06:28:43.743603+0000","last_unstale":"2024-04-24T06:28:45.991438+0000","last_undegraded":"2024-04-24T06:28:45.991438+0000","last_fullsized":"2024-04-24T06:28:45.991438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741915+0000","last_change":"2024-04-24T06:28:44.752430+0000","last_active":"2024-04-24T06:28:45.741915+0000","last_peered":"2024-04-24T06:28:45.741915+0000","last_clean":"2024-04-24T06:28:45.741915+0000","last_became_active":"2024-04-24T06:28:44.752129+0000","last_became_peered":"2024-04-24T06:28:44.752129+0000","last_unstale":"2024-04-24T06:28:45.741915+0000","last_undegraded":"2024-04-24T06:28:45.741915+0000","last_fullsized":"2024-04-24T06:28:45.741915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:11:40.427236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989718+0000","last_change":"2024-04-24T06:28:43.746350+0000","last_active":"2024-04-24T06:28:45.989718+0000","last_peered":"2024-04-24T06:28:45.989718+0000","last_clean":"2024-04-24T06:28:45.989718+0000","last_became_active":"2024-04-24T06:28:43.745990+0000","last_became_peered":"2024-04-24T06:28:43.745990+0000","last_unstale":"2024-04-24T06:28:45.989718+0000","last_undegraded":"2024-04-24T06:28:45.989718+0000","last_fullsized":"2024-04-24T06:28:45.989718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994027+0000","last_change":"2024-04-24T06:28:44.755036+0000","last_active":"2024-04-24T06:28:45.994027+0000","last_peered":"2024-04-24T06:28:45.994027+0000","last_clean":"2024-04-24T06:28:45.994027+0000","last_became_active":"2024-04-24T06:28:44.754677+0000","last_became_peered":"2024-04-24T06:28:44.754677+0000","last_unstale":"2024-04-24T06:28:45.994027+0000","last_undegraded":"2024-04-24T06:28:45.994027+0000","last_fullsized":"2024-04-24T06:28:45.994027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:26.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990120+0000","last_change":"2024-04-24T06:28:43.733886+0000","last_active":"2024-04-24T06:28:45.990120+0000","last_peered":"2024-04-24T06:28:45.990120+0000","last_clean":"2024-04-24T06:28:45.990120+0000","last_became_active":"2024-04-24T06:28:43.733540+0000","last_became_peered":"2024-04-24T06:28:43.733540+0000","last_unstale":"2024-04-24T06:28:45.990120+0000","last_undegraded":"2024-04-24T06:28:45.990120+0000","last_fullsized":"2024-04-24T06:28:45.990120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990757+0000","last_change":"2024-04-24T06:28:44.740692+0000","last_active":"2024-04-24T06:28:45.990757+0000","last_peered":"2024-04-24T06:28:45.990757+0000","last_clean":"2024-04-24T06:28:45.990757+0000","last_became_active":"2024-04-24T06:28:44.740305+0000","last_became_peered":"2024-04-24T06:28:44.740305+0000","last_unstale":"2024-04-24T06:28:45.990757+0000","last_undegraded":"2024-04-24T06:28:45.990757+0000","last_fullsized":"2024-04-24T06:28:45.990757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992763+0000","last_change":"2024-04-24T06:28:43.735350+0000","last_active":"2024-04-24T06:28:45.992763+0000","last_peered":"2024-04-24T06:28:45.992763+0000","last_clean":"2024-04-24T06:28:45.992763+0000","last_became_active":"2024-04-24T06:28:43.735053+0000","last_became_peered":"2024-04-24T06:28:43.735053+0000","last_unstale":"2024-04-24T06:28:45.992763+0000","last_undegraded":"2024-04-24T06:28:45.992763+0000","last_fullsized":"2024-04-24T06:28:45.992763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993602+0000","last_change":"2024-04-24T06:28:44.758074+0000","last_active":"2024-04-24T06:28:45.993602+0000","last_peered":"2024-04-24T06:28:45.993602+0000","last_clean":"2024-04-24T06:28:45.993602+0000","last_became_active":"2024-04-24T06:28:44.757701+0000","last_became_peered":"2024-04-24T06:28:44.757701+0000","last_unstale":"2024-04-24T06:28:45.993602+0000","last_undegraded":"2024-04-24T06:28:45.993602+0000","last_fullsized":"2024-04-24T06:28:45.993602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:15.829745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988739+0000","last_change":"2024-04-24T06:28:43.743727+0000","last_active":"2024-04-24T06:28:45.988739+0000","last_peered":"2024-04-24T06:28:45.988739+0000","last_clean":"2024-04-24T06:28:45.988739+0000","last_became_active":"2024-04-24T06:28:43.743425+0000","last_became_peered":"2024-04-24T06:28:43.743425+0000","last_unstale":"2024-04-24T06:28:45.988739+0000","last_undegraded":"2024-04-24T06:28:45.988739+0000","last_fullsized":"2024-04-24T06:28:45.988739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989586+0000","last_change":"2024-04-24T06:28:44.743620+0000","last_active":"2024-04-24T06:28:45.989586+0000","last_peered":"2024-04-24T06:28:45.989586+0000","last_clean":"2024-04-24T06:28:45.989586+0000","last_became_active":"2024-04-24T06:28:44.743033+0000","last_became_peered":"2024-04-24T06:28:44.743033+0000","last_unstale":"2024-04-24T06:28:45.989586+0000","last_undegraded":"2024-04-24T06:28:45.989586+0000","last_fullsized":"2024-04-24T06:28:45.989586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987173+0000","last_change":"2024-04-24T06:28:43.736607+0000","last_active":"2024-04-24T06:28:45.987173+0000","last_peered":"2024-04-24T06:28:45.987173+0000","last_clean":"2024-04-24T06:28:45.987173+0000","last_became_active":"2024-04-24T06:28:43.735768+0000","last_became_peered":"2024-04-24T06:28:43.735768+0000","last_unstale":"2024-04-24T06:28:45.987173+0000","last_undegraded":"2024-04-24T06:28:45.987173+0000","last_fullsized":"2024-04-24T06:28:45.987173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:36.170123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:44.752144+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:44.751815+0000","last_became_peered":"2024-04-24T06:28:44.751815+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741519+0000","last_change":"2024-04-24T06:28:43.738580+0000","last_active":"2024-04-24T06:28:45.741519+0000","last_peered":"2024-04-24T06:28:45.741519+0000","last_clean":"2024-04-24T06:28:45.741519+0000","last_became_active":"2024-04-24T06:28:43.737916+0000","last_became_peered":"2024-04-24T06:28:43.737916+0000","last_unstale":"2024-04-24T06:28:45.741519+0000","last_undegraded":"2024-04-24T06:28:45.741519+0000","last_fullsized":"2024-04-24T06:28:45.741519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994443+0000","last_change":"2024-04-24T06:28:44.753155+0000","last_active":"2024-04-24T06:28:45.994443+0000","last_peered":"2024-04-24T06:28:45.994443+0000","last_clean":"2024-04-24T06:28:45.994443+0000","last_became_active":"2024-04-24T06:28:44.752638+0000","last_became_peered":"2024-04-24T06:28:44.752638+0000","last_unstale":"2024-04-24T06:28:45.994443+0000","last_undegraded":"2024-04-24T06:28:45.994443+0000","last_fullsized":"2024-04-24T06:28:45.994443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:40:44.642204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987739+0000","last_change":"2024-04-24T06:28:43.738005+0000","last_active":"2024-04-24T06:28:45.987739+0000","last_peered":"2024-04-24T06:28:45.987739+0000","last_clean":"2024-04-24T06:28:45.987739+0000","last_became_active":"2024-04-24T06:28:43.737449+0000","last_became_peered":"2024-04-24T06:28:43.737449+0000","last_unstale":"2024-04-24T06:28:45.987739+0000","last_undegraded":"2024-04-24T06:28:45.987739+0000","last_fullsized":"2024-04-24T06:28:45.987739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994839+0000","last_change":"2024-04-24T06:28:44.754240+0000","last_active":"2024-04-24T06:28:45.994839+0000","last_peered":"2024-04-24T06:28:45.994839+0000","last_clean":"2024-04-24T06:28:45.994839+0000","last_became_active":"2024-04-24T06:28:44.753698+0000","last_became_peered":"2024-04-24T06:28:44.753698+0000","last_unstale":"2024-04-24T06:28:45.994839+0000","last_undegraded":"2024-04-24T06:28:45.994839+0000","last_fullsized":"2024-04-24T06:28:45.994839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:57:38.145042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991465+0000","last_change":"2024-04-24T06:28:43.744401+0000","last_active":"2024-04-24T06:28:45.991465+0000","last_peered":"2024-04-24T06:28:45.991465+0000","last_clean":"2024-04-24T06:28:45.991465+0000","last_became_active":"2024-04-24T06:28:43.744085+0000","last_became_peered":"2024-04-24T06:28:43.744085+0000","last_unstale":"2024-04-24T06:28:45.991465+0000","last_undegraded":"2024-04-24T06:28:45.991465+0000","last_fullsized":"2024-04-24T06:28:45.991465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992064+0000","last_change":"2024-04-24T06:28:44.747939+0000","last_active":"2024-04-24T06:28:45.992064+0000","last_peered":"2024-04-24T06:28:45.992064+0000","last_clean":"2024-04-24T06:28:45.992064+0000","last_became_active":"2024-04-24T06:28:44.747593+0000","last_became_peered":"2024-04-24T06:28:44.747593+0000","last_unstale":"2024-04-24T06:28:45.992064+0000","last_undegraded":"2024-04-24T06:28:45.992064+0000","last_fullsized":"2024-04-24T06:28:45.992064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987616+0000","last_change":"2024-04-24T06:28:43.736080+0000","last_active":"2024-04-24T06:28:45.987616+0000","last_peered":"2024-04-24T06:28:45.987616+0000","last_clean":"2024-04-24T06:28:45.987616+0000","last_became_active":"2024-04-24T06:28:43.735262+0000","last_became_peered":"2024-04-24T06:28:43.735262+0000","last_unstale":"2024-04-24T06:28:45.987616+0000","last_undegraded":"2024-04-24T06:28:45.987616+0000","last_fullsized":"2024-04-24T06:28:45.987616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:50:23.356432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994003+0000","last_change":"2024-04-24T06:28:44.748730+0000","last_active":"2024-04-24T06:28:45.994003+0000","last_peered":"2024-04-24T06:28:45.994003+0000","last_clean":"2024-04-24T06:28:45.994003+0000","last_became_active":"2024-04-24T06:28:44.748473+0000","last_became_peered":"2024-04-24T06:28:44.748473+0000","last_unstale":"2024-04-24T06:28:45.994003+0000","last_undegraded":"2024-04-24T06:28:45.994003+0000","last_fullsized":"2024-04-24T06:28:45.994003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:13.432258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992450+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:28:45.992450+0000","last_peered":"2024-04-24T06:28:45.992450+0000","last_clean":"2024-04-24T06:28:45.992450+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:28:45.992450+0000","last_undegraded":"2024-04-24T06:28:45.992450+0000","last_fullsized":"2024-04-24T06:28:45.992450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989929+0000","last_change":"2024-04-24T06:28:43.732980+0000","last_active":"2024-04-24T06:28:45.989929+0000","last_peered":"2024-04-24T06:28:45.989929+0000","last_clean":"2024-04-24T06:28:45.989929+0000","last_became_active":"2024-04-24T06:28:43.732613+0000","last_became_peered":"2024-04-24T06:28:43.732613+0000","last_unstale":"2024-04-24T06:28:45.989929+0000","last_undegraded":"2024-04-24T06:28:45.989929+0000","last_fullsized":"2024-04-24T06:28:45.989929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741850+0000","last_change":"2024-04-24T06:28:44.741822+0000","last_active":"2024-04-24T06:28:45.741850+0000","last_peered":"2024-04-24T06:28:45.741850+0000","last_clean":"2024-04-24T06:28:45.741850+0000","last_became_active":"2024-04-24T06:28:44.737221+0000","last_became_peered":"2024-04-24T06:28:44.737221+0000","last_unstale":"2024-04-24T06:28:45.741850+0000","last_undegraded":"2024-04-24T06:28:45.741850+0000","last_fullsized":"2024-04-24T06:28:45.741850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992125+0000","last_change":"2024-04-24T06:28:43.742633+0000","last_active":"2024-04-24T06:28:45.992125+0000","last_peered":"2024-04-24T06:28:45.992125+0000","last_clean":"2024-04-24T06:28:45.992125+0000","last_became_active":"2024-04-24T06:28:43.742329+0000","last_became_peered":"2024-04-24T06:28:43.742329+0000","last_unstale":"2024-04-24T06:28:45.992125+0000","last_undegraded":"2024-04-24T06:28:45.992125+0000","last_fullsized":"2024-04-24T06:28:45.992125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991037+0000","last_change":"2024-04-24T06:28:44.729217+0000","last_active":"2024-04-24T06:28:45.991037+0000","last_peered":"2024-04-24T06:28:45.991037+0000","last_clean":"2024-04-24T06:28:45.991037+0000","last_became_active":"2024-04-24T06:28:44.728584+0000","last_became_peered":"2024-04-24T06:28:44.728584+0000","last_unstale":"2024-04-24T06:28:45.991037+0000","last_undegraded":"2024-04-24T06:28:45.991037+0000","last_fullsized":"2024-04-24T06:28:45.991037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.987414+0000","last_change":"2024-04-24T06:28:43.737893+0000","last_active":"2024-04-24T06:28:45.987414+0000","last_peered":"2024-04-24T06:28:45.987414+0000","last_clean":"2024-04-24T06:28:45.987414+0000","last_became_active":"2024-04-24T06:28:43.737490+0000","last_became_peered":"2024-04-24T06:28:43.737490+0000","last_unstale":"2024-04-24T06:28:45.987414+0000","last_undegraded":"2024-04-24T06:28:45.987414+0000","last_fullsized":"2024-04-24T06:28:45.987414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.988935+0000","last_change":"2024-04-24T06:28:44.727674+0000","last_active":"2024-04-24T06:28:45.988935+0000","last_peered":"2024-04-24T06:28:45.988935+0000","last_clean":"2024-04-24T06:28:45.988935+0000","last_became_active":"2024-04-24T06:28:44.726973+0000","last_became_peered":"2024-04-24T06:28:44.726973+0000","last_unstale":"2024-04-24T06:28:45.988935+0000","last_undegraded":"2024-04-24T06:28:45.988935+0000","last_fullsized":"2024-04-24T06:28:45.988935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.992992+0000","last_change":"2024-04-24T06:28:43.734680+0000","last_active":"2024-04-24T06:28:45.992992+0000","last_peered":"2024-04-24T06:28:45.992992+0000","last_clean":"2024-04-24T06:28:45.992992+0000","last_became_active":"2024-04-24T06:28:43.734291+0000","last_became_peered":"2024-04-24T06:28:43.734291+0000","last_unstale":"2024-04-24T06:28:45.992992+0000","last_undegraded":"2024-04-24T06:28:45.992992+0000","last_fullsized":"2024-04-24T06:28:45.992992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741903+0000","last_change":"2024-04-24T06:28:44.747768+0000","last_active":"2024-04-24T06:28:45.741903+0000","last_peered":"2024-04-24T06:28:45.741903+0000","last_clean":"2024-04-24T06:28:45.741903+0000","last_became_active":"2024-04-24T06:28:44.747439+0000","last_became_peered":"2024-04-24T06:28:44.747439+0000","last_unstale":"2024-04-24T06:28:45.741903+0000","last_undegraded":"2024-04-24T06:28:45.741903+0000","last_fullsized":"2024-04-24T06:28:45.741903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:18.635305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991108+0000","last_change":"2024-04-24T06:28:43.736187+0000","last_active":"2024-04-24T06:28:45.991108+0000","last_peered":"2024-04-24T06:28:45.991108+0000","last_clean":"2024-04-24T06:28:45.991108+0000","last_became_active":"2024-04-24T06:28:43.735013+0000","last_became_peered":"2024-04-24T06:28:43.735013+0000","last_unstale":"2024-04-24T06:28:45.991108+0000","last_undegraded":"2024-04-24T06:28:45.991108+0000","last_fullsized":"2024-04-24T06:28:45.991108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:31.045051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991099+0000","last_change":"2024-04-24T06:28:44.748068+0000","last_active":"2024-04-24T06:28:45.991099+0000","last_peered":"2024-04-24T06:28:45.991099+0000","last_clean":"2024-04-24T06:28:45.991099+0000","last_became_active":"2024-04-24T06:28:44.747692+0000","last_became_peered":"2024-04-24T06:28:44.747692+0000","last_unstale":"2024-04-24T06:28:45.991099+0000","last_undegraded":"2024-04-24T06:28:45.991099+0000","last_fullsized":"2024-04-24T06:28:45.991099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:29.846561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994396+0000","last_change":"2024-04-24T06:28:43.746631+0000","last_active":"2024-04-24T06:28:45.994396+0000","last_peered":"2024-04-24T06:28:45.994396+0000","last_clean":"2024-04-24T06:28:45.994396+0000","last_became_active":"2024-04-24T06:28:43.746051+0000","last_became_peered":"2024-04-24T06:28:43.746051+0000","last_unstale":"2024-04-24T06:28:45.994396+0000","last_undegraded":"2024-04-24T06:28:45.994396+0000","last_fullsized":"2024-04-24T06:28:45.994396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:21:17.617739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991283+0000","last_change":"2024-04-24T06:28:44.730831+0000","last_active":"2024-04-24T06:28:45.991283+0000","last_peered":"2024-04-24T06:28:45.991283+0000","last_clean":"2024-04-24T06:28:45.991283+0000","last_became_active":"2024-04-24T06:28:44.730464+0000","last_became_peered":"2024-04-24T06:28:44.730464+0000","last_unstale":"2024-04-24T06:28:45.991283+0000","last_undegraded":"2024-04-24T06:28:45.991283+0000","last_fullsized":"2024-04-24T06:28:45.991283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991856+0000","last_change":"2024-04-24T06:28:43.746118+0000","last_active":"2024-04-24T06:28:45.991856+0000","last_peered":"2024-04-24T06:28:45.991856+0000","last_clean":"2024-04-24T06:28:45.991856+0000","last_became_active":"2024-04-24T06:28:43.745789+0000","last_became_peered":"2024-04-24T06:28:43.745789+0000","last_unstale":"2024-04-24T06:28:45.991856+0000","last_undegraded":"2024-04-24T06:28:45.991856+0000","last_fullsized":"2024-04-24T06:28:45.991856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742491+0000","last_change":"2024-04-24T06:28:44.760317+0000","last_active":"2024-04-24T06:28:45.742491+0000","last_peered":"2024-04-24T06:28:45.742491+0000","last_clean":"2024-04-24T06:28:45.742491+0000","last_became_active":"2024-04-24T06:28:44.759974+0000","last_became_peered":"2024-04-24T06:28:44.759974+0000","last_unstale":"2024-04-24T06:28:45.742491+0000","last_undegraded":"2024-04-24T06:28:45.742491+0000","last_fullsized":"2024-04-24T06:28:45.742491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:39:07.972109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989548+0000","last_change":"2024-04-24T06:28:43.751042+0000","last_active":"2024-04-24T06:28:45.989548+0000","last_peered":"2024-04-24T06:28:45.989548+0000","last_clean":"2024-04-24T06:28:45.989548+0000","last_became_active":"2024-04-24T06:28:43.750648+0000","last_became_peered":"2024-04-24T06:28:43.750648+0000","last_unstale":"2024-04-24T06:28:45.989548+0000","last_undegraded":"2024-04-24T06:28:45.989548+0000","last_fullsized":"2024-04-24T06:28:45.989548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991697+0000","last_change":"2024-04-24T06:28:44.735037+0000","last_active":"2024-04-24T06:28:45.991697+0000","last_peered":"2024-04-24T06:28:45.991697+0000","last_clean":"2024-04-24T06:28:45.991697+0000","last_became_active":"2024-04-24T06:28:44.734809+0000","last_became_peered":"2024-04-24T06:28:44.734809+0000","last_unstale":"2024-04-24T06:28:45.991697+0000","last_undegraded":"2024-04-24T06:28:45.991697+0000","last_fullsized":"2024-04-24T06:28:45.991697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:39.310457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991820+0000","last_change":"2024-04-24T06:28:43.744133+0000","last_active":"2024-04-24T06:28:45.991820+0000","last_peered":"2024-04-24T06:28:45.991820+0000","last_clean":"2024-04-24T06:28:45.991820+0000","last_became_active":"2024-04-24T06:28:43.743825+0000","last_became_peered":"2024-04-24T06:28:43.743825+0000","last_unstale":"2024-04-24T06:28:45.991820+0000","last_undegraded":"2024-04-24T06:28:45.991820+0000","last_fullsized":"2024-04-24T06:28:45.991820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742147+0000","last_change":"2024-04-24T06:28:44.739332+0000","last_active":"2024-04-24T06:28:45.742147+0000","last_peered":"2024-04-24T06:28:45.742147+0000","last_clean":"2024-04-24T06:28:45.742147+0000","last_became_active":"2024-04-24T06:28:44.738899+0000","last_became_peered":"2024-04-24T06:28:44.738899+0000","last_unstale":"2024-04-24T06:28:45.742147+0000","last_undegraded":"2024-04-24T06:28:45.742147+0000","last_fullsized":"2024-04-24T06:28:45.742147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:23:54.989726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989171+0000","last_change":"2024-04-24T06:28:43.742255+0000","last_active":"2024-04-24T06:28:45.989171+0000","last_peered":"2024-04-24T06:28:45.989171+0000","last_clean":"2024-04-24T06:28:45.989171+0000","last_became_active":"2024-04-24T06:28:43.741940+0000","last_became_peered":"2024-04-24T06:28:43.741940+0000","last_unstale":"2024-04-24T06:28:45.989171+0000","last_undegraded":"2024-04-24T06:28:45.989171+0000","last_fullsized":"2024-04-24T06:28:45.989171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:04:08.452885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741541+0000","last_change":"2024-04-24T06:28:44.739763+0000","last_active":"2024-04-24T06:28:45.741541+0000","last_peered":"2024-04-24T06:28:45.741541+0000","last_clean":"2024-04-24T06:28:45.741541+0000","last_became_active":"2024-04-24T06:28:44.739490+0000","last_became_peered":"2024-04-24T06:28:44.739490+0000","last_unstale":"2024-04-24T06:28:45.741541+0000","last_undegraded":"2024-04-24T06:28:45.741541+0000","last_fullsized":"2024-04-24T06:28:45.741541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:19:12.070136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990882+0000","last_change":"2024-04-24T06:28:43.735231+0000","last_active":"2024-04-24T06:28:45.990882+0000","last_peered":"2024-04-24T06:28:45.990882+0000","last_clean":"2024-04-24T06:28:45.990882+0000","last_became_active":"2024-04-24T06:28:43.734870+0000","last_became_peered":"2024-04-24T06:28:43.734870+0000","last_unstale":"2024-04-24T06:28:45.990882+0000","last_undegraded":"2024-04-24T06:28:45.990882+0000","last_fullsized":"2024-04-24T06:28:45.990882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:35.789235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.741685+0000","last_change":"2024-04-24T06:28:44.746720+0000","last_active":"2024-04-24T06:28:45.741685+0000","last_peered":"2024-04-24T06:28:45.741685+0000","last_clean":"2024-04-24T06:28:45.741685+0000","last_became_active":"2024-04-24T06:28:44.746083+0000","last_became_peered":"2024-04-24T06:28:44.746083+0000","last_unstale":"2024-04-24T06:28:45.741685+0000","last_undegraded":"2024-04-24T06:28:45.741685+0000","last_fullsized":"2024-04-24T06:28:45.741685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:53.629416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993302+0000","last_change":"2024-04-24T06:28:43.748833+0000","last_active":"2024-04-24T06:28:45.993302+0000","last_peered":"2024-04-24T06:28:45.993302+0000","last_clean":"2024-04-24T06:28:45.993302+0000","last_became_active":"2024-04-24T06:28:43.748530+0000","last_became_peered":"2024-04-24T06:28:43.748530+0000","last_unstale":"2024-04-24T06:28:45.993302+0000","last_undegraded":"2024-04-24T06:28:45.993302+0000","last_fullsized":"2024-04-24T06:28:45.993302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:42.803883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.989154+0000","last_change":"2024-04-24T06:28:44.755539+0000","last_active":"2024-04-24T06:28:45.989154+0000","last_peered":"2024-04-24T06:28:45.989154+0000","last_clean":"2024-04-24T06:28:45.989154+0000","last_became_active":"2024-04-24T06:28:44.755179+0000","last_became_peered":"2024-04-24T06:28:44.755179+0000","last_unstale":"2024-04-24T06:28:45.989154+0000","last_undegraded":"2024-04-24T06:28:45.989154+0000","last_fullsized":"2024-04-24T06:28:45.989154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:49.318690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991127+0000","last_change":"2024-04-24T06:28:44.738377+0000","last_active":"2024-04-24T06:28:45.991127+0000","last_peered":"2024-04-24T06:28:45.991127+0000","last_clean":"2024-04-24T06:28:45.991127+0000","last_became_active":"2024-04-24T06:28:44.738086+0000","last_became_peered":"2024-04-24T06:28:44.738086+0000","last_unstale":"2024-04-24T06:28:45.991127+0000","last_undegraded":"2024-04-24T06:28:45.991127+0000","last_fullsized":"2024-04-24T06:28:45.991127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:25.363169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.993626+0000","last_change":"2024-04-24T06:28:43.747066+0000","last_active":"2024-04-24T06:28:45.993626+0000","last_peered":"2024-04-24T06:28:45.993626+0000","last_clean":"2024-04-24T06:28:45.993626+0000","last_became_active":"2024-04-24T06:28:43.746745+0000","last_became_peered":"2024-04-24T06:28:43.746745+0000","last_unstale":"2024-04-24T06:28:45.993626+0000","last_undegraded":"2024-04-24T06:28:45.993626+0000","last_fullsized":"2024-04-24T06:28:45.993626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:58.688914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991013+0000","last_change":"2024-04-24T06:28:44.754533+0000","last_active":"2024-04-24T06:28:45.991013+0000","last_peered":"2024-04-24T06:28:45.991013+0000","last_clean":"2024-04-24T06:28:45.991013+0000","last_became_active":"2024-04-24T06:28:44.754250+0000","last_became_peered":"2024-04-24T06:28:44.754250+0000","last_unstale":"2024-04-24T06:28:45.991013+0000","last_undegraded":"2024-04-24T06:28:45.991013+0000","last_fullsized":"2024-04-24T06:28:45.991013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.994203+0000","last_change":"2024-04-24T06:28:43.749898+0000","last_active":"2024-04-24T06:28:45.994203+0000","last_peered":"2024-04-24T06:28:45.994203+0000","last_clean":"2024-04-24T06:28:45.994203+0000","last_became_active":"2024-04-24T06:28:43.749544+0000","last_became_peered":"2024-04-24T06:28:43.749544+0000","last_unstale":"2024-04-24T06:28:45.994203+0000","last_undegraded":"2024-04-24T06:28:45.994203+0000","last_fullsized":"2024-04-24T06:28:45.994203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:48.937858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.991965+0000","last_change":"2024-04-24T06:28:44.736764+0000","last_active":"2024-04-24T06:28:45.991965+0000","last_peered":"2024-04-24T06:28:45.991965+0000","last_clean":"2024-04-24T06:28:45.991965+0000","last_became_active":"2024-04-24T06:28:44.736187+0000","last_became_peered":"2024-04-24T06:28:44.736187+0000","last_unstale":"2024-04-24T06:28:45.991965+0000","last_undegraded":"2024-04-24T06:28:45.991965+0000","last_fullsized":"2024-04-24T06:28:45.991965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:55.395726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742308+0000","last_change":"2024-04-24T06:28:43.739421+0000","last_active":"2024-04-24T06:28:45.742308+0000","last_peered":"2024-04-24T06:28:45.742308+0000","last_clean":"2024-04-24T06:28:45.742308+0000","last_became_active":"2024-04-24T06:28:43.737666+0000","last_became_peered":"2024-04-24T06:28:43.737666+0000","last_unstale":"2024-04-24T06:28:45.742308+0000","last_undegraded":"2024-04-24T06:28:45.742308+0000","last_fullsized":"2024-04-24T06:28:45.742308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:32.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.742640+0000","last_change":"2024-04-24T06:28:44.741299+0000","last_active":"2024-04-24T06:28:45.742640+0000","last_peered":"2024-04-24T06:28:45.742640+0000","last_clean":"2024-04-24T06:28:45.742640+0000","last_became_active":"2024-04-24T06:28:44.740960+0000","last_became_peered":"2024-04-24T06:28:44.740960+0000","last_unstale":"2024-04-24T06:28:45.742640+0000","last_undegraded":"2024-04-24T06:28:45.742640+0000","last_fullsized":"2024-04-24T06:28:45.742640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:43.684851+0000","last_clean_scrub_stamp":"2024-04-24T06:28:43.684851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:43.750488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-24T06:28:45.990886+0000","last_change":"2024-04-24T06:28:43.734445+0000","last_active":"2024-04-24T06:28:45.990886+0000","last_peered":"2024-04-24T06:28:45.990886+0000","last_clean":"2024-04-24T06:28:45.990886+0000","last_became_active":"2024-04-24T06:28:43.733088+0000","last_became_peered":"2024-04-24T06:28:43.733088+0000","last_unstale":"2024-04-24T06:28:45.990886+0000","last_undegraded":"2024-04-24T06:28:45.990886+0000","last_fullsized":"2024-04-24T06:28:45.990886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:42.667285+0000","last_clean_scrub_stamp":"2024-04-24T06:28:42.667285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:11.461086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:28:55.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T06:28:55.378 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T06:28:55.378 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T06:28:55.378 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:28:55.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.615+0000 7fee1fc00640 1 Processor -- start 2024-04-24T06:28:55.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.616+0000 7fee1fc00640 1 -- start start 2024-04-24T06:28:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.617+0000 7fee1fc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fee180afaf0 0x7fee180d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.617+0000 7fee1fc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1815f0f0 con 0x7fee180d06e0 2024-04-24T06:28:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.617+0000 7fee1fc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1815f3d0 con 0x7fee1815c5e0 2024-04-24T06:28:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.617+0000 7fee1fc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fee1815fa20 con 0x7fee180afaf0 2024-04-24T06:28:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee1dc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fee180d06e0 0x7fee181624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60068/0 (socket says 172.21.15.59:60068) 2024-04-24T06:28:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee1dc00640 1 -- 172.21.15.59:0/3657773304 learned_addr learned my addr 172.21.15.59:0/3657773304 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17e00640 1 --2- 172.21.15.59:0/3657773304 >> v2:172.21.15.66:3300/0 conn(0x7fee180afaf0 0x7fee180d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433118527 0 0) 0x7fee1815f0f0 con 0x7fee180d06e0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee00003520 con 0x7fee180d06e0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2119814976 0 0) 0x7fee00003520 con 0x7fee180d06e0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 >> v2:172.21.15.66:3300/0 conn(0x7fee180afaf0 msgr2=0x7fee180d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 --2- 172.21.15.59:0/3657773304 >> v2:172.21.15.66:3300/0 conn(0x7fee180afaf0 0x7fee180d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 >> v1:172.21.15.66:6789/0 conn(0x7fee1815c5e0 legacy=0x7fee180af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1815f6b0 con 0x7fee180d06e0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fee100019f0 con 0x7fee180d06e0 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.618+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 >> v1:172.21.15.59:6789/0 conn(0x7fee180d06e0 legacy=0x7fee181624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 0x7fee00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00004800 0x7fee00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee000051c0 0x7fee0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee1815fa20 con 0x7fee00003a00 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee0000c0c0 con 0x7fee000051c0 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fee0000c2c0 con 0x7fee00004800 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 0x7fee00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee17e00640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00004800 0x7fee00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 -- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00004800 msgr2=0x7fee00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00004800 0x7fee00004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 -- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee000051c0 msgr2=0x7fee0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1dc00640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee000051c0 0x7fee0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee000051c0 0x7fee0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1d200640 1 -- 172.21.15.59:0/3657773304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee00003520 con 0x7fee00003a00 2024-04-24T06:28:55.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1dc00640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee000051c0 0x7fee0000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:55.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.619+0000 7fee1dc00640 1 -- 172.21.15.59:0/3657773304 reap_dead start 2024-04-24T06:28:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.620+0000 7fee1d200640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 0x7fee00003e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee0c001770 tx=0x7fee00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c83ad481fbc8f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.621+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c001390 con 0x7fee00003a00 2024-04-24T06:28:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.621+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c0052d0 con 0x7fee00003a00 2024-04-24T06:28:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.621+0000 7fee17400640 1 -- 172.21.15.59:0/3657773304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c02ab20 con 0x7fee00003a00 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.621+0000 7fee1fc00640 1 -- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 msgr2=0x7fee00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.621+0000 7fee1fc00640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 0x7fee00003e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee0c001770 tx=0x7fee00005830 comp rx=0 tx=0).stop 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.622+0000 7fee1fc00640 1 -- 172.21.15.59:0/3657773304 shutdown_connections 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.622+0000 7fee1fc00640 1 --2- 172.21.15.59:0/3657773304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee00003a00 0x7fee00003e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.622+0000 7fee1fc00640 1 -- 172.21.15.59:0/3657773304 >> 172.21.15.59:0/3657773304 conn(0x7fee18067ea0 msgr2=0x7fee1806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.622+0000 7fee1fc00640 1 -- 172.21.15.59:0/3657773304 shutdown_connections 2024-04-24T06:28:55.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.622+0000 7fee1fc00640 1 -- 172.21.15.59:0/3657773304 wait complete. 2024-04-24T06:28:55.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.623+0000 7fee1fc00640 1 Processor -- start 2024-04-24T06:28:55.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 -- start start 2024-04-24T06:28:55.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00003a00 0x7fee18146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee18147640 0x7fee18147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee1806e600 con 0x7fee18154270 2024-04-24T06:28:55.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee1806e480 con 0x7fee18147640 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.624+0000 7fee1fc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fee180d01a0 con 0x7fee00003a00 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38206/0 (socket says 172.21.15.59:38206) 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 -- 172.21.15.59:0/1231909747 learned_addr learned my addr 172.21.15.59:0/1231909747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee1d200640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00003a00 0x7fee18146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee1dc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee18147640 0x7fee18147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 -- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00003a00 msgr2=0x7fee18146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00003a00 0x7fee18146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 -- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee18147640 msgr2=0x7fee18147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee18147640 0x7fee18147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 -- 172.21.15.59:0/1231909747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee18160620 con 0x7fee18154270 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.625+0000 7fee17e00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee040108e0 tx=0x7fee04010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.626+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0400f040 con 0x7fee18154270 2024-04-24T06:28:55.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.626+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0400eb00 con 0x7fee18154270 2024-04-24T06:28:55.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.626+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0401c400 con 0x7fee18154270 2024-04-24T06:28:55.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.626+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee18160900 con 0x7fee18154270 2024-04-24T06:28:55.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.626+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee18160ec0 con 0x7fee18154270 2024-04-24T06:28:55.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.627+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee18068a80 con 0x7fee18154270 2024-04-24T06:28:55.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.628+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee0401c5a0 con 0x7fee18154270 2024-04-24T06:28:55.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.629+0000 7fee14c00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 0x7fedf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:55.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.629+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee0408c280 con 0x7fee18154270 2024-04-24T06:28:55.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.629+0000 7fee1d200640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 0x7fedf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:55.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.630+0000 7fee1d200640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 0x7fedf806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee0c032000 tx=0x7fee0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:55.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.635+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee04018030 con 0x7fee18154270 2024-04-24T06:28:55.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.872+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fee1806e260 con 0x7fee18154270 2024-04-24T06:28:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.873+0000 7fee14c00640 1 -- 172.21.15.59:0/1231909747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fee04059fc0 con 0x7fee18154270 2024-04-24T06:28:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:55.874 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 msgr2=0x7fedf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 0x7fedf806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee0c032000 tx=0x7fee0c02e000 comp rx=0 tx=0).stop 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 msgr2=0x7fee18158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee040108e0 tx=0x7fee04010da0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 shutdown_connections 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedf806c8f0 0x7fedf806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee18147640 0x7fee18147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee18154270 0x7fee18158530 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 --2- 172.21.15.59:0/1231909747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fee00003a00 0x7fee18146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.875+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 >> 172.21.15.59:0/1231909747 conn(0x7fee18067ea0 msgr2=0x7fee1806ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.876+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 shutdown_connections 2024-04-24T06:28:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:55.876+0000 7fee1fc00640 1 -- 172.21.15.59:0/1231909747 wait complete. 2024-04-24T06:28:55.894 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T06:28:55.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:56.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.100+0000 7f8a36a00640 1 Processor -- start 2024-04-24T06:28:56.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.100+0000 7f8a36a00640 1 -- start start 2024-04-24T06:28:56.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.101+0000 7f8a36a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a3015e7c0 0x7f8a300d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a36a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8a300d0d90 0x7f8a300d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a36a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a300cf030 con 0x7f8a300d2630 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a36a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8a300b7000 con 0x7f8a3015e7c0 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a36a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8a300cf620 con 0x7f8a300d0d90 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a3015e7c0 0x7f8a300d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a3015e7c0 0x7f8a300d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38222/0 (socket says 172.21.15.59:38222) 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 -- 172.21.15.59:0/3014542733 learned_addr learned my addr 172.21.15.59:0/3014542733 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 -- 172.21.15.59:0/3014542733 >> v2:172.21.15.66:3300/0 conn(0x7f8a300d0d90 msgr2=0x7f8a300d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 --2- 172.21.15.59:0/3014542733 >> v2:172.21.15.66:3300/0 conn(0x7f8a300d0d90 0x7f8a300d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 -- 172.21.15.59:0/3014542733 >> v1:172.21.15.66:6789/0 conn(0x7f8a300d2630 legacy=0x7f8a300b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.102+0000 7f8a35600640 1 -- 172.21.15.59:0/3014542733 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a300b6a40 con 0x7f8a3015e7c0 2024-04-24T06:28:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.103+0000 7f8a35600640 1 --2- 172.21.15.59:0/3014542733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a3015e7c0 0x7f8a300d20f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a300cb580 tx=0x7f8a2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd40da2ce466e753 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.103+0000 7f8a2fe00640 1 -- 172.21.15.59:0/3014542733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24036070 con 0x7f8a3015e7c0 2024-04-24T06:28:56.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a2fe00640 1 -- 172.21.15.59:0/3014542733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a24031600 con 0x7f8a3015e7c0 2024-04-24T06:28:56.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a2fe00640 1 -- 172.21.15.59:0/3014542733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240319d0 con 0x7f8a3015e7c0 2024-04-24T06:28:56.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 -- 172.21.15.59:0/3014542733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a3015e7c0 msgr2=0x7f8a300d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 --2- 172.21.15.59:0/3014542733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a3015e7c0 0x7f8a300d20f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a300cb580 tx=0x7f8a2402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 -- 172.21.15.59:0/3014542733 shutdown_connections 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 --2- 172.21.15.59:0/3014542733 >> v2:172.21.15.66:3300/0 conn(0x7f8a300d0d90 0x7f8a300d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 --2- 172.21.15.59:0/3014542733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a3015e7c0 0x7f8a300d20f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 -- 172.21.15.59:0/3014542733 >> 172.21.15.59:0/3014542733 conn(0x7f8a3009d920 msgr2=0x7f8a3008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 -- 172.21.15.59:0/3014542733 shutdown_connections 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.104+0000 7f8a36a00640 1 -- 172.21.15.59:0/3014542733 wait complete. 2024-04-24T06:28:56.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.106+0000 7f8a36a00640 1 Processor -- start 2024-04-24T06:28:56.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.106+0000 7f8a36a00640 1 -- start start 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a300d0d90 0x7f8a30151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3015e7c0 0x7f8a300b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300cf2b0 con 0x7f8a300d2630 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300d1700 con 0x7f8a3015e7c0 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a36a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a300b6700 con 0x7f8a300d0d90 2024-04-24T06:28:56.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a34c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a34c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38238/0 (socket says 172.21.15.59:38238) 2024-04-24T06:28:56.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a34c00640 1 -- 172.21.15.59:0/989316768 learned_addr learned my addr 172.21.15.59:0/989316768 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a35600640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a300d0d90 0x7f8a30151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.107+0000 7f8a34c00640 1 -- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a300d0d90 msgr2=0x7f8a30151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a34c00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a300d0d90 0x7f8a30151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a34c00640 1 -- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3015e7c0 msgr2=0x7f8a300b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a34c00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3015e7c0 0x7f8a300b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a34c00640 1 -- 172.21.15.59:0/989316768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a300c81d0 con 0x7f8a300d2630 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a34c00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c430 tx=0x7f8a1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c017070 con 0x7f8a300d2630 2024-04-24T06:28:56.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0035f0 con 0x7f8a300d2630 2024-04-24T06:28:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0039e0 con 0x7f8a300d2630 2024-04-24T06:28:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a300c84b0 con 0x7f8a300d2630 2024-04-24T06:28:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.108+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a300c8a70 con 0x7f8a300d2630 2024-04-24T06:28:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.109+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a300c8c70 con 0x7f8a300d2630 2024-04-24T06:28:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.110+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a1c003b80 con 0x7f8a300d2630 2024-04-24T06:28:56.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.111+0000 7f8a2d600640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.111+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a1c08aff0 con 0x7f8a300d2630 2024-04-24T06:28:56.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.112+0000 7f8a35600640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.112+0000 7f8a35600640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 0x7f8a1006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a24033000 tx=0x7f8a24009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.116+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a300c8c70 con 0x7f8a300d2630 2024-04-24T06:28:56.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.321+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a3004fd90 con 0x7f8a300d2630 2024-04-24T06:28:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.323+0000 7f8a2d600640 1 -- 172.21.15.59:0/989316768 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f8a1c059380 con 0x7f8a300d2630 2024-04-24T06:28:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:56.324 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.325+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 msgr2=0x7f8a1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.325+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 0x7f8a1006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a24033000 tx=0x7f8a24009000 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.325+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 msgr2=0x7f8a300b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.325+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c430 tx=0x7f8a1c00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 shutdown_connections 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a3015e7c0 0x7f8a300b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a300d2630 0x7f8a300b5520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 --2- 172.21.15.59:0/989316768 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a300d0d90 0x7f8a30151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 >> 172.21.15.59:0/989316768 conn(0x7f8a3009d920 msgr2=0x7f8a30160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 shutdown_connections 2024-04-24T06:28:56.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.326+0000 7f8a36a00640 1 -- 172.21.15.59:0/989316768 wait complete. 2024-04-24T06:28:56.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:28:56.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:28:56.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.557+0000 7f0f83000640 1 Processor -- start 2024-04-24T06:28:56.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.557+0000 7f0f83000640 1 -- start start 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f83000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f7c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f83000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f7c0b5da0 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f83000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f7c0d0a20 con 0x7f0f7c15ed20 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f83000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7c15f100 con 0x7f0f7c0d15b0 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f82600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0f7c0b4f20 0x7f0f7c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60072/0 (socket says 172.21.15.59:60072) 2024-04-24T06:28:56.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f82600640 1 -- 172.21.15.59:0/1878223836 learned_addr learned my addr 172.21.15.59:0/1878223836 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.559+0000 7f0f81200640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f7c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235107897 0 0) 0x7f0f7c0b5da0 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f64003520 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3542727970 0 0) 0x7f0f64003520 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 msgr2=0x7f0f7c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f7c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 >> v1:172.21.15.66:6789/0 conn(0x7f0f7c15ed20 legacy=0x7f0f7c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c0d2ed0 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0f740019f0 con 0x7f0f7c0b4f20 2024-04-24T06:28:56.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 >> v1:172.21.15.59:6789/0 conn(0x7f0f7c0b4f20 legacy=0x7f0f7c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640039e0 0x7f0f64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f64004d40 0x7f0f640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6400bb50 con 0x7f0f640039e0 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6400bd00 con 0x7f0f64004d40 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6400bf00 con 0x7f0f7c0d15b0 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.560+0000 7f0f81c00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640039e0 0x7f0f64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f82600640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f64004d40 0x7f0f640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81200640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81c00640 1 -- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 msgr2=0x7f0f64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81c00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0d15b0 0x7f0f64004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81c00640 1 -- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f64004d40 msgr2=0x7f0f640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81c00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f64004d40 0x7f0f640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.561+0000 7f0f81c00640 1 -- 172.21.15.59:0/1878223836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f64003520 con 0x7f0f640039e0 2024-04-24T06:28:56.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.562+0000 7f0f81c00640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640039e0 0x7f0f64003e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001750 tx=0x7f0f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b9c5f1f601df38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.562+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70001370 con 0x7f0f640039e0 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.562+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f700052d0 con 0x7f0f640039e0 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.562+0000 7f0f7be00640 1 -- 172.21.15.59:0/1878223836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7002abe0 con 0x7f0f640039e0 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 -- 172.21.15.59:0/1878223836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640039e0 msgr2=0x7f0f64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 --2- 172.21.15.59:0/1878223836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640039e0 0x7f0f64003e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001750 tx=0x7f0f64005830 comp rx=0 tx=0).stop 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f82600640 1 -- 172.21.15.59:0/1878223836 reap_dead start 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 -- 172.21.15.59:0/1878223836 shutdown_connections 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 -- 172.21.15.59:0/1878223836 >> 172.21.15.59:0/1878223836 conn(0x7f0f7c08bf90 msgr2=0x7f0f7c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 -- 172.21.15.59:0/1878223836 shutdown_connections 2024-04-24T06:28:56.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.563+0000 7f0f83000640 1 -- 172.21.15.59:0/1878223836 wait complete. 2024-04-24T06:28:56.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.565+0000 7f0f83000640 1 Processor -- start 2024-04-24T06:28:56.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.565+0000 7f0f83000640 1 -- start start 2024-04-24T06:28:56.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 0x7f0f7c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0b9500 0x7f0f7c0b9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b3600 con 0x7f0f7c1601b0 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c0d3490 con 0x7f0f640039e0 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f83000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7c0b3780 con 0x7f0f7c0b9500 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38254/0 (socket says 172.21.15.59:38254) 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 -- 172.21.15.59:0/3148069287 learned_addr learned my addr 172.21.15.59:0/3148069287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81c00640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 0x7f0f7c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f82600640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0b9500 0x7f0f7c0b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 -- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0b9500 msgr2=0x7f0f7c0b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0b9500 0x7f0f7c0b9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 -- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 msgr2=0x7f0f7c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.566+0000 7f0f81200640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 0x7f0f7c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f81200640 1 -- 172.21.15.59:0/3148069287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c15ce20 con 0x7f0f7c1601b0 2024-04-24T06:28:56.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f81c00640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 0x7f0f7c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:56.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f81200640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f68010890 tx=0x7f0f68010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800f040 con 0x7f0f7c1601b0 2024-04-24T06:28:56.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6800eb60 con 0x7f0f7c1601b0 2024-04-24T06:28:56.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6801c400 con 0x7f0f7c1601b0 2024-04-24T06:28:56.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c160720 con 0x7f0f7c1601b0 2024-04-24T06:28:56.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.567+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c160ce0 con 0x7f0f7c1601b0 2024-04-24T06:28:56.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.568+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c04fd90 con 0x7f0f7c1601b0 2024-04-24T06:28:56.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.569+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f6801c5a0 con 0x7f0f7c1601b0 2024-04-24T06:28:56.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.570+0000 7f0f79600640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 0x7f0f5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.571+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6808c130 con 0x7f0f7c1601b0 2024-04-24T06:28:56.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.571+0000 7f0f81c00640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 0x7f0f5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.572+0000 7f0f81c00640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 0x7f0f5c06ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f70032000 tx=0x7f0f7002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.575+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f68018030 con 0x7f0f7c1601b0 2024-04-24T06:28:56.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.773+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f7c15d070 con 0x7f0f7c1601b0 2024-04-24T06:28:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.774+0000 7f0f79600640 1 -- 172.21.15.59:0/3148069287 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0f68059e70 con 0x7f0f7c1601b0 2024-04-24T06:28:56.775 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:28:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.776+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 msgr2=0x7f0f5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.776+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 0x7f0f5c06ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f70032000 tx=0x7f0f7002e000 comp rx=0 tx=0).stop 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.776+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 msgr2=0x7f0f7c15c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f68010890 tx=0x7f0f68010d50 comp rx=0 tx=0).stop 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 shutdown_connections 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f5c06c920 0x7f0f5c06ede0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f7c0b9500 0x7f0f7c0b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f7c1601b0 0x7f0f7c15c460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 --2- 172.21.15.59:0/3148069287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f640039e0 0x7f0f7c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 >> 172.21.15.59:0/3148069287 conn(0x7f0f7c08bf90 msgr2=0x7f0f7c0d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 shutdown_connections 2024-04-24T06:28:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.777+0000 7f0f83000640 1 -- 172.21.15.59:0/3148069287 wait complete. 2024-04-24T06:28:56.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:28:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.986+0000 7fc01f200640 1 Processor -- start 2024-04-24T06:28:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.986+0000 7fc01f200640 1 -- start start 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.987+0000 7fc01f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc020072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc020072b50 0x7fc0200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02006efe0 con 0x7fc020155280 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc020072fc0 con 0x7fc020072b50 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc02015c0c0 con 0x7fc02015bc80 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc020155280 0x7fc02006e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60074/0 (socket says 172.21.15.59:60074) 2024-04-24T06:28:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01de00640 1 -- 172.21.15.59:0/2297070505 learned_addr learned my addr 172.21.15.59:0/2297070505 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191253011 0 0) 0x7fc02006efe0 con 0x7fc020155280 2024-04-24T06:28:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> v2:172.21.15.66:3300/0 conn(0x7fc020072b50 0x7fc0200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01d400640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc020072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.988+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc004003520 con 0x7fc020155280 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 854332967 0 0) 0x7fc004003520 con 0x7fc020155280 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 msgr2=0x7fc020072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc020072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 >> v2:172.21.15.66:3300/0 conn(0x7fc020072b50 msgr2=0x7fc0200b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 --2- 172.21.15.59:0/2297070505 >> v2:172.21.15.66:3300/0 conn(0x7fc020072b50 0x7fc0200b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02006f2c0 con 0x7fc020155280 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc010001670 con 0x7fc020155280 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> v2:172.21.15.66:3300/0 conn(0x7fc020072b50 0x7fc0200b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 >> v1:172.21.15.59:6789/0 conn(0x7fc020155280 legacy=0x7fc02006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc004003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc004004150 0x7fc00400a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00400af20 0x7fc00400d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc020072fc0 con 0x7fc00400af20 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc02015c0c0 con 0x7fc004004150 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc00400dc70 con 0x7fc02015bc80 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00400af20 0x7fc00400d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.989+0000 7fc01de00640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc004003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01d400640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc004004150 0x7fc00400a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01e800640 1 -- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 msgr2=0x7fc004003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc02015bc80 0x7fc004003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01e800640 1 -- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc004004150 msgr2=0x7fc00400a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc004004150 0x7fc00400a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01e800640 1 -- 172.21.15.59:0/2297070505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc004003520 con 0x7fc00400af20 2024-04-24T06:28:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.990+0000 7fc01d400640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc004004150 0x7fc00400a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.991+0000 7fc01e800640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00400af20 0x7fc00400d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc0180073e0 tx=0x7fc004003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43056b3c2571feb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.991+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018009c80 con 0x7fc00400af20 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.991+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc01802c680 con 0x7fc00400af20 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.991+0000 7fc01ca00640 1 -- 172.21.15.59:0/2297070505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01802ca30 con 0x7fc00400af20 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 -- 172.21.15.59:0/2297070505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00400af20 msgr2=0x7fc00400d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 --2- 172.21.15.59:0/2297070505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc00400af20 0x7fc00400d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc0180073e0 tx=0x7fc004003810 comp rx=0 tx=0).stop 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01e800640 1 -- 172.21.15.59:0/2297070505 reap_dead start 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 -- 172.21.15.59:0/2297070505 shutdown_connections 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 -- 172.21.15.59:0/2297070505 >> 172.21.15.59:0/2297070505 conn(0x7fc02006bdf0 msgr2=0x7fc02006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 -- 172.21.15.59:0/2297070505 shutdown_connections 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.992+0000 7fc01f200640 1 -- 172.21.15.59:0/2297070505 wait complete. 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.994+0000 7fc01f200640 1 Processor -- start 2024-04-24T06:28:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.994+0000 7fc01f200640 1 -- start start 2024-04-24T06:28:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc00400af20 0x7fc0200ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc020154500 0x7fc0200c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc02006fb60 con 0x7fc02015dd30 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc02006f880 con 0x7fc00400af20 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc020072610 con 0x7fc020154500 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38268/0 (socket says 172.21.15.59:38268) 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01d400640 1 -- 172.21.15.59:0/3128364365 learned_addr learned my addr 172.21.15.59:0/3128364365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.995+0000 7fc01de00640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc00400af20 0x7fc0200ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 -- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc020154500 msgr2=0x7fc0200c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01e800640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc020154500 0x7fc0200c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc020154500 0x7fc0200c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 -- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc00400af20 msgr2=0x7fc0200ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc00400af20 0x7fc0200ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 -- 172.21.15.59:0/3128364365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02015cc90 con 0x7fc02015dd30 2024-04-24T06:28:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01d400640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0080108c0 tx=0x7fc008010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00800f040 con 0x7fc02015dd30 2024-04-24T06:28:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00800eb00 con 0x7fc02015dd30 2024-04-24T06:28:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02015cf70 con 0x7fc02015dd30 2024-04-24T06:28:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00801c400 con 0x7fc02015dd30 2024-04-24T06:28:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.996+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02015d530 con 0x7fc02015dd30 2024-04-24T06:28:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.997+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02004fd90 con 0x7fc02015dd30 2024-04-24T06:28:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.998+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc00801c5a0 con 0x7fc02015dd30 2024-04-24T06:28:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.999+0000 7fc016000640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 0x7fbffc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:56.999+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc00808c130 con 0x7fc02015dd30 2024-04-24T06:28:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.000+0000 7fc01de00640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 0x7fbffc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.000+0000 7fc01de00640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 0x7fbffc06ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0100057f0 tx=0x7fc010000f50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.004+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc008018030 con 0x7fc02015dd30 2024-04-24T06:28:57.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.208+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc02006e2f0 con 0x7fc02015dd30 2024-04-24T06:28:57.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.208+0000 7fc016000640 1 -- 172.21.15.59:0/3128364365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc00805a550 con 0x7fc02015dd30 2024-04-24T06:28:57.209 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:28:57.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 msgr2=0x7fbffc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 0x7fbffc06ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0100057f0 tx=0x7fc010000f50 comp rx=0 tx=0).stop 2024-04-24T06:28:57.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 msgr2=0x7fc02015c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0080108c0 tx=0x7fc008010d80 comp rx=0 tx=0).stop 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 shutdown_connections 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbffc06c920 0x7fbffc06ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc020154500 0x7fc0200c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc02015dd30 0x7fc02015c520 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 --2- 172.21.15.59:0/3128364365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc00400af20 0x7fc0200ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.211+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 >> 172.21.15.59:0/3128364365 conn(0x7fc02006bdf0 msgr2=0x7fc02015e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.212+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 shutdown_connections 2024-04-24T06:28:57.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.212+0000 7fc01f200640 1 -- 172.21.15.59:0/3128364365 wait complete. 2024-04-24T06:28:57.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:28:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.442+0000 7f69ad400640 1 Processor -- start 2024-04-24T06:28:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.442+0000 7f69ad400640 1 -- start start 2024-04-24T06:28:57.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.443+0000 7f69ad400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d0600 0x7f69a80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.443+0000 7f69ad400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69a806dcb0 0x7f69a8162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.443+0000 7f69ad400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a8163240 con 0x7f69a80d09e0 2024-04-24T06:28:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f69ad400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f69a8072a20 con 0x7f69a806dcb0 2024-04-24T06:28:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f69ad400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69a8071e10 con 0x7f69a80d0600 2024-04-24T06:28:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f699f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69a806dcb0 0x7f69a8162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f69aca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f69a80d09e0 0x7f69a806e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60086/0 (socket says 172.21.15.59:60086) 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f69aca00640 1 -- 172.21.15.59:0/3596609597 learned_addr learned my addr 172.21.15.59:0/3596609597 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.444+0000 7f69a7e00640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d0600 0x7f69a80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.445+0000 7f699f400640 1 -- 172.21.15.59:0/3596609597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d0600 msgr2=0x7f69a80ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.445+0000 7f699f400640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d0600 0x7f69a80ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.445+0000 7f699f400640 1 -- 172.21.15.59:0/3596609597 >> v1:172.21.15.59:6789/0 conn(0x7f69a80d09e0 legacy=0x7f69a806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.445+0000 7f699f400640 1 -- 172.21.15.59:0/3596609597 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a80720f0 con 0x7f69a806dcb0 2024-04-24T06:28:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.445+0000 7f69a7400640 1 -- 172.21.15.59:0/3596609597 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674912761 0 0) 0x7f69a0001110 con 0x7f69a80d09e0 2024-04-24T06:28:57.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f699f400640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a806dcb0 0x7f69a8162d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48a869632ed0373 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f69a7400640 1 -- 172.21.15.59:0/3596609597 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990037070 con 0x7f69a806dcb0 2024-04-24T06:28:57.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f69a7400640 1 -- 172.21.15.59:0/3596609597 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6990032440 con 0x7f69a806dcb0 2024-04-24T06:28:57.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f69a7400640 1 -- 172.21.15.59:0/3596609597 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990032810 con 0x7f69a806dcb0 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f69ad400640 1 -- 172.21.15.59:0/3596609597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a806dcb0 msgr2=0x7f69a8162d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.446+0000 7f69ad400640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a806dcb0 0x7f69a8162d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002a180 comp rx=0 tx=0).stop 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 -- 172.21.15.59:0/3596609597 shutdown_connections 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a806dcb0 0x7f69a8162d00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 --2- 172.21.15.59:0/3596609597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d0600 0x7f69a80ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 -- 172.21.15.59:0/3596609597 >> 172.21.15.59:0/3596609597 conn(0x7f69a806bdf0 msgr2=0x7f69a806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 -- 172.21.15.59:0/3596609597 shutdown_connections 2024-04-24T06:28:57.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.447+0000 7f69ad400640 1 -- 172.21.15.59:0/3596609597 wait complete. 2024-04-24T06:28:57.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.448+0000 7f69ad400640 1 Processor -- start 2024-04-24T06:28:57.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.448+0000 7f69ad400640 1 -- start start 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69a806dcb0 0x7f69a81493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d09e0 0x7f69a80b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69a815eaf0 con 0x7f69a80d0600 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f69a80726b0 con 0x7f69a806dcb0 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.449+0000 7f69ad400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69a80d0e70 con 0x7f69a80d09e0 2024-04-24T06:28:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38278/0 (socket says 172.21.15.59:38278) 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 -- 172.21.15.59:0/561026799 learned_addr learned my addr 172.21.15.59:0/561026799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 -- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d09e0 msgr2=0x7f69a80b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d09e0 0x7f69a80b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 -- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69a806dcb0 msgr2=0x7f69a81493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69a806dcb0 0x7f69a81493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 -- 172.21.15.59:0/561026799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8160cf0 con 0x7f69a80d0600 2024-04-24T06:28:57.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.450+0000 7f699f400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69900098c0 tx=0x7f69900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.451+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990037040 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.451+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6990030500 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.451+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900308d0 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.451+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a8160450 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.451+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a8160a10 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.452+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a804fd90 con 0x7f69a80d0600 2024-04-24T06:28:57.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.453+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6990044020 con 0x7f69a80d0600 2024-04-24T06:28:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.454+0000 7f69a4c00640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 0x7f698406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.454+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f69900abd70 con 0x7f69a80d0600 2024-04-24T06:28:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.454+0000 7f69a7e00640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 0x7f698406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.455+0000 7f69a7e00640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 0x7f698406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f699800e190 tx=0x7f699800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.458+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f699007a190 con 0x7f69a80d0600 2024-04-24T06:28:57.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.651+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f69a80cff50 con 0x7f69a80d0600 2024-04-24T06:28:57.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.652+0000 7f69a4c00640 1 -- 172.21.15.59:0/561026799 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6990079a90 con 0x7f69a80d0600 2024-04-24T06:28:57.653 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:28:57.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.654+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 msgr2=0x7f698406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.654+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 0x7f698406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f699800e190 tx=0x7f699800b040 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.654+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 msgr2=0x7f69a815fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.654+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69900098c0 tx=0x7f69900046c0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 shutdown_connections 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f698406c8f0 0x7f698406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69a80d09e0 0x7f69a80b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69a80d0600 0x7f69a815fa00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 --2- 172.21.15.59:0/561026799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69a806dcb0 0x7f69a81493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 >> 172.21.15.59:0/561026799 conn(0x7f69a806bdf0 msgr2=0x7f69a80ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:57.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 shutdown_connections 2024-04-24T06:28:57.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.655+0000 7f69ad400640 1 -- 172.21.15.59:0/561026799 wait complete. 2024-04-24T06:28:57.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:28:57.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.874+0000 7f76fd400640 1 Processor -- start 2024-04-24T06:28:57.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.874+0000 7f76fd400640 1 -- start start 2024-04-24T06:28:57.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76fd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f76f806f7c0 0x7f76f80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76fd400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f806ec60 con 0x7f76f806df00 2024-04-24T06:28:57.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76fd400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f80b46e0 con 0x7f76f815b840 2024-04-24T06:28:57.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76fd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80b49c0 con 0x7f76f806f7c0 2024-04-24T06:28:57.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76f7200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f76f806df00 0x7f76f806e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60100/0 (socket says 172.21.15.59:60100) 2024-04-24T06:28:57.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76f7200640 1 -- 172.21.15.59:0/2407610136 learned_addr learned my addr 172.21.15.59:0/2407610136 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76f6800640 1 --2- 172.21.15.59:0/2407610136 >> v2:172.21.15.66:3300/0 conn(0x7f76f806f7c0 0x7f76f80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209532309 0 0) 0x7f76f806ec60 con 0x7f76f806df00 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.876+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76dc005730 con 0x7f76f806df00 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 159000893 0 0) 0x7f76dc005730 con 0x7f76f806df00 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 >> v2:172.21.15.66:3300/0 conn(0x7f76f806f7c0 msgr2=0x7f76f80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 --2- 172.21.15.59:0/2407610136 >> v2:172.21.15.66:3300/0 conn(0x7f76f806f7c0 0x7f76f80b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 >> v1:172.21.15.66:6789/0 conn(0x7f76f815b840 legacy=0x7f76f806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80b4ca0 con 0x7f76f806df00 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f76f0001670 con 0x7f76f806df00 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 >> v1:172.21.15.59:6789/0 conn(0x7f76f806df00 legacy=0x7f76f806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76dc005bc0 0x7f76dc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76dc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76dc0073d0 0x7f76dc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b49c0 con 0x7f76dc006a10 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc00e270 con 0x7f76dc005bc0 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc00e470 con 0x7f76dc0073d0 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.877+0000 7f76ede00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76dc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76ede00640 1 -- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76dc0073d0 msgr2=0x7f76dc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76ede00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76dc0073d0 0x7f76dc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76ede00640 1 -- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76dc005bc0 msgr2=0x7f76dc005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76ede00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76dc005bc0 0x7f76dc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76ede00640 1 -- 172.21.15.59:0/2407610136 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc005730 con 0x7f76dc006a10 2024-04-24T06:28:57.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.878+0000 7f76f7200640 1 -- 172.21.15.59:0/2407610136 reap_dead start 2024-04-24T06:28:57.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76ede00640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76dc006e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76e0001750 tx=0x7f76dc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5282c552c8a329 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0001370 con 0x7f76dc006a10 2024-04-24T06:28:57.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e00052d0 con 0x7f76dc006a10 2024-04-24T06:28:57.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76f5e00640 1 -- 172.21.15.59:0/2407610136 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e002abc0 con 0x7f76dc006a10 2024-04-24T06:28:57.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76fd400640 1 -- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 msgr2=0x7f76dc006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.879+0000 7f76fd400640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76dc006e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76e0001750 tx=0x7f76dc007a40 comp rx=0 tx=0).stop 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.880+0000 7f76fd400640 1 -- 172.21.15.59:0/2407610136 shutdown_connections 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.880+0000 7f76fd400640 1 --2- 172.21.15.59:0/2407610136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76dc006e90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.880+0000 7f76fd400640 1 -- 172.21.15.59:0/2407610136 >> 172.21.15.59:0/2407610136 conn(0x7f76f8067ea0 msgr2=0x7f76f806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.880+0000 7f76fd400640 1 -- 172.21.15.59:0/2407610136 shutdown_connections 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.880+0000 7f76fd400640 1 -- 172.21.15.59:0/2407610136 wait complete. 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.881+0000 7f76fd400640 1 Processor -- start 2024-04-24T06:28:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.882+0000 7f76fd400640 1 -- start start 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.882+0000 7f76fd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76fd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76f815d1e0 0x7f76f81f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76fd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f81f85d0 0x7f76f81f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76fd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b5620 con 0x7f76dc006a10 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76fd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b54a0 con 0x7f76f81f85d0 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76fd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80b57a0 con 0x7f76f815d1e0 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38302/0 (socket says 172.21.15.59:38302) 2024-04-24T06:28:57.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 -- 172.21.15.59:0/2144640031 learned_addr learned my addr 172.21.15.59:0/2144640031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 -- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76f815d1e0 msgr2=0x7f76f81f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76f815d1e0 0x7f76f81f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 -- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f81f85d0 msgr2=0x7f76f81f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f81f85d0 0x7f76f81f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 -- 172.21.15.59:0/2144640031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f815c850 con 0x7f76dc006a10 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.883+0000 7f76f6800640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76e800ef40 tx=0x7f76e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.884+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8018020 con 0x7f76dc006a10 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.884+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e8014480 con 0x7f76dc006a10 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.884+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8014870 con 0x7f76dc006a10 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.884+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f815cb30 con 0x7f76dc006a10 2024-04-24T06:28:57.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.884+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f8159780 con 0x7f76dc006a10 2024-04-24T06:28:57.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.885+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f8068a80 con 0x7f76dc006a10 2024-04-24T06:28:57.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.886+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76e8005d10 con 0x7f76dc006a10 2024-04-24T06:28:57.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.887+0000 7f76ef400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 0x7f76d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:57.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.887+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f76e808c770 con 0x7f76dc006a10 2024-04-24T06:28:57.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.887+0000 7f76ede00640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 0x7f76d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:57.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.888+0000 7f76ede00640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 0x7f76d406ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76e0032000 tx=0x7f76e002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:57.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:57.892+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76e8022030 con 0x7f76dc006a10 2024-04-24T06:28:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.128+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f806f370 con 0x7f76dc006a10 2024-04-24T06:28:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.129+0000 7f76ef400640 1 -- 172.21.15.59:0/2144640031 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f76e8010040 con 0x7f76dc006a10 2024-04-24T06:28:58.130 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:58.130 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:28:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 msgr2=0x7f76d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 0x7f76d406ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76e0032000 tx=0x7f76e002c000 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 msgr2=0x7f76f81e5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76e800ef40 tx=0x7f76e800c490 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 shutdown_connections 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76d406c940 0x7f76d406ee00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76f81f85d0 0x7f76f81f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f76f815d1e0 0x7f76f81f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 --2- 172.21.15.59:0/2144640031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76dc006a10 0x7f76f81e5920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 >> 172.21.15.59:0/2144640031 conn(0x7f76f8067ea0 msgr2=0x7f76f815f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 shutdown_connections 2024-04-24T06:28:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.132+0000 7f76fd400640 1 -- 172.21.15.59:0/2144640031 wait complete. 2024-04-24T06:28:58.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:28:58.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.357+0000 7f6515400640 1 Processor -- start 2024-04-24T06:28:58.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.357+0000 7f6515400640 1 -- start start 2024-04-24T06:28:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6515400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f651015c3d0 0x7f65100cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6515400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6510160210 con 0x7f65100cd590 2024-04-24T06:28:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6515400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65101604f0 con 0x7f65100b4860 2024-04-24T06:28:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6515400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f65101607d0 con 0x7f651015c3d0 2024-04-24T06:28:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6507e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65100cd590 0x7f65100b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60108/0 (socket says 172.21.15.59:60108) 2024-04-24T06:28:58.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.359+0000 7f6507e00640 1 -- 172.21.15.59:0/2852453784 learned_addr learned my addr 172.21.15.59:0/2852453784 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650fe00640 1 --2- 172.21.15.59:0/2852453784 >> v2:172.21.15.66:3300/0 conn(0x7f651015c3d0 0x7f65100cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189887435 0 0) 0x7f6510160210 con 0x7f65100cd590 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f4003520 con 0x7f65100cd590 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2287455908 0 0) 0x7f64f4003520 con 0x7f65100cd590 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 >> v2:172.21.15.66:3300/0 conn(0x7f651015c3d0 msgr2=0x7f65100cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 --2- 172.21.15.59:0/2852453784 >> v2:172.21.15.66:3300/0 conn(0x7f651015c3d0 0x7f65100cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 >> v1:172.21.15.66:6789/0 conn(0x7f65100b4860 legacy=0x7f65100cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510160ab0 con 0x7f65100cd590 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f64f8001670 con 0x7f65100cd590 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 >> v1:172.21.15.59:6789/0 conn(0x7f65100cd590 legacy=0x7f65100b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.360+0000 7f650f400640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f40039b0 0x7f64f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650f400640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650f400640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 0x7f64f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65101607d0 con 0x7f64f40051c0 2024-04-24T06:28:58.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64f400c060 con 0x7f64f40039b0 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f64f400c260 con 0x7f64f4004800 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 0x7f64f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650fe00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f40039b0 0x7f64f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 -- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f4004800 msgr2=0x7f64f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6507e00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f4004800 0x7f64f4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 -- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f40039b0 msgr2=0x7f64f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f40039b0 0x7f64f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f6514a00640 1 -- 172.21.15.59:0/2852453784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f4003520 con 0x7f64f40051c0 2024-04-24T06:28:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.361+0000 7f650fe00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f40039b0 0x7f64f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:28:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.362+0000 7f6514a00640 1 -- 172.21.15.59:0/2852453784 reap_dead start 2024-04-24T06:28:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.362+0000 7f6514a00640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 0x7f64f400b9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6508001e50 tx=0x7f64f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f5be4f656dc2bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508001a70 con 0x7f64f40051c0 2024-04-24T06:28:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6508003970 con 0x7f64f40051c0 2024-04-24T06:28:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f650f400640 1 -- 172.21.15.59:0/2852453784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508003d60 con 0x7f64f40051c0 2024-04-24T06:28:58.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 -- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 msgr2=0x7f64f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 0x7f64f400b9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6508001e50 tx=0x7f64f4005830 comp rx=0 tx=0).stop 2024-04-24T06:28:58.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 -- 172.21.15.59:0/2852453784 shutdown_connections 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 --2- 172.21.15.59:0/2852453784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64f40051c0 0x7f64f400b9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 -- 172.21.15.59:0/2852453784 >> 172.21.15.59:0/2852453784 conn(0x7f65100b21a0 msgr2=0x7f65100b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.363+0000 7f6515400640 1 -- 172.21.15.59:0/2852453784 shutdown_connections 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.364+0000 7f6515400640 1 -- 172.21.15.59:0/2852453784 wait complete. 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.365+0000 7f6515400640 1 Processor -- start 2024-04-24T06:28:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.365+0000 7f6515400640 1 -- start start 2024-04-24T06:28:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f40051c0 0x7f6510156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 0x7f6510072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f651015c930 con 0x7f65100ceab0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f651015c7b0 con 0x7f6510159ce0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.366+0000 7f6515400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65100cd050 con 0x7f64f40051c0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38324/0 (socket says 172.21.15.59:38324) 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 -- 172.21.15.59:0/1037965672 learned_addr learned my addr 172.21.15.59:0/1037965672 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f650fe00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f40051c0 0x7f6510156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6514a00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 0x7f6510072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 -- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f40051c0 msgr2=0x7f6510156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f40051c0 0x7f6510156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 -- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 msgr2=0x7f6510072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 0x7f6510072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 -- 172.21.15.59:0/1037965672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510099980 con 0x7f65100ceab0 2024-04-24T06:28:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6514a00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 0x7f6510072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.367+0000 7f6507e00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64f8000f60 tx=0x7f64f800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.368+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8019070 con 0x7f65100ceab0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.368+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f80074e0 con 0x7f65100ceab0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.368+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8004a00 con 0x7f65100ceab0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.368+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651015a380 con 0x7f65100ceab0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.368+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65100ce2b0 con 0x7f65100ceab0 2024-04-24T06:28:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.369+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651004fd90 con 0x7f65100ceab0 2024-04-24T06:28:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.370+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f8004ba0 con 0x7f65100ceab0 2024-04-24T06:28:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.372+0000 7f650cc00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 0x7f64ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.372+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f64f808b600 con 0x7f65100ceab0 2024-04-24T06:28:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.372+0000 7f650fe00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 0x7f64ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.373+0000 7f650fe00640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 0x7f64ec06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65100af750 tx=0x7f6500008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.376+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64f8059990 con 0x7f65100ceab0 2024-04-24T06:28:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.586+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f651015a0c0 con 0x7f65100ceab0 2024-04-24T06:28:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.588+0000 7f650cc00640 1 -- 172.21.15.59:0/1037965672 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f64f8059290 con 0x7f65100ceab0 2024-04-24T06:28:58.589 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:28:58.589 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:28:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 msgr2=0x7f64ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 0x7f64ec06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65100af750 tx=0x7f6500008040 comp rx=0 tx=0).stop 2024-04-24T06:28:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 msgr2=0x7f65101597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f64f8000f60 tx=0x7f64f800dc70 comp rx=0 tx=0).stop 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 shutdown_connections 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64ec06c8f0 0x7f64ec06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6510159ce0 0x7f6510072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65100ceab0 0x7f65101597a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 --2- 172.21.15.59:0/1037965672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64f40051c0 0x7f6510156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 >> 172.21.15.59:0/1037965672 conn(0x7f65100b21a0 msgr2=0x7f65100b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 shutdown_connections 2024-04-24T06:28:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.591+0000 7f6515400640 1 -- 172.21.15.59:0/1037965672 wait complete. 2024-04-24T06:28:58.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:28:58.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:28:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.826+0000 7f6564e00640 1 Processor -- start 2024-04-24T06:28:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.827+0000 7f6564e00640 1 -- start start 2024-04-24T06:28:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.828+0000 7f6564e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6560161c50 0x7f65600d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.828+0000 7f6564e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65600d22e0 con 0x7f65600d0de0 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.828+0000 7f6564e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65600d2550 con 0x7f65600cfc50 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.828+0000 7f6564e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f65600d2ba0 con 0x7f6560161c50 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.829+0000 7f655fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65600d0de0 0x7f656006cb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60116/0 (socket says 172.21.15.59:60116) 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.829+0000 7f655fe00640 1 -- 172.21.15.59:0/1913480159 learned_addr learned my addr 172.21.15.59:0/1913480159 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.829+0000 7f655f400640 1 --2- 172.21.15.59:0/1913480159 >> v2:172.21.15.66:3300/0 conn(0x7f6560161c50 0x7f65600d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.829+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931163771 0 0) 0x7f65600d22e0 con 0x7f65600d0de0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.829+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6540003520 con 0x7f65600d0de0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1199623714 0 0) 0x7f6540003520 con 0x7f65600d0de0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 >> v2:172.21.15.66:3300/0 conn(0x7f6560161c50 msgr2=0x7f65600d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 --2- 172.21.15.59:0/1913480159 >> v2:172.21.15.66:3300/0 conn(0x7f6560161c50 0x7f65600d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 >> v1:172.21.15.66:6789/0 conn(0x7f65600cfc50 legacy=0x7f65600d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65600d2830 con 0x7f65600d0de0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6558001670 con 0x7f65600d0de0 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 >> v1:172.21.15.59:6789/0 conn(0x7f65600d0de0 legacy=0x7f656006cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 0x7f6540003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6540004800 0x7f6540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65400051c0 0x7f654000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d2ba0 con 0x7f6540003a00 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f654000c0c0 con 0x7f6540004800 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f654000c2c0 con 0x7f65400051c0 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.830+0000 7f655f400640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 0x7f6540003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655f400640 1 -- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65400051c0 msgr2=0x7f654000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655f400640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65400051c0 0x7f654000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655ea00640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6540004800 0x7f6540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655f400640 1 -- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6540004800 msgr2=0x7f6540004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655f400640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6540004800 0x7f6540004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655f400640 1 -- 172.21.15.59:0/1913480159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6540003520 con 0x7f6540003a00 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655fe00640 1 -- 172.21.15.59:0/1913480159 reap_dead start 2024-04-24T06:28:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.831+0000 7f655ea00640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6540004800 0x7f6540004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:58.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.832+0000 7f655f400640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 0x7f6540003e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6550009ac0 tx=0x7f6540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae1efb342f93626 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550036070 con 0x7f6540003a00 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6550031400 con 0x7f6540003a00 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f655e000640 1 -- 172.21.15.59:0/1913480159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550009690 con 0x7f6540003a00 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f6564e00640 1 -- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 msgr2=0x7f6540003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f6564e00640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 0x7f6540003e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6550009ac0 tx=0x7f6540005830 comp rx=0 tx=0).stop 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f6564e00640 1 -- 172.21.15.59:0/1913480159 shutdown_connections 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f6564e00640 1 --2- 172.21.15.59:0/1913480159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6540003a00 0x7f6540003e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.833+0000 7f6564e00640 1 -- 172.21.15.59:0/1913480159 >> 172.21.15.59:0/1913480159 conn(0x7f656008bf90 msgr2=0x7f65600d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.834+0000 7f6564e00640 1 -- 172.21.15.59:0/1913480159 shutdown_connections 2024-04-24T06:28:58.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.834+0000 7f6564e00640 1 -- 172.21.15.59:0/1913480159 wait complete. 2024-04-24T06:28:58.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.835+0000 7f6564e00640 1 Processor -- start 2024-04-24T06:28:58.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.835+0000 7f6564e00640 1 -- start start 2024-04-24T06:28:58.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6540003a00 0x7f65600c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65600ce260 0x7f656014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d0090 con 0x7f656014e180 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d0460 con 0x7f65600ce260 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.836+0000 7f6564e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65600d0210 con 0x7f6540003a00 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38328/0 (socket says 172.21.15.59:38328) 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 -- 172.21.15.59:0/3791943208 learned_addr learned my addr 172.21.15.59:0/3791943208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655f400640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6540003a00 0x7f65600c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655ea00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65600ce260 0x7f656014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 -- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6540003a00 msgr2=0x7f65600c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6540003a00 0x7f65600c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 -- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65600ce260 msgr2=0x7f656014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65600ce260 0x7f656014dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.837+0000 7f655fe00640 1 -- 172.21.15.59:0/3791943208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65600c86b0 con 0x7f656014e180 2024-04-24T06:28:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f655fe00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65600c8140 tx=0x7f655800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558004bf0 con 0x7f656014e180 2024-04-24T06:28:58.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6558004db0 con 0x7f656014e180 2024-04-24T06:28:58.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65600b81f0 con 0x7f656014e180 2024-04-24T06:28:58.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558007270 con 0x7f656014e180 2024-04-24T06:28:58.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.838+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600b87b0 con 0x7f656014e180 2024-04-24T06:28:58.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.839+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656004fd90 con 0x7f656014e180 2024-04-24T06:28:58.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.840+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6558007410 con 0x7f656014e180 2024-04-24T06:28:58.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.841+0000 7f6557400640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 0x7f653c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:58.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.842+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f655808b3b0 con 0x7f656014e180 2024-04-24T06:28:58.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.842+0000 7f655f400640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 0x7f653c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:58.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.842+0000 7f655f400640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 0x7f653c06ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6550033000 tx=0x7f6550008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:58.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:58.846+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65580597d0 con 0x7f656014e180 2024-04-24T06:28:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.040+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65600d3180 con 0x7f656014e180 2024-04-24T06:28:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.041+0000 7f6557400640 1 -- 172.21.15.59:0/3791943208 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f65580590d0 con 0x7f656014e180 2024-04-24T06:28:59.043 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:28:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 msgr2=0x7f653c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 0x7f653c06ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6550033000 tx=0x7f6550008030 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 msgr2=0x7f656015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65600c8140 tx=0x7f655800c960 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 shutdown_connections 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f653c06c920 0x7f653c06ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f656014e180 0x7f656015a570 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65600ce260 0x7f656014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 --2- 172.21.15.59:0/3791943208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6540003a00 0x7f65600c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 >> 172.21.15.59:0/3791943208 conn(0x7f656008bf90 msgr2=0x7f65600b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 shutdown_connections 2024-04-24T06:28:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.044+0000 7f6564e00640 1 -- 172.21.15.59:0/3791943208 wait complete. 2024-04-24T06:28:59.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:28:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.273+0000 7fa41ea00640 1 Processor -- start 2024-04-24T06:28:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.273+0000 7fa41ea00640 1 -- start start 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa41806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa4180d0cc0 0x7fa4180cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4180cfc20 con 0x7fa418071d70 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa4180cff00 con 0x7fa4180d0cc0 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4180d01e0 con 0x7fa418145e10 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa418071d70 0x7fa41815cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43610/0 (socket says 172.21.15.59:43610) 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa41ca00640 1 -- 172.21.15.59:0/2401244880 learned_addr learned my addr 172.21.15.59:0/2401244880 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa417e00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa41806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa417400640 1 --2- 172.21.15.59:0/2401244880 >> v2:172.21.15.66:3300/0 conn(0x7fa4180d0cc0 0x7fa4180cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.275+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711364840 0 0) 0x7fa4180cfc20 con 0x7fa418071d70 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3fc003520 con 0x7fa418071d70 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2669853825 0 0) 0x7fa3fc003520 con 0x7fa418071d70 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 msgr2=0x7fa41806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa41806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 >> v2:172.21.15.66:3300/0 conn(0x7fa4180d0cc0 msgr2=0x7fa4180cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 --2- 172.21.15.59:0/2401244880 >> v2:172.21.15.66:3300/0 conn(0x7fa4180d0cc0 0x7fa4180cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4180d04c0 con 0x7fa418071d70 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa4100019f0 con 0x7fa418071d70 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 >> v1:172.21.15.59:6789/0 conn(0x7fa418071d70 legacy=0x7fa41815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa3fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4180cff00 con 0x7fa3fc0039b0 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4180d01e0 con 0x7fa3fc004d40 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc00b9c0 con 0x7fa418145e10 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.276+0000 7fa417e00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417400640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa3fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417e00640 1 -- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 msgr2=0x7fa3fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417e00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa418145e10 0x7fa3fc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417e00640 1 -- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc004d40 msgr2=0x7fa3fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa41ca00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417e00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa417e00640 1 -- 172.21.15.59:0/2401244880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc003520 con 0x7fa3fc0039b0 2024-04-24T06:28:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.277+0000 7fa41ca00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc004d40 0x7fa3fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.278+0000 7fa417e00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa408009ac0 tx=0x7fa3fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956871a5225aad7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.278+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408036070 con 0x7fa3fc0039b0 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.278+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408031400 con 0x7fa3fc0039b0 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.278+0000 7fa416a00640 1 -- 172.21.15.59:0/2401244880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408009690 con 0x7fa3fc0039b0 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 -- 172.21.15.59:0/2401244880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3fc0039b0 msgr2=0x7fa3fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 --2- 172.21.15.59:0/2401244880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3fc0039b0 0x7fa3fc003dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa408009ac0 tx=0x7fa3fc005830 comp rx=0 tx=0).stop 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ca00640 1 -- 172.21.15.59:0/2401244880 reap_dead start 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 -- 172.21.15.59:0/2401244880 shutdown_connections 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 -- 172.21.15.59:0/2401244880 >> 172.21.15.59:0/2401244880 conn(0x7fa418067d90 msgr2=0x7fa41806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 -- 172.21.15.59:0/2401244880 shutdown_connections 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.279+0000 7fa41ea00640 1 -- 172.21.15.59:0/2401244880 wait complete. 2024-04-24T06:28:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.281+0000 7fa41ea00640 1 Processor -- start 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.281+0000 7fa41ea00640 1 -- start start 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc0039b0 0x7fa41813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4180b4ff0 0x7fa41815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4180cfa00 con 0x7fa41815e830 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4180cf880 con 0x7fa3fc0039b0 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4181461f0 con 0x7fa4180b4ff0 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36670/0 (socket says 172.21.15.59:36670) 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.282+0000 7fa41ca00640 1 -- 172.21.15.59:0/4053362655 learned_addr learned my addr 172.21.15.59:0/4053362655 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa417e00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc0039b0 0x7fa41813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa417400640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4180b4ff0 0x7fa41815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 -- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4180b4ff0 msgr2=0x7fa41815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4180b4ff0 0x7fa41815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 -- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc0039b0 msgr2=0x7fa41813e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc0039b0 0x7fa41813e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 -- 172.21.15.59:0/4053362655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4180b5520 con 0x7fa41815e830 2024-04-24T06:28:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ca00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4180b5920 tx=0x7fa41000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41000b8d0 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41000ba70 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41000be60 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.283+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41813d880 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.284+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41813de40 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.285+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41815ecb0 con 0x7fa41815e830 2024-04-24T06:28:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.286+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa41001b020 con 0x7fa41815e830 2024-04-24T06:28:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.287+0000 7fa40fe00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.287+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa41008bcb0 con 0x7fa41815e830 2024-04-24T06:28:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.287+0000 7fa417e00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.288+0000 7fa417e00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 0x7fa3f806ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa408033000 tx=0x7fa408008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.292+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa41005a040 con 0x7fa41815e830 2024-04-24T06:28:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.496+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa418068a80 con 0x7fa41815e830 2024-04-24T06:28:59.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.497+0000 7fa40fe00640 1 -- 172.21.15.59:0/4053362655 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa410059940 con 0x7fa41815e830 2024-04-24T06:28:59.498 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 msgr2=0x7fa3f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 0x7fa3f806ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa408033000 tx=0x7fa408008030 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 msgr2=0x7fa4180b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4180b5920 tx=0x7fa41000b0d0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 shutdown_connections 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41815e830 0x7fa4180b3760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4180b4ff0 0x7fa41815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.499+0000 7fa41ea00640 1 --2- 172.21.15.59:0/4053362655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3fc0039b0 0x7fa41813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.500+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 >> 172.21.15.59:0/4053362655 conn(0x7fa418067d90 msgr2=0x7fa41806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.500+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 shutdown_connections 2024-04-24T06:28:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.500+0000 7fa41ea00640 1 -- 172.21.15.59:0/4053362655 wait complete. 2024-04-24T06:28:59.519 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:28:59.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.733+0000 7f9395200640 1 Processor -- start 2024-04-24T06:28:59.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.733+0000 7f9395200640 1 -- start start 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f9395200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9390163770 0x7f93900d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f9395200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93900cff50 con 0x7f9390160180 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f9395200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93900d01c0 con 0x7f939015cb20 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f9395200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f939015cf00 con 0x7f9390163770 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f9387e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9390163770 0x7f93900d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f938e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9390160180 0x7f93900d2be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43616/0 (socket says 172.21.15.59:43616) 2024-04-24T06:28:59.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.736+0000 7f938e800640 1 -- 172.21.15.59:0/2918044047 learned_addr learned my addr 172.21.15.59:0/2918044047 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:28:59.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.737+0000 7f9387e00640 1 -- 172.21.15.59:0/2918044047 >> v1:172.21.15.66:6789/0 conn(0x7f939015cb20 legacy=0x7f93900cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:28:59.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.737+0000 7f938d400640 1 -- 172.21.15.59:0/2918044047 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401979319 0 0) 0x7f93900cff50 con 0x7f9390160180 2024-04-24T06:28:59.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.737+0000 7f9387e00640 1 -- 172.21.15.59:0/2918044047 >> v1:172.21.15.59:6789/0 conn(0x7f9390160180 legacy=0x7f93900d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.737+0000 7f9387e00640 1 -- 172.21.15.59:0/2918044047 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939015f6a0 con 0x7f9390163770 2024-04-24T06:28:59.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.738+0000 7f9387e00640 1 --2- 172.21.15.59:0/2918044047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9390163770 0x7f93900d1240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93780097a0 tx=0x7f937802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b21f6a6391ff18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f938d400640 1 -- 172.21.15.59:0/2918044047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378037070 con 0x7f9390163770 2024-04-24T06:28:59.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f938d400640 1 -- 172.21.15.59:0/2918044047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9378032440 con 0x7f9390163770 2024-04-24T06:28:59.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f938d400640 1 -- 172.21.15.59:0/2918044047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378032810 con 0x7f9390163770 2024-04-24T06:28:59.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 -- 172.21.15.59:0/2918044047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9390163770 msgr2=0x7f93900d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 --2- 172.21.15.59:0/2918044047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9390163770 0x7f93900d1240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93780097a0 tx=0x7f937802a1a0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 -- 172.21.15.59:0/2918044047 shutdown_connections 2024-04-24T06:28:59.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 --2- 172.21.15.59:0/2918044047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9390163770 0x7f93900d1240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 -- 172.21.15.59:0/2918044047 >> 172.21.15.59:0/2918044047 conn(0x7f939006cc70 msgr2=0x7f939006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:59.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.739+0000 7f9395200640 1 -- 172.21.15.59:0/2918044047 shutdown_connections 2024-04-24T06:28:59.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.740+0000 7f9395200640 1 -- 172.21.15.59:0/2918044047 wait complete. 2024-04-24T06:28:59.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.741+0000 7f9395200640 1 Processor -- start 2024-04-24T06:28:59.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.741+0000 7f9395200640 1 -- start start 2024-04-24T06:28:59.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 0x7f9390149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 0x7f9390161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93900d1900 con 0x7f939015cb20 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93900d1780 con 0x7f9390160180 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.742+0000 7f9395200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93900d1a80 con 0x7f9390163770 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36682/0 (socket says 172.21.15.59:36682) 2024-04-24T06:28:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 -- 172.21.15.59:0/2722775428 learned_addr learned my addr 172.21.15.59:0/2722775428 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:28:59.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938e800640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 0x7f9390161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f9387e00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 0x7f9390149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 -- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 msgr2=0x7f9390161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 0x7f9390161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 -- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 msgr2=0x7f9390149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 0x7f9390149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 -- 172.21.15.59:0/2722775428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93900cc530 con 0x7f939015cb20 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938e800640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 0x7f9390161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f9387e00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 0x7f9390149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.743+0000 7f938de00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f938000c450 tx=0x7f938000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.744+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800052d0 con 0x7f939015cb20 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.744+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9380005470 con 0x7f939015cb20 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.744+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380012870 con 0x7f939015cb20 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.744+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93900cc810 con 0x7f939015cb20 2024-04-24T06:28:59.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.744+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93900ccdd0 con 0x7f939015cb20 2024-04-24T06:28:59.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.745+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9390068a80 con 0x7f939015cb20 2024-04-24T06:28:59.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.746+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9380012a10 con 0x7f939015cb20 2024-04-24T06:28:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.747+0000 7f9386000640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 0x7f936c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:28:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.748+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f938008b6c0 con 0x7f939015cb20 2024-04-24T06:28:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.748+0000 7f9387e00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 0x7f936c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:28:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.748+0000 7f9387e00640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 0x7f936c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9378034000 tx=0x7f937802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:28:59.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.752+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9380059b80 con 0x7f939015cb20 2024-04-24T06:28:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.956+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f93900d29c0 con 0x7f939015cb20 2024-04-24T06:28:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.957+0000 7f9386000640 1 -- 172.21.15.59:0/2722775428 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9380059480 con 0x7f939015cb20 2024-04-24T06:28:59.958 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.959+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 msgr2=0x7f936c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.959+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 0x7f936c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9378034000 tx=0x7f937802e000 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.959+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 msgr2=0x7f93900cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.959+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f938000c450 tx=0x7f938000c910 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 shutdown_connections 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f936c06c8f0 0x7f936c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9390163770 0x7f9390161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9390160180 0x7f9390149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 --2- 172.21.15.59:0/2722775428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f939015cb20 0x7f93900cd5d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:28:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 >> 172.21.15.59:0/2722775428 conn(0x7f939006cc70 msgr2=0x7f939015a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:28:59.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 shutdown_connections 2024-04-24T06:28:59.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:28:59.960+0000 7f9395200640 1 -- 172.21.15.59:0/2722775428 wait complete. 2024-04-24T06:28:59.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:00.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.176+0000 7f81c3e00640 1 Processor -- start 2024-04-24T06:29:00.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.177+0000 7f81c3e00640 1 -- start start 2024-04-24T06:29:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 0x7f81c40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f81c40b4860 0x7f81c40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c4160210 con 0x7f81c40cd590 2024-04-24T06:29:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f81c40cd9f0 con 0x7f81c40b4860 2024-04-24T06:29:00.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f81c41604f0 con 0x7f81c415c3d0 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81c3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f81c40b4860 0x7f81c40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81ba000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f81c40cd590 0x7f81c40b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43630/0 (socket says 172.21.15.59:43630) 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.178+0000 7f81ba000640 1 -- 172.21.15.59:0/2250251222 learned_addr learned my addr 172.21.15.59:0/2250251222 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c2a00640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 0x7f81c40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c3400640 1 -- 172.21.15.59:0/2250251222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 msgr2=0x7f81c40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c3400640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 0x7f81c40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c3400640 1 -- 172.21.15.59:0/2250251222 >> v1:172.21.15.59:6789/0 conn(0x7f81c40cd590 legacy=0x7f81c40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c3400640 1 -- 172.21.15.59:0/2250251222 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41607d0 con 0x7f81c40b4860 2024-04-24T06:29:00.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.179+0000 7f81c2a00640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 0x7f81c40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:00.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.180+0000 7f81c3400640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40cb7d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009d90 tx=0x7f81bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad29dc547a17f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.180+0000 7f81c2000640 1 -- 172.21.15.59:0/2250251222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc007740 con 0x7f81c40b4860 2024-04-24T06:29:00.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.180+0000 7f81c2000640 1 -- 172.21.15.59:0/2250251222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc0078e0 con 0x7f81c40b4860 2024-04-24T06:29:00.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.180+0000 7f81c2000640 1 -- 172.21.15.59:0/2250251222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc007cb0 con 0x7f81c40b4860 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 -- 172.21.15.59:0/2250251222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 msgr2=0x7f81c40cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40cb7d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009d90 tx=0x7f81bc02a660 comp rx=0 tx=0).stop 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 -- 172.21.15.59:0/2250251222 shutdown_connections 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40cb7d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2250251222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c415c3d0 0x7f81c40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 -- 172.21.15.59:0/2250251222 >> 172.21.15.59:0/2250251222 conn(0x7f81c40b21a0 msgr2=0x7f81c40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 -- 172.21.15.59:0/2250251222 shutdown_connections 2024-04-24T06:29:00.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.181+0000 7f81c3e00640 1 -- 172.21.15.59:0/2250251222 wait complete. 2024-04-24T06:29:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.183+0000 7f81c3e00640 1 Processor -- start 2024-04-24T06:29:00.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.184+0000 7f81c3e00640 1 -- start start 2024-04-24T06:29:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c40cd590 0x7f81c415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c415c3d0 0x7f81c415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415c7b0 con 0x7f81c40b4860 2024-04-24T06:29:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4160d90 con 0x7f81c415c3d0 2024-04-24T06:29:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f81c415c930 con 0x7f81c40cd590 2024-04-24T06:29:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36710/0 (socket says 172.21.15.59:36710) 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.185+0000 7f81c2a00640 1 -- 172.21.15.59:0/2507956597 learned_addr learned my addr 172.21.15.59:0/2507956597 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81ba000640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c40cd590 0x7f81c415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 -- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c40cd590 msgr2=0x7f81c415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c40cd590 0x7f81c415b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 -- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c415c3d0 msgr2=0x7f81c415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c415c3d0 0x7f81c415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 -- 172.21.15.59:0/2507956597 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c40c86b0 con 0x7f81c40b4860 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c2a00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c7f0 tx=0x7f81b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4015070 con 0x7f81c40b4860 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b4007820 con 0x7f81c40b4860 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c40c8990 con 0x7f81c40b4860 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.186+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4007bd0 con 0x7f81c40b4860 2024-04-24T06:29:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.187+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c40c8f50 con 0x7f81c40b4860 2024-04-24T06:29:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.188+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c40adf80 con 0x7f81c40b4860 2024-04-24T06:29:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.189+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81b4018070 con 0x7f81c40b4860 2024-04-24T06:29:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.189+0000 7f81bb400640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.190+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81b408bd60 con 0x7f81c40b4860 2024-04-24T06:29:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.190+0000 7f81ba000640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.191+0000 7f81ba000640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 0x7f81a006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b3cf0 tx=0x7f81ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.194+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81b405a0f0 con 0x7f81c40b4860 2024-04-24T06:29:00.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.436+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c404fd90 con 0x7f81c40b4860 2024-04-24T06:29:00.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.438+0000 7f81bb400640 1 -- 172.21.15.59:0/2507956597 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f81b40599f0 con 0x7f81c40b4860 2024-04-24T06:29:00.439 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:00.439 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:00.439 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.440+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 msgr2=0x7f81a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.440+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 0x7f81a006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b3cf0 tx=0x7f81ac009040 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 msgr2=0x7f81c40ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c7f0 tx=0x7f81b400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 shutdown_connections 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f81a006c8f0 0x7f81a006edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81c415c3d0 0x7f81c415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f81c40cd590 0x7f81c415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 --2- 172.21.15.59:0/2507956597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81c40b4860 0x7f81c40ad9e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 >> 172.21.15.59:0/2507956597 conn(0x7f81c40b21a0 msgr2=0x7f81c40b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 shutdown_connections 2024-04-24T06:29:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.441+0000 7f81c3e00640 1 -- 172.21.15.59:0/2507956597 wait complete. 2024-04-24T06:29:00.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:00.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.667+0000 7ff29fe00640 1 Processor -- start 2024-04-24T06:29:00.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.667+0000 7ff29fe00640 1 -- start start 2024-04-24T06:29:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a0162fb0 0x7ff2a01673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2a00c42b0 0x7ff2a00c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a0164420 con 0x7ff2a00d1a60 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a0163e60 con 0x7ff2a00c42b0 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a0164140 con 0x7ff2a0162fb0 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2a00c42b0 0x7ff2a00c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2a00c42b0 0x7ff2a00c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36720/0 (socket says 172.21.15.59:36720) 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29f400640 1 -- 172.21.15.59:0/1099060110 learned_addr learned my addr 172.21.15.59:0/1099060110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.669+0000 7ff29ea00640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a0162fb0 0x7ff2a01673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.670+0000 7ff29f400640 1 -- 172.21.15.59:0/1099060110 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a0162fb0 msgr2=0x7ff2a01673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.670+0000 7ff29f400640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a0162fb0 0x7ff2a01673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.670+0000 7ff29f400640 1 -- 172.21.15.59:0/1099060110 >> v1:172.21.15.66:6789/0 conn(0x7ff2a00d1a60 legacy=0x7ff2a0163390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:00.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.670+0000 7ff29f400640 1 -- 172.21.15.59:0/1099060110 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a0166bf0 con 0x7ff2a00c42b0 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.671+0000 7ff29f400640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a00c42b0 0x7ff2a00c4760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff298009870 tx=0x7ff29802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abed7894375b5453 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.671+0000 7ff29d600640 1 -- 172.21.15.59:0/1099060110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298037070 con 0x7ff2a00c42b0 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.671+0000 7ff29d600640 1 -- 172.21.15.59:0/1099060110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff298032440 con 0x7ff2a00c42b0 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.671+0000 7ff29d600640 1 -- 172.21.15.59:0/1099060110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298032810 con 0x7ff2a00c42b0 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 -- 172.21.15.59:0/1099060110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a00c42b0 msgr2=0x7ff2a00c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a00c42b0 0x7ff2a00c4760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff298009870 tx=0x7ff29802a180 comp rx=0 tx=0).stop 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 -- 172.21.15.59:0/1099060110 shutdown_connections 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a00c42b0 0x7ff2a00c4760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 --2- 172.21.15.59:0/1099060110 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a0162fb0 0x7ff2a01673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 -- 172.21.15.59:0/1099060110 >> 172.21.15.59:0/1099060110 conn(0x7ff2a009aa00 msgr2=0x7ff2a00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 -- 172.21.15.59:0/1099060110 shutdown_connections 2024-04-24T06:29:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.672+0000 7ff29fe00640 1 -- 172.21.15.59:0/1099060110 wait complete. 2024-04-24T06:29:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.673+0000 7ff29fe00640 1 Processor -- start 2024-04-24T06:29:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.674+0000 7ff29fe00640 1 -- start start 2024-04-24T06:29:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a00c42b0 0x7ff2a015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a00d1a60 0x7ff2a015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0163aa0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0167930 con 0x7ff2a00d1a60 2024-04-24T06:29:00.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a0163c20 con 0x7ff2a00c42b0 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36734/0 (socket says 172.21.15.59:36734) 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 -- 172.21.15.59:0/2039645935 learned_addr learned my addr 172.21.15.59:0/2039645935 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29e000640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a00d1a60 0x7ff2a015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29ea00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a00c42b0 0x7ff2a015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 -- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a00c42b0 msgr2=0x7ff2a015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a00c42b0 0x7ff2a015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 -- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a00d1a60 msgr2=0x7ff2a015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a00d1a60 0x7ff2a015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.675+0000 7ff29f400640 1 -- 172.21.15.59:0/2039645935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a015f9e0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff29f400640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff29802a1c0 tx=0x7ff2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298041020 con 0x7ff2a0162fb0 2024-04-24T06:29:00.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2980306c0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298037040 con 0x7ff2a0162fb0 2024-04-24T06:29:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a0165900 con 0x7ff2a0162fb0 2024-04-24T06:29:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.676+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a0165ba0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.677+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a004fd90 con 0x7ff2a0162fb0 2024-04-24T06:29:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.678+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff298044020 con 0x7ff2a0162fb0 2024-04-24T06:29:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.679+0000 7ff296a00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.679+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2980ac340 con 0x7ff2a0162fb0 2024-04-24T06:29:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.680+0000 7ff29ea00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:00.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.680+0000 7ff29ea00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff290009a60 tx=0x7ff290003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:00.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.684+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff29804b030 con 0x7ff2a0162fb0 2024-04-24T06:29:00.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.897+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2a006e8a0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.899+0000 7ff296a00640 1 -- 172.21.15.59:0/2039645935 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7ff2a015f9e0 con 0x7ff2a0162fb0 2024-04-24T06:29:00.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:00.901 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:00.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 msgr2=0x7ff27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 0x7ff27c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff290009a60 tx=0x7ff290003990 comp rx=0 tx=0).stop 2024-04-24T06:29:00.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 msgr2=0x7ff2a01660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:00.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff29802a1c0 tx=0x7ff2980046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 shutdown_connections 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27c06c8f0 0x7ff27c06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0162fb0 0x7ff2a01660c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2a00d1a60 0x7ff2a015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 --2- 172.21.15.59:0/2039645935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2a00c42b0 0x7ff2a015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 >> 172.21.15.59:0/2039645935 conn(0x7ff2a009aa00 msgr2=0x7ff2a0163390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 shutdown_connections 2024-04-24T06:29:00.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:00.902+0000 7ff29fe00640 1 -- 172.21.15.59:0/2039645935 wait complete. 2024-04-24T06:29:00.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.128+0000 7f668f000640 1 Processor -- start 2024-04-24T06:29:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.129+0000 7f668f000640 1 -- start start 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.130+0000 7f668f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66880d0370 0x7f66880cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.130+0000 7f668f000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f668815e200 0x7f668815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.130+0000 7f668f000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66880cd9f0 con 0x7f66880d0b70 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.130+0000 7f668f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f668809b6f0 con 0x7f66880d0370 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.130+0000 7f668f000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f66880cd790 con 0x7f668815e200 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66880d0370 0x7f66880cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66880d0370 0x7f66880cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36750/0 (socket says 172.21.15.59:36750) 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 -- 172.21.15.59:0/903350138 learned_addr learned my addr 172.21.15.59:0/903350138 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f668d000640 1 --2- 172.21.15.59:0/903350138 >> v2:172.21.15.66:3300/0 conn(0x7f668815e200 0x7f668815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 -- 172.21.15.59:0/903350138 >> v2:172.21.15.66:3300/0 conn(0x7f668815e200 msgr2=0x7f668815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 --2- 172.21.15.59:0/903350138 >> v2:172.21.15.66:3300/0 conn(0x7f668815e200 0x7f668815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 -- 172.21.15.59:0/903350138 >> v1:172.21.15.66:6789/0 conn(0x7f66880d0b70 legacy=0x7f66880d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.131+0000 7f6687400640 1 -- 172.21.15.59:0/903350138 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66880cdcb0 con 0x7f66880d0370 2024-04-24T06:29:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.132+0000 7f6687400640 1 --2- 172.21.15.59:0/903350138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66880d0370 0x7f66880cd250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be86d7ab1359eab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f6686a00640 1 -- 172.21.15.59:0/903350138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670036070 con 0x7f66880d0370 2024-04-24T06:29:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f6686a00640 1 -- 172.21.15.59:0/903350138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6670031600 con 0x7f66880d0370 2024-04-24T06:29:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f6686a00640 1 -- 172.21.15.59:0/903350138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700319d0 con 0x7f66880d0370 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 -- 172.21.15.59:0/903350138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66880d0370 msgr2=0x7f66880cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 --2- 172.21.15.59:0/903350138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66880d0370 0x7f66880cd250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667002a6d0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 -- 172.21.15.59:0/903350138 shutdown_connections 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 --2- 172.21.15.59:0/903350138 >> v2:172.21.15.66:3300/0 conn(0x7f668815e200 0x7f668815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 --2- 172.21.15.59:0/903350138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66880d0370 0x7f66880cd250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.133+0000 7f668f000640 1 -- 172.21.15.59:0/903350138 >> 172.21.15.59:0/903350138 conn(0x7f66880b4080 msgr2=0x7f66880b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.134+0000 7f668f000640 1 -- 172.21.15.59:0/903350138 shutdown_connections 2024-04-24T06:29:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.134+0000 7f668f000640 1 -- 172.21.15.59:0/903350138 wait complete. 2024-04-24T06:29:01.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.136+0000 7f668f000640 1 Processor -- start 2024-04-24T06:29:01.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.136+0000 7f668f000640 1 -- start start 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66880d0370 0x7f66881f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66880d0b70 0x7f6688150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f668815eb80 con 0x7f668815e200 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66880ce270 con 0x7f66880d0370 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668f000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f668815ed00 con 0x7f66880d0b70 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36754/0 (socket says 172.21.15.59:36754) 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.137+0000 7f668d000640 1 -- 172.21.15.59:0/3365551592 learned_addr learned my addr 172.21.15.59:0/3365551592 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f6687400640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66880d0b70 0x7f6688150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f6687e00640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66880d0370 0x7f66881f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 -- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66880d0b70 msgr2=0x7f6688150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66880d0b70 0x7f6688150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 -- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66880d0370 msgr2=0x7f66881f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66880d0370 0x7f66881f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 -- 172.21.15.59:0/3365551592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66880c60c0 con 0x7f668815e200 2024-04-24T06:29:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668d000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f668000d5d0 tx=0x7f668000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800125f0 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6680019070 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800129f0 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.138+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66880c63a0 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.139+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66880c6960 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.140+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668815e020 con 0x7f668815e200 2024-04-24T06:29:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.141+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6680012b90 con 0x7f668815e200 2024-04-24T06:29:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.142+0000 7f667fe00640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:01.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.142+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f668008bfd0 con 0x7f668815e200 2024-04-24T06:29:01.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.142+0000 7f6687e00640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:01.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.142+0000 7f6687e00640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 0x7f666406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6678001640 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.146+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f668815e020 con 0x7f668815e200 2024-04-24T06:29:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.395+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6688068a80 con 0x7f668815e200 2024-04-24T06:29:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.396+0000 7f667fe00640 1 -- 172.21.15.59:0/3365551592 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f6680060030 con 0x7f668815e200 2024-04-24T06:29:01.397 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:01.397 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:01.398 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 msgr2=0x7f666406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 0x7f666406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6678001640 tx=0x7f6678009040 comp rx=0 tx=0).stop 2024-04-24T06:29:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 msgr2=0x7f66881e94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f668000d5d0 tx=0x7f668000da90 comp rx=0 tx=0).stop 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 shutdown_connections 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f666406c8f0 0x7f666406edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f668815e200 0x7f66881e94e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66880d0b70 0x7f6688150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 --2- 172.21.15.59:0/3365551592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66880d0370 0x7f66881f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 >> 172.21.15.59:0/3365551592 conn(0x7f66880b4080 msgr2=0x7f668815f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 shutdown_connections 2024-04-24T06:29:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:01.399+0000 7f668f000640 1 -- 172.21.15.59:0/3365551592 wait complete. 2024-04-24T06:29:01.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:28:44.183608+0000', 'modified': '2024-04-24T06:28:47.252737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/1796814529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 1796814529}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 1796814529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-24T06:29:01.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:29:01.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:29:02.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.042+0000 7f0590400640 1 Processor -- start 2024-04-24T06:29:02.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.042+0000 7f0590400640 1 -- start start 2024-04-24T06:29:02.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.044+0000 7f0590400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f058815e9c0 0x7f05880d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.044+0000 7f0590400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05880d1800 0x7f05880d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.044+0000 7f0590400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05880cf3c0 con 0x7f05880d2d70 2024-04-24T06:29:02.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.044+0000 7f0590400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f05880d3710 con 0x7f05880d1800 2024-04-24T06:29:02.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.044+0000 7f0590400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05880cf640 con 0x7f058815e9c0 2024-04-24T06:29:02.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.045+0000 7f058d000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f05880d2d70 0x7f05880d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:40734/0 (socket says 172.21.15.59:40734) 2024-04-24T06:29:02.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.045+0000 7f058d000640 1 -- 172.21.15.59:0/3383574075 learned_addr learned my addr 172.21.15.59:0/3383574075 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:02.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.045+0000 7f058da00640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f058815e9c0 0x7f05880d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.045+0000 7f058e400640 1 --2- 172.21.15.59:0/3383574075 >> v2:172.21.15.66:3300/0 conn(0x7f05880d1800 0x7f05880d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.046+0000 7f058e400640 1 -- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f058815e9c0 msgr2=0x7f05880d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.046+0000 7f058e400640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f058815e9c0 0x7f05880d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.046+0000 7f058e400640 1 -- 172.21.15.59:0/3383574075 >> v1:172.21.15.66:6789/0 conn(0x7f05880d2d70 legacy=0x7f05880d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.046+0000 7f058e400640 1 -- 172.21.15.59:0/3383574075 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05880cf900 con 0x7f05880d1800 2024-04-24T06:29:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.047+0000 7f058e400640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d1800 0x7f05880d31d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3676d1c70bb233b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.047+0000 7f0587e00640 1 -- 172.21.15.59:0/3383574075 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c037070 con 0x7f05880d1800 2024-04-24T06:29:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.047+0000 7f0587e00640 1 -- 172.21.15.59:0/3383574075 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c02ae50 con 0x7f05880d1800 2024-04-24T06:29:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.047+0000 7f0587e00640 1 -- 172.21.15.59:0/3383574075 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c032660 con 0x7f05880d1800 2024-04-24T06:29:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.048+0000 7f0590400640 1 -- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d1800 msgr2=0x7f05880d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.048+0000 7f0590400640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d1800 0x7f05880d31d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f057c0097a0 tx=0x7f057c02a100 comp rx=0 tx=0).stop 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 -- 172.21.15.59:0/3383574075 shutdown_connections 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d1800 0x7f05880d31d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 --2- 172.21.15.59:0/3383574075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f058815e9c0 0x7f05880d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 -- 172.21.15.59:0/3383574075 >> 172.21.15.59:0/3383574075 conn(0x7f058808bf90 msgr2=0x7f05880d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 -- 172.21.15.59:0/3383574075 shutdown_connections 2024-04-24T06:29:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.049+0000 7f0590400640 1 -- 172.21.15.59:0/3383574075 wait complete. 2024-04-24T06:29:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.051+0000 7f0590400640 1 Processor -- start 2024-04-24T06:29:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.051+0000 7f0590400640 1 -- start start 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05880d1800 0x7f058815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d2d70 0x7f058815f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f058806cb90 con 0x7f058815e9c0 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f058815eda0 con 0x7f05880d2d70 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.052+0000 7f0590400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f058806cd10 con 0x7f05880d1800 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36766/0 (socket says 172.21.15.59:36766) 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 -- 172.21.15.59:0/1765015234 learned_addr learned my addr 172.21.15.59:0/1765015234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058d000640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d2d70 0x7f058815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058da00640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05880d1800 0x7f058815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 -- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05880d1800 msgr2=0x7f058815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05880d1800 0x7f058815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 -- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d2d70 msgr2=0x7f058815f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d2d70 0x7f058815f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.053+0000 7f058e400640 1 -- 172.21.15.59:0/1765015234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05880c8a90 con 0x7f058815e9c0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f058e400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f057c035040 tx=0x7f057c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f0585600640 1 -- 172.21.15.59:0/1765015234 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c041070 con 0x7f058815e9c0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f0585600640 1 -- 172.21.15.59:0/1765015234 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c03c460 con 0x7f058815e9c0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f0585600640 1 -- 172.21.15.59:0/1765015234 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c037040 con 0x7f058815e9c0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05880c8d70 con 0x7f058815e9c0 2024-04-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.054+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05880c9330 con 0x7f058815e9c0 2024-04-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.057+0000 7f0585600640 1 -- 172.21.15.59:0/1765015234 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f057c044070 con 0x7f058815e9c0 2024-04-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.057+0000 7f0585600640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 0x7f056806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.058+0000 7f0585600640 1 -- 172.21.15.59:0/1765015234 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f057c0ac220 con 0x7f058815e9c0 2024-04-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.058+0000 7f058da00640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 0x7f056806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.058+0000 7f0593255740 1 Processor -- start 2024-04-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.058+0000 7f0593255740 1 -- start start 2024-04-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.058+0000 7f058da00640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 0x7f056806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f058815b4f0 tx=0x7f058000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5803a0 0x55a19a57d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a19a4e3b10 con 0x55a19a5a6430 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a19a48a2b0 con 0x55a19a5a8e40 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.059+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a19a4dfad0 con 0x55a19a5803a0 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36770/0 (socket says 172.21.15.59:36770) 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 -- 172.21.15.59:0/103079582 learned_addr learned my addr 172.21.15.59:0/103079582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058da00640 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058e400640 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5803a0 0x55a19a57d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 -- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5803a0 msgr2=0x55a19a57d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5803a0 0x55a19a57d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 -- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 msgr2=0x55a19a5a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 -- 172.21.15.59:0/103079582 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19a1f62d0 con 0x55a19a5a6430 2024-04-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.060+0000 7f058d000640 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05740015d0 tx=0x7f0574001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf5c4c502dee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f057a000640 1 -- 172.21.15.59:0/103079582 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574019070 con 0x55a19a5a6430 2024-04-24T06:29:02.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f057a000640 1 -- 172.21.15.59:0/103079582 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574005900 con 0x55a19a5a6430 2024-04-24T06:29:02.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f057a000640 1 -- 172.21.15.59:0/103079582 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574005cd0 con 0x55a19a5a6430 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 -- 172.21.15.59:0/103079582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 msgr2=0x55a19a579140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05740015d0 tx=0x7f0574001a90 comp rx=0 tx=0).stop 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 -- 172.21.15.59:0/103079582 shutdown_connections 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5803a0 0x55a19a57d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a579140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 --2- 172.21.15.59:0/103079582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 -- 172.21.15.59:0/103079582 >> 172.21.15.59:0/103079582 conn(0x55a19a57b3b0 msgr2=0x55a19a5c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.061+0000 7f0593255740 1 -- 172.21.15.59:0/103079582 shutdown_connections 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.062+0000 7f0593255740 1 -- 172.21.15.59:0/103079582 wait complete. 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.062+0000 7f0593255740 1 -- start start 2024-04-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.062+0000 7f0593255740 1 Processor -- start 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5b69d0 0x55a19a5a9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a19a5a6150 con 0x55a19a5a6430 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a19a49d590 con 0x55a19a5a8e40 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.063+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a19a5c6eb0 con 0x55a19a5b69d0 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f0593255740 10 client.0 _ll_register_callbacks cb 0x55a19a4803a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36784/0 (socket says 172.21.15.59:36784) 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 -- 172.21.15.59:0/408805383 learned_addr learned my addr 172.21.15.59:0/408805383 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058d000640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058e400640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5b69d0 0x55a19a5a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 -- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5b69d0 msgr2=0x55a19a5a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5b69d0 0x55a19a5a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 -- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 msgr2=0x55a19a5a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 -- 172.21.15.59:0/408805383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19a2c06b0 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f058da00640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0580016f90 tx=0x7f0580004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e39ae64a0ec33fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.064+0000 7f056ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05800225d0 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0580022770 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574005900 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f0580003990 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 10 client.4629 fetch_fsmap learned FSMap version 8 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a19a5a50e0 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574005900 con 0x55a19a5a6430 2024-04-24T06:29:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f0580033080 con 0x55a19a5a6430 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 10 client.4629 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 20 client.4629 populate_metadata read hostname 'smithi059' 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5ac1c0 0x55a19a5eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 4 client.4629 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.065+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 --> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19a4f1530 con 0x55a19a5ac1c0 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.066+0000 7f058d000640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5ac1c0 0x55a19a5eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.067+0000 7f058d000640 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5ac1c0 0x55a19a5eb470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550860efde083332 server_cookie=d5d61a09df3a10b9 in_seq=0 out_seq=0 2024-04-24T06:29:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.067+0000 7f056ea00640 10 client.4629 ms_handle_connect on v2:172.21.15.66:6834/1796814529 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.069+0000 7f056ea00640 1 -- 172.21.15.59:0/408805383 <== mds.0 v2:172.21.15.66:6834/1796814529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a19a4f1530 con 0x55a19a5ac1c0 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.069+0000 7f056ea00640 10 client.4629 handle_command_reply: tid=0 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.069+0000 7f0593255740 1 client.4629 shutdown 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 msgr2=0x55a19a5a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0580016f90 tx=0x7f0580004ab0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 shutdown_connections 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5b69d0 0x55a19a5a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a8e40 0x55a19a5a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5ac1c0 0x55a19a5eb470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 --2- 172.21.15.59:0/408805383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a6430 0x55a19a5a0560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f056ea00640 0 client.4629 ms_handle_reset on v2:172.21.15.66:6834/1796814529 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 >> 172.21.15.59:0/408805383 conn(0x55a19a5803a0 msgr2=0x55a19a57a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 shutdown_connections 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 1 -- 172.21.15.59:0/408805383 wait complete. 2024-04-24T06:29:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.070+0000 7f0593255740 20 client.4629 trim_cache size 0 max 16384 2024-04-24T06:29:02.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.090+0000 7f0593255740 1 Processor -- start 2024-04-24T06:29:02.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.090+0000 7f0593255740 1 -- start start 2024-04-24T06:29:02.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a6430 0x55a19a579e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a8e40 0x55a19a579160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0588070ba0 con 0x55a19a5b69d0 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f058806ff00 con 0x55a19a5a6430 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0588070550 con 0x55a19a5a8e40 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f058e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f058e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36798/0 (socket says 172.21.15.59:36798) 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f058e400640 1 -- 172.21.15.59:0/722265451 learned_addr learned my addr 172.21.15.59:0/722265451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f058da00640 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a6430 0x55a19a579e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.091+0000 7f058d000640 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a8e40 0x55a19a579160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 -- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a8e40 msgr2=0x55a19a579160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a8e40 0x55a19a579160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 -- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a6430 msgr2=0x55a19a579e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a6430 0x55a19a579e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 -- 172.21.15.59:0/722265451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19a5801d0 con 0x55a19a5b69d0 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f058e400640 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f057c0aa650 tx=0x7f057c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e686852d7af14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f0579600640 1 -- 172.21.15.59:0/722265451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c04b040 con 0x55a19a5b69d0 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f0579600640 1 -- 172.21.15.59:0/722265451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c02ae50 con 0x55a19a5b69d0 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.092+0000 7f0579600640 1 -- 172.21.15.59:0/722265451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c079810 con 0x55a19a5b69d0 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 -- 172.21.15.59:0/722265451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 msgr2=0x55a19a5c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f057c0aa650 tx=0x7f057c02a680 comp rx=0 tx=0).stop 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 -- 172.21.15.59:0/722265451 shutdown_connections 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5b69d0 0x55a19a5c5940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a8e40 0x55a19a579160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 --2- 172.21.15.59:0/722265451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5a6430 0x55a19a579e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 -- 172.21.15.59:0/722265451 >> 172.21.15.59:0/722265451 conn(0x55a19a5803a0 msgr2=0x55a19a574750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 -- 172.21.15.59:0/722265451 shutdown_connections 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.093+0000 7f0593255740 1 -- 172.21.15.59:0/722265451 wait complete. 2024-04-24T06:29:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.094+0000 7f0593255740 1 -- start start 2024-04-24T06:29:02.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.094+0000 7f0593255740 1 Processor -- start 2024-04-24T06:29:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a6430 0x55a19a5aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5b69d0 0x55a19a5ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a19a5b6e50 con 0x55a19a5a8e40 2024-04-24T06:29:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a19a5aa1b0 con 0x55a19a5b69d0 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a19a5b7bf0 con 0x55a19a5a6430 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f0593255740 10 client.0 _ll_register_callbacks cb 0x7f0580016c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36810/0 (socket says 172.21.15.59:36810) 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058d000640 1 -- 172.21.15.59:0/123675014 learned_addr learned my addr 172.21.15.59:0/123675014 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058e400640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5b69d0 0x55a19a5ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058da00640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a6430 0x55a19a5aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058d000640 1 -- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a6430 msgr2=0x55a19a5aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.095+0000 7f058d000640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a6430 0x55a19a5aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f058d000640 1 -- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5b69d0 msgr2=0x55a19a5ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f058d000640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5b69d0 0x55a19a5ecda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f058d000640 1 -- 172.21.15.59:0/123675014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19a5b6270 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f058d000640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f057400e530 tx=0x7f0574000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16433f9ed936bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f056ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057402a060 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f05740074e0 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c02ae50 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f0574030050 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f0593255740 10 client.4635 fetch_fsmap learned FSMap version 8 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.096+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a19a5ed800 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c02ae50 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f0574033020 con 0x55a19a5a8e40 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 20 client.4635 populate_metadata read hostname 'smithi059' 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5c5000 0x55a19a5c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 4 client.4635 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:29:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 --> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] -- command(tid 0: {"prefix": "status"}) -- 0x55a19a4eb3a0 con 0x55a19a5c5000 2024-04-24T06:29:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.097+0000 7f058da00640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5c5000 0x55a19a5c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.098+0000 7f058da00640 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5c5000 0x55a19a5c53e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e20f79b545586c server_cookie=be4a97192ce48a41 in_seq=0 out_seq=0 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.098+0000 7f056ea00640 10 client.4635 ms_handle_connect on v2:172.21.15.66:6834/1796814529 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f056ea00640 1 -- 172.21.15.59:0/123675014 <== mds.0 v2:172.21.15.66:6834/1796814529 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55a19a4eb3a0 con 0x55a19a5c5000 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f056ea00640 10 client.4635 handle_command_reply: tid=0 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 client.4635 shutdown 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 msgr2=0x55a19a64a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f057400e530 tx=0x7f0574000bc0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 shutdown_connections 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a19a5b69d0 0x55a19a5ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:6834/1796814529,v1:172.21.15.66:6835/1796814529] conn(0x55a19a5c5000 0x55a19a5c53e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a19a5a8e40 0x55a19a64a140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 --2- 172.21.15.59:0/123675014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a19a5a6430 0x55a19a5aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f056ea00640 0 client.4635 ms_handle_reset on v2:172.21.15.66:6834/1796814529 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 >> 172.21.15.59:0/123675014 conn(0x55a19a5803a0 msgr2=0x55a19a579160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 shutdown_connections 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.099+0000 7f0593255740 1 -- 172.21.15.59:0/123675014 wait complete. 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0593255740 20 client.4635 trim_cache size 0 max 16384 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4273, 2024-04-24T06:29:02.103 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 17.391562159999999, 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 8, 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 20, 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 20, 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 22.532447296000001 2024-04-24T06:29:02.104 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 msgr2=0x7f056806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 0x7f056806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f058815b4f0 tx=0x7f058000b040 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 msgr2=0x7f05880b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f057c035040 tx=0x7f057c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 shutdown_connections 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f056806c8f0 0x7f056806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f058815e9c0 0x7f05880b87d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05880d2d70 0x7f058815f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 --2- 172.21.15.59:0/1765015234 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05880d1800 0x7f058815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.100+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 >> 172.21.15.59:0/1765015234 conn(0x7f058808bf90 msgr2=0x7f05880b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.101+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 shutdown_connections 2024-04-24T06:29:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.101+0000 7f0590400640 1 -- 172.21.15.59:0/1765015234 wait complete. 2024-04-24T06:29:02.120 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T06:29:02.131 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T06:29:02.131 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T06:29:02.131 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T06:29:02.131 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T06:29:02.231 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T06:29:02.232 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-24T06:29:02.232 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T06:29:02.232 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T06:29:02.232 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:02.232 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-24T06:29:02.232 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T06:29:02.232 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:02.232 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-24T06:29:02.232 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T06:29:02.233 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:02.233 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-24T06:29:02.233 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T06:29:02.233 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:02.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:02.233 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:29:02.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:02.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-24T06:29:02.391 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2024-04-24T06:29:02.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:29:02.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:02.396 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-24T06:29:02.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:02.460 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-04-24T06:29:02.563 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-04-24T06:29:02.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:29:02.568 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T06:29:02.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:29:02.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.788+0000 7f1fda800640 1 Processor -- start 2024-04-24T06:29:02.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.789+0000 7f1fda800640 1 -- start start 2024-04-24T06:29:02.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.790+0000 7f1fda800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1fd40cd050 0x7f1fd406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.790+0000 7f1fda800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1fd40b3080 0x7f1fd40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.790+0000 7f1fda800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd415f160 con 0x7f1fd415ece0 2024-04-24T06:29:02.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.790+0000 7f1fda800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd40ced20 con 0x7f1fd40b3080 2024-04-24T06:29:02.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.790+0000 7f1fda800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd415fcd0 con 0x7f1fd40cd050 2024-04-24T06:29:02.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd8a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1fd415ece0 0x7f1fd40cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43638/0 (socket says 172.21.15.59:43638) 2024-04-24T06:29:02.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd8a00640 1 -- 172.21.15.59:0/3161133826 learned_addr learned my addr 172.21.15.59:0/3161133826 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:02.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/3161133826 >> v2:172.21.15.59:3300/0 conn(0x7f1fd40b3080 0x7f1fd40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9400640 1 --2- 172.21.15.59:0/3161133826 >> v2:172.21.15.66:3300/0 conn(0x7f1fd40cd050 0x7f1fd406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd3e00640 1 -- 172.21.15.59:0/3161133826 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770129133 0 0) 0x7f1fd415f160 con 0x7f1fd415ece0 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd3e00640 1 -- 172.21.15.59:0/3161133826 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fbc003520 con 0x7f1fd415ece0 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9e00640 1 -- 172.21.15.59:0/3161133826 >> v2:172.21.15.66:3300/0 conn(0x7f1fd40cd050 msgr2=0x7f1fd406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/3161133826 >> v2:172.21.15.66:3300/0 conn(0x7f1fd40cd050 0x7f1fd406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9e00640 1 -- 172.21.15.59:0/3161133826 >> v1:172.21.15.59:6789/0 conn(0x7f1fd415ece0 legacy=0x7f1fd40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.791+0000 7f1fd9e00640 1 -- 172.21.15.59:0/3161133826 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd415f430 con 0x7f1fd40b3080 2024-04-24T06:29:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.792+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/3161133826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd40b3080 0x7f1fd40cf470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fd406e080 tx=0x7f1fcc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38eb6a14eb8e917 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.793+0000 7f1fd3e00640 1 -- 172.21.15.59:0/3161133826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc007740 con 0x7f1fd40b3080 2024-04-24T06:29:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.793+0000 7f1fd3e00640 1 -- 172.21.15.59:0/3161133826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc0078e0 con 0x7f1fd40b3080 2024-04-24T06:29:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.793+0000 7f1fd3e00640 1 -- 172.21.15.59:0/3161133826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc007cb0 con 0x7f1fd40b3080 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.793+0000 7f1fda800640 1 -- 172.21.15.59:0/3161133826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd40b3080 msgr2=0x7f1fd40cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.793+0000 7f1fda800640 1 --2- 172.21.15.59:0/3161133826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd40b3080 0x7f1fd40cf470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fd406e080 tx=0x7f1fcc02a5f0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 -- 172.21.15.59:0/3161133826 shutdown_connections 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 --2- 172.21.15.59:0/3161133826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd40b3080 0x7f1fd40cf470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 --2- 172.21.15.59:0/3161133826 >> v2:172.21.15.66:3300/0 conn(0x7f1fd40cd050 0x7f1fd406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 -- 172.21.15.59:0/3161133826 >> 172.21.15.59:0/3161133826 conn(0x7f1fd40b21a0 msgr2=0x7f1fd40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 -- 172.21.15.59:0/3161133826 shutdown_connections 2024-04-24T06:29:02.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.794+0000 7f1fda800640 1 -- 172.21.15.59:0/3161133826 wait complete. 2024-04-24T06:29:02.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.796+0000 7f1fda800640 1 Processor -- start 2024-04-24T06:29:02.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.796+0000 7f1fda800640 1 -- start start 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fd40b3080 0x7f1fd41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fd40cd050 0x7f1fd41f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd40cef50 con 0x7f1fd415ece0 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd415f9f0 con 0x7f1fd40cd050 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fda800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd4159110 con 0x7f1fd40b3080 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36820/0 (socket says 172.21.15.59:36820) 2024-04-24T06:29:02.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fd9e00640 1 -- 172.21.15.59:0/2663971633 learned_addr learned my addr 172.21.15.59:0/2663971633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:02.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fd9400640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fd40b3080 0x7f1fd41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.797+0000 7f1fd8a00640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fd40cd050 0x7f1fd41f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 -- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fd40b3080 msgr2=0x7f1fd41f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fd40b3080 0x7f1fd41f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 -- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fd40cd050 msgr2=0x7f1fd41f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fd40cd050 0x7f1fd41f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 -- 172.21.15.59:0/2663971633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd41f48d0 con 0x7f1fd415ece0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd9e00640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009e50 tx=0x7f1fcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc040070 con 0x7f1fd415ece0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc031bf0 con 0x7f1fd415ece0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc009450 con 0x7f1fd415ece0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd41f4bb0 con 0x7f1fd415ece0 2024-04-24T06:29:02.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.798+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd40c7320 con 0x7f1fd415ece0 2024-04-24T06:29:02.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.800+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd404fd90 con 0x7f1fd415ece0 2024-04-24T06:29:02.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.800+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fcc039040 con 0x7f1fd415ece0 2024-04-24T06:29:02.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.801+0000 7f1fd1600640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 0x7f1fb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:02.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.801+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fcc0ad4c0 con 0x7f1fd415ece0 2024-04-24T06:29:02.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.802+0000 7f1fd9400640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 0x7f1fb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:02.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.802+0000 7f1fd9400640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 0x7f1fb406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8006fd0 tx=0x7f1fc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:02.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:02.806+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fcc07b200 con 0x7f1fd415ece0 2024-04-24T06:29:03.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.054+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1fd40c74f0 con 0x7f1fd415ece0 2024-04-24T06:29:03.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.055+0000 7f1fd1600640 1 -- 172.21.15.59:0/2663971633 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f1fcc07f500 con 0x7f1fd415ece0 2024-04-24T06:29:03.056 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:29:03.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 msgr2=0x7f1fb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:03.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 0x7f1fb406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8006fd0 tx=0x7f1fc8008040 comp rx=0 tx=0).stop 2024-04-24T06:29:03.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 msgr2=0x7f1fd40cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:03.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009e50 tx=0x7f1fcc004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 shutdown_connections 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fb406c8f0 0x7f1fb406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fd415ece0 0x7f1fd40cb1f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fd40cd050 0x7f1fd41f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 --2- 172.21.15.59:0/2663971633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fd40b3080 0x7f1fd41f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.057+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 >> 172.21.15.59:0/2663971633 conn(0x7f1fd40b21a0 msgr2=0x7f1fd40ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.058+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 shutdown_connections 2024-04-24T06:29:03.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03.058+0000 7f1fda800640 1 -- 172.21.15.59:0/2663971633 wait complete. 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:29:03.079 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:29:03.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:03.079 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:29:03.131 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42601sec preferred_lft 42601sec 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:29:03.133 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T06:29:03.133 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:03.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:03.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:03.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:03.650 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:29:03.724 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:29:03.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:03.728 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:29:03.778 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:29:03.778 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:29:03.778 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:29:03.778 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:29:03.779 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:29:03.779 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:03.779 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:03.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:29:03.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:29:03.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:29:03.780 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:03.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:04.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:04.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:04.539 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:29:04.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:04.587 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-04-24T06:29:04.631 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:04.631 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:04.631 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:04.631 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T06:29:04.631 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:04.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:04.852 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T06:29:04.853 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:05.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:29:05.006 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:29:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:05.156 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:05.157 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:05.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:29:05.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T06:29:05.157 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:05.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:05.407 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:29:05.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:05.407 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:05.450 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:29:05.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:05.450 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:05.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:29:05.647 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:29:05.701 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:05.702 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:05.703 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:05.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:05.704 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:29:05.758 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:29:05.758 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:29:05.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:29:05.831 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:29:05.876 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:29:05.877 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:05.878 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:05.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:05.879 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:29:05.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:29:05.946+0000 7f28aa51b480 -1 init, newargv = 0x5576941b4460 newargc=15 2024-04-24T06:29:05.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[41687]: starting ceph client 2024-04-24T06:29:05.962 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[41687]: starting fuse 2024-04-24T06:29:05.966 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:29:05.966 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:05.966 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:29:05.967 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:29:05.968 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:29:05.968 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:05.968 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:29:05.968 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:29:06.154 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.41687.asok 2024-04-24T06:29:06.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:06.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41687.asok 2024-04-24T06:29:06.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:06.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41687.asok status 2024-04-24T06:29:06.401 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "41687", 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4650, 2024-04-24T06:29:06.402 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4650 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 8337777 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:29:06.403 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 8337777 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4650 192.168.0.1:0/8337777", 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/8337777", 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 8, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 20, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:29:06.404 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:29:06.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:29:06.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.627+0000 7ff6ab200640 1 Processor -- start 2024-04-24T06:29:06.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.627+0000 7ff6ab200640 1 -- start start 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6ab200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6ac0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6ab200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ac0cf030 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6ab200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ac0b6a40 con 0x7ff6ac15e7c0 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6ab200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac0b7000 con 0x7ff6ac0d14a0 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6a9400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff6ac0b5ff0 0x7ff6ac0d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43642/0 (socket says 172.21.15.59:43642) 2024-04-24T06:29:06.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6a9400640 1 -- 172.21.15.59:0/2108197902 learned_addr learned my addr 172.21.15.59:0/2108197902 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.629+0000 7ff6aa800640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6ac0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575784207 0 0) 0x7ff698001110 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff690003520 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2503945020 0 0) 0x7ff690003520 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 msgr2=0x7ff6ac0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:06.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6ac0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 >> v1:172.21.15.66:6789/0 conn(0x7ff6ac15e7c0 legacy=0x7ff6ac0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac0b6d20 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff698001840 con 0x7ff6ac0b5ff0 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 >> v1:172.21.15.59:6789/0 conn(0x7ff6ac0b5ff0 legacy=0x7ff6ac0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690003f60 0x7ff690004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690004d40 0x7ff6900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0b7000 con 0x7ff690004d40 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff69000baa0 con 0x7ff690003f60 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.630+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff69000bca0 con 0x7ff6ac0d14a0 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690004d40 0x7ff6900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6a9400640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690003f60 0x7ff690004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6a9e00640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 -- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 msgr2=0x7ff6900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:06.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac0d14a0 0x7ff6900039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 -- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690003f60 msgr2=0x7ff690004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:06.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690003f60 0x7ff690004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.631+0000 7ff6aa800640 1 -- 172.21.15.59:0/2108197902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff690003520 con 0x7ff690004d40 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.632+0000 7ff6aa800640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690004d40 0x7ff6900051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0d0b50 tx=0x7ff690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ffa7b002b9dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.632+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4037070 con 0x7ff690004d40 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.632+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a4032620 con 0x7ff690004d40 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.632+0000 7ff6a8a00640 1 -- 172.21.15.59:0/2108197902 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4009c10 con 0x7ff690004d40 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 -- 172.21.15.59:0/2108197902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690004d40 msgr2=0x7ff6900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 --2- 172.21.15.59:0/2108197902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff690004d40 0x7ff6900051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0d0b50 tx=0x7ff690005830 comp rx=0 tx=0).stop 2024-04-24T06:29:06.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6aa800640 1 -- 172.21.15.59:0/2108197902 reap_dead start 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 -- 172.21.15.59:0/2108197902 shutdown_connections 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 -- 172.21.15.59:0/2108197902 >> 172.21.15.59:0/2108197902 conn(0x7ff6ac09d920 msgr2=0x7ff6ac08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 -- 172.21.15.59:0/2108197902 shutdown_connections 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.633+0000 7ff6ab200640 1 -- 172.21.15.59:0/2108197902 wait complete. 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.635+0000 7ff6ab200640 1 Processor -- start 2024-04-24T06:29:06.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.635+0000 7ff6ab200640 1 -- start start 2024-04-24T06:29:06.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 0x7ff6ac0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac14e960 0x7ff6ac0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0d2800 con 0x7ff6ac0cc190 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0cf620 con 0x7ff690004d40 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6ab200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac0d1a80 con 0x7ff6ac14e960 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36840/0 (socket says 172.21.15.59:36840) 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.636+0000 7ff6aa800640 1 -- 172.21.15.59:0/1031634920 learned_addr learned my addr 172.21.15.59:0/1031634920 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:06.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a9400640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac14e960 0x7ff6ac0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a9e00640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 0x7ff6ac0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 -- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac14e960 msgr2=0x7ff6ac0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac14e960 0x7ff6ac0b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 -- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 msgr2=0x7ff6ac0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 0x7ff6ac0cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 -- 172.21.15.59:0/1031634920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac0b2b60 con 0x7ff6ac0cc190 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a9e00640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 0x7ff6ac0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6aa800640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e230 tx=0x7ff6a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4037040 con 0x7ff6ac0cc190 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a4044070 con 0x7ff6ac0cc190 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.637+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4009c10 con 0x7ff6ac0cc190 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.638+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac0b1580 con 0x7ff6ac0cc190 2024-04-24T06:29:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.638+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac0b1b40 con 0x7ff6ac0cc190 2024-04-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.639+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ac04fd90 con 0x7ff6ac0cc190 2024-04-24T06:29:06.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.640+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6a4005d10 con 0x7ff6ac0cc190 2024-04-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.641+0000 7ff6a2000640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 0x7ff68c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.641+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6a40ac2c0 con 0x7ff6ac0cc190 2024-04-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.641+0000 7ff6a9e00640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 0x7ff68c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.642+0000 7ff6a9e00640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 0x7ff68c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0caaf0 tx=0x7ff69c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:06.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.646+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6a407a660 con 0x7ff6ac0cc190 2024-04-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.895+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff6ac15e0e0 con 0x7ff6ac0cc190 2024-04-24T06:29:06.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.895+0000 7ff6a2000640 1 -- 172.21.15.59:0/1031634920 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff6a4079f60 con 0x7ff6ac0cc190 2024-04-24T06:29:06.897 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 msgr2=0x7ff68c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 0x7ff68c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0caaf0 tx=0x7ff69c009040 comp rx=0 tx=0).stop 2024-04-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 msgr2=0x7ff6ac0cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400e230 tx=0x7ff6a4009750 comp rx=0 tx=0).stop 2024-04-24T06:29:06.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 shutdown_connections 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff68c06c8f0 0x7ff68c06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6ac0cc190 0x7ff6ac0cc610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff6ac14e960 0x7ff6ac0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 --2- 172.21.15.59:0/1031634920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff690004d40 0x7ff6ac0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 >> 172.21.15.59:0/1031634920 conn(0x7ff6ac09d920 msgr2=0x7ff6ac15d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 shutdown_connections 2024-04-24T06:29:06.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:06.898+0000 7ff6ab200640 1 -- 172.21.15.59:0/1031634920 wait complete. 2024-04-24T06:29:06.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:29:06.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:29:06.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:29:06.920 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-24T06:29:06.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:06.920 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42597sec preferred_lft 42597sec 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:06.977 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:06.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:06.978 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:29:07.060 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:29:07.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:07.065 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:29:07.121 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:29:07.121 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:29:07.121 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:29:07.122 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:29:07.122 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:29:07.122 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:07.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:07.346 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:29:07.399 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:29:07.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:07.401 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-04-24T06:29:07.460 INFO:teuthology.orchestra.run.smithi059.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-24T06:29:07.461 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:07.461 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:07.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:29:07.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-24T06:29:07.461 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:07.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:07.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:07.736 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 62:d4:b3:3c:7e:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T06:29:07.737 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T06:29:07.737 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.737 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::60d4:b3ff:fe3c:7efe/64 scope link 2024-04-24T06:29:07.737 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:07.741 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T06:29:07.827 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:07.827 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 62:d4:b3:3c:7e:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::60d4:b3ff:fe3c:7efe/64 scope link 2024-04-24T06:29:07.828 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:07.832 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-24T06:29:07.832 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:07.832 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:07.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-24T06:29:07.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T06:29:07.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-24T06:29:07.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-24T06:29:07.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-24T06:29:07.833 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:08.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:08.132 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:08.132 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:29:08.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.1 up 2024-04-24T06:29:08.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T06:29:08.133 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:29:08.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:08.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:08.353 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-24T06:29:08.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:08.353 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:08.401 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-24T06:29:08.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:08.402 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:08.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:29:08.534 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:29:08.590 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:29:08.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:08.594 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:29:08.649 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:29:08.649 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-24T06:29:08.649 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-24T06:29:08.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:29:08.725 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:29:08.771 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:29:08.772 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:29:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:29:08.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:08.774 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:29:08.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:2024-04-24T06:29:08.831+0000 7f487d91d480 -1 init, newargv = 0x55d6ce2ba5f0 newargc=15 2024-04-24T06:29:08.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:ceph-fuse[42185]: starting ceph client 2024-04-24T06:29:08.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:ceph-fuse[42185]: starting fuse 2024-04-24T06:29:08.859 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:29:08.860 INFO:teuthology.orchestra.run.smithi059.stdout:43 2024-04-24T06:29:08.860 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:08.860 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:29:08.861 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.2")) 2024-04-24T06:29:08.862 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:29:09.056 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.2.42185.asok 2024-04-24T06:29:09.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:09.073 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.42185.asok 2024-04-24T06:29:09.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:09.074 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.42185.asok status 2024-04-24T06:29:09.309 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:29:09.309 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:29:09.309 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "2", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "42185", 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4665, 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:29:09.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4665 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2391891645 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2391891645 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4665 192.168.0.2:0/2391891645", 2024-04-24T06:29:09.311 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.2:0/2391891645", 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 8, 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 20, 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:29:09.312 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:29:09.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:29:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.533+0000 7f975d600640 1 Processor -- start 2024-04-24T06:29:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.534+0000 7f975d600640 1 -- start start 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.535+0000 7f975d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f975815e7c0 0x7f97580d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.535+0000 7f975d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d0d90 0x7f97580d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.535+0000 7f975d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97580cf030 con 0x7f97580d2630 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.535+0000 7f975d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f97580d1740 con 0x7f975815e7c0 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.535+0000 7f975d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97580b7000 con 0x7f97580d0d90 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.536+0000 7f9757e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f975815e7c0 0x7f97580d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.536+0000 7f9757e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f975815e7c0 0x7f97580d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:38030/0 (socket says 172.21.15.59:38030) 2024-04-24T06:29:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.536+0000 7f9757e00640 1 -- 172.21.15.59:0/2620258357 learned_addr learned my addr 172.21.15.59:0/2620258357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.536+0000 7f975cc00640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d0d90 0x7f97580d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.537+0000 7f9757e00640 1 -- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d0d90 msgr2=0x7f97580d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.537+0000 7f9757e00640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d0d90 0x7f97580d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.537+0000 7f9757e00640 1 -- 172.21.15.59:0/2620258357 >> v1:172.21.15.66:6789/0 conn(0x7f97580d2630 legacy=0x7f97580b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.537+0000 7f9757e00640 1 -- 172.21.15.59:0/2620258357 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580b6a40 con 0x7f975815e7c0 2024-04-24T06:29:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.538+0000 7f9757e00640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580d20f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9748009870 tx=0x7f974802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7bff7dcd9578d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:09.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.539+0000 7f9757400640 1 -- 172.21.15.59:0/2620258357 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748037070 con 0x7f975815e7c0 2024-04-24T06:29:09.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.539+0000 7f9757400640 1 -- 172.21.15.59:0/2620258357 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748032440 con 0x7f975815e7c0 2024-04-24T06:29:09.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.539+0000 7f9757400640 1 -- 172.21.15.59:0/2620258357 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748032810 con 0x7f975815e7c0 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 -- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 msgr2=0x7f97580d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580d20f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9748009870 tx=0x7f974802a270 comp rx=0 tx=0).stop 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 -- 172.21.15.59:0/2620258357 shutdown_connections 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d0d90 0x7f97580d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 --2- 172.21.15.59:0/2620258357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580d20f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 -- 172.21.15.59:0/2620258357 >> 172.21.15.59:0/2620258357 conn(0x7f975809d920 msgr2=0x7f975808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 -- 172.21.15.59:0/2620258357 shutdown_connections 2024-04-24T06:29:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.540+0000 7f975d600640 1 -- 172.21.15.59:0/2620258357 wait complete. 2024-04-24T06:29:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.543+0000 7f975d600640 1 Processor -- start 2024-04-24T06:29:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.543+0000 7f975d600640 1 -- start start 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 0x7f97580cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97580cf2b0 con 0x7f97580d0d90 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97580cf620 con 0x7f975815e7c0 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f975d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97580b6700 con 0x7f97580d2630 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f9757e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f9757e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37254/0 (socket says 172.21.15.59:37254) 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.544+0000 7f9757e00640 1 -- 172.21.15.59:0/3214222295 learned_addr learned my addr 172.21.15.59:0/3214222295 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f975cc00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f974fe00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 0x7f97580cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 -- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 msgr2=0x7f97580cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 0x7f97580cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 -- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 msgr2=0x7f97580cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 -- 172.21.15.59:0/3214222295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580c85d0 con 0x7f97580d0d90 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f974fe00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 0x7f97580cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:09.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.545+0000 7f9757e00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9748000c00 tx=0x7f97480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.546+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748037040 con 0x7f97580d0d90 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.546+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748042070 con 0x7f97580d0d90 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.546+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97580b7300 con 0x7f97580d0d90 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.546+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748030940 con 0x7f97580d0d90 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.546+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97580b78c0 con 0x7f97580d0d90 2024-04-24T06:29:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.547+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975804fd90 con 0x7f97580d0d90 2024-04-24T06:29:09.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.548+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9748030ae0 con 0x7f97580d0d90 2024-04-24T06:29:09.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.549+0000 7f9754c00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 0x7f973406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:09.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.550+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f97480ac3f0 con 0x7f97580d0d90 2024-04-24T06:29:09.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.550+0000 7f974fe00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 0x7f973406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:09.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.550+0000 7f974fe00640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 0x7f973406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f97580d2a30 tx=0x7f9740009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.554+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97580c85d0 con 0x7f97580d0d90 2024-04-24T06:29:09.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.793+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f97580d2410 con 0x7f97580d0d90 2024-04-24T06:29:09.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.794+0000 7f9754c00640 1 -- 172.21.15.59:0/3214222295 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f974807a810 con 0x7f97580d0d90 2024-04-24T06:29:09.795 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 msgr2=0x7f973406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 0x7f973406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f97580d2a30 tx=0x7f9740009040 comp rx=0 tx=0).stop 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 msgr2=0x7f9758153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9748000c00 tx=0x7f97480046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 shutdown_connections 2024-04-24T06:29:09.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f973406c8f0 0x7f973406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975815e7c0 0x7f97580cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97580d2630 0x7f97580cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 --2- 172.21.15.59:0/3214222295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97580d0d90 0x7f9758153ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.796+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 >> 172.21.15.59:0/3214222295 conn(0x7f975809d920 msgr2=0x7f975815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.797+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 shutdown_connections 2024-04-24T06:29:09.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:09.797+0000 7f975d600640 1 -- 172.21.15.59:0/3214222295 wait complete. 2024-04-24T06:29:09.816 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:29:09.816 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:29:09.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:29:09.817 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T06:29:09.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:09.817 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-04-24T06:29:09.881 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42632sec preferred_lft 42632sec 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-04-24T06:29:09.882 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:09.883 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T06:29:09.883 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:09.883 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:09.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-04-24T06:29:09.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-04-24T06:29:09.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-04-24T06:29:09.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T06:29:09.884 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:09.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:10.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:10.385 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:29:10.456 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-04-24T06:29:10.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:10.460 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-04-24T06:29:10.513 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-04-24T06:29:10.513 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:29:10.513 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T06:29:10.513 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T06:29:10.513 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:29:10.513 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:10.514 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:10.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T06:29:10.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T06:29:10.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T06:29:10.514 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:10.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:11.290 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:11.301 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-24T06:29:11.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:11.358 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-04-24T06:29:11.416 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:11.417 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:11.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:11.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T06:29:11.417 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:11.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:11.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:11.619 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T06:29:11.620 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:11.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:11.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:11.931 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:11.931 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:11.932 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-04-24T06:29:11.932 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T06:29:11.932 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:12.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:12.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:12.147 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-24T06:29:12.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:12.147 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:12.200 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T06:29:12.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:12.200 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:12.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-24T06:29:12.329 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-24T06:29:12.385 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-24T06:29:12.386 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-24T06:29:12.387 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:12.388 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:12.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:12.389 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-24T06:29:12.444 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:29:12.444 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-24T06:29:12.449 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-24T06:29:12.523 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:12.568 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-24T06:29:12.569 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-24T06:29:12.570 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:12.571 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:12.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:12.572 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-24T06:29:12.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi066.stderr:2024-04-24T06:29:12.636+0000 7fbf52f41480 -1 init, newargv = 0x5597a04b8580 newargc=15 2024-04-24T06:29:12.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi066.stderr:ceph-fuse[39096]: starting ceph client 2024-04-24T06:29:12.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi066.stderr:ceph-fuse[39096]: starting fuse 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi066.stdout:42 2024-04-24T06:29:12.658 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:12.659 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.1")) 2024-04-24T06:29:12.660 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-04-24T06:29:12.862 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.1.39096.asok 2024-04-24T06:29:12.867 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:12.883 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39096.asok 2024-04-24T06:29:12.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:12.884 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39096.asok status 2024-04-24T06:29:13.122 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-24T06:29:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-04-24T06:29:13.122 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "1", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "39096", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4457, 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4457 2024-04-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1349990826 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1349990826 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4457 192.168.0.1:0/1349990826", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/1349990826", 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 8, 2024-04-24T06:29:13.124 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 20, 2024-04-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-04-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-04-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-24T06:29:13.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:29:13.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.358+0000 7fe141000640 1 Processor -- start 2024-04-24T06:29:13.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.359+0000 7fe141000640 1 -- start start 2024-04-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.360+0000 7fe141000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe13c1454e0 0x7fe13c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.360+0000 7fe141000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe13c0cd520 con 0x7fe13c0cd020 2024-04-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.360+0000 7fe141000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe13c0cd800 con 0x7fe13c160910 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.360+0000 7fe141000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe13c0cddc0 con 0x7fe13c1454e0 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.360+0000 7fe139e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe13c1454e0 0x7fe13c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe13c1454e0 0x7fe13c145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37256/0 (socket says 172.21.15.59:37256) 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139e00640 1 -- 172.21.15.59:0/1815402679 learned_addr learned my addr 172.21.15.59:0/1815402679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139400640 1 --1- 172.21.15.59:0/1815402679 >> v1:172.21.15.59:6789/0 conn(0x7fe13c0cd020 0x7fe13c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39878/0 (socket says 172.21.15.59:39878) 2024-04-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139e00640 1 -- 172.21.15.59:0/1815402679 >> v1:172.21.15.66:6789/0 conn(0x7fe13c160910 legacy=0x7fe13c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139e00640 1 -- 172.21.15.59:0/1815402679 >> v1:172.21.15.59:6789/0 conn(0x7fe13c0cd020 legacy=0x7fe13c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe139e00640 1 -- 172.21.15.59:0/1815402679 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c0cdae0 con 0x7fe13c1454e0 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.361+0000 7fe138a00640 1 -- 172.21.15.59:0/1815402679 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987046811 0 0) 0x7fe13c0cd520 con 0x7fe13c0cd020 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.362+0000 7fe139e00640 1 --2- 172.21.15.59:0/1815402679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c1454e0 0x7fe13c145bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009e50 tx=0x7fe12c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986ea304ad89a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe138a00640 1 -- 172.21.15.59:0/1815402679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c036070 con 0x7fe13c1454e0 2024-04-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe138a00640 1 -- 172.21.15.59:0/1815402679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c031600 con 0x7fe13c1454e0 2024-04-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe138a00640 1 -- 172.21.15.59:0/1815402679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0319d0 con 0x7fe13c1454e0 2024-04-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe141000640 1 -- 172.21.15.59:0/1815402679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c1454e0 msgr2=0x7fe13c145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe141000640 1 --2- 172.21.15.59:0/1815402679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c1454e0 0x7fe13c145bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009e50 tx=0x7fe12c02a6d0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe141000640 1 -- 172.21.15.59:0/1815402679 shutdown_connections 2024-04-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe141000640 1 --2- 172.21.15.59:0/1815402679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c1454e0 0x7fe13c145bb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.363+0000 7fe141000640 1 -- 172.21.15.59:0/1815402679 >> 172.21.15.59:0/1815402679 conn(0x7fe13c067ea0 msgr2=0x7fe13c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.364+0000 7fe141000640 1 -- 172.21.15.59:0/1815402679 shutdown_connections 2024-04-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.364+0000 7fe141000640 1 -- 172.21.15.59:0/1815402679 wait complete. 2024-04-24T06:29:13.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.365+0000 7fe141000640 1 Processor -- start 2024-04-24T06:29:13.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.365+0000 7fe141000640 1 -- start start 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe13c0cd020 0x7fe13c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13c1454e0 0x7fe13c0a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c0b4010 con 0x7fe13c160910 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c1460f0 con 0x7fe13c1454e0 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.366+0000 7fe141000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe13c0a9ee0 con 0x7fe13c0cd020 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37262/0 (socket says 172.21.15.59:37262) 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 -- 172.21.15.59:0/3963055764 learned_addr learned my addr 172.21.15.59:0/3963055764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe139e00640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe13c0cd020 0x7fe13c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe139400640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13c1454e0 0x7fe13c0a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 -- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe13c0cd020 msgr2=0x7fe13c0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe13c0cd020 0x7fe13c0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 -- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13c1454e0 msgr2=0x7fe13c0a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13c1454e0 0x7fe13c0a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.367+0000 7fe13a800640 1 -- 172.21.15.59:0/3963055764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c0adcc0 con 0x7fe13c160910 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe13a800640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b410 tx=0x7fe13400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134015030 con 0x7fe13c160910 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe134004020 con 0x7fe13c160910 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134004410 con 0x7fe13c160910 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c0aa150 con 0x7fe13c160910 2024-04-24T06:29:13.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.368+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c0aa710 con 0x7fe13c160910 2024-04-24T06:29:13.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.369+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13c068a80 con 0x7fe13c160910 2024-04-24T06:29:13.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.370+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe134018070 con 0x7fe13c160910 2024-04-24T06:29:13.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.372+0000 7fe132000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 0x7fe11806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:13.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.372+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe13408b6e0 con 0x7fe13c160910 2024-04-24T06:29:13.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.372+0000 7fe139e00640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 0x7fe11806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:13.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.373+0000 7fe139e00640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 0x7fe11806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe12c033000 tx=0x7fe12c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:13.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.377+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe134059a70 con 0x7fe13c160910 2024-04-24T06:29:13.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.626+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe13c0ade90 con 0x7fe13c160910 2024-04-24T06:29:13.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.627+0000 7fe132000640 1 -- 172.21.15.59:0/3963055764 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe134059370 con 0x7fe13c160910 2024-04-24T06:29:13.630 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:29:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 msgr2=0x7fe11806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 0x7fe11806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe12c033000 tx=0x7fe12c009000 comp rx=0 tx=0).stop 2024-04-24T06:29:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 msgr2=0x7fe13c1f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b410 tx=0x7fe13400b8d0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 shutdown_connections 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11806c8f0 0x7fe11806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe13c160910 0x7fe13c1f26b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13c1454e0 0x7fe13c0a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 --2- 172.21.15.59:0/3963055764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe13c0cd020 0x7fe13c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 >> 172.21.15.59:0/3963055764 conn(0x7fe13c067ea0 msgr2=0x7fe13c0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 shutdown_connections 2024-04-24T06:29:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:13.630+0000 7fe141000640 1 -- 172.21.15.59:0/3963055764 wait complete. 2024-04-24T06:29:13.652 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:29:13.652 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:29:13.652 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-04-24T06:29:13.652 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-24T06:29:13.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:13.653 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:29:13.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:29:13.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:29:13.653 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-24T06:29:13.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:13.653 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-04-24T06:29:13.714 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:13.714 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:13.714 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:13.714 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:29:13.715 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42628sec preferred_lft 42628sec 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether e2:5a:64:9f:5d:21 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::4035:bdff:fee1:c3a6/64 scope link 2024-04-24T06:29:13.716 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.717 INFO:teuthology.orchestra.run.smithi066.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:29:13.717 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether e2:5a:64:9f:5d:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:13.717 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::e05a:64ff:fe9f:5d21/64 scope link 2024-04-24T06:29:13.717 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:13.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:13.718 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:29:13.806 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-04-24T06:29:13.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:13.811 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-04-24T06:29:13.867 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-04-24T06:29:13.867 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:29:13.867 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T06:29:13.868 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T06:29:13.868 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T06:29:13.869 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:13.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:14.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:14.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:14.095 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-24T06:29:14.147 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T06:29:14.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:14.149 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-04-24T06:29:14.206 INFO:teuthology.orchestra.run.smithi066.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-24T06:29:14.207 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:14.207 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:14.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:29:14.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-24T06:29:14.207 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:14.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:14.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:14.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:14.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 36:52:fc:74:a4:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T06:29:14.484 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::3452:fcff:fe74:a49b/64 scope link 2024-04-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:14.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 36:52:fc:74:a4:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.571 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::3452:fcff:fe74:a49b/64 scope link 2024-04-24T06:29:14.572 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:29:14.574 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-24T06:29:14.575 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:14.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:14.871 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:29:14.871 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-04-24T06:29:14.871 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.1 up 2024-04-24T06:29:14.871 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T06:29:14.871 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-04-24T06:29:14.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:29:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:15.090 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-24T06:29:15.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:15.090 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:15.141 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-24T06:29:15.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:29:15.142 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:15.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-24T06:29:15.286 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:15.345 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-24T06:29:15.345 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-24T06:29:15.345 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:15.346 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-24T06:29:15.347 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:29:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:29:15.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:15.349 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-24T06:29:15.407 INFO:teuthology.orchestra.run.smithi066.stdout:42 2024-04-24T06:29:15.407 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-24T06:29:15.408 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-24T06:29:15.413 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2024-04-24T06:29:15.482 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:29:15.523 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/bpf 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2024-04-24T06:29:15.525 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:29:15.526 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:29:15.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:15.527 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2024-04-24T06:29:15.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi066.stderr:2024-04-24T06:29:15.581+0000 7f9f9cd21480 -1 init, newargv = 0x55bf9675a410 newargc=15 2024-04-24T06:29:15.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi066.stderr:ceph-fuse[39552]: starting ceph client 2024-04-24T06:29:15.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi066.stderr:ceph-fuse[39552]: starting fuse 2024-04-24T06:29:15.614 INFO:teuthology.orchestra.run.smithi066.stdout:42 2024-04-24T06:29:15.614 INFO:teuthology.orchestra.run.smithi066.stdout:43 2024-04-24T06:29:15.614 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-24T06:29:15.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:29:15.614 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> import re 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:15.615 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> return f 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.3")) 2024-04-24T06:29:15.616 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-04-24T06:29:15.809 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.3.39552.asok 2024-04-24T06:29:15.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-24T06:29:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:29:15.827 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39552.asok 2024-04-24T06:29:15.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:15.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39552.asok status 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "3", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "39552", 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2024-04-24T06:29:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4564, 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 4564 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1496770419 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-24T06:29:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.2:0", 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 1496770419 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.4564 192.168.0.2:0/1496770419", 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.2:0/1496770419", 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 8, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 20, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2024-04-24T06:29:16.058 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-24T06:29:16.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.072 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:29:16.073 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:16.136 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:29:16.136 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:16.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:16.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.223 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:29:16.223 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:16.278 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:29:16.278 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:16.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:16.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.369 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-04-24T06:29:16.369 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:16.433 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-04-24T06:29:16.433 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:16.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:16.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.529 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2024-04-24T06:29:16.529 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:16.588 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2024-04-24T06:29:16.589 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:16.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:16.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:16.671 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-24T06:29:16.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:16.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.933+0000 7f71a5200640 1 Processor -- start 2024-04-24T06:29:16.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.933+0000 7f71a5200640 1 -- start start 2024-04-24T06:29:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.934+0000 7f71a5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f71a00d14a0 0x7f71a00b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.934+0000 7f71a5200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a00cf030 con 0x7f71a00b5ff0 2024-04-24T06:29:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.934+0000 7f71a5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a00b6a40 con 0x7f71a015e7c0 2024-04-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.934+0000 7f71a5200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f71a00b7000 con 0x7f71a00d14a0 2024-04-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.935+0000 7f719ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f71a00b5ff0 0x7f71a00d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39884/0 (socket says 172.21.15.59:39884) 2024-04-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.935+0000 7f719ea00640 1 -- 172.21.15.59:0/855372276 learned_addr learned my addr 172.21.15.59:0/855372276 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.935+0000 7f719fe00640 1 --2- 172.21.15.59:0/855372276 >> v2:172.21.15.66:3300/0 conn(0x7f71a00d14a0 0x7f71a00b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:16.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678640121 0 0) 0x7f718c001110 con 0x7f71a00b5ff0 2024-04-24T06:29:16.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7184003520 con 0x7f71a00b5ff0 2024-04-24T06:29:16.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3858496004 0 0) 0x7f7184003520 con 0x7f71a00b5ff0 2024-04-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719fe00640 1 -- 172.21.15.59:0/855372276 >> v1:172.21.15.66:6789/0 conn(0x7f71a015e7c0 legacy=0x7f71a00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719fe00640 1 -- 172.21.15.59:0/855372276 >> v1:172.21.15.59:6789/0 conn(0x7f71a00b5ff0 legacy=0x7f71a00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.936+0000 7f719fe00640 1 -- 172.21.15.59:0/855372276 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a00b6d20 con 0x7f71a00d14a0 2024-04-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.937+0000 7f719fe00640 1 --2- 172.21.15.59:0/855372276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a00b6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7198009d90 tx=0x7f719802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208af0ee0174137a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:16.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.938+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c001840 con 0x7f71a00d14a0 2024-04-24T06:29:16.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.938+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198007540 con 0x7f71a00d14a0 2024-04-24T06:29:16.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.938+0000 7f719e000640 1 -- 172.21.15.59:0/855372276 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980078f0 con 0x7f71a00d14a0 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 -- 172.21.15.59:0/855372276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 msgr2=0x7f71a00b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 --2- 172.21.15.59:0/855372276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a00b6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7198009d90 tx=0x7f719802a680 comp rx=0 tx=0).stop 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 -- 172.21.15.59:0/855372276 shutdown_connections 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 --2- 172.21.15.59:0/855372276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a00b6450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 -- 172.21.15.59:0/855372276 >> 172.21.15.59:0/855372276 conn(0x7f71a009d920 msgr2=0x7f71a008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 -- 172.21.15.59:0/855372276 shutdown_connections 2024-04-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.939+0000 7f71a5200640 1 -- 172.21.15.59:0/855372276 wait complete. 2024-04-24T06:29:16.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.941+0000 7f71a5200640 1 Processor -- start 2024-04-24T06:29:16.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.941+0000 7f71a5200640 1 -- start start 2024-04-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71a015e7c0 0x7f71a01e8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00d2800 con 0x7f71a00b5ff0 2024-04-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00cf620 con 0x7f71a00d14a0 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f71a5200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f71a00d1a80 con 0x7f71a015e7c0 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f719f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f719f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37264/0 (socket says 172.21.15.59:37264) 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f719f400640 1 -- 172.21.15.59:0/1342616185 learned_addr learned my addr 172.21.15.59:0/1342616185 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f719fe00640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71a015e7c0 0x7f71a01e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.943+0000 7f719ea00640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 -- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71a015e7c0 msgr2=0x7f71a01e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71a015e7c0 0x7f71a01e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 -- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 msgr2=0x7f71a014e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a014e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 -- 172.21.15.59:0/1342616185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0160ae0 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f719f400640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f719000c3c0 tx=0x7f719000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71900052d0 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7190005490 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190007260 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.944+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a0160d80 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.945+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a0161340 con 0x7f71a00b5ff0 2024-04-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.946+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a004fd90 con 0x7f71a00b5ff0 2024-04-24T06:29:16.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.947+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7190007400 con 0x7f71a00b5ff0 2024-04-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.948+0000 7f7197400640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.948+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f719008b060 con 0x7f71a00b5ff0 2024-04-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.948+0000 7f719ea00640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.949+0000 7f719ea00640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 0x7f718006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a00cd6f0 tx=0x7f718c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:16.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:16.953+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7190059480 con 0x7f71a00b5ff0 2024-04-24T06:29:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.157+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a008c250 con 0x7f71a00b5ff0 2024-04-24T06:29:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.160+0000 7f7197400640 1 -- 172.21.15.59:0/1342616185 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f7190058d80 con 0x7f71a00b5ff0 2024-04-24T06:29:17.162 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:17.162 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.162+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 msgr2=0x7f718006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 0x7f718006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a00cd6f0 tx=0x7f718c009040 comp rx=0 tx=0).stop 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 msgr2=0x7f71a0161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f719000c3c0 tx=0x7f719000c880 comp rx=0 tx=0).stop 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 shutdown_connections 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f71a015e7c0 0x7f71a01e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71a00d14a0 0x7f71a014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 --2- 172.21.15.59:0/1342616185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71a00b5ff0 0x7f71a0161b40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 >> 172.21.15.59:0/1342616185 conn(0x7f71a009d920 msgr2=0x7f71a015f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 shutdown_connections 2024-04-24T06:29:17.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.163+0000 7f71a5200640 1 -- 172.21.15.59:0/1342616185 wait complete. 2024-04-24T06:29:17.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:17.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.401+0000 7f7e39200640 1 Processor -- start 2024-04-24T06:29:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.401+0000 7f7e39200640 1 -- start start 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e39200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e3415d810 0x7f7e34160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e39200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3415cab0 con 0x7f7e341567e0 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e39200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e34160e40 con 0x7f7e340d0d10 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e39200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3415cd30 con 0x7f7e3415d810 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e33400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7e341567e0 0x7f7e340d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39894/0 (socket says 172.21.15.59:39894) 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e33400640 1 -- 172.21.15.59:0/1681446364 learned_addr learned my addr 172.21.15.59:0/1681446364 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:17.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.403+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1681446364 >> v2:172.21.15.66:3300/0 conn(0x7f7e3415d810 0x7f7e34160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215881498 0 0) 0x7f7e24001110 con 0x7f7e341567e0 2024-04-24T06:29:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e18003520 con 0x7f7e341567e0 2024-04-24T06:29:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 965883868 0 0) 0x7f7e18003520 con 0x7f7e341567e0 2024-04-24T06:29:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 >> v2:172.21.15.66:3300/0 conn(0x7f7e3415d810 msgr2=0x7f7e34160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 --2- 172.21.15.59:0/1681446364 >> v2:172.21.15.66:3300/0 conn(0x7f7e3415d810 0x7f7e34160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 >> v1:172.21.15.66:6789/0 conn(0x7f7e340d0d10 legacy=0x7f7e34160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3415d010 con 0x7f7e341567e0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7e24001840 con 0x7f7e341567e0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 >> v1:172.21.15.59:6789/0 conn(0x7f7e341567e0 legacy=0x7f7e340d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e18003a00 0x7f7e18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e18004800 0x7f7e18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 0x7f7e1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3415cd30 con 0x7f7e180051c0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1800c0c0 con 0x7f7e18003a00 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1800c2c0 con 0x7f7e18004800 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.404+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 0x7f7e1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33400640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e18003a00 0x7f7e18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e32a00640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e18004800 0x7f7e18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33e00640 1 -- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e18004800 msgr2=0x7f7e18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:17.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e18004800 0x7f7e18004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33e00640 1 -- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e18003a00 msgr2=0x7f7e18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:17.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e18003a00 0x7f7e18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.405+0000 7f7e33e00640 1 -- 172.21.15.59:0/1681446364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18003520 con 0x7f7e180051c0 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.406+0000 7f7e33e00640 1 -- 172.21.15.59:0/1681446364 reap_dead start 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.406+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 0x7f7e1800b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009f90 tx=0x7f7e18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c61c2999ad92869 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.406+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c035020 con 0x7f7e180051c0 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.406+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c003b90 con 0x7f7e180051c0 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.406+0000 7f7e32000640 1 -- 172.21.15.59:0/1681446364 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c009c10 con 0x7f7e180051c0 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 -- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 msgr2=0x7f7e1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 0x7f7e1800b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009f90 tx=0x7f7e18005830 comp rx=0 tx=0).stop 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 -- 172.21.15.59:0/1681446364 shutdown_connections 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 --2- 172.21.15.59:0/1681446364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180051c0 0x7f7e1800b9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 -- 172.21.15.59:0/1681446364 >> 172.21.15.59:0/1681446364 conn(0x7f7e3406bdf0 msgr2=0x7f7e3406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 -- 172.21.15.59:0/1681446364 shutdown_connections 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.407+0000 7f7e39200640 1 -- 172.21.15.59:0/1681446364 wait complete. 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.409+0000 7f7e39200640 1 Processor -- start 2024-04-24T06:29:17.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.409+0000 7f7e39200640 1 -- start start 2024-04-24T06:29:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e180051c0 0x7f7e34151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 0x7f7e34159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34156d40 con 0x7f7e34150520 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34156bc0 con 0x7f7e180051c0 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e39200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e34156ec0 con 0x7f7e341500c0 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37290/0 (socket says 172.21.15.59:37290) 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.410+0000 7f7e33e00640 1 -- 172.21.15.59:0/1768322267 learned_addr learned my addr 172.21.15.59:0/1768322267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e32a00640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 0x7f7e34159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33400640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e180051c0 0x7f7e34151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 -- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 msgr2=0x7f7e34159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 0x7f7e34159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 -- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e180051c0 msgr2=0x7f7e34151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e180051c0 0x7f7e34151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 -- 172.21.15.59:0/1768322267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3414dfa0 con 0x7f7e34150520 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e32a00640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 0x7f7e34159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e33e00640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00e230 tx=0x7f7e2c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c035030 con 0x7f7e34150520 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c042070 con 0x7f7e34150520 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c009c10 con 0x7f7e34150520 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.411+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3414e280 con 0x7f7e34150520 2024-04-24T06:29:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.412+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3414e840 con 0x7f7e34150520 2024-04-24T06:29:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.413+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3404fd90 con 0x7f7e34150520 2024-04-24T06:29:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.414+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e2c005d10 con 0x7f7e34150520 2024-04-24T06:29:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.415+0000 7f7e2b400640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.415+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e2c0ac650 con 0x7f7e34150520 2024-04-24T06:29:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.415+0000 7f7e33400640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.416+0000 7f7e33400640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 0x7f7e1406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e24000f30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.420+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e2c07a950 con 0x7f7e34150520 2024-04-24T06:29:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.623+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e340b5150 con 0x7f7e34150520 2024-04-24T06:29:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.624+0000 7f7e2b400640 1 -- 172.21.15.59:0/1768322267 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7e2c07a250 con 0x7f7e34150520 2024-04-24T06:29:17.626 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.626+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 msgr2=0x7f7e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.626+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 0x7f7e1406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e24000f30 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.626+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 msgr2=0x7f7e3414c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.626+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00e230 tx=0x7f7e2c009750 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 shutdown_connections 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e34150520 0x7f7e3414c860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e341500c0 0x7f7e34159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 --2- 172.21.15.59:0/1768322267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e180051c0 0x7f7e34151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 >> 172.21.15.59:0/1768322267 conn(0x7f7e3406bdf0 msgr2=0x7f7e3415bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 shutdown_connections 2024-04-24T06:29:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.627+0000 7f7e39200640 1 -- 172.21.15.59:0/1768322267 wait complete. 2024-04-24T06:29:17.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:17.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.847+0000 7ffab9200640 1 Processor -- start 2024-04-24T06:29:17.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.847+0000 7ffab9200640 1 -- start start 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffab40d0cc0 0x7ffab406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab9200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab40cfc20 con 0x7ffab4145e10 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab9200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab40cff00 con 0x7ffab4071d70 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab9200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffab40d01e0 con 0x7ffab40d0cc0 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffab40d0cc0 0x7ffab406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffab4145e10 0x7ffab40cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39904/0 (socket says 172.21.15.59:39904) 2024-04-24T06:29:17.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.849+0000 7ffab1e00640 1 -- 172.21.15.59:0/2519156461 learned_addr learned my addr 172.21.15.59:0/2519156461 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:17.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.850+0000 7ffab1400640 1 -- 172.21.15.59:0/2519156461 >> v1:172.21.15.66:6789/0 conn(0x7ffab4071d70 legacy=0x7ffab415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:17.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.850+0000 7ffab1400640 1 -- 172.21.15.59:0/2519156461 >> v1:172.21.15.59:6789/0 conn(0x7ffab4145e10 legacy=0x7ffab40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.850+0000 7ffab1400640 1 -- 172.21.15.59:0/2519156461 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab40d04c0 con 0x7ffab40d0cc0 2024-04-24T06:29:17.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.850+0000 7ffab0a00640 1 -- 172.21.15.59:0/2519156461 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475725062 0 0) 0x7ffab40cfc20 con 0x7ffab4145e10 2024-04-24T06:29:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.851+0000 7ffab1400640 1 --2- 172.21.15.59:0/2519156461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab40d0cc0 0x7ffab406f130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097d0 tx=0x7ffaa002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cf1bd0f11c66ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab0a00640 1 -- 172.21.15.59:0/2519156461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0037070 con 0x7ffab40d0cc0 2024-04-24T06:29:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab0a00640 1 -- 172.21.15.59:0/2519156461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa0032440 con 0x7ffab40d0cc0 2024-04-24T06:29:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab0a00640 1 -- 172.21.15.59:0/2519156461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0032810 con 0x7ffab40d0cc0 2024-04-24T06:29:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 -- 172.21.15.59:0/2519156461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab40d0cc0 msgr2=0x7ffab406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 --2- 172.21.15.59:0/2519156461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab40d0cc0 0x7ffab406f130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097d0 tx=0x7ffaa002a210 comp rx=0 tx=0).stop 2024-04-24T06:29:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 -- 172.21.15.59:0/2519156461 shutdown_connections 2024-04-24T06:29:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 --2- 172.21.15.59:0/2519156461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab40d0cc0 0x7ffab406f130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 -- 172.21.15.59:0/2519156461 >> 172.21.15.59:0/2519156461 conn(0x7ffab4067d90 msgr2=0x7ffab406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 -- 172.21.15.59:0/2519156461 shutdown_connections 2024-04-24T06:29:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.852+0000 7ffab9200640 1 -- 172.21.15.59:0/2519156461 wait complete. 2024-04-24T06:29:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.854+0000 7ffab9200640 1 Processor -- start 2024-04-24T06:29:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.854+0000 7ffab9200640 1 -- start start 2024-04-24T06:29:17.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab40d0cc0 0x7ffab41e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffab4145e10 0x7ffab40ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41461f0 con 0x7ffab4071d70 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4072200 con 0x7ffab40d0cc0 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.855+0000 7ffab9200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffab41e8400 con 0x7ffab4145e10 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37316/0 (socket says 172.21.15.59:37316) 2024-04-24T06:29:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 -- 172.21.15.59:0/2232362570 learned_addr learned my addr 172.21.15.59:0/2232362570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab2800640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffab4145e10 0x7ffab40ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1400640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab40d0cc0 0x7ffab41e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 -- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffab4145e10 msgr2=0x7ffab40ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffab4145e10 0x7ffab40ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 -- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab40d0cc0 msgr2=0x7ffab41e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab40d0cc0 0x7ffab41e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.856+0000 7ffab1e00640 1 -- 172.21.15.59:0/2232362570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41e86e0 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffab1e00640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c430 tx=0x7ffaa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40074e0 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa40047e0 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004bd0 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41e89c0 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.857+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41e8110 con 0x7ffab4071d70 2024-04-24T06:29:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.858+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab4068a80 con 0x7ffab4071d70 2024-04-24T06:29:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.859+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffaa4011070 con 0x7ffab4071d70 2024-04-24T06:29:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.861+0000 7ffaaa000640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 0x7ffa9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.861+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffaa408b710 con 0x7ffab4071d70 2024-04-24T06:29:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.861+0000 7ffab1400640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 0x7ffa9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.862+0000 7ffab1400640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 0x7ffa9406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0034000 tx=0x7ffaa002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:17.865+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffaa4059bd0 con 0x7ffab4071d70 2024-04-24T06:29:18.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.078+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffab40d1190 con 0x7ffab4071d70 2024-04-24T06:29:18.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.080+0000 7ffaaa000640 1 -- 172.21.15.59:0/2232362570 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ffaa40594d0 con 0x7ffab4071d70 2024-04-24T06:29:18.082 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 msgr2=0x7ffa9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 0x7ffa9406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0034000 tx=0x7ffaa002e000 comp rx=0 tx=0).stop 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 msgr2=0x7ffab40ab230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400c430 tx=0x7ffaa400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 shutdown_connections 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa9406c8f0 0x7ffa9406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffab4145e10 0x7ffab40ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffab40d0cc0 0x7ffab41e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 --2- 172.21.15.59:0/2232362570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffab4071d70 0x7ffab40ab230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 >> 172.21.15.59:0/2232362570 conn(0x7ffab4067d90 msgr2=0x7ffab40a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 shutdown_connections 2024-04-24T06:29:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.083+0000 7ffab9200640 1 -- 172.21.15.59:0/2232362570 wait complete. 2024-04-24T06:29:18.104 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.321+0000 7f5fb5200640 1 Processor -- start 2024-04-24T06:29:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.322+0000 7f5fb5200640 1 -- start start 2024-04-24T06:29:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fb015d810 0x7f5fb00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fb00d0b40 0x7f5fb0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb00d0f70 con 0x7f5fb00d0b40 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb015cab0 con 0x7f5fb015d810 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.323+0000 7f5fb5200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb0160e40 con 0x7f5fb01567e0 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fb00d0b40 0x7f5fb0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fb00d0b40 0x7f5fb0160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37322/0 (socket says 172.21.15.59:37322) 2024-04-24T06:29:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 -- 172.21.15.59:0/3106448982 learned_addr learned my addr 172.21.15.59:0/3106448982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5faf400640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5faea00640 1 --2- 172.21.15.59:0/3106448982 >> v2:172.21.15.66:3300/0 conn(0x7f5fb015d810 0x7f5fb00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 -- 172.21.15.59:0/3106448982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 msgr2=0x7f5fb0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 -- 172.21.15.59:0/3106448982 >> v2:172.21.15.66:3300/0 conn(0x7f5fb015d810 msgr2=0x7f5fb00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 --2- 172.21.15.59:0/3106448982 >> v2:172.21.15.66:3300/0 conn(0x7f5fb015d810 0x7f5fb00d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.324+0000 7f5fafe00640 1 -- 172.21.15.59:0/3106448982 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb015cd30 con 0x7f5fb00d0b40 2024-04-24T06:29:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.325+0000 7f5fafe00640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb00d0b40 0x7f5fb0160110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00b5660 tx=0x7f5fa802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad3e1a4cdb71647 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fae000640 1 -- 172.21.15.59:0/3106448982 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8036070 con 0x7f5fb00d0b40 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fae000640 1 -- 172.21.15.59:0/3106448982 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8031600 con 0x7f5fb00d0b40 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fae000640 1 -- 172.21.15.59:0/3106448982 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80319d0 con 0x7f5fb00d0b40 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 -- 172.21.15.59:0/3106448982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb00d0b40 msgr2=0x7f5fb0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb00d0b40 0x7f5fb0160110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00b5660 tx=0x7f5fa802a6a0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 -- 172.21.15.59:0/3106448982 shutdown_connections 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb00d0b40 0x7f5fb0160110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 --2- 172.21.15.59:0/3106448982 >> v2:172.21.15.66:3300/0 conn(0x7f5fb015d810 0x7f5fb00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 --2- 172.21.15.59:0/3106448982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 -- 172.21.15.59:0/3106448982 >> 172.21.15.59:0/3106448982 conn(0x7f5fb006bdf0 msgr2=0x7f5fb006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 -- 172.21.15.59:0/3106448982 shutdown_connections 2024-04-24T06:29:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.326+0000 7f5fb5200640 1 -- 172.21.15.59:0/3106448982 wait complete. 2024-04-24T06:29:18.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.328+0000 7f5fb5200640 1 Processor -- start 2024-04-24T06:29:18.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.328+0000 7f5fb5200640 1 -- start start 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb00d0b40 0x7f5fb009a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb009dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0160650 con 0x7f5fb015d810 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb015d2f0 con 0x7f5fb00d0b40 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.329+0000 7f5fb5200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb01607d0 con 0x7f5fb01567e0 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37324/0 (socket says 172.21.15.59:37324) 2024-04-24T06:29:18.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 -- 172.21.15.59:0/1941586540 learned_addr learned my addr 172.21.15.59:0/1941586540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5faf400640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb00d0b40 0x7f5fb009a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5faea00640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb009dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 -- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 msgr2=0x7f5fb009dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb009dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 -- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb00d0b40 msgr2=0x7f5fb009a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb00d0b40 0x7f5fb009a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.330+0000 7f5fafe00640 1 -- 172.21.15.59:0/1941586540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb009d700 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fafe00640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802ac90 tx=0x7f5fa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa803f020 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8037040 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8036070 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb014fb80 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.331+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb0152260 con 0x7f5fb015d810 2024-04-24T06:29:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.332+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb004fd90 con 0x7f5fb015d810 2024-04-24T06:29:18.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.333+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fa8042020 con 0x7f5fb015d810 2024-04-24T06:29:18.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.334+0000 7f5fa7400640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 0x7f5f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.334+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5fa80abfe0 con 0x7f5fb015d810 2024-04-24T06:29:18.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.334+0000 7f5faf400640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 0x7f5f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.335+0000 7f5faf400640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 0x7f5f8c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006fd0 tx=0x7f5fa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.339+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fa8049030 con 0x7f5fb015d810 2024-04-24T06:29:18.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.545+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5fb009d8d0 con 0x7f5fb015d810 2024-04-24T06:29:18.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.546+0000 7f5fa7400640 1 -- 172.21.15.59:0/1941586540 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5fa807a460 con 0x7f5fb015d810 2024-04-24T06:29:18.548 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:18.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.548+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 msgr2=0x7f5f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.548+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 0x7f5f8c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006fd0 tx=0x7f5fa0008040 comp rx=0 tx=0).stop 2024-04-24T06:29:18.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.548+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 msgr2=0x7f5fb014f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.548+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fa802ac90 tx=0x7f5fa8004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.548+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 shutdown_connections 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f8c06c8f0 0x7f5f8c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fb015d810 0x7f5fb014f160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fb01567e0 0x7f5fb009dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 --2- 172.21.15.59:0/1941586540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fb00d0b40 0x7f5fb009a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 >> 172.21.15.59:0/1941586540 conn(0x7f5fb006bdf0 msgr2=0x7f5fb015dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 shutdown_connections 2024-04-24T06:29:18.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.549+0000 7f5fb5200640 1 -- 172.21.15.59:0/1941586540 wait complete. 2024-04-24T06:29:18.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.785+0000 7f73fa600640 1 Processor -- start 2024-04-24T06:29:18.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.785+0000 7f73fa600640 1 -- start start 2024-04-24T06:29:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.786+0000 7f73fa600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.786+0000 7f73fa600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f73f409ba00 0x7f73f40d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.786+0000 7f73fa600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f40cd2b0 con 0x7f73f40b3820 2024-04-24T06:29:18.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.786+0000 7f73fa600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f73f40d06e0 con 0x7f73f409ba00 2024-04-24T06:29:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.786+0000 7f73fa600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73f40cd550 con 0x7f73f409b110 2024-04-24T06:29:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.787+0000 7f73f3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.787+0000 7f73f3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:48070/0 (socket says 172.21.15.59:48070) 2024-04-24T06:29:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.787+0000 7f73f3e00640 1 -- 172.21.15.59:0/768828793 learned_addr learned my addr 172.21.15.59:0/768828793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.787+0000 7f73f9c00640 1 --2- 172.21.15.59:0/768828793 >> v2:172.21.15.66:3300/0 conn(0x7f73f409ba00 0x7f73f40d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.788+0000 7f73f3e00640 1 -- 172.21.15.59:0/768828793 >> v2:172.21.15.66:3300/0 conn(0x7f73f409ba00 msgr2=0x7f73f40d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.788+0000 7f73f3e00640 1 --2- 172.21.15.59:0/768828793 >> v2:172.21.15.66:3300/0 conn(0x7f73f409ba00 0x7f73f40d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.788+0000 7f73f3e00640 1 -- 172.21.15.59:0/768828793 >> v1:172.21.15.66:6789/0 conn(0x7f73f40b3820 legacy=0x7f73f409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.788+0000 7f73f3400640 1 -- 172.21.15.59:0/768828793 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483960603 0 0) 0x7f73f40cd2b0 con 0x7f73f40b3820 2024-04-24T06:29:18.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.788+0000 7f73f3e00640 1 -- 172.21.15.59:0/768828793 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f40cd830 con 0x7f73f409b110 2024-04-24T06:29:18.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.789+0000 7f73f3e00640 1 --2- 172.21.15.59:0/768828793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009870 tx=0x7f73dc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3112e0aedc0857 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.790+0000 7f73f3400640 1 -- 172.21.15.59:0/768828793 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc037070 con 0x7f73f409b110 2024-04-24T06:29:18.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.790+0000 7f73f3400640 1 -- 172.21.15.59:0/768828793 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc032440 con 0x7f73f409b110 2024-04-24T06:29:18.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.790+0000 7f73f3400640 1 -- 172.21.15.59:0/768828793 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc032810 con 0x7f73f409b110 2024-04-24T06:29:18.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 -- 172.21.15.59:0/768828793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 msgr2=0x7f73f40d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 --2- 172.21.15.59:0/768828793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009870 tx=0x7f73dc02a1c0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 -- 172.21.15.59:0/768828793 shutdown_connections 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 --2- 172.21.15.59:0/768828793 >> v2:172.21.15.66:3300/0 conn(0x7f73f409ba00 0x7f73f40d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 --2- 172.21.15.59:0/768828793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f40d08c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.791+0000 7f73fa600640 1 -- 172.21.15.59:0/768828793 >> 172.21.15.59:0/768828793 conn(0x7f73f40b3ef0 msgr2=0x7f73f40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.792+0000 7f73fa600640 1 -- 172.21.15.59:0/768828793 shutdown_connections 2024-04-24T06:29:18.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.792+0000 7f73fa600640 1 -- 172.21.15.59:0/768828793 wait complete. 2024-04-24T06:29:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.794+0000 7f73fa600640 1 Processor -- start 2024-04-24T06:29:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.794+0000 7f73fa600640 1 -- start start 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f40b3820 0x7f73f4156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40ce0d0 con 0x7f73f409ba00 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73f40cddf0 con 0x7f73f40b3820 2024-04-24T06:29:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.795+0000 7f73fa600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73f40ce5f0 con 0x7f73f409b110 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37340/0 (socket says 172.21.15.59:37340) 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 -- 172.21.15.59:0/101222948 learned_addr learned my addr 172.21.15.59:0/101222948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f9200640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f9c00640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f40b3820 0x7f73f4156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 -- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 msgr2=0x7f73f4153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f4153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 -- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f40b3820 msgr2=0x7f73f4156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f40b3820 0x7f73f4156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.796+0000 7f73f3e00640 1 -- 172.21.15.59:0/101222948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f40c7560 con 0x7f73f409ba00 2024-04-24T06:29:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73f3e00640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000c00 tx=0x7f73dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc037040 con 0x7f73f409ba00 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc042070 con 0x7f73f409ba00 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc030940 con 0x7f73f409ba00 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f40c7840 con 0x7f73f409ba00 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.797+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f40c7e00 con 0x7f73f409ba00 2024-04-24T06:29:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.798+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f404fd90 con 0x7f73f409ba00 2024-04-24T06:29:18.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.800+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73dc02ad10 con 0x7f73f409ba00 2024-04-24T06:29:18.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.801+0000 7f73f0c00640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 0x7f73d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:18.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.801+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f73dc0abff0 con 0x7f73f409ba00 2024-04-24T06:29:18.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.802+0000 7f73f9200640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 0x7f73d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:18.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.803+0000 7f73f9200640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 0x7f73d006ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73e40012e0 tx=0x7f73e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:18.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:18.806+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73dc07a450 con 0x7f73f409ba00 2024-04-24T06:29:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.013+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73f40cc930 con 0x7f73f409ba00 2024-04-24T06:29:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.016+0000 7f73f0c00640 1 -- 172.21.15.59:0/101222948 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f73dc079d50 con 0x7f73f409ba00 2024-04-24T06:29:19.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:19.018 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.018+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 msgr2=0x7f73d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.018+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 0x7f73d006ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73e40012e0 tx=0x7f73e4009040 comp rx=0 tx=0).stop 2024-04-24T06:29:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.018+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 msgr2=0x7f73f40c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.018+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000c00 tx=0x7f73dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 shutdown_connections 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f73d006c920 0x7f73d006ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73f40b3820 0x7f73f4156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f409ba00 0x7f73f40c94d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 --2- 172.21.15.59:0/101222948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73f409b110 0x7f73f4153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 >> 172.21.15.59:0/101222948 conn(0x7f73f40b3ef0 msgr2=0x7f73f41609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 shutdown_connections 2024-04-24T06:29:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.019+0000 7f73fa600640 1 -- 172.21.15.59:0/101222948 wait complete. 2024-04-24T06:29:19.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:19.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.251+0000 7ff056200640 1 Processor -- start 2024-04-24T06:29:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.251+0000 7ff056200640 1 -- start start 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.252+0000 7ff056200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff0501567e0 0x7ff050156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff056200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff0500d0d10 0x7ff050160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff056200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff05015cab0 con 0x7ff05015d810 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff056200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff050160e40 con 0x7ff0501567e0 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff056200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff05015cd30 con 0x7ff0500d0d10 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff0501567e0 0x7ff050156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff0501567e0 0x7ff050156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37158/0 (socket says 172.21.15.59:37158) 2024-04-24T06:29:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 -- 172.21.15.59:0/3347728776 learned_addr learned my addr 172.21.15.59:0/3347728776 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 -- 172.21.15.59:0/3347728776 >> v2:172.21.15.66:3300/0 conn(0x7ff0500d0d10 msgr2=0x7ff050160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 --2- 172.21.15.59:0/3347728776 >> v2:172.21.15.66:3300/0 conn(0x7ff0500d0d10 0x7ff050160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 -- 172.21.15.59:0/3347728776 >> v1:172.21.15.66:6789/0 conn(0x7ff05015d810 legacy=0x7ff0500d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.253+0000 7ff054e00640 1 -- 172.21.15.59:0/3347728776 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05015d010 con 0x7ff0501567e0 2024-04-24T06:29:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.255+0000 7ff054e00640 1 --2- 172.21.15.59:0/3347728776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0501567e0 0x7ff050156bc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b4cb0 tx=0x7ff04002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77ce4adf322f44b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.255+0000 7ff04f400640 1 -- 172.21.15.59:0/3347728776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040036070 con 0x7ff0501567e0 2024-04-24T06:29:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.255+0000 7ff04f400640 1 -- 172.21.15.59:0/3347728776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff040031600 con 0x7ff0501567e0 2024-04-24T06:29:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.255+0000 7ff04f400640 1 -- 172.21.15.59:0/3347728776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400319d0 con 0x7ff0501567e0 2024-04-24T06:29:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 -- 172.21.15.59:0/3347728776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0501567e0 msgr2=0x7ff050156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 --2- 172.21.15.59:0/3347728776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0501567e0 0x7ff050156bc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b4cb0 tx=0x7ff04002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 -- 172.21.15.59:0/3347728776 shutdown_connections 2024-04-24T06:29:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 --2- 172.21.15.59:0/3347728776 >> v2:172.21.15.66:3300/0 conn(0x7ff0500d0d10 0x7ff050160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 --2- 172.21.15.59:0/3347728776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0501567e0 0x7ff050156bc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 -- 172.21.15.59:0/3347728776 >> 172.21.15.59:0/3347728776 conn(0x7ff05006bdf0 msgr2=0x7ff05006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 -- 172.21.15.59:0/3347728776 shutdown_connections 2024-04-24T06:29:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.256+0000 7ff056200640 1 -- 172.21.15.59:0/3347728776 wait complete. 2024-04-24T06:29:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.258+0000 7ff056200640 1 Processor -- start 2024-04-24T06:29:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.258+0000 7ff056200640 1 -- start start 2024-04-24T06:29:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0501567e0 0x7ff05006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05015d810 0x7ff050157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501607d0 con 0x7ff0500d0d10 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff050160650 con 0x7ff05015d810 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff056200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff050160950 con 0x7ff0501567e0 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.259+0000 7ff054e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37164/0 (socket says 172.21.15.59:37164) 2024-04-24T06:29:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 -- 172.21.15.59:0/834604339 learned_addr learned my addr 172.21.15.59:0/834604339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 -- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0501567e0 msgr2=0x7ff05006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0501567e0 0x7ff05006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 -- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05015d810 msgr2=0x7ff050157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05015d810 0x7ff050157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 -- 172.21.15.59:0/834604339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0500cf0d0 con 0x7ff0500d0d10 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.260+0000 7ff054e00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff0400093e0 tx=0x7ff040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.261+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040041020 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.261+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff040007800 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.261+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040036040 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.261+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05006efc0 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.261+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05006f580 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.262+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05004fd90 con 0x7ff0500d0d10 2024-04-24T06:29:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.263+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff040044020 con 0x7ff0500d0d10 2024-04-24T06:29:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.264+0000 7ff04cc00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 0x7ff03006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.264+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff0400acb40 con 0x7ff0500d0d10 2024-04-24T06:29:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.264+0000 7ff04fe00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 0x7ff03006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.265+0000 7ff04fe00640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 0x7ff03006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff050149590 tx=0x7ff03c007990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.269+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff04004b030 con 0x7ff0500d0d10 2024-04-24T06:29:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.473+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff050068db0 con 0x7ff0500d0d10 2024-04-24T06:29:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.474+0000 7ff04cc00640 1 -- 172.21.15.59:0/834604339 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff05015cab0 con 0x7ff0500d0d10 2024-04-24T06:29:19.476 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.476+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 msgr2=0x7ff03006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 0x7ff03006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff050149590 tx=0x7ff03c007990 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 msgr2=0x7ff0500ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff0400093e0 tx=0x7ff040004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 shutdown_connections 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff03006c8f0 0x7ff03006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05015d810 0x7ff050157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0501567e0 0x7ff05006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 --2- 172.21.15.59:0/834604339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0500d0d10 0x7ff0500ceb90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 >> 172.21.15.59:0/834604339 conn(0x7ff05006bdf0 msgr2=0x7ff05015a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 shutdown_connections 2024-04-24T06:29:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.477+0000 7ff056200640 1 -- 172.21.15.59:0/834604339 wait complete. 2024-04-24T06:29:19.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:19.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.714+0000 7f7e97e00640 1 Processor -- start 2024-04-24T06:29:19.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.715+0000 7f7e97e00640 1 -- start start 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.716+0000 7f7e97e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e980d0d10 0x7f7e98156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.716+0000 7f7e97e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e9815d810 0x7f7e98160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.716+0000 7f7e97e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9815cab0 con 0x7f7e981567e0 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.716+0000 7f7e97e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7e98160e40 con 0x7f7e9815d810 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.716+0000 7f7e97e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9815cd30 con 0x7f7e980d0d10 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e9815d810 0x7f7e98160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e9815d810 0x7f7e98160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37180/0 (socket says 172.21.15.59:37180) 2024-04-24T06:29:19.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 -- 172.21.15.59:0/3175882725 learned_addr learned my addr 172.21.15.59:0/3175882725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e96000640 1 --2- 172.21.15.59:0/3175882725 >> v2:172.21.15.66:3300/0 conn(0x7f7e980d0d10 0x7f7e98156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 -- 172.21.15.59:0/3175882725 >> v2:172.21.15.66:3300/0 conn(0x7f7e980d0d10 msgr2=0x7f7e98156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 --2- 172.21.15.59:0/3175882725 >> v2:172.21.15.66:3300/0 conn(0x7f7e980d0d10 0x7f7e98156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 -- 172.21.15.59:0/3175882725 >> v1:172.21.15.66:6789/0 conn(0x7f7e981567e0 legacy=0x7f7e980d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.717+0000 7f7e97400640 1 -- 172.21.15.59:0/3175882725 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9815d010 con 0x7f7e9815d810 2024-04-24T06:29:19.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.718+0000 7f7e97400640 1 --2- 172.21.15.59:0/3175882725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e98160110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e98074140 tx=0x7f7e9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7d75f8afa55383 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e95600640 1 -- 172.21.15.59:0/3175882725 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90036070 con 0x7f7e9815d810 2024-04-24T06:29:19.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e95600640 1 -- 172.21.15.59:0/3175882725 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e90031600 con 0x7f7e9815d810 2024-04-24T06:29:19.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e95600640 1 -- 172.21.15.59:0/3175882725 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900319d0 con 0x7f7e9815d810 2024-04-24T06:29:19.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 -- 172.21.15.59:0/3175882725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 msgr2=0x7f7e98160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 --2- 172.21.15.59:0/3175882725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e98160110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e98074140 tx=0x7f7e9002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 -- 172.21.15.59:0/3175882725 shutdown_connections 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 --2- 172.21.15.59:0/3175882725 >> v2:172.21.15.66:3300/0 conn(0x7f7e980d0d10 0x7f7e98156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 --2- 172.21.15.59:0/3175882725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e98160110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.719+0000 7f7e97e00640 1 -- 172.21.15.59:0/3175882725 >> 172.21.15.59:0/3175882725 conn(0x7f7e9806bdf0 msgr2=0x7f7e9806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.720+0000 7f7e97e00640 1 -- 172.21.15.59:0/3175882725 shutdown_connections 2024-04-24T06:29:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.720+0000 7f7e97e00640 1 -- 172.21.15.59:0/3175882725 wait complete. 2024-04-24T06:29:19.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.722+0000 7f7e97e00640 1 Processor -- start 2024-04-24T06:29:19.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.722+0000 7f7e97e00640 1 -- start start 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980d0d10 0x7f7e98146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e981567e0 0x7f7e9814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e981607d0 con 0x7f7e9815d810 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e98160650 con 0x7f7e980d0d10 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e98160950 con 0x7f7e981567e0 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.723+0000 7f7e97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37184/0 (socket says 172.21.15.59:37184) 2024-04-24T06:29:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 -- 172.21.15.59:0/356599256 learned_addr learned my addr 172.21.15.59:0/356599256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e96a00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980d0d10 0x7f7e98146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e96000640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e981567e0 0x7f7e9814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 -- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e981567e0 msgr2=0x7f7e9814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e981567e0 0x7f7e9814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 -- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980d0d10 msgr2=0x7f7e98146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980d0d10 0x7f7e98146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 -- 172.21.15.59:0/356599256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98158540 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.724+0000 7f7e97400640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009f40 tx=0x7f7e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.725+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90036050 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.725+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e90042070 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.725+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90009510 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.725+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e98158820 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.725+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e98158de0 con 0x7f7e9815d810 2024-04-24T06:29:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.726+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e980b6410 con 0x7f7e9815d810 2024-04-24T06:29:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.727+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e90007bd0 con 0x7f7e9815d810 2024-04-24T06:29:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.728+0000 7f7e8f400640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.728+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e900ac980 con 0x7f7e9815d810 2024-04-24T06:29:19.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.728+0000 7f7e96a00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.729+0000 7f7e96a00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 0x7f7e7806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e88004ea0 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.733+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e980b6410 con 0x7f7e9815d810 2024-04-24T06:29:19.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.938+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e9804fd90 con 0x7f7e9815d810 2024-04-24T06:29:19.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.939+0000 7f7e8f400640 1 -- 172.21.15.59:0/356599256 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7e9815cab0 con 0x7f7e9815d810 2024-04-24T06:29:19.941 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 msgr2=0x7f7e7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 0x7f7e7806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e88004ea0 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 msgr2=0x7f7e9814fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009f40 tx=0x7f7e90004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 shutdown_connections 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e7806c8f0 0x7f7e7806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e9815d810 0x7f7e9814fbe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e981567e0 0x7f7e9814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 --2- 172.21.15.59:0/356599256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e980d0d10 0x7f7e98146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 >> 172.21.15.59:0/356599256 conn(0x7f7e9806bdf0 msgr2=0x7f7e9815dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 shutdown_connections 2024-04-24T06:29:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:19.942+0000 7f7e97e00640 1 -- 172.21.15.59:0/356599256 wait complete. 2024-04-24T06:29:19.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.180+0000 7f0148200640 1 Processor -- start 2024-04-24T06:29:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.181+0000 7f0148200640 1 -- start start 2024-04-24T06:29:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0140075370 0x7f014006e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f0140154af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 0x7f014006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f01400b4fe0 con 0x7f0140075370 2024-04-24T06:29:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f014006ee50 con 0x7f01400b3c80 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.182+0000 7f0148200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f014015f3b0 con 0x7f01400b3820 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0140075370 0x7f014006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0140075370 0x7f014006e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37196/0 (socket says 172.21.15.59:37196) 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 -- 172.21.15.59:0/3765412765 learned_addr learned my addr 172.21.15.59:0/3765412765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0144e00640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f0140154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0146200640 1 --2- 172.21.15.59:0/3765412765 >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 0x7f014006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 -- 172.21.15.59:0/3765412765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 msgr2=0x7f0140154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f0140154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 -- 172.21.15.59:0/3765412765 >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 msgr2=0x7f014006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 --2- 172.21.15.59:0/3765412765 >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 0x7f014006b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0145800640 1 -- 172.21.15.59:0/3765412765 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014015e830 con 0x7f0140075370 2024-04-24T06:29:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.183+0000 7f0146200640 1 --2- 172.21.15.59:0/3765412765 >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 0x7f014006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.184+0000 7f0145800640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0140075370 0x7f014006e910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01400b5c70 tx=0x7f013402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1260c3e8d43930db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.185+0000 7f013fe00640 1 -- 172.21.15.59:0/3765412765 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134037070 con 0x7f0140075370 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.185+0000 7f013fe00640 1 -- 172.21.15.59:0/3765412765 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0134032440 con 0x7f0140075370 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.185+0000 7f013fe00640 1 -- 172.21.15.59:0/3765412765 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134032810 con 0x7f0140075370 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.185+0000 7f0148200640 1 -- 172.21.15.59:0/3765412765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0140075370 msgr2=0x7f014006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.185+0000 7f0148200640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0140075370 0x7f014006e910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01400b5c70 tx=0x7f013402a210 comp rx=0 tx=0).stop 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 -- 172.21.15.59:0/3765412765 shutdown_connections 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 --2- 172.21.15.59:0/3765412765 >> v2:172.21.15.66:3300/0 conn(0x7f01400b3c80 0x7f014006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f0140154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 --2- 172.21.15.59:0/3765412765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0140075370 0x7f014006e910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 -- 172.21.15.59:0/3765412765 >> 172.21.15.59:0/3765412765 conn(0x7f01400b2520 msgr2=0x7f01400b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 -- 172.21.15.59:0/3765412765 shutdown_connections 2024-04-24T06:29:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.186+0000 7f0148200640 1 -- 172.21.15.59:0/3765412765 wait complete. 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.188+0000 7f0148200640 1 Processor -- start 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.188+0000 7f0148200640 1 -- start start 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0140075370 0x7f0140149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f01401482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f014015f0d0 con 0x7f01400b3c80 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f014015edf0 con 0x7f0140075370 2024-04-24T06:29:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0148200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f014015e470 con 0x7f01400b3820 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37212/0 (socket says 172.21.15.59:37212) 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 -- 172.21.15.59:0/3018139623 learned_addr learned my addr 172.21.15.59:0/3018139623 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0145800640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0140075370 0x7f0140149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0144e00640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f01401482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 -- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 msgr2=0x7f01401482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f01401482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 -- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0140075370 msgr2=0x7f0140149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0140075370 0x7f0140149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.189+0000 7f0146200640 1 -- 172.21.15.59:0/3018139623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014014c400 con 0x7f01400b3c80 2024-04-24T06:29:20.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f0146200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f013800cd80 tx=0x7f013800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800f680 con 0x7f01400b3c80 2024-04-24T06:29:20.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013800f840 con 0x7f01400b3c80 2024-04-24T06:29:20.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800fc30 con 0x7f01400b3c80 2024-04-24T06:29:20.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014014c6e0 con 0x7f01400b3c80 2024-04-24T06:29:20.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.190+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014014bf40 con 0x7f01400b3c80 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.191+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0140068a80 con 0x7f01400b3c80 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.192+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01380213f0 con 0x7f01400b3c80 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.193+0000 7f013d600640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 0x7f012006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.194+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f013808c8e0 con 0x7f01400b3c80 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.194+0000 7f0145800640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 0x7f012006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.194+0000 7f0145800640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 0x7f012006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0134034000 tx=0x7f013402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.198+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0138014030 con 0x7f01400b3c80 2024-04-24T06:29:20.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.402+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0140152710 con 0x7f01400b3c80 2024-04-24T06:29:20.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.403+0000 7f013d600640 1 -- 172.21.15.59:0/3018139623 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f013805a6b0 con 0x7f01400b3c80 2024-04-24T06:29:20.405 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 msgr2=0x7f012006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 0x7f012006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0134034000 tx=0x7f013402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 msgr2=0x7f014014a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f013800cd80 tx=0x7f013800a650 comp rx=0 tx=0).stop 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 shutdown_connections 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f012006c8f0 0x7f012006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01400b3c80 0x7f014014a8e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01400b3820 0x7f01401482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 --2- 172.21.15.59:0/3018139623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0140075370 0x7f0140149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.406+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 >> 172.21.15.59:0/3018139623 conn(0x7f01400b2520 msgr2=0x7f014006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:20.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.407+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 shutdown_connections 2024-04-24T06:29:20.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.407+0000 7f0148200640 1 -- 172.21.15.59:0/3018139623 wait complete. 2024-04-24T06:29:20.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:20.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.641+0000 7f9394800640 1 Processor -- start 2024-04-24T06:29:20.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.642+0000 7f9394800640 1 -- start start 2024-04-24T06:29:20.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c1454e0 0x7f938c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f938c0cd020 0x7f938c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f938c0b3e40 0x7f938c160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f938c0cddc0 con 0x7f938c0cd020 2024-04-24T06:29:20.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f938c0cd520 con 0x7f938c0b3e40 2024-04-24T06:29:20.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.643+0000 7f9394800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f938c0cd800 con 0x7f938c1454e0 2024-04-24T06:29:20.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f938c0cd020 0x7f938c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f938c0cd020 0x7f938c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37224/0 (socket says 172.21.15.59:37224) 2024-04-24T06:29:20.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 -- 172.21.15.59:0/791058601 learned_addr learned my addr 172.21.15.59:0/791058601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:20.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9392800640 1 --2- 172.21.15.59:0/791058601 >> v2:172.21.15.66:3300/0 conn(0x7f938c0b3e40 0x7f938c160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391e00640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c1454e0 0x7f938c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 -- 172.21.15.59:0/791058601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c1454e0 msgr2=0x7f938c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:20.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c1454e0 0x7f938c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 -- 172.21.15.59:0/791058601 >> v2:172.21.15.66:3300/0 conn(0x7f938c0b3e40 msgr2=0x7f938c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:20.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 --2- 172.21.15.59:0/791058601 >> v2:172.21.15.66:3300/0 conn(0x7f938c0b3e40 0x7f938c160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.644+0000 7f9391400640 1 -- 172.21.15.59:0/791058601 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c0cdae0 con 0x7f938c0cd020 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.645+0000 7f9391400640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0cd020 0x7f938c0b3900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f938c0d0270 tx=0x7f937c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80db38006d087fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9390a00640 1 -- 172.21.15.59:0/791058601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c037070 con 0x7f938c0cd020 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9390a00640 1 -- 172.21.15.59:0/791058601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937c032440 con 0x7f938c0cd020 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9390a00640 1 -- 172.21.15.59:0/791058601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c032810 con 0x7f938c0cd020 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 -- 172.21.15.59:0/791058601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0cd020 msgr2=0x7f938c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0cd020 0x7f938c0b3900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f938c0d0270 tx=0x7f937c02a140 comp rx=0 tx=0).stop 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 -- 172.21.15.59:0/791058601 shutdown_connections 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 --2- 172.21.15.59:0/791058601 >> v2:172.21.15.66:3300/0 conn(0x7f938c0b3e40 0x7f938c160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0cd020 0x7f938c0b3900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 --2- 172.21.15.59:0/791058601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c1454e0 0x7f938c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.646+0000 7f9394800640 1 -- 172.21.15.59:0/791058601 >> 172.21.15.59:0/791058601 conn(0x7f938c067ea0 msgr2=0x7f938c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.647+0000 7f9394800640 1 -- 172.21.15.59:0/791058601 shutdown_connections 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.647+0000 7f9394800640 1 -- 172.21.15.59:0/791058601 wait complete. 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.648+0000 7f9394800640 1 Processor -- start 2024-04-24T06:29:20.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.649+0000 7f9394800640 1 -- start start 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.649+0000 7f9394800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.649+0000 7f9394800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c0cd020 0x7f938c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9394800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f938c1454e0 0x7f938c1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9394800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f938c146270 con 0x7f938c0b3e40 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9394800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1460f0 con 0x7f938c1454e0 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9394800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f938c160e50 con 0x7f938c0cd020 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37230/0 (socket says 172.21.15.59:37230) 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 -- 172.21.15.59:0/1658069324 learned_addr learned my addr 172.21.15.59:0/1658069324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9392800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f938c1454e0 0x7f938c1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391400640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c0cd020 0x7f938c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 -- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c0cd020 msgr2=0x7f938c13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c0cd020 0x7f938c13eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 -- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f938c1454e0 msgr2=0x7f938c1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f938c1454e0 0x7f938c1f5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.650+0000 7f9391e00640 1 -- 172.21.15.59:0/1658069324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c1444a0 con 0x7f938c0b3e40 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9391e00640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f938000d5f0 tx=0x7f938000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800125f0 con 0x7f938c0b3e40 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9380019070 con 0x7f938c0b3e40 2024-04-24T06:29:20.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800129d0 con 0x7f938c0b3e40 2024-04-24T06:29:20.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c144720 con 0x7f938c0b3e40 2024-04-24T06:29:20.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.651+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c144ce0 con 0x7f938c0b3e40 2024-04-24T06:29:20.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.652+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c068a80 con 0x7f938c0b3e40 2024-04-24T06:29:20.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.653+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9380012b70 con 0x7f938c0b3e40 2024-04-24T06:29:20.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.654+0000 7f9386000640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:20.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.654+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f938008bf30 con 0x7f938c0b3e40 2024-04-24T06:29:20.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.655+0000 7f9391400640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:20.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.655+0000 7f9391400640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 0x7f937006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f937c034000 tx=0x7f937c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:20.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.661+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f938c1444a0 con 0x7f938c0b3e40 2024-04-24T06:29:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.868+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f938c160030 con 0x7f938c0b3e40 2024-04-24T06:29:20.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.871+0000 7f9386000640 1 -- 172.21.15.59:0/1658069324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f938005a2c0 con 0x7f938c0b3e40 2024-04-24T06:29:20.873 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:20.873 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.873+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 msgr2=0x7f937006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 0x7f937006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f937c034000 tx=0x7f937c02e000 comp rx=0 tx=0).stop 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 msgr2=0x7f938c06f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f938000d5f0 tx=0x7f938000dab0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 shutdown_connections 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f938c1454e0 0x7f938c1f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f938c0cd020 0x7f938c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 --2- 172.21.15.59:0/1658069324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938c0b3e40 0x7f938c06f290 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 >> 172.21.15.59:0/1658069324 conn(0x7f938c067ea0 msgr2=0x7f938c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 shutdown_connections 2024-04-24T06:29:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:20.874+0000 7f9394800640 1 -- 172.21.15.59:0/1658069324 wait complete. 2024-04-24T06:29:20.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:20.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:21.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.107+0000 7f440fe00640 1 Processor -- start 2024-04-24T06:29:21.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.107+0000 7f440fe00640 1 -- start start 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f441006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f44100d0600 0x7f44100d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f441015ef20 con 0x7f44100b43a0 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f44100d0fa0 con 0x7f44100d0600 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f441015f1c0 con 0x7f4410155a80 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f44100b43a0 0x7f44100b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54428/0 (socket says 172.21.15.59:54428) 2024-04-24T06:29:21.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.109+0000 7f440f400640 1 -- 172.21.15.59:0/706012905 learned_addr learned my addr 172.21.15.59:0/706012905 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:21.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f4406000640 1 --2- 172.21.15.59:0/706012905 >> v2:172.21.15.66:3300/0 conn(0x7f44100d0600 0x7f44100d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440ea00640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f441006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029696859 0 0) 0x7f441015ef20 con 0x7f44100b43a0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43f4005730 con 0x7f44100b43a0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2082989690 0 0) 0x7f43f4005730 con 0x7f44100b43a0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 msgr2=0x7f441006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f441006f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 >> v2:172.21.15.66:3300/0 conn(0x7f44100d0600 msgr2=0x7f44100d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 --2- 172.21.15.59:0/706012905 >> v2:172.21.15.66:3300/0 conn(0x7f44100d0600 0x7f44100d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441015f4a0 con 0x7f44100b43a0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4408001670 con 0x7f44100b43a0 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 >> v1:172.21.15.59:6789/0 conn(0x7f44100b43a0 legacy=0x7f44100b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43f4005bc0 0x7f43f4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4006a10 0x7f43f4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f43f40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f441015f1c0 con 0x7f43f4006a10 2024-04-24T06:29:21.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43f400dbd0 con 0x7f43f4005bc0 2024-04-24T06:29:21.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.110+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f43f400dd50 con 0x7f4410155a80 2024-04-24T06:29:21.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4006a10 0x7f43f4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f440f400640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f43f40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 -- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 msgr2=0x7f43f40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f43f40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f440ea00640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43f4005bc0 0x7f43f4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 -- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43f4005bc0 msgr2=0x7f43f4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43f4005bc0 0x7f43f4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f4406000640 1 -- 172.21.15.59:0/706012905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f4005730 con 0x7f43f4006a10 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f440f400640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4410155a80 0x7f43f40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.111+0000 7f440ea00640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43f4005bc0 0x7f43f4005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.112+0000 7f4406000640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4006a10 0x7f43f4006e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009ac0 tx=0x7f43f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a5a6f252526a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8036070 con 0x7f43f4006a10 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f8031400 con 0x7f43f4006a10 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440e000640 1 -- 172.21.15.59:0/706012905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8009690 con 0x7f43f4006a10 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440fe00640 1 -- 172.21.15.59:0/706012905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4006a10 msgr2=0x7f43f4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440fe00640 1 --2- 172.21.15.59:0/706012905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43f4006a10 0x7f43f4006e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009ac0 tx=0x7f43f4007a40 comp rx=0 tx=0).stop 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440f400640 1 -- 172.21.15.59:0/706012905 reap_dead start 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440fe00640 1 -- 172.21.15.59:0/706012905 shutdown_connections 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.113+0000 7f440fe00640 1 -- 172.21.15.59:0/706012905 >> 172.21.15.59:0/706012905 conn(0x7f441006bdf0 msgr2=0x7f441006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:21.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.114+0000 7f440fe00640 1 -- 172.21.15.59:0/706012905 shutdown_connections 2024-04-24T06:29:21.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.114+0000 7f440fe00640 1 -- 172.21.15.59:0/706012905 wait complete. 2024-04-24T06:29:21.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.115+0000 7f440fe00640 1 Processor -- start 2024-04-24T06:29:21.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 -- start start 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43f4006a10 0x7f441014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44101509d0 0x7f44100ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f441015fd40 con 0x7f44101e4d40 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f441015fa60 con 0x7f44101509d0 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.116+0000 7f440fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f441006ef80 con 0x7f43f4006a10 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37240/0 (socket says 172.21.15.59:37240) 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 -- 172.21.15.59:0/2148177641 learned_addr learned my addr 172.21.15.59:0/2148177641 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f4406000640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44101509d0 0x7f44100ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440ea00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43f4006a10 0x7f441014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 -- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43f4006a10 msgr2=0x7f441014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43f4006a10 0x7f441014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 -- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44101509d0 msgr2=0x7f44100ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44101509d0 0x7f44100ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.117+0000 7f440f400640 1 -- 172.21.15.59:0/2148177641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44101e5da0 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f440f400640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f441006e700 tx=0x7f440800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408007440 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44080075e0 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44101e5220 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408004a20 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.118+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101e57e0 con 0x7f44101e4d40 2024-04-24T06:29:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.119+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441004fd90 con 0x7f44101e4d40 2024-04-24T06:29:21.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.120+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f440801b020 con 0x7f44101e4d40 2024-04-24T06:29:21.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.122+0000 7f4407400640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.122+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f440808b5c0 con 0x7f44101e4d40 2024-04-24T06:29:21.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.122+0000 7f440ea00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.123+0000 7f440ea00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 0x7f43ec06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4400003be0 tx=0x7f440000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.126+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44080599f0 con 0x7f44101e4d40 2024-04-24T06:29:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.336+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44101f43a0 con 0x7f44101e4d40 2024-04-24T06:29:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.336+0000 7f4407400640 1 -- 172.21.15.59:0/2148177641 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f44080592f0 con 0x7f44101e4d40 2024-04-24T06:29:21.338 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 msgr2=0x7f43ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 0x7f43ec06edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4400003be0 tx=0x7f440000b040 comp rx=0 tx=0).stop 2024-04-24T06:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 msgr2=0x7f44100cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f441006e700 tx=0x7f440800d9d0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 shutdown_connections 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43ec06c8f0 0x7f43ec06edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44101e4d40 0x7f44100cee90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44101509d0 0x7f44100ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 --2- 172.21.15.59:0/2148177641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43f4006a10 0x7f441014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 >> 172.21.15.59:0/2148177641 conn(0x7f441006bdf0 msgr2=0x7f44100ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 shutdown_connections 2024-04-24T06:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.339+0000 7f440fe00640 1 -- 172.21.15.59:0/2148177641 wait complete. 2024-04-24T06:29:21.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.571+0000 7fa1a5200640 1 Processor -- start 2024-04-24T06:29:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.571+0000 7fa1a5200640 1 -- start start 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.572+0000 7fa1a5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a01567e0 0x7fa1a0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa1a5200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa1a015d810 0x7fa1a00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa1a5200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a015cab0 con 0x7fa1a00d0b40 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa1a5200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a00d0f90 con 0x7fa1a01567e0 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa1a5200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a0160e40 con 0x7fa1a015d810 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a01567e0 0x7fa1a0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a01567e0 0x7fa1a0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37252/0 (socket says 172.21.15.59:37252) 2024-04-24T06:29:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 -- 172.21.15.59:0/2236364509 learned_addr learned my addr 172.21.15.59:0/2236364509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 -- 172.21.15.59:0/2236364509 >> v2:172.21.15.66:3300/0 conn(0x7fa1a015d810 msgr2=0x7fa1a00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 --2- 172.21.15.59:0/2236364509 >> v2:172.21.15.66:3300/0 conn(0x7fa1a015d810 0x7fa1a00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 -- 172.21.15.59:0/2236364509 >> v1:172.21.15.66:6789/0 conn(0x7fa1a00d0b40 legacy=0x7fa1a0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.573+0000 7fa19f400640 1 -- 172.21.15.59:0/2236364509 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a015cd30 con 0x7fa1a01567e0 2024-04-24T06:29:21.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa19f400640 1 --2- 172.21.15.59:0/2236364509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a0156bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0074140 tx=0x7fa19002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9705c63b9c1ea98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa19e000640 1 -- 172.21.15.59:0/2236364509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190036070 con 0x7fa1a01567e0 2024-04-24T06:29:21.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa19e000640 1 -- 172.21.15.59:0/2236364509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa190031600 con 0x7fa1a01567e0 2024-04-24T06:29:21.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa19e000640 1 -- 172.21.15.59:0/2236364509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900319d0 con 0x7fa1a01567e0 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa1a5200640 1 -- 172.21.15.59:0/2236364509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 msgr2=0x7fa1a0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.575+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2236364509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a0156bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0074140 tx=0x7fa19002a710 comp rx=0 tx=0).stop 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 -- 172.21.15.59:0/2236364509 shutdown_connections 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2236364509 >> v2:172.21.15.66:3300/0 conn(0x7fa1a015d810 0x7fa1a00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2236364509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a0156bc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 -- 172.21.15.59:0/2236364509 >> 172.21.15.59:0/2236364509 conn(0x7fa1a006bdf0 msgr2=0x7fa1a006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 -- 172.21.15.59:0/2236364509 shutdown_connections 2024-04-24T06:29:21.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.576+0000 7fa1a5200640 1 -- 172.21.15.59:0/2236364509 wait complete. 2024-04-24T06:29:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.578+0000 7fa1a5200640 1 Processor -- start 2024-04-24T06:29:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.578+0000 7fa1a5200640 1 -- start start 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1a00d0b40 0x7fa1a0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a015d810 0x7fa1a0152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0160820 con 0x7fa1a01567e0 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a015d2f0 con 0x7fa1a015d810 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa1a5200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a01609a0 con 0x7fa1a00d0b40 2024-04-24T06:29:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa19ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa19ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37260/0 (socket says 172.21.15.59:37260) 2024-04-24T06:29:21.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.579+0000 7fa19ea00640 1 -- 172.21.15.59:0/2269511705 learned_addr learned my addr 172.21.15.59:0/2269511705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 -- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1a00d0b40 msgr2=0x7fa1a0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1a00d0b40 0x7fa1a0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 -- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a015d810 msgr2=0x7fa1a0152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a015d810 0x7fa1a0152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 -- 172.21.15.59:0/2269511705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a01f5b30 con 0x7fa1a01567e0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa19ea00640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa18800b460 tx=0x7fa18800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188015020 con 0x7fa1a01567e0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1880118b0 con 0x7fa1a01567e0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188011ca0 con 0x7fa1a01567e0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.580+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a0154670 con 0x7fa1a01567e0 2024-04-24T06:29:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.581+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a0151f10 con 0x7fa1a01567e0 2024-04-24T06:29:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.582+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a004fd90 con 0x7fa1a01567e0 2024-04-24T06:29:21.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.583+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa188018020 con 0x7fa1a01567e0 2024-04-24T06:29:21.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.584+0000 7fa197400640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 0x7fa17c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:21.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.584+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa18808bb90 con 0x7fa1a01567e0 2024-04-24T06:29:21.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.584+0000 7fa19f400640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 0x7fa17c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.585+0000 7fa19f400640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 0x7fa17c06ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa190033000 tx=0x7fa19002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.589+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa18801f020 con 0x7fa1a01567e0 2024-04-24T06:29:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.793+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1a0156590 con 0x7fa1a01567e0 2024-04-24T06:29:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.794+0000 7fa197400640 1 -- 172.21.15.59:0/2269511705 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa18800d040 con 0x7fa1a01567e0 2024-04-24T06:29:21.796 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 msgr2=0x7fa17c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 0x7fa17c06ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa190033000 tx=0x7fa19002e000 comp rx=0 tx=0).stop 2024-04-24T06:29:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 msgr2=0x7fa1a01f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa18800b460 tx=0x7fa18800b920 comp rx=0 tx=0).stop 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 shutdown_connections 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17c06c940 0x7fa17c06ee00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1a015d810 0x7fa1a0152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a01567e0 0x7fa1a01f55f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 --2- 172.21.15.59:0/2269511705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1a00d0b40 0x7fa1a0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.796+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 >> 172.21.15.59:0/2269511705 conn(0x7fa1a006bdf0 msgr2=0x7fa1a00af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.797+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 shutdown_connections 2024-04-24T06:29:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:21.797+0000 7fa1a5200640 1 -- 172.21.15.59:0/2269511705 wait complete. 2024-04-24T06:29:21.817 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:22.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.033+0000 7fc0eee00640 1 Processor -- start 2024-04-24T06:29:22.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.033+0000 7fc0eee00640 1 -- start start 2024-04-24T06:29:22.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0e81567e0 0x7fc0e8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 0x7fc0e8160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e80d0f70 con 0x7fc0e81567e0 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e815cab0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.034+0000 7fc0eee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e8160e40 con 0x7fc0e815d810 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0e81567e0 0x7fc0e8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0e81567e0 0x7fc0e8156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37266/0 (socket says 172.21.15.59:37266) 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 -- 172.21.15.59:0/3603211225 learned_addr learned my addr 172.21.15.59:0/3603211225 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 -- 172.21.15.59:0/3603211225 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 msgr2=0x7fc0e80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0ee400640 1 --2- 172.21.15.59:0/3603211225 >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 0x7fc0e8160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0ed000640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e80d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 -- 172.21.15.59:0/3603211225 >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 msgr2=0x7fc0e8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 --2- 172.21.15.59:0/3603211225 >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 0x7fc0e8160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0eda00640 1 -- 172.21.15.59:0/3603211225 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e815cd30 con 0x7fc0e81567e0 2024-04-24T06:29:22.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0ee400640 1 --2- 172.21.15.59:0/3603211225 >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 0x7fc0e8160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:22.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.035+0000 7fc0ed000640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.036+0000 7fc0eda00640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e81567e0 0x7fc0e8156bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8142d50 tx=0x7fc0dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450adf4d056a40ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.037+0000 7fc0e7e00640 1 -- 172.21.15.59:0/3603211225 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc037070 con 0x7fc0e81567e0 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.037+0000 7fc0e7e00640 1 -- 172.21.15.59:0/3603211225 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc032440 con 0x7fc0e81567e0 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.037+0000 7fc0e7e00640 1 -- 172.21.15.59:0/3603211225 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc032810 con 0x7fc0e81567e0 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 -- 172.21.15.59:0/3603211225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e81567e0 msgr2=0x7fc0e8156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e81567e0 0x7fc0e8156bc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8142d50 tx=0x7fc0dc02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 -- 172.21.15.59:0/3603211225 shutdown_connections 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 --2- 172.21.15.59:0/3603211225 >> v2:172.21.15.66:3300/0 conn(0x7fc0e80d0b40 0x7fc0e8160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 --2- 172.21.15.59:0/3603211225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e81567e0 0x7fc0e8156bc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 -- 172.21.15.59:0/3603211225 >> 172.21.15.59:0/3603211225 conn(0x7fc0e806bdf0 msgr2=0x7fc0e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:22.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 -- 172.21.15.59:0/3603211225 shutdown_connections 2024-04-24T06:29:22.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.038+0000 7fc0eee00640 1 -- 172.21.15.59:0/3603211225 wait complete. 2024-04-24T06:29:22.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.040+0000 7fc0eee00640 1 Processor -- start 2024-04-24T06:29:22.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.040+0000 7fc0eee00640 1 -- start start 2024-04-24T06:29:22.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 0x7fc0e814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e81f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8160650 con 0x7fc0e80d0b40 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e815d2f0 con 0x7fc0e81567e0 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e81607d0 con 0x7fc0e815d810 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37278/0 (socket says 172.21.15.59:37278) 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 -- 172.21.15.59:0/1198387771 learned_addr learned my addr 172.21.15.59:0/1198387771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0ed000640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 0x7fc0e814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 -- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 msgr2=0x7fc0e81f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e81f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 -- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 msgr2=0x7fc0e814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 0x7fc0e814ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.042+0000 7fc0eda00640 1 -- 172.21.15.59:0/1198387771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e814f150 con 0x7fc0e80d0b40 2024-04-24T06:29:22.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0ed000640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 0x7fc0e814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:22.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0eda00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc000c00 tx=0x7fc0dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc037040 con 0x7fc0e80d0b40 2024-04-24T06:29:22.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc042070 con 0x7fc0e80d0b40 2024-04-24T06:29:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e815a0b0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc030bb0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.043+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e8159dd0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.045+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e804fd90 con 0x7fc0e80d0b40 2024-04-24T06:29:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.045+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0dc047460 con 0x7fc0e80d0b40 2024-04-24T06:29:22.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.047+0000 7fc0e5600640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 0x7fc0cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.047+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0dc0ac810 con 0x7fc0e80d0b40 2024-04-24T06:29:22.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.047+0000 7fc0ed000640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 0x7fc0cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.048+0000 7fc0ed000640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 0x7fc0cc06ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0e814e070 tx=0x7fc0d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.051+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0dc07aba0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.255+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0e80b5150 con 0x7fc0e80d0b40 2024-04-24T06:29:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.256+0000 7fc0e5600640 1 -- 172.21.15.59:0/1198387771 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc0dc07a4a0 con 0x7fc0e80d0b40 2024-04-24T06:29:22.258 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 msgr2=0x7fc0cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 0x7fc0cc06ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0e814e070 tx=0x7fc0d8008040 comp rx=0 tx=0).stop 2024-04-24T06:29:22.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 msgr2=0x7fc0e814b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc000c00 tx=0x7fc0dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 shutdown_connections 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0cc06c920 0x7fc0cc06ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0e815d810 0x7fc0e81f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0e81567e0 0x7fc0e814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 --2- 172.21.15.59:0/1198387771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0e80d0b40 0x7fc0e814b7c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 >> 172.21.15.59:0/1198387771 conn(0x7fc0e806bdf0 msgr2=0x7fc0e8155460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 shutdown_connections 2024-04-24T06:29:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.259+0000 7fc0eee00640 1 -- 172.21.15.59:0/1198387771 wait complete. 2024-04-24T06:29:22.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:22.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.485+0000 7f10f2600640 1 Processor -- start 2024-04-24T06:29:22.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.485+0000 7f10f2600640 1 -- start start 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f2600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f10ec0b4480 0x7f10ec0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f2600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f2600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ec0b4910 con 0x7f10ec15e8c0 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f2600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f10ec0b5ef0 con 0x7f10ec0d32a0 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f2600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f10ec0b4bf0 con 0x7f10ec0b4480 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37294/0 (socket says 172.21.15.59:37294) 2024-04-24T06:29:22.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.487+0000 7f10f1c00640 1 -- 172.21.15.59:0/1106228356 learned_addr learned my addr 172.21.15.59:0/1106228356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:22.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.488+0000 7f10ebe00640 1 --2- 172.21.15.59:0/1106228356 >> v2:172.21.15.66:3300/0 conn(0x7f10ec0b4480 0x7f10ec0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.488+0000 7f10f1c00640 1 -- 172.21.15.59:0/1106228356 >> v2:172.21.15.66:3300/0 conn(0x7f10ec0b4480 msgr2=0x7f10ec0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.488+0000 7f10f1c00640 1 --2- 172.21.15.59:0/1106228356 >> v2:172.21.15.66:3300/0 conn(0x7f10ec0b4480 0x7f10ec0b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.488+0000 7f10f1c00640 1 -- 172.21.15.59:0/1106228356 >> v1:172.21.15.66:6789/0 conn(0x7f10ec15e8c0 legacy=0x7f10ec0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.488+0000 7f10f1c00640 1 -- 172.21.15.59:0/1106228356 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec0b4ed0 con 0x7f10ec0d32a0 2024-04-24T06:29:22.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.489+0000 7f10f1c00640 1 --2- 172.21.15.59:0/1106228356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f10ec06c630 tx=0x7f10e402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932ff67d9701db4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.489+0000 7f10eb400640 1 -- 172.21.15.59:0/1106228356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4037070 con 0x7f10ec0d32a0 2024-04-24T06:29:22.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.489+0000 7f10eb400640 1 -- 172.21.15.59:0/1106228356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e4032440 con 0x7f10ec0d32a0 2024-04-24T06:29:22.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.489+0000 7f10eb400640 1 -- 172.21.15.59:0/1106228356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4032810 con 0x7f10ec0d32a0 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 -- 172.21.15.59:0/1106228356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0d32a0 msgr2=0x7f10ec0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 --2- 172.21.15.59:0/1106228356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f10ec06c630 tx=0x7f10e402a2c0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 -- 172.21.15.59:0/1106228356 shutdown_connections 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 --2- 172.21.15.59:0/1106228356 >> v2:172.21.15.66:3300/0 conn(0x7f10ec0b4480 0x7f10ec0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 --2- 172.21.15.59:0/1106228356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0cf3a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 -- 172.21.15.59:0/1106228356 >> 172.21.15.59:0/1106228356 conn(0x7f10ec08bf90 msgr2=0x7f10ec0d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 -- 172.21.15.59:0/1106228356 shutdown_connections 2024-04-24T06:29:22.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.490+0000 7f10f2600640 1 -- 172.21.15.59:0/1106228356 wait complete. 2024-04-24T06:29:22.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.493+0000 7f10f2600640 1 Processor -- start 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.493+0000 7f10f2600640 1 -- start start 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10ec15e8c0 0x7f10ec0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec15eca0 con 0x7f10ec0ca630 2024-04-24T06:29:22.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0b5490 con 0x7f10ec0d32a0 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.494+0000 7f10f2600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec15ee20 con 0x7f10ec15e8c0 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37308/0 (socket says 172.21.15.59:37308) 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 -- 172.21.15.59:0/3818628382 learned_addr learned my addr 172.21.15.59:0/3818628382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10ebe00640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10ec15e8c0 0x7f10ec0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1200640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 -- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10ec15e8c0 msgr2=0x7f10ec0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10ec15e8c0 0x7f10ec0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 -- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0d32a0 msgr2=0x7f10ec0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.495+0000 7f10f1c00640 1 -- 172.21.15.59:0/3818628382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec0cd610 con 0x7f10ec0ca630 2024-04-24T06:29:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10f1c00640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f10e40090b0 tx=0x7f10e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4037040 con 0x7f10ec0ca630 2024-04-24T06:29:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e4042070 con 0x7f10ec0ca630 2024-04-24T06:29:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4030bd0 con 0x7f10ec0ca630 2024-04-24T06:29:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec0cd8f0 con 0x7f10ec0ca630 2024-04-24T06:29:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.496+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec0cdeb0 con 0x7f10ec0ca630 2024-04-24T06:29:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.497+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec04fd90 con 0x7f10ec0ca630 2024-04-24T06:29:22.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.498+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10e4047460 con 0x7f10ec0ca630 2024-04-24T06:29:22.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.499+0000 7f10e8c00640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.499+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f10e40ac7a0 con 0x7f10ec0ca630 2024-04-24T06:29:22.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.500+0000 7f10f1200640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.501+0000 7f10f1200640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004e50 tx=0x7f10d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.504+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10e407ab30 con 0x7f10ec0ca630 2024-04-24T06:29:22.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.708+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10ec0ba530 con 0x7f10ec0ca630 2024-04-24T06:29:22.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.712+0000 7f10e8c00640 1 -- 172.21.15.59:0/3818628382 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f10e407a430 con 0x7f10ec0ca630 2024-04-24T06:29:22.714 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:22.715 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:22.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.714+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 msgr2=0x7f10c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.714+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004e50 tx=0x7f10d4009040 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.714+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 msgr2=0x7f10ec0cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.714+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f10e40090b0 tx=0x7f10e40046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 shutdown_connections 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10ec0ca630 0x7f10ec0cc870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10ec15e8c0 0x7f10ec0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 --2- 172.21.15.59:0/3818628382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0d32a0 0x7f10ec0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 >> 172.21.15.59:0/3818628382 conn(0x7f10ec08bf90 msgr2=0x7f10ec0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:22.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 shutdown_connections 2024-04-24T06:29:22.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.715+0000 7f10f2600640 1 -- 172.21.15.59:0/3818628382 wait complete. 2024-04-24T06:29:22.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:22.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:22.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.954+0000 7fc677000640 1 Processor -- start 2024-04-24T06:29:22.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.954+0000 7fc677000640 1 -- start start 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.956+0000 7fc677000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc67006f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.956+0000 7fc677000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700cfc20 con 0x7fc670145e10 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc677000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700cff00 con 0x7fc670071d70 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc677000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc6700d01e0 con 0x7fc6700d0cc0 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc66fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc670145e10 0x7fc6700cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54432/0 (socket says 172.21.15.59:54432) 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc66fe00640 1 -- 172.21.15.59:0/442191764 learned_addr learned my addr 172.21.15.59:0/442191764 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc66f400640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc67006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150037401 0 0) 0x7fc6700cfc20 con 0x7fc670145e10 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.957+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc654003520 con 0x7fc670145e10 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2901467217 0 0) 0x7fc654003520 con 0x7fc670145e10 2024-04-24T06:29:22.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 msgr2=0x7fc67006f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc67006f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 >> v1:172.21.15.66:6789/0 conn(0x7fc670071d70 legacy=0x7fc67015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6700d04c0 con 0x7fc670145e10 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc660001670 con 0x7fc670145e10 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 >> v1:172.21.15.59:6789/0 conn(0x7fc670145e10 legacy=0x7fc6700cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc6540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc654003f60 0x7fc654004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc654004d40 0x7fc6540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6700d01e0 con 0x7fc654003f60 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc65400baa0 con 0x7fc654004d40 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc65400bca0 con 0x7fc6700d0cc0 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.958+0000 7fc66f400640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc654003f60 0x7fc654004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66fe00640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc6540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc675000640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc654004d40 0x7fc6540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66f400640 1 -- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 msgr2=0x7fc6540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66f400640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc6700d0cc0 0x7fc6540039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66f400640 1 -- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc654004d40 msgr2=0x7fc6540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66f400640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc654004d40 0x7fc6540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc66f400640 1 -- 172.21.15.59:0/442191764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654003520 con 0x7fc654003f60 2024-04-24T06:29:22.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.959+0000 7fc675000640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc654004d40 0x7fc6540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.960+0000 7fc66f400640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc654003f60 0x7fc654004800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009ac0 tx=0x7fc654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492d7c2fedb3c789 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.960+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c036070 con 0x7fc654003f60 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.960+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65c031400 con 0x7fc654003f60 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.960+0000 7fc66ea00640 1 -- 172.21.15.59:0/442191764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c009690 con 0x7fc654003f60 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 -- 172.21.15.59:0/442191764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc654003f60 msgr2=0x7fc654004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 --2- 172.21.15.59:0/442191764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc654003f60 0x7fc654004800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009ac0 tx=0x7fc654005830 comp rx=0 tx=0).stop 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc675000640 1 -- 172.21.15.59:0/442191764 reap_dead start 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 -- 172.21.15.59:0/442191764 shutdown_connections 2024-04-24T06:29:22.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 -- 172.21.15.59:0/442191764 >> 172.21.15.59:0/442191764 conn(0x7fc670067d90 msgr2=0x7fc67006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 -- 172.21.15.59:0/442191764 shutdown_connections 2024-04-24T06:29:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.961+0000 7fc677000640 1 -- 172.21.15.59:0/442191764 wait complete. 2024-04-24T06:29:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.963+0000 7fc677000640 1 Processor -- start 2024-04-24T06:29:22.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.963+0000 7fc677000640 1 -- start start 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.964+0000 7fc677000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc654003f60 0x7fc6701f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.964+0000 7fc677000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc67013c520 0x7fc67013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc677000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc677000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701461f0 con 0x7fc67013c980 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc677000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc67006f130 con 0x7fc67013c520 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc677000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc6700b3910 con 0x7fc654003f60 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37318/0 (socket says 172.21.15.59:37318) 2024-04-24T06:29:22.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc675000640 1 -- 172.21.15.59:0/3288191506 learned_addr learned my addr 172.21.15.59:0/3288191506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:22.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc654003f60 0x7fc6701f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.965+0000 7fc66f400640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc67013c520 0x7fc67013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 -- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc654003f60 msgr2=0x7fc6701f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc654003f60 0x7fc6701f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 -- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc67013c520 msgr2=0x7fc67013d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:22.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc67013c520 0x7fc67013d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:22.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 -- 172.21.15.59:0/3288191506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6700b3bf0 con 0x7fc67013c980 2024-04-24T06:29:22.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.966+0000 7fc675000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc6700b58c0 tx=0x7fc66800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.970+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668006d50 con 0x7fc67013c980 2024-04-24T06:29:22.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.970+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc668003970 con 0x7fc67013c980 2024-04-24T06:29:22.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.970+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668003da0 con 0x7fc67013c980 2024-04-24T06:29:22.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.970+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6700b3ed0 con 0x7fc67013c980 2024-04-24T06:29:22.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.970+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6700b4490 con 0x7fc67013c980 2024-04-24T06:29:22.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.971+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc670068a80 con 0x7fc67013c980 2024-04-24T06:29:22.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.973+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc66801b020 con 0x7fc67013c980 2024-04-24T06:29:22.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.974+0000 7fc667e00640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 0x7fc65006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:22.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.974+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc66808b640 con 0x7fc67013c980 2024-04-24T06:29:22.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.974+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 0x7fc65006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:22.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.975+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 0x7fc65006ee00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc660000f60 tx=0x7fc660009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:22.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:22.979+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc668059480 con 0x7fc67013c980 2024-04-24T06:29:23.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.193+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6701455c0 con 0x7fc67013c980 2024-04-24T06:29:23.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.194+0000 7fc667e00640 1 -- 172.21.15.59:0/3288191506 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc66805d780 con 0x7fc67013c980 2024-04-24T06:29:23.196 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.196+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 msgr2=0x7fc65006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.196+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 0x7fc65006ee00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc660000f60 tx=0x7fc660009040 comp rx=0 tx=0).stop 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 msgr2=0x7fc6701426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc6700b58c0 tx=0x7fc66800dc00 comp rx=0 tx=0).stop 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 shutdown_connections 2024-04-24T06:29:23.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc65006c940 0x7fc65006ee00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc67013c980 0x7fc6701426f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc67013c520 0x7fc67013d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 --2- 172.21.15.59:0/3288191506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc654003f60 0x7fc6701f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 >> 172.21.15.59:0/3288191506 conn(0x7fc670067d90 msgr2=0x7fc670072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 shutdown_connections 2024-04-24T06:29:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.197+0000 7fc677000640 1 -- 172.21.15.59:0/3288191506 wait complete. 2024-04-24T06:29:23.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:23.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.432+0000 7f0be6600640 1 Processor -- start 2024-04-24T06:29:23.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.433+0000 7f0be6600640 1 -- start start 2024-04-24T06:29:23.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.434+0000 7f0be6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 0x7f0be00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.434+0000 7f0be6600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be00b5da0 con 0x7f0be00d15b0 2024-04-24T06:29:23.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.434+0000 7f0be6600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be00d0a20 con 0x7f0be015ed20 2024-04-24T06:29:23.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.434+0000 7f0be6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0be015f100 con 0x7f0be00b4f20 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.435+0000 7f0bdfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0be00d15b0 0x7f0be00cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54440/0 (socket says 172.21.15.59:54440) 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.435+0000 7f0bdfe00640 1 -- 172.21.15.59:0/1257549463 learned_addr learned my addr 172.21.15.59:0/1257549463 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.435+0000 7f0be5c00640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 0x7f0be00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.435+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 269627142 0 0) 0x7f0be00b5da0 con 0x7f0be00d15b0 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.435+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc8003520 con 0x7f0be00d15b0 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861613781 0 0) 0x7f0be00d0a20 con 0x7f0be015ed20 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be00b5da0 con 0x7f0be015ed20 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3646133283 0 0) 0x7f0bc8003520 con 0x7f0be00d15b0 2024-04-24T06:29:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 msgr2=0x7f0be00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 0x7f0be00cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 >> v1:172.21.15.66:6789/0 conn(0x7f0be015ed20 legacy=0x7f0be00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be00d2ed0 con 0x7f0be00d15b0 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0bcc001670 con 0x7f0be00d15b0 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 >> v1:172.21.15.59:6789/0 conn(0x7f0be00d15b0 legacy=0x7f0be00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc8003b70 0x7f0bc8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8004800 0x7f0bc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 0x7f0bc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8005520 con 0x7f0bc8003b70 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc800bf00 con 0x7f0bc8004800 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc800c080 con 0x7f0be00b4f20 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.436+0000 7f0be5200640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc8003b70 0x7f0bc8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0be5200640 1 -- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 msgr2=0x7f0bc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0be5200640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00b4f20 0x7f0bc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0be5200640 1 -- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8004800 msgr2=0x7f0bc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0bdfe00640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8004800 0x7f0bc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0be5200640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8004800 0x7f0bc8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.437+0000 7f0be5200640 1 -- 172.21.15.59:0/1257549463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc8003520 con 0x7f0bc8003b70 2024-04-24T06:29:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.438+0000 7f0be5200640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc8003b70 0x7f0bc8003f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40018d0 tx=0x7f0bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c2cd61cf69f248 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.438+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40014f0 con 0x7f0bc8003b70 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.438+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd40052d0 con 0x7f0bc8003b70 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.438+0000 7f0bdf400640 1 -- 172.21.15.59:0/1257549463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd402aca0 con 0x7f0bc8003b70 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 -- 172.21.15.59:0/1257549463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc8003b70 msgr2=0x7f0bc8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 --2- 172.21.15.59:0/1257549463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bc8003b70 0x7f0bc8003f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40018d0 tx=0x7f0bc8005830 comp rx=0 tx=0).stop 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be5c00640 1 -- 172.21.15.59:0/1257549463 reap_dead start 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 -- 172.21.15.59:0/1257549463 shutdown_connections 2024-04-24T06:29:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 -- 172.21.15.59:0/1257549463 >> 172.21.15.59:0/1257549463 conn(0x7f0be008bf90 msgr2=0x7f0be00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:23.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 -- 172.21.15.59:0/1257549463 shutdown_connections 2024-04-24T06:29:23.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.439+0000 7f0be6600640 1 -- 172.21.15.59:0/1257549463 wait complete. 2024-04-24T06:29:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.441+0000 7f0be6600640 1 Processor -- start 2024-04-24T06:29:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.441+0000 7f0be6600640 1 -- start start 2024-04-24T06:29:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 0x7f0be015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00c80f0 0x7f0be00c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00b3600 con 0x7f0be006cb60 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00d3490 con 0x7f0bc8003b70 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00b3780 con 0x7f0be00c80f0 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37338/0 (socket says 172.21.15.59:37338) 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.442+0000 7f0be5c00640 1 -- 172.21.15.59:0/4079123826 learned_addr learned my addr 172.21.15.59:0/4079123826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:23.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5200640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 0x7f0be015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 -- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00c80f0 msgr2=0x7f0be00c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00c80f0 0x7f0be00c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 -- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 msgr2=0x7f0be015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 0x7f0be015bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 -- 172.21.15.59:0/4079123826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01e7480 con 0x7f0be006cb60 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5200640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 0x7f0be015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.443+0000 7f0be5c00640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0be00d1c80 tx=0x7f0bd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.444+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800eb30 con 0x7f0be006cb60 2024-04-24T06:29:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.444+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd800ecf0 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.444+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800eea0 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.444+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be015c430 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.444+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be00c8e90 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.445+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be004fd90 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.446+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bd801b020 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.447+0000 7f0bdcc00640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 0x7f0bc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.447+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd808b9a0 con 0x7f0be006cb60 2024-04-24T06:29:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.447+0000 7f0be5200640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 0x7f0bc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.448+0000 7f0be5200640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 0x7f0bc006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4032000 tx=0x7f0bd4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.452+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0be01e7480 con 0x7f0be006cb60 2024-04-24T06:29:23.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.653+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0be01ea2a0 con 0x7f0be006cb60 2024-04-24T06:29:23.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.654+0000 7f0bdcc00640 1 -- 172.21.15.59:0/4079123826 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f0bd8059d30 con 0x7f0be006cb60 2024-04-24T06:29:23.657 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.657+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 msgr2=0x7f0bc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.657+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 0x7f0bc006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4032000 tx=0x7f0bd4009030 comp rx=0 tx=0).stop 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.657+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 msgr2=0x7f0be006cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.657+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0be00d1c80 tx=0x7f0bd800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 shutdown_connections 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bc006c920 0x7f0bc006ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be006cb60 0x7f0be006cfe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0be00c80f0 0x7f0be00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 --2- 172.21.15.59:0/4079123826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0bc8003b70 0x7f0be015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 >> 172.21.15.59:0/4079123826 conn(0x7f0be008bf90 msgr2=0x7f0be006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:23.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 shutdown_connections 2024-04-24T06:29:23.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.658+0000 7f0be6600640 1 -- 172.21.15.59:0/4079123826 wait complete. 2024-04-24T06:29:23.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.898+0000 7fe47ae00640 1 Processor -- start 2024-04-24T06:29:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.898+0000 7fe47ae00640 1 -- start start 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4740cec60 0x7fe4741668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe474166e10 0x7fe4740c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe4740ce290 con 0x7fe4740cec60 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe4740d1560 con 0x7fe474166e10 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe47ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe474167330 con 0x7fe4740d1180 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe473400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4740cec60 0x7fe4741668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe473400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4740cec60 0x7fe4741668d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37352/0 (socket says 172.21.15.59:37352) 2024-04-24T06:29:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe473400640 1 -- 172.21.15.59:0/2728615065 learned_addr learned my addr 172.21.15.59:0/2728615065 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.900+0000 7fe473400640 1 -- 172.21.15.59:0/2728615065 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 msgr2=0x7fe4740ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe473e00640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe473400640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe473400640 1 -- 172.21.15.59:0/2728615065 >> v2:172.21.15.66:3300/0 conn(0x7fe474166e10 msgr2=0x7fe4740c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe478e00640 1 --2- 172.21.15.59:0/2728615065 >> v2:172.21.15.66:3300/0 conn(0x7fe474166e10 0x7fe4740c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe473400640 1 --2- 172.21.15.59:0/2728615065 >> v2:172.21.15.66:3300/0 conn(0x7fe474166e10 0x7fe4740c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.901+0000 7fe473400640 1 -- 172.21.15.59:0/2728615065 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4741675f0 con 0x7fe4740cec60 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.902+0000 7fe473400640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4740cec60 0x7fe4741668d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe474073dd0 tx=0x7fe45c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34106bd1f3ec5968 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.902+0000 7fe472a00640 1 -- 172.21.15.59:0/2728615065 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c037070 con 0x7fe4740cec60 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.902+0000 7fe472a00640 1 -- 172.21.15.59:0/2728615065 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c032440 con 0x7fe4740cec60 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.902+0000 7fe472a00640 1 -- 172.21.15.59:0/2728615065 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c032810 con 0x7fe4740cec60 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 -- 172.21.15.59:0/2728615065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4740cec60 msgr2=0x7fe4741668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4740cec60 0x7fe4741668d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe474073dd0 tx=0x7fe45c02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 -- 172.21.15.59:0/2728615065 shutdown_connections 2024-04-24T06:29:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 --2- 172.21.15.59:0/2728615065 >> v2:172.21.15.66:3300/0 conn(0x7fe474166e10 0x7fe4740c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4740cec60 0x7fe4741668d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 --2- 172.21.15.59:0/2728615065 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 -- 172.21.15.59:0/2728615065 >> 172.21.15.59:0/2728615065 conn(0x7fe474073430 msgr2=0x7fe47409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 -- 172.21.15.59:0/2728615065 shutdown_connections 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.903+0000 7fe47ae00640 1 -- 172.21.15.59:0/2728615065 wait complete. 2024-04-24T06:29:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.905+0000 7fe47ae00640 1 Processor -- start 2024-04-24T06:29:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.905+0000 7fe47ae00640 1 -- start start 2024-04-24T06:29:23.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 0x7fe4740c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe47406f680 con 0x7fe474166e10 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe47406f500 con 0x7fe474154a60 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe47ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4740c4a30 con 0x7fe4740d1180 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe473400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe473400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37356/0 (socket says 172.21.15.59:37356) 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.906+0000 7fe473400640 1 -- 172.21.15.59:0/3149717 learned_addr learned my addr 172.21.15.59:0/3149717 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 -- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 msgr2=0x7fe4740b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe478e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 0x7fe4740c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 -- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 msgr2=0x7fe4740c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 0x7fe4740c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 -- 172.21.15.59:0/3149717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe474163820 con 0x7fe474166e10 2024-04-24T06:29:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe478e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 0x7fe4740c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe473400640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe45c000c00 tx=0x7fe45c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c037040 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c042070 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.907+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4740c6160 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.908+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c030bb0 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.908+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4740b4510 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.909+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4740c6680 con 0x7fe474166e10 2024-04-24T06:29:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.910+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe45c047460 con 0x7fe474166e10 2024-04-24T06:29:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.911+0000 7fe46be00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 0x7fe45006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.911+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe45c0ac7b0 con 0x7fe474166e10 2024-04-24T06:29:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.911+0000 7fe473e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 0x7fe45006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.912+0000 7fe473e00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 0x7fe45006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe464008f10 tx=0x7fe464008460 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:23.915+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4740c6680 con 0x7fe474166e10 2024-04-24T06:29:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.125+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe474069190 con 0x7fe474166e10 2024-04-24T06:29:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.126+0000 7fe46be00640 1 -- 172.21.15.59:0/3149717 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe45c045360 con 0x7fe474166e10 2024-04-24T06:29:24.128 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 msgr2=0x7fe45006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 0x7fe45006ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe464008f10 tx=0x7fe464008460 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 msgr2=0x7fe4740c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe45c000c00 tx=0x7fe45c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 shutdown_connections 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45006c920 0x7fe45006ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe474154a60 0x7fe4740c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe474166e10 0x7fe4740c5a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 --2- 172.21.15.59:0/3149717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4740d1180 0x7fe4740b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 >> 172.21.15.59:0/3149717 conn(0x7fe474073430 msgr2=0x7fe474161e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 shutdown_connections 2024-04-24T06:29:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.129+0000 7fe47ae00640 1 -- 172.21.15.59:0/3149717 wait complete. 2024-04-24T06:29:24.169 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:29:24.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.377+0000 7fc08d600640 1 Processor -- start 2024-04-24T06:29:24.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.378+0000 7fc08d600640 1 -- start start 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc08d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc0880d14a0 0x7fc0880b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc08d600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0880cf030 con 0x7fc08815e7c0 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc08d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0880b6a40 con 0x7fc0880b5ff0 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc08d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc0880b7000 con 0x7fc0880d14a0 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc087e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc08815e7c0 0x7fc0880d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54446/0 (socket says 172.21.15.59:54446) 2024-04-24T06:29:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.379+0000 7fc087e00640 1 -- 172.21.15.59:0/564848173 learned_addr learned my addr 172.21.15.59:0/564848173 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:24.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445309637 0 0) 0x7fc078001490 con 0x7fc08815e7c0 2024-04-24T06:29:24.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc06c003520 con 0x7fc08815e7c0 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc08cc00640 1 --2- 172.21.15.59:0/564848173 >> v2:172.21.15.66:3300/0 conn(0x7fc0880d14a0 0x7fc0880b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4021890716 0 0) 0x7fc06c003520 con 0x7fc08815e7c0 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 >> v2:172.21.15.66:3300/0 conn(0x7fc0880d14a0 msgr2=0x7fc0880b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 --2- 172.21.15.59:0/564848173 >> v2:172.21.15.66:3300/0 conn(0x7fc0880d14a0 0x7fc0880b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 >> v1:172.21.15.66:6789/0 conn(0x7fc0880b5ff0 legacy=0x7fc0880d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880b6d20 con 0x7fc08815e7c0 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc078001bc0 con 0x7fc08815e7c0 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 >> v1:172.21.15.59:6789/0 conn(0x7fc08815e7c0 legacy=0x7fc0880d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc06c003a00 0x7fc06c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c004800 0x7fc06c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 0x7fc06c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880b7000 con 0x7fc06c005210 2024-04-24T06:29:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c00c140 con 0x7fc06c004800 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.380+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c00c340 con 0x7fc06c003a00 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 0x7fc06c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc087e00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc06c003a00 0x7fc06c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 -- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc06c003a00 msgr2=0x7fc06c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc06c003a00 0x7fc06c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 -- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c004800 msgr2=0x7fc06c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c004800 0x7fc06c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc08cc00640 1 -- 172.21.15.59:0/564848173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c003520 con 0x7fc06c005210 2024-04-24T06:29:24.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.381+0000 7fc087e00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc06c003a00 0x7fc06c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:24.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.382+0000 7fc08cc00640 1 -- 172.21.15.59:0/564848173 reap_dead start 2024-04-24T06:29:24.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.382+0000 7fc08cc00640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 0x7fc06c00b9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0880cb5a0 tx=0x7fc06c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51e942f20ae3d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080030070 con 0x7fc06c005210 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08002adb0 con 0x7fc06c005210 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc086a00640 1 -- 172.21.15.59:0/564848173 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08000b600 con 0x7fc06c005210 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 -- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 msgr2=0x7fc06c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 0x7fc06c00b9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0880cb5a0 tx=0x7fc06c005830 comp rx=0 tx=0).stop 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 -- 172.21.15.59:0/564848173 shutdown_connections 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 --2- 172.21.15.59:0/564848173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c005210 0x7fc06c00b9c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 -- 172.21.15.59:0/564848173 >> 172.21.15.59:0/564848173 conn(0x7fc08809d920 msgr2=0x7fc08808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 -- 172.21.15.59:0/564848173 shutdown_connections 2024-04-24T06:29:24.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.383+0000 7fc08d600640 1 -- 172.21.15.59:0/564848173 wait complete. 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.385+0000 7fc08d600640 1 Processor -- start 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.385+0000 7fc08d600640 1 -- start start 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 0x7fc0880b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0881f4650 0x7fc0881f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880d1a80 con 0x7fc088152f10 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0880cf620 con 0x7fc06c005210 2024-04-24T06:29:24.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.386+0000 7fc08d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0880d1c00 con 0x7fc0881f4650 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37378/0 (socket says 172.21.15.59:37378) 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 -- 172.21.15.59:0/2137466815 learned_addr learned my addr 172.21.15.59:0/2137466815 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc08cc00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0881f4650 0x7fc0881f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087e00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 0x7fc0880b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 -- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0881f4650 msgr2=0x7fc0881f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0881f4650 0x7fc0881f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 -- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 msgr2=0x7fc0880b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 0x7fc0880b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087400640 1 -- 172.21.15.59:0/2137466815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880b1c50 con 0x7fc088152f10 2024-04-24T06:29:24.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.387+0000 7fc087e00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 0x7fc0880b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:24.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc087400640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c480 tx=0x7fc07000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070016020 con 0x7fc088152f10 2024-04-24T06:29:24.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc070012830 con 0x7fc088152f10 2024-04-24T06:29:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070012c20 con 0x7fc088152f10 2024-04-24T06:29:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc088160090 con 0x7fc088152f10 2024-04-24T06:29:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.388+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0880b21d0 con 0x7fc088152f10 2024-04-24T06:29:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.389+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc088068f60 con 0x7fc088152f10 2024-04-24T06:29:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.391+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc070019020 con 0x7fc088152f10 2024-04-24T06:29:24.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.392+0000 7fc07fe00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 0x7fc06406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.392+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc07008b850 con 0x7fc088152f10 2024-04-24T06:29:24.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.392+0000 7fc087e00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 0x7fc06406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.393+0000 7fc087e00640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 0x7fc06406ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc078002230 tx=0x7fc078009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.397+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc070020030 con 0x7fc088152f10 2024-04-24T06:29:24.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.594+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc08804fd90 con 0x7fc088152f10 2024-04-24T06:29:24.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.595+0000 7fc07fe00640 1 -- 172.21.15.59:0/2137466815 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fc0700182c0 con 0x7fc088152f10 2024-04-24T06:29:24.599 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:24.599 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:28:45.724518+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.598+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 msgr2=0x7fc06406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.598+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 0x7fc06406ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc078002230 tx=0x7fc078009040 comp rx=0 tx=0).stop 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.598+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 msgr2=0x7fc0880cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.598+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c480 tx=0x7fc07000c940 comp rx=0 tx=0).stop 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 shutdown_connections 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc06406c920 0x7fc06406ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0881f4650 0x7fc0881f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088152f10 0x7fc0880cdf10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 --2- 172.21.15.59:0/2137466815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06c005210 0x7fc0880b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 >> 172.21.15.59:0/2137466815 conn(0x7fc08809d920 msgr2=0x7fc0880b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:24.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 shutdown_connections 2024-04-24T06:29:24.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.599+0000 7fc08d600640 1 -- 172.21.15.59:0/2137466815 wait complete. 2024-04-24T06:29:24.621 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:24.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:24.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.837+0000 7fe3b7000640 1 Processor -- start 2024-04-24T06:29:24.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.837+0000 7fe3b7000640 1 -- start start 2024-04-24T06:29:24.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 0x7fe3b00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3b00b4670 0x7fe3b0071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe3b00cfba0 0x7fe3b00722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b00d0fd0 con 0x7fe3b00b4670 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b00b35d0 con 0x7fe3b00cfba0 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3b7000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b00b38b0 con 0x7fe3b0162c10 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.839+0000 7fe3a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3b00b4670 0x7fe3b0071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3b00b4670 0x7fe3b0071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37382/0 (socket says 172.21.15.59:37382) 2024-04-24T06:29:24.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 -- 172.21.15.59:0/958985705 learned_addr learned my addr 172.21.15.59:0/958985705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3b5000640 1 --2- 172.21.15.59:0/958985705 >> v2:172.21.15.66:3300/0 conn(0x7fe3b00cfba0 0x7fe3b00722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3afe00640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 0x7fe3b00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 -- 172.21.15.59:0/958985705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 msgr2=0x7fe3b00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 0x7fe3b00d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 -- 172.21.15.59:0/958985705 >> v2:172.21.15.66:3300/0 conn(0x7fe3b00cfba0 msgr2=0x7fe3b00722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 --2- 172.21.15.59:0/958985705 >> v2:172.21.15.66:3300/0 conn(0x7fe3b00cfba0 0x7fe3b00722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3a7e00640 1 -- 172.21.15.59:0/958985705 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b00b3b90 con 0x7fe3b00b4670 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.840+0000 7fe3afe00640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 0x7fe3b00d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.842+0000 7fe3a7e00640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b0071d70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0070110 tx=0x7fe39802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9925a4a4885a8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.842+0000 7fe3af400640 1 -- 172.21.15.59:0/958985705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398037070 con 0x7fe3b00b4670 2024-04-24T06:29:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.842+0000 7fe3af400640 1 -- 172.21.15.59:0/958985705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe398032440 con 0x7fe3b00b4670 2024-04-24T06:29:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.842+0000 7fe3af400640 1 -- 172.21.15.59:0/958985705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398032810 con 0x7fe3b00b4670 2024-04-24T06:29:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 -- 172.21.15.59:0/958985705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 msgr2=0x7fe3b0071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b0071d70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0070110 tx=0x7fe39802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 -- 172.21.15.59:0/958985705 shutdown_connections 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 --2- 172.21.15.59:0/958985705 >> v2:172.21.15.66:3300/0 conn(0x7fe3b00cfba0 0x7fe3b00722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b0071d70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 --2- 172.21.15.59:0/958985705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b0162c10 0x7fe3b00d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 -- 172.21.15.59:0/958985705 >> 172.21.15.59:0/958985705 conn(0x7fe3b0067d90 msgr2=0x7fe3b006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 -- 172.21.15.59:0/958985705 shutdown_connections 2024-04-24T06:29:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.843+0000 7fe3b7000640 1 -- 172.21.15.59:0/958985705 wait complete. 2024-04-24T06:29:24.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.845+0000 7fe3b7000640 1 Processor -- start 2024-04-24T06:29:24.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.845+0000 7fe3b7000640 1 -- start start 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b00cfba0 0x7fe3b00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b0162c10 0x7fe3b0164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0162ff0 con 0x7fe3b00b4670 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b00b4150 con 0x7fe3b0162c10 2024-04-24T06:29:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3b7000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b0163170 con 0x7fe3b00cfba0 2024-04-24T06:29:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37392/0 (socket says 172.21.15.59:37392) 2024-04-24T06:29:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.846+0000 7fe3afe00640 1 -- 172.21.15.59:0/2829779645 learned_addr learned my addr 172.21.15.59:0/2829779645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 -- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b00cfba0 msgr2=0x7fe3b00afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b00cfba0 0x7fe3b00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 -- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b0162c10 msgr2=0x7fe3b0164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b0162c10 0x7fe3b0164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 -- 172.21.15.59:0/2829779645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b00b2130 con 0x7fe3b00b4670 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3afe00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000e730 tx=0x7fe3a000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe3b00b4670 2024-04-24T06:29:24.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a0004230 con 0x7fe3b00b4670 2024-04-24T06:29:24.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000ee40 con 0x7fe3b00b4670 2024-04-24T06:29:24.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.847+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b00b2410 con 0x7fe3b00b4670 2024-04-24T06:29:24.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.848+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b00b00f0 con 0x7fe3b00b4670 2024-04-24T06:29:24.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.849+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b01fc170 con 0x7fe3b00b4670 2024-04-24T06:29:24.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.849+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3a000c6d0 con 0x7fe3b00b4670 2024-04-24T06:29:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.850+0000 7fe3acc00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 0x7fe38c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.851+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3a008b7c0 con 0x7fe3b00b4670 2024-04-24T06:29:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.851+0000 7fe3a7e00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 0x7fe38c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.851+0000 7fe3a7e00640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 0x7fe38c06ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe398034000 tx=0x7fe39802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:24.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:24.855+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3a0017020 con 0x7fe3b00b4670 2024-04-24T06:29:25.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.061+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3b0068a80 con 0x7fe3b00b4670 2024-04-24T06:29:25.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.062+0000 7fe3acc00640 1 -- 172.21.15.59:0/2829779645 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe3a0059be0 con 0x7fe3b00b4670 2024-04-24T06:29:25.064 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:25.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 msgr2=0x7fe38c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 0x7fe38c06ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe398034000 tx=0x7fe39802c000 comp rx=0 tx=0).stop 2024-04-24T06:29:25.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 msgr2=0x7fe3b01568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000e730 tx=0x7fe3a000ebf0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 shutdown_connections 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe38c06c920 0x7fe38c06ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3b0162c10 0x7fe3b0164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3b00cfba0 0x7fe3b00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 --2- 172.21.15.59:0/2829779645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b00b4670 0x7fe3b01568c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 >> 172.21.15.59:0/2829779645 conn(0x7fe3b0067d90 msgr2=0x7fe3b015d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 shutdown_connections 2024-04-24T06:29:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.065+0000 7fe3b7000640 1 -- 172.21.15.59:0/2829779645 wait complete. 2024-04-24T06:29:25.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.302+0000 7f7bbfe00640 1 Processor -- start 2024-04-24T06:29:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.303+0000 7f7bbfe00640 1 -- start start 2024-04-24T06:29:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.304+0000 7f7bbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7bc01567e0 0x7f7bc0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.304+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc015d810 0x7f7bc00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.304+0000 7f7bbfe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc015cab0 con 0x7f7bc00d0b40 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.304+0000 7f7bbfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc00d0f70 con 0x7f7bc01567e0 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.304+0000 7f7bbfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc0160e40 con 0x7f7bc015d810 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7bc01567e0 0x7f7bc0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7bc01567e0 0x7f7bc0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37396/0 (socket says 172.21.15.59:37396) 2024-04-24T06:29:25.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 -- 172.21.15.59:0/1047205152 learned_addr learned my addr 172.21.15.59:0/1047205152 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:25.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bb6000640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc015d810 0x7f7bc00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 -- 172.21.15.59:0/1047205152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc015d810 msgr2=0x7f7bc00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc015d810 0x7f7bc00d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 -- 172.21.15.59:0/1047205152 >> v1:172.21.15.66:6789/0 conn(0x7f7bc00d0b40 legacy=0x7f7bc0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.305+0000 7f7bbea00640 1 -- 172.21.15.59:0/1047205152 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc015cd30 con 0x7f7bc01567e0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.306+0000 7f7bbea00640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc01567e0 0x7f7bc0156bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009870 tx=0x7f7bb002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199c80877d02d997 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.307+0000 7f7bbe000640 1 -- 172.21.15.59:0/1047205152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0037070 con 0x7f7bc01567e0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.307+0000 7f7bbe000640 1 -- 172.21.15.59:0/1047205152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb0032440 con 0x7f7bc01567e0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.307+0000 7f7bbe000640 1 -- 172.21.15.59:0/1047205152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0032810 con 0x7f7bc01567e0 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1047205152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc01567e0 msgr2=0x7f7bc0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc01567e0 0x7f7bc0156bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009870 tx=0x7f7bb002a210 comp rx=0 tx=0).stop 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1047205152 shutdown_connections 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc015d810 0x7f7bc00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1047205152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc01567e0 0x7f7bc0156bc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1047205152 >> 172.21.15.59:0/1047205152 conn(0x7f7bc006bdf0 msgr2=0x7f7bc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1047205152 shutdown_connections 2024-04-24T06:29:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.308+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1047205152 wait complete. 2024-04-24T06:29:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.310+0000 7f7bbfe00640 1 Processor -- start 2024-04-24T06:29:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.310+0000 7f7bbfe00640 1 -- start start 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.311+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc00d0b40 0x7f7bc01f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc01567e0 0x7f7bc01513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc0160820 con 0x7f7bc015d810 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc015d2f0 con 0x7f7bc01567e0 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc01609a0 con 0x7f7bc00d0b40 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37410/0 (socket says 172.21.15.59:37410) 2024-04-24T06:29:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbf400640 1 -- 172.21.15.59:0/1128832130 learned_addr learned my addr 172.21.15.59:0/1128832130 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bb6000640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc01567e0 0x7f7bc01513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.312+0000 7f7bbea00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc00d0b40 0x7f7bc01f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 -- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc00d0b40 msgr2=0x7f7bc01f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc00d0b40 0x7f7bc01f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 -- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc01567e0 msgr2=0x7f7bc01513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc01567e0 0x7f7bc01513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 -- 172.21.15.59:0/1128832130 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc006efa0 con 0x7f7bc015d810 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbf400640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8019070 con 0x7f7bc015d810 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb800cb40 con 0x7f7bc015d810 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8014440 con 0x7f7bc015d810 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc006f280 con 0x7f7bc015d810 2024-04-24T06:29:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.313+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc006f840 con 0x7f7bc015d810 2024-04-24T06:29:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.314+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc00d0960 con 0x7f7bc015d810 2024-04-24T06:29:25.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.316+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bb801c070 con 0x7f7bc015d810 2024-04-24T06:29:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.317+0000 7f7bb7400640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 0x7f7b9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.317+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7bb808b7f0 con 0x7f7bc015d810 2024-04-24T06:29:25.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.317+0000 7f7bbea00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 0x7f7b9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.318+0000 7f7bbea00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 0x7f7b9c06ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.321+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bc00d0960 con 0x7f7bc015d810 2024-04-24T06:29:25.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.522+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7bc004fd90 con 0x7f7bc015d810 2024-04-24T06:29:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.523+0000 7f7bb7400640 1 -- 172.21.15.59:0/1128832130 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7bb8059b80 con 0x7f7bc015d810 2024-04-24T06:29:25.525 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:25.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.525+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 msgr2=0x7f7b9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.525+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 0x7f7b9c06ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb002e000 comp rx=0 tx=0).stop 2024-04-24T06:29:25.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.525+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 msgr2=0x7f7bc0151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.525+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb800be30 comp rx=0 tx=0).stop 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 shutdown_connections 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7b9c06c920 0x7f7b9c06ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bc015d810 0x7f7bc0151930 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bc01567e0 0x7f7bc01513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 --2- 172.21.15.59:0/1128832130 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bc00d0b40 0x7f7bc01f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 >> 172.21.15.59:0/1128832130 conn(0x7f7bc006bdf0 msgr2=0x7f7bc00af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 shutdown_connections 2024-04-24T06:29:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.526+0000 7f7bbfe00640 1 -- 172.21.15.59:0/1128832130 wait complete. 2024-04-24T06:29:25.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:25.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.761+0000 7fda12600640 1 Processor -- start 2024-04-24T06:29:25.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.761+0000 7fda12600640 1 -- start start 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.762+0000 7fda12600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fda0c072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda12600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c06efe0 con 0x7fda0c15bc80 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda12600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c06f2c0 con 0x7fda0c155280 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda12600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0d0d10 con 0x7fda0c072610 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda0be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fda0c15bc80 0x7fda0c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54458/0 (socket says 172.21.15.59:54458) 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda0be00640 1 -- 172.21.15.59:0/2781915009 learned_addr learned my addr 172.21.15.59:0/2781915009 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730238165 0 0) 0x7fda0c06efe0 con 0x7fda0c15bc80 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda11c00640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fda0c072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.763+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9f4005730 con 0x7fda0c15bc80 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3516852028 0 0) 0x7fd9f4005730 con 0x7fda0c15bc80 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 msgr2=0x7fda0c072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fda0c072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 >> v1:172.21.15.66:6789/0 conn(0x7fda0c155280 legacy=0x7fda0c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c06f5a0 con 0x7fda0c15bc80 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd9f8001670 con 0x7fda0c15bc80 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 >> v1:172.21.15.59:6789/0 conn(0x7fda0c15bc80 legacy=0x7fda0c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fd9f4005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fd9f4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f4006f50 0x7fd9f40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0d0d10 con 0x7fd9f4006220 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f400dd90 con 0x7fd9f4006f50 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f400df90 con 0x7fda0c072610 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0be00640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fd9f4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda11200640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fd9f4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0be00640 1 -- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 msgr2=0x7fd9f4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.764+0000 7fda0be00640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fd9f4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.765+0000 7fda0be00640 1 -- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f4006f50 msgr2=0x7fd9f40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.765+0000 7fda0be00640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9f4006f50 0x7fd9f40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.765+0000 7fda0be00640 1 -- 172.21.15.59:0/2781915009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f4005730 con 0x7fd9f4006220 2024-04-24T06:29:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.765+0000 7fda11200640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c072610 0x7fd9f4005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda0be00640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fd9f4006a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c240 tx=0x7fd9f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caae0c7ba6dbe0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8007cf0 con 0x7fd9f4006220 2024-04-24T06:29:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f80058d0 con 0x7fd9f4006220 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda0b400640 1 -- 172.21.15.59:0/2781915009 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8005d20 con 0x7fd9f4006220 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda12600640 1 -- 172.21.15.59:0/2781915009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 msgr2=0x7fd9f4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda12600640 1 --2- 172.21.15.59:0/2781915009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fd9f4006a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c240 tx=0x7fd9f4007a40 comp rx=0 tx=0).stop 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda11c00640 1 -- 172.21.15.59:0/2781915009 reap_dead start 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda12600640 1 -- 172.21.15.59:0/2781915009 shutdown_connections 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.766+0000 7fda12600640 1 -- 172.21.15.59:0/2781915009 >> 172.21.15.59:0/2781915009 conn(0x7fda0c06bdf0 msgr2=0x7fda0c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.767+0000 7fda12600640 1 -- 172.21.15.59:0/2781915009 shutdown_connections 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.767+0000 7fda12600640 1 -- 172.21.15.59:0/2781915009 wait complete. 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.768+0000 7fda12600640 1 Processor -- start 2024-04-24T06:29:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.769+0000 7fda12600640 1 -- start start 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.769+0000 7fda12600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.769+0000 7fda12600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0c1f7ea0 0x7fda0c0aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda12600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c0ae410 0x7fda0c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda12600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c06fb60 con 0x7fd9f4006220 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda12600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c15c0c0 con 0x7fda0c1f7ea0 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda12600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c072a60 con 0x7fda0c0ae410 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37432/0 (socket says 172.21.15.59:37432) 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 -- 172.21.15.59:0/822454494 learned_addr learned my addr 172.21.15.59:0/822454494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 -- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c0ae410 msgr2=0x7fda0c0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c0ae410 0x7fda0c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 -- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0c1f7ea0 msgr2=0x7fda0c0aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0c1f7ea0 0x7fda0c0aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.770+0000 7fda11200640 1 -- 172.21.15.59:0/822454494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1f8860 con 0x7fd9f4006220 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda11200640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d7a0 tx=0x7fda0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00005070 con 0x7fd9f4006220 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda00005210 con 0x7fd9f4006220 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00007030 con 0x7fd9f4006220 2024-04-24T06:29:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1f8b40 con 0x7fd9f4006220 2024-04-24T06:29:25.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.771+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c0ad9b0 con 0x7fd9f4006220 2024-04-24T06:29:25.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.772+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c04fd90 con 0x7fd9f4006220 2024-04-24T06:29:25.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.773+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda0001a020 con 0x7fd9f4006220 2024-04-24T06:29:25.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.774+0000 7fda08c00640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:25.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.774+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda0008b400 con 0x7fd9f4006220 2024-04-24T06:29:25.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.774+0000 7fda0be00640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:25.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.775+0000 7fda0be00640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8033000 tx=0x7fd9f8007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:25.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.778+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda00059790 con 0x7fd9f4006220 2024-04-24T06:29:25.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.983+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda0c06e2f0 con 0x7fd9f4006220 2024-04-24T06:29:25.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.984+0000 7fda08c00640 1 -- 172.21.15.59:0/822454494 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fda00059090 con 0x7fd9f4006220 2024-04-24T06:29:25.986 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:25.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 msgr2=0x7fd9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8033000 tx=0x7fd9f8007000 comp rx=0 tx=0).stop 2024-04-24T06:29:25.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 msgr2=0x7fda0c1f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d7a0 tx=0x7fda0000dc60 comp rx=0 tx=0).stop 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 shutdown_connections 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fda0c0ae410 0x7fda0c0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda0c1f7ea0 0x7fda0c0aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 --2- 172.21.15.59:0/822454494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9f4006220 0x7fda0c1f1d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 >> 172.21.15.59:0/822454494 conn(0x7fda0c06bdf0 msgr2=0x7fda0c159010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.987+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 shutdown_connections 2024-04-24T06:29:25.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:25.988+0000 7fda12600640 1 -- 172.21.15.59:0/822454494 wait complete. 2024-04-24T06:29:26.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-24T06:29:26.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.211+0000 7f2af7200640 1 Processor -- start 2024-04-24T06:29:26.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.212+0000 7f2af7200640 1 -- start start 2024-04-24T06:29:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.213+0000 7f2af7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 0x7f2af80d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.213+0000 7f2af7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2af80d0330 0x7f2af80d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.213+0000 7f2af7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af815fa20 con 0x7f2af80d0ec0 2024-04-24T06:29:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.213+0000 7f2af7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2af80b3b60 con 0x7f2af80d0330 2024-04-24T06:29:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.213+0000 7f2af7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2af815fd00 con 0x7f2af80cff10 2024-04-24T06:29:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.214+0000 7f2af5400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 0x7f2af80d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.214+0000 7f2af5e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2af80d0ec0 0x7f2af80d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:50358/0 (socket says 172.21.15.59:50358) 2024-04-24T06:29:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.214+0000 7f2af5e00640 1 -- 172.21.15.59:0/513338814 learned_addr learned my addr 172.21.15.59:0/513338814 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.214+0000 7f2af6800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2af80d0330 0x7f2af80d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.214+0000 7f2af5400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 0x7f2af80d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:55750/0 (socket says 172.21.15.59:55750) 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af4a00640 1 -- 172.21.15.59:0/513338814 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655136139 0 0) 0x7f2af815fa20 con 0x7f2af80d0ec0 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af4a00640 1 -- 172.21.15.59:0/513338814 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2adc003520 con 0x7f2af80d0ec0 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af6800640 1 -- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 msgr2=0x7f2af80d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af6800640 1 --2- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 0x7f2af80d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af6800640 1 -- 172.21.15.59:0/513338814 >> v1:172.21.15.66:6789/0 conn(0x7f2af80d0ec0 legacy=0x7f2af80d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:26.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.215+0000 7f2af6800640 1 -- 172.21.15.59:0/513338814 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af80cf430 con 0x7f2af80d0330 2024-04-24T06:29:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.216+0000 7f2af6800640 1 --2- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0330 0x7f2af80d26e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009d90 tx=0x7f2af002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd3e38262ac3c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.217+0000 7f2af4a00640 1 -- 172.21.15.59:0/513338814 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007740 con 0x7f2af80d0330 2024-04-24T06:29:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.217+0000 7f2af4a00640 1 -- 172.21.15.59:0/513338814 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af00078e0 con 0x7f2af80d0330 2024-04-24T06:29:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.217+0000 7f2af4a00640 1 -- 172.21.15.59:0/513338814 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007cb0 con 0x7f2af80d0330 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 -- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0330 msgr2=0x7f2af80d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 --2- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0330 0x7f2af80d26e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009d90 tx=0x7f2af002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 -- 172.21.15.59:0/513338814 shutdown_connections 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 --2- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0330 0x7f2af80d26e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 --2- 172.21.15.59:0/513338814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80cff10 0x7f2af80d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 -- 172.21.15.59:0/513338814 >> 172.21.15.59:0/513338814 conn(0x7f2af808bf90 msgr2=0x7f2af80d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 -- 172.21.15.59:0/513338814 shutdown_connections 2024-04-24T06:29:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.218+0000 7f2af7200640 1 -- 172.21.15.59:0/513338814 wait complete. 2024-04-24T06:29:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.219+0000 7f2af7200640 1 Processor -- start 2024-04-24T06:29:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.220+0000 7f2af7200640 1 -- start start 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.220+0000 7f2af7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80d0330 0x7f2af8145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0ec0 0x7f2af80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2af815eb10 con 0x7f2af80cff10 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80cf9f0 con 0x7f2af80d0ec0 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2af815ec90 con 0x7f2af80d0330 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37438/0 (socket says 172.21.15.59:37438) 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 -- 172.21.15.59:0/1414008428 learned_addr learned my addr 172.21.15.59:0/1414008428 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5400640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80d0330 0x7f2af8145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af6800640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0ec0 0x7f2af80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 -- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80d0330 msgr2=0x7f2af8145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80d0330 0x7f2af8145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.221+0000 7f2af5e00640 1 -- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0ec0 msgr2=0x7f2af80cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2af5e00640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0ec0 0x7f2af80cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2af5e00640 1 -- 172.21.15.59:0/1414008428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af80ce620 con 0x7f2af80cff10 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2af5e00640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c440 tx=0x7f2ae800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8017070 con 0x7f2af80cff10 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae8005ce0 con 0x7f2af80cff10 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80072b0 con 0x7f2af80cff10 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af80ce340 con 0x7f2af80cff10 2024-04-24T06:29:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.222+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81e94f0 con 0x7f2af80cff10 2024-04-24T06:29:26.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.224+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af8068b70 con 0x7f2af80cff10 2024-04-24T06:29:26.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.225+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ae801a070 con 0x7f2af80cff10 2024-04-24T06:29:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.226+0000 7f2aee000640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 0x7f2ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.226+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f2ae808af40 con 0x7f2af80cff10 2024-04-24T06:29:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.226+0000 7f2af5400640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 0x7f2ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.227+0000 7f2af5400640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 0x7f2ad406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cd300 tx=0x7f2ae0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.230+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2af8068b70 con 0x7f2af80cff10 2024-04-24T06:29:26.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:26.510+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f2af815e000 con 0x7f2af80cff10 2024-04-24T06:29:27.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.010+0000 7f2aee000640 1 -- 172.21.15.59:0/1414008428 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v62) ==== 145+0+0 (secure 0 0 0) 0x7f2ae80593b0 con 0x7f2af80cff10 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.013+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 msgr2=0x7f2ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.013+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 0x7f2ad406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2af80cd300 tx=0x7f2ae0008040 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 msgr2=0x7f2af8145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c440 tx=0x7f2ae800c900 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 shutdown_connections 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ad406c8f0 0x7f2ad406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2af80d0ec0 0x7f2af80cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2af80d0330 0x7f2af8145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 --2- 172.21.15.59:0/1414008428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cff10 0x7f2af8145e80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 >> 172.21.15.59:0/1414008428 conn(0x7f2af808bf90 msgr2=0x7f2af815ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 shutdown_connections 2024-04-24T06:29:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.014+0000 7f2af7200640 1 -- 172.21.15.59:0/1414008428 wait complete. 2024-04-24T06:29:27.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:29:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.252+0000 7f9297200640 1 Processor -- start 2024-04-24T06:29:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.253+0000 7f9297200640 1 -- start start 2024-04-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.254+0000 7f9297200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 0x7f929806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.254+0000 7f9297200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.254+0000 7f9297200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929815ef20 con 0x7f92980b43a0 2024-04-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.254+0000 7f9297200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f92980d0fa0 con 0x7f9298155a80 2024-04-24T06:29:27.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.254+0000 7f9297200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f929815f1c0 con 0x7f92980d0600 2024-04-24T06:29:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.255+0000 7f928fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.255+0000 7f9296800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f92980b43a0 0x7f92980b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:50362/0 (socket says 172.21.15.59:50362) 2024-04-24T06:29:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.255+0000 7f9296800640 1 -- 172.21.15.59:0/257079353 learned_addr learned my addr 172.21.15.59:0/257079353 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:27.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.255+0000 7f9295e00640 1 --2- 172.21.15.59:0/257079353 >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 0x7f929806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.256+0000 7f928fe00640 1 -- 172.21.15.59:0/257079353 >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 msgr2=0x7f929806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.256+0000 7f928fe00640 1 --2- 172.21.15.59:0/257079353 >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 0x7f929806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.256+0000 7f928fe00640 1 -- 172.21.15.59:0/257079353 >> v1:172.21.15.66:6789/0 conn(0x7f92980b43a0 legacy=0x7f92980b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.256+0000 7f928fe00640 1 -- 172.21.15.59:0/257079353 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929815f4a0 con 0x7f92980d0600 2024-04-24T06:29:27.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.256+0000 7f9295e00640 1 --2- 172.21.15.59:0/257079353 >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 0x7f929806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:29:27.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.257+0000 7f928fe00640 1 --2- 172.21.15.59:0/257079353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980d0a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92800097a0 tx=0x7f928002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93fc9dc38bc3085a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9295400640 1 -- 172.21.15.59:0/257079353 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280037070 con 0x7f92980d0600 2024-04-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9295400640 1 -- 172.21.15.59:0/257079353 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9280032440 con 0x7f92980d0600 2024-04-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9295400640 1 -- 172.21.15.59:0/257079353 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280032810 con 0x7f92980d0600 2024-04-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 -- 172.21.15.59:0/257079353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 msgr2=0x7f92980d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 --2- 172.21.15.59:0/257079353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980d0a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92800097a0 tx=0x7f928002a180 comp rx=0 tx=0).stop 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 -- 172.21.15.59:0/257079353 shutdown_connections 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 --2- 172.21.15.59:0/257079353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980d0a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 --2- 172.21.15.59:0/257079353 >> v2:172.21.15.66:3300/0 conn(0x7f9298155a80 0x7f929806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.258+0000 7f9297200640 1 -- 172.21.15.59:0/257079353 >> 172.21.15.59:0/257079353 conn(0x7f929806bdf0 msgr2=0x7f929806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.259+0000 7f9297200640 1 -- 172.21.15.59:0/257079353 shutdown_connections 2024-04-24T06:29:27.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.259+0000 7f9297200640 1 -- 172.21.15.59:0/257079353 wait complete. 2024-04-24T06:29:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.260+0000 7f9297200640 1 Processor -- start 2024-04-24T06:29:27.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.260+0000 7f9297200640 1 -- start start 2024-04-24T06:29:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 0x7f92980b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f929815fd40 con 0x7f9298155a80 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f929815fa60 con 0x7f92980b43a0 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.261+0000 7f9297200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f929806ff00 con 0x7f92980d0600 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37454/0 (socket says 172.21.15.59:37454) 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 -- 172.21.15.59:0/1235295034 learned_addr learned my addr 172.21.15.59:0/1235295034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9295e00640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 0x7f92980b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f928fe00640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 -- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 msgr2=0x7f92980b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 -- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 msgr2=0x7f92980b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 0x7f92980b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9296800640 1 -- 172.21.15.59:0/1235295034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92980b1510 con 0x7f9298155a80 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.262+0000 7f9295e00640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 0x7f92980b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:27.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f9296800640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f929000cbc0 tx=0x7f929000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:27.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290017070 con 0x7f9298155a80 2024-04-24T06:29:27.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929000ad40 con 0x7f9298155a80 2024-04-24T06:29:27.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290012600 con 0x7f9298155a80 2024-04-24T06:29:27.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9298141e50 con 0x7f9298155a80 2024-04-24T06:29:27.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.263+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92981426f0 con 0x7f9298155a80 2024-04-24T06:29:27.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.264+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9298142240 con 0x7f9298155a80 2024-04-24T06:29:27.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.265+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f929001a070 con 0x7f9298155a80 2024-04-24T06:29:27.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.266+0000 7f928e000640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:27.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.266+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f929008b900 con 0x7f9298155a80 2024-04-24T06:29:27.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.267+0000 7f9295e00640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:27.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.267+0000 7f9295e00640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 0x7f927406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f928800b240 tx=0x7f928800b3a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.271+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92900595c0 con 0x7f9298155a80 2024-04-24T06:29:27.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.474+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f929804fd90 con 0x7f9298155a80 2024-04-24T06:29:27.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.480+0000 7f928e000640 1 -- 172.21.15.59:0/1235295034 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f929005d8c0 con 0x7f9298155a80 2024-04-24T06:29:27.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 msgr2=0x7f927406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 0x7f927406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f928800b240 tx=0x7f928800b3a0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 msgr2=0x7f9298143e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f929000cbc0 tx=0x7f929000a430 comp rx=0 tx=0).stop 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 shutdown_connections 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9298155a80 0x7f9298143e40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92980d0600 0x7f92980b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 --2- 172.21.15.59:0/1235295034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92980b43a0 0x7f92980b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:27.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 >> 172.21.15.59:0/1235295034 conn(0x7f929806bdf0 msgr2=0x7f92980b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:27.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.483+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 shutdown_connections 2024-04-24T06:29:27.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:27.484+0000 7f9297200640 1 -- 172.21.15.59:0/1235295034 wait complete. 2024-04-24T06:29:27.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:29:27.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:29:27.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:29:27.752 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:29:27.752 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:27.752 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:27.811 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:27.812 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:27.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:28.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.074+0000 7faa13e00640 1 Processor -- start 2024-04-24T06:29:28.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.075+0000 7faa13e00640 1 -- start start 2024-04-24T06:29:28.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.076+0000 7faa13e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7faa140d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.076+0000 7faa13e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa140cf030 con 0x7faa140d2630 2024-04-24T06:29:28.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.076+0000 7faa13e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa140b6a40 con 0x7faa140d0d90 2024-04-24T06:29:28.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.076+0000 7faa13e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa140b7000 con 0x7faa1415e7c0 2024-04-24T06:29:28.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa0a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faa140d2630 0x7faa140b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54464/0 (socket says 172.21.15.59:54464) 2024-04-24T06:29:28.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa0a000640 1 -- 172.21.15.59:0/1182616437 learned_addr learned my addr 172.21.15.59:0/1182616437 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa12a00640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7faa140d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413215297 0 0) 0x7fa9fc001110 con 0x7faa140d2630 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9f8003520 con 0x7faa140d2630 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.077+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3777784095 0 0) 0x7fa9f8003520 con 0x7faa140d2630 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 msgr2=0x7faa140d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7faa140d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 >> v1:172.21.15.66:6789/0 conn(0x7faa140d0d90 legacy=0x7faa140cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa140b6d20 con 0x7faa140d2630 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa9fc001840 con 0x7faa140d2630 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 >> v1:172.21.15.59:6789/0 conn(0x7faa140d2630 legacy=0x7faa140b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80039e0 0x7fa9f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7fa9f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7fa9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa140b7000 con 0x7fa9f8004d40 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f800ba70 con 0x7fa9f80039e0 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f800bc70 con 0x7faa1415e7c0 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa13400640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7fa9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa12a00640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80039e0 0x7fa9f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.078+0000 7faa0a000640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7fa9f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.079+0000 7faa13400640 1 -- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 msgr2=0x7fa9f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.079+0000 7faa13400640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa1415e7c0 0x7fa9f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.079+0000 7faa13400640 1 -- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80039e0 msgr2=0x7fa9f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.079+0000 7faa13400640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9f80039e0 0x7fa9f8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.079+0000 7faa13400640 1 -- 172.21.15.59:0/1182616437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f8003520 con 0x7fa9f8004d40 2024-04-24T06:29:28.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa13400640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7fa9f80051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faa0c001e20 tx=0x7fa9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8991b1bb555313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c001a70 con 0x7fa9f8004d40 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0c02ae40 con 0x7fa9f8004d40 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa12000640 1 -- 172.21.15.59:0/1182616437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c003bb0 con 0x7fa9f8004d40 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa13e00640 1 -- 172.21.15.59:0/1182616437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 msgr2=0x7fa9f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa13e00640 1 --2- 172.21.15.59:0/1182616437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7fa9f80051c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faa0c001e20 tx=0x7fa9f8005830 comp rx=0 tx=0).stop 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.080+0000 7faa13400640 1 -- 172.21.15.59:0/1182616437 reap_dead start 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.081+0000 7faa13e00640 1 -- 172.21.15.59:0/1182616437 shutdown_connections 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.081+0000 7faa13e00640 1 -- 172.21.15.59:0/1182616437 >> 172.21.15.59:0/1182616437 conn(0x7faa1409d920 msgr2=0x7faa1408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.081+0000 7faa13e00640 1 -- 172.21.15.59:0/1182616437 shutdown_connections 2024-04-24T06:29:28.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.081+0000 7faa13e00640 1 -- 172.21.15.59:0/1182616437 wait complete. 2024-04-24T06:29:28.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.082+0000 7faa13e00640 1 Processor -- start 2024-04-24T06:29:28.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.083+0000 7faa13e00640 1 -- start start 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 0x7faa1414c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 0x7faa140b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa140b6700 con 0x7fa9f8004d40 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa140cf620 con 0x7faa140cdc70 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa140d1a80 con 0x7faa140b2540 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37462/0 (socket says 172.21.15.59:37462) 2024-04-24T06:29:28.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 -- 172.21.15.59:0/861820782 learned_addr learned my addr 172.21.15.59:0/861820782 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa0a000640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 0x7faa1414c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13400640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 0x7faa140b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 -- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 msgr2=0x7faa140b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 0x7faa140b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 -- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 msgr2=0x7faa1414c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 0x7faa1414c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa12a00640 1 -- 172.21.15.59:0/861820782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa140b1780 con 0x7fa9f8004d40 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa13400640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 0x7faa140b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.084+0000 7faa0a000640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 0x7faa1414c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa12a00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa04011600 tx=0x7faa04011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0400f920 con 0x7fa9f8004d40 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0400fac0 con 0x7fa9f8004d40 2024-04-24T06:29:28.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0400fe70 con 0x7fa9f8004d40 2024-04-24T06:29:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa140b1a60 con 0x7fa9f8004d40 2024-04-24T06:29:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.085+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa140b2020 con 0x7fa9f8004d40 2024-04-24T06:29:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.086+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1404fd90 con 0x7fa9f8004d40 2024-04-24T06:29:28.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.088+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa04015460 con 0x7fa9f8004d40 2024-04-24T06:29:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.088+0000 7faa0b400640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 0x7fa9f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.089+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7faa0408c610 con 0x7fa9f8004d40 2024-04-24T06:29:28.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.089+0000 7faa0a000640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 0x7fa9f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.090+0000 7faa0a000640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 0x7fa9f006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa1415a430 tx=0x7fa9fc001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.093+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa04019030 con 0x7fa9f8004d40 2024-04-24T06:29:28.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.301+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa140ce950 con 0x7fa9f8004d40 2024-04-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.303+0000 7faa0b400640 1 -- 172.21.15.59:0/861820782 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7faa0405a920 con 0x7fa9f8004d40 2024-04-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.306+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 msgr2=0x7fa9f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.306+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 0x7fa9f006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa1415a430 tx=0x7fa9fc001110 comp rx=0 tx=0).stop 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.306+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 msgr2=0x7faa140cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.306+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa04011600 tx=0x7faa04011ac0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 shutdown_connections 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa9f006c920 0x7fa9f006ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa140b2540 0x7faa140b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa140cdc70 0x7faa1414c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 --2- 172.21.15.59:0/861820782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f8004d40 0x7faa140cb740 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 >> 172.21.15.59:0/861820782 conn(0x7faa1409d920 msgr2=0x7faa1415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:28.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 shutdown_connections 2024-04-24T06:29:28.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.307+0000 7faa13e00640 1 -- 172.21.15.59:0/861820782 wait complete. 2024-04-24T06:29:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:28.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.539+0000 7f4907e00640 1 Processor -- start 2024-04-24T06:29:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.539+0000 7f4907e00640 1 -- start start 2024-04-24T06:29:28.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4907e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49080d0600 0x7f49080ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4907e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f490806dcb0 0x7f4908162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4907e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4908163240 con 0x7f49080d09e0 2024-04-24T06:29:28.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4907e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4908072a20 con 0x7f49080d0600 2024-04-24T06:29:28.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4907e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4908071e10 con 0x7f490806dcb0 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4906a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49080d0600 0x7f49080ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.541+0000 7f4906a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49080d0600 0x7f49080ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37466/0 (socket says 172.21.15.59:37466) 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906a00640 1 -- 172.21.15.59:0/3973966514 learned_addr learned my addr 172.21.15.59:0/3973966514 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906000640 1 --2- 172.21.15.59:0/3973966514 >> v2:172.21.15.66:3300/0 conn(0x7f490806dcb0 0x7f4908162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906a00640 1 -- 172.21.15.59:0/3973966514 >> v2:172.21.15.66:3300/0 conn(0x7f490806dcb0 msgr2=0x7f4908162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906a00640 1 --2- 172.21.15.59:0/3973966514 >> v2:172.21.15.66:3300/0 conn(0x7f490806dcb0 0x7f4908162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906a00640 1 -- 172.21.15.59:0/3973966514 >> v1:172.21.15.66:6789/0 conn(0x7f49080d09e0 legacy=0x7f490806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:28.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.542+0000 7f4906a00640 1 -- 172.21.15.59:0/3973966514 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49080720f0 con 0x7f49080d0600 2024-04-24T06:29:28.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.543+0000 7f4906a00640 1 --2- 172.21.15.59:0/3973966514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d0600 0x7f49080ad170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49080d11c0 tx=0x7f48f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467869a97ce19661 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.543+0000 7f4905600640 1 -- 172.21.15.59:0/3973966514 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8037070 con 0x7f49080d0600 2024-04-24T06:29:28.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4905600640 1 -- 172.21.15.59:0/3973966514 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f8032440 con 0x7f49080d0600 2024-04-24T06:29:28.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4905600640 1 -- 172.21.15.59:0/3973966514 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8032810 con 0x7f49080d0600 2024-04-24T06:29:28.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 -- 172.21.15.59:0/3973966514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d0600 msgr2=0x7f49080ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 --2- 172.21.15.59:0/3973966514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d0600 0x7f49080ad170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49080d11c0 tx=0x7f48f802a230 comp rx=0 tx=0).stop 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 -- 172.21.15.59:0/3973966514 shutdown_connections 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 --2- 172.21.15.59:0/3973966514 >> v2:172.21.15.66:3300/0 conn(0x7f490806dcb0 0x7f4908162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 --2- 172.21.15.59:0/3973966514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d0600 0x7f49080ad170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 -- 172.21.15.59:0/3973966514 >> 172.21.15.59:0/3973966514 conn(0x7f490806bdf0 msgr2=0x7f490806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 -- 172.21.15.59:0/3973966514 shutdown_connections 2024-04-24T06:29:28.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.544+0000 7f4907e00640 1 -- 172.21.15.59:0/3973966514 wait complete. 2024-04-24T06:29:28.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.546+0000 7f4907e00640 1 Processor -- start 2024-04-24T06:29:28.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.546+0000 7f4907e00640 1 -- start start 2024-04-24T06:29:28.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f490806dcb0 0x7f4908147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 0x7f49080c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f490815eaf0 con 0x7f49080d09e0 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f49080726b0 con 0x7f49080d0600 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f49080d0e90 con 0x7f490806dcb0 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.547+0000 7f4907400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37470/0 (socket says 172.21.15.59:37470) 2024-04-24T06:29:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 -- 172.21.15.59:0/388311729 learned_addr learned my addr 172.21.15.59:0/388311729 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4906000640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 0x7f49080c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 -- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f490806dcb0 msgr2=0x7f4908147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4906a00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f490806dcb0 0x7f4908147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f490806dcb0 0x7f4908147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 -- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 msgr2=0x7f49080c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 0x7f49080c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 -- 172.21.15.59:0/388311729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081481d0 con 0x7f49080d09e0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4906000640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 0x7f49080c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.548+0000 7f4907400640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f490000b970 tx=0x7f490000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.549+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900019070 con 0x7f49080d09e0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.549+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490000cb40 con 0x7f49080d09e0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.549+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900014440 con 0x7f49080d09e0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.549+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081484b0 con 0x7f49080d09e0 2024-04-24T06:29:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.549+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4908148a70 con 0x7f49080d09e0 2024-04-24T06:29:28.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.550+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490804fd90 con 0x7f49080d09e0 2024-04-24T06:29:28.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.551+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f490001c070 con 0x7f49080d09e0 2024-04-24T06:29:28.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.552+0000 7f48ff400640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 0x7f48e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:28.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.552+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f490008b7e0 con 0x7f49080d09e0 2024-04-24T06:29:28.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.552+0000 7f4906a00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 0x7f48e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:28.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.553+0000 7f4906a00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 0x7f48e806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:28.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.556+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4900059b80 con 0x7f49080d09e0 2024-04-24T06:29:28.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.761+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49080aeec0 con 0x7f49080d09e0 2024-04-24T06:29:28.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.762+0000 7f48ff400640 1 -- 172.21.15.59:0/388311729 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4900059480 con 0x7f49080d09e0 2024-04-24T06:29:28.764 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.764+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 msgr2=0x7f48e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 0x7f48e806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 msgr2=0x7f490815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f490000b970 tx=0x7f490000be30 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 shutdown_connections 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48e806c920 0x7f48e806ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49080d09e0 0x7f490815fcc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49080d0600 0x7f49080c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 --2- 172.21.15.59:0/388311729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f490806dcb0 0x7f4908147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:28.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 >> 172.21.15.59:0/388311729 conn(0x7f490806bdf0 msgr2=0x7f4908161050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:28.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 shutdown_connections 2024-04-24T06:29:28.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:28.765+0000 7f4907e00640 1 -- 172.21.15.59:0/388311729 wait complete. 2024-04-24T06:29:28.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.005+0000 7fbaad800640 1 Processor -- start 2024-04-24T06:29:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.005+0000 7fbaad800640 1 -- start start 2024-04-24T06:29:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.006+0000 7fbaad800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa80d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.006+0000 7fbaad800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa80d0de0 0x7fbaa80d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.006+0000 7fbaad800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa80d22e0 con 0x7fbaa80cfc50 2024-04-24T06:29:29.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.006+0000 7fbaad800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa80d2ba0 con 0x7fbaa80d0de0 2024-04-24T06:29:29.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.006+0000 7fbaad800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa80d2550 con 0x7fbaa8161c50 2024-04-24T06:29:29.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaace00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa80d0de0 0x7fbaa80d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaace00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa80d0de0 0x7fbaa80d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60864/0 (socket says 172.21.15.59:60864) 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaace00640 1 -- 172.21.15.59:0/3626788018 learned_addr learned my addr 172.21.15.59:0/3626788018 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaa7e00640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa80d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaace00640 1 -- 172.21.15.59:0/3626788018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 msgr2=0x7fbaa80d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.007+0000 7fbaace00640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa80d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.008+0000 7fbaace00640 1 -- 172.21.15.59:0/3626788018 >> v1:172.21.15.66:6789/0 conn(0x7fbaa80cfc50 legacy=0x7fbaa80d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:29.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.008+0000 7fbaace00640 1 -- 172.21.15.59:0/3626788018 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80d2830 con 0x7fbaa80d0de0 2024-04-24T06:29:29.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.009+0000 7fbaace00640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa80d1270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009e50 tx=0x7fbaa002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0d790640d516df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.009+0000 7fbaa6a00640 1 -- 172.21.15.59:0/3626788018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0036070 con 0x7fbaa80d0de0 2024-04-24T06:29:29.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.009+0000 7fbaa6a00640 1 -- 172.21.15.59:0/3626788018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa0031600 con 0x7fbaa80d0de0 2024-04-24T06:29:29.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.009+0000 7fbaa6a00640 1 -- 172.21.15.59:0/3626788018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00319d0 con 0x7fbaa80d0de0 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 -- 172.21.15.59:0/3626788018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 msgr2=0x7fbaa80d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa80d1270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009e50 tx=0x7fbaa002a710 comp rx=0 tx=0).stop 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 -- 172.21.15.59:0/3626788018 shutdown_connections 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa80d1270 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 --2- 172.21.15.59:0/3626788018 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa80d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 -- 172.21.15.59:0/3626788018 >> 172.21.15.59:0/3626788018 conn(0x7fbaa808bf90 msgr2=0x7fbaa80d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 -- 172.21.15.59:0/3626788018 shutdown_connections 2024-04-24T06:29:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.010+0000 7fbaad800640 1 -- 172.21.15.59:0/3626788018 wait complete. 2024-04-24T06:29:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.012+0000 7fbaad800640 1 Processor -- start 2024-04-24T06:29:29.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.012+0000 7fbaad800640 1 -- start start 2024-04-24T06:29:29.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 0x7fbaa814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa8151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80d0090 con 0x7fbaa80d0de0 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80d0460 con 0x7fbaa80cfc50 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaad800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa80d0210 con 0x7fbaa8161c50 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaa7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaa7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60876/0 (socket says 172.21.15.59:60876) 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.013+0000 7fbaa7400640 1 -- 172.21.15.59:0/2791827520 learned_addr learned my addr 172.21.15.59:0/2791827520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7e00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 0x7fbaa814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaace00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa8151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 -- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 msgr2=0x7fbaa8151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa8151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 -- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 msgr2=0x7fbaa814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 0x7fbaa814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 -- 172.21.15.59:0/2791827520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80cd6c0 con 0x7fbaa80d0de0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaace00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7e00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 0x7fbaa814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaa7400640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c480 tx=0x7fba9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940052d0 con 0x7fbaa80d0de0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba94005470 con 0x7fbaa80d0de0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa80cd9a0 con 0x7fbaa80d0de0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.014+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94012870 con 0x7fbaa80d0de0 2024-04-24T06:29:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.015+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa80cdf60 con 0x7fbaa80d0de0 2024-04-24T06:29:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.016+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa80cfa60 con 0x7fbaa80d0de0 2024-04-24T06:29:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.017+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba94012a10 con 0x7fbaa80d0de0 2024-04-24T06:29:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.018+0000 7fba9fe00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 0x7fba8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.018+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fba9408b880 con 0x7fbaa80d0de0 2024-04-24T06:29:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.018+0000 7fbaa7e00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 0x7fba8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.019+0000 7fbaa7e00640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 0x7fba8806ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba98006fd0 tx=0x7fba98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.023+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba94059c20 con 0x7fbaa80d0de0 2024-04-24T06:29:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.226+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbaa804fd90 con 0x7fbaa80d0de0 2024-04-24T06:29:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.227+0000 7fba9fe00640 1 -- 172.21.15.59:0/2791827520 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fba94059520 con 0x7fbaa80d0de0 2024-04-24T06:29:29.229 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.230+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 msgr2=0x7fba8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.230+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 0x7fba8806ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba98006fd0 tx=0x7fba98008040 comp rx=0 tx=0).stop 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.230+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 msgr2=0x7fbaa8150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.230+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c480 tx=0x7fba9400c940 comp rx=0 tx=0).stop 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 shutdown_connections 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba8806c920 0x7fba8806ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbaa8161c50 0x7fbaa8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa80d0de0 0x7fbaa8150c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 --2- 172.21.15.59:0/2791827520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbaa80cfc50 0x7fbaa814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 >> 172.21.15.59:0/2791827520 conn(0x7fbaa808bf90 msgr2=0x7fbaa80b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 shutdown_connections 2024-04-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.231+0000 7fbaad800640 1 -- 172.21.15.59:0/2791827520 wait complete. 2024-04-24T06:29:29.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.462+0000 7f5fdb000640 1 Processor -- start 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.462+0000 7f5fdb000640 1 -- start start 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fdb000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd415ed20 0x7f5fd40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fdb000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40d13e0 0x7f5fd40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fdb000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd40b5da0 con 0x7f5fd40b3600 2024-04-24T06:29:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fdb000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd415f100 con 0x7f5fd415ed20 2024-04-24T06:29:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fdb000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd40d07d0 con 0x7f5fd40d13e0 2024-04-24T06:29:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fd9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd415ed20 0x7f5fd40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fd9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd415ed20 0x7f5fd40d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60880/0 (socket says 172.21.15.59:60880) 2024-04-24T06:29:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.464+0000 7f5fd9c00640 1 -- 172.21.15.59:0/2424067767 learned_addr learned my addr 172.21.15.59:0/2424067767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.465+0000 7f5fda600640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40d13e0 0x7f5fd40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.465+0000 7f5fd9c00640 1 -- 172.21.15.59:0/2424067767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40d13e0 msgr2=0x7f5fd40d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.465+0000 7f5fd9c00640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40d13e0 0x7f5fd40d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.465+0000 7f5fd9c00640 1 -- 172.21.15.59:0/2424067767 >> v1:172.21.15.66:6789/0 conn(0x7f5fd40b3600 legacy=0x7f5fd40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.465+0000 7f5fd9c00640 1 -- 172.21.15.59:0/2424067767 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd40d0a20 con 0x7f5fd415ed20 2024-04-24T06:29:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.466+0000 7f5fd9c00640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd40d0ea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009870 tx=0x7f5fc402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811b5274cd19c326 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2424067767 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4037070 con 0x7f5fd415ed20 2024-04-24T06:29:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2424067767 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc402ae10 con 0x7f5fd415ed20 2024-04-24T06:29:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2424067767 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4032660 con 0x7f5fd415ed20 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 -- 172.21.15.59:0/2424067767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 msgr2=0x7f5fd40d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd40d0ea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009870 tx=0x7f5fc402a100 comp rx=0 tx=0).stop 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 -- 172.21.15.59:0/2424067767 shutdown_connections 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40d13e0 0x7f5fd40d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 --2- 172.21.15.59:0/2424067767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd40d0ea0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.467+0000 7f5fdb000640 1 -- 172.21.15.59:0/2424067767 >> 172.21.15.59:0/2424067767 conn(0x7f5fd408bf90 msgr2=0x7f5fd40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.468+0000 7f5fdb000640 1 -- 172.21.15.59:0/2424067767 shutdown_connections 2024-04-24T06:29:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.468+0000 7f5fdb000640 1 -- 172.21.15.59:0/2424067767 wait complete. 2024-04-24T06:29:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.469+0000 7f5fdb000640 1 Processor -- start 2024-04-24T06:29:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.469+0000 7f5fdb000640 1 -- start start 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40b3600 0x7f5fd40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40d13e0 0x7f5fd40b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd40d3490 con 0x7f5fd415ed20 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd40d31b0 con 0x7f5fd40d13e0 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.470+0000 7f5fdb000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd40b4620 con 0x7f5fd40b3600 2024-04-24T06:29:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60892/0 (socket says 172.21.15.59:60892) 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 -- 172.21.15.59:0/1659038280 learned_addr learned my addr 172.21.15.59:0/1659038280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fd9200640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40d13e0 0x7f5fd40b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fd9c00640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40b3600 0x7f5fd40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 -- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40b3600 msgr2=0x7f5fd40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40b3600 0x7f5fd40ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 -- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40d13e0 msgr2=0x7f5fd40b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40d13e0 0x7f5fd40b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.471+0000 7f5fda600640 1 -- 172.21.15.59:0/1659038280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd40ba110 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fda600640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc007e40 tx=0x7f5fcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc014620 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc019070 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd4151b90 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc014a20 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.472+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd4152150 con 0x7f5fd415ed20 2024-04-24T06:29:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.473+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd404fd90 con 0x7f5fd415ed20 2024-04-24T06:29:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.474+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fcc014bc0 con 0x7f5fd415ed20 2024-04-24T06:29:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.475+0000 7f5fd1600640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 0x7f5fb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.475+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5fcc08c440 con 0x7f5fd415ed20 2024-04-24T06:29:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.476+0000 7f5fd9c00640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 0x7f5fb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.477+0000 7f5fd9c00640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 0x7f5fb406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4034000 tx=0x7f5fc402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.480+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fcc011260 con 0x7f5fd415ed20 2024-04-24T06:29:29.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.685+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5fd40b4290 con 0x7f5fd415ed20 2024-04-24T06:29:29.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.686+0000 7f5fd1600640 1 -- 172.21.15.59:0/1659038280 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f5fcc05a7e0 con 0x7f5fd415ed20 2024-04-24T06:29:29.688 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 msgr2=0x7f5fb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 0x7f5fb406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4034000 tx=0x7f5fc402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 msgr2=0x7f5fd409f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc007e40 tx=0x7f5fcc00e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 shutdown_connections 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb406c920 0x7f5fb406ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd415ed20 0x7f5fd409f350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40d13e0 0x7f5fd40b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 --2- 172.21.15.59:0/1659038280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd40b3600 0x7f5fd40ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 >> 172.21.15.59:0/1659038280 conn(0x7f5fd408bf90 msgr2=0x7f5fd40d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 shutdown_connections 2024-04-24T06:29:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.689+0000 7f5fdb000640 1 -- 172.21.15.59:0/1659038280 wait complete. 2024-04-24T06:29:29.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:29.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.922+0000 7fc4ab200640 1 Processor -- start 2024-04-24T06:29:29.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.922+0000 7fc4ab200640 1 -- start start 2024-04-24T06:29:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4ab200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc4ac15c670 0x7fc4ac0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4ab200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4ac0d0cd0 0x7fc4ac0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4ab200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ac0b4cf0 con 0x7fc4ac0b4230 2024-04-24T06:29:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4ab200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ac15ca50 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4ab200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ac0ccb80 con 0x7fc4ac15c670 2024-04-24T06:29:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4a1400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc4ac0b4230 0x7fc4ac0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54298/0 (socket says 172.21.15.59:54298) 2024-04-24T06:29:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.924+0000 7fc4a1400640 1 -- 172.21.15.59:0/2147129984 learned_addr learned my addr 172.21.15.59:0/2147129984 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4aa800640 1 --2- 172.21.15.59:0/2147129984 >> v2:172.21.15.59:3300/0 conn(0x7fc4ac0d0cd0 0x7fc4ac0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4a9e00640 1 --2- 172.21.15.59:0/2147129984 >> v2:172.21.15.66:3300/0 conn(0x7fc4ac15c670 0x7fc4ac0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4a9400640 1 -- 172.21.15.59:0/2147129984 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839336226 0 0) 0x7fc4ac0b4cf0 con 0x7fc4ac0b4230 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4a9400640 1 -- 172.21.15.59:0/2147129984 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc490003520 con 0x7fc4ac0b4230 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4aa800640 1 -- 172.21.15.59:0/2147129984 >> v2:172.21.15.66:3300/0 conn(0x7fc4ac15c670 msgr2=0x7fc4ac0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4aa800640 1 --2- 172.21.15.59:0/2147129984 >> v2:172.21.15.66:3300/0 conn(0x7fc4ac15c670 0x7fc4ac0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4aa800640 1 -- 172.21.15.59:0/2147129984 >> v1:172.21.15.59:6789/0 conn(0x7fc4ac0b4230 legacy=0x7fc4ac0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.925+0000 7fc4aa800640 1 -- 172.21.15.59:0/2147129984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac0ccdd0 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.926+0000 7fc4aa800640 1 --2- 172.21.15.59:0/2147129984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac0b4690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0d1270 tx=0x7fc4a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac7e292a6ba2b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4a9400640 1 -- 172.21.15.59:0/2147129984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4007740 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4a9400640 1 -- 172.21.15.59:0/2147129984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a40078e0 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4a9400640 1 -- 172.21.15.59:0/2147129984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4007cb0 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 -- 172.21.15.59:0/2147129984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 msgr2=0x7fc4ac0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 --2- 172.21.15.59:0/2147129984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac0b4690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0d1270 tx=0x7fc4a402a680 comp rx=0 tx=0).stop 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 -- 172.21.15.59:0/2147129984 shutdown_connections 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 --2- 172.21.15.59:0/2147129984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac0b4690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 --2- 172.21.15.59:0/2147129984 >> v2:172.21.15.66:3300/0 conn(0x7fc4ac15c670 0x7fc4ac0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 -- 172.21.15.59:0/2147129984 >> 172.21.15.59:0/2147129984 conn(0x7fc4ac06bdf0 msgr2=0x7fc4ac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.927+0000 7fc4ab200640 1 -- 172.21.15.59:0/2147129984 shutdown_connections 2024-04-24T06:29:29.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.928+0000 7fc4ab200640 1 -- 172.21.15.59:0/2147129984 wait complete. 2024-04-24T06:29:29.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.929+0000 7fc4ab200640 1 Processor -- start 2024-04-24T06:29:29.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.929+0000 7fc4ab200640 1 -- start start 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4ac0b4230 0x7fc4ac0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4ac15c670 0x7fc4ac073360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac0b3e60 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac0cd390 con 0x7fc4ac0b4230 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.930+0000 7fc4ab200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ac0b3fe0 con 0x7fc4ac15c670 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60908/0 (socket says 172.21.15.59:60908) 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 -- 172.21.15.59:0/1669943508 learned_addr learned my addr 172.21.15.59:0/1669943508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a9e00640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4ac0b4230 0x7fc4ac0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 -- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4ac15c670 msgr2=0x7fc4ac073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4aa800640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4ac15c670 0x7fc4ac073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4ac15c670 0x7fc4ac073360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 -- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4ac0b4230 msgr2=0x7fc4ac0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4ac0b4230 0x7fc4ac0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:29.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.931+0000 7fc4a1400640 1 -- 172.21.15.59:0/1669943508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac0b1400 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4a1400640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc49400c430 tx=0x7fc49400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494017070 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494005d00 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494007310 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac0b16e0 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.932+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac0b1f80 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.933+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ac04fd90 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.934+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc49401a070 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.936+0000 7fc4a2a00640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 0x7fc48806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:29.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.936+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc49408b110 con 0x7fc4ac0d0cd0 2024-04-24T06:29:29.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.936+0000 7fc4a9e00640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 0x7fc48806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:29.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.937+0000 7fc4a9e00640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 0x7fc48806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006fd0 tx=0x7fc49c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:29.942+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc494059420 con 0x7fc4ac0d0cd0 2024-04-24T06:29:30.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.161+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4ac075dd0 con 0x7fc4ac0d0cd0 2024-04-24T06:29:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.161+0000 7fc4a2a00640 1 -- 172.21.15.59:0/1669943508 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc494058d20 con 0x7fc4ac0d0cd0 2024-04-24T06:29:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 msgr2=0x7fc48806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 0x7fc48806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc49c006fd0 tx=0x7fc49c008040 comp rx=0 tx=0).stop 2024-04-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 msgr2=0x7fc4ac1f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc49400c430 tx=0x7fc49400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 shutdown_connections 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc48806c8f0 0x7fc48806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4ac15c670 0x7fc4ac073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4ac0d0cd0 0x7fc4ac1f46b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 --2- 172.21.15.59:0/1669943508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4ac0b4230 0x7fc4ac0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 >> 172.21.15.59:0/1669943508 conn(0x7fc4ac06bdf0 msgr2=0x7fc4ac09a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.164+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 shutdown_connections 2024-04-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.165+0000 7fc4ab200640 1 -- 172.21.15.59:0/1669943508 wait complete. 2024-04-24T06:29:30.186 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:30.186 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:30.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:30.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.438+0000 7f55d5000640 1 Processor -- start 2024-04-24T06:29:30.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.439+0000 7f55d5000640 1 -- start start 2024-04-24T06:29:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55d5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55d015c3d0 0x7f55d00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55d5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55d5000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d0160210 con 0x7f55d00cd590 2024-04-24T06:29:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55d5000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f55d01604f0 con 0x7f55d015c3d0 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55d5000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f55d01607d0 con 0x7f55d00b4860 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55cf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55d015c3d0 0x7f55d00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55cf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55d015c3d0 0x7f55d00cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60916/0 (socket says 172.21.15.59:60916) 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.440+0000 7f55cf400640 1 -- 172.21.15.59:0/2695273851 learned_addr learned my addr 172.21.15.59:0/2695273851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.441+0000 7f55cf400640 1 -- 172.21.15.59:0/2695273851 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 msgr2=0x7f55d00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.441+0000 7f55cfe00640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.441+0000 7f55cf400640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.441+0000 7f55cf400640 1 -- 172.21.15.59:0/2695273851 >> v1:172.21.15.66:6789/0 conn(0x7f55d00cd590 legacy=0x7f55d00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.441+0000 7f55cf400640 1 -- 172.21.15.59:0/2695273851 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d0160ab0 con 0x7f55d015c3d0 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.442+0000 7f55cf400640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cd050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21203b4e91aaf578 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.442+0000 7f55ce000640 1 -- 172.21.15.59:0/2695273851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0037070 con 0x7f55d015c3d0 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55ce000640 1 -- 172.21.15.59:0/2695273851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0032440 con 0x7f55d015c3d0 2024-04-24T06:29:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55ce000640 1 -- 172.21.15.59:0/2695273851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0032810 con 0x7f55d015c3d0 2024-04-24T06:29:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 -- 172.21.15.59:0/2695273851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 msgr2=0x7f55d00cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cd050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002a210 comp rx=0 tx=0).stop 2024-04-24T06:29:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 -- 172.21.15.59:0/2695273851 shutdown_connections 2024-04-24T06:29:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 --2- 172.21.15.59:0/2695273851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cd050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 -- 172.21.15.59:0/2695273851 >> 172.21.15.59:0/2695273851 conn(0x7f55d00b21a0 msgr2=0x7f55d00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 -- 172.21.15.59:0/2695273851 shutdown_connections 2024-04-24T06:29:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.443+0000 7f55d5000640 1 -- 172.21.15.59:0/2695273851 wait complete. 2024-04-24T06:29:30.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.445+0000 7f55d5000640 1 Processor -- start 2024-04-24T06:29:30.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.445+0000 7f55d5000640 1 -- start start 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 0x7f55d00cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55d015c930 con 0x7f55d015c3d0 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55d015c7b0 con 0x7f55d00cd590 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55d5000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f55d00cbd10 con 0x7f55d00b4860 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.446+0000 7f55cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60930/0 (socket says 172.21.15.59:60930) 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 -- 172.21.15.59:0/643097751 learned_addr learned my addr 172.21.15.59:0/643097751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cf400640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cea00640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 0x7f55d00cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 -- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 msgr2=0x7f55d00c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 -- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 msgr2=0x7f55d00cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 0x7f55d00cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 -- 172.21.15.59:0/643097751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d00cdeb0 con 0x7f55d015c3d0 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cf400640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cea00640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 0x7f55d00cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:30.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.447+0000 7f55cfe00640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55c800cd70 tx=0x7f55c800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.448+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8003b90 con 0x7f55d015c3d0 2024-04-24T06:29:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.448+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c8003d30 con 0x7f55d015c3d0 2024-04-24T06:29:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.448+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004240 con 0x7f55d015c3d0 2024-04-24T06:29:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.448+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d00ce190 con 0x7f55d015c3d0 2024-04-24T06:29:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.448+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d00b0390 con 0x7f55d015c3d0 2024-04-24T06:29:30.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.449+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d004fd90 con 0x7f55d015c3d0 2024-04-24T06:29:30.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.450+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55c80043e0 con 0x7f55d015c3d0 2024-04-24T06:29:30.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.451+0000 7f55c7400640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 0x7f55b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.451+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f55c808c6c0 con 0x7f55d015c3d0 2024-04-24T06:29:30.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.451+0000 7f55cf400640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 0x7f55b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.452+0000 7f55cf400640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 0x7f55b006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55c0034000 tx=0x7f55c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.455+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55c801d340 con 0x7f55d015c3d0 2024-04-24T06:29:30.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.658+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55d006e400 con 0x7f55d015c3d0 2024-04-24T06:29:30.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.660+0000 7f55c7400640 1 -- 172.21.15.59:0/643097751 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f55c805aa60 con 0x7f55d015c3d0 2024-04-24T06:29:30.663 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:30.663 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.662+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 msgr2=0x7f55b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.662+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 0x7f55b006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55c0034000 tx=0x7f55c002e000 comp rx=0 tx=0).stop 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.662+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 msgr2=0x7f55d00cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.662+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f55c800cd70 tx=0x7f55c800a660 comp rx=0 tx=0).stop 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 shutdown_connections 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f55b006c8f0 0x7f55b006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55d015c3d0 0x7f55d00cf760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55d00cd590 0x7f55d00cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 --2- 172.21.15.59:0/643097751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55d00b4860 0x7f55d00c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 >> 172.21.15.59:0/643097751 conn(0x7f55d00b21a0 msgr2=0x7f55d006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:30.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 shutdown_connections 2024-04-24T06:29:30.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.663+0000 7f55d5000640 1 -- 172.21.15.59:0/643097751 wait complete. 2024-04-24T06:29:30.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:30.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:30.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.901+0000 7f9c7e800640 1 Processor -- start 2024-04-24T06:29:30.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.902+0000 7f9c7e800640 1 -- start start 2024-04-24T06:29:30.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.903+0000 7f9c7e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c780b56f0 0x7f9c7806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.903+0000 7f9c7e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c7806f340 0x7f9c7806f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.903+0000 7f9c7e800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7815bf60 con 0x7f9c780d0350 2024-04-24T06:29:30.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.903+0000 7f9c7e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7815c240 con 0x7f9c780b56f0 2024-04-24T06:29:30.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.903+0000 7f9c7e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7815c520 con 0x7f9c7806f340 2024-04-24T06:29:30.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c780b56f0 0x7f9c7806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9c780d0350 0x7f9c780b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54308/0 (socket says 172.21.15.59:54308) 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77e00640 1 -- 172.21.15.59:0/2480592047 learned_addr learned my addr 172.21.15.59:0/2480592047 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c76a00640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c7806f340 0x7f9c7806f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77400640 1 -- 172.21.15.59:0/2480592047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c7806f340 msgr2=0x7f9c7806f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77400640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c7806f340 0x7f9c7806f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77400640 1 -- 172.21.15.59:0/2480592047 >> v1:172.21.15.59:6789/0 conn(0x7f9c780d0350 legacy=0x7f9c780b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.904+0000 7f9c77400640 1 -- 172.21.15.59:0/2480592047 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7815c800 con 0x7f9c780b56f0 2024-04-24T06:29:30.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.906+0000 7f9c77400640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7806ddd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009d90 tx=0x7f9c6802c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5f397d63f39d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.906+0000 7f9c76000640 1 -- 172.21.15.59:0/2480592047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68032770 con 0x7f9c780b56f0 2024-04-24T06:29:30.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.906+0000 7f9c76000640 1 -- 172.21.15.59:0/2480592047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68038070 con 0x7f9c780b56f0 2024-04-24T06:29:30.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.906+0000 7f9c76000640 1 -- 172.21.15.59:0/2480592047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68032b30 con 0x7f9c780b56f0 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 -- 172.21.15.59:0/2480592047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 msgr2=0x7f9c7806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7806ddd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009d90 tx=0x7f9c6802c870 comp rx=0 tx=0).stop 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 -- 172.21.15.59:0/2480592047 shutdown_connections 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c7806f340 0x7f9c7806f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2480592047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7806ddd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 -- 172.21.15.59:0/2480592047 >> 172.21.15.59:0/2480592047 conn(0x7f9c78067d90 msgr2=0x7f9c7806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 -- 172.21.15.59:0/2480592047 shutdown_connections 2024-04-24T06:29:30.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.907+0000 7f9c7e800640 1 -- 172.21.15.59:0/2480592047 wait complete. 2024-04-24T06:29:30.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.909+0000 7f9c7e800640 1 Processor -- start 2024-04-24T06:29:30.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.909+0000 7f9c7e800640 1 -- start start 2024-04-24T06:29:30.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7806f340 0x7f9c78155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c780d0350 0x7f9c78156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780d07e0 con 0x7f9c780b56f0 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7815bc10 con 0x7f9c7806f340 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c7e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780d0960 con 0x7f9c780d0350 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60944/0 (socket says 172.21.15.59:60944) 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.910+0000 7f9c76a00640 1 -- 172.21.15.59:0/2854859390 learned_addr learned my addr 172.21.15.59:0/2854859390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 -- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c780d0350 msgr2=0x7f9c78156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c77400640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7806f340 0x7f9c78155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c780d0350 0x7f9c78156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 -- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7806f340 msgr2=0x7f9c78155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7806f340 0x7f9c78155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 -- 172.21.15.59:0/2854859390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c781f4a40 con 0x7f9c780b56f0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c76a00640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d800 tx=0x7f9c6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000b6c0 con 0x7f9c780b56f0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c60019070 con 0x7f9c780b56f0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000ba80 con 0x7f9c780b56f0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.911+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7815d7a0 con 0x7f9c780b56f0 2024-04-24T06:29:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.912+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7815dd60 con 0x7f9c780b56f0 2024-04-24T06:29:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.913+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c78068a80 con 0x7f9c780b56f0 2024-04-24T06:29:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.914+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c6000bc20 con 0x7f9c780b56f0 2024-04-24T06:29:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.914+0000 7f9c6f400640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.915+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9c6008c160 con 0x7f9c780b56f0 2024-04-24T06:29:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.915+0000 7f9c77400640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.916+0000 7f9c77400640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 0x7f9c5406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c68037000 tx=0x7f9c6800d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:30.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:30.919+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c781f4a40 con 0x7f9c780b56f0 2024-04-24T06:29:31.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.122+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c7806ef20 con 0x7f9c780b56f0 2024-04-24T06:29:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.123+0000 7f9c6f400640 1 -- 172.21.15.59:0/2854859390 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f9c6001c360 con 0x7f9c780b56f0 2024-04-24T06:29:31.125 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 msgr2=0x7f9c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 0x7f9c5406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c68037000 tx=0x7f9c6800d040 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 msgr2=0x7f9c7815e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d800 tx=0x7f9c6000dcc0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 shutdown_connections 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9c780d0350 0x7f9c78156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c780b56f0 0x7f9c7815e560 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 --2- 172.21.15.59:0/2854859390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c7806f340 0x7f9c78155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.125+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 >> 172.21.15.59:0/2854859390 conn(0x7f9c78067d90 msgr2=0x7f9c780b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.126+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 shutdown_connections 2024-04-24T06:29:31.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.126+0000 7f9c7e800640 1 -- 172.21.15.59:0/2854859390 wait complete. 2024-04-24T06:29:31.147 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.355+0000 7ff329c00640 1 Processor -- start 2024-04-24T06:29:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.355+0000 7ff329c00640 1 -- start start 2024-04-24T06:29:31.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff329c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff3241567e0 0x7ff324156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff329c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32415cab0 con 0x7ff32415d810 2024-04-24T06:29:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff329c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff324160e40 con 0x7ff3240d0d10 2024-04-24T06:29:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff329c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff32415cd30 con 0x7ff3241567e0 2024-04-24T06:29:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff323400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff32415d810 0x7ff3240d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54322/0 (socket says 172.21.15.59:54322) 2024-04-24T06:29:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.357+0000 7ff323400640 1 -- 172.21.15.59:0/1213373193 learned_addr learned my addr 172.21.15.59:0/1213373193 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:31.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.358+0000 7ff323e00640 1 --2- 172.21.15.59:0/1213373193 >> v2:172.21.15.66:3300/0 conn(0x7ff3241567e0 0x7ff324156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.358+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487220824 0 0) 0x7ff310001110 con 0x7ff32415d810 2024-04-24T06:29:31.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.358+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff308003520 con 0x7ff32415d810 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.358+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2968878749 0 0) 0x7ff308003520 con 0x7ff32415d810 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 >> v2:172.21.15.66:3300/0 conn(0x7ff3241567e0 msgr2=0x7ff324156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 --2- 172.21.15.59:0/1213373193 >> v2:172.21.15.66:3300/0 conn(0x7ff3241567e0 0x7ff324156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 >> v1:172.21.15.66:6789/0 conn(0x7ff3240d0d10 legacy=0x7ff324160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32415d010 con 0x7ff32415d810 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff310001840 con 0x7ff32415d810 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 >> v1:172.21.15.59:6789/0 conn(0x7ff32415d810 legacy=0x7ff3240d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff308003a00 0x7ff308003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3080051c0 0x7ff30800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff32415cd30 con 0x7ff308004800 2024-04-24T06:29:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff30800c0c0 con 0x7ff308003a00 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff30800c2c0 con 0x7ff3080051c0 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff323400640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff323e00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff308003a00 0x7ff308003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff329200640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3080051c0 0x7ff30800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.359+0000 7ff323400640 1 -- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3080051c0 msgr2=0x7ff30800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.360+0000 7ff323400640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3080051c0 0x7ff30800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.360+0000 7ff323400640 1 -- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff308003a00 msgr2=0x7ff308003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.360+0000 7ff323400640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff308003a00 0x7ff308003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.360+0000 7ff323400640 1 -- 172.21.15.59:0/1213373193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff308003520 con 0x7ff308004800 2024-04-24T06:29:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.360+0000 7ff329200640 1 -- 172.21.15.59:0/1213373193 reap_dead start 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff323400640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff308004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff32415dc20 tx=0x7ff308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8247ef05781d3945 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310036070 con 0x7ff308004800 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff310005ad0 con 0x7ff308004800 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff322a00640 1 -- 172.21.15.59:0/1213373193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310007e30 con 0x7ff308004800 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff329c00640 1 -- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 msgr2=0x7ff308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.361+0000 7ff329c00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff308004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff32415dc20 tx=0x7ff308005830 comp rx=0 tx=0).stop 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.362+0000 7ff329c00640 1 -- 172.21.15.59:0/1213373193 shutdown_connections 2024-04-24T06:29:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.362+0000 7ff329c00640 1 --2- 172.21.15.59:0/1213373193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff308004c80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.362+0000 7ff329c00640 1 -- 172.21.15.59:0/1213373193 >> 172.21.15.59:0/1213373193 conn(0x7ff32406bdf0 msgr2=0x7ff32406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.362+0000 7ff329c00640 1 -- 172.21.15.59:0/1213373193 shutdown_connections 2024-04-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.362+0000 7ff329c00640 1 -- 172.21.15.59:0/1213373193 wait complete. 2024-04-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.363+0000 7ff329c00640 1 Processor -- start 2024-04-24T06:29:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.364+0000 7ff329c00640 1 -- start start 2024-04-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff32414eed0 0x7ff32415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32414f350 0x7ff32415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff324156d40 con 0x7ff308004800 2024-04-24T06:29:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff324156bc0 con 0x7ff32414f350 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff329c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff324156ec0 con 0x7ff32414eed0 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60952/0 (socket says 172.21.15.59:60952) 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 -- 172.21.15.59:0/1849212620 learned_addr learned my addr 172.21.15.59:0/1849212620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 -- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff32414eed0 msgr2=0x7ff32415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff32414eed0 0x7ff32415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 -- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32414f350 msgr2=0x7ff32415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32414f350 0x7ff32415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.365+0000 7ff323e00640 1 -- 172.21.15.59:0/1849212620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3241f7fb0 con 0x7ff308004800 2024-04-24T06:29:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff323e00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff3140116d0 tx=0x7ff314011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400f9c0 con 0x7ff308004800 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31400fb60 con 0x7ff308004800 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314015460 con 0x7ff308004800 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3241f8290 con 0x7ff308004800 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.366+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3241f8850 con 0x7ff308004800 2024-04-24T06:29:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.367+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32404fd90 con 0x7ff308004800 2024-04-24T06:29:31.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.368+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff314015600 con 0x7ff308004800 2024-04-24T06:29:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.369+0000 7ff31be00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 0x7ff30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.370+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff31408c630 con 0x7ff308004800 2024-04-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.370+0000 7ff323400640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 0x7ff30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.371+0000 7ff323400640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 0x7ff30406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff310033000 tx=0x7ff310009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.374+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff314019030 con 0x7ff308004800 2024-04-24T06:29:31.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.579+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3240b5150 con 0x7ff308004800 2024-04-24T06:29:31.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.579+0000 7ff31be00640 1 -- 172.21.15.59:0/1849212620 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff31405a9d0 con 0x7ff308004800 2024-04-24T06:29:31.581 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:31.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 msgr2=0x7ff30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 0x7ff30406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff310033000 tx=0x7ff310009030 comp rx=0 tx=0).stop 2024-04-24T06:29:31.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 msgr2=0x7ff3241f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff3140116d0 tx=0x7ff314011b90 comp rx=0 tx=0).stop 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 shutdown_connections 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff30406c8f0 0x7ff30406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff32414f350 0x7ff32415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff32414eed0 0x7ff32415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 --2- 172.21.15.59:0/1849212620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff308004800 0x7ff3241f75d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 >> 172.21.15.59:0/1849212620 conn(0x7ff32406bdf0 msgr2=0x7ff324071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 shutdown_connections 2024-04-24T06:29:31.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.582+0000 7ff329c00640 1 -- 172.21.15.59:0/1849212620 wait complete. 2024-04-24T06:29:31.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:31.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.821+0000 7fecefe00640 1 Processor -- start 2024-04-24T06:29:31.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.822+0000 7fecefe00640 1 -- start start 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecefe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf00d0d10 0x7fecf0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecefe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fecf015d810 0x7fecf0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecefe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf015cab0 con 0x7fecf01567e0 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecefe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fecf0160e40 con 0x7fecf015d810 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecefe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fecf015cd30 con 0x7fecf00d0d10 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecef400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fecf015d810 0x7fecf0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecef400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fecf015d810 0x7fecf0160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60958/0 (socket says 172.21.15.59:60958) 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.823+0000 7fecef400640 1 -- 172.21.15.59:0/514573022 learned_addr learned my addr 172.21.15.59:0/514573022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.824+0000 7fecef400640 1 -- 172.21.15.59:0/514573022 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf00d0d10 msgr2=0x7fecf0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.824+0000 7fecef400640 1 --2- 172.21.15.59:0/514573022 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf00d0d10 0x7fecf0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.824+0000 7fecef400640 1 -- 172.21.15.59:0/514573022 >> v1:172.21.15.66:6789/0 conn(0x7fecf01567e0 legacy=0x7fecf00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.824+0000 7fecef400640 1 -- 172.21.15.59:0/514573022 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf015d010 con 0x7fecf015d810 2024-04-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.825+0000 7fecef400640 1 --2- 172.21.15.59:0/514573022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf0160110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fece8009e50 tx=0x7fece802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a940763ce9259c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.825+0000 7fecee000640 1 -- 172.21.15.59:0/514573022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8036070 con 0x7fecf015d810 2024-04-24T06:29:31.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.825+0000 7fecee000640 1 -- 172.21.15.59:0/514573022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8031600 con 0x7fecf015d810 2024-04-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.825+0000 7fecee000640 1 -- 172.21.15.59:0/514573022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece80319d0 con 0x7fecf015d810 2024-04-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 -- 172.21.15.59:0/514573022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 msgr2=0x7fecf0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 --2- 172.21.15.59:0/514573022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf0160110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fece8009e50 tx=0x7fece802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 -- 172.21.15.59:0/514573022 shutdown_connections 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 --2- 172.21.15.59:0/514573022 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf00d0d10 0x7fecf0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 --2- 172.21.15.59:0/514573022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf0160110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 -- 172.21.15.59:0/514573022 >> 172.21.15.59:0/514573022 conn(0x7fecf006bdf0 msgr2=0x7fecf006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 -- 172.21.15.59:0/514573022 shutdown_connections 2024-04-24T06:29:31.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.826+0000 7fecefe00640 1 -- 172.21.15.59:0/514573022 wait complete. 2024-04-24T06:29:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.828+0000 7fecefe00640 1 Processor -- start 2024-04-24T06:29:31.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.828+0000 7fecefe00640 1 -- start start 2024-04-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecf00d0d10 0x7fecf01436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf01567e0 0x7fecf006e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fecf01607d0 con 0x7fecf015d810 2024-04-24T06:29:31.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fecf0160650 con 0x7fecf00d0d10 2024-04-24T06:29:31.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecefe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fecf0160950 con 0x7fecf01567e0 2024-04-24T06:29:31.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecef400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecef400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60962/0 (socket says 172.21.15.59:60962) 2024-04-24T06:29:31.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.829+0000 7fecef400640 1 -- 172.21.15.59:0/2360289075 learned_addr learned my addr 172.21.15.59:0/2360289075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:31.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 -- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf01567e0 msgr2=0x7fecf006e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf01567e0 0x7fecf006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 -- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecf00d0d10 msgr2=0x7fecf01436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:31.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7feceea00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecf00d0d10 0x7fecf01436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecf00d0d10 0x7fecf01436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 -- 172.21.15.59:0/2360289075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf006ea00 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fecef400640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fece8034040 tx=0x7fece8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.830+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8041020 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.831+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8007780 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.831+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8036040 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.831+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf006ece0 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.831+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf006f2a0 con 0x7fecf015d810 2024-04-24T06:29:31.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.832+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf004fd90 con 0x7fecf015d810 2024-04-24T06:29:31.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.833+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fece8044020 con 0x7fecf015d810 2024-04-24T06:29:31.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.834+0000 7fece6a00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:31.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.834+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fece80aca20 con 0x7fecf015d810 2024-04-24T06:29:31.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.834+0000 7feceea00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:31.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.835+0000 7feceea00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fece0003d90 tx=0x7fece0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:31.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:31.838+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fece804b030 con 0x7fecf015d810 2024-04-24T06:29:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.042+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fecf00b5150 con 0x7fecf015d810 2024-04-24T06:29:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.045+0000 7fece6a00640 1 -- 172.21.15.59:0/2360289075 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fecf015cab0 con 0x7fecf015d810 2024-04-24T06:29:32.047 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 msgr2=0x7feccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 0x7feccc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fece0003d90 tx=0x7fece0009040 comp rx=0 tx=0).stop 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 msgr2=0x7fecf009ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fece8034040 tx=0x7fece8004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 shutdown_connections 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feccc06c8f0 0x7feccc06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf015d810 0x7fecf009ce10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fecf01567e0 0x7fecf006e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 --2- 172.21.15.59:0/2360289075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecf00d0d10 0x7fecf01436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 >> 172.21.15.59:0/2360289075 conn(0x7fecf006bdf0 msgr2=0x7fecf015e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 shutdown_connections 2024-04-24T06:29:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.048+0000 7fecefe00640 1 -- 172.21.15.59:0/2360289075 wait complete. 2024-04-24T06:29:32.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.274+0000 7fc8cf200640 1 Processor -- start 2024-04-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.274+0000 7fc8cf200640 1 -- start start 2024-04-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.276+0000 7fc8cf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8c815c670 0x7fc8c80cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.276+0000 7fc8cf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80b4cf0 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.276+0000 7fc8cf200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80ccdd0 con 0x7fc8c80b4230 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.276+0000 7fc8cf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c815ca50 con 0x7fc8c815c670 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.276+0000 7fc8cde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8c815c670 0x7fc8c80cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.277+0000 7fc8cde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8c815c670 0x7fc8c80cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60972/0 (socket says 172.21.15.59:60972) 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.277+0000 7fc8cde00640 1 -- 172.21.15.59:0/53241976 learned_addr learned my addr 172.21.15.59:0/53241976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.277+0000 7fc8cde00640 1 -- 172.21.15.59:0/53241976 >> v1:172.21.15.66:6789/0 conn(0x7fc8c80b4230 legacy=0x7fc8c80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.277+0000 7fc8cde00640 1 -- 172.21.15.59:0/53241976 >> v1:172.21.15.59:6789/0 conn(0x7fc8c80d0cd0 legacy=0x7fc8c806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.277+0000 7fc8cde00640 1 -- 172.21.15.59:0/53241976 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80cd0b0 con 0x7fc8c815c670 2024-04-24T06:29:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.278+0000 7fc8cde00640 1 --2- 172.21.15.59:0/53241976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c815c670 0x7fc8c80cd8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009870 tx=0x7fc8bc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c447c18f778c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.278+0000 7fc8cca00640 1 -- 172.21.15.59:0/53241976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc037070 con 0x7fc8c815c670 2024-04-24T06:29:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.278+0000 7fc8cca00640 1 -- 172.21.15.59:0/53241976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8bc032440 con 0x7fc8c815c670 2024-04-24T06:29:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.278+0000 7fc8cca00640 1 -- 172.21.15.59:0/53241976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc032810 con 0x7fc8c815c670 2024-04-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 -- 172.21.15.59:0/53241976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c815c670 msgr2=0x7fc8c80cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 --2- 172.21.15.59:0/53241976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c815c670 0x7fc8c80cd8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009870 tx=0x7fc8bc02a2a0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 -- 172.21.15.59:0/53241976 shutdown_connections 2024-04-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 --2- 172.21.15.59:0/53241976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c815c670 0x7fc8c80cd8b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 -- 172.21.15.59:0/53241976 >> 172.21.15.59:0/53241976 conn(0x7fc8c806bdf0 msgr2=0x7fc8c806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:32.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 -- 172.21.15.59:0/53241976 shutdown_connections 2024-04-24T06:29:32.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.279+0000 7fc8cf200640 1 -- 172.21.15.59:0/53241976 wait complete. 2024-04-24T06:29:32.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.281+0000 7fc8cf200640 1 Processor -- start 2024-04-24T06:29:32.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.281+0000 7fc8cf200640 1 -- start start 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 0x7fc8c814a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8c815c670 0x7fc8c8153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80b3e60 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80ccb80 con 0x7fc8c80b4230 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c80b3fe0 con 0x7fc8c815c670 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60986/0 (socket says 172.21.15.59:60986) 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.282+0000 7fc8cd400640 1 -- 172.21.15.59:0/1972613162 learned_addr learned my addr 172.21.15.59:0/1972613162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cde00640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 0x7fc8c814a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 -- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8c815c670 msgr2=0x7fc8c8153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8c815c670 0x7fc8c8153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 -- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 msgr2=0x7fc8c814a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 0x7fc8c814a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 -- 172.21.15.59:0/1972613162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c81f4640 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cde00640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 0x7fc8c814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cd400640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c430 tx=0x7fc8b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011790 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8017070 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.283+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c8153360 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.284+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011b70 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.284+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c8158300 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.285+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c804fd90 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.286+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b8011d10 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.288+0000 7fc8c2000640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 0x7fc8ac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.288+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc8b808bab0 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.288+0000 7fc8cde00640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 0x7fc8ac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:32.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.289+0000 7fc8cde00640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 0x7fc8ac06ee00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc034000 tx=0x7fc8bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.292+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8b8059e40 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.507+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8c80b3570 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.508+0000 7fc8c2000640 1 -- 172.21.15.59:0/1972613162 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc8b8059740 con 0x7fc8c80d0cd0 2024-04-24T06:29:32.510 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 msgr2=0x7fc8ac06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 0x7fc8ac06ee00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc034000 tx=0x7fc8bc02e000 comp rx=0 tx=0).stop 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 msgr2=0x7fc8c81528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c430 tx=0x7fc8b800c8f0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 shutdown_connections 2024-04-24T06:29:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8ac06c940 0x7fc8ac06ee00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8c815c670 0x7fc8c8153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8c80d0cd0 0x7fc8c81528d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 --2- 172.21.15.59:0/1972613162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8c80b4230 0x7fc8c814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 >> 172.21.15.59:0/1972613162 conn(0x7fc8c806bdf0 msgr2=0x7fc8c8072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 shutdown_connections 2024-04-24T06:29:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:32.511+0000 7fc8cf200640 1 -- 172.21.15.59:0/1972613162 wait complete. 2024-04-24T06:29:32.531 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:29:32.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:32.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:29:32.611 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:32.612 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:29:32.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:32.612 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:29:32.667 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:32.667 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:32.729 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:32.729 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:32.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:33.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.005+0000 7f07afe00640 1 Processor -- start 2024-04-24T06:29:33.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.006+0000 7f07afe00640 1 -- start start 2024-04-24T06:29:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.007+0000 7f07afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07b00d02e0 0x7f07b00d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.007+0000 7f07afe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00cf520 con 0x7f07b00d3260 2024-04-24T06:29:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.007+0000 7f07afe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00cf800 con 0x7f07b015e9d0 2024-04-24T06:29:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.007+0000 7f07afe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f07b00cfae0 con 0x7f07b00d02e0 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07af400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07b00d02e0 0x7f07b00d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07ae000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f07b00d3260 0x7f07b00d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54354/0 (socket says 172.21.15.59:54354) 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07ae000640 1 -- 172.21.15.59:0/2959843058 learned_addr learned my addr 172.21.15.59:0/2959843058 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07af400640 1 -- 172.21.15.59:0/2959843058 >> v1:172.21.15.66:6789/0 conn(0x7f07b015e9d0 legacy=0x7f07b00d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07af400640 1 -- 172.21.15.59:0/2959843058 >> v1:172.21.15.59:6789/0 conn(0x7f07b00d3260 legacy=0x7f07b00d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.008+0000 7f07af400640 1 -- 172.21.15.59:0/2959843058 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00cfdc0 con 0x7f07b00d02e0 2024-04-24T06:29:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.009+0000 7f07af400640 1 --2- 172.21.15.59:0/2959843058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00d02e0 0x7f07b00d1680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009d90 tx=0x7f07a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22ce8299bcbb029 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07ad600640 1 -- 172.21.15.59:0/2959843058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8007740 con 0x7f07b00d02e0 2024-04-24T06:29:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07ad600640 1 -- 172.21.15.59:0/2959843058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a80078e0 con 0x7f07b00d02e0 2024-04-24T06:29:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07ad600640 1 -- 172.21.15.59:0/2959843058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8007cb0 con 0x7f07b00d02e0 2024-04-24T06:29:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 -- 172.21.15.59:0/2959843058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00d02e0 msgr2=0x7f07b00d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 --2- 172.21.15.59:0/2959843058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00d02e0 0x7f07b00d1680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009d90 tx=0x7f07a802a6f0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 -- 172.21.15.59:0/2959843058 shutdown_connections 2024-04-24T06:29:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 --2- 172.21.15.59:0/2959843058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00d02e0 0x7f07b00d1680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 -- 172.21.15.59:0/2959843058 >> 172.21.15.59:0/2959843058 conn(0x7f07b008bf90 msgr2=0x7f07b00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 -- 172.21.15.59:0/2959843058 shutdown_connections 2024-04-24T06:29:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.010+0000 7f07afe00640 1 -- 172.21.15.59:0/2959843058 wait complete. 2024-04-24T06:29:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.012+0000 7f07afe00640 1 Processor -- start 2024-04-24T06:29:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.012+0000 7f07afe00640 1 -- start start 2024-04-24T06:29:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07b00d02e0 0x7f07b00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 0x7f07b00b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01697e0 con 0x7f07b00cdd70 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07b0169660 con 0x7f07b00d02e0 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07afe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f07b0169960 con 0x7f07b00d3260 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.013+0000 7f07af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60990/0 (socket says 172.21.15.59:60990) 2024-04-24T06:29:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 -- 172.21.15.59:0/2571794566 learned_addr learned my addr 172.21.15.59:0/2571794566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07ae000640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 0x7f07b00b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07aea00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07b00d02e0 0x7f07b00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 -- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 msgr2=0x7f07b00b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 0x7f07b00b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 -- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07b00d02e0 msgr2=0x7f07b00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07b00d02e0 0x7f07b00cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 -- 172.21.15.59:0/2571794566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00cbfc0 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07ae000640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 0x7f07b00b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.014+0000 7f07af400640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07a802ace0 tx=0x7f07a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.015+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8040070 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.015+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8031c90 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.015+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8045680 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.015+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b00cc2a0 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.015+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b00b5010 con 0x7f07b00cdd70 2024-04-24T06:29:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.016+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b004fd90 con 0x7f07b00cdd70 2024-04-24T06:29:33.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.017+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07a8039040 con 0x7f07b00cdd70 2024-04-24T06:29:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.018+0000 7f07a6a00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.018+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f07a80ad5e0 con 0x7f07b00cdd70 2024-04-24T06:29:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.018+0000 7f07aea00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.019+0000 7f07aea00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 0x7f079006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000f60 tx=0x7f07a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.022+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07a8043360 con 0x7f07b00cdd70 2024-04-24T06:29:33.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.229+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07b00cc470 con 0x7f07b00cdd70 2024-04-24T06:29:33.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.231+0000 7f07a6a00640 1 -- 172.21.15.59:0/2571794566 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f07a807ba50 con 0x7f07b00cdd70 2024-04-24T06:29:33.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:33.233 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 msgr2=0x7f079006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 0x7f079006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000f60 tx=0x7f07a0009040 comp rx=0 tx=0).stop 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 msgr2=0x7f07b015fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07a802ace0 tx=0x7f07a8004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 shutdown_connections 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079006c8f0 0x7f079006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07b00cdd70 0x7f07b015fe10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07b00d3260 0x7f07b00b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 --2- 172.21.15.59:0/2571794566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07b00d02e0 0x7f07b00cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 >> 172.21.15.59:0/2571794566 conn(0x7f07b008bf90 msgr2=0x7f07b00b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:33.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 shutdown_connections 2024-04-24T06:29:33.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.234+0000 7f07afe00640 1 -- 172.21.15.59:0/2571794566 wait complete. 2024-04-24T06:29:33.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:33.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:33.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.473+0000 7fb4cd600640 1 Processor -- start 2024-04-24T06:29:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.473+0000 7fb4cd600640 1 -- start start 2024-04-24T06:29:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4cd600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4c806dcb0 0x7fb4c80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4cd600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb4c80d09e0 0x7fb4c806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4cd600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4c8071e10 con 0x7fb4c80d0600 2024-04-24T06:29:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4cd600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c8072a20 con 0x7fb4c806dcb0 2024-04-24T06:29:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4cd600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb4c80720f0 con 0x7fb4c80d09e0 2024-04-24T06:29:33.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.475+0000 7fb4bfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4c806dcb0 0x7fb4c80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4bfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4c806dcb0 0x7fb4c80ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32768/0 (socket says 172.21.15.59:32768) 2024-04-24T06:29:33.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4bfe00640 1 -- 172.21.15.59:0/3184155620 learned_addr learned my addr 172.21.15.59:0/3184155620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:33.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4ccc00640 1 --2- 172.21.15.59:0/3184155620 >> v2:172.21.15.66:3300/0 conn(0x7fb4c80d09e0 0x7fb4c806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4bfe00640 1 -- 172.21.15.59:0/3184155620 >> v2:172.21.15.66:3300/0 conn(0x7fb4c80d09e0 msgr2=0x7fb4c806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:33.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4bfe00640 1 --2- 172.21.15.59:0/3184155620 >> v2:172.21.15.66:3300/0 conn(0x7fb4c80d09e0 0x7fb4c806e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.476+0000 7fb4bfe00640 1 -- 172.21.15.59:0/3184155620 >> v1:172.21.15.66:6789/0 conn(0x7fb4c80d0600 legacy=0x7fb4c8162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.477+0000 7fb4bfe00640 1 -- 172.21.15.59:0/3184155620 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c80723d0 con 0x7fb4c806dcb0 2024-04-24T06:29:33.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.478+0000 7fb4bfe00640 1 --2- 172.21.15.59:0/3184155620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c806dcb0 0x7fb4c80ad170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80b4230 tx=0x7fb4b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1049e59e85dccdad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.478+0000 7fb4c7400640 1 -- 172.21.15.59:0/3184155620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0037070 con 0x7fb4c806dcb0 2024-04-24T06:29:33.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.478+0000 7fb4c7400640 1 -- 172.21.15.59:0/3184155620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0032440 con 0x7fb4c806dcb0 2024-04-24T06:29:33.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.478+0000 7fb4c7400640 1 -- 172.21.15.59:0/3184155620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0032810 con 0x7fb4c806dcb0 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 -- 172.21.15.59:0/3184155620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c806dcb0 msgr2=0x7fb4c80ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 --2- 172.21.15.59:0/3184155620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c806dcb0 0x7fb4c80ad170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80b4230 tx=0x7fb4b002a270 comp rx=0 tx=0).stop 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 -- 172.21.15.59:0/3184155620 shutdown_connections 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 --2- 172.21.15.59:0/3184155620 >> v2:172.21.15.66:3300/0 conn(0x7fb4c80d09e0 0x7fb4c806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 --2- 172.21.15.59:0/3184155620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c806dcb0 0x7fb4c80ad170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 -- 172.21.15.59:0/3184155620 >> 172.21.15.59:0/3184155620 conn(0x7fb4c806bdf0 msgr2=0x7fb4c806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 -- 172.21.15.59:0/3184155620 shutdown_connections 2024-04-24T06:29:33.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.479+0000 7fb4cd600640 1 -- 172.21.15.59:0/3184155620 wait complete. 2024-04-24T06:29:33.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.481+0000 7fb4cd600640 1 Processor -- start 2024-04-24T06:29:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.481+0000 7fb4cd600640 1 -- start start 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4c806dcb0 0x7fb4c8151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c80d0600 0x7fb4c80cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c80d0e90 con 0x7fb4c80d09e0 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c815eaf0 con 0x7fb4c80d0600 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4cd600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c806eb00 con 0x7fb4c806dcb0 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4ccc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4ccc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32780/0 (socket says 172.21.15.59:32780) 2024-04-24T06:29:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4ccc00640 1 -- 172.21.15.59:0/267110909 learned_addr learned my addr 172.21.15.59:0/267110909 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4c7e00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4c806dcb0 0x7fb4c8151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4bfe00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c80d0600 0x7fb4c80cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4ccc00640 1 -- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4c806dcb0 msgr2=0x7fb4c8151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.482+0000 7fb4ccc00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4c806dcb0 0x7fb4c8151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4ccc00640 1 -- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c80d0600 msgr2=0x7fb4c80cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4ccc00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c80d0600 0x7fb4c80cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4ccc00640 1 -- 172.21.15.59:0/267110909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c8156cb0 con 0x7fb4c80d09e0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4ccc00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000efc0 tx=0x7fb4c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0018020 con 0x7fb4c80d09e0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c0014480 con 0x7fb4c80d09e0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0014870 con 0x7fb4c80d09e0 2024-04-24T06:29:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c8156f50 con 0x7fb4c80d09e0 2024-04-24T06:29:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.483+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c8157510 con 0x7fb4c80d09e0 2024-04-24T06:29:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.485+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c80705e0 con 0x7fb4c80d09e0 2024-04-24T06:29:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.486+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4c0005d10 con 0x7fb4c80d09e0 2024-04-24T06:29:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.486+0000 7fb4c4c00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 0x7fb4a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.487+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb4c008c890 con 0x7fb4c80d09e0 2024-04-24T06:29:33.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.487+0000 7fb4c7e00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 0x7fb4a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.487+0000 7fb4c7e00640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 0x7fb4a406ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80015a0 tx=0x7fb4b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.491+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4c80705e0 con 0x7fb4c80d09e0 2024-04-24T06:29:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.696+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4c804fd90 con 0x7fb4c80d09e0 2024-04-24T06:29:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.697+0000 7fb4c4c00640 1 -- 172.21.15.59:0/267110909 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fb4c0022030 con 0x7fb4c80d09e0 2024-04-24T06:29:33.699 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 msgr2=0x7fb4a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 0x7fb4a406ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80015a0 tx=0x7fb4b8009040 comp rx=0 tx=0).stop 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 msgr2=0x7fb4c80cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000efc0 tx=0x7fb4c000c490 comp rx=0 tx=0).stop 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 shutdown_connections 2024-04-24T06:29:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4a406c920 0x7fb4a406ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c80d09e0 0x7fb4c80cb5f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c80d0600 0x7fb4c80cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 --2- 172.21.15.59:0/267110909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4c806dcb0 0x7fb4c8151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 >> 172.21.15.59:0/267110909 conn(0x7fb4c806bdf0 msgr2=0x7fb4c80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 shutdown_connections 2024-04-24T06:29:33.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.700+0000 7fb4cd600640 1 -- 172.21.15.59:0/267110909 wait complete. 2024-04-24T06:29:33.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:33.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.934+0000 7f4c79c00640 1 Processor -- start 2024-04-24T06:29:33.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.935+0000 7f4c79c00640 1 -- start start 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.936+0000 7f4c79c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c740d15b0 0x7f4c740cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.936+0000 7f4c79c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c740b5da0 con 0x7f4c7415ed20 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c79c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c740d0a20 con 0x7f4c740b4f20 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c79c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4c7415f100 con 0x7f4c740d15b0 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4c7415ed20 0x7f4c740d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54360/0 (socket says 172.21.15.59:54360) 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73e00640 1 -- 172.21.15.59:0/1053884441 learned_addr learned my addr 172.21.15.59:0/1053884441 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:33.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73400640 1 --2- 172.21.15.59:0/1053884441 >> v2:172.21.15.59:3300/0 conn(0x7f4c740d15b0 0x7f4c740cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c72a00640 1 -- 172.21.15.59:0/1053884441 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993866521 0 0) 0x7f4c740b5da0 con 0x7f4c7415ed20 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c72a00640 1 -- 172.21.15.59:0/1053884441 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c58003520 con 0x7f4c7415ed20 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73400640 1 -- 172.21.15.59:0/1053884441 >> v1:172.21.15.66:6789/0 conn(0x7f4c740b4f20 legacy=0x7f4c740cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73400640 1 -- 172.21.15.59:0/1053884441 >> v1:172.21.15.59:6789/0 conn(0x7f4c7415ed20 legacy=0x7f4c740d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.937+0000 7f4c73400640 1 -- 172.21.15.59:0/1053884441 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c740d2ed0 con 0x7f4c740d15b0 2024-04-24T06:29:33.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.939+0000 7f4c73400640 1 --2- 172.21.15.59:0/1053884441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740cf280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c6002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b23bddce28a49a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.939+0000 7f4c72a00640 1 -- 172.21.15.59:0/1053884441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60031600 con 0x7f4c740d15b0 2024-04-24T06:29:33.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.939+0000 7f4c72a00640 1 -- 172.21.15.59:0/1053884441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c600317a0 con 0x7f4c740d15b0 2024-04-24T06:29:33.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.939+0000 7f4c72a00640 1 -- 172.21.15.59:0/1053884441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60031b50 con 0x7f4c740d15b0 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 -- 172.21.15.59:0/1053884441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 msgr2=0x7f4c740cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 --2- 172.21.15.59:0/1053884441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740cf280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c6002a5d0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 -- 172.21.15.59:0/1053884441 shutdown_connections 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 --2- 172.21.15.59:0/1053884441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740cf280 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 -- 172.21.15.59:0/1053884441 >> 172.21.15.59:0/1053884441 conn(0x7f4c7408bf90 msgr2=0x7f4c740d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 -- 172.21.15.59:0/1053884441 shutdown_connections 2024-04-24T06:29:33.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.940+0000 7f4c79c00640 1 -- 172.21.15.59:0/1053884441 wait complete. 2024-04-24T06:29:33.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.942+0000 7f4c79c00640 1 Processor -- start 2024-04-24T06:29:33.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.942+0000 7f4c79c00640 1 -- start start 2024-04-24T06:29:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4c740b4f20 0x7f4c74155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c7415ed20 0x7f4c7415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c740b3600 con 0x7f4c740d15b0 2024-04-24T06:29:33.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c740d3490 con 0x7f4c7415ed20 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4c740b3780 con 0x7f4c740b4f20 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c73400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c73400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32796/0 (socket says 172.21.15.59:32796) 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c73400640 1 -- 172.21.15.59:0/3812359641 learned_addr learned my addr 172.21.15.59:0/3812359641 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.943+0000 7f4c79200640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c7415ed20 0x7f4c7415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73e00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4c740b4f20 0x7f4c74155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 -- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4c740b4f20 msgr2=0x7f4c74155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4c740b4f20 0x7f4c74155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 -- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c7415ed20 msgr2=0x7f4c7415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c7415ed20 0x7f4c7415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 -- 172.21.15.59:0/3812359641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c740c9390 con 0x7f4c740d15b0 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c73400640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.944+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60040070 con 0x7f4c740d15b0 2024-04-24T06:29:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.945+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c60030be0 con 0x7f4c740d15b0 2024-04-24T06:29:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.945+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60009450 con 0x7f4c740d15b0 2024-04-24T06:29:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.945+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c740c9670 con 0x7f4c740d15b0 2024-04-24T06:29:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.945+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c740c9c30 con 0x7f4c740d15b0 2024-04-24T06:29:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.946+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7404fd90 con 0x7f4c740d15b0 2024-04-24T06:29:33.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.947+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c60039040 con 0x7f4c740d15b0 2024-04-24T06:29:33.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.948+0000 7f4c6be00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 0x7f4c5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:33.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.948+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4c600ac850 con 0x7f4c740d15b0 2024-04-24T06:29:33.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.948+0000 7f4c73e00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 0x7f4c5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:33.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.949+0000 7f4c73e00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 0x7f4c5406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c64007fd0 tx=0x7f4c64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:33.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:33.952+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c6007ab60 con 0x7f4c740d15b0 2024-04-24T06:29:34.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.158+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c7415e330 con 0x7f4c740d15b0 2024-04-24T06:29:34.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.159+0000 7f4c6be00640 1 -- 172.21.15.59:0/3812359641 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f4c6007a460 con 0x7f4c740d15b0 2024-04-24T06:29:34.161 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:34.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 msgr2=0x7f4c5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 0x7f4c5406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4c64007fd0 tx=0x7f4c64009040 comp rx=0 tx=0).stop 2024-04-24T06:29:34.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 msgr2=0x7f4c740ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c60004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 shutdown_connections 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4c5406c920 0x7f4c5406ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c7415ed20 0x7f4c7415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c740d15b0 0x7f4c740ca430 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 --2- 172.21.15.59:0/3812359641 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4c740b4f20 0x7f4c74155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 >> 172.21.15.59:0/3812359641 conn(0x7f4c7408bf90 msgr2=0x7f4c740d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 shutdown_connections 2024-04-24T06:29:34.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.162+0000 7f4c79c00640 1 -- 172.21.15.59:0/3812359641 wait complete. 2024-04-24T06:29:34.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:34.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.402+0000 7fec0d200640 1 Processor -- start 2024-04-24T06:29:34.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.402+0000 7fec0d200640 1 -- start start 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.403+0000 7fec0d200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7fec08156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.403+0000 7fec0d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0815cab0 con 0x7fec0815d810 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7fec0d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec08160e40 con 0x7fec081567e0 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7fec0d200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fec0815cd30 con 0x7fec080d0d10 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7fec07e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fec0815d810 0x7fec08160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54370/0 (socket says 172.21.15.59:54370) 2024-04-24T06:29:34.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7fec07e00640 1 -- 172.21.15.59:0/1323938316 learned_addr learned my addr 172.21.15.59:0/1323938316 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7febffe00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7fec08156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.404+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983224078 0 0) 0x7fec00001490 con 0x7fec0815d810 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febec003520 con 0x7fec0815d810 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156116067 0 0) 0x7febec003520 con 0x7fec0815d810 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 msgr2=0x7fec08156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7fec08156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 >> v1:172.21.15.66:6789/0 conn(0x7fec081567e0 legacy=0x7fec080d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0815d010 con 0x7fec0815d810 2024-04-24T06:29:34.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fec00001bc0 con 0x7fec0815d810 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 >> v1:172.21.15.59:6789/0 conn(0x7fec0815d810 legacy=0x7fec08160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7febec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7febec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febec004d40 0x7febec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec0815cd30 con 0x7febec0039e0 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febec00baa0 con 0x7febec004d40 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.405+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7febec00bca0 con 0x7fec080d0d10 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7febec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07e00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febec004d40 0x7febec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7febffe00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7febec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 -- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 msgr2=0x7febec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7febec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 -- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febec004d40 msgr2=0x7febec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febec004d40 0x7febec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7fec07400640 1 -- 172.21.15.59:0/1323938316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec003520 con 0x7febec0039e0 2024-04-24T06:29:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.406+0000 7febffe00640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec080d0d10 0x7febec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.407+0000 7fec07400640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7febec003e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febf8001720 tx=0x7febec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7111534e0ca4bac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.408+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8001370 con 0x7febec0039e0 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.408+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf80052d0 con 0x7febec0039e0 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.408+0000 7fec06a00640 1 -- 172.21.15.59:0/1323938316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf802abc0 con 0x7febec0039e0 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.408+0000 7fec0d200640 1 -- 172.21.15.59:0/1323938316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 msgr2=0x7febec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.408+0000 7fec0d200640 1 --2- 172.21.15.59:0/1323938316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7febec003e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febf8001720 tx=0x7febec005830 comp rx=0 tx=0).stop 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.409+0000 7fec07e00640 1 -- 172.21.15.59:0/1323938316 reap_dead start 2024-04-24T06:29:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.409+0000 7fec0d200640 1 -- 172.21.15.59:0/1323938316 shutdown_connections 2024-04-24T06:29:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.409+0000 7fec0d200640 1 -- 172.21.15.59:0/1323938316 >> 172.21.15.59:0/1323938316 conn(0x7fec0806bdf0 msgr2=0x7fec0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.409+0000 7fec0d200640 1 -- 172.21.15.59:0/1323938316 shutdown_connections 2024-04-24T06:29:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.409+0000 7fec0d200640 1 -- 172.21.15.59:0/1323938316 wait complete. 2024-04-24T06:29:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.410+0000 7fec0d200640 1 Processor -- start 2024-04-24T06:29:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.411+0000 7fec0d200640 1 -- start start 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec0806f890 0x7fec08153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec08157b60 0x7fec08072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec08156d40 con 0x7febec0039e0 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec08156bc0 con 0x7fec08157b60 2024-04-24T06:29:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec0d200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fec08156ec0 con 0x7fec0806f890 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32816/0 (socket says 172.21.15.59:32816) 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 -- 172.21.15.59:0/556198068 learned_addr learned my addr 172.21.15.59:0/556198068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 -- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec0806f890 msgr2=0x7fec08153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec0806f890 0x7fec08153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 -- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec08157b60 msgr2=0x7fec08072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec08157b60 0x7fec08072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.412+0000 7fec07400640 1 -- 172.21.15.59:0/556198068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec08158320 con 0x7febec0039e0 2024-04-24T06:29:34.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7fec07400640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febf8008fd0 tx=0x7febf8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf803f020 con 0x7febec0039e0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf802be50 con 0x7febec0039e0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0806dd50 con 0x7febec0039e0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8007a30 con 0x7febec0039e0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.413+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0806e310 con 0x7febec0039e0 2024-04-24T06:29:34.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.414+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0806fcd0 con 0x7febec0039e0 2024-04-24T06:29:34.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.415+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febf8042020 con 0x7febec0039e0 2024-04-24T06:29:34.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.416+0000 7febff400640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.416+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7febf80ab770 con 0x7febec0039e0 2024-04-24T06:29:34.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.416+0000 7febffe00640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.417+0000 7febffe00640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 0x7febe406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec08073730 tx=0x7febf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.420+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febf8049030 con 0x7febec0039e0 2024-04-24T06:29:34.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.623+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec0804fd90 con 0x7febec0039e0 2024-04-24T06:29:34.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.624+0000 7febff400640 1 -- 172.21.15.59:0/556198068 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7febf8079b10 con 0x7febec0039e0 2024-04-24T06:29:34.626 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.626+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 msgr2=0x7febe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.626+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 0x7febe406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec08073730 tx=0x7febf0008040 comp rx=0 tx=0).stop 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 msgr2=0x7fec0806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febf8008fd0 tx=0x7febf8001120 comp rx=0 tx=0).stop 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 shutdown_connections 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7febe406c8f0 0x7febe406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec08157b60 0x7fec08072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec0806f890 0x7fec08153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 --2- 172.21.15.59:0/556198068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febec0039e0 0x7fec0806eb10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 >> 172.21.15.59:0/556198068 conn(0x7fec0806bdf0 msgr2=0x7fec080b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 shutdown_connections 2024-04-24T06:29:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.627+0000 7fec0d200640 1 -- 172.21.15.59:0/556198068 wait complete. 2024-04-24T06:29:34.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.859+0000 7f85f1800640 1 Processor -- start 2024-04-24T06:29:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.859+0000 7f85f1800640 1 -- start start 2024-04-24T06:29:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f1800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f85ec0cd8b0 0x7f85ec0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec0d0b00 0x7f85ec06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f1800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ec0b4cf0 con 0x7f85ec15c670 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f1800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec15ca50 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0ccb80 con 0x7f85ec0d0b00 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85eb400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f85ec0cd8b0 0x7f85ec0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85f0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec0d0b00 0x7f85ec06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85ebe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f85ec15c670 0x7f85ec0b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:35830/0 (socket says 172.21.15.59:35830) 2024-04-24T06:29:34.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.861+0000 7f85ebe00640 1 -- 172.21.15.59:0/4292089215 learned_addr learned my addr 172.21.15.59:0/4292089215 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:34.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.862+0000 7f85eb400640 1 -- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec0d0b00 msgr2=0x7f85ec06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:34.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.862+0000 7f85eb400640 1 --2- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec0d0b00 0x7f85ec06ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.862+0000 7f85eb400640 1 -- 172.21.15.59:0/4292089215 >> v1:172.21.15.66:6789/0 conn(0x7f85ec15c670 legacy=0x7f85ec0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.862+0000 7f85eb400640 1 -- 172.21.15.59:0/4292089215 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0ccdd0 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.864+0000 7f85eb400640 1 --2- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0d05c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85d80097a0 tx=0x7f85d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d269b212e959db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.864+0000 7f85eaa00640 1 -- 172.21.15.59:0/4292089215 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8037070 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.864+0000 7f85eaa00640 1 -- 172.21.15.59:0/4292089215 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d8032440 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.864+0000 7f85eaa00640 1 -- 172.21.15.59:0/4292089215 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8032810 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 -- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 msgr2=0x7f85ec0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 --2- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0d05c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85d80097a0 tx=0x7f85d802a180 comp rx=0 tx=0).stop 2024-04-24T06:29:34.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 -- 172.21.15.59:0/4292089215 shutdown_connections 2024-04-24T06:29:34.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 --2- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec0d0b00 0x7f85ec06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 --2- 172.21.15.59:0/4292089215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0d05c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 -- 172.21.15.59:0/4292089215 >> 172.21.15.59:0/4292089215 conn(0x7f85ec06bdf0 msgr2=0x7f85ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 -- 172.21.15.59:0/4292089215 shutdown_connections 2024-04-24T06:29:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.865+0000 7f85f1800640 1 -- 172.21.15.59:0/4292089215 wait complete. 2024-04-24T06:29:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.867+0000 7f85f1800640 1 Processor -- start 2024-04-24T06:29:34.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.867+0000 7f85f1800640 1 -- start start 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec15c670 0x7f85ec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0b3e60 con 0x7f85ec0d0b00 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0cd390 con 0x7f85ec0cd8b0 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85f1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0b3fe0 con 0x7f85ec15c670 2024-04-24T06:29:34.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85eb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85eb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32818/0 (socket says 172.21.15.59:32818) 2024-04-24T06:29:34.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.868+0000 7f85eb400640 1 -- 172.21.15.59:0/156219480 learned_addr learned my addr 172.21.15.59:0/156219480 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85ebe00640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 -- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec15c670 msgr2=0x7f85ec0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec15c670 0x7f85ec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 -- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 msgr2=0x7f85ec0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 -- 172.21.15.59:0/156219480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec1582d0 con 0x7f85ec0d0b00 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85ebe00640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85eb400640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85d802a770 tx=0x7f85d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.869+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8037050 con 0x7f85ec0d0b00 2024-04-24T06:29:34.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.870+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d8043070 con 0x7f85ec0d0b00 2024-04-24T06:29:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.870+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d803cb30 con 0x7f85ec0d0b00 2024-04-24T06:29:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.870+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec1585b0 con 0x7f85ec0d0b00 2024-04-24T06:29:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.870+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec158b70 con 0x7f85ec0d0b00 2024-04-24T06:29:34.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.871+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec04fd90 con 0x7f85ec0d0b00 2024-04-24T06:29:34.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.872+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85d803ccd0 con 0x7f85ec0d0b00 2024-04-24T06:29:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.873+0000 7f85e3e00640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.873+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f85d80ac9f0 con 0x7f85ec0d0b00 2024-04-24T06:29:34.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.873+0000 7f85ebe00640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:34.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.874+0000 7f85ebe00640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 0x7f85cc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000f60 tx=0x7f85dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:34.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:34.878+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85d807ae20 con 0x7f85ec0d0b00 2024-04-24T06:29:35.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.095+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85ec1551d0 con 0x7f85ec0d0b00 2024-04-24T06:29:35.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.096+0000 7f85e3e00640 1 -- 172.21.15.59:0/156219480 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f85d807a720 con 0x7f85ec0d0b00 2024-04-24T06:29:35.098 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 msgr2=0x7f85cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 0x7f85cc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000f60 tx=0x7f85dc009040 comp rx=0 tx=0).stop 2024-04-24T06:29:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 msgr2=0x7f85ec0b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85d802a770 tx=0x7f85d80046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 shutdown_connections 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85ec15c670 0x7f85ec0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ec0d0b00 0x7f85ec0b0ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 --2- 172.21.15.59:0/156219480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec0cd8b0 0x7f85ec0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 >> 172.21.15.59:0/156219480 conn(0x7f85ec06bdf0 msgr2=0x7f85ec0b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 shutdown_connections 2024-04-24T06:29:35.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.099+0000 7f85f1800640 1 -- 172.21.15.59:0/156219480 wait complete. 2024-04-24T06:29:35.122 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:35.122 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:35.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:29:35.145 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:29:35.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.387+0000 7f0c85400640 1 Processor -- start 2024-04-24T06:29:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.387+0000 7f0c85400640 1 -- start start 2024-04-24T06:29:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c85400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c8006f3f0 0x7f0c800cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c85400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c800b3080 0x7f0c800cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c85400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c800b3870 con 0x7f0c8015b450 2024-04-24T06:29:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c85400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0c800b41a0 con 0x7f0c8006f3f0 2024-04-24T06:29:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c85400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800b3b50 con 0x7f0c800b3080 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c77e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c8006f3f0 0x7f0c800cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.389+0000 7f0c7fe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f0c8015b450 0x7f0c800b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:35840/0 (socket says 172.21.15.59:35840) 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c7fe00640 1 -- 172.21.15.59:0/1097748008 learned_addr learned my addr 172.21.15.59:0/1097748008 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c77e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c8006f3f0 0x7f0c800cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:36234/0 (socket says 172.21.15.59:36234) 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c84a00640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c800b3080 0x7f0c800cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c77e00640 1 -- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c800b3080 msgr2=0x7f0c800cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c77e00640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c800b3080 0x7f0c800cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c77e00640 1 -- 172.21.15.59:0/1097748008 >> v1:172.21.15.66:6789/0 conn(0x7f0c8015b450 legacy=0x7f0c800b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.390+0000 7f0c77e00640 1 -- 172.21.15.59:0/1097748008 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c800b3e30 con 0x7f0c8006f3f0 2024-04-24T06:29:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.392+0000 7f0c77e00640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c800cd050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c680097a0 tx=0x7f0c6802c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886cd1419e8b58ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.392+0000 7f0c7f400640 1 -- 172.21.15.59:0/1097748008 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68038070 con 0x7f0c8006f3f0 2024-04-24T06:29:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.392+0000 7f0c7f400640 1 -- 172.21.15.59:0/1097748008 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c68031710 con 0x7f0c8006f3f0 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.392+0000 7f0c7f400640 1 -- 172.21.15.59:0/1097748008 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68031ae0 con 0x7f0c8006f3f0 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 -- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 msgr2=0x7f0c800cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c800cd050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c680097a0 tx=0x7f0c6802c420 comp rx=0 tx=0).stop 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 -- 172.21.15.59:0/1097748008 shutdown_connections 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c800b3080 0x7f0c800cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 --2- 172.21.15.59:0/1097748008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c800cd050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 -- 172.21.15.59:0/1097748008 >> 172.21.15.59:0/1097748008 conn(0x7f0c800b21a0 msgr2=0x7f0c800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 -- 172.21.15.59:0/1097748008 shutdown_connections 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.393+0000 7f0c85400640 1 -- 172.21.15.59:0/1097748008 wait complete. 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.396+0000 7f0c85400640 1 Processor -- start 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.396+0000 7f0c85400640 1 -- start start 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c8009dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 0x7f0c80158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800ad060 con 0x7f0c800b3080 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c800b4ff0 con 0x7f0c8006f3f0 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c85400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800ad1e0 con 0x7f0c8015b450 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32828/0 (socket says 172.21.15.59:32828) 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.397+0000 7f0c77e00640 1 -- 172.21.15.59:0/1493191341 learned_addr learned my addr 172.21.15.59:0/1493191341 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c7fe00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c8009dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c84a00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 0x7f0c80158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 -- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 msgr2=0x7f0c80158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 0x7f0c80158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 -- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 msgr2=0x7f0c8009dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c8009dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 -- 172.21.15.59:0/1493191341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c801f7d80 con 0x7f0c800b3080 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c84a00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 0x7f0c80158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c77e00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098c0 tx=0x7f0c680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.398+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68042020 con 0x7f0c800b3080 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.399+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c68033830 con 0x7f0c800b3080 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.399+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68038040 con 0x7f0c800b3080 2024-04-24T06:29:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.399+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c801f8060 con 0x7f0c800b3080 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.399+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801f8620 con 0x7f0c800b3080 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.400+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8004fd90 con 0x7f0c800b3080 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.401+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c68033c20 con 0x7f0c800b3080 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.402+0000 7f0c7cc00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 0x7f0c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.402+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0c680ac450 con 0x7f0c800b3080 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.402+0000 7f0c7fe00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 0x7f0c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.403+0000 7f0c7fe00640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 0x7f0c5c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f60 tx=0x7f0c70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.406+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c6804c020 con 0x7f0c800b3080 2024-04-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.614+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c801f87f0 con 0x7f0c800b3080 2024-04-24T06:29:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.617+0000 7f0c7cc00640 1 -- 172.21.15.59:0/1493191341 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f0c6807a7f0 con 0x7f0c800b3080 2024-04-24T06:29:35.619 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:35.619 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 msgr2=0x7f0c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 0x7f0c5c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f60 tx=0x7f0c70009040 comp rx=0 tx=0).stop 2024-04-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 msgr2=0x7f0c80158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098c0 tx=0x7f0c680046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 shutdown_connections 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c5c06c8f0 0x7f0c5c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c8015b450 0x7f0c80158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c800b3080 0x7f0c80158730 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 --2- 172.21.15.59:0/1493191341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c8006f3f0 0x7f0c8009dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 >> 172.21.15.59:0/1493191341 conn(0x7f0c800b21a0 msgr2=0x7f0c800cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 shutdown_connections 2024-04-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.620+0000 7f0c85400640 1 -- 172.21.15.59:0/1493191341 wait complete. 2024-04-24T06:29:35.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:35.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.847+0000 7f70b2a00640 1 Processor -- start 2024-04-24T06:29:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.847+0000 7f70b2a00640 1 -- start start 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.848+0000 7f70b2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70ac161c50 0x7f70ac0d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b2a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70ac0d22e0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b2a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70ac0d2550 con 0x7f70ac0cfc50 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b2a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f70ac0d2ba0 con 0x7f70ac161c50 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f70ac0d0de0 0x7f70ac06cb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54384/0 (socket says 172.21.15.59:54384) 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b2000640 1 -- 172.21.15.59:0/1148692189 learned_addr learned my addr 172.21.15.59:0/1148692189 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:35.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.849+0000 7f70b1600640 1 --2- 172.21.15.59:0/1148692189 >> v2:172.21.15.59:3300/0 conn(0x7f70ac161c50 0x7f70ac0d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.850+0000 7f70abe00640 1 -- 172.21.15.59:0/1148692189 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1387269828 0 0) 0x7f70ac0d22e0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.850+0000 7f70abe00640 1 -- 172.21.15.59:0/1148692189 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7094003520 con 0x7f70ac0d0de0 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.850+0000 7f70b1600640 1 -- 172.21.15.59:0/1148692189 >> v1:172.21.15.66:6789/0 conn(0x7f70ac0cfc50 legacy=0x7f70ac0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.850+0000 7f70b1600640 1 -- 172.21.15.59:0/1148692189 >> v1:172.21.15.59:6789/0 conn(0x7f70ac0d0de0 legacy=0x7f70ac06cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.850+0000 7f70b1600640 1 -- 172.21.15.59:0/1148692189 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac0d2830 con 0x7f70ac161c50 2024-04-24T06:29:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.851+0000 7f70b1600640 1 --2- 172.21.15.59:0/1148692189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac161c50 0x7f70ac0d5240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f70a00097a0 tx=0x7f70a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32386df86496aa25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.851+0000 7f70abe00640 1 -- 172.21.15.59:0/1148692189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0037070 con 0x7f70ac161c50 2024-04-24T06:29:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.851+0000 7f70abe00640 1 -- 172.21.15.59:0/1148692189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a0032440 con 0x7f70ac161c50 2024-04-24T06:29:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.851+0000 7f70abe00640 1 -- 172.21.15.59:0/1148692189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0032810 con 0x7f70ac161c50 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 -- 172.21.15.59:0/1148692189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac161c50 msgr2=0x7f70ac0d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 --2- 172.21.15.59:0/1148692189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac161c50 0x7f70ac0d5240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f70a00097a0 tx=0x7f70a002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 -- 172.21.15.59:0/1148692189 shutdown_connections 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 --2- 172.21.15.59:0/1148692189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac161c50 0x7f70ac0d5240 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 -- 172.21.15.59:0/1148692189 >> 172.21.15.59:0/1148692189 conn(0x7f70ac08bf90 msgr2=0x7f70ac0d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 -- 172.21.15.59:0/1148692189 shutdown_connections 2024-04-24T06:29:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.852+0000 7f70b2a00640 1 -- 172.21.15.59:0/1148692189 wait complete. 2024-04-24T06:29:35.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.854+0000 7f70b2a00640 1 Processor -- start 2024-04-24T06:29:35.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.854+0000 7f70b2a00640 1 -- start start 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70ac0cfc50 0x7f70ac0b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70ac161c50 0x7f70ac0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac0d0090 con 0x7f70ac0d0de0 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac0d0460 con 0x7f70ac161c50 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b2a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac0d0210 con 0x7f70ac0cfc50 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32848/0 (socket says 172.21.15.59:32848) 2024-04-24T06:29:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.855+0000 7f70b0c00640 1 -- 172.21.15.59:0/82959603 learned_addr learned my addr 172.21.15.59:0/82959603 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 -- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70ac0cfc50 msgr2=0x7f70ac0b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70ac0cfc50 0x7f70ac0b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 -- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70ac161c50 msgr2=0x7f70ac0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70ac161c50 0x7f70ac0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 -- 172.21.15.59:0/82959603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac0b90e0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b0c00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f709800c460 tx=0x7f709800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098011790 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7098017070 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098011bb0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac0b93c0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.856+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac0b9ea0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.858+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac04fd90 con 0x7f70ac0d0de0 2024-04-24T06:29:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.859+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f709801c460 con 0x7f70ac0d0de0 2024-04-24T06:29:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.859+0000 7f70a9600640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 0x7f708c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.860+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f709808bad0 con 0x7f70ac0d0de0 2024-04-24T06:29:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.860+0000 7f70b1600640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 0x7f708c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.861+0000 7f70b1600640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 0x7f708c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70a0034000 tx=0x7f70a002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:35.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:35.864+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7098059de0 con 0x7f70ac0d0de0 2024-04-24T06:29:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.070+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70ac0d1cf0 con 0x7f70ac0d0de0 2024-04-24T06:29:36.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.073+0000 7f70a9600640 1 -- 172.21.15.59:0/82959603 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f70980596e0 con 0x7f70ac0d0de0 2024-04-24T06:29:36.075 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 msgr2=0x7f708c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 0x7f708c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70a0034000 tx=0x7f70a002c000 comp rx=0 tx=0).stop 2024-04-24T06:29:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 msgr2=0x7f70ac0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f709800c460 tx=0x7f709800c920 comp rx=0 tx=0).stop 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 shutdown_connections 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f708c06c8f0 0x7f708c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70ac161c50 0x7f70ac0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70ac0d0de0 0x7f70ac0b40d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 --2- 172.21.15.59:0/82959603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70ac0cfc50 0x7f70ac0b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 >> 172.21.15.59:0/82959603 conn(0x7f70ac08bf90 msgr2=0x7f70ac0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 shutdown_connections 2024-04-24T06:29:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.077+0000 7f70b2a00640 1 -- 172.21.15.59:0/82959603 wait complete. 2024-04-24T06:29:36.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:36.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.281+0000 7fe1d5a00640 1 Processor -- start 2024-04-24T06:29:36.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.281+0000 7fe1d5a00640 1 -- start start 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1d006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1d5a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe1d00d0350 0x7fe1d00d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1d5a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d015bf60 con 0x7fe1d00b56f0 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1d5a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d015c240 con 0x7fe1d00d0350 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d015c520 con 0x7fe1d006fa50 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1cea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe1d00b56f0 0x7fe1d006f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54394/0 (socket says 172.21.15.59:54394) 2024-04-24T06:29:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1cea00640 1 -- 172.21.15.59:0/477593686 learned_addr learned my addr 172.21.15.59:0/477593686 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:36.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1cf400640 1 --2- 172.21.15.59:0/477593686 >> v2:172.21.15.66:3300/0 conn(0x7fe1d00d0350 0x7fe1d00d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.283+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1d006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182921193 0 0) 0x7fe1d015bf60 con 0x7fe1d00b56f0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b4003520 con 0x7fe1d00b56f0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4147434575 0 0) 0x7fe1b4003520 con 0x7fe1d00b56f0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 msgr2=0x7fe1d006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1d006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 >> v2:172.21.15.66:3300/0 conn(0x7fe1d00d0350 msgr2=0x7fe1d00d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 --2- 172.21.15.59:0/477593686 >> v2:172.21.15.66:3300/0 conn(0x7fe1d00d0350 0x7fe1d00d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d015c800 con 0x7fe1d00b56f0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe1c0001670 con 0x7fe1d00b56f0 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 >> v1:172.21.15.59:6789/0 conn(0x7fe1d00b56f0 legacy=0x7fe1d006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1b4003f30 0x7fe1b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d015c240 con 0x7fe1b4004d40 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d015c520 con 0x7fe1b4003f30 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.284+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b400b9c0 con 0x7fe1d006fa50 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cea00640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1b4003f30 0x7fe1b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 -- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 msgr2=0x7fe1b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1d006fa50 0x7fe1b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 -- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1b4003f30 msgr2=0x7fe1b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1b4003f30 0x7fe1b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1cf400640 1 -- 172.21.15.59:0/477593686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b4003520 con 0x7fe1b4004d40 2024-04-24T06:29:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.285+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1b4003f30 0x7fe1b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:36.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.287+0000 7fe1cf400640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009fc0 tx=0x7fe1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771786b4f6bd51b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.287+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8009c10 con 0x7fe1b4004d40 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.287+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c8036070 con 0x7fe1b4004d40 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.287+0000 7fe1ce000640 1 -- 172.21.15.59:0/477593686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c802d070 con 0x7fe1b4004d40 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 -- 172.21.15.59:0/477593686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b4004d40 msgr2=0x7fe1b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/477593686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009fc0 tx=0x7fe1b4005830 comp rx=0 tx=0).stop 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1cf400640 1 -- 172.21.15.59:0/477593686 reap_dead start 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 -- 172.21.15.59:0/477593686 shutdown_connections 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 -- 172.21.15.59:0/477593686 >> 172.21.15.59:0/477593686 conn(0x7fe1d0067d90 msgr2=0x7fe1d006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 -- 172.21.15.59:0/477593686 shutdown_connections 2024-04-24T06:29:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.288+0000 7fe1d5a00640 1 -- 172.21.15.59:0/477593686 wait complete. 2024-04-24T06:29:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.291+0000 7fe1d5a00640 1 Processor -- start 2024-04-24T06:29:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.291+0000 7fe1d5a00640 1 -- start start 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1b4004d40 0x7fe1d0152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d015a9d0 0x7fe1d00ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d006ddd0 con 0x7fe1d01ee810 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d015bc10 con 0x7fe1d015a9d0 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1d5a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d006df50 con 0x7fe1b4004d40 2024-04-24T06:29:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32866/0 (socket says 172.21.15.59:32866) 2024-04-24T06:29:36.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.293+0000 7fe1c7e00640 1 -- 172.21.15.59:0/3172118727 learned_addr learned my addr 172.21.15.59:0/3172118727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1cea00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1b4004d40 0x7fe1d0152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1cf400640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d015a9d0 0x7fe1d00ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 -- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1b4004d40 msgr2=0x7fe1d0152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1b4004d40 0x7fe1d0152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 -- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d015a9d0 msgr2=0x7fe1d00ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d015a9d0 0x7fe1d00ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 -- 172.21.15.59:0/3172118727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d01531f0 con 0x7fe1d01ee810 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c7e00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8010890 tx=0x7fe1b8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800f040 con 0x7fe1d01ee810 2024-04-24T06:29:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b800eb00 con 0x7fe1d01ee810 2024-04-24T06:29:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.294+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d01534d0 con 0x7fe1d01ee810 2024-04-24T06:29:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.295+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b801c400 con 0x7fe1d01ee810 2024-04-24T06:29:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.295+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d015ae50 con 0x7fe1d01ee810 2024-04-24T06:29:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.296+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d0068a80 con 0x7fe1d01ee810 2024-04-24T06:29:36.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.297+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1b801c5a0 con 0x7fe1d01ee810 2024-04-24T06:29:36.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.298+0000 7fe1c6a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 0x7fe1ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.298+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe1b808c1c0 con 0x7fe1d01ee810 2024-04-24T06:29:36.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.298+0000 7fe1cea00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 0x7fe1ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.299+0000 7fe1cea00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 0x7fe1ac06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000fb0 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.302+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1b8018030 con 0x7fe1d01ee810 2024-04-24T06:29:36.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.506+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe1d01536a0 con 0x7fe1d01ee810 2024-04-24T06:29:36.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.506+0000 7fe1c6a00640 1 -- 172.21.15.59:0/3172118727 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe1b805a4d0 con 0x7fe1d01ee810 2024-04-24T06:29:36.508 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 msgr2=0x7fe1ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 0x7fe1ac06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000fb0 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2024-04-24T06:29:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 msgr2=0x7fe1d0152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8010890 tx=0x7fe1b8010d50 comp rx=0 tx=0).stop 2024-04-24T06:29:36.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 shutdown_connections 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c920 0x7fe1ac06ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d015a9d0 0x7fe1d00ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1d01ee810 0x7fe1d0152540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 --2- 172.21.15.59:0/3172118727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1b4004d40 0x7fe1d0152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 >> 172.21.15.59:0/3172118727 conn(0x7fe1d0067d90 msgr2=0x7fe1d00b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 shutdown_connections 2024-04-24T06:29:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.509+0000 7fe1d5a00640 1 -- 172.21.15.59:0/3172118727 wait complete. 2024-04-24T06:29:36.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:36.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.724+0000 7f2a9ea00640 1 Processor -- start 2024-04-24T06:29:36.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.724+0000 7f2a9ea00640 1 -- start start 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.725+0000 7f2a9ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 0x7f2a980d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a9ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2a98160180 0x7f2a980d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a9ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a980cff50 con 0x7f2a9815cb20 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a9ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9815cf00 con 0x7f2a98160180 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a9ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2a980d01c0 con 0x7f2a98163770 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a97e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2a9815cb20 0x7f2a980cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54400/0 (socket says 172.21.15.59:54400) 2024-04-24T06:29:36.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a97e00640 1 -- 172.21.15.59:0/2747915169 learned_addr learned my addr 172.21.15.59:0/2747915169 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:36.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32525077 0 0) 0x7f2a980cff50 con 0x7f2a9815cb20 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a9ca00640 1 --2- 172.21.15.59:0/2747915169 >> v2:172.21.15.66:3300/0 conn(0x7f2a98160180 0x7f2a980d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a97400640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 0x7f2a980d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.726+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a7c003520 con 0x7f2a9815cb20 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 765401859 0 0) 0x7f2a7c003520 con 0x7f2a9815cb20 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 msgr2=0x7f2a980d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 0x7f2a980d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 >> v2:172.21.15.66:3300/0 conn(0x7f2a98160180 msgr2=0x7f2a980d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 --2- 172.21.15.59:0/2747915169 >> v2:172.21.15.66:3300/0 conn(0x7f2a98160180 0x7f2a980d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9815f6a0 con 0x7f2a9815cb20 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2a88001670 con 0x7f2a9815cb20 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 >> v1:172.21.15.59:6789/0 conn(0x7f2a9815cb20 legacy=0x7f2a980cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a7c0039b0 0x7f2a7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 0x7f2a7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a7c004d40 0x7f2a7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9815cf00 con 0x7f2a7c0039b0 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980d01c0 con 0x7f2a7c004d40 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c00baa0 con 0x7f2a98163770 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a7c0039b0 0x7f2a7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 -- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 msgr2=0x7f2a7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a98163770 0x7f2a7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 -- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a7c004d40 msgr2=0x7f2a7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a7c004d40 0x7f2a7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.727+0000 7f2a97e00640 1 -- 172.21.15.59:0/2747915169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c003520 con 0x7f2a7c0039b0 2024-04-24T06:29:36.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.728+0000 7f2a97e00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a7c0039b0 0x7f2a7c003dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c160 tx=0x7f2a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd70f13d2e1b55da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88007c80 con 0x7f2a7c0039b0 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a88005ad0 con 0x7f2a7c0039b0 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a96a00640 1 -- 172.21.15.59:0/2747915169 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88007070 con 0x7f2a7c0039b0 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 -- 172.21.15.59:0/2747915169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a7c0039b0 msgr2=0x7f2a7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/2747915169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a7c0039b0 0x7f2a7c003dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c160 tx=0x7f2a7c005830 comp rx=0 tx=0).stop 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ca00640 1 -- 172.21.15.59:0/2747915169 reap_dead start 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 -- 172.21.15.59:0/2747915169 shutdown_connections 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 -- 172.21.15.59:0/2747915169 >> 172.21.15.59:0/2747915169 conn(0x7f2a9806cc70 msgr2=0x7f2a9806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 -- 172.21.15.59:0/2747915169 shutdown_connections 2024-04-24T06:29:36.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.729+0000 7f2a9ea00640 1 -- 172.21.15.59:0/2747915169 wait complete. 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.731+0000 7f2a9ea00640 1 Processor -- start 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.731+0000 7f2a9ea00640 1 -- start start 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a7c0039b0 0x7f2a98151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98156710 0x7f2a98156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980d13c0 con 0x7f2a981596d0 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980d1240 con 0x7f2a98156710 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2a980d1540 con 0x7f2a7c0039b0 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32884/0 (socket says 172.21.15.59:32884) 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.732+0000 7f2a9ca00640 1 -- 172.21.15.59:0/698373798 learned_addr learned my addr 172.21.15.59:0/698373798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a97400640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98156710 0x7f2a98156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a97e00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a7c0039b0 0x7f2a98151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 -- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a7c0039b0 msgr2=0x7f2a98151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a7c0039b0 0x7f2a98151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 -- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98156710 msgr2=0x7f2a98156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98156710 0x7f2a98156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 -- 172.21.15.59:0/698373798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a98158670 con 0x7f2a981596d0 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ca00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2a980d58d0 tx=0x7f2a9000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90018020 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9000bb10 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90014400 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.733+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a98158910 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.734+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a98158ed0 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.735+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98068a80 con 0x7f2a981596d0 2024-04-24T06:29:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.736+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a90005d10 con 0x7f2a981596d0 2024-04-24T06:29:36.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.737+0000 7f2a8fe00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 0x7f2a7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:36.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.737+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2a9008c0c0 con 0x7f2a981596d0 2024-04-24T06:29:36.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.737+0000 7f2a97e00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 0x7f2a7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.738+0000 7f2a97e00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 0x7f2a7806ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2a88032000 tx=0x7f2a88007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.741+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a90022030 con 0x7f2a981596d0 2024-04-24T06:29:36.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.945+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a980d29c0 con 0x7f2a981596d0 2024-04-24T06:29:36.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.946+0000 7f2a8fe00640 1 -- 172.21.15.59:0/698373798 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2a9005a460 con 0x7f2a981596d0 2024-04-24T06:29:36.949 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:36.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 msgr2=0x7f2a7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 0x7f2a7806ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2a88032000 tx=0x7f2a88007000 comp rx=0 tx=0).stop 2024-04-24T06:29:36.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 msgr2=0x7f2a98159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2a980d58d0 tx=0x7f2a9000b480 comp rx=0 tx=0).stop 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 shutdown_connections 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a7806c920 0x7f2a7806ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a981596d0 0x7f2a98159b60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a98156710 0x7f2a98156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 --2- 172.21.15.59:0/698373798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a7c0039b0 0x7f2a98151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 >> 172.21.15.59:0/698373798 conn(0x7f2a9806cc70 msgr2=0x7f2a980cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 shutdown_connections 2024-04-24T06:29:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:36.949+0000 7f2a9ea00640 1 -- 172.21.15.59:0/698373798 wait complete. 2024-04-24T06:29:36.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.165+0000 7f0788e00640 1 Processor -- start 2024-04-24T06:29:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.165+0000 7f0788e00640 1 -- start start 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f0788e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07840d4bc0 0x7f07840b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f0788e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0784162c20 con 0x7f0784162640 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f0788e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0784162f00 con 0x7f078415ea50 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f0788e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f07840cf260 con 0x7f07840d4bc0 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f0783e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07840d4bc0 0x7f07840b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f077be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0784162640 0x7f07840d44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54414/0 (socket says 172.21.15.59:54414) 2024-04-24T06:29:37.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.167+0000 7f077be00640 1 -- 172.21.15.59:0/2995653624 learned_addr learned my addr 172.21.15.59:0/2995653624 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:37.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.168+0000 7f0783e00640 1 -- 172.21.15.59:0/2995653624 >> v1:172.21.15.66:6789/0 conn(0x7f078415ea50 legacy=0x7f07840b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:37.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.168+0000 7f0783e00640 1 -- 172.21.15.59:0/2995653624 >> v1:172.21.15.59:6789/0 conn(0x7f0784162640 legacy=0x7f07840d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.168+0000 7f0783e00640 1 -- 172.21.15.59:0/2995653624 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07841631e0 con 0x7f07840d4bc0 2024-04-24T06:29:37.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0783e00640 1 --2- 172.21.15.59:0/2995653624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07840d4bc0 0x7f07840b7410 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ef5513cfbb3fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0782a00640 1 -- 172.21.15.59:0/2995653624 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c031600 con 0x7f07840d4bc0 2024-04-24T06:29:37.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0782a00640 1 -- 172.21.15.59:0/2995653624 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c0317a0 con 0x7f07840d4bc0 2024-04-24T06:29:37.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0782a00640 1 -- 172.21.15.59:0/2995653624 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c031b70 con 0x7f07840d4bc0 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0788e00640 1 -- 172.21.15.59:0/2995653624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07840d4bc0 msgr2=0x7f07840b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.169+0000 7f0788e00640 1 --2- 172.21.15.59:0/2995653624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07840d4bc0 0x7f07840b7410 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c02a610 comp rx=0 tx=0).stop 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.170+0000 7f0788e00640 1 -- 172.21.15.59:0/2995653624 shutdown_connections 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.170+0000 7f0788e00640 1 --2- 172.21.15.59:0/2995653624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07840d4bc0 0x7f07840b7410 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.170+0000 7f0788e00640 1 -- 172.21.15.59:0/2995653624 >> 172.21.15.59:0/2995653624 conn(0x7f078409d950 msgr2=0x7f078408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.170+0000 7f0788e00640 1 -- 172.21.15.59:0/2995653624 shutdown_connections 2024-04-24T06:29:37.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.170+0000 7f0788e00640 1 -- 172.21.15.59:0/2995653624 wait complete. 2024-04-24T06:29:37.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.172+0000 7f0788e00640 1 Processor -- start 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.172+0000 7f0788e00640 1 -- start start 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 0x7f0784155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f078415ea50 0x7f07840c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f078415ee30 con 0x7f0784162640 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07840cf6f0 con 0x7f07840d4bc0 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0788e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f07840cce00 con 0x7f078415ea50 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0783e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0783e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32912/0 (socket says 172.21.15.59:32912) 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0783e00640 1 -- 172.21.15.59:0/4247630982 learned_addr learned my addr 172.21.15.59:0/4247630982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f0783400640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 0x7f0784155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.173+0000 7f077be00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f078415ea50 0x7f07840c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 -- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f078415ea50 msgr2=0x7f07840c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f078415ea50 0x7f07840c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 -- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 msgr2=0x7f0784155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 0x7f0784155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 -- 172.21.15.59:0/4247630982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07841558d0 con 0x7f0784162640 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783400640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 0x7f0784155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0783e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c040070 con 0x7f0784162640 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c030b60 con 0x7f0784162640 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c03c420 con 0x7f0784162640 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.174+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07840cd050 con 0x7f0784162640 2024-04-24T06:29:37.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.175+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07840cd610 con 0x7f0784162640 2024-04-24T06:29:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.176+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078404fd90 con 0x7f0784162640 2024-04-24T06:29:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.177+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f077c039040 con 0x7f0784162640 2024-04-24T06:29:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.178+0000 7f077b400640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 0x7f076006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.178+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f077c0acd00 con 0x7f0784162640 2024-04-24T06:29:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.179+0000 7f0783400640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 0x7f076006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.179+0000 7f0783400640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 0x7f076006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07841f8460 tx=0x7f0774009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.183+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f077c07aa80 con 0x7f0784162640 2024-04-24T06:29:37.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.405+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0784074cf0 con 0x7f0784162640 2024-04-24T06:29:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.406+0000 7f077b400640 1 -- 172.21.15.59:0/4247630982 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f077c07ed80 con 0x7f0784162640 2024-04-24T06:29:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.408+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 msgr2=0x7f076006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.408+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 0x7f076006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07841f8460 tx=0x7f0774009040 comp rx=0 tx=0).stop 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.408+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 msgr2=0x7f07840ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.408+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 shutdown_connections 2024-04-24T06:29:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f076006c920 0x7f076006ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0784162640 0x7f07840ce0f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f078415ea50 0x7f07840c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 --2- 172.21.15.59:0/4247630982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07840d4bc0 0x7f0784155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 >> 172.21.15.59:0/4247630982 conn(0x7f078409d950 msgr2=0x7f078415d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 shutdown_connections 2024-04-24T06:29:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.409+0000 7f0788e00640 1 -- 172.21.15.59:0/4247630982 wait complete. 2024-04-24T06:29:37.430 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-04-24T06:29:37.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:37.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-24T06:29:37.516 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:37.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:29:37.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:37.517 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:29:37.569 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:37.569 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:37.628 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:37.628 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:37.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:37.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.885+0000 7fb752800640 1 Processor -- start 2024-04-24T06:29:37.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.886+0000 7fb752800640 1 -- start start 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.887+0000 7fb752800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb74c15e7c0 0x7fb74c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb752800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb74c0d0d90 0x7fb74c0d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb752800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb74c0cf030 con 0x7fb74c0d2630 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb752800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb74c0b7000 con 0x7fb74c0d0d90 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb752800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb74c0cf620 con 0x7fb74c15e7c0 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb74c0d0d90 0x7fb74c0d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb750a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb74c0d2630 0x7fb74c0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54430/0 (socket says 172.21.15.59:54430) 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb74c0d0d90 0x7fb74c0d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32914/0 (socket says 172.21.15.59:32914) 2024-04-24T06:29:37.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb750a00640 1 -- 172.21.15.59:0/1824952303 learned_addr learned my addr 172.21.15.59:0/1824952303 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751400640 1 --2- 172.21.15.59:0/1824952303 >> v2:172.21.15.66:3300/0 conn(0x7fb74c15e7c0 0x7fb74c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751e00640 1 -- 172.21.15.59:0/1824952303 >> v2:172.21.15.66:3300/0 conn(0x7fb74c15e7c0 msgr2=0x7fb74c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751e00640 1 --2- 172.21.15.59:0/1824952303 >> v2:172.21.15.66:3300/0 conn(0x7fb74c15e7c0 0x7fb74c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.888+0000 7fb751e00640 1 -- 172.21.15.59:0/1824952303 >> v1:172.21.15.59:6789/0 conn(0x7fb74c0d2630 legacy=0x7fb74c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.889+0000 7fb751e00640 1 -- 172.21.15.59:0/1824952303 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c0b6a40 con 0x7fb74c0d0d90 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb751e00640 1 --2- 172.21.15.59:0/1824952303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d0d90 0x7fb74c0d11c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb74c08c6d0 tx=0x7fb74402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f04bb71434b5fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb74be00640 1 -- 172.21.15.59:0/1824952303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744007740 con 0x7fb74c0d0d90 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb74be00640 1 -- 172.21.15.59:0/1824952303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7440078e0 con 0x7fb74c0d0d90 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb74be00640 1 -- 172.21.15.59:0/1824952303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744007cb0 con 0x7fb74c0d0d90 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb752800640 1 -- 172.21.15.59:0/1824952303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d0d90 msgr2=0x7fb74c0d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.890+0000 7fb752800640 1 --2- 172.21.15.59:0/1824952303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d0d90 0x7fb74c0d11c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb74c08c6d0 tx=0x7fb74402a680 comp rx=0 tx=0).stop 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 -- 172.21.15.59:0/1824952303 shutdown_connections 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 --2- 172.21.15.59:0/1824952303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d0d90 0x7fb74c0d11c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 --2- 172.21.15.59:0/1824952303 >> v2:172.21.15.66:3300/0 conn(0x7fb74c15e7c0 0x7fb74c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 -- 172.21.15.59:0/1824952303 >> 172.21.15.59:0/1824952303 conn(0x7fb74c09d920 msgr2=0x7fb74c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 -- 172.21.15.59:0/1824952303 shutdown_connections 2024-04-24T06:29:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.891+0000 7fb752800640 1 -- 172.21.15.59:0/1824952303 wait complete. 2024-04-24T06:29:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.893+0000 7fb752800640 1 Processor -- start 2024-04-24T06:29:37.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.893+0000 7fb752800640 1 -- start start 2024-04-24T06:29:37.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74c0d0d90 0x7fb74c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb74c15e7c0 0x7fb74c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c0cf2b0 con 0x7fb74c0d2630 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c0d1700 con 0x7fb74c0d0d90 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb752800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb74c0b6700 con 0x7fb74c15e7c0 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb750a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb750a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32928/0 (socket says 172.21.15.59:32928) 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb750a00640 1 -- 172.21.15.59:0/2931854135 learned_addr learned my addr 172.21.15.59:0/2931854135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb751400640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74c0d0d90 0x7fb74c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.894+0000 7fb751e00640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb74c15e7c0 0x7fb74c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 -- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb74c15e7c0 msgr2=0x7fb74c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb74c15e7c0 0x7fb74c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 -- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74c0d0d90 msgr2=0x7fb74c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74c0d0d90 0x7fb74c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 -- 172.21.15.59:0/2931854135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c0cdeb0 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb750a00640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c470 tx=0x7fb738001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738017070 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb738005ce0 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7380072f0 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c0cca60 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.895+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c0cd020 con 0x7fb74c0d2630 2024-04-24T06:29:37.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.897+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74c04fd90 con 0x7fb74c0d2630 2024-04-24T06:29:37.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.898+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb73801a070 con 0x7fb74c0d2630 2024-04-24T06:29:37.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.899+0000 7fb749600640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 0x7fb72c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:37.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.899+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb73808afa0 con 0x7fb74c0d2630 2024-04-24T06:29:37.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.899+0000 7fb751400640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 0x7fb72c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:37.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.899+0000 7fb751400640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 0x7fb72c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb740006fd0 tx=0x7fb740008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:37.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:37.903+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb74c0cf030 con 0x7fb74c0d2630 2024-04-24T06:29:38.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.103+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb74c1609a0 con 0x7fb74c0d2630 2024-04-24T06:29:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.105+0000 7fb749600640 1 -- 172.21.15.59:0/2931854135 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7fb738059340 con 0x7fb74c0d2630 2024-04-24T06:29:38.107 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:38.107 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 msgr2=0x7fb72c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 0x7fb72c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb740006fd0 tx=0x7fb740008040 comp rx=0 tx=0).stop 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 msgr2=0x7fb74c0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c470 tx=0x7fb738001e70 comp rx=0 tx=0).stop 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 shutdown_connections 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb72c06c8f0 0x7fb72c06edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb74c15e7c0 0x7fb74c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb74c0d2630 0x7fb74c0cd540 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 --2- 172.21.15.59:0/2931854135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb74c0d0d90 0x7fb74c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.107+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 >> 172.21.15.59:0/2931854135 conn(0x7fb74c09d920 msgr2=0x7fb74c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:38.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.108+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 shutdown_connections 2024-04-24T06:29:38.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.108+0000 7fb752800640 1 -- 172.21.15.59:0/2931854135 wait complete. 2024-04-24T06:29:38.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:38.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.333+0000 7fc5b1800640 1 Processor -- start 2024-04-24T06:29:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.333+0000 7fc5b1800640 1 -- start start 2024-04-24T06:29:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5b1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5ac0b4f20 0x7fc5ac0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5b1800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac0b5da0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5b1800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac0d0a20 con 0x7fc5ac15ed20 2024-04-24T06:29:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5b1800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac15f100 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5a3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc5ac0d15b0 0x7fc5ac0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54436/0 (socket says 172.21.15.59:54436) 2024-04-24T06:29:38.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.335+0000 7fc5a3e00640 1 -- 172.21.15.59:0/1588699385 learned_addr learned my addr 172.21.15.59:0/1588699385 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:38.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5b0e00640 1 --2- 172.21.15.59:0/1588699385 >> v2:172.21.15.59:3300/0 conn(0x7fc5ac0b4f20 0x7fc5ac0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982224731 0 0) 0x7fc5ac0b5da0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003520 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2681017028 0 0) 0x7fc5ac0d0a20 con 0x7fc5ac15ed20 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5ac0b5da0 con 0x7fc5ac15ed20 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5b0e00640 1 -- 172.21.15.59:0/1588699385 >> v1:172.21.15.66:6789/0 conn(0x7fc5ac15ed20 legacy=0x7fc5ac0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 259514243 0 0) 0x7fc590003520 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5b0e00640 1 -- 172.21.15.59:0/1588699385 >> v1:172.21.15.59:6789/0 conn(0x7fc5ac0d15b0 legacy=0x7fc5ac0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.336+0000 7fc5b0e00640 1 -- 172.21.15.59:0/1588699385 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac0d2ed0 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.338+0000 7fc5b0e00640 1 --2- 172.21.15.59:0/1588699385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cf990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d90 tx=0x7fc5a402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed33a27efd17bb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.338+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594001670 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4007540 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5ab400640 1 -- 172.21.15.59:0/1588699385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40078f0 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 -- 172.21.15.59:0/1588699385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0b4f20 msgr2=0x7fc5ac0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1588699385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cf990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009d90 tx=0x7fc5a402a560 comp rx=0 tx=0).stop 2024-04-24T06:29:38.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 -- 172.21.15.59:0/1588699385 shutdown_connections 2024-04-24T06:29:38.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1588699385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cf990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 -- 172.21.15.59:0/1588699385 >> 172.21.15.59:0/1588699385 conn(0x7fc5ac08bf90 msgr2=0x7fc5ac0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 -- 172.21.15.59:0/1588699385 shutdown_connections 2024-04-24T06:29:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.339+0000 7fc5b1800640 1 -- 172.21.15.59:0/1588699385 wait complete. 2024-04-24T06:29:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.341+0000 7fc5b1800640 1 Processor -- start 2024-04-24T06:29:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.341+0000 7fc5b1800640 1 -- start start 2024-04-24T06:29:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5ac15ed20 0x7fc5ac152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0b3600 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0d3490 con 0x7fc5ac15ed20 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.342+0000 7fc5b1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac0b3780 con 0x7fc5ac0b4f20 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32944/0 (socket says 172.21.15.59:32944) 2024-04-24T06:29:38.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 -- 172.21.15.59:0/1924036973 learned_addr learned my addr 172.21.15.59:0/1924036973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:38.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5abe00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 -- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5ac0b4f20 msgr2=0x7fc5ac0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 -- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5ac15ed20 msgr2=0x7fc5ac152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5ac15ed20 0x7fc5ac152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 -- 172.21.15.59:0/1924036973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac152f80 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.343+0000 7fc5a3e00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5940017e0 tx=0x7fc59400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.344+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594017070 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.344+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc594005ad0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.344+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594007070 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.344+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac0ca5b0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.344+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac0cbc70 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.345+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac1f47b0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.346+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc59401a070 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.347+0000 7fc5a8c00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 0x7fc58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.347+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc59408b060 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.348+0000 7fc5abe00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 0x7fc58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.348+0000 7fc5abe00640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 0x7fc58806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0011e0 tx=0x7fc59c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.352+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5ac152f80 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.553+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5ac0d42e0 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.554+0000 7fc5a8c00640 1 -- 172.21.15.59:0/1924036973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fc59405f030 con 0x7fc5ac0d15b0 2024-04-24T06:29:38.557 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 msgr2=0x7fc58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 0x7fc58806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0011e0 tx=0x7fc59c009040 comp rx=0 tx=0).stop 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 msgr2=0x7fc5ac0cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5940017e0 tx=0x7fc59400cc00 comp rx=0 tx=0).stop 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 shutdown_connections 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc58806c8f0 0x7fc58806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5ac15ed20 0x7fc5ac152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ac0d15b0 0x7fc5ac0cd730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 --2- 172.21.15.59:0/1924036973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5ac0b4f20 0x7fc5ac0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 >> 172.21.15.59:0/1924036973 conn(0x7fc5ac08bf90 msgr2=0x7fc5ac15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 shutdown_connections 2024-04-24T06:29:38.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.557+0000 7fc5b1800640 1 -- 172.21.15.59:0/1924036973 wait complete. 2024-04-24T06:29:38.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:38.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.789+0000 7f2f0e600640 1 Processor -- start 2024-04-24T06:29:38.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.789+0000 7f2f0e600640 1 -- start start 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.790+0000 7f2f0e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f0815d300 0x7f2f08071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.790+0000 7f2f0e600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 0x7f2f08163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.790+0000 7f2f0e600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f080d0600 0x7f2f080d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0e600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2f08163650 con 0x7f2f0815d300 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0e600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2f08163320 con 0x7f2f080d0600 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0e600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08072380 con 0x7f2f08167180 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f0815d300 0x7f2f08071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f0815d300 0x7f2f08071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32946/0 (socket says 172.21.15.59:32946) 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 -- 172.21.15.59:0/1713175679 learned_addr learned my addr 172.21.15.59:0/1713175679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0dc00640 1 --2- 172.21.15.59:0/1713175679 >> v2:172.21.15.66:3300/0 conn(0x7f2f080d0600 0x7f2f080d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f07e00640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 0x7f2f08163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 -- 172.21.15.59:0/1713175679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 msgr2=0x7f2f08163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 0x7f2f08163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 -- 172.21.15.59:0/1713175679 >> v2:172.21.15.66:3300/0 conn(0x7f2f080d0600 msgr2=0x7f2f080d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 --2- 172.21.15.59:0/1713175679 >> v2:172.21.15.66:3300/0 conn(0x7f2f080d0600 0x7f2f080d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f0d200640 1 -- 172.21.15.59:0/1713175679 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08072660 con 0x7f2f0815d300 2024-04-24T06:29:38.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.791+0000 7f2f07e00640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 0x7f2f08163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:38.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.793+0000 7f2f0d200640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f0815d300 0x7f2f08071c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009870 tx=0x7f2ef802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c526fc63f07f61e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.793+0000 7f2f07400640 1 -- 172.21.15.59:0/1713175679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8037070 con 0x7f2f0815d300 2024-04-24T06:29:38.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.793+0000 7f2f07400640 1 -- 172.21.15.59:0/1713175679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8032440 con 0x7f2f0815d300 2024-04-24T06:29:38.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.793+0000 7f2f07400640 1 -- 172.21.15.59:0/1713175679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8032810 con 0x7f2f0815d300 2024-04-24T06:29:38.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 -- 172.21.15.59:0/1713175679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f0815d300 msgr2=0x7f2f08071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f0815d300 0x7f2f08071c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009870 tx=0x7f2ef802a210 comp rx=0 tx=0).stop 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 -- 172.21.15.59:0/1713175679 shutdown_connections 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 --2- 172.21.15.59:0/1713175679 >> v2:172.21.15.66:3300/0 conn(0x7f2f080d0600 0x7f2f080d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f08167180 0x7f2f08163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 --2- 172.21.15.59:0/1713175679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f0815d300 0x7f2f08071c70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 -- 172.21.15.59:0/1713175679 >> 172.21.15.59:0/1713175679 conn(0x7f2f0806bdf0 msgr2=0x7f2f0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 -- 172.21.15.59:0/1713175679 shutdown_connections 2024-04-24T06:29:38.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.794+0000 7f2f0e600640 1 -- 172.21.15.59:0/1713175679 wait complete. 2024-04-24T06:29:38.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.796+0000 7f2f0e600640 1 Processor -- start 2024-04-24T06:29:38.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.796+0000 7f2f0e600640 1 -- start start 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f080d0600 0x7f2f0809d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 0x7f2f0814bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08164020 con 0x7f2f08167180 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081675e0 con 0x7f2f0815d300 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0e600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0806e1e0 con 0x7f2f080d0600 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32954/0 (socket says 172.21.15.59:32954) 2024-04-24T06:29:38.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f0dc00640 1 -- 172.21.15.59:0/3523320272 learned_addr learned my addr 172.21.15.59:0/3523320272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:38.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.797+0000 7f2f07e00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 0x7f2f0814bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 -- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f080d0600 msgr2=0x7f2f0809d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f080d0600 0x7f2f0809d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 -- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 msgr2=0x7f2f0814bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 0x7f2f0814bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 -- 172.21.15.59:0/3523320272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0809e010 con 0x7f2f08167180 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f07e00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 0x7f2f0814bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0dc00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007e60 tx=0x7f2f0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00014620 con 0x7f2f08167180 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00019070 con 0x7f2f08167180 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00014a20 con 0x7f2f08167180 2024-04-24T06:29:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08158f30 con 0x7f2f08167180 2024-04-24T06:29:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.798+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081594f0 con 0x7f2f08167180 2024-04-24T06:29:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.800+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0804fd90 con 0x7f2f08167180 2024-04-24T06:29:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.801+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f00014bc0 con 0x7f2f08167180 2024-04-24T06:29:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.801+0000 7f2f04c00640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 0x7f2ee406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.802+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2f0008c2b0 con 0x7f2f08167180 2024-04-24T06:29:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.802+0000 7f2f0d200640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 0x7f2ee406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.802+0000 7f2f0d200640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 0x7f2ee406ee00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8034000 tx=0x7f2ef802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:38.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:38.806+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f0005a7d0 con 0x7f2f08167180 2024-04-24T06:29:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.011+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f080cff50 con 0x7f2f08167180 2024-04-24T06:29:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.012+0000 7f2f04c00640 1 -- 172.21.15.59:0/3523320272 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f2f0005a0d0 con 0x7f2f08167180 2024-04-24T06:29:39.014 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 msgr2=0x7f2ee406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 0x7f2ee406ee00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8034000 tx=0x7f2ef802c000 comp rx=0 tx=0).stop 2024-04-24T06:29:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 msgr2=0x7f2f0814c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007e60 tx=0x7f2f0000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 shutdown_connections 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2ee406c940 0x7f2ee406ee00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f08167180 0x7f2f0814c4c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f0815d300 0x7f2f0814bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 --2- 172.21.15.59:0/3523320272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f080d0600 0x7f2f0809d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 >> 172.21.15.59:0/3523320272 conn(0x7f2f0806bdf0 msgr2=0x7f2f081666a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 shutdown_connections 2024-04-24T06:29:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.015+0000 7f2f0e600640 1 -- 172.21.15.59:0/3523320272 wait complete. 2024-04-24T06:29:39.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:39.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.255+0000 7fc7c8800640 1 Processor -- start 2024-04-24T06:29:39.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.256+0000 7fc7c8800640 1 -- start start 2024-04-24T06:29:39.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.257+0000 7fc7c8800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7c0160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.257+0000 7fc7c8800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c0160450 con 0x7fc7c0146120 2024-04-24T06:29:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.257+0000 7fc7c8800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c00b2b60 con 0x7fc7c006b9d0 2024-04-24T06:29:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.257+0000 7fc7c8800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c01606f0 con 0x7fc7c00cd790 2024-04-24T06:29:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.258+0000 7fc7c5400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7c0146120 0x7fc7c00b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52994/0 (socket says 172.21.15.59:52994) 2024-04-24T06:29:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.258+0000 7fc7c5400640 1 -- 172.21.15.59:0/1175484537 learned_addr learned my addr 172.21.15.59:0/1175484537 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.258+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209536533 0 0) 0x7fc7c0160450 con 0x7fc7c0146120 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.258+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7a8003520 con 0x7fc7c0146120 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.258+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7c0160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3828319406 0 0) 0x7fc7a8003520 con 0x7fc7c0146120 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 msgr2=0x7fc7c0160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7c0160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 >> v1:172.21.15.66:6789/0 conn(0x7fc7c006b9d0 legacy=0x7fc7c00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c01609d0 con 0x7fc7c0146120 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc7b0001690 con 0x7fc7c0146120 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 >> v1:172.21.15.59:6789/0 conn(0x7fc7c0146120 legacy=0x7fc7c00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7a8004d40 0x7fc7a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c01606f0 con 0x7fc7a80039e0 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a800bb50 con 0x7fc7a8004d40 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a800bd20 con 0x7fc7c00cd790 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c5400640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.259+0000 7fc7c6800640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7a8004d40 0x7fc7a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.260+0000 7fc7c5e00640 1 -- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 msgr2=0x7fc7a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.260+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00cd790 0x7fc7a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.260+0000 7fc7c5e00640 1 -- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7a8004d40 msgr2=0x7fc7a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.260+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7a8004d40 0x7fc7a80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.260+0000 7fc7c5e00640 1 -- 172.21.15.59:0/1175484537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a8003520 con 0x7fc7a80039e0 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7a8003e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4007390 tx=0x7fc7a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b312835c3cac6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4009be0 con 0x7fc7a80039e0 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b402e620 con 0x7fc7a80039e0 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c4a00640 1 -- 172.21.15.59:0/1175484537 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b402e9f0 con 0x7fc7a80039e0 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c8800640 1 -- 172.21.15.59:0/1175484537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 msgr2=0x7fc7a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.261+0000 7fc7c8800640 1 --2- 172.21.15.59:0/1175484537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7a8003e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4007390 tx=0x7fc7a8005830 comp rx=0 tx=0).stop 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.262+0000 7fc7c6800640 1 -- 172.21.15.59:0/1175484537 reap_dead start 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.262+0000 7fc7c8800640 1 -- 172.21.15.59:0/1175484537 shutdown_connections 2024-04-24T06:29:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.262+0000 7fc7c8800640 1 -- 172.21.15.59:0/1175484537 >> 172.21.15.59:0/1175484537 conn(0x7fc7c0073430 msgr2=0x7fc7c009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.262+0000 7fc7c8800640 1 -- 172.21.15.59:0/1175484537 shutdown_connections 2024-04-24T06:29:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.262+0000 7fc7c8800640 1 -- 172.21.15.59:0/1175484537 wait complete. 2024-04-24T06:29:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.264+0000 7fc7c8800640 1 Processor -- start 2024-04-24T06:29:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.264+0000 7fc7c8800640 1 -- start start 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c00ae360 0x7fc7c00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00ae7e0 0x7fc7c00b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0161070 con 0x7fc7a80039e0 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0160ef0 con 0x7fc7c00ae360 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c8800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c01611f0 con 0x7fc7c00ae7e0 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48220/0 (socket says 172.21.15.59:48220) 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5e00640 1 -- 172.21.15.59:0/3729513262 learned_addr learned my addr 172.21.15.59:0/3729513262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5e00640 1 -- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00ae7e0 msgr2=0x7fc7c00b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c6800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00ae7e0 0x7fc7c00b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5400640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c00ae360 0x7fc7c00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.265+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00ae7e0 0x7fc7c00b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c5e00640 1 -- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c00ae360 msgr2=0x7fc7c00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c00ae360 0x7fc7c00aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c5e00640 1 -- 172.21.15.59:0/3729513262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c00b0d50 con 0x7fc7a80039e0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c5e00640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc7b402b0a0 tx=0x7fc7b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4045070 con 0x7fc7a80039e0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4003990 con 0x7fc7a80039e0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c00b1030 con 0x7fc7a80039e0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4009be0 con 0x7fc7a80039e0 2024-04-24T06:29:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.266+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c00c8ce0 con 0x7fc7a80039e0 2024-04-24T06:29:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.268+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c00b1550 con 0x7fc7a80039e0 2024-04-24T06:29:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.269+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7b4048070 con 0x7fc7a80039e0 2024-04-24T06:29:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.270+0000 7fc7ba000640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 0x7fc7a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.270+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc7b40ac5a0 con 0x7fc7a80039e0 2024-04-24T06:29:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.270+0000 7fc7c5400640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 0x7fc7a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.271+0000 7fc7c5400640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 0x7fc7a406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0001eb0 tx=0x7fc7b0001da0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.274+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7c00b1550 con 0x7fc7a80039e0 2024-04-24T06:29:39.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.476+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7c0069190 con 0x7fc7a80039e0 2024-04-24T06:29:39.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.477+0000 7fc7ba000640 1 -- 172.21.15.59:0/3729513262 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc7b403d070 con 0x7fc7a80039e0 2024-04-24T06:29:39.479 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 msgr2=0x7fc7a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 0x7fc7a406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0001eb0 tx=0x7fc7b0001da0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 msgr2=0x7fc7c00c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc7b402b0a0 tx=0x7fc7b4004ab0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 shutdown_connections 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.479+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7a406c8f0 0x7fc7a406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c00ae7e0 0x7fc7c00b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c00ae360 0x7fc7c00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 --2- 172.21.15.59:0/3729513262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7a80039e0 0x7fc7c00c7030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 >> 172.21.15.59:0/3729513262 conn(0x7fc7c0073430 msgr2=0x7fc7c00cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 shutdown_connections 2024-04-24T06:29:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.480+0000 7fc7c8800640 1 -- 172.21.15.59:0/3729513262 wait complete. 2024-04-24T06:29:39.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.704+0000 7fcab6a00640 1 Processor -- start 2024-04-24T06:29:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.704+0000 7fcab6a00640 1 -- start start 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcab6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcab00b44c0 0x7fcab00aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcab6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcab0162de0 0x7fcab006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcab6a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcab01632e0 con 0x7fcab0161e60 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcab6a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcab00aefd0 con 0x7fcab00b44c0 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcab6a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcab00adcd0 con 0x7fcab0162de0 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcaaf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcab00b44c0 0x7fcab00aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcaaf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcab00b44c0 0x7fcab00aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48234/0 (socket says 172.21.15.59:48234) 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.706+0000 7fcaaf400640 1 -- 172.21.15.59:0/195785264 learned_addr learned my addr 172.21.15.59:0/195785264 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.707+0000 7fcab4a00640 1 --2- 172.21.15.59:0/195785264 >> v2:172.21.15.66:3300/0 conn(0x7fcab0162de0 0x7fcab006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.707+0000 7fcaaf400640 1 -- 172.21.15.59:0/195785264 >> v2:172.21.15.66:3300/0 conn(0x7fcab0162de0 msgr2=0x7fcab006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.707+0000 7fcaaf400640 1 --2- 172.21.15.59:0/195785264 >> v2:172.21.15.66:3300/0 conn(0x7fcab0162de0 0x7fcab006ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.707+0000 7fcaaf400640 1 -- 172.21.15.59:0/195785264 >> v1:172.21.15.66:6789/0 conn(0x7fcab0161e60 legacy=0x7fcab00b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.707+0000 7fcaaf400640 1 -- 172.21.15.59:0/195785264 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab00adfb0 con 0x7fcab00b44c0 2024-04-24T06:29:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.709+0000 7fcaaf400640 1 --2- 172.21.15.59:0/195785264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab00b44c0 0x7fcab00aea90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcab0075670 tx=0x7fca9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42a74ccae0e24ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.709+0000 7fcaaea00640 1 -- 172.21.15.59:0/195785264 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c037070 con 0x7fcab00b44c0 2024-04-24T06:29:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.709+0000 7fcaaea00640 1 -- 172.21.15.59:0/195785264 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9c032440 con 0x7fcab00b44c0 2024-04-24T06:29:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.709+0000 7fcaaea00640 1 -- 172.21.15.59:0/195785264 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c032810 con 0x7fcab00b44c0 2024-04-24T06:29:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 -- 172.21.15.59:0/195785264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab00b44c0 msgr2=0x7fcab00aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 --2- 172.21.15.59:0/195785264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab00b44c0 0x7fcab00aea90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcab0075670 tx=0x7fca9c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 -- 172.21.15.59:0/195785264 shutdown_connections 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 --2- 172.21.15.59:0/195785264 >> v2:172.21.15.66:3300/0 conn(0x7fcab0162de0 0x7fcab006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 --2- 172.21.15.59:0/195785264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab00b44c0 0x7fcab00aea90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 -- 172.21.15.59:0/195785264 >> 172.21.15.59:0/195785264 conn(0x7fcab00b2130 msgr2=0x7fcab00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 -- 172.21.15.59:0/195785264 shutdown_connections 2024-04-24T06:29:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.710+0000 7fcab6a00640 1 -- 172.21.15.59:0/195785264 wait complete. 2024-04-24T06:29:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.712+0000 7fcab6a00640 1 Processor -- start 2024-04-24T06:29:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.713+0000 7fcab6a00640 1 -- start start 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcab00b44c0 0x7fcab0153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcab0162de0 0x7fcab01609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00af480 con 0x7fcab0161e60 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00ae570 con 0x7fcab00b44c0 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab6a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcab00af600 con 0x7fcab0162de0 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48246/0 (socket says 172.21.15.59:48246) 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcaaf400640 1 -- 172.21.15.59:0/3571030066 learned_addr learned my addr 172.21.15.59:0/3571030066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcaafe00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcab00b44c0 0x7fcab0153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.714+0000 7fcab4a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcab0162de0 0x7fcab01609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 -- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcab0162de0 msgr2=0x7fcab01609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcab0162de0 0x7fcab01609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 -- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcab00b44c0 msgr2=0x7fcab0153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcab00b44c0 0x7fcab0153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 -- 172.21.15.59:0/3571030066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab01595a0 con 0x7fcab0161e60 2024-04-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaaf400640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000c00 tx=0x7fca9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c041020 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9c030970 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c037040 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab0159880 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.715+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab0153d70 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.717+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab0068a80 con 0x7fcab0161e60 2024-04-24T06:29:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.718+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca9c044020 con 0x7fcab0161e60 2024-04-24T06:29:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.718+0000 7fcaa7e00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 0x7fca9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.718+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fca9c0ac650 con 0x7fcab0161e60 2024-04-24T06:29:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.719+0000 7fcaafe00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 0x7fca9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.719+0000 7fcaafe00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 0x7fca9006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0004ea0 tx=0x7fcaa0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:39.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.724+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca9c04b030 con 0x7fcab0161e60 2024-04-24T06:29:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.942+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcab0158410 con 0x7fcab0161e60 2024-04-24T06:29:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.942+0000 7fcaa7e00640 1 -- 172.21.15.59:0/3571030066 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fca9c07a9f0 con 0x7fcab0161e60 2024-04-24T06:29:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 msgr2=0x7fca9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 0x7fca9006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0004ea0 tx=0x7fcaa0009040 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 msgr2=0x7fcab01604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000c00 tx=0x7fca9c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 shutdown_connections 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca9006c8f0 0x7fca9006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcab0162de0 0x7fcab01609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab0161e60 0x7fcab01604a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 --2- 172.21.15.59:0/3571030066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcab00b44c0 0x7fcab0153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 >> 172.21.15.59:0/3571030066 conn(0x7fcab00b2130 msgr2=0x7fcab006ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.945+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 shutdown_connections 2024-04-24T06:29:39.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:39.946+0000 7fcab6a00640 1 -- 172.21.15.59:0/3571030066 wait complete. 2024-04-24T06:29:39.966 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:29:39.966 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:40.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:40.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.227+0000 7f826fe00640 1 Processor -- start 2024-04-24T06:29:40.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.228+0000 7f826fe00640 1 -- start start 2024-04-24T06:29:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.229+0000 7f826fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f826806f7c0 0x7f82680b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.229+0000 7f826fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826806ec60 con 0x7f826815b840 2024-04-24T06:29:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.229+0000 7f826fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82680b46e0 con 0x7f826806df00 2024-04-24T06:29:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.229+0000 7f826fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f82680b49c0 con 0x7f826806f7c0 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f826815b840 0x7f826806fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53008/0 (socket says 172.21.15.59:53008) 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826ca00640 1 -- 172.21.15.59:0/3433061721 learned_addr learned my addr 172.21.15.59:0/3433061721 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826d400640 1 --2- 172.21.15.59:0/3433061721 >> v2:172.21.15.59:3300/0 conn(0x7f826806f7c0 0x7f82680b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826d400640 1 -- 172.21.15.59:0/3433061721 >> v1:172.21.15.66:6789/0 conn(0x7f826806df00 legacy=0x7f826806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826d400640 1 -- 172.21.15.59:0/3433061721 >> v1:172.21.15.59:6789/0 conn(0x7f826815b840 legacy=0x7f826806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f826d400640 1 -- 172.21.15.59:0/3433061721 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82680b4ca0 con 0x7f826806f7c0 2024-04-24T06:29:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.230+0000 7f8267e00640 1 -- 172.21.15.59:0/3433061721 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645076298 0 0) 0x7f826806ec60 con 0x7f826815b840 2024-04-24T06:29:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.231+0000 7f826d400640 1 --2- 172.21.15.59:0/3433061721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f82680b54a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f825802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a7f2df47d4ed06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.232+0000 7f8267e00640 1 -- 172.21.15.59:0/3433061721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258007740 con 0x7f826806f7c0 2024-04-24T06:29:40.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.232+0000 7f8267e00640 1 -- 172.21.15.59:0/3433061721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82580078e0 con 0x7f826806f7c0 2024-04-24T06:29:40.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.232+0000 7f8267e00640 1 -- 172.21.15.59:0/3433061721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258007cb0 con 0x7f826806f7c0 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.232+0000 7f826fe00640 1 -- 172.21.15.59:0/3433061721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 msgr2=0x7f82680b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.232+0000 7f826fe00640 1 --2- 172.21.15.59:0/3433061721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f82680b54a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f825802a6a0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.233+0000 7f826fe00640 1 -- 172.21.15.59:0/3433061721 shutdown_connections 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.233+0000 7f826fe00640 1 --2- 172.21.15.59:0/3433061721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f82680b54a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.233+0000 7f826fe00640 1 -- 172.21.15.59:0/3433061721 >> 172.21.15.59:0/3433061721 conn(0x7f8268067ea0 msgr2=0x7f826806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.233+0000 7f826fe00640 1 -- 172.21.15.59:0/3433061721 shutdown_connections 2024-04-24T06:29:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.233+0000 7f826fe00640 1 -- 172.21.15.59:0/3433061721 wait complete. 2024-04-24T06:29:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.235+0000 7f826fe00640 1 Processor -- start 2024-04-24T06:29:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.235+0000 7f826fe00640 1 -- start start 2024-04-24T06:29:40.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826806df00 0x7f8268159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8268072320 0x7f8268152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82680ce340 con 0x7f826806f7c0 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f82680ce1c0 con 0x7f826806df00 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f82680ce4c0 con 0x7f8268072320 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48258/0 (socket says 172.21.15.59:48258) 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 -- 172.21.15.59:0/3317484936 learned_addr learned my addr 172.21.15.59:0/3317484936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826d400640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826806df00 0x7f8268159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 -- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8268072320 msgr2=0x7f8268152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8268072320 0x7f8268152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 -- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826806df00 msgr2=0x7f8268159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826806df00 0x7f8268159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.236+0000 7f826ca00640 1 -- 172.21.15.59:0/3317484936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8268159bb0 con 0x7f826806f7c0 2024-04-24T06:29:40.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f826ca00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f825400c460 tx=0x7f825400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82540074e0 con 0x7f826806f7c0 2024-04-24T06:29:40.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82540047e0 con 0x7f826806f7c0 2024-04-24T06:29:40.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254004bd0 con 0x7f826806f7c0 2024-04-24T06:29:40.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8268153700 con 0x7f826806f7c0 2024-04-24T06:29:40.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.237+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8268153cc0 con 0x7f826806f7c0 2024-04-24T06:29:40.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.238+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8268068a80 con 0x7f826806f7c0 2024-04-24T06:29:40.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.239+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8254011070 con 0x7f826806f7c0 2024-04-24T06:29:40.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.240+0000 7f8265600640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.240+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f825408b7e0 con 0x7f826806f7c0 2024-04-24T06:29:40.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.241+0000 7f826d400640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.241+0000 7f826d400640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 0x7f824806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8258034000 tx=0x7f825802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.245+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8254059b90 con 0x7f826806f7c0 2024-04-24T06:29:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.451+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f826806f370 con 0x7f826806f7c0 2024-04-24T06:29:40.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.454+0000 7f8265600640 1 -- 172.21.15.59:0/3317484936 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f8254059490 con 0x7f826806f7c0 2024-04-24T06:29:40.456 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:40.456 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 msgr2=0x7f824806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 0x7f824806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8258034000 tx=0x7f825802f000 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 msgr2=0x7f8268071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f825400c460 tx=0x7f825400c920 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 shutdown_connections 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8268072320 0x7f8268152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f826806f7c0 0x7f8268071de0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 --2- 172.21.15.59:0/3317484936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f826806df00 0x7f8268159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 >> 172.21.15.59:0/3317484936 conn(0x7f8268067ea0 msgr2=0x7f82680b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.457+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 shutdown_connections 2024-04-24T06:29:40.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.458+0000 7f826fe00640 1 -- 172.21.15.59:0/3317484936 wait complete. 2024-04-24T06:29:40.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:40.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.692+0000 7fb78f200640 1 Processor -- start 2024-04-24T06:29:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.692+0000 7fb78f200640 1 -- start start 2024-04-24T06:29:40.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.693+0000 7fb78f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.693+0000 7fb78f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 0x7fb788168660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.694+0000 7fb78f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7880ce390 con 0x7fb78815e920 2024-04-24T06:29:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.694+0000 7fb78f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb7880b0bb0 con 0x7fb7880cdc40 2024-04-24T06:29:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.694+0000 7fb78f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880cec30 con 0x7fb78806c0e0 2024-04-24T06:29:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.694+0000 7fb78d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.694+0000 7fb78e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 0x7fb788168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.695+0000 7fb78de00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fb78815e920 0x7fb78806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:52882/0 (socket says 172.21.15.59:52882) 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.695+0000 7fb78d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:33520/0 (socket says 172.21.15.59:33520) 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.695+0000 7fb78de00640 1 -- 172.21.15.59:0/3523749499 learned_addr learned my addr 172.21.15.59:0/3523749499 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.695+0000 7fb78d400640 1 -- 172.21.15.59:0/3523749499 >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 msgr2=0x7fb788168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.696+0000 7fb78d400640 1 --2- 172.21.15.59:0/3523749499 >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 0x7fb788168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.696+0000 7fb78d400640 1 -- 172.21.15.59:0/3523749499 >> v1:172.21.15.66:6789/0 conn(0x7fb78815e920 legacy=0x7fb78806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.696+0000 7fb78d400640 1 -- 172.21.15.59:0/3523749499 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7880ce670 con 0x7fb78806c0e0 2024-04-24T06:29:40.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.696+0000 7fb78e800640 1 --2- 172.21.15.59:0/3523749499 >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 0x7fb788168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:29:40.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.697+0000 7fb78d400640 1 --2- 172.21.15.59:0/3523749499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7780097a0 tx=0x7fb77802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f909c1bb7af8c449 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.697+0000 7fb78ca00640 1 -- 172.21.15.59:0/3523749499 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb78806c0e0 2024-04-24T06:29:40.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.697+0000 7fb78ca00640 1 -- 172.21.15.59:0/3523749499 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77802adb0 con 0x7fb78806c0e0 2024-04-24T06:29:40.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78ca00640 1 -- 172.21.15.59:0/3523749499 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778032660 con 0x7fb78806c0e0 2024-04-24T06:29:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 -- 172.21.15.59:0/3523749499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 msgr2=0x7fb7880cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 --2- 172.21.15.59:0/3523749499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7780097a0 tx=0x7fb77802a060 comp rx=0 tx=0).stop 2024-04-24T06:29:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 -- 172.21.15.59:0/3523749499 shutdown_connections 2024-04-24T06:29:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 --2- 172.21.15.59:0/3523749499 >> v2:172.21.15.66:3300/0 conn(0x7fb7880cdc40 0x7fb788168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 --2- 172.21.15.59:0/3523749499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb78806c0e0 0x7fb7880cd5f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 -- 172.21.15.59:0/3523749499 >> 172.21.15.59:0/3523749499 conn(0x7fb7880b2f10 msgr2=0x7fb7880b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:40.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 -- 172.21.15.59:0/3523749499 shutdown_connections 2024-04-24T06:29:40.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.698+0000 7fb78f200640 1 -- 172.21.15.59:0/3523749499 wait complete. 2024-04-24T06:29:40.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.700+0000 7fb78f200640 1 Processor -- start 2024-04-24T06:29:40.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.700+0000 7fb78f200640 1 -- start start 2024-04-24T06:29:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7880cdc40 0x7fb7880c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb78815e920 0x7fb7880b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880ccff0 con 0x7fb78806c0e0 2024-04-24T06:29:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880ce070 con 0x7fb78815e920 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.701+0000 7fb78f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880cd170 con 0x7fb7880cdc40 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48268/0 (socket says 172.21.15.59:48268) 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 -- 172.21.15.59:0/4231463399 learned_addr learned my addr 172.21.15.59:0/4231463399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78e800640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb78815e920 0x7fb7880b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78d400640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7880cdc40 0x7fb7880c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 -- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7880cdc40 msgr2=0x7fb7880c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7880cdc40 0x7fb7880c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 -- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb78815e920 msgr2=0x7fb7880b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb78815e920 0x7fb7880b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 -- 172.21.15.59:0/4231463399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7880ca700 con 0x7fb78806c0e0 2024-04-24T06:29:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.702+0000 7fb78de00640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c430 tx=0x7fb77c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.703+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c017070 con 0x7fb78806c0e0 2024-04-24T06:29:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.703+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c005ce0 con 0x7fb78806c0e0 2024-04-24T06:29:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.703+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0072f0 con 0x7fb78806c0e0 2024-04-24T06:29:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.703+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7880ca9e0 con 0x7fb78806c0e0 2024-04-24T06:29:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.703+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7880b5550 con 0x7fb78806c0e0 2024-04-24T06:29:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.704+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7880b1890 con 0x7fb78806c0e0 2024-04-24T06:29:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.705+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c01a070 con 0x7fb78806c0e0 2024-04-24T06:29:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.707+0000 7fb782000640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.707+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb77c08afa0 con 0x7fb78806c0e0 2024-04-24T06:29:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.707+0000 7fb78d400640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.708+0000 7fb78d400640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:40.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.711+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7880b1890 con 0x7fb78806c0e0 2024-04-24T06:29:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.927+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb78804fd90 con 0x7fb78806c0e0 2024-04-24T06:29:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.928+0000 7fb782000640 1 -- 172.21.15.59:0/4231463399 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fb77c059340 con 0x7fb78806c0e0 2024-04-24T06:29:40.930 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.930+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 msgr2=0x7fb76c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.930+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).stop 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.930+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 msgr2=0x7fb7881531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.930+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c430 tx=0x7fb77c00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 shutdown_connections 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb78815e920 0x7fb7880b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7880cdc40 0x7fb7880c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 --2- 172.21.15.59:0/4231463399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78806c0e0 0x7fb7881531a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 >> 172.21.15.59:0/4231463399 conn(0x7fb7880b2f10 msgr2=0x7fb7880b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 shutdown_connections 2024-04-24T06:29:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:40.931+0000 7fb78f200640 1 -- 172.21.15.59:0/4231463399 wait complete. 2024-04-24T06:29:40.954 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:41.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.166+0000 7f0bc4600640 1 Processor -- start 2024-04-24T06:29:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.167+0000 7f0bc4600640 1 -- start start 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc4600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0bbc16d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc4600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bbc0b35d0 con 0x7f0bbc162c10 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc4600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bbc0b38b0 con 0x7f0bbc0b4670 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc4600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc0b3b90 con 0x7f0bbc072480 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc1c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0bbc162c10 0x7f0bbc0d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53016/0 (socket says 172.21.15.59:53016) 2024-04-24T06:29:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.168+0000 7f0bc1c00640 1 -- 172.21.15.59:0/913043477 learned_addr learned my addr 172.21.15.59:0/913043477 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bc2600640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0bbc16d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695939586 0 0) 0x7f0bbc0b35d0 con 0x7f0bbc162c10 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ba0003520 con 0x7f0bbc162c10 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2472730417 0 0) 0x7f0ba0003520 con 0x7f0bbc162c10 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 msgr2=0x7f0bbc16d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0bbc16d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 >> v1:172.21.15.66:6789/0 conn(0x7f0bbc0b4670 legacy=0x7f0bbc071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc0b3e70 con 0x7f0bbc162c10 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0bac001670 con 0x7f0bbc162c10 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.169+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 >> v1:172.21.15.59:6789/0 conn(0x7f0bbc162c10 legacy=0x7f0bbc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba00039e0 0x7f0ba0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0ba0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0004d40 0x7f0ba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc0b3b90 con 0x7f0ba00039e0 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba000baa0 con 0x7f0ba0004d40 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba000bca0 con 0x7f0bbc072480 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba00039e0 0x7f0ba0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc1200640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0ba0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc1c00640 1 -- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 msgr2=0x7f0ba0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0ba0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc2600640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0004d40 0x7f0ba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.170+0000 7f0bc1c00640 1 -- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0004d40 msgr2=0x7f0ba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.171+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba0004d40 0x7f0ba00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.171+0000 7f0bc1c00640 1 -- 172.21.15.59:0/913043477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba0003520 con 0x7f0ba00039e0 2024-04-24T06:29:41.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.171+0000 7f0bc1200640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc072480 0x7f0ba0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:41.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.172+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba00039e0 0x7f0ba0003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c210 tx=0x7f0ba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bbb84577908530 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.172+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac007cf0 con 0x7f0ba00039e0 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.172+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac005af0 con 0x7f0ba00039e0 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.172+0000 7f0bbbe00640 1 -- 172.21.15.59:0/913043477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac007070 con 0x7f0ba00039e0 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 -- 172.21.15.59:0/913043477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba00039e0 msgr2=0x7f0ba0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 --2- 172.21.15.59:0/913043477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba00039e0 0x7f0ba0003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c210 tx=0x7f0ba0005830 comp rx=0 tx=0).stop 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc2600640 1 -- 172.21.15.59:0/913043477 reap_dead start 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 -- 172.21.15.59:0/913043477 shutdown_connections 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 -- 172.21.15.59:0/913043477 >> 172.21.15.59:0/913043477 conn(0x7f0bbc067d90 msgr2=0x7f0bbc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 -- 172.21.15.59:0/913043477 shutdown_connections 2024-04-24T06:29:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.173+0000 7f0bc4600640 1 -- 172.21.15.59:0/913043477 wait complete. 2024-04-24T06:29:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.175+0000 7f0bc4600640 1 Processor -- start 2024-04-24T06:29:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.176+0000 7f0bc4600640 1 -- start start 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba00039e0 0x7f0bbc163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc15b7e0 0x7f0bbc15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc163170 con 0x7f0bbc159d50 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc162ff0 con 0x7f0ba00039e0 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc4600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc0cfba0 con 0x7f0bbc15b7e0 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48282/0 (socket says 172.21.15.59:48282) 2024-04-24T06:29:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 -- 172.21.15.59:0/3948431728 learned_addr learned my addr 172.21.15.59:0/3948431728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba00039e0 0x7f0bbc163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc2600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc15b7e0 0x7f0bbc15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 -- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc15b7e0 msgr2=0x7f0bbc15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc15b7e0 0x7f0bbc15bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 -- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba00039e0 msgr2=0x7f0bbc163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.177+0000 7f0bc1200640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba00039e0 0x7f0bbc163320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bc1200640 1 -- 172.21.15.59:0/3948431728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc163a90 con 0x7f0bbc159d50 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bc1200640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800d800 tx=0x7f0ba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8005070 con 0x7f0bbc159d50 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba8005210 con 0x7f0bbc159d50 2024-04-24T06:29:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8007030 con 0x7f0bbc159d50 2024-04-24T06:29:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc163d70 con 0x7f0bbc159d50 2024-04-24T06:29:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.178+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1f84f0 con 0x7f0bbc159d50 2024-04-24T06:29:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.180+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc068a80 con 0x7f0bbc159d50 2024-04-24T06:29:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.181+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ba801a020 con 0x7f0bbc159d50 2024-04-24T06:29:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.181+0000 7f0bb9600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 0x7f0b9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.182+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0ba808b350 con 0x7f0bbc159d50 2024-04-24T06:29:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.182+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 0x7f0b9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.183+0000 7f0bc1c00640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 0x7f0b9c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0bac032000 tx=0x7f0bac009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.186+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ba80597c0 con 0x7f0bbc159d50 2024-04-24T06:29:41.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.390+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0bbc071ea0 con 0x7f0bbc159d50 2024-04-24T06:29:41.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.391+0000 7f0bb9600640 1 -- 172.21.15.59:0/3948431728 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f0ba80590c0 con 0x7f0bbc159d50 2024-04-24T06:29:41.393 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.393+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 msgr2=0x7f0b9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.393+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 0x7f0b9c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0bac032000 tx=0x7f0bac009030 comp rx=0 tx=0).stop 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.393+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 msgr2=0x7f0bbc1f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.393+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800d800 tx=0x7f0ba800dcc0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 shutdown_connections 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b9c06c920 0x7f0b9c06ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0bbc15b7e0 0x7f0bbc15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bbc159d50 0x7f0bbc1f7fb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 --2- 172.21.15.59:0/3948431728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ba00039e0 0x7f0bbc163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 >> 172.21.15.59:0/3948431728 conn(0x7f0bbc067d90 msgr2=0x7f0bbc15e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 shutdown_connections 2024-04-24T06:29:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.394+0000 7f0bc4600640 1 -- 172.21.15.59:0/3948431728 wait complete. 2024-04-24T06:29:41.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.618+0000 7ff5c8a00640 1 Processor -- start 2024-04-24T06:29:41.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.618+0000 7ff5c8a00640 1 -- start start 2024-04-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff5c41567e0 0x7ff5c4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c40d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c8a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c415cab0 con 0x7ff5c40d0b40 2024-04-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c8a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c40d0f70 con 0x7ff5c41567e0 2024-04-24T06:29:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c4160e40 con 0x7ff5c415d810 2024-04-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.619+0000 7ff5c2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff5c41567e0 0x7ff5c4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c3200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff5c40d0b40 0x7ff5c4160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53030/0 (socket says 172.21.15.59:53030) 2024-04-24T06:29:41.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c3200640 1 -- 172.21.15.59:0/2801594671 learned_addr learned my addr 172.21.15.59:0/2801594671 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c1e00640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c40d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c2800640 1 -- 172.21.15.59:0/2801594671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 msgr2=0x7ff5c40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c2800640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c40d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c2800640 1 -- 172.21.15.59:0/2801594671 >> v1:172.21.15.59:6789/0 conn(0x7ff5c40d0b40 legacy=0x7ff5c4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c2800640 1 -- 172.21.15.59:0/2801594671 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c415cd30 con 0x7ff5c41567e0 2024-04-24T06:29:41.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.620+0000 7ff5c1400640 1 -- 172.21.15.59:0/2801594671 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27130002 0 0) 0x7ff5bc001110 con 0x7ff5c40d0b40 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.621+0000 7ff5c2800640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c4156bc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009d90 tx=0x7ff5b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80b8763999fd73c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.621+0000 7ff5c1400640 1 -- 172.21.15.59:0/2801594671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4007740 con 0x7ff5c41567e0 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.621+0000 7ff5c1400640 1 -- 172.21.15.59:0/2801594671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b40078e0 con 0x7ff5c41567e0 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.621+0000 7ff5c1400640 1 -- 172.21.15.59:0/2801594671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4007cb0 con 0x7ff5c41567e0 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 -- 172.21.15.59:0/2801594671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 msgr2=0x7ff5c4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c4156bc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009d90 tx=0x7ff5b402a660 comp rx=0 tx=0).stop 2024-04-24T06:29:41.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 -- 172.21.15.59:0/2801594671 shutdown_connections 2024-04-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c40d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/2801594671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c4156bc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 -- 172.21.15.59:0/2801594671 >> 172.21.15.59:0/2801594671 conn(0x7ff5c406bdf0 msgr2=0x7ff5c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 -- 172.21.15.59:0/2801594671 shutdown_connections 2024-04-24T06:29:41.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.622+0000 7ff5c8a00640 1 -- 172.21.15.59:0/2801594671 wait complete. 2024-04-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.624+0000 7ff5c8a00640 1 Processor -- start 2024-04-24T06:29:41.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.624+0000 7ff5c8a00640 1 -- start start 2024-04-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40d0b40 0x7ff5c415e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4160820 con 0x7ff5c41567e0 2024-04-24T06:29:41.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c415d2f0 con 0x7ff5c40d0b40 2024-04-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c41609a0 con 0x7ff5c415d810 2024-04-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.625+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48304/0 (socket says 172.21.15.59:48304) 2024-04-24T06:29:41.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 -- 172.21.15.59:0/4238229171 learned_addr learned my addr 172.21.15.59:0/4238229171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c2800640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40d0b40 0x7ff5c415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c3200640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 -- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 msgr2=0x7ff5c414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c414f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 -- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40d0b40 msgr2=0x7ff5c415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40d0b40 0x7ff5c415e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 -- 172.21.15.59:0/4238229171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c414fdf0 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.626+0000 7ff5c1e00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000d830 tx=0x7ff5b000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.627+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000b6c0 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.627+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b0019070 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.627+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000ba80 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.627+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c41500d0 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.627+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c4150690 con 0x7ff5c41567e0 2024-04-24T06:29:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.628+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c404fd90 con 0x7ff5c41567e0 2024-04-24T06:29:41.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.629+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b001e460 con 0x7ff5c41567e0 2024-04-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.630+0000 7ff5baa00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.630+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff5b008c2e0 con 0x7ff5c41567e0 2024-04-24T06:29:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.630+0000 7ff5c2800640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.631+0000 7ff5c2800640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 0x7ff5a406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4034000 tx=0x7ff5b402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.634+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5b005a680 con 0x7ff5c41567e0 2024-04-24T06:29:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.838+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5c40b5150 con 0x7ff5c41567e0 2024-04-24T06:29:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.839+0000 7ff5baa00640 1 -- 172.21.15.59:0/4238229171 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff5b0059f80 con 0x7ff5c41567e0 2024-04-24T06:29:41.842 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 msgr2=0x7ff5a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 0x7ff5a406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4034000 tx=0x7ff5b402f000 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 msgr2=0x7ff5c41520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000d830 tx=0x7ff5b000dcf0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 shutdown_connections 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff5c415d810 0x7ff5c414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5c41567e0 0x7ff5c41520c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.842+0000 7ff5c8a00640 1 --2- 172.21.15.59:0/4238229171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40d0b40 0x7ff5c415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.843+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 >> 172.21.15.59:0/4238229171 conn(0x7ff5c406bdf0 msgr2=0x7ff5c415f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.843+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 shutdown_connections 2024-04-24T06:29:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:41.843+0000 7ff5c8a00640 1 -- 172.21.15.59:0/4238229171 wait complete. 2024-04-24T06:29:41.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.071+0000 7fe78b200640 1 Processor -- start 2024-04-24T06:29:42.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.071+0000 7fe78b200640 1 -- start start 2024-04-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.072+0000 7fe78b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 0x7fe78c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.072+0000 7fe78b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe78c0b4230 0x7fe78c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.072+0000 7fe78b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe78c0b4cf0 con 0x7fe78c0d05c0 2024-04-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.072+0000 7fe78b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe78c15ca50 con 0x7fe78c0b4230 2024-04-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.072+0000 7fe78b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe78c0ccb80 con 0x7fe78c15c670 2024-04-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe78a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe78c0d05c0 0x7fe78c160160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53042/0 (socket says 172.21.15.59:53042) 2024-04-24T06:29:42.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe78a800640 1 -- 172.21.15.59:0/2190954774 learned_addr learned my addr 172.21.15.59:0/2190954774 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe789e00640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 0x7fe78c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe789400640 1 --2- 172.21.15.59:0/2190954774 >> v2:172.21.15.66:3300/0 conn(0x7fe78c0b4230 0x7fe78c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962875505 0 0) 0x7fe78c0b4cf0 con 0x7fe78c0d05c0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.073+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe770003520 con 0x7fe78c0d05c0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3826637624 0 0) 0x7fe770003520 con 0x7fe78c0d05c0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 msgr2=0x7fe78c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 0x7fe78c0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 >> v2:172.21.15.66:3300/0 conn(0x7fe78c0b4230 msgr2=0x7fe78c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 --2- 172.21.15.59:0/2190954774 >> v2:172.21.15.66:3300/0 conn(0x7fe78c0b4230 0x7fe78c0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c0ccdd0 con 0x7fe78c0d05c0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe784001670 con 0x7fe78c0d05c0 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 >> v1:172.21.15.59:6789/0 conn(0x7fe78c0d05c0 legacy=0x7fe78c160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7700039b0 0x7fe770003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 0x7fe770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe7700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c0ccb80 con 0x7fe770004d40 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe77000b9c0 con 0x7fe7700039b0 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe77000bb40 con 0x7fe78c15c670 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.074+0000 7fe78a800640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe7700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.075+0000 7fe78a800640 1 -- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 msgr2=0x7fe770004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.075+0000 7fe78a800640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c15c670 0x7fe770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.075+0000 7fe78a800640 1 -- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7700039b0 msgr2=0x7fe770003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.075+0000 7fe78a800640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7700039b0 0x7fe770003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.075+0000 7fe78a800640 1 -- 172.21.15.59:0/2190954774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770003520 con 0x7fe770004d40 2024-04-24T06:29:42.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe78a800640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe7700051c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c210 tx=0x7fe770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303a9feb70c87158 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784007d30 con 0x7fe770004d40 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe784005ad0 con 0x7fe770004d40 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe788a00640 1 -- 172.21.15.59:0/2190954774 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784007070 con 0x7fe770004d40 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe78b200640 1 -- 172.21.15.59:0/2190954774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 msgr2=0x7fe7700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.076+0000 7fe78b200640 1 --2- 172.21.15.59:0/2190954774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe7700051c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c210 tx=0x7fe770005830 comp rx=0 tx=0).stop 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.077+0000 7fe78a800640 1 -- 172.21.15.59:0/2190954774 reap_dead start 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.077+0000 7fe78b200640 1 -- 172.21.15.59:0/2190954774 shutdown_connections 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.077+0000 7fe78b200640 1 -- 172.21.15.59:0/2190954774 >> 172.21.15.59:0/2190954774 conn(0x7fe78c06bdf0 msgr2=0x7fe78c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.077+0000 7fe78b200640 1 -- 172.21.15.59:0/2190954774 shutdown_connections 2024-04-24T06:29:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.077+0000 7fe78b200640 1 -- 172.21.15.59:0/2190954774 wait complete. 2024-04-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.078+0000 7fe78b200640 1 Processor -- start 2024-04-24T06:29:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.079+0000 7fe78b200640 1 -- start start 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c0b1c00 0x7fe78c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c1532c0 0x7fe78c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c0b3e60 con 0x7fe770004d40 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c0cd390 con 0x7fe78c1532c0 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe78b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe78c0b3fe0 con 0x7fe78c0b1c00 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48324/0 (socket says 172.21.15.59:48324) 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 -- 172.21.15.59:0/4081496517 learned_addr learned my addr 172.21.15.59:0/4081496517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 -- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c0b1c00 msgr2=0x7fe78c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c0b1c00 0x7fe78c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 -- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c1532c0 msgr2=0x7fe78c153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c1532c0 0x7fe78c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.080+0000 7fe789e00640 1 -- 172.21.15.59:0/4081496517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c15caf0 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe789e00640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00e7b0 tx=0x7fe77c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c017070 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77c00c7e0 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00cbd0 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c15cdd0 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.081+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c15d390 con 0x7fe770004d40 2024-04-24T06:29:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.082+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c0b25a0 con 0x7fe770004d40 2024-04-24T06:29:42.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.083+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe77c01a070 con 0x7fe770004d40 2024-04-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.084+0000 7fe782000640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 0x7fe76c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.084+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe77c08bcc0 con 0x7fe770004d40 2024-04-24T06:29:42.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.084+0000 7fe789400640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 0x7fe76c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.085+0000 7fe789400640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 0x7fe76c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe778004b30 tx=0x7fe77800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.089+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe77c092080 con 0x7fe770004d40 2024-04-24T06:29:42.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.305+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe78c04fd90 con 0x7fe770004d40 2024-04-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.306+0000 7fe782000640 1 -- 172.21.15.59:0/4081496517 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe77c059fd0 con 0x7fe770004d40 2024-04-24T06:29:42.308 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 msgr2=0x7fe76c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 0x7fe76c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe778004b30 tx=0x7fe77800b040 comp rx=0 tx=0).stop 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 msgr2=0x7fe78c150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00e7b0 tx=0x7fe77c00ec70 comp rx=0 tx=0).stop 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 shutdown_connections 2024-04-24T06:29:42.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe76c06c8f0 0x7fe76c06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c1532c0 0x7fe78c153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe78c0b1c00 0x7fe78c0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 --2- 172.21.15.59:0/4081496517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe770004d40 0x7fe78c150490 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 >> 172.21.15.59:0/4081496517 conn(0x7fe78c06bdf0 msgr2=0x7fe78c0acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 shutdown_connections 2024-04-24T06:29:42.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.309+0000 7fe78b200640 1 -- 172.21.15.59:0/4081496517 wait complete. 2024-04-24T06:29:42.330 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:29:42.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:42.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-24T06:29:42.416 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:42.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:29:42.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:42.416 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:29:42.470 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:42.470 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:42.533 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:42.534 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:42.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.808+0000 7fa826600640 1 Processor -- start 2024-04-24T06:29:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.808+0000 7fa826600640 1 -- start start 2024-04-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.809+0000 7fa826600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa82015c700 0x7fa8200d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.809+0000 7fa826600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 0x7fa82006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.809+0000 7fa826600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8200d0c70 con 0x7fa82006bf10 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.809+0000 7fa826600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa8200b46b0 con 0x7fa82015c700 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.809+0000 7fa826600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa8200d0f50 con 0x7fa820160d50 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.810+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 0x7fa82006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.810+0000 7fa825200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa82015c700 0x7fa8200d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.810+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 0x7fa82006b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:33594/0 (socket says 172.21.15.59:33594) 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.810+0000 7fa825200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa82015c700 0x7fa8200d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:40100/0 (socket says 172.21.15.59:40100) 2024-04-24T06:29:42.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.810+0000 7fa81fe00640 1 -- 172.21.15.59:0/4035193072 learned_addr learned my addr 172.21.15.59:0/4035193072 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.811+0000 7fa825200640 1 -- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 msgr2=0x7fa82006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.811+0000 7fa825200640 1 --2- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 0x7fa82006b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.811+0000 7fa825200640 1 -- 172.21.15.59:0/4035193072 >> v1:172.21.15.66:6789/0 conn(0x7fa82006bf10 legacy=0x7fa8200cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.811+0000 7fa825200640 1 -- 172.21.15.59:0/4035193072 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8200b4340 con 0x7fa82015c700 2024-04-24T06:29:42.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.812+0000 7fa825200640 1 --2- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015c700 0x7fa8200d0600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6977da50515dbf8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.813+0000 7fa81f400640 1 -- 172.21.15.59:0/4035193072 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810037070 con 0x7fa82015c700 2024-04-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.813+0000 7fa81f400640 1 -- 172.21.15.59:0/4035193072 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81002ae40 con 0x7fa82015c700 2024-04-24T06:29:42.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.813+0000 7fa81f400640 1 -- 172.21.15.59:0/4035193072 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810032660 con 0x7fa82015c700 2024-04-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 -- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015c700 msgr2=0x7fa8200d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 --2- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015c700 0x7fa8200d0600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa8100097a0 tx=0x7fa81002a0f0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 -- 172.21.15.59:0/4035193072 shutdown_connections 2024-04-24T06:29:42.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 --2- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa820160d50 0x7fa82006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 --2- 172.21.15.59:0/4035193072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82015c700 0x7fa8200d0600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 -- 172.21.15.59:0/4035193072 >> 172.21.15.59:0/4035193072 conn(0x7fa82009aa00 msgr2=0x7fa8200d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 -- 172.21.15.59:0/4035193072 shutdown_connections 2024-04-24T06:29:42.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.814+0000 7fa826600640 1 -- 172.21.15.59:0/4035193072 wait complete. 2024-04-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.816+0000 7fa826600640 1 Processor -- start 2024-04-24T06:29:42.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.816+0000 7fa826600640 1 -- start start 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 0x7fa8200c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820160d50 0x7fa8200c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8200b5b50 con 0x7fa82006bf10 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201611b0 con 0x7fa820160d50 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.817+0000 7fa826600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa8200b5cd0 con 0x7fa82015c700 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48336/0 (socket says 172.21.15.59:48336) 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 -- 172.21.15.59:0/1571255150 learned_addr learned my addr 172.21.15.59:0/1571255150 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825c00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820160d50 0x7fa8200c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa81fe00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 0x7fa8200c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 -- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 msgr2=0x7fa8200c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 0x7fa8200c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 -- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820160d50 msgr2=0x7fa8200c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820160d50 0x7fa8200c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa825200640 1 -- 172.21.15.59:0/1571255150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa820157e60 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.818+0000 7fa81fe00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 0x7fa8200c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa825200640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a6e0 tx=0x7fa8100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810042020 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81003d690 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810037040 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82015d660 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.819+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82015dc20 con 0x7fa82006bf10 2024-04-24T06:29:42.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.820+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82015e330 con 0x7fa82006bf10 2024-04-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.821+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa810045020 con 0x7fa82006bf10 2024-04-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.823+0000 7fa81cc00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 0x7fa7fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.823+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa8100ac380 con 0x7fa82006bf10 2024-04-24T06:29:42.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.823+0000 7fa81fe00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 0x7fa7fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:42.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.824+0000 7fa81fe00640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 0x7fa7fc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa820160140 tx=0x7fa808008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:42.828+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa81004c030 con 0x7fa82006bf10 2024-04-24T06:29:43.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.032+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8200b7260 con 0x7fa82006bf10 2024-04-24T06:29:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.034+0000 7fa81cc00640 1 -- 172.21.15.59:0/1571255150 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7fa81007a690 con 0x7fa82006bf10 2024-04-24T06:29:43.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:43.037 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 msgr2=0x7fa7fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 0x7fa7fc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa820160140 tx=0x7fa808008040 comp rx=0 tx=0).stop 2024-04-24T06:29:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 msgr2=0x7fa820157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa81002a6e0 tx=0x7fa8100046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 shutdown_connections 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.037+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7fc06c8f0 0x7fa7fc06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820160d50 0x7fa8200c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa82015c700 0x7fa8200c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 --2- 172.21.15.59:0/1571255150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82006bf10 0x7fa820157370 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 >> 172.21.15.59:0/1571255150 conn(0x7fa82009aa00 msgr2=0x7fa820070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 shutdown_connections 2024-04-24T06:29:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.038+0000 7fa826600640 1 -- 172.21.15.59:0/1571255150 wait complete. 2024-04-24T06:29:43.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:43.059 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:43.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.270+0000 7fcedca00640 1 Processor -- start 2024-04-24T06:29:43.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.271+0000 7fcedca00640 1 -- start start 2024-04-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.272+0000 7fcedca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fced8160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.272+0000 7fcedca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced815cab0 con 0x7fced80d0d10 2024-04-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.272+0000 7fcedca00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced8160e40 con 0x7fced81567e0 2024-04-24T06:29:43.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.272+0000 7fcedca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fced815cd30 con 0x7fced815d810 2024-04-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fcecfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fced80d0d10 0x7fced8160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53048/0 (socket says 172.21.15.59:53048) 2024-04-24T06:29:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fcecfe00640 1 -- 172.21.15.59:0/2994623564 learned_addr learned my addr 172.21.15.59:0/2994623564 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fced7e00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fced8160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177703333 0 0) 0x7fcec0001110 con 0x7fced80d0d10 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcebc003520 con 0x7fced80d0d10 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.273+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879934 0 0) 0x7fcebc003520 con 0x7fced80d0d10 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 msgr2=0x7fced8160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fced8160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 >> v1:172.21.15.66:6789/0 conn(0x7fced81567e0 legacy=0x7fced80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced815d010 con 0x7fced80d0d10 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fcec0001840 con 0x7fced80d0d10 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 >> v1:172.21.15.59:6789/0 conn(0x7fced80d0d10 legacy=0x7fced8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcebc0039e0 0x7fcebc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fcebc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fced815cd30 con 0x7fcebc0039e0 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc00ba70 con 0x7fcebc004800 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcebc00bc70 con 0x7fced815d810 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced7400640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcebc0039e0 0x7fcebc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced7e00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fcebc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced7400640 1 -- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 msgr2=0x7fcebc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced7400640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fced815d810 0x7fcebc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fcecfe00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.274+0000 7fced7400640 1 -- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcebc004800 msgr2=0x7fcebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.275+0000 7fced7400640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.275+0000 7fced7400640 1 -- 172.21.15.59:0/2994623564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc003520 con 0x7fcebc0039e0 2024-04-24T06:29:43.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fced7400640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcebc0039e0 0x7fcebc003e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcec8001720 tx=0x7fcebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b639e2304b84eec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8001370 con 0x7fcebc0039e0 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec80052d0 con 0x7fcebc0039e0 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fced6a00640 1 -- 172.21.15.59:0/2994623564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec802ab90 con 0x7fcebc0039e0 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fcedca00640 1 -- 172.21.15.59:0/2994623564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcebc0039e0 msgr2=0x7fcebc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fcedca00640 1 --2- 172.21.15.59:0/2994623564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcebc0039e0 0x7fcebc003e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcec8001720 tx=0x7fcebc005830 comp rx=0 tx=0).stop 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.276+0000 7fced7e00640 1 -- 172.21.15.59:0/2994623564 reap_dead start 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.277+0000 7fcedca00640 1 -- 172.21.15.59:0/2994623564 shutdown_connections 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.277+0000 7fcedca00640 1 -- 172.21.15.59:0/2994623564 >> 172.21.15.59:0/2994623564 conn(0x7fced806bdf0 msgr2=0x7fced806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.277+0000 7fcedca00640 1 -- 172.21.15.59:0/2994623564 shutdown_connections 2024-04-24T06:29:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.277+0000 7fcedca00640 1 -- 172.21.15.59:0/2994623564 wait complete. 2024-04-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.278+0000 7fcedca00640 1 Processor -- start 2024-04-24T06:29:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.279+0000 7fcedca00640 1 -- start start 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcebc0039e0 0x7fced81f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced8153930 0x7fced8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fced81609a0 con 0x7fced814c7f0 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fced8160820 con 0x7fced8153930 2024-04-24T06:29:43.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcedca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fced8160b20 con 0x7fcebc0039e0 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48352/0 (socket says 172.21.15.59:48352) 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 -- 172.21.15.59:0/464135332 learned_addr learned my addr 172.21.15.59:0/464135332 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fced7400640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcebc0039e0 0x7fced81f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fced7e00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced8153930 0x7fced8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 -- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcebc0039e0 msgr2=0x7fced81f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcebc0039e0 0x7fced81f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 -- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced8153930 msgr2=0x7fced8153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced8153930 0x7fced8153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.280+0000 7fcecfe00640 1 -- 172.21.15.59:0/464135332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81f7fc0 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcecfe00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcec000d7a0 tx=0x7fcec000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0019070 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec000de50 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00047e0 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced814ccf0 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.281+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced814d2b0 con 0x7fced814c7f0 2024-04-24T06:29:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.282+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced804fd90 con 0x7fced814c7f0 2024-04-24T06:29:43.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.283+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcec001c070 con 0x7fced814c7f0 2024-04-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.284+0000 7fcecf400640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 0x7fceb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.285+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcec008b510 con 0x7fced814c7f0 2024-04-24T06:29:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.285+0000 7fced7400640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 0x7fceb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.285+0000 7fced7400640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 0x7fceb406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcec8032000 tx=0x7fcec802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.289+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fced815cab0 con 0x7fced814c7f0 2024-04-24T06:29:43.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.491+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fced80b4a60 con 0x7fced814c7f0 2024-04-24T06:29:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.492+0000 7fcecf400640 1 -- 172.21.15.59:0/464135332 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fcec00598b0 con 0x7fced814c7f0 2024-04-24T06:29:43.494 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.494+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 msgr2=0x7fceb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.494+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 0x7fceb406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcec8032000 tx=0x7fcec802e000 comp rx=0 tx=0).stop 2024-04-24T06:29:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.494+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 msgr2=0x7fced8158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.494+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcec000d7a0 tx=0x7fcec000dc60 comp rx=0 tx=0).stop 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 shutdown_connections 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fceb406c8f0 0x7fceb406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fced8153930 0x7fced8153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fced814c7f0 0x7fced8158780 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 --2- 172.21.15.59:0/464135332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcebc0039e0 0x7fced81f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 >> 172.21.15.59:0/464135332 conn(0x7fced806bdf0 msgr2=0x7fced80735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 shutdown_connections 2024-04-24T06:29:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.495+0000 7fcedca00640 1 -- 172.21.15.59:0/464135332 wait complete. 2024-04-24T06:29:43.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:43.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.728+0000 7fe88ac00640 1 Processor -- start 2024-04-24T06:29:43.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.728+0000 7fe88ac00640 1 -- start start 2024-04-24T06:29:43.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88ac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8840d0600 0x7fe8840ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88ac00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 0x7fe884162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88ac00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe884163240 con 0x7fe8840d09e0 2024-04-24T06:29:43.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88ac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe884072a20 con 0x7fe8840d0600 2024-04-24T06:29:43.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88ac00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe884071e10 con 0x7fe88406dcb0 2024-04-24T06:29:43.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88a200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe8840d09e0 0x7fe88406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53052/0 (socket says 172.21.15.59:53052) 2024-04-24T06:29:43.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe88a200640 1 -- 172.21.15.59:0/1462223150 learned_addr learned my addr 172.21.15.59:0/1462223150 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.729+0000 7fe889800640 1 --2- 172.21.15.59:0/1462223150 >> v2:172.21.15.59:3300/0 conn(0x7fe8840d0600 0x7fe8840ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe888e00640 1 --2- 172.21.15.59:0/1462223150 >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 0x7fe884162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe883e00640 1 -- 172.21.15.59:0/1462223150 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133422094 0 0) 0x7fe87c001110 con 0x7fe8840d09e0 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe883e00640 1 -- 172.21.15.59:0/1462223150 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe868003520 con 0x7fe8840d09e0 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe889800640 1 -- 172.21.15.59:0/1462223150 >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 msgr2=0x7fe884162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe889800640 1 --2- 172.21.15.59:0/1462223150 >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 0x7fe884162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe889800640 1 -- 172.21.15.59:0/1462223150 >> v1:172.21.15.59:6789/0 conn(0x7fe8840d09e0 legacy=0x7fe88406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe889800640 1 -- 172.21.15.59:0/1462223150 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840720f0 con 0x7fe8840d0600 2024-04-24T06:29:43.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.730+0000 7fe888e00640 1 --2- 172.21.15.59:0/1462223150 >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 0x7fe884162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:43.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.731+0000 7fe889800640 1 --2- 172.21.15.59:0/1462223150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d0600 0x7fe8840ad170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8840d11c0 tx=0x7fe87402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abf643f0ba19113 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe883e00640 1 -- 172.21.15.59:0/1462223150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874007740 con 0x7fe8840d0600 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe883e00640 1 -- 172.21.15.59:0/1462223150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8740078e0 con 0x7fe8840d0600 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe883e00640 1 -- 172.21.15.59:0/1462223150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874007cb0 con 0x7fe8840d0600 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 -- 172.21.15.59:0/1462223150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d0600 msgr2=0x7fe8840ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 --2- 172.21.15.59:0/1462223150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d0600 0x7fe8840ad170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8840d11c0 tx=0x7fe87402a680 comp rx=0 tx=0).stop 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 -- 172.21.15.59:0/1462223150 shutdown_connections 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 --2- 172.21.15.59:0/1462223150 >> v2:172.21.15.66:3300/0 conn(0x7fe88406dcb0 0x7fe884162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 --2- 172.21.15.59:0/1462223150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d0600 0x7fe8840ad170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.732+0000 7fe88ac00640 1 -- 172.21.15.59:0/1462223150 >> 172.21.15.59:0/1462223150 conn(0x7fe88406bdf0 msgr2=0x7fe88406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.733+0000 7fe88ac00640 1 -- 172.21.15.59:0/1462223150 shutdown_connections 2024-04-24T06:29:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.733+0000 7fe88ac00640 1 -- 172.21.15.59:0/1462223150 wait complete. 2024-04-24T06:29:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.734+0000 7fe88ac00640 1 Processor -- start 2024-04-24T06:29:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.735+0000 7fe88ac00640 1 -- start start 2024-04-24T06:29:43.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe88406dcb0 0x7fe8840ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8840d0600 0x7fe88414c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe88415eaf0 con 0x7fe8840d09e0 2024-04-24T06:29:43.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840726b0 con 0x7fe8840d0600 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88ac00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe8840d0e90 con 0x7fe88406dcb0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48368/0 (socket says 172.21.15.59:48368) 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe88a200640 1 -- 172.21.15.59:0/2383643357 learned_addr learned my addr 172.21.15.59:0/2383643357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe888e00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8840d0600 0x7fe88414c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.736+0000 7fe889800640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe88406dcb0 0x7fe8840ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 -- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe88406dcb0 msgr2=0x7fe8840ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe88406dcb0 0x7fe8840ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 -- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8840d0600 msgr2=0x7fe88414c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8840d0600 0x7fe88414c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 -- 172.21.15.59:0/2383643357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8840cab20 con 0x7fe8840d09e0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88a200640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c3a0 tx=0x7fe87c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c017070 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c005ce0 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88415bcd0 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0072f0 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.737+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8840cadc0 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.738+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88404fd90 con 0x7fe8840d09e0 2024-04-24T06:29:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.740+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe87c01a070 con 0x7fe8840d09e0 2024-04-24T06:29:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.741+0000 7fe881600640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.741+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe87c08afb0 con 0x7fe8840d09e0 2024-04-24T06:29:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.741+0000 7fe889800640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.742+0000 7fe889800640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe874034000 tx=0x7fe87402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:43.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.746+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe884163240 con 0x7fe8840d09e0 2024-04-24T06:29:43.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.950+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe8840cff50 con 0x7fe8840d09e0 2024-04-24T06:29:43.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.950+0000 7fe881600640 1 -- 172.21.15.59:0/2383643357 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe87c0592c0 con 0x7fe8840d09e0 2024-04-24T06:29:43.952 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.953+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 msgr2=0x7fe86406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.953+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 0x7fe86406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe874034000 tx=0x7fe87402f000 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.953+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 msgr2=0x7fe88414c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.953+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c3a0 tx=0x7fe87c00c860 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 shutdown_connections 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe86406c8f0 0x7fe86406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8840d09e0 0x7fe88414c770 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8840d0600 0x7fe88414c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 --2- 172.21.15.59:0/2383643357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe88406dcb0 0x7fe8840ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 >> 172.21.15.59:0/2383643357 conn(0x7fe88406bdf0 msgr2=0x7fe8840cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 shutdown_connections 2024-04-24T06:29:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:43.954+0000 7fe88ac00640 1 -- 172.21.15.59:0/2383643357 wait complete. 2024-04-24T06:29:43.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.194+0000 7fa12d600640 1 Processor -- start 2024-04-24T06:29:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.194+0000 7fa12d600640 1 -- start start 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa12d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa128155a80 0x7fa12806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa12d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1280d0600 0x7fa1280d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa12d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12815ef20 con 0x7fa1280b43a0 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa12d600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa12815f1c0 con 0x7fa1280d0600 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa12d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa12815f4a0 con 0x7fa128155a80 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa11fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1280d0600 0x7fa1280d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa11fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1280d0600 0x7fa1280d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48372/0 (socket says 172.21.15.59:48372) 2024-04-24T06:29:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa11fe00640 1 -- 172.21.15.59:0/2894320999 learned_addr learned my addr 172.21.15.59:0/2894320999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:44.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.196+0000 7fa127e00640 1 --2- 172.21.15.59:0/2894320999 >> v2:172.21.15.66:3300/0 conn(0x7fa128155a80 0x7fa12806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.197+0000 7fa11fe00640 1 -- 172.21.15.59:0/2894320999 >> v2:172.21.15.66:3300/0 conn(0x7fa128155a80 msgr2=0x7fa12806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:44.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.197+0000 7fa11fe00640 1 --2- 172.21.15.59:0/2894320999 >> v2:172.21.15.66:3300/0 conn(0x7fa128155a80 0x7fa12806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.197+0000 7fa11fe00640 1 -- 172.21.15.59:0/2894320999 >> v1:172.21.15.66:6789/0 conn(0x7fa1280b43a0 legacy=0x7fa1280b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:44.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.197+0000 7fa11fe00640 1 -- 172.21.15.59:0/2894320999 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12815f780 con 0x7fa1280d0600 2024-04-24T06:29:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.198+0000 7fa11fe00640 1 --2- 172.21.15.59:0/2894320999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa1280d0a90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa12806dfb0 tx=0x7fa11002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2824028d07b7903b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.198+0000 7fa127400640 1 -- 172.21.15.59:0/2894320999 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037070 con 0x7fa1280d0600 2024-04-24T06:29:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.198+0000 7fa127400640 1 -- 172.21.15.59:0/2894320999 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110032440 con 0x7fa1280d0600 2024-04-24T06:29:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa127400640 1 -- 172.21.15.59:0/2894320999 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110032810 con 0x7fa1280d0600 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 -- 172.21.15.59:0/2894320999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 msgr2=0x7fa1280d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 --2- 172.21.15.59:0/2894320999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa1280d0a90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa12806dfb0 tx=0x7fa11002a230 comp rx=0 tx=0).stop 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 -- 172.21.15.59:0/2894320999 shutdown_connections 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 --2- 172.21.15.59:0/2894320999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa1280d0a90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 --2- 172.21.15.59:0/2894320999 >> v2:172.21.15.66:3300/0 conn(0x7fa128155a80 0x7fa12806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 -- 172.21.15.59:0/2894320999 >> 172.21.15.59:0/2894320999 conn(0x7fa12806bdf0 msgr2=0x7fa12806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 -- 172.21.15.59:0/2894320999 shutdown_connections 2024-04-24T06:29:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.199+0000 7fa12d600640 1 -- 172.21.15.59:0/2894320999 wait complete. 2024-04-24T06:29:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.201+0000 7fa12d600640 1 Processor -- start 2024-04-24T06:29:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.201+0000 7fa12d600640 1 -- start start 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1280b43a0 0x7fa128148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa128155a80 0x7fa1280999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa12806ff00 con 0x7fa1280d0600 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa12815fd40 con 0x7fa128155a80 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa12d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa12806ef80 con 0x7fa1280b43a0 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48378/0 (socket says 172.21.15.59:48378) 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.202+0000 7fa11fe00640 1 -- 172.21.15.59:0/969832629 learned_addr learned my addr 172.21.15.59:0/969832629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa127e00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1280b43a0 0x7fa128148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa12cc00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa128155a80 0x7fa1280999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 -- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1280b43a0 msgr2=0x7fa128148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1280b43a0 0x7fa128148210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 -- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa128155a80 msgr2=0x7fa1280999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa128155a80 0x7fa1280999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 -- 172.21.15.59:0/969832629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281542d0 con 0x7fa1280d0600 2024-04-24T06:29:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.203+0000 7fa11fe00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa1100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.204+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110041020 con 0x7fa1280d0600 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.204+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1100306c0 con 0x7fa1280d0600 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.204+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037040 con 0x7fa1280d0600 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.204+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1281545b0 con 0x7fa1280d0600 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.204+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12814be00 con 0x7fa1280d0600 2024-04-24T06:29:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.205+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12804fd90 con 0x7fa1280d0600 2024-04-24T06:29:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.206+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa110044020 con 0x7fa1280d0600 2024-04-24T06:29:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.207+0000 7fa124c00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 0x7fa10406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.207+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa1100ac4c0 con 0x7fa1280d0600 2024-04-24T06:29:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.207+0000 7fa127e00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 0x7fa10406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.208+0000 7fa127e00640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 0x7fa10406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa118006fd0 tx=0x7fa118008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.211+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa11004b030 con 0x7fa1280d0600 2024-04-24T06:29:44.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.414+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa12806eba0 con 0x7fa1280d0600 2024-04-24T06:29:44.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.416+0000 7fa124c00640 1 -- 172.21.15.59:0/969832629 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa11007a7d0 con 0x7fa1280d0600 2024-04-24T06:29:44.418 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 msgr2=0x7fa10406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 0x7fa10406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa118006fd0 tx=0x7fa118008040 comp rx=0 tx=0).stop 2024-04-24T06:29:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 msgr2=0x7fa128099490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa1100046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 shutdown_connections 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa10406c8f0 0x7fa10406edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa128155a80 0x7fa1280999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1280d0600 0x7fa128099490 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 --2- 172.21.15.59:0/969832629 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1280b43a0 0x7fa128148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 >> 172.21.15.59:0/969832629 conn(0x7fa12806bdf0 msgr2=0x7fa12815ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 shutdown_connections 2024-04-24T06:29:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.419+0000 7fa12d600640 1 -- 172.21.15.59:0/969832629 wait complete. 2024-04-24T06:29:44.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:44.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.645+0000 7effc4000640 1 Processor -- start 2024-04-24T06:29:44.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.646+0000 7effc4000640 1 -- start start 2024-04-24T06:29:44.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc4000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7effbc06d4d0 0x7effbc09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc4000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 0x7effbc0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc4000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0b1d90 con 0x7effbc06d8b0 2024-04-24T06:29:44.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc4000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0d21f0 con 0x7effbc06d4d0 2024-04-24T06:29:44.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc4000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7effbc1623a0 con 0x7effbc070020 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc1600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7effbc06d4d0 0x7effbc09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc0c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7effbc06d8b0 0x7effbc0d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53058/0 (socket says 172.21.15.59:53058) 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc0c00640 1 -- 172.21.15.59:0/2384526672 learned_addr learned my addr 172.21.15.59:0/2384526672 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc2000640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 0x7effbc0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc1600640 1 -- 172.21.15.59:0/2384526672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 msgr2=0x7effbc0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.648+0000 7effc1600640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 0x7effbc0704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.649+0000 7effc1600640 1 -- 172.21.15.59:0/2384526672 >> v1:172.21.15.59:6789/0 conn(0x7effbc06d8b0 legacy=0x7effbc0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.649+0000 7effc1600640 1 -- 172.21.15.59:0/2384526672 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0d1150 con 0x7effbc06d4d0 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.649+0000 7effbbe00640 1 -- 172.21.15.59:0/2384526672 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498284244 0 0) 0x7effbc0b1d90 con 0x7effbc06d8b0 2024-04-24T06:29:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.649+0000 7effc2000640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 0x7effbc0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:44.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.650+0000 7effc1600640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d4d0 0x7effbc09b5a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7effb0009d90 tx=0x7effb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aef17bd8259b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.650+0000 7effbbe00640 1 -- 172.21.15.59:0/2384526672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0007740 con 0x7effbc06d4d0 2024-04-24T06:29:44.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.650+0000 7effbbe00640 1 -- 172.21.15.59:0/2384526672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb00078e0 con 0x7effbc06d4d0 2024-04-24T06:29:44.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.650+0000 7effbbe00640 1 -- 172.21.15.59:0/2384526672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0007cb0 con 0x7effbc06d4d0 2024-04-24T06:29:44.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 -- 172.21.15.59:0/2384526672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d4d0 msgr2=0x7effbc09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d4d0 0x7effbc09b5a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7effb0009d90 tx=0x7effb002a660 comp rx=0 tx=0).stop 2024-04-24T06:29:44.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 -- 172.21.15.59:0/2384526672 shutdown_connections 2024-04-24T06:29:44.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc070020 0x7effbc0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 --2- 172.21.15.59:0/2384526672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d4d0 0x7effbc09b5a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 -- 172.21.15.59:0/2384526672 >> 172.21.15.59:0/2384526672 conn(0x7effbc0b4050 msgr2=0x7effbc0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 -- 172.21.15.59:0/2384526672 shutdown_connections 2024-04-24T06:29:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.651+0000 7effc4000640 1 -- 172.21.15.59:0/2384526672 wait complete. 2024-04-24T06:29:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.653+0000 7effc4000640 1 Processor -- start 2024-04-24T06:29:44.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.653+0000 7effc4000640 1 -- start start 2024-04-24T06:29:44.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 0x7effbc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 0x7effbc14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d19f0 con 0x7effbc06d8b0 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d1710 con 0x7effbc070020 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc4000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7effbc06dd30 con 0x7effbc06d4d0 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48384/0 (socket says 172.21.15.59:48384) 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc0c00640 1 -- 172.21.15.59:0/2174100321 learned_addr learned my addr 172.21.15.59:0/2174100321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc1600640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 0x7effbc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.654+0000 7effc2000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 0x7effbc14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 -- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 msgr2=0x7effbc15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 0x7effbc15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 -- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 msgr2=0x7effbc14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 0x7effbc14f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 -- 172.21.15.59:0/2174100321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1f8430 con 0x7effbc06d8b0 2024-04-24T06:29:44.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc1600640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 0x7effbc15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc2000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 0x7effbc14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc0c00640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effa800c460 tx=0x7effa800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8018070 con 0x7effbc06d8b0 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa8007500 con 0x7effbc06d8b0 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8004a20 con 0x7effbc06d8b0 2024-04-24T06:29:44.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1f8710 con 0x7effbc06d8b0 2024-04-24T06:29:44.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.655+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc1f8cd0 con 0x7effbc06d8b0 2024-04-24T06:29:44.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.656+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc069190 con 0x7effbc06d8b0 2024-04-24T06:29:44.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.657+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effa801b070 con 0x7effbc06d8b0 2024-04-24T06:29:44.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.658+0000 7effb9600640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:44.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.658+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7effa808af00 con 0x7effbc06d8b0 2024-04-24T06:29:44.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.659+0000 7effc1600640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:44.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.659+0000 7effc1600640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7effb0034000 tx=0x7effb0033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:44.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.663+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effa8059210 con 0x7effbc06d8b0 2024-04-24T06:29:44.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.886+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7effbc0cea40 con 0x7effbc06d8b0 2024-04-24T06:29:44.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.887+0000 7effb9600640 1 -- 172.21.15.59:0/2174100321 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7effa8058b10 con 0x7effbc06d8b0 2024-04-24T06:29:44.888 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:44.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 msgr2=0x7eff9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 0x7eff9c06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7effb0034000 tx=0x7effb0033040 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 msgr2=0x7effbc09c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effa800c460 tx=0x7effa800c920 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 shutdown_connections 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7eff9c06c8f0 0x7eff9c06edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effbc070020 0x7effbc14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effbc06d8b0 0x7effbc09c370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 --2- 172.21.15.59:0/2174100321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7effbc06d4d0 0x7effbc15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.889+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 >> 172.21.15.59:0/2174100321 conn(0x7effbc0b4050 msgr2=0x7effbc0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.890+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 shutdown_connections 2024-04-24T06:29:44.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:44.890+0000 7effc4000640 1 -- 172.21.15.59:0/2174100321 wait complete. 2024-04-24T06:29:44.912 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:29:44.912 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:29:44.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:45.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.184+0000 7f3984e00640 1 Processor -- start 2024-04-24T06:29:45.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.185+0000 7f3984e00640 1 -- start start 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f3984e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39800cd760 0x7f39800b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f3984e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f3984e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3980160210 con 0x7f398015c3d0 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f3984e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f39801604f0 con 0x7f39800cd760 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f3984e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f39801607d0 con 0x7f39800b4690 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f397ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39800cd760 0x7f39800b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f397ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39800cd760 0x7f39800b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48400/0 (socket says 172.21.15.59:48400) 2024-04-24T06:29:45.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f397ea00640 1 -- 172.21.15.59:0/2484063905 learned_addr learned my addr 172.21.15.59:0/2484063905 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:45.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f397fe00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.186+0000 7f397ea00640 1 -- 172.21.15.59:0/2484063905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 msgr2=0x7f39800cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:45.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.187+0000 7f397ea00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.187+0000 7f397ea00640 1 -- 172.21.15.59:0/2484063905 >> v1:172.21.15.66:6789/0 conn(0x7f398015c3d0 legacy=0x7f39800cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:45.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.187+0000 7f397ea00640 1 -- 172.21.15.59:0/2484063905 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3980160ab0 con 0x7f39800cd760 2024-04-24T06:29:45.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.187+0000 7f397fe00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:45.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.188+0000 7f397ea00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f39800b4150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2a61d7a8539a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.188+0000 7f397e000640 1 -- 172.21.15.59:0/2484063905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f39800cd760 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.188+0000 7f397e000640 1 -- 172.21.15.59:0/2484063905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c032440 con 0x7f39800cd760 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.188+0000 7f397e000640 1 -- 172.21.15.59:0/2484063905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c032810 con 0x7f39800cd760 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 -- 172.21.15.59:0/2484063905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 msgr2=0x7f39800b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f39800b4150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a210 comp rx=0 tx=0).stop 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 -- 172.21.15.59:0/2484063905 shutdown_connections 2024-04-24T06:29:45.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 --2- 172.21.15.59:0/2484063905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f39800b4150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 -- 172.21.15.59:0/2484063905 >> 172.21.15.59:0/2484063905 conn(0x7f39800b21a0 msgr2=0x7f39800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 -- 172.21.15.59:0/2484063905 shutdown_connections 2024-04-24T06:29:45.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.189+0000 7f3984e00640 1 -- 172.21.15.59:0/2484063905 wait complete. 2024-04-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.190+0000 7f3984e00640 1 Processor -- start 2024-04-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.191+0000 7f3984e00640 1 -- start start 2024-04-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 0x7f398015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f398015c7b0 con 0x7f39800cd760 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f39800b4b00 con 0x7f398015c3d0 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f3984e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f398015c930 con 0x7f39800b4690 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48410/0 (socket says 172.21.15.59:48410) 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 -- 172.21.15.59:0/2080682935 learned_addr learned my addr 172.21.15.59:0/2080682935 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 -- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 msgr2=0x7f39800c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397f400640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397fe00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 0x7f398015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 -- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 msgr2=0x7f398015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 0x7f398015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.192+0000 7f397ea00640 1 -- 172.21.15.59:0/2080682935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3980158e80 con 0x7f39800cd760 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f397fe00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 0x7f398015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f397ea00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037040 con 0x7f39800cd760 2024-04-24T06:29:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c042070 con 0x7f39800cd760 2024-04-24T06:29:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c030bb0 con 0x7f39800cd760 2024-04-24T06:29:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.193+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3980159160 con 0x7f39800cd760 2024-04-24T06:29:45.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.194+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39800cc770 con 0x7f39800cd760 2024-04-24T06:29:45.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.195+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39800c7e70 con 0x7f39800cd760 2024-04-24T06:29:45.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.196+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f396c047460 con 0x7f39800cd760 2024-04-24T06:29:45.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.197+0000 7f3977400640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.197+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f396c0ac910 con 0x7f39800cd760 2024-04-24T06:29:45.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.197+0000 7f397f400640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.198+0000 7f397f400640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3970006fd0 tx=0x7f3970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.202+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f396c07ac20 con 0x7f39800cd760 2024-04-24T06:29:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.407+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f398004fd90 con 0x7f39800cd760 2024-04-24T06:29:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.410+0000 7f3977400640 1 -- 172.21.15.59:0/2080682935 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f396c07a520 con 0x7f39800cd760 2024-04-24T06:29:45.414 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:45.415 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.412+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 msgr2=0x7f396006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.412+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3970006fd0 tx=0x7f3970008040 comp rx=0 tx=0).stop 2024-04-24T06:29:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.412+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 msgr2=0x7f3980158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.412+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 shutdown_connections 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f398015c3d0 0x7f398015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39800cd760 0x7f3980158440 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 --2- 172.21.15.59:0/2080682935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f39800b4690 0x7f39800c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 >> 172.21.15.59:0/2080682935 conn(0x7f39800b21a0 msgr2=0x7f39800afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 shutdown_connections 2024-04-24T06:29:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.413+0000 7f3984e00640 1 -- 172.21.15.59:0/2080682935 wait complete. 2024-04-24T06:29:45.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:45.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.639+0000 7f6016000640 1 Processor -- start 2024-04-24T06:29:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.639+0000 7f6016000640 1 -- start start 2024-04-24T06:29:45.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.640+0000 7f6016000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f601015be10 0x7f60100b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f6016000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60100b4700 con 0x7f6010072480 2024-04-24T06:29:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f6016000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60100b49e0 con 0x7f60100b4260 2024-04-24T06:29:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f6016000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f60100b4cc0 con 0x7f601015be10 2024-04-24T06:29:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f600fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6010072480 0x7f6010151670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53074/0 (socket says 172.21.15.59:53074) 2024-04-24T06:29:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f600fe00640 1 -- 172.21.15.59:0/1684445031 learned_addr learned my addr 172.21.15.59:0/1684445031 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f600f400640 1 --2- 172.21.15.59:0/1684445031 >> v2:172.21.15.66:3300/0 conn(0x7f601015be10 0x7f60100b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.641+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725235367 0 0) 0x7f60100b4700 con 0x7f6010072480 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff4005730 con 0x7f6010072480 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114720768 0 0) 0x7f5ff4005730 con 0x7f6010072480 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 >> v2:172.21.15.66:3300/0 conn(0x7f601015be10 msgr2=0x7f60100b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 --2- 172.21.15.59:0/1684445031 >> v2:172.21.15.66:3300/0 conn(0x7f601015be10 0x7f60100b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 >> v1:172.21.15.66:6789/0 conn(0x7f60100b4260 legacy=0x7f6010071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60100b4fa0 con 0x7f6010072480 2024-04-24T06:29:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f60080019f0 con 0x7f6010072480 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 >> v1:172.21.15.59:6789/0 conn(0x7f6010072480 legacy=0x7f6010151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff4006a10 0x7f5ff4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff40073d0 0x7f5ff400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60100b4cc0 con 0x7f5ff4005c10 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff400e2d0 con 0x7f5ff40073d0 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff400e4d0 con 0x7f5ff4006a10 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.642+0000 7f600f400640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600fe00640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff40073d0 0x7f5ff400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600f400640 1 -- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff4006a10 msgr2=0x7f5ff4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600f400640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff4006a10 0x7f5ff4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600f400640 1 -- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff40073d0 msgr2=0x7f5ff400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600f400640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff40073d0 0x7f5ff400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600f400640 1 -- 172.21.15.59:0/1684445031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff4005730 con 0x7f5ff4005c10 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600fe00640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff40073d0 0x7f5ff400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.643+0000 7f600fe00640 1 -- 172.21.15.59:0/1684445031 reap_dead start 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.644+0000 7f600f400640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6000009ac0 tx=0x7f5ff4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126385c4be38ce76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.644+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000036070 con 0x7f5ff4005c10 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.644+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6000031400 con 0x7f5ff4005c10 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.644+0000 7f600ea00640 1 -- 172.21.15.59:0/1684445031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000009690 con 0x7f5ff4005c10 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 -- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 msgr2=0x7f5ff4006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6000009ac0 tx=0x7f5ff4007a40 comp rx=0 tx=0).stop 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 -- 172.21.15.59:0/1684445031 shutdown_connections 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 --2- 172.21.15.59:0/1684445031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 -- 172.21.15.59:0/1684445031 >> 172.21.15.59:0/1684445031 conn(0x7f6010067d90 msgr2=0x7f601006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 -- 172.21.15.59:0/1684445031 shutdown_connections 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.645+0000 7f6016000640 1 -- 172.21.15.59:0/1684445031 wait complete. 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.647+0000 7f6016000640 1 Processor -- start 2024-04-24T06:29:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.647+0000 7f6016000640 1 -- start start 2024-04-24T06:29:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6010157a80 0x7f60100b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60100b21b0 0x7f60100cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f601015c370 con 0x7f5ff4005c10 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f601015c1f0 con 0x7f6010157a80 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f6016000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f601015b550 con 0x7f60100b21b0 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f600f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f600f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48426/0 (socket says 172.21.15.59:48426) 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.648+0000 7f600f400640 1 -- 172.21.15.59:0/1696733762 learned_addr learned my addr 172.21.15.59:0/1696733762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 -- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60100b21b0 msgr2=0x7f60100cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60100b21b0 0x7f60100cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 -- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6010157a80 msgr2=0x7f60100b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6010157a80 0x7f60100b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 -- 172.21.15.59:0/1696733762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6010157650 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f600f400640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6000034040 tx=0x7f6000004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f020 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6000037040 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000036070 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6010155e30 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.649+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101563f0 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.650+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6010068a80 con 0x7f5ff4005c10 2024-04-24T06:29:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.651+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6000042020 con 0x7f5ff4005c10 2024-04-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.652+0000 7f6007400640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.652+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f60000abd70 con 0x7f5ff4005c10 2024-04-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.653+0000 7f6007e00640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.653+0000 7f6007e00640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60100cea90 tx=0x7f5ff8007990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:45.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.657+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6000049030 con 0x7f5ff4005c10 2024-04-24T06:29:45.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.858+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60100b25d0 con 0x7f5ff4005c10 2024-04-24T06:29:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.859+0000 7f6007400640 1 -- 172.21.15.59:0/1696733762 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f600007a0b0 con 0x7f5ff4005c10 2024-04-24T06:29:45.861 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 msgr2=0x7f5fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60100cea90 tx=0x7f5ff8007990 comp rx=0 tx=0).stop 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 msgr2=0x7f601014acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6000034040 tx=0x7f6000004730 comp rx=0 tx=0).stop 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 shutdown_connections 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60100b21b0 0x7f60100cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6010157a80 0x7f60100b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 --2- 172.21.15.59:0/1696733762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f601014acc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 >> 172.21.15.59:0/1696733762 conn(0x7f6010067d90 msgr2=0x7f6010159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 shutdown_connections 2024-04-24T06:29:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:45.862+0000 7f6016000640 1 -- 172.21.15.59:0/1696733762 wait complete. 2024-04-24T06:29:45.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:46.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.086+0000 7fe246800640 1 Processor -- start 2024-04-24T06:29:46.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.087+0000 7fe246800640 1 -- start start 2024-04-24T06:29:46.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.088+0000 7fe246800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2400d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.088+0000 7fe246800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400b5da0 con 0x7fe2400b3600 2024-04-24T06:29:46.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.088+0000 7fe246800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400d0a20 con 0x7fe2400d13e0 2024-04-24T06:29:46.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.088+0000 7fe246800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe24015f100 con 0x7fe24015ed20 2024-04-24T06:29:46.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.089+0000 7fe244a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe2400b3600 0x7fe2400b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53086/0 (socket says 172.21.15.59:53086) 2024-04-24T06:29:46.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.089+0000 7fe244a00640 1 -- 172.21.15.59:0/4194912804 learned_addr learned my addr 172.21.15.59:0/4194912804 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.089+0000 7fe245400640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2400d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.089+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480728806 0 0) 0x7fe2400b5da0 con 0x7fe2400b3600 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.089+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe228003520 con 0x7fe2400b3600 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 613323151 0 0) 0x7fe228003520 con 0x7fe2400b3600 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 msgr2=0x7fe2400d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2400d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 >> v1:172.21.15.66:6789/0 conn(0x7fe2400d13e0 legacy=0x7fe2400b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400d2ed0 con 0x7fe2400b3600 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe22c001670 con 0x7fe2400b3600 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 >> v1:172.21.15.59:6789/0 conn(0x7fe2400b3600 legacy=0x7fe2400b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe228003fb0 0x7fe228004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228004d40 0x7fe2280051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe22800bc20 con 0x7fe228003fb0 2024-04-24T06:29:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe22800bdd0 con 0x7fe228004d40 2024-04-24T06:29:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe22800bfd0 con 0x7fe24015ed20 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe244a00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe228003fb0 0x7fe228004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe245400640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.090+0000 7fe245e00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228004d40 0x7fe2280051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe244a00640 1 -- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 msgr2=0x7fe2280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe244a00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe24015ed20 0x7fe2280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe244a00640 1 -- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228004d40 msgr2=0x7fe2280051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe244a00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228004d40 0x7fe2280051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe244a00640 1 -- 172.21.15.59:0/4194912804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe228003520 con 0x7fe228003fb0 2024-04-24T06:29:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.091+0000 7fe245e00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228004d40 0x7fe2280051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.092+0000 7fe244a00640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe228003fb0 0x7fe228004800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c190 tx=0x7fe228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6cb2d700f31a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.092+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c007c60 con 0x7fe228003fb0 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.092+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c005ad0 con 0x7fe228003fb0 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.092+0000 7fe23fe00640 1 -- 172.21.15.59:0/4194912804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c007070 con 0x7fe228003fb0 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 -- 172.21.15.59:0/4194912804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe228003fb0 msgr2=0x7fe228004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 --2- 172.21.15.59:0/4194912804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe228003fb0 0x7fe228004800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c190 tx=0x7fe228005830 comp rx=0 tx=0).stop 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe245e00640 1 -- 172.21.15.59:0/4194912804 reap_dead start 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 -- 172.21.15.59:0/4194912804 shutdown_connections 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 -- 172.21.15.59:0/4194912804 >> 172.21.15.59:0/4194912804 conn(0x7fe24008bf90 msgr2=0x7fe2400d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 -- 172.21.15.59:0/4194912804 shutdown_connections 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.093+0000 7fe246800640 1 -- 172.21.15.59:0/4194912804 wait complete. 2024-04-24T06:29:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.095+0000 7fe246800640 1 Processor -- start 2024-04-24T06:29:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.095+0000 7fe246800640 1 -- start start 2024-04-24T06:29:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228003fb0 0x7fe2400b9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2400b23e0 0x7fe240152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d0ea0 con 0x7fe2400b7b80 2024-04-24T06:29:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d3490 con 0x7fe228003fb0 2024-04-24T06:29:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe246800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400d1020 con 0x7fe2400b23e0 2024-04-24T06:29:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48454/0 (socket says 172.21.15.59:48454) 2024-04-24T06:29:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 -- 172.21.15.59:0/91581229 learned_addr learned my addr 172.21.15.59:0/91581229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245400640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228003fb0 0x7fe2400b9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 -- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2400b23e0 msgr2=0x7fe240152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2400b23e0 0x7fe240152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 -- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228003fb0 msgr2=0x7fe2400b9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228003fb0 0x7fe2400b9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.096+0000 7fe245e00640 1 -- 172.21.15.59:0/91581229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400b2860 con 0x7fe2400b7b80 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe245e00640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe24008dad0 tx=0x7fe23800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238006d50 con 0x7fe2400b7b80 2024-04-24T06:29:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238003970 con 0x7fe2400b7b80 2024-04-24T06:29:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238003da0 con 0x7fe2400b7b80 2024-04-24T06:29:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2400b73a0 con 0x7fe2400b7b80 2024-04-24T06:29:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.097+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2400c8ca0 con 0x7fe2400b7b80 2024-04-24T06:29:46.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.098+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24004fd90 con 0x7fe2400b7b80 2024-04-24T06:29:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.099+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe23801b020 con 0x7fe2400b7b80 2024-04-24T06:29:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.101+0000 7fe23d600640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 0x7fe22006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.101+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe23808b8e0 con 0x7fe2400b7b80 2024-04-24T06:29:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.101+0000 7fe245400640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 0x7fe22006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.102+0000 7fe245400640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 0x7fe22006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe234009950 tx=0x7fe234008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.105+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2400b2860 con 0x7fe2400b7b80 2024-04-24T06:29:46.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.312+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2400d5320 con 0x7fe2400b7b80 2024-04-24T06:29:46.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.313+0000 7fe23d600640 1 -- 172.21.15.59:0/91581229 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe238059bf0 con 0x7fe2400b7b80 2024-04-24T06:29:46.315 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 msgr2=0x7fe22006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 0x7fe22006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe234009950 tx=0x7fe234008040 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 msgr2=0x7fe2400b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe24008dad0 tx=0x7fe23800dc00 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 shutdown_connections 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe22006c8f0 0x7fe22006edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2400b7b80 0x7fe2400b8000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2400b23e0 0x7fe240152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 --2- 172.21.15.59:0/91581229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe228003fb0 0x7fe2400b9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 >> 172.21.15.59:0/91581229 conn(0x7fe24008bf90 msgr2=0x7fe2400b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 shutdown_connections 2024-04-24T06:29:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.316+0000 7fe246800640 1 -- 172.21.15.59:0/91581229 wait complete. 2024-04-24T06:29:46.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:46.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.547+0000 7f920d400640 1 Processor -- start 2024-04-24T06:29:46.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.547+0000 7f920d400640 1 -- start start 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f920d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9208145e10 0x7f920806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f920d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 0x7f92080cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f920d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92080cfc20 con 0x7f9208071d70 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f920d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f92080cff00 con 0x7f9208145e10 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f920d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92080d01e0 con 0x7f92080d0cc0 2024-04-24T06:29:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f9207200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9208071d70 0x7f920815cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53092/0 (socket says 172.21.15.59:53092) 2024-04-24T06:29:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.549+0000 7f9207200640 1 -- 172.21.15.59:0/1548105600 learned_addr learned my addr 172.21.15.59:0/1548105600 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9206800640 1 --2- 172.21.15.59:0/1548105600 >> v2:172.21.15.66:3300/0 conn(0x7f9208145e10 0x7f920806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f91ffe00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 0x7f92080cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246038888 0 0) 0x7f92080cfc20 con 0x7f9208071d70 2024-04-24T06:29:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91ec003520 con 0x7f9208071d70 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 501561117 0 0) 0x7f91ec003520 con 0x7f9208071d70 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 msgr2=0x7f92080cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 0x7f92080cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 >> v2:172.21.15.66:3300/0 conn(0x7f9208145e10 msgr2=0x7f920806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 --2- 172.21.15.59:0/1548105600 >> v2:172.21.15.66:3300/0 conn(0x7f9208145e10 0x7f920806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92080d04c0 con 0x7f9208071d70 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f92000019f0 con 0x7f9208071d70 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 >> v1:172.21.15.59:6789/0 conn(0x7f9208071d70 legacy=0x7f920815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ec0039b0 0x7f91ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 0x7f91ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92080cff00 con 0x7f91ec0039b0 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92080d01e0 con 0x7f91ec004800 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.550+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec00b9c0 con 0x7f92080d0cc0 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ec0039b0 0x7f91ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 -- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 msgr2=0x7f91ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080d0cc0 0x7f91ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f91ffe00640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 -- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec004800 msgr2=0x7f91ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.551+0000 7f9206800640 1 -- 172.21.15.59:0/1548105600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec003520 con 0x7f91ec0039b0 2024-04-24T06:29:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.552+0000 7f9206800640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ec0039b0 0x7f91ec003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ac0 tx=0x7f91ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5527aa15e981ca4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.552+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8036070 con 0x7f91ec0039b0 2024-04-24T06:29:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.552+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8031400 con 0x7f91ec0039b0 2024-04-24T06:29:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.552+0000 7f9205e00640 1 -- 172.21.15.59:0/1548105600 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8009690 con 0x7f91ec0039b0 2024-04-24T06:29:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 -- 172.21.15.59:0/1548105600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ec0039b0 msgr2=0x7f91ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 --2- 172.21.15.59:0/1548105600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91ec0039b0 0x7f91ec003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ac0 tx=0x7f91ec005830 comp rx=0 tx=0).stop 2024-04-24T06:29:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f9207200640 1 -- 172.21.15.59:0/1548105600 reap_dead start 2024-04-24T06:29:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 -- 172.21.15.59:0/1548105600 shutdown_connections 2024-04-24T06:29:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 -- 172.21.15.59:0/1548105600 >> 172.21.15.59:0/1548105600 conn(0x7f9208067d90 msgr2=0x7f920806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 -- 172.21.15.59:0/1548105600 shutdown_connections 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.553+0000 7f920d400640 1 -- 172.21.15.59:0/1548105600 wait complete. 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.555+0000 7f920d400640 1 Processor -- start 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.555+0000 7f920d400640 1 -- start start 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec0039b0 0x7f920815bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080caf30 0x7f9208141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92081461f0 con 0x7f920815c1b0 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9208072200 con 0x7f91ec0039b0 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.556+0000 7f920d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f920813e770 con 0x7f92080caf30 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48472/0 (socket says 172.21.15.59:48472) 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 -- 172.21.15.59:0/1136510799 learned_addr learned my addr 172.21.15.59:0/1136510799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:46.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f9207200640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080caf30 0x7f9208141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f9206800640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec0039b0 0x7f920815bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 -- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080caf30 msgr2=0x7f9208141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080caf30 0x7f9208141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 -- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec0039b0 msgr2=0x7f920815bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec0039b0 0x7f920815bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.557+0000 7f91ffe00640 1 -- 172.21.15.59:0/1136510799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920813ea50 con 0x7f920815c1b0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f91ffe00640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91f0010890 tx=0x7f91f0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f000f040 con 0x7f920815c1b0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f000eb60 con 0x7f920815c1b0 2024-04-24T06:29:46.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f001c400 con 0x7f920815c1b0 2024-04-24T06:29:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920813ed30 con 0x7f920815c1b0 2024-04-24T06:29:46.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.558+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920813f2f0 con 0x7f920815c1b0 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.559+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92080cdec0 con 0x7f920815c1b0 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.560+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91f001c5a0 con 0x7f920815c1b0 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.562+0000 7f91fea00640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 0x7f91e406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.562+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f91f008c1f0 con 0x7f920815c1b0 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.562+0000 7f9206800640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 0x7f91e406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:46.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.563+0000 7f9206800640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 0x7f91e406ee30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91f8033000 tx=0x7f91f8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:46.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.566+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92080cdec0 con 0x7f920815c1b0 2024-04-24T06:29:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.774+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9208068a80 con 0x7f920815c1b0 2024-04-24T06:29:46.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.775+0000 7f91fea00640 1 -- 172.21.15.59:0/1136510799 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f91f0018030 con 0x7f920815c1b0 2024-04-24T06:29:46.777 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:46.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.777+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 msgr2=0x7f91e406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.777+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 0x7f91e406ee30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91f8033000 tx=0x7f91f8008030 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.777+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 msgr2=0x7f92080ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.777+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91f0010890 tx=0x7f91f0010d50 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.777+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 shutdown_connections 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e406c970 0x7f91e406ee30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92080caf30 0x7f9208141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920815c1b0 0x7f92080ca9f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 --2- 172.21.15.59:0/1136510799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91ec0039b0 0x7f920815bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 >> 172.21.15.59:0/1136510799 conn(0x7f9208067d90 msgr2=0x7f920806e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:46.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 shutdown_connections 2024-04-24T06:29:46.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:46.778+0000 7f920d400640 1 -- 172.21.15.59:0/1136510799 wait complete. 2024-04-24T06:29:46.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:29:47.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.008+0000 7f3316a00640 1 Processor -- start 2024-04-24T06:29:47.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.008+0000 7f3316a00640 1 -- start start 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.009+0000 7f3316a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33100d0f80 0x7f33100cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.009+0000 7f3316a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f331015f880 0x7f33100d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.009+0000 7f3316a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f331015fd30 con 0x7f331015ea40 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3316a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f33100cf930 con 0x7f33100d0f80 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3316a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f33100cfc10 con 0x7f331015f880 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33100d0f80 0x7f33100cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33100d0f80 0x7f33100cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48484/0 (socket says 172.21.15.59:48484) 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 -- 172.21.15.59:0/3823557051 learned_addr learned my addr 172.21.15.59:0/3823557051 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 -- 172.21.15.59:0/3823557051 >> v2:172.21.15.66:3300/0 conn(0x7f331015f880 msgr2=0x7f33100d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 --2- 172.21.15.59:0/3823557051 >> v2:172.21.15.66:3300/0 conn(0x7f331015f880 0x7f33100d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 -- 172.21.15.59:0/3823557051 >> v1:172.21.15.66:6789/0 conn(0x7f331015ea40 legacy=0x7f33100d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.010+0000 7f3315600640 1 -- 172.21.15.59:0/3823557051 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33100cfef0 con 0x7f33100d0f80 2024-04-24T06:29:47.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.011+0000 7f3315600640 1 --2- 172.21.15.59:0/3823557051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100d0f80 0x7f33100cf250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33100ceb70 tx=0x7f330402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d0edd03818d4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.012+0000 7f330fe00640 1 -- 172.21.15.59:0/3823557051 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304037070 con 0x7f33100d0f80 2024-04-24T06:29:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.012+0000 7f330fe00640 1 -- 172.21.15.59:0/3823557051 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3304032440 con 0x7f33100d0f80 2024-04-24T06:29:47.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.012+0000 7f330fe00640 1 -- 172.21.15.59:0/3823557051 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304032810 con 0x7f33100d0f80 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.012+0000 7f3316a00640 1 -- 172.21.15.59:0/3823557051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100d0f80 msgr2=0x7f33100cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.012+0000 7f3316a00640 1 --2- 172.21.15.59:0/3823557051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100d0f80 0x7f33100cf250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33100ceb70 tx=0x7f330402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 -- 172.21.15.59:0/3823557051 shutdown_connections 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 --2- 172.21.15.59:0/3823557051 >> v2:172.21.15.66:3300/0 conn(0x7f331015f880 0x7f33100d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 --2- 172.21.15.59:0/3823557051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100d0f80 0x7f33100cf250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 -- 172.21.15.59:0/3823557051 >> 172.21.15.59:0/3823557051 conn(0x7f331008bf90 msgr2=0x7f33100d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 -- 172.21.15.59:0/3823557051 shutdown_connections 2024-04-24T06:29:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.013+0000 7f3316a00640 1 -- 172.21.15.59:0/3823557051 wait complete. 2024-04-24T06:29:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.014+0000 7f3316a00640 1 Processor -- start 2024-04-24T06:29:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 -- start start 2024-04-24T06:29:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0f80 0x7f3310158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f331015ea40 0x7f33100c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33100d2850 con 0x7f331015f880 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33100d04b0 con 0x7f33100d0f80 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.015+0000 7f3316a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f33100d29d0 con 0x7f331015ea40 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48492/0 (socket says 172.21.15.59:48492) 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 -- 172.21.15.59:0/3069881142 learned_addr learned my addr 172.21.15.59:0/3069881142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3315600640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0f80 0x7f3310158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3314c00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f331015ea40 0x7f33100c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 -- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f331015ea40 msgr2=0x7f33100c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f331015ea40 0x7f33100c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 -- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0f80 msgr2=0x7f3310158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0f80 0x7f3310158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.016+0000 7f3316000640 1 -- 172.21.15.59:0/3069881142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33100b60a0 con 0x7f331015f880 2024-04-24T06:29:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f3316000640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f330800b410 tx=0x7f330800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308015020 con 0x7f331015f880 2024-04-24T06:29:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308011850 con 0x7f331015f880 2024-04-24T06:29:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308011c40 con 0x7f331015f880 2024-04-24T06:29:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33100b6380 con 0x7f331015f880 2024-04-24T06:29:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.017+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33100b7350 con 0x7f331015f880 2024-04-24T06:29:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.018+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33100b7550 con 0x7f331015f880 2024-04-24T06:29:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.020+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3308018020 con 0x7f331015f880 2024-04-24T06:29:47.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.021+0000 7f330d600640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.021+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f330808bb70 con 0x7f331015f880 2024-04-24T06:29:47.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.021+0000 7f3315600640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.022+0000 7f3315600640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 0x7f32f406ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3304034000 tx=0x7f330402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.025+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33100b7550 con 0x7f331015f880 2024-04-24T06:29:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.248+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f331004fd90 con 0x7f331015f880 2024-04-24T06:29:47.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.248+0000 7f330d600640 1 -- 172.21.15.59:0/3069881142 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f330801f020 con 0x7f331015f880 2024-04-24T06:29:47.251 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:29:47.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 msgr2=0x7f32f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 0x7f32f406ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3304034000 tx=0x7f330402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:47.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 msgr2=0x7f33100cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f330800b410 tx=0x7f330800b8d0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 shutdown_connections 2024-04-24T06:29:47.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015f880 0x7f33100cacd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f331015ea40 0x7f33100c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 --2- 172.21.15.59:0/3069881142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33100d0f80 0x7f3310158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 >> 172.21.15.59:0/3069881142 conn(0x7f331008bf90 msgr2=0x7f33100b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:47.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 shutdown_connections 2024-04-24T06:29:47.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.251+0000 7f3316a00640 1 -- 172.21.15.59:0/3069881142 wait complete. 2024-04-24T06:29:47.272 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2024-04-24T06:29:47.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:47.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-24T06:29:47.367 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:47.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:29:47.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:29:47.367 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:29:47.424 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:29:47.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:47.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.638+0000 7f1973e00640 1 Processor -- start 2024-04-24T06:29:47.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.638+0000 7f1973e00640 1 -- start start 2024-04-24T06:29:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.639+0000 7f1973e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f197415c3d0 0x7f19740cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.639+0000 7f1973e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1974160210 con 0x7f19740b4860 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.639+0000 7f1973e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19741604f0 con 0x7f19740cd590 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.639+0000 7f1973e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f19741607d0 con 0x7f197415c3d0 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1972a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f197415c3d0 0x7f19740cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1973400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f19740b4860 0x7f19740cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53100/0 (socket says 172.21.15.59:53100) 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1973400640 1 -- 172.21.15.59:0/952217053 learned_addr learned my addr 172.21.15.59:0/952217053 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1972a00640 1 -- 172.21.15.59:0/952217053 >> v1:172.21.15.66:6789/0 conn(0x7f19740cd590 legacy=0x7f19740b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1972a00640 1 -- 172.21.15.59:0/952217053 >> v1:172.21.15.59:6789/0 conn(0x7f19740b4860 legacy=0x7f19740cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.640+0000 7f1972a00640 1 -- 172.21.15.59:0/952217053 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1974160ab0 con 0x7f197415c3d0 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.641+0000 7f1972a00640 1 --2- 172.21.15.59:0/952217053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f197415c3d0 0x7f19740cd050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cacc9bc152ed997 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.641+0000 7f1972000640 1 -- 172.21.15.59:0/952217053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964037070 con 0x7f197415c3d0 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1972000640 1 -- 172.21.15.59:0/952217053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1964032440 con 0x7f197415c3d0 2024-04-24T06:29:47.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1972000640 1 -- 172.21.15.59:0/952217053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964032810 con 0x7f197415c3d0 2024-04-24T06:29:47.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 -- 172.21.15.59:0/952217053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f197415c3d0 msgr2=0x7f19740cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 --2- 172.21.15.59:0/952217053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f197415c3d0 0x7f19740cd050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 -- 172.21.15.59:0/952217053 shutdown_connections 2024-04-24T06:29:47.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 --2- 172.21.15.59:0/952217053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f197415c3d0 0x7f19740cd050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 -- 172.21.15.59:0/952217053 >> 172.21.15.59:0/952217053 conn(0x7f19740b21a0 msgr2=0x7f19740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:47.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 -- 172.21.15.59:0/952217053 shutdown_connections 2024-04-24T06:29:47.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.642+0000 7f1973e00640 1 -- 172.21.15.59:0/952217053 wait complete. 2024-04-24T06:29:47.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.644+0000 7f1973e00640 1 Processor -- start 2024-04-24T06:29:47.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.644+0000 7f1973e00640 1 -- start start 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740cd590 0x7f19740ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f197415c3d0 0x7f19740cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f197415c930 con 0x7f19740b4860 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f197415c7b0 con 0x7f19740cd590 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1973e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f19740cbee0 con 0x7f197415c3d0 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.645+0000 7f1972a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48506/0 (socket says 172.21.15.59:48506) 2024-04-24T06:29:47.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 -- 172.21.15.59:0/807592813 learned_addr learned my addr 172.21.15.59:0/807592813 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 -- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f197415c3d0 msgr2=0x7f19740cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1973400640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f197415c3d0 0x7f19740cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f196be00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740cd590 0x7f19740ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f197415c3d0 0x7f19740cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 -- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740cd590 msgr2=0x7f19740ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740cd590 0x7f19740ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 -- 172.21.15.59:0/807592813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740c9020 con 0x7f19740b4860 2024-04-24T06:29:47.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.646+0000 7f1972a00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f19640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.647+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964041070 con 0x7f19740b4860 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.647+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196403c930 con 0x7f19740b4860 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.647+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19740c9300 con 0x7f19740b4860 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.647+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964037040 con 0x7f19740b4860 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.647+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741586b0 con 0x7f19740b4860 2024-04-24T06:29:47.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.648+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197404fd90 con 0x7f19740b4860 2024-04-24T06:29:47.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.649+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1964044070 con 0x7f19740b4860 2024-04-24T06:29:47.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.650+0000 7f196aa00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:47.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.650+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f19640ac810 con 0x7f19740b4860 2024-04-24T06:29:47.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.650+0000 7f196be00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:47.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.651+0000 7f196be00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 0x7f195006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19741581c0 tx=0x7f195c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:47.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.654+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f196407ab20 con 0x7f19740b4860 2024-04-24T06:29:47.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.898+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f197406e400 con 0x7f19740b4860 2024-04-24T06:29:47.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.899+0000 7f196aa00640 1 -- 172.21.15.59:0/807592813 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f196407a420 con 0x7f19740b4860 2024-04-24T06:29:47.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:47.901 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:47.902 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.901+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 msgr2=0x7f195006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.901+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 0x7f195006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19741581c0 tx=0x7f195c009040 comp rx=0 tx=0).stop 2024-04-24T06:29:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.901+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 msgr2=0x7f19740c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.901+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f19640046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 shutdown_connections 2024-04-24T06:29:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f197415c3d0 0x7f19740cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19740cd590 0x7f19740ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 --2- 172.21.15.59:0/807592813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19740b4860 0x7f19740c5390 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 >> 172.21.15.59:0/807592813 conn(0x7f19740b21a0 msgr2=0x7f1974074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 shutdown_connections 2024-04-24T06:29:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:47.902+0000 7f1973e00640 1 -- 172.21.15.59:0/807592813 wait complete. 2024-04-24T06:29:47.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.125+0000 7f2147e00640 1 Processor -- start 2024-04-24T06:29:48.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.125+0000 7f2147e00640 1 -- start start 2024-04-24T06:29:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 0x7f21480ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214809aaa0 con 0x7f21480d0b80 2024-04-24T06:29:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214815ca50 con 0x7f214809bad0 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f214809b640 con 0x7f214815c670 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f21480d0b80 0x7f2148166550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53112/0 (socket says 172.21.15.59:53112) 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2147400640 1 -- 172.21.15.59:0/1401852596 learned_addr learned my addr 172.21.15.59:0/1401852596 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2146a00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 0x7f21480ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233664066 0 0) 0x7f214809aaa0 con 0x7f21480d0b80 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.127+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f212c003520 con 0x7f21480d0b80 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1190227815 0 0) 0x7f212c003520 con 0x7f21480d0b80 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 msgr2=0x7f21480ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 0x7f21480ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 >> v1:172.21.15.66:6789/0 conn(0x7f214809bad0 legacy=0x7f21480d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214809ad10 con 0x7f21480d0b80 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2140001670 con 0x7f21480d0b80 2024-04-24T06:29:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 >> v1:172.21.15.59:6789/0 conn(0x7f21480d0b80 legacy=0x7f2148166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f212c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 0x7f212c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f212c004d40 0x7f212c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f214809b640 con 0x7f212c0039e0 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f212c00b9c0 con 0x7f212c004d40 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f212c00bbc0 con 0x7f214815c670 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.128+0000 7f2146a00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f212c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.129+0000 7f2146a00640 1 -- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 msgr2=0x7f212c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.129+0000 7f2146a00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214815c670 0x7f212c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.129+0000 7f2146a00640 1 -- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f212c004d40 msgr2=0x7f212c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.129+0000 7f2146a00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f212c004d40 0x7f212c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.129+0000 7f2146a00640 1 -- 172.21.15.59:0/1401852596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c003520 con 0x7f212c0039e0 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.130+0000 7f2146a00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f212c003dc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2138009fc0 tx=0x7f212c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b61fcb2aea4197d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.130+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138037070 con 0x7f212c0039e0 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.130+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2138032620 con 0x7f212c0039e0 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.130+0000 7f2145600640 1 -- 172.21.15.59:0/1401852596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138009c10 con 0x7f212c0039e0 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 -- 172.21.15.59:0/1401852596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 msgr2=0x7f212c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 --2- 172.21.15.59:0/1401852596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f212c003dc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2138009fc0 tx=0x7f212c005830 comp rx=0 tx=0).stop 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147400640 1 -- 172.21.15.59:0/1401852596 reap_dead start 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 -- 172.21.15.59:0/1401852596 shutdown_connections 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 -- 172.21.15.59:0/1401852596 >> 172.21.15.59:0/1401852596 conn(0x7f21480b3ee0 msgr2=0x7f21480b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:48.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 -- 172.21.15.59:0/1401852596 shutdown_connections 2024-04-24T06:29:48.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.131+0000 7f2147e00640 1 -- 172.21.15.59:0/1401852596 wait complete. 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.133+0000 7f2147e00640 1 Processor -- start 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.133+0000 7f2147e00640 1 -- start start 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21480b5010 0x7f21480b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214806ce10 0x7f214806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21480d1190 con 0x7f212c0039e0 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f214809b2d0 con 0x7f21480b5010 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2147e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f21480d1310 con 0x7f214806ce10 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48534/0 (socket says 172.21.15.59:48534) 2024-04-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 -- 172.21.15.59:0/4159443505 learned_addr learned my addr 172.21.15.59:0/4159443505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 -- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214806ce10 msgr2=0x7f214806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214806ce10 0x7f214806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 -- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21480b5010 msgr2=0x7f21480b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21480b5010 0x7f21480b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.134+0000 7f2146a00640 1 -- 172.21.15.59:0/4159443505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214806e430 con 0x7f212c0039e0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f2146a00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f213802ae60 tx=0x7f2138009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138042030 con 0x7f212c0039e0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2138032420 con 0x7f212c0039e0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138037040 con 0x7f212c0039e0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214806db90 con 0x7f212c0039e0 2024-04-24T06:29:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.135+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214806e150 con 0x7f212c0039e0 2024-04-24T06:29:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.136+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214804fd90 con 0x7f212c0039e0 2024-04-24T06:29:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.137+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2138045070 con 0x7f212c0039e0 2024-04-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.138+0000 7f213ea00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 0x7f212406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.138+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f21380ac8b0 con 0x7f212c0039e0 2024-04-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.138+0000 7f2146000640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 0x7f212406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.139+0000 7f2146000640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 0x7f212406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f214815bb80 tx=0x7f2130007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.142+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f213807ace0 con 0x7f212c0039e0 2024-04-24T06:29:48.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.348+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21480b4b40 con 0x7f212c0039e0 2024-04-24T06:29:48.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.351+0000 7f213ea00640 1 -- 172.21.15.59:0/4159443505 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f213807a5e0 con 0x7f212c0039e0 2024-04-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 msgr2=0x7f212406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 0x7f212406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f214815bb80 tx=0x7f2130007930 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 msgr2=0x7f214814bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f213802ae60 tx=0x7f2138009750 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 shutdown_connections 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f212406c8f0 0x7f212406edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f214806ce10 0x7f214806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21480b5010 0x7f21480b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 --2- 172.21.15.59:0/4159443505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f212c0039e0 0x7f214814bc40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.354+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 >> 172.21.15.59:0/4159443505 conn(0x7f21480b3ee0 msgr2=0x7f21480af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.355+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 shutdown_connections 2024-04-24T06:29:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.355+0000 7f2147e00640 1 -- 172.21.15.59:0/4159443505 wait complete. 2024-04-24T06:29:48.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:28:42.577295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:28:43.152430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:29:48.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:29:48.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.586+0000 7f25dbe00640 1 Processor -- start 2024-04-24T06:29:48.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.586+0000 7f25dbe00640 1 -- start start 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 0x7f25dc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25dc15d810 0x7f25dc0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f25dc0d0f60 con 0x7f25dc15d810 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f25dc15cab0 con 0x7f25dc1567e0 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25dbe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc160e40 con 0x7f25dc0d0b40 2024-04-24T06:29:48.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25dc15d810 0x7f25dc0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25dc15d810 0x7f25dc0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48536/0 (socket says 172.21.15.59:48536) 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 -- 172.21.15.59:0/3727629441 learned_addr learned my addr 172.21.15.59:0/3727629441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 -- 172.21.15.59:0/3727629441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 msgr2=0x7f25dc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25daa00640 1 --2- 172.21.15.59:0/3727629441 >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 0x7f25dc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25db400640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 -- 172.21.15.59:0/3727629441 >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 msgr2=0x7f25dc156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 --2- 172.21.15.59:0/3727629441 >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 0x7f25dc156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25d3e00640 1 -- 172.21.15.59:0/3727629441 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc15cd30 con 0x7f25dc15d810 2024-04-24T06:29:48.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.588+0000 7f25daa00640 1 --2- 172.21.15.59:0/3727629441 >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 0x7f25dc156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25d3e00640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc15d810 0x7f25dc0d0600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0b6360 tx=0x7f25c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f64608eb0b5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25da000640 1 -- 172.21.15.59:0/3727629441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4037070 con 0x7f25dc15d810 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25da000640 1 -- 172.21.15.59:0/3727629441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c4032440 con 0x7f25dc15d810 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25da000640 1 -- 172.21.15.59:0/3727629441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4032810 con 0x7f25dc15d810 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 -- 172.21.15.59:0/3727629441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc15d810 msgr2=0x7f25dc0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc15d810 0x7f25dc0d0600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0b6360 tx=0x7f25c402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 -- 172.21.15.59:0/3727629441 shutdown_connections 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 --2- 172.21.15.59:0/3727629441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc15d810 0x7f25dc0d0600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 --2- 172.21.15.59:0/3727629441 >> v2:172.21.15.66:3300/0 conn(0x7f25dc1567e0 0x7f25dc156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 -- 172.21.15.59:0/3727629441 >> 172.21.15.59:0/3727629441 conn(0x7f25dc06bdf0 msgr2=0x7f25dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 -- 172.21.15.59:0/3727629441 shutdown_connections 2024-04-24T06:29:48.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.591+0000 7f25dbe00640 1 -- 172.21.15.59:0/3727629441 wait complete. 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.593+0000 7f25dbe00640 1 Processor -- start 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.593+0000 7f25dbe00640 1 -- start start 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 0x7f25dc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc160650 con 0x7f25dc1567e0 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc15d2f0 con 0x7f25dc15d810 2024-04-24T06:29:48.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.594+0000 7f25dbe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc1607d0 con 0x7f25dc0d0b40 2024-04-24T06:29:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48550/0 (socket says 172.21.15.59:48550) 2024-04-24T06:29:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 -- 172.21.15.59:0/1111105955 learned_addr learned my addr 172.21.15.59:0/1111105955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25daa00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25db400640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 0x7f25dc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 -- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 msgr2=0x7f25dc099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 -- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 msgr2=0x7f25dc06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 0x7f25dc06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 -- 172.21.15.59:0/1111105955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc14faa0 con 0x7f25dc1567e0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25db400640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 0x7f25dc06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.595+0000 7f25d3e00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000c00 tx=0x7f25c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.596+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4041020 con 0x7f25dc1567e0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.596+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c4030800 con 0x7f25dc1567e0 2024-04-24T06:29:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.596+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4037040 con 0x7f25dc1567e0 2024-04-24T06:29:48.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.596+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc14fd80 con 0x7f25dc1567e0 2024-04-24T06:29:48.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.596+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc150340 con 0x7f25dc1567e0 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.597+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc04fd90 con 0x7f25dc1567e0 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.598+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25c4044020 con 0x7f25dc1567e0 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.599+0000 7f25d2a00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.599+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f25c40ac560 con 0x7f25dc1567e0 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.599+0000 7f25daa00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.600+0000 7f25daa00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00a910 tx=0x7f25cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:48.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.603+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25c404b030 con 0x7f25dc1567e0 2024-04-24T06:29:48.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.812+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25dc0b5150 con 0x7f25dc1567e0 2024-04-24T06:29:48.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.812+0000 7f25d2a00640 1 -- 172.21.15.59:0/1111105955 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f25c407a870 con 0x7f25dc1567e0 2024-04-24T06:29:48.815 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 msgr2=0x7f25b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00a910 tx=0x7f25cc008040 comp rx=0 tx=0).stop 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 msgr2=0x7f25dc14f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000c00 tx=0x7f25c40046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 shutdown_connections 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25dc15d810 0x7f25dc06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25dc1567e0 0x7f25dc14f130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 --2- 172.21.15.59:0/1111105955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25dc0d0b40 0x7f25dc099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:48.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.815+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 >> 172.21.15.59:0/1111105955 conn(0x7f25dc06bdf0 msgr2=0x7f25dc071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:48.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.816+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 shutdown_connections 2024-04-24T06:29:48.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:48.816+0000 7f25dbe00640 1 -- 172.21.15.59:0/1111105955 wait complete. 2024-04-24T06:29:48.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:29:49.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.060+0000 7fedcb200640 1 Processor -- start 2024-04-24T06:29:49.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.060+0000 7fedcb200640 1 -- start start 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedcb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 0x7fedcc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedcb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fedcc0af2a0 0x7fedcc0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedcb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedcc15ef20 con 0x7fedcc0d0600 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedcb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fedcc0d0fa0 con 0x7fedcc0af2a0 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedcb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedcc15f1c0 con 0x7fedcc155a80 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc9400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fedcc0d0600 0x7fedcc0b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48838/0 (socket says 172.21.15.59:48838) 2024-04-24T06:29:49.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc9400640 1 -- 172.21.15.59:0/616681721 learned_addr learned my addr 172.21.15.59:0/616681721 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc9e00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 0x7fedcc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832479862 0 0) 0x7fedcc15ef20 con 0x7fedcc0d0600 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedb0003520 con 0x7fedcc0d0600 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedca800640 1 --2- 172.21.15.59:0/616681721 >> v2:172.21.15.66:3300/0 conn(0x7fedcc0af2a0 0x7fedcc0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3882260863 0 0) 0x7fedb0003520 con 0x7fedcc0d0600 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 msgr2=0x7fedcc06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 0x7fedcc06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 >> v2:172.21.15.66:3300/0 conn(0x7fedcc0af2a0 msgr2=0x7fedcc0d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 --2- 172.21.15.59:0/616681721 >> v2:172.21.15.66:3300/0 conn(0x7fedcc0af2a0 0x7fedcc0d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc15f4a0 con 0x7fedcc0d0600 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fedbc001670 con 0x7fedcc0d0600 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 >> v1:172.21.15.59:6789/0 conn(0x7fedcc0d0600 legacy=0x7fedcc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedb0004800 0x7fedb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 0x7fedb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc15f1c0 con 0x7fedb00039b0 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedb000b9c0 con 0x7fedb0004800 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedb000bb90 con 0x7fedcc155a80 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.062+0000 7fedc9e00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9400640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedb0004800 0x7fedb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9e00640 1 -- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 msgr2=0x7fedb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9e00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc155a80 0x7fedb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9e00640 1 -- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedb0004800 msgr2=0x7fedb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9e00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedb0004800 0x7fedb0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.063+0000 7fedc9e00640 1 -- 172.21.15.59:0/616681721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb0003520 con 0x7fedb00039b0 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.064+0000 7fedc9e00640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedb0003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009ac0 tx=0x7fedb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb38ce9cd377b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.064+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4036070 con 0x7fedb00039b0 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.064+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb4031400 con 0x7fedb00039b0 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.064+0000 7fedc8a00640 1 -- 172.21.15.59:0/616681721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4009690 con 0x7fedb00039b0 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 -- 172.21.15.59:0/616681721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 msgr2=0x7fedb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 --2- 172.21.15.59:0/616681721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedb0003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009ac0 tx=0x7fedb0005830 comp rx=0 tx=0).stop 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedca800640 1 -- 172.21.15.59:0/616681721 reap_dead start 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 -- 172.21.15.59:0/616681721 shutdown_connections 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 -- 172.21.15.59:0/616681721 >> 172.21.15.59:0/616681721 conn(0x7fedcc06bdf0 msgr2=0x7fedcc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 -- 172.21.15.59:0/616681721 shutdown_connections 2024-04-24T06:29:49.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.065+0000 7fedcb200640 1 -- 172.21.15.59:0/616681721 wait complete. 2024-04-24T06:29:49.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.068+0000 7fedcb200640 1 Processor -- start 2024-04-24T06:29:49.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.068+0000 7fedcb200640 1 -- start start 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 0x7fedcc15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc14f350 0x7fedcc1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc15fd40 con 0x7fedb00039b0 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc15fa60 con 0x7fedcc148b60 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedcb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedcc06ef80 con 0x7fedcc14f350 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51830/0 (socket says 172.21.15.59:51830) 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 -- 172.21.15.59:0/3623974633 learned_addr learned my addr 172.21.15.59:0/3623974633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9400640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 0x7fedcc15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 -- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc14f350 msgr2=0x7fedcc1f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedca800640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc14f350 0x7fedcc1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc14f350 0x7fedcc1f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 -- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 msgr2=0x7fedcc15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 0x7fedcc15cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 -- 172.21.15.59:0/3623974633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc14eb90 con 0x7fedb00039b0 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9400640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 0x7fedcc15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc9e00640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009ac0 tx=0x7fedb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.069+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4040020 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.070+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb4037040 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.070+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedcc1f89f0 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.070+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4036070 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.070+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedcc1f8fb0 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.072+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedcc04fd90 con 0x7fedb00039b0 2024-04-24T06:29:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.073+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedb4043020 con 0x7fedb00039b0 2024-04-24T06:29:49.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.073+0000 7fedc2000640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 0x7feda806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.074+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fedb40abfd0 con 0x7fedb00039b0 2024-04-24T06:29:49.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.074+0000 7fedc9400640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 0x7feda806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.074+0000 7fedc9400640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 0x7feda806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fedbc007970 tx=0x7fedbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.079+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedb404a030 con 0x7fedb00039b0 2024-04-24T06:29:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.289+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fedcc06eba0 con 0x7fedb00039b0 2024-04-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.290+0000 7fedc2000640 1 -- 172.21.15.59:0/3623974633 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fedb407a410 con 0x7fedb00039b0 2024-04-24T06:29:49.291 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 msgr2=0x7feda806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 0x7feda806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fedbc007970 tx=0x7fedbc009040 comp rx=0 tx=0).stop 2024-04-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 msgr2=0x7fedcc1f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009ac0 tx=0x7fedb4004730 comp rx=0 tx=0).stop 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 shutdown_connections 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feda806c920 0x7feda806ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedcc14f350 0x7fedcc1f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedcc148b60 0x7fedcc15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 --2- 172.21.15.59:0/3623974633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00039b0 0x7fedcc1f81a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 >> 172.21.15.59:0/3623974633 conn(0x7fedcc06bdf0 msgr2=0x7fedcc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 shutdown_connections 2024-04-24T06:29:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.293+0000 7fedcb200640 1 -- 172.21.15.59:0/3623974633 wait complete. 2024-04-24T06:29:49.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:29:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.532+0000 7f9e87e00640 1 Processor -- start 2024-04-24T06:29:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.533+0000 7f9e87e00640 1 -- start start 2024-04-24T06:29:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e88156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e8815d810 0x7f9e880d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9e880d0b40 0x7f9e88160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9e880d0f70 con 0x7f9e8815d810 2024-04-24T06:29:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8815cab0 con 0x7f9e880d0b40 2024-04-24T06:29:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.534+0000 7f9e87e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e88160e40 con 0x7f9e881567e0 2024-04-24T06:29:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e8815d810 0x7f9e880d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e8815d810 0x7f9e880d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51844/0 (socket says 172.21.15.59:51844) 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 -- 172.21.15.59:0/2970943342 learned_addr learned my addr 172.21.15.59:0/2970943342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e87400640 1 --2- 172.21.15.59:0/2970943342 >> v2:172.21.15.66:3300/0 conn(0x7f9e880d0b40 0x7f9e88160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e86a00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e88156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 -- 172.21.15.59:0/2970943342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 msgr2=0x7f9e88156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e88156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 -- 172.21.15.59:0/2970943342 >> v2:172.21.15.66:3300/0 conn(0x7f9e880d0b40 msgr2=0x7f9e88160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 --2- 172.21.15.59:0/2970943342 >> v2:172.21.15.66:3300/0 conn(0x7f9e880d0b40 0x7f9e88160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.535+0000 7f9e7fe00640 1 -- 172.21.15.59:0/2970943342 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8815cd30 con 0x7f9e8815d810 2024-04-24T06:29:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.536+0000 7f9e7fe00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e880d0600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e88142d50 tx=0x7f9e7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962312cf58c8949 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.537+0000 7f9e86000640 1 -- 172.21.15.59:0/2970943342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70037070 con 0x7f9e8815d810 2024-04-24T06:29:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.537+0000 7f9e86000640 1 -- 172.21.15.59:0/2970943342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e70032440 con 0x7f9e8815d810 2024-04-24T06:29:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.537+0000 7f9e86000640 1 -- 172.21.15.59:0/2970943342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70032810 con 0x7f9e8815d810 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.537+0000 7f9e87e00640 1 -- 172.21.15.59:0/2970943342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 msgr2=0x7f9e880d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.537+0000 7f9e87e00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e880d0600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e88142d50 tx=0x7f9e7002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 -- 172.21.15.59:0/2970943342 shutdown_connections 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 --2- 172.21.15.59:0/2970943342 >> v2:172.21.15.66:3300/0 conn(0x7f9e880d0b40 0x7f9e88160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e880d0600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 --2- 172.21.15.59:0/2970943342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e88156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 -- 172.21.15.59:0/2970943342 >> 172.21.15.59:0/2970943342 conn(0x7f9e8806bdf0 msgr2=0x7f9e8806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 -- 172.21.15.59:0/2970943342 shutdown_connections 2024-04-24T06:29:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.538+0000 7f9e87e00640 1 -- 172.21.15.59:0/2970943342 wait complete. 2024-04-24T06:29:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.540+0000 7f9e87e00640 1 Processor -- start 2024-04-24T06:29:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.540+0000 7f9e87e00640 1 -- start start 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 0x7f9e8809bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e8809dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88160650 con 0x7f9e8815d810 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8815d2f0 con 0x7f9e880d0b40 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e881607d0 con 0x7f9e881567e0 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51846/0 (socket says 172.21.15.59:51846) 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e87400640 1 -- 172.21.15.59:0/1859423496 learned_addr learned my addr 172.21.15.59:0/1859423496 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.541+0000 7f9e86a00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 0x7f9e8809bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 -- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 msgr2=0x7f9e8809dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e7fe00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e8809dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e8809dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 -- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 msgr2=0x7f9e8809bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 0x7f9e8809bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 -- 172.21.15.59:0/1859423496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8809c320 con 0x7f9e8815d810 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e86a00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 0x7f9e8809bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87400640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e80007e60 tx=0x7f9e8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80014420 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e80019070 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80014820 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.542+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e88143040 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.543+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881438a0 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.544+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e881433f0 con 0x7f9e8815d810 2024-04-24T06:29:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.545+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e800149c0 con 0x7f9e8815d810 2024-04-24T06:29:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.546+0000 7f9e7ea00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 0x7f9e6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.546+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9e8008c4a0 con 0x7f9e8815d810 2024-04-24T06:29:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.546+0000 7f9e86a00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 0x7f9e6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.547+0000 7f9e86a00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 0x7f9e6406ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:49.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.550+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e8005a7b0 con 0x7f9e8815d810 2024-04-24T06:29:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.770+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e88156bc0 con 0x7f9e8815d810 2024-04-24T06:29:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.770+0000 7f9e7ea00640 1 -- 172.21.15.59:0/1859423496 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f9e88156bc0 con 0x7f9e8815d810 2024-04-24T06:29:49.771 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 msgr2=0x7f9e6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 0x7f9e6406ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).stop 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 msgr2=0x7f9e8809a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e80007e60 tx=0x7f9e8000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 shutdown_connections 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e6406c920 0x7f9e6406ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e8815d810 0x7f9e8809a280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e881567e0 0x7f9e8809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 --2- 172.21.15.59:0/1859423496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e880d0b40 0x7f9e8809bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.773+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 >> 172.21.15.59:0/1859423496 conn(0x7f9e8806bdf0 msgr2=0x7f9e88071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.774+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 shutdown_connections 2024-04-24T06:29:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.774+0000 7f9e87e00640 1 -- 172.21.15.59:0/1859423496 wait complete. 2024-04-24T06:29:49.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.997+0000 7efe2be00640 1 Processor -- start 2024-04-24T06:29:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.997+0000 7efe2be00640 1 -- start start 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.998+0000 7efe2be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c15ea10 0x7efe2c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.998+0000 7efe2be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efe2c0b02f0 0x7efe2c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efe2c0b4230 0x7efe2c06bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efe2c0b5d00 con 0x7efe2c0b02f0 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efe2c0b2840 con 0x7efe2c0b4230 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efe2c0b2b00 con 0x7efe2c15ea10 2024-04-24T06:29:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efe2c0b02f0 0x7efe2c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efe2c0b02f0 0x7efe2c06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51862/0 (socket says 172.21.15.59:51862) 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2a000640 1 -- 172.21.15.59:0/2811062063 learned_addr learned my addr 172.21.15.59:0/2811062063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2aa00640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c15ea10 0x7efe2c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:49.999+0000 7efe2b400640 1 --2- 172.21.15.59:0/2811062063 >> v2:172.21.15.66:3300/0 conn(0x7efe2c0b4230 0x7efe2c06bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.000+0000 7efe2a000640 1 -- 172.21.15.59:0/2811062063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c15ea10 msgr2=0x7efe2c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.000+0000 7efe2a000640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c15ea10 0x7efe2c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.000+0000 7efe2a000640 1 -- 172.21.15.59:0/2811062063 >> v2:172.21.15.66:3300/0 conn(0x7efe2c0b4230 msgr2=0x7efe2c06bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.000+0000 7efe2a000640 1 --2- 172.21.15.59:0/2811062063 >> v2:172.21.15.66:3300/0 conn(0x7efe2c0b4230 0x7efe2c06bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.000+0000 7efe2a000640 1 -- 172.21.15.59:0/2811062063 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c0b5fa0 con 0x7efe2c0b02f0 2024-04-24T06:29:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.001+0000 7efe2a000640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0b02f0 0x7efe2c06b9d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0afd90 tx=0x7efe1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa2911a154152ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.002+0000 7efe29600640 1 -- 172.21.15.59:0/2811062063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14037070 con 0x7efe2c0b02f0 2024-04-24T06:29:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.002+0000 7efe29600640 1 -- 172.21.15.59:0/2811062063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe14032440 con 0x7efe2c0b02f0 2024-04-24T06:29:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.002+0000 7efe29600640 1 -- 172.21.15.59:0/2811062063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14032810 con 0x7efe2c0b02f0 2024-04-24T06:29:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 -- 172.21.15.59:0/2811062063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0b02f0 msgr2=0x7efe2c06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0b02f0 0x7efe2c06b9d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0afd90 tx=0x7efe1402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 -- 172.21.15.59:0/2811062063 shutdown_connections 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 --2- 172.21.15.59:0/2811062063 >> v2:172.21.15.66:3300/0 conn(0x7efe2c0b4230 0x7efe2c06bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0b02f0 0x7efe2c06b9d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 --2- 172.21.15.59:0/2811062063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c15ea10 0x7efe2c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 -- 172.21.15.59:0/2811062063 >> 172.21.15.59:0/2811062063 conn(0x7efe2c0b6750 msgr2=0x7efe2c0b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 -- 172.21.15.59:0/2811062063 shutdown_connections 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.003+0000 7efe2be00640 1 -- 172.21.15.59:0/2811062063 wait complete. 2024-04-24T06:29:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.005+0000 7efe2be00640 1 Processor -- start 2024-04-24T06:29:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.005+0000 7efe2be00640 1 -- start start 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe2c0b02f0 0x7efe2c1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 0x7efe2c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c0b1450 con 0x7efe2c0caf70 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c06c690 con 0x7efe2c0b02f0 2024-04-24T06:29:50.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.006+0000 7efe2be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efe2c0b15d0 con 0x7efe2c0b4230 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51874/0 (socket says 172.21.15.59:51874) 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 -- 172.21.15.59:0/574015026 learned_addr learned my addr 172.21.15.59:0/574015026 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2aa00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe2c0b02f0 0x7efe2c1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2a000640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 0x7efe2c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 -- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 msgr2=0x7efe2c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 0x7efe2c0af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 -- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe2c0b02f0 msgr2=0x7efe2c1f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe2c0b02f0 0x7efe2c1f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2b400640 1 -- 172.21.15.59:0/574015026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c1f7a20 con 0x7efe2c0caf70 2024-04-24T06:29:50.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.007+0000 7efe2a000640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 0x7efe2c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe2b400640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe2400eed0 tx=0x7efe2400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24018020 con 0x7efe2c0caf70 2024-04-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24014400 con 0x7efe2c0caf70 2024-04-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c0cb530 con 0x7efe2c0caf70 2024-04-24T06:29:50.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240147f0 con 0x7efe2c0caf70 2024-04-24T06:29:50.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.008+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c0cbaf0 con 0x7efe2c0caf70 2024-04-24T06:29:50.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.009+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2c04fd90 con 0x7efe2c0caf70 2024-04-24T06:29:50.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.010+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe24005d10 con 0x7efe2c0caf70 2024-04-24T06:29:50.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.011+0000 7efe22a00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 0x7efe0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.011+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efe2408c1e0 con 0x7efe2c0caf70 2024-04-24T06:29:50.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.012+0000 7efe2aa00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 0x7efe0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.012+0000 7efe2aa00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 0x7efe0806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009a30 tx=0x7efe1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.016+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe24022070 con 0x7efe2c0caf70 2024-04-24T06:29:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.262+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe2c0aea20 con 0x7efe2c0caf70 2024-04-24T06:29:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.263+0000 7efe22a00640 1 -- 172.21.15.59:0/574015026 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7efe2405a620 con 0x7efe2c0caf70 2024-04-24T06:29:50.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:50.264 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 msgr2=0x7efe0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 0x7efe0806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009a30 tx=0x7efe1c008040 comp rx=0 tx=0).stop 2024-04-24T06:29:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 msgr2=0x7efe2c0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe2400eed0 tx=0x7efe2400b480 comp rx=0 tx=0).stop 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 shutdown_connections 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efe0806c8f0 0x7efe0806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe2c0caf70 0x7efe2c0af7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efe2c0b4230 0x7efe2c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 --2- 172.21.15.59:0/574015026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe2c0b02f0 0x7efe2c1f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 >> 172.21.15.59:0/574015026 conn(0x7efe2c0b6750 msgr2=0x7efe2c0b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 shutdown_connections 2024-04-24T06:29:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.266+0000 7efe2be00640 1 -- 172.21.15.59:0/574015026 wait complete. 2024-04-24T06:29:50.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.501+0000 7f7b21c00640 1 Processor -- start 2024-04-24T06:29:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.501+0000 7f7b21c00640 1 -- start start 2024-04-24T06:29:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.502+0000 7f7b21c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7b1c15ece0 0x7f7b1c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.502+0000 7f7b21c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1c15f430 con 0x7f7b1c06f620 2024-04-24T06:29:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.502+0000 7f7b21c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1c15f710 con 0x7f7b1c0cd050 2024-04-24T06:29:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.502+0000 7f7b21c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1c0ced20 con 0x7f7b1c15ece0 2024-04-24T06:29:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b13400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7b1c06f620 0x7f7b1c0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48848/0 (socket says 172.21.15.59:48848) 2024-04-24T06:29:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b13400640 1 -- 172.21.15.59:0/3044142396 learned_addr learned my addr 172.21.15.59:0/3044142396 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b21200640 1 --2- 172.21.15.59:0/3044142396 >> v2:172.21.15.66:3300/0 conn(0x7f7b1c15ece0 0x7f7b1c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756350563 0 0) 0x7f7b1c15f430 con 0x7f7b1c06f620 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b00003520 con 0x7f7b1c06f620 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2728080277 0 0) 0x7f7b00003520 con 0x7f7b1c06f620 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 >> v2:172.21.15.66:3300/0 conn(0x7f7b1c15ece0 msgr2=0x7f7b1c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 --2- 172.21.15.59:0/3044142396 >> v2:172.21.15.66:3300/0 conn(0x7f7b1c15ece0 0x7f7b1c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 >> v1:172.21.15.66:6789/0 conn(0x7f7b1c0cd050 legacy=0x7f7b1c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c15f9f0 con 0x7f7b1c06f620 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7b04001670 con 0x7f7b1c06f620 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.503+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 >> v1:172.21.15.59:6789/0 conn(0x7f7b1c06f620 legacy=0x7f7b1c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000039b0 0x7f7b00003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b000052e0 0x7f7b0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0000c160 con 0x7f7b00004800 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0000c310 con 0x7f7b000039b0 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0000c510 con 0x7f7b000052e0 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b1be00640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000039b0 0x7f7b00003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b21200640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b000052e0 0x7f7b0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 -- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b000052e0 msgr2=0x7f7b0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b000052e0 0x7f7b0000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 -- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000039b0 msgr2=0x7f7b00003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000039b0 0x7f7b00003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b13400640 1 -- 172.21.15.59:0/3044142396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b00003520 con 0x7f7b00004800 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.504+0000 7f7b21200640 1 -- 172.21.15.59:0/3044142396 reap_dead start 2024-04-24T06:29:50.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.505+0000 7f7b13400640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b00004c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c210 tx=0x7f7b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd05c57f17c6dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.506+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04007d30 con 0x7f7b00004800 2024-04-24T06:29:50.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.506+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04005af0 con 0x7f7b00004800 2024-04-24T06:29:50.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.506+0000 7f7b1b400640 1 -- 172.21.15.59:0/3044142396 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04007070 con 0x7f7b00004800 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.506+0000 7f7b21c00640 1 -- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 msgr2=0x7f7b00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.506+0000 7f7b21c00640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b00004c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c210 tx=0x7f7b00005830 comp rx=0 tx=0).stop 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.507+0000 7f7b21c00640 1 -- 172.21.15.59:0/3044142396 shutdown_connections 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.507+0000 7f7b21c00640 1 --2- 172.21.15.59:0/3044142396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b00004c80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.507+0000 7f7b21c00640 1 -- 172.21.15.59:0/3044142396 >> 172.21.15.59:0/3044142396 conn(0x7f7b1c0b21a0 msgr2=0x7f7b1c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.507+0000 7f7b21c00640 1 -- 172.21.15.59:0/3044142396 shutdown_connections 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.507+0000 7f7b21c00640 1 -- 172.21.15.59:0/3044142396 wait complete. 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.508+0000 7f7b21c00640 1 Processor -- start 2024-04-24T06:29:50.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.508+0000 7f7b21c00640 1 -- start start 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1c0c4a20 0x7f7b1c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1c0b2580 0x7f7b1c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c06faa0 con 0x7f7b00004800 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c15fcd0 con 0x7f7b1c0b2580 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.509+0000 7f7b21c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1c15f130 con 0x7f7b1c0c4a20 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51888/0 (socket says 172.21.15.59:51888) 2024-04-24T06:29:50.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 -- 172.21.15.59:0/866527904 learned_addr learned my addr 172.21.15.59:0/866527904 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 -- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1c0c4a20 msgr2=0x7f7b1c158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1c0c4a20 0x7f7b1c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 -- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1c0b2580 msgr2=0x7f7b1c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1c0b2580 0x7f7b1c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 -- 172.21.15.59:0/866527904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c1f7e70 con 0x7f7b00004800 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.510+0000 7f7b1be00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00e410 tx=0x7f7b0c00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.511+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003a70 con 0x7f7b00004800 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.511+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c018070 con 0x7f7b00004800 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.511+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c004050 con 0x7f7b00004800 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.511+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c1f8150 con 0x7f7b00004800 2024-04-24T06:29:50.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.511+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c09de70 con 0x7f7b00004800 2024-04-24T06:29:50.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.512+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1c04fd90 con 0x7f7b00004800 2024-04-24T06:29:50.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.513+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b0c0041f0 con 0x7f7b00004800 2024-04-24T06:29:50.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.514+0000 7f7b18c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 0x7f7af806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.514+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7b0c08b770 con 0x7f7b00004800 2024-04-24T06:29:50.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.514+0000 7f7b13400640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 0x7f7af806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.515+0000 7f7b13400640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 0x7f7af806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b000057e0 tx=0x7f7b04009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.518+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b0c059b90 con 0x7f7b00004800 2024-04-24T06:29:50.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.726+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b1c09e140 con 0x7f7b00004800 2024-04-24T06:29:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.729+0000 7f7b18c00640 1 -- 172.21.15.59:0/866527904 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7f7b0c059490 con 0x7f7b00004800 2024-04-24T06:29:50.730 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:50.730 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 msgr2=0x7f7af806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 0x7f7af806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b000057e0 tx=0x7f7b04009030 comp rx=0 tx=0).stop 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 msgr2=0x7f7b1c1f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00e410 tx=0x7f7b0c00e8d0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 shutdown_connections 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7af806c970 0x7f7af806ee30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b1c0b2580 0x7f7b1c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7b1c0c4a20 0x7f7b1c158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 --2- 172.21.15.59:0/866527904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b00004800 0x7f7b1c1f7890 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 >> 172.21.15.59:0/866527904 conn(0x7f7b1c0b21a0 msgr2=0x7f7b1c0ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 shutdown_connections 2024-04-24T06:29:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.732+0000 7f7b21c00640 1 -- 172.21.15.59:0/866527904 wait complete. 2024-04-24T06:29:50.754 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:29:50.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.965+0000 7f9fbe400640 1 Processor -- start 2024-04-24T06:29:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.966+0000 7f9fbe400640 1 -- start start 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.967+0000 7f9fbe400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.967+0000 7f9fbe400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9fb80b4690 0x7f9fb80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.967+0000 7f9fbe400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb8160210 con 0x7f9fb815c3d0 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.967+0000 7f9fbe400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb81604f0 con 0x7f9fb80b4690 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.967+0000 7f9fbe400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb81607d0 con 0x7f9fb80cd760 2024-04-24T06:29:50.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.968+0000 7f9fafe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.968+0000 7f9fbd000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f9fb815c3d0 0x7f9fb80cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:36668/0 (socket says 172.21.15.59:36668) 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.968+0000 7f9fbd000640 1 -- 172.21.15.59:0/1213671319 learned_addr learned my addr 172.21.15.59:0/1213671319 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.968+0000 7f9fbda00640 1 --2- 172.21.15.59:0/1213671319 >> v2:172.21.15.66:3300/0 conn(0x7f9fb80b4690 0x7f9fb80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.969+0000 7f9fbda00640 1 -- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 msgr2=0x7f9fb80b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.969+0000 7f9fbda00640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb80b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.969+0000 7f9fbda00640 1 -- 172.21.15.59:0/1213671319 >> v1:172.21.15.66:6789/0 conn(0x7f9fb815c3d0 legacy=0x7f9fb80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.969+0000 7f9fbda00640 1 -- 172.21.15.59:0/1213671319 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb8160ab0 con 0x7f9fb80b4690 2024-04-24T06:29:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.969+0000 7f9fafe00640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb80b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.970+0000 7f9fbda00640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb80cb7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009d90 tx=0x7f9fb002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1ecd91581b3de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.970+0000 7f9fb7e00640 1 -- 172.21.15.59:0/1213671319 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0007740 con 0x7f9fb80b4690 2024-04-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fb7e00640 1 -- 172.21.15.59:0/1213671319 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb00078e0 con 0x7f9fb80b4690 2024-04-24T06:29:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fb7e00640 1 -- 172.21.15.59:0/1213671319 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0007cb0 con 0x7f9fb80b4690 2024-04-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 -- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 msgr2=0x7f9fb80cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb80cb7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009d90 tx=0x7f9fb002a660 comp rx=0 tx=0).stop 2024-04-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 -- 172.21.15.59:0/1213671319 shutdown_connections 2024-04-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb80cb7d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 --2- 172.21.15.59:0/1213671319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb80b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.971+0000 7f9fbe400640 1 -- 172.21.15.59:0/1213671319 >> 172.21.15.59:0/1213671319 conn(0x7f9fb80b21a0 msgr2=0x7f9fb80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:50.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.972+0000 7f9fbe400640 1 -- 172.21.15.59:0/1213671319 shutdown_connections 2024-04-24T06:29:50.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.972+0000 7f9fbe400640 1 -- 172.21.15.59:0/1213671319 wait complete. 2024-04-24T06:29:50.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.974+0000 7f9fbe400640 1 Processor -- start 2024-04-24T06:29:50.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.974+0000 7f9fbe400640 1 -- start start 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.975+0000 7f9fbe400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.975+0000 7f9fbe400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb81f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbe400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbe400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb815c7b0 con 0x7f9fb815c3d0 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbe400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80b4ac0 con 0x7f9fb80b4690 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbe400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb815c930 con 0x7f9fb80cd760 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51890/0 (socket says 172.21.15.59:51890) 2024-04-24T06:29:50.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbda00640 1 -- 172.21.15.59:0/3917094962 learned_addr learned my addr 172.21.15.59:0/3917094962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:50.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fbd000640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.976+0000 7f9fafe00640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb81f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 -- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 msgr2=0x7f9fb81f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb81f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 -- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 msgr2=0x7f9fb815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 -- 172.21.15.59:0/3917094962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80c7aa0 con 0x7f9fb815c3d0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fafe00640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbda00640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9fb002ac50 tx=0x7f9fb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0040070 con 0x7f9fb815c3d0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb0031bc0 con 0x7f9fb815c3d0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009450 con 0x7f9fb815c3d0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.977+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb80c7d80 con 0x7f9fb815c3d0 2024-04-24T06:29:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.978+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb80c8340 con 0x7f9fb815c3d0 2024-04-24T06:29:50.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.979+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb80c8540 con 0x7f9fb815c3d0 2024-04-24T06:29:50.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.980+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fb0039040 con 0x7f9fb815c3d0 2024-04-24T06:29:50.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.981+0000 7f9fb5600640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:50.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.982+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9fb00ad320 con 0x7f9fb815c3d0 2024-04-24T06:29:50.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.982+0000 7f9fbd000640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.983+0000 7f9fbd000640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 0x7f9f9406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005880 tx=0x7f9fa0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:50.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:50.985+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fb80c8540 con 0x7f9fb815c3d0 2024-04-24T06:29:51.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.230+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fb804fd90 con 0x7f9fb815c3d0 2024-04-24T06:29:51.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.231+0000 7f9fb5600640 1 -- 172.21.15.59:0/3917094962 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f9fb007b6c0 con 0x7f9fb815c3d0 2024-04-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:51.233 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:51.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 msgr2=0x7f9f9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 0x7f9f9406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005880 tx=0x7f9fa0009040 comp rx=0 tx=0).stop 2024-04-24T06:29:51.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 msgr2=0x7f9fb815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9fb002ac50 tx=0x7f9fb0004a40 comp rx=0 tx=0).stop 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 shutdown_connections 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f9406c920 0x7f9f9406ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb815c3d0 0x7f9fb815f3c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9fb80cd760 0x7f9fb81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 --2- 172.21.15.59:0/3917094962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fb80b4690 0x7f9fb815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 >> 172.21.15.59:0/3917094962 conn(0x7f9fb80b21a0 msgr2=0x7f9fb80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 shutdown_connections 2024-04-24T06:29:51.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.234+0000 7f9fbe400640 1 -- 172.21.15.59:0/3917094962 wait complete. 2024-04-24T06:29:51.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:29:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.457+0000 7fb21ec00640 1 Processor -- start 2024-04-24T06:29:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.458+0000 7fb21ec00640 1 -- start start 2024-04-24T06:29:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.459+0000 7fb21ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2181454e0 0x7fb218145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.459+0000 7fb21ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb2180b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.459+0000 7fb21ec00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2180cd520 con 0x7fb2180b3e40 2024-04-24T06:29:51.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.459+0000 7fb21ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb2180cddc0 con 0x7fb2181454e0 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.459+0000 7fb21ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb2180cd800 con 0x7fb2180cd020 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb21cc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb2180b3e40 0x7fb218160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48850/0 (socket says 172.21.15.59:48850) 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb21cc00640 1 -- 172.21.15.59:0/3450158237 learned_addr learned my addr 172.21.15.59:0/3450158237 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb20f400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb2180b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb217e00640 1 --2- 172.21.15.59:0/3450158237 >> v2:172.21.15.66:3300/0 conn(0x7fb2181454e0 0x7fb218145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1791469565 0 0) 0x7fb2180cd520 con 0x7fb2180b3e40 2024-04-24T06:29:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.460+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1fc003520 con 0x7fb2180b3e40 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 375282457 0 0) 0x7fb1fc003520 con 0x7fb2180b3e40 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 msgr2=0x7fb2180b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb2180b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 >> v2:172.21.15.66:3300/0 conn(0x7fb2181454e0 msgr2=0x7fb218145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 --2- 172.21.15.59:0/3450158237 >> v2:172.21.15.66:3300/0 conn(0x7fb2181454e0 0x7fb218145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2180cdae0 con 0x7fb2180b3e40 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb2100019f0 con 0x7fb2180b3e40 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 >> v1:172.21.15.59:6789/0 conn(0x7fb2180b3e40 legacy=0x7fb218160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb1fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc004d40 0x7fb1fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180cddc0 con 0x7fb1fc004d40 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180cd800 con 0x7fb1fc0039b0 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc00b9f0 con 0x7fb2180cd020 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.461+0000 7fb21cc00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc004d40 0x7fb1fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb20f400640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb1fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb217e00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb21cc00640 1 -- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 msgr2=0x7fb1fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb21cc00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2180cd020 0x7fb1fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb21cc00640 1 -- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc0039b0 msgr2=0x7fb1fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb21cc00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc0039b0 0x7fb1fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.462+0000 7fb21cc00640 1 -- 172.21.15.59:0/3450158237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc003520 con 0x7fb1fc004d40 2024-04-24T06:29:51.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.463+0000 7fb21cc00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc004d40 0x7fb1fc0051c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a7d0 tx=0x7fb1fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75965251e9be48b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.463+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21000a3f0 con 0x7fb1fc004d40 2024-04-24T06:29:51.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.463+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21002f070 con 0x7fb1fc004d40 2024-04-24T06:29:51.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.463+0000 7fb217400640 1 -- 172.21.15.59:0/3450158237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21002bc50 con 0x7fb1fc004d40 2024-04-24T06:29:51.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 -- 172.21.15.59:0/3450158237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc004d40 msgr2=0x7fb1fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 --2- 172.21.15.59:0/3450158237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc004d40 0x7fb1fc0051c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a7d0 tx=0x7fb1fc005830 comp rx=0 tx=0).stop 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21cc00640 1 -- 172.21.15.59:0/3450158237 reap_dead start 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 -- 172.21.15.59:0/3450158237 shutdown_connections 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 -- 172.21.15.59:0/3450158237 >> 172.21.15.59:0/3450158237 conn(0x7fb218067ea0 msgr2=0x7fb21806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 -- 172.21.15.59:0/3450158237 shutdown_connections 2024-04-24T06:29:51.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.464+0000 7fb21ec00640 1 -- 172.21.15.59:0/3450158237 wait complete. 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.466+0000 7fb21ec00640 1 Processor -- start 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.466+0000 7fb21ec00640 1 -- start start 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc004d40 0x7fb21813aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb21815db10 0x7fb21815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb218146270 con 0x7fb21806f6a0 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2181460f0 con 0x7fb1fc004d40 2024-04-24T06:29:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb218161020 con 0x7fb21815db10 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51910/0 (socket says 172.21.15.59:51910) 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.467+0000 7fb21cc00640 1 -- 172.21.15.59:0/6432909 learned_addr learned my addr 172.21.15.59:0/6432909 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb20f400640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb21815db10 0x7fb21815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb217e00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc004d40 0x7fb21813aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 -- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb21815db10 msgr2=0x7fb21815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb21815db10 0x7fb21815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 -- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc004d40 msgr2=0x7fb21813aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc004d40 0x7fb21813aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 -- 172.21.15.59:0/6432909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21815c620 con 0x7fb21806f6a0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21cc00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a8f0 tx=0x7fb210001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21000a3f0 con 0x7fb21806f6a0 2024-04-24T06:29:51.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210044070 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210001f40 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.468+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21815cd50 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.469+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21815d310 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.470+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb218068a80 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.471+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb210034670 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.472+0000 7fb214c00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 0x7fb1f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.472+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb2100ac330 con 0x7fb21806f6a0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.472+0000 7fb217e00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 0x7fb1f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.473+0000 7fb217e00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 0x7fb1f406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb208009950 tx=0x7fb208008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.476+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb21815c620 con 0x7fb21806f6a0 2024-04-24T06:29:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.720+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb218160030 con 0x7fb21806f6a0 2024-04-24T06:29:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.722+0000 7fb214c00640 1 -- 172.21.15.59:0/6432909 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fb21007a770 con 0x7fb21806f6a0 2024-04-24T06:29:51.723 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:51.723 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2539965486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2539965486},{"type":"v1","addr":"172.21.15.59:6837","nonce":2539965486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3521576970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3521576970},{"type":"v1","addr":"172.21.15.59:6835","nonce":3521576970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/186924553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":186924553},{"type":"v1","addr":"172.21.15.66:6833","nonce":186924553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:28:44.183608+0000","modified":"2024-04-24T06:28:47.252737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1796814529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1796814529},{"type":"v1","addr":"172.21.15.66:6835","nonce":1796814529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T06:29:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-04-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.724+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 msgr2=0x7fb1f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.724+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 0x7fb1f406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb208009950 tx=0x7fb208008040 comp rx=0 tx=0).stop 2024-04-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.724+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 msgr2=0x7fb21813ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.724+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a8f0 tx=0x7fb210001cf0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 shutdown_connections 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1f406c8f0 0x7fb1f406edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb21806f6a0 0x7fb21813ec40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb21815db10 0x7fb21815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 --2- 172.21.15.59:0/6432909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1fc004d40 0x7fb21813aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 >> 172.21.15.59:0/6432909 conn(0x7fb218067ea0 msgr2=0x7fb2180b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:51.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 shutdown_connections 2024-04-24T06:29:51.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.725+0000 7fb21ec00640 1 -- 172.21.15.59:0/6432909 wait complete. 2024-04-24T06:29:51.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:29:51.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.952+0000 7fcd5e200640 1 Processor -- start 2024-04-24T06:29:51.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.952+0000 7fcd5e200640 1 -- start start 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd5806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd5815ece0 0x7fcd580cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 0x7fcd580cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcd580ced20 con 0x7fcd5815ece0 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5815fcd0 con 0x7fcd580b3080 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd5e200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcd580cd970 con 0x7fcd580cd050 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd57e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd5815ece0 0x7fcd580cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd57e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd5815ece0 0x7fcd580cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51918/0 (socket says 172.21.15.59:51918) 2024-04-24T06:29:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.954+0000 7fcd57e00640 1 -- 172.21.15.59:0/3119902488 learned_addr learned my addr 172.21.15.59:0/3119902488 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd5d800640 1 --2- 172.21.15.59:0/3119902488 >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 0x7fcd580cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd57e00640 1 -- 172.21.15.59:0/3119902488 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 msgr2=0x7fcd5806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd5ce00640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd5806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd57e00640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd5806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd57e00640 1 -- 172.21.15.59:0/3119902488 >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 msgr2=0x7fcd580cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd57e00640 1 --2- 172.21.15.59:0/3119902488 >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 0x7fcd580cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd57e00640 1 -- 172.21.15.59:0/3119902488 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5815f160 con 0x7fcd5815ece0 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd5d800640 1 --2- 172.21.15.59:0/3119902488 >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 0x7fcd580cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.955+0000 7fcd5ce00640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd5806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.956+0000 7fcd57e00640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cd430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcd5815c8b0 tx=0x7fcd4402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc0051d682356fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.956+0000 7fcd57400640 1 -- 172.21.15.59:0/3119902488 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44037070 con 0x7fcd5815ece0 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.956+0000 7fcd57400640 1 -- 172.21.15.59:0/3119902488 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd44032440 con 0x7fcd5815ece0 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.956+0000 7fcd57400640 1 -- 172.21.15.59:0/3119902488 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44032810 con 0x7fcd5815ece0 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 -- 172.21.15.59:0/3119902488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 msgr2=0x7fcd580cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cd430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcd5815c8b0 tx=0x7fcd4402a140 comp rx=0 tx=0).stop 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 -- 172.21.15.59:0/3119902488 shutdown_connections 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3119902488 >> v2:172.21.15.66:3300/0 conn(0x7fcd580b3080 0x7fcd580cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cd430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3119902488 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd5806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 -- 172.21.15.59:0/3119902488 >> 172.21.15.59:0/3119902488 conn(0x7fcd580b21a0 msgr2=0x7fcd580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 -- 172.21.15.59:0/3119902488 shutdown_connections 2024-04-24T06:29:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.957+0000 7fcd5e200640 1 -- 172.21.15.59:0/3119902488 wait complete. 2024-04-24T06:29:51.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.959+0000 7fcd5e200640 1 Processor -- start 2024-04-24T06:29:51.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.959+0000 7fcd5e200640 1 -- start start 2024-04-24T06:29:51.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd580b3080 0x7fcd58158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd580cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815f9f0 con 0x7fcd5815ece0 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815f710 con 0x7fcd580b3080 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.960+0000 7fcd5e200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcd580cef50 con 0x7fcd580cd050 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51922/0 (socket says 172.21.15.59:51922) 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 -- 172.21.15.59:0/3579154787 learned_addr learned my addr 172.21.15.59:0/3579154787 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5ce00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd580b3080 0x7fcd58158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd57e00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd580cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 -- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 msgr2=0x7fcd580cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd580cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 -- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd580b3080 msgr2=0x7fcd58158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd580b3080 0x7fcd58158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 -- 172.21.15.59:0/3579154787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd58158690 con 0x7fcd5815ece0 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd57e00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd580cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.961+0000 7fcd5d800640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000cd80 tx=0x7fcd5000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.962+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000f680 con 0x7fcd5815ece0 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.962+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd5000f840 con 0x7fcd5815ece0 2024-04-24T06:29:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.962+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000fc30 con 0x7fcd5815ece0 2024-04-24T06:29:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.962+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd580c8c00 con 0x7fcd5815ece0 2024-04-24T06:29:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.962+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd580c91c0 con 0x7fcd5815ece0 2024-04-24T06:29:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.963+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd580af400 con 0x7fcd5815ece0 2024-04-24T06:29:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.964+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd5000e3e0 con 0x7fcd5815ece0 2024-04-24T06:29:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.965+0000 7fcd54c00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 0x7fcd3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:51.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.965+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcd5008c960 con 0x7fcd5815ece0 2024-04-24T06:29:51.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.966+0000 7fcd5ce00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 0x7fcd3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:51.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.966+0000 7fcd5ce00640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 0x7fcd3806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a910 tx=0x7fcd4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:51.970+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd580af400 con 0x7fcd5815ece0 2024-04-24T06:29:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.174+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd580731c0 con 0x7fcd5815ece0 2024-04-24T06:29:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.176+0000 7fcd54c00640 1 -- 172.21.15.59:0/3579154787 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16770 (secure 0 0 0) 0x7fcd50014030 con 0x7fcd5815ece0 2024-04-24T06:29:52.177 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:52.178 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:26.993776+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:28:42.577295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:28:43.152430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 msgr2=0x7fcd3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 0x7fcd3806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a910 tx=0x7fcd4c008040 comp rx=0 tx=0).stop 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 msgr2=0x7fcd580cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000cd80 tx=0x7fcd5000a650 comp rx=0 tx=0).stop 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 shutdown_connections 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd3806c8f0 0x7fcd3806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd5815ece0 0x7fcd580cbd70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd580cd050 0x7fcd580cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 --2- 172.21.15.59:0/3579154787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd580b3080 0x7fcd58158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 >> 172.21.15.59:0/3579154787 conn(0x7fcd580b21a0 msgr2=0x7fcd580b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:52.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 shutdown_connections 2024-04-24T06:29:52.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.179+0000 7fcd5e200640 1 -- 172.21.15.59:0/3579154787 wait complete. 2024-04-24T06:29:52.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:29:52.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.392+0000 7fc8e8600640 1 Processor -- start 2024-04-24T06:29:52.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.392+0000 7fc8e8600640 1 -- start start 2024-04-24T06:29:52.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e8600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e00cf410 0x7fc8e015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e8600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e015dda0 con 0x7fc8e006ddf0 2024-04-24T06:29:52.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e8600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e00cd880 con 0x7fc8e006e250 2024-04-24T06:29:52.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e8600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e0071ec0 con 0x7fc8e00cf410 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e5c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e00cf410 0x7fc8e015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e5200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc8e006ddf0 0x7fc8e0072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48858/0 (socket says 172.21.15.59:48858) 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.394+0000 7fc8e5200640 1 -- 172.21.15.59:0/3973662482 learned_addr learned my addr 172.21.15.59:0/3973662482 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.395+0000 7fc8dfe00640 1 -- 172.21.15.59:0/3973662482 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722691473 0 0) 0x7fc8cc001110 con 0x7fc8e006ddf0 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.395+0000 7fc8e5c00640 1 -- 172.21.15.59:0/3973662482 >> v1:172.21.15.66:6789/0 conn(0x7fc8e006e250 legacy=0x7fc8e00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.395+0000 7fc8e5c00640 1 -- 172.21.15.59:0/3973662482 >> v1:172.21.15.59:6789/0 conn(0x7fc8e006ddf0 legacy=0x7fc8e0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.395+0000 7fc8e5c00640 1 -- 172.21.15.59:0/3973662482 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e00cdb20 con 0x7fc8e00cf410 2024-04-24T06:29:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.397+0000 7fc8e5c00640 1 --2- 172.21.15.59:0/3973662482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e015d860 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954443bc94676ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.397+0000 7fc8dfe00640 1 -- 172.21.15.59:0/3973662482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4037070 con 0x7fc8e00cf410 2024-04-24T06:29:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.397+0000 7fc8dfe00640 1 -- 172.21.15.59:0/3973662482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d4032440 con 0x7fc8e00cf410 2024-04-24T06:29:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.397+0000 7fc8dfe00640 1 -- 172.21.15.59:0/3973662482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032810 con 0x7fc8e00cf410 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 -- 172.21.15.59:0/3973662482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 msgr2=0x7fc8e015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3973662482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e015d860 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 -- 172.21.15.59:0/3973662482 shutdown_connections 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3973662482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e015d860 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 -- 172.21.15.59:0/3973662482 >> 172.21.15.59:0/3973662482 conn(0x7fc8e00b2130 msgr2=0x7fc8e00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 -- 172.21.15.59:0/3973662482 shutdown_connections 2024-04-24T06:29:52.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.398+0000 7fc8e8600640 1 -- 172.21.15.59:0/3973662482 wait complete. 2024-04-24T06:29:52.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.400+0000 7fc8e8600640 1 Processor -- start 2024-04-24T06:29:52.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.400+0000 7fc8e8600640 1 -- start start 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e006ddf0 0x7fc8e01f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 0x7fc8e00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e015a6a0 con 0x7fc8e00cf410 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00cde00 con 0x7fc8e006e250 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e8600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e015a820 con 0x7fc8e006ddf0 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51932/0 (socket says 172.21.15.59:51932) 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.401+0000 7fc8e6600640 1 -- 172.21.15.59:0/3590261076 learned_addr learned my addr 172.21.15.59:0/3590261076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e5c00640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e006ddf0 0x7fc8e01f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e5200640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 0x7fc8e00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 -- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e006ddf0 msgr2=0x7fc8e01f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e006ddf0 0x7fc8e01f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 -- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 msgr2=0x7fc8e00b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 0x7fc8e00b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 -- 172.21.15.59:0/3590261076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e00b4310 con 0x7fc8e00cf410 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e5200640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 0x7fc8e00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:52.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e6600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7a0 tx=0x7fc8d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003b70 con 0x7fc8e00cf410 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d8003d30 con 0x7fc8e00cf410 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800aa90 con 0x7fc8e00cf410 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.402+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e00b45f0 con 0x7fc8e00cf410 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.403+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e0156e50 con 0x7fc8e00cf410 2024-04-24T06:29:52.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.404+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e0068a80 con 0x7fc8e00cf410 2024-04-24T06:29:52.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.405+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d800ac30 con 0x7fc8e00cf410 2024-04-24T06:29:52.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.406+0000 7fc8dd600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 0x7fc8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.406+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc8d808bb80 con 0x7fc8e00cf410 2024-04-24T06:29:52.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.406+0000 7fc8e5c00640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 0x7fc8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.407+0000 7fc8e5c00640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 0x7fc8c006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.410+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8d8059f20 con 0x7fc8e00cf410 2024-04-24T06:29:52.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.655+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8e00700b0 con 0x7fc8e00cf410 2024-04-24T06:29:52.656 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:29:52.656+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:29:52.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.660+0000 7fc8dd600640 1 -- 172.21.15.59:0/3590261076 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fc8d8059820 con 0x7fc8e00cf410 2024-04-24T06:29:52.661 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:29:52.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 msgr2=0x7fc8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 0x7fc8c006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:52.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 msgr2=0x7fc8e00b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c7a0 tx=0x7fc8d800cc60 comp rx=0 tx=0).stop 2024-04-24T06:29:52.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 shutdown_connections 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c006c8f0 0x7fc8c006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e00cf410 0x7fc8e00b3a90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e006e250 0x7fc8e00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 --2- 172.21.15.59:0/3590261076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e006ddf0 0x7fc8e01f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 >> 172.21.15.59:0/3590261076 conn(0x7fc8e00b2130 msgr2=0x7fc8e015e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 shutdown_connections 2024-04-24T06:29:52.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.663+0000 7fc8e8600640 1 -- 172.21.15.59:0/3590261076 wait complete. 2024-04-24T06:29:52.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:29:52.694 INFO:tasks.ceph.mds.b.smithi066.stderr: -4040> 2024-04-24T06:28:50.555+0000 7f8881000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:29:52.798 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:29:52.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.889+0000 7fe449000640 1 Processor -- start 2024-04-24T06:29:52.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.890+0000 7fe449000640 1 -- start start 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe449000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe44415ed20 0x7fe4440d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe449000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 0x7fe4440d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe449000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4440b5da0 con 0x7fe4440b3f10 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe449000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe44415f100 con 0x7fe44415ed20 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe449000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440d0a20 con 0x7fe4440b3600 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe443e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe4440b3f10 0x7fe4440b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48874/0 (socket says 172.21.15.59:48874) 2024-04-24T06:29:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.891+0000 7fe443e00640 1 -- 172.21.15.59:0/1575969491 learned_addr learned my addr 172.21.15.59:0/1575969491 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:52.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe443400640 1 --2- 172.21.15.59:0/1575969491 >> v2:172.21.15.66:3300/0 conn(0x7fe44415ed20 0x7fe4440d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe43aa00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 0x7fe4440d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892311477 0 0) 0x7fe4440b5da0 con 0x7fe4440b3f10 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe428003520 con 0x7fe4440b3f10 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3317960429 0 0) 0x7fe428003520 con 0x7fe4440b3f10 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 msgr2=0x7fe4440d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 0x7fe4440d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 >> v2:172.21.15.66:3300/0 conn(0x7fe44415ed20 msgr2=0x7fe4440d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 --2- 172.21.15.59:0/1575969491 >> v2:172.21.15.66:3300/0 conn(0x7fe44415ed20 0x7fe4440d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440d2ed0 con 0x7fe4440b3f10 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.892+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe43c001670 con 0x7fe4440b3f10 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 >> v1:172.21.15.59:6789/0 conn(0x7fe4440b3f10 legacy=0x7fe4440b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 0x7fe428004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d0a20 con 0x7fe4280039b0 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe42800ba10 con 0x7fe428004d40 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe42800bc10 con 0x7fe4440b3600 2024-04-24T06:29:52.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443e00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 -- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 msgr2=0x7fe428004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:52.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440b3600 0x7fe428004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 -- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe428004d40 msgr2=0x7fe4280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:52.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443400640 1 -- 172.21.15.59:0/1575969491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe428003520 con 0x7fe4280039b0 2024-04-24T06:29:52.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.893+0000 7fe443e00640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.894+0000 7fe443400640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe434009fc0 tx=0x7fe428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95a05bf24fa79c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434009c10 con 0x7fe4280039b0 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe434036070 con 0x7fe4280039b0 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe442a00640 1 -- 172.21.15.59:0/1575969491 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43402d070 con 0x7fe4280039b0 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe449000640 1 -- 172.21.15.59:0/1575969491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4280039b0 msgr2=0x7fe428003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe449000640 1 --2- 172.21.15.59:0/1575969491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe434009fc0 tx=0x7fe428005830 comp rx=0 tx=0).stop 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe443e00640 1 -- 172.21.15.59:0/1575969491 reap_dead start 2024-04-24T06:29:52.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe449000640 1 -- 172.21.15.59:0/1575969491 shutdown_connections 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.895+0000 7fe449000640 1 -- 172.21.15.59:0/1575969491 >> 172.21.15.59:0/1575969491 conn(0x7fe44408bf90 msgr2=0x7fe4440d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.896+0000 7fe449000640 1 -- 172.21.15.59:0/1575969491 shutdown_connections 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.896+0000 7fe449000640 1 -- 172.21.15.59:0/1575969491 wait complete. 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.897+0000 7fe449000640 1 Processor -- start 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.898+0000 7fe449000640 1 -- start start 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.898+0000 7fe449000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4280039b0 0x7fe4441f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.898+0000 7fe449000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440c9540 0x7fe4440c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.898+0000 7fe449000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe449000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d13e0 con 0x7fe4440b20a0 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe449000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d3490 con 0x7fe4280039b0 2024-04-24T06:29:52.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe449000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440d1560 con 0x7fe4440c9540 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51956/0 (socket says 172.21.15.59:51956) 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 -- 172.21.15.59:0/3709477927 learned_addr learned my addr 172.21.15.59:0/3709477927 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443400640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4280039b0 0x7fe4441f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe43aa00640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440c9540 0x7fe4440c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 -- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440c9540 msgr2=0x7fe4440c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440c9540 0x7fe4440c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 -- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4280039b0 msgr2=0x7fe4441f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4280039b0 0x7fe4441f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.899+0000 7fe443e00640 1 -- 172.21.15.59:0/3709477927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441f4bc0 con 0x7fe4440b20a0 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe443e00640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000fb0 tx=0x7fe43c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c007440 con 0x7fe4440b20a0 2024-04-24T06:29:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c007600 con 0x7fe4440b20a0 2024-04-24T06:29:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004a20 con 0x7fe4440b20a0 2024-04-24T06:29:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441f4ea0 con 0x7fe4440b20a0 2024-04-24T06:29:52.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.900+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441f5460 con 0x7fe4440b20a0 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.901+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440b2a60 con 0x7fe4440b20a0 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.902+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe43c01b060 con 0x7fe4440b20a0 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.903+0000 7fe43be00640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 0x7fe42006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.903+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fe43c08b4e0 con 0x7fe4440b20a0 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.903+0000 7fe443400640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 0x7fe42006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:52.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.904+0000 7fe443400640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 0x7fe42006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe434035000 tx=0x7fe43402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:52.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:52.908+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe43c059840 con 0x7fe4440b20a0 2024-04-24T06:29:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.153+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe4440d2ce0 con 0x7fe4440b20a0 2024-04-24T06:29:53.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.663+0000 7fe43be00640 1 -- 172.21.15.59:0/3709477927 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fe43c059140 con 0x7fe4440b20a0 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 msgr2=0x7fe42006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 0x7fe42006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe434035000 tx=0x7fe43402f000 comp rx=0 tx=0).stop 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 msgr2=0x7fe4440b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000fb0 tx=0x7fe43c00d9d0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 shutdown_connections 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe42006c8f0 0x7fe42006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4440b20a0 0x7fe4440b2520 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4440c9540 0x7fe4440c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 --2- 172.21.15.59:0/3709477927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4280039b0 0x7fe4441f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 >> 172.21.15.59:0/3709477927 conn(0x7fe44408bf90 msgr2=0x7fe4440d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 shutdown_connections 2024-04-24T06:29:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.668+0000 7fe449000640 1 -- 172.21.15.59:0/3709477927 wait complete. 2024-04-24T06:29:53.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:29:53.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.912+0000 7f2de9200640 1 Processor -- start 2024-04-24T06:29:53.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.912+0000 7f2de9200640 1 -- start start 2024-04-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.914+0000 7f2de9200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2de415acf0 0x7f2de4073810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.914+0000 7f2de9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2de415ef00 0x7f2de40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.914+0000 7f2de9200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de415b0d0 con 0x7f2de4071cb0 2024-04-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.914+0000 7f2de9200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2de40b5830 con 0x7f2de415ef00 2024-04-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.914+0000 7f2de9200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2de4073e10 con 0x7f2de415acf0 2024-04-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2de415ef00 0x7f2de40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2de4071cb0 0x7f2de415e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48888/0 (socket says 172.21.15.59:48888) 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de2a00640 1 -- 172.21.15.59:0/503986192 learned_addr learned my addr 172.21.15.59:0/503986192 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3400640 1 --2- 172.21.15.59:0/503986192 >> v2:172.21.15.66:3300/0 conn(0x7f2de415acf0 0x7f2de4073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3e00640 1 -- 172.21.15.59:0/503986192 >> v2:172.21.15.66:3300/0 conn(0x7f2de415acf0 msgr2=0x7f2de4073810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3e00640 1 --2- 172.21.15.59:0/503986192 >> v2:172.21.15.66:3300/0 conn(0x7f2de415acf0 0x7f2de4073810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3e00640 1 -- 172.21.15.59:0/503986192 >> v1:172.21.15.59:6789/0 conn(0x7f2de4071cb0 legacy=0x7f2de415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de2000640 1 -- 172.21.15.59:0/503986192 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920249346 0 0) 0x7f2de415b0d0 con 0x7f2de4071cb0 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.915+0000 7f2de3e00640 1 -- 172.21.15.59:0/503986192 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40b51e0 con 0x7f2de415ef00 2024-04-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.916+0000 7f2de3e00640 1 --2- 172.21.15.59:0/503986192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de415ef00 0x7f2de40b3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2de40710e0 tx=0x7f2ddc02b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c70491ace1d779 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de2000640 1 -- 172.21.15.59:0/503986192 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00b040 con 0x7f2de415ef00 2024-04-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de2000640 1 -- 172.21.15.59:0/503986192 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ddc030710 con 0x7f2de415ef00 2024-04-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de2000640 1 -- 172.21.15.59:0/503986192 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc030ae0 con 0x7f2de415ef00 2024-04-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 -- 172.21.15.59:0/503986192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de415ef00 msgr2=0x7f2de40b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 --2- 172.21.15.59:0/503986192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de415ef00 0x7f2de40b3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2de40710e0 tx=0x7f2ddc02b4d0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 -- 172.21.15.59:0/503986192 shutdown_connections 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 --2- 172.21.15.59:0/503986192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de415ef00 0x7f2de40b3600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 --2- 172.21.15.59:0/503986192 >> v2:172.21.15.66:3300/0 conn(0x7f2de415acf0 0x7f2de4073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.917+0000 7f2de9200640 1 -- 172.21.15.59:0/503986192 >> 172.21.15.59:0/503986192 conn(0x7f2de40b1f90 msgr2=0x7f2de40b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.918+0000 7f2de9200640 1 -- 172.21.15.59:0/503986192 shutdown_connections 2024-04-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.918+0000 7f2de9200640 1 -- 172.21.15.59:0/503986192 wait complete. 2024-04-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.919+0000 7f2de9200640 1 Processor -- start 2024-04-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 -- start start 2024-04-24T06:29:53.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2de415acf0 0x7f2de4150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2de415ef00 0x7f2de40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4072270 con 0x7f2de4071cb0 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2de40720f0 con 0x7f2de415acf0 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.920+0000 7f2de9200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2de40ccd20 con 0x7f2de415ef00 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51968/0 (socket says 172.21.15.59:51968) 2024-04-24T06:29:53.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 -- 172.21.15.59:0/3696231494 learned_addr learned my addr 172.21.15.59:0/3696231494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 -- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2de415ef00 msgr2=0x7f2de40cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2de415ef00 0x7f2de40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 -- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2de415acf0 msgr2=0x7f2de4150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2de415acf0 0x7f2de4150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 -- 172.21.15.59:0/3696231494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40ce220 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.921+0000 7f2de3400640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4007ef0 tx=0x7f2dd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.922+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4014620 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.922+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd4019070 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.922+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de4150a80 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.922+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4014a00 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.922+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de4151040 con 0x7f2de4071cb0 2024-04-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.923+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de404fd90 con 0x7f2de4071cb0 2024-04-24T06:29:53.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.924+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dd4014ba0 con 0x7f2de4071cb0 2024-04-24T06:29:53.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.925+0000 7f2ddb400640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 0x7f2dc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:53.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.925+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f2dd408c500 con 0x7f2de4071cb0 2024-04-24T06:29:53.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.926+0000 7f2de2a00640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 0x7f2dc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:53.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.926+0000 7f2de2a00640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 0x7f2dc006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc007fd0 tx=0x7f2dcc007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:53.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:53.930+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dd405a850 con 0x7f2de4071cb0 2024-04-24T06:29:54.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.133+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2de406fe40 con 0x7f2de4071cb0 2024-04-24T06:29:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.667+0000 7f2ddb400640 1 -- 172.21.15.59:0/3696231494 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f2dd405a150 con 0x7f2de4071cb0 2024-04-24T06:29:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:29:54.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.670+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 msgr2=0x7f2dc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.670+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 0x7f2dc006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc007fd0 tx=0x7f2dcc007920 comp rx=0 tx=0).stop 2024-04-24T06:29:54.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.670+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 msgr2=0x7f2de414dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.670+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4007ef0 tx=0x7f2dd400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 shutdown_connections 2024-04-24T06:29:54.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2dc006c920 0x7f2dc006ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2de415ef00 0x7f2de40cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2de415acf0 0x7f2de4150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 --2- 172.21.15.59:0/3696231494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4071cb0 0x7f2de414dc40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 >> 172.21.15.59:0/3696231494 conn(0x7f2de40b1f90 msgr2=0x7f2de406e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 shutdown_connections 2024-04-24T06:29:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.671+0000 7f2de9200640 1 -- 172.21.15.59:0/3696231494 wait complete. 2024-04-24T06:29:54.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:29:54.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.918+0000 7f6bd7e00640 1 Processor -- start 2024-04-24T06:29:54.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.919+0000 7f6bd7e00640 1 -- start start 2024-04-24T06:29:54.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.920+0000 7f6bd7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6bd815e760 0x7f6bd80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.920+0000 7f6bd7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd80cefc0 con 0x7f6bd80d4b60 2024-04-24T06:29:54.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.920+0000 7f6bd7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd80b1b30 con 0x7f6bd80d3100 2024-04-24T06:29:54.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.920+0000 7f6bd7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd80b2740 con 0x7f6bd815e760 2024-04-24T06:29:54.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.921+0000 7f6bd7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6bd80d4b60 0x7f6bd80b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48904/0 (socket says 172.21.15.59:48904) 2024-04-24T06:29:54.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.921+0000 7f6bd7400640 1 -- 172.21.15.59:0/185009214 learned_addr learned my addr 172.21.15.59:0/185009214 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.921+0000 7f6bd6a00640 1 --2- 172.21.15.59:0/185009214 >> v2:172.21.15.66:3300/0 conn(0x7f6bd815e760 0x7f6bd80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.921+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825647270 0 0) 0x7f6bd0001490 con 0x7f6bd80d4b60 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.921+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bbc003520 con 0x7f6bd80d4b60 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 820854293 0 0) 0x7f6bbc003520 con 0x7f6bd80d4b60 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 >> v2:172.21.15.66:3300/0 conn(0x7f6bd815e760 msgr2=0x7f6bd80b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 --2- 172.21.15.59:0/185009214 >> v2:172.21.15.66:3300/0 conn(0x7f6bd815e760 0x7f6bd80b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 >> v1:172.21.15.66:6789/0 conn(0x7f6bd80d3100 legacy=0x7f6bd80d4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd80b1e10 con 0x7f6bd80d4b60 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6bd0001bc0 con 0x7f6bd80d4b60 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 >> v1:172.21.15.59:6789/0 conn(0x7f6bd80d4b60 legacy=0x7f6bd80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 0x7f6bbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bbc0051c0 0x7f6bbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80b2740 con 0x7f6bbc004800 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc00c060 con 0x7f6bbc0039b0 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc00c260 con 0x7f6bbc0051c0 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd6000640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 0x7f6bbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd7400640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bbc0051c0 0x7f6bbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.922+0000 7f6bd6a00640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd6000640 1 -- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bbc0051c0 msgr2=0x7f6bbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd6000640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bbc0051c0 0x7f6bbc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd6000640 1 -- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc0039b0 msgr2=0x7f6bbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd6000640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc0039b0 0x7f6bbc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd6000640 1 -- 172.21.15.59:0/185009214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc003520 con 0x7f6bbc004800 2024-04-24T06:29:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.923+0000 7f6bd7400640 1 -- 172.21.15.59:0/185009214 reap_dead start 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.924+0000 7f6bd6000640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 0x7f6bbc004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001750 tx=0x7f6bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8002cc32ef4f2cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.924+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8001370 con 0x7f6bbc004800 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.924+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc80052d0 con 0x7f6bbc004800 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.924+0000 7f6bd5600640 1 -- 172.21.15.59:0/185009214 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc802abc0 con 0x7f6bbc004800 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 -- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 msgr2=0x7f6bbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 0x7f6bbc004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001750 tx=0x7f6bbc005830 comp rx=0 tx=0).stop 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 -- 172.21.15.59:0/185009214 shutdown_connections 2024-04-24T06:29:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/185009214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc004800 0x7f6bbc004c80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 -- 172.21.15.59:0/185009214 >> 172.21.15.59:0/185009214 conn(0x7f6bd809d980 msgr2=0x7f6bd808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:54.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 -- 172.21.15.59:0/185009214 shutdown_connections 2024-04-24T06:29:54.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.925+0000 7f6bd7e00640 1 -- 172.21.15.59:0/185009214 wait complete. 2024-04-24T06:29:54.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.927+0000 7f6bd7e00640 1 Processor -- start 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.927+0000 7f6bd7e00640 1 -- start start 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc004800 0x7f6bd815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bd80cd4f0 0x7f6bd80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd815eb40 con 0x7f6bd80c8250 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd80b23d0 con 0x7f6bbc004800 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd815ecc0 con 0x7f6bd80cd4f0 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51982/0 (socket says 172.21.15.59:51982) 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd6000640 1 -- 172.21.15.59:0/2523390669 learned_addr learned my addr 172.21.15.59:0/2523390669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd7400640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bd80cd4f0 0x7f6bd80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd6a00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc004800 0x7f6bd815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.928+0000 7f6bd6000640 1 -- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bd80cd4f0 msgr2=0x7f6bd80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd6000640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bd80cd4f0 0x7f6bd80cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd6000640 1 -- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc004800 msgr2=0x7f6bd815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd6000640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc004800 0x7f6bd815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd6000640 1 -- 172.21.15.59:0/2523390669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd815c6f0 con 0x7f6bd80c8250 2024-04-24T06:29:54.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd6000640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005040 tx=0x7f6bc8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc803f020 con 0x7f6bd80c8250 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc8005480 con 0x7f6bd80c8250 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8037040 con 0x7f6bd80c8250 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd80ca740 con 0x7f6bd80c8250 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.929+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd80cad00 con 0x7f6bd80c8250 2024-04-24T06:29:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.931+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd804fd90 con 0x7f6bd80c8250 2024-04-24T06:29:54.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.932+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bc8042020 con 0x7f6bd80c8250 2024-04-24T06:29:54.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.932+0000 7f6bcea00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 0x7f6bb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:54.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.933+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f6bc80ab5e0 con 0x7f6bd80c8250 2024-04-24T06:29:54.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.933+0000 7f6bd6a00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 0x7f6bb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:54.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.933+0000 7f6bd6a00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 0x7f6bb406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806a7b0 tx=0x7f6bc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:54.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:54.945+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bc8049030 con 0x7f6bd80c8250 2024-04-24T06:29:55.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.156+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bd80b2fb0 con 0x7f6bd80c8250 2024-04-24T06:29:55.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.679+0000 7f6bcea00640 1 -- 172.21.15.59:0/2523390669 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f6bd815c6f0 con 0x7f6bd80c8250 2024-04-24T06:29:55.680 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:29:55.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 msgr2=0x7f6bb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:55.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 0x7f6bb406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806a7b0 tx=0x7f6bc0008040 comp rx=0 tx=0).stop 2024-04-24T06:29:55.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 msgr2=0x7f6bd80c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:55.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005040 tx=0x7f6bc8001120 comp rx=0 tx=0).stop 2024-04-24T06:29:55.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 shutdown_connections 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6bb406c8f0 0x7f6bb406edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6bd80cd4f0 0x7f6bd80cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bd80c8250 0x7f6bd80c9dd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 --2- 172.21.15.59:0/2523390669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6bbc004800 0x7f6bd815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.683+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 >> 172.21.15.59:0/2523390669 conn(0x7f6bd809d980 msgr2=0x7f6bd808bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.684+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 shutdown_connections 2024-04-24T06:29:55.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.684+0000 7f6bd7e00640 1 -- 172.21.15.59:0/2523390669 wait complete. 2024-04-24T06:29:55.711 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:29:55.711 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:29:55.711 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:29:55.711 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:55.712 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:29:55.712 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:29:55.712 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:29:55.712 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:55.713 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:29:55.713 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:29:55.713 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:29:55.713 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:55.713 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:29:55.713 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:29:55.714 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:29:55.714 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:29:55.733 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:29:55.733 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:29:55.735 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:29:55.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:29:55.737 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:29:55.737 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:29:55.738 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:29:55.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:29:55.783 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:29:55.784 DEBUG:teuthology.parallel:result is None 2024-04-24T06:29:55.785 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:29:55.785 DEBUG:teuthology.parallel:result is None 2024-04-24T06:29:55.790 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:29:55.790 DEBUG:teuthology.parallel:result is None 2024-04-24T06:29:55.791 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:29:55.792 DEBUG:teuthology.parallel:result is None 2024-04-24T06:29:55.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:29:55.937 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:29:56.049 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:29:56.050 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:29:55.939+0000 7f8e33c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-24T06:29:56.056 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:29:56.056 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:29:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.975+0000 7f3e50400640 1 Processor -- start 2024-04-24T06:29:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.975+0000 7f3e50400640 1 -- start start 2024-04-24T06:29:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e4815ec20 0x7f3e480cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480b1e10 0x7f3e480cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e480d00a0 0x7f3e4815f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3e480b2ec0 con 0x7f3e4815ec20 2024-04-24T06:29:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3e480b22b0 con 0x7f3e480d00a0 2024-04-24T06:29:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e50400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3e480b2590 con 0x7f3e480b1e10 2024-04-24T06:29:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e4da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e4815ec20 0x7f3e480cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e4da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e4815ec20 0x7f3e480cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52028/0 (socket says 172.21.15.59:52028) 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.976+0000 7f3e4da00640 1 -- 172.21.15.59:0/4107526094 learned_addr learned my addr 172.21.15.59:0/4107526094 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4e400640 1 --2- 172.21.15.59:0/4107526094 >> v2:172.21.15.66:3300/0 conn(0x7f3e480d00a0 0x7f3e4815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4d000640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480b1e10 0x7f3e480cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4da00640 1 -- 172.21.15.59:0/4107526094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480b1e10 msgr2=0x7f3e480cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:56.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4da00640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480b1e10 0x7f3e480cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4da00640 1 -- 172.21.15.59:0/4107526094 >> v2:172.21.15.66:3300/0 conn(0x7f3e480d00a0 msgr2=0x7f3e4815f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4da00640 1 --2- 172.21.15.59:0/4107526094 >> v2:172.21.15.66:3300/0 conn(0x7f3e480d00a0 0x7f3e4815f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.977+0000 7f3e4da00640 1 -- 172.21.15.59:0/4107526094 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e480b2870 con 0x7f3e4815ec20 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.978+0000 7f3e4da00640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480cf490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3e480b4180 tx=0x7f3e3c02c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0bddc301ebb37f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.978+0000 7f3e47e00640 1 -- 172.21.15.59:0/4107526094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c038070 con 0x7f3e4815ec20 2024-04-24T06:29:56.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.978+0000 7f3e47e00640 1 -- 172.21.15.59:0/4107526094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c031710 con 0x7f3e4815ec20 2024-04-24T06:29:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.978+0000 7f3e47e00640 1 -- 172.21.15.59:0/4107526094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c031b20 con 0x7f3e4815ec20 2024-04-24T06:29:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 -- 172.21.15.59:0/4107526094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 msgr2=0x7f3e480cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480cf490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3e480b4180 tx=0x7f3e3c02c400 comp rx=0 tx=0).stop 2024-04-24T06:29:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 -- 172.21.15.59:0/4107526094 shutdown_connections 2024-04-24T06:29:56.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 --2- 172.21.15.59:0/4107526094 >> v2:172.21.15.66:3300/0 conn(0x7f3e480d00a0 0x7f3e4815f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480b1e10 0x7f3e480cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 --2- 172.21.15.59:0/4107526094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480cf490 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 -- 172.21.15.59:0/4107526094 >> 172.21.15.59:0/4107526094 conn(0x7f3e4809aa00 msgr2=0x7f3e480b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 -- 172.21.15.59:0/4107526094 shutdown_connections 2024-04-24T06:29:56.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.979+0000 7f3e50400640 1 -- 172.21.15.59:0/4107526094 wait complete. 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.981+0000 7f3e50400640 1 Processor -- start 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.981+0000 7f3e50400640 1 -- start start 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 0x7f3e4815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480d00a0 0x7f3e480d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4815f180 con 0x7f3e4815ec20 2024-04-24T06:29:56.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4815f000 con 0x7f3e480b1e10 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e50400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3e480b0f20 con 0x7f3e480d00a0 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e4e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e4e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52040/0 (socket says 172.21.15.59:52040) 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e4e400640 1 -- 172.21.15.59:0/890261631 learned_addr learned my addr 172.21.15.59:0/890261631 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e4da00640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 0x7f3e4815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.982+0000 7f3e4d000640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480d00a0 0x7f3e480d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 -- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480d00a0 msgr2=0x7f3e480d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480d00a0 0x7f3e480d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 -- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 msgr2=0x7f3e4815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 0x7f3e4815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 -- 172.21.15.59:0/890261631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481fc430 con 0x7f3e4815ec20 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4da00640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 0x7f3e4815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e4e400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007e40 tx=0x7f3e4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.983+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40014620 con 0x7f3e4815ec20 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.984+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e40019070 con 0x7f3e4815ec20 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.984+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40014a20 con 0x7f3e4815ec20 2024-04-24T06:29:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.984+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481fc710 con 0x7f3e4815ec20 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.984+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481fccd0 con 0x7f3e4815ec20 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.985+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e48068a80 con 0x7f3e4815ec20 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.986+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e40014bc0 con 0x7f3e4815ec20 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.987+0000 7f3e45600640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.987+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f3e4008bff0 con 0x7f3e4815ec20 2024-04-24T06:29:56.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.989+0000 7f3e4da00640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.989+0000 7f3e4da00640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 0x7f3e2806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c035000 tx=0x7f3e3c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:55.992+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e4005a0c0 con 0x7f3e4815ec20 2024-04-24T06:29:56.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.219+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3e48073180 con 0x7f3e4815ec20 2024-04-24T06:29:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.706+0000 7f3e45600640 1 -- 172.21.15.59:0/890261631 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f3e4005e3c0 con 0x7f3e4815ec20 2024-04-24T06:29:56.707 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.709+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 msgr2=0x7f3e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.709+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 0x7f3e2806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c035000 tx=0x7f3e3c008030 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.709+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 msgr2=0x7f3e480d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.709+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e40007e40 tx=0x7f3e4000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 shutdown_connections 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e2806c8f0 0x7f3e2806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e4815ec20 0x7f3e480d19e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e480d00a0 0x7f3e480d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 --2- 172.21.15.59:0/890261631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e480b1e10 0x7f3e4815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 >> 172.21.15.59:0/890261631 conn(0x7f3e4809aa00 msgr2=0x7f3e480aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 shutdown_connections 2024-04-24T06:29:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.710+0000 7f3e50400640 1 -- 172.21.15.59:0/890261631 wait complete. 2024-04-24T06:29:56.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:29:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.937+0000 7f95c1c00640 1 Processor -- start 2024-04-24T06:29:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.937+0000 7f95c1c00640 1 -- start start 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95c1c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 0x7f95bc145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95c1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95bc0cd020 0x7f95bc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95c1c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc0cd520 con 0x7f95bc0b3e40 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95c1c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f95bc0cddc0 con 0x7f95bc0cd020 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95c1c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc0cd800 con 0x7f95bc1454e0 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95ba000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95bc0cd020 0x7f95bc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95ba000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95bc0cd020 0x7f95bc0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52042/0 (socket says 172.21.15.59:52042) 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.939+0000 7f95ba000640 1 -- 172.21.15.59:0/3403950357 learned_addr learned my addr 172.21.15.59:0/3403950357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95baa00640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 0x7f95bc145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95ba000640 1 -- 172.21.15.59:0/3403950357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 msgr2=0x7f95bc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95ba000640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 0x7f95bc145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95ba000640 1 -- 172.21.15.59:0/3403950357 >> v1:172.21.15.66:6789/0 conn(0x7f95bc0b3e40 legacy=0x7f95bc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95ba000640 1 -- 172.21.15.59:0/3403950357 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0cdae0 con 0x7f95bc0cd020 2024-04-24T06:29:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.940+0000 7f95baa00640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 0x7f95bc145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:56.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.941+0000 7f95ba000640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0cd020 0x7f95bc0b3900 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009870 tx=0x7f95a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1be3b40396fa4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.941+0000 7f95b9600640 1 -- 172.21.15.59:0/3403950357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4037070 con 0x7f95bc0cd020 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.941+0000 7f95b9600640 1 -- 172.21.15.59:0/3403950357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4032440 con 0x7f95bc0cd020 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.941+0000 7f95b9600640 1 -- 172.21.15.59:0/3403950357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4032810 con 0x7f95bc0cd020 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 -- 172.21.15.59:0/3403950357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0cd020 msgr2=0x7f95bc0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0cd020 0x7f95bc0b3900 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009870 tx=0x7f95a402a180 comp rx=0 tx=0).stop 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 -- 172.21.15.59:0/3403950357 shutdown_connections 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0cd020 0x7f95bc0b3900 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3403950357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc1454e0 0x7f95bc145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 -- 172.21.15.59:0/3403950357 >> 172.21.15.59:0/3403950357 conn(0x7f95bc067ea0 msgr2=0x7f95bc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 -- 172.21.15.59:0/3403950357 shutdown_connections 2024-04-24T06:29:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.942+0000 7f95c1c00640 1 -- 172.21.15.59:0/3403950357 wait complete. 2024-04-24T06:29:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.944+0000 7f95c1c00640 1 Processor -- start 2024-04-24T06:29:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.944+0000 7f95c1c00640 1 -- start start 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc0cd020 0x7f95bc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95bc1454e0 0x7f95bc13ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc146270 con 0x7f95bc0b3e40 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1460f0 con 0x7f95bc1454e0 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95c1c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95bc161020 con 0x7f95bc0cd020 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52052/0 (socket says 172.21.15.59:52052) 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.945+0000 7f95baa00640 1 -- 172.21.15.59:0/3561405929 learned_addr learned my addr 172.21.15.59:0/3561405929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 -- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc0cd020 msgr2=0x7f95bc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95ba000640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc0cd020 0x7f95bc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc0cd020 0x7f95bc0ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 -- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95bc1454e0 msgr2=0x7f95bc13ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:56.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95bc1454e0 0x7f95bc13ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:56.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 -- 172.21.15.59:0/3561405929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc06fe20 con 0x7f95bc0b3e40 2024-04-24T06:29:56.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95baa00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c820 tx=0x7f95ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac015070 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac007a30 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac007de0 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.946+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc0adf30 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.947+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc0ae4f0 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.948+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc068a80 con 0x7f95bc0b3e40 2024-04-24T06:29:56.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.949+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95ac018070 con 0x7f95bc0b3e40 2024-04-24T06:29:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.949+0000 7f95b2a00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.950+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f95ac08b740 con 0x7f95bc0b3e40 2024-04-24T06:29:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.950+0000 7f95ba000640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.950+0000 7f95ba000640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034000 tx=0x7f95a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:56.954+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95ac059e20 con 0x7f95bc0b3e40 2024-04-24T06:29:57.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.234+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f95bc075150 con 0x7f95bc0b3e40 2024-04-24T06:29:57.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.235+0000 7f95b2a00640 1 -- 172.21.15.59:0/3561405929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f95ac059720 con 0x7f95bc0b3e40 2024-04-24T06:29:57.236 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.237 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:29:57.238 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:29:57.239 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.240 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.241 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:29:57.243 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:29:57.245 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:29:57.246 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:29:57.248 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mds", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihmt0YDNxAAc2kiEJQGSiYR8zJXuul7ag==", 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.249 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mds" 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mgr", 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihm+VcDNxAAj2pslhXN17B+yFp3PVFSuw==", 2024-04-24T06:29:57.250 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.251 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-24T06:29:57.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.251 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-osd", 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihmiWUDNxAA4eeuAJQxJbh3l9ZZPNLF0w==", 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-osd" 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd", 2024-04-24T06:29:57.252 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihm8nIDNxAA7DMCV/H8UACWNTS7lk+k2w==", 2024-04-24T06:29:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-24T06:29:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihma4ADNxAAc3VTmmweLjlQlZdY6V57zw==", 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.255 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rgw", 2024-04-24T06:29:57.255 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQCDpihmL5IDNxAAYpLsQ1tE96mKC/TcxkNdYQ==", 2024-04-24T06:29:57.255 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:29:57.256 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:29:57.257 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.257 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:29:57.257 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:29:57.258 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.258 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:29:57.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:29:57.259 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:29:57.259 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:29:57.259 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:29:57.260 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:29:57.260 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:29:57.260 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:29:57.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.261 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:29:57.261 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:29:57.261 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:29:57.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 msgr2=0x7f959806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034000 tx=0x7f95a402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:57.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 msgr2=0x7f95bc15f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c820 tx=0x7f95ac00cce0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 shutdown_connections 2024-04-24T06:29:57.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95bc1454e0 0x7f95bc13ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95bc0cd020 0x7f95bc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 --2- 172.21.15.59:0/3561405929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95bc0b3e40 0x7f95bc15f330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.238+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 >> 172.21.15.59:0/3561405929 conn(0x7f95bc067ea0 msgr2=0x7f95bc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:57.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.239+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 shutdown_connections 2024-04-24T06:29:57.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.239+0000 7f95c1c00640 1 -- 172.21.15.59:0/3561405929 wait complete. 2024-04-24T06:29:57.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-24T06:29:57.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.485+0000 7fe369000640 1 Processor -- start 2024-04-24T06:29:57.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.486+0000 7fe369000640 1 -- start start 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.487+0000 7fe369000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 0x7fe3640aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.487+0000 7fe369000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe364162de0 0x7fe36406ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe369000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3641632e0 con 0x7fe364161e60 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe369000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe3640aefd0 con 0x7fe364162de0 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe369000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640adcd0 con 0x7fe3640b44c0 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe361e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe364161e60 0x7fe3640b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48922/0 (socket says 172.21.15.59:48922) 2024-04-24T06:29:57.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe361e00640 1 -- 172.21.15.59:0/3926084950 learned_addr learned my addr 172.21.15.59:0/3926084950 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe361400640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 0x7fe3640aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe362800640 1 --2- 172.21.15.59:0/3926084950 >> v2:172.21.15.66:3300/0 conn(0x7fe364162de0 0x7fe36406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.488+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024839572 0 0) 0x7fe3641632e0 con 0x7fe364161e60 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe348003520 con 0x7fe364161e60 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4239631948 0 0) 0x7fe348003520 con 0x7fe364161e60 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 msgr2=0x7fe3640aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 0x7fe3640aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 >> v2:172.21.15.66:3300/0 conn(0x7fe364162de0 msgr2=0x7fe36406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 --2- 172.21.15.59:0/3926084950 >> v2:172.21.15.66:3300/0 conn(0x7fe364162de0 0x7fe36406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640adfb0 con 0x7fe364161e60 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe3540019f0 con 0x7fe364161e60 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 >> v1:172.21.15.59:6789/0 conn(0x7fe364161e60 legacy=0x7fe3640b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe348004800 0x7fe348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 0x7fe3480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640aefd0 con 0x7fe3480039b0 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640adcd0 con 0x7fe348004800 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe34800ba70 con 0x7fe3640b44c0 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.489+0000 7fe361e00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.490+0000 7fe361e00640 1 -- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 msgr2=0x7fe3480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.490+0000 7fe361e00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640b44c0 0x7fe3480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.490+0000 7fe361e00640 1 -- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe348004800 msgr2=0x7fe348004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.490+0000 7fe361e00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe348004800 0x7fe348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.490+0000 7fe361e00640 1 -- 172.21.15.59:0/3926084950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348003520 con 0x7fe3480039b0 2024-04-24T06:29:57.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.491+0000 7fe361e00640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe348003dd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe35400a870 tx=0x7fe348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c50ac3937a846a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.491+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35400a4c0 con 0x7fe3480039b0 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.491+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35402f070 con 0x7fe3480039b0 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.491+0000 7fe360a00640 1 -- 172.21.15.59:0/3926084950 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35402bc10 con 0x7fe3480039b0 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 -- 172.21.15.59:0/3926084950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 msgr2=0x7fe348003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 --2- 172.21.15.59:0/3926084950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe348003dd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe35400a870 tx=0x7fe348005830 comp rx=0 tx=0).stop 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe362800640 1 -- 172.21.15.59:0/3926084950 reap_dead start 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 -- 172.21.15.59:0/3926084950 shutdown_connections 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 -- 172.21.15.59:0/3926084950 >> 172.21.15.59:0/3926084950 conn(0x7fe3640b2130 msgr2=0x7fe3640b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 -- 172.21.15.59:0/3926084950 shutdown_connections 2024-04-24T06:29:57.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.492+0000 7fe369000640 1 -- 172.21.15.59:0/3926084950 wait complete. 2024-04-24T06:29:57.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.494+0000 7fe369000640 1 Processor -- start 2024-04-24T06:29:57.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.494+0000 7fe369000640 1 -- start start 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640d0dd0 0x7fe364153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe364151290 0x7fe364151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640af480 con 0x7fe3480039b0 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640ae570 con 0x7fe364151290 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe369000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640af600 con 0x7fe3640d0dd0 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe361e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe361e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52080/0 (socket says 172.21.15.59:52080) 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.495+0000 7fe361e00640 1 -- 172.21.15.59:0/1992057084 learned_addr learned my addr 172.21.15.59:0/1992057084 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:57.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361400640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640d0dd0 0x7fe364153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 -- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640d0dd0 msgr2=0x7fe364153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640d0dd0 0x7fe364153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 -- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe364151290 msgr2=0x7fe364151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe364151290 0x7fe364151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 -- 172.21.15.59:0/1992057084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640d0140 con 0x7fe3480039b0 2024-04-24T06:29:57.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe361e00640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe35400a8d0 tx=0x7fe354001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354044070 con 0x7fe3480039b0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.496+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe354004b60 con 0x7fe3480039b0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.497+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354002010 con 0x7fe3480039b0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.497+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe364154510 con 0x7fe3480039b0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.497+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36414e9c0 con 0x7fe3480039b0 2024-04-24T06:29:57.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.498+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe364068a80 con 0x7fe3480039b0 2024-04-24T06:29:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.499+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe354047070 con 0x7fe3480039b0 2024-04-24T06:29:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.500+0000 7fe35a000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 0x7fe34006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.500+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fe3540ab7f0 con 0x7fe3480039b0 2024-04-24T06:29:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.500+0000 7fe361400640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 0x7fe34006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.501+0000 7fe361400640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 0x7fe34006ee30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe364143c20 tx=0x7fe34c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:57.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.504+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe354079fe0 con 0x7fe3480039b0 2024-04-24T06:29:57.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.790+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fe364069190 con 0x7fe3480039b0 2024-04-24T06:29:57.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.794+0000 7fe35a000640 1 -- 172.21.15.59:0/1992057084 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fe35407ee20 con 0x7fe3480039b0 2024-04-24T06:29:57.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 msgr2=0x7fe34006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 0x7fe34006ee30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe364143c20 tx=0x7fe34c008040 comp rx=0 tx=0).stop 2024-04-24T06:29:57.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 msgr2=0x7fe3640cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe35400a8d0 tx=0x7fe354001dc0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 shutdown_connections 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe34006c970 0x7fe34006ee30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe364151290 0x7fe364151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3640d0dd0 0x7fe364153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 --2- 172.21.15.59:0/1992057084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3480039b0 0x7fe3640cfc00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 >> 172.21.15.59:0/1992057084 conn(0x7fe3640b2130 msgr2=0x7fe36406ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 shutdown_connections 2024-04-24T06:29:57.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:57.797+0000 7fe369000640 1 -- 172.21.15.59:0/1992057084 wait complete. 2024-04-24T06:29:57.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.013+0000 7f81a4600640 1 Processor -- start 2024-04-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.013+0000 7f81a4600640 1 -- start start 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a4600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 0x7f819c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a4600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f819c06ddf0 0x7f819c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a4600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819c15dda0 con 0x7f819c072680 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a4600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f819c0cd880 con 0x7f819c06ddf0 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a4600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f819c071ec0 con 0x7f819c0cf410 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a2600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f819c072680 0x7f819c0cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48938/0 (socket says 172.21.15.59:48938) 2024-04-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a2600640 1 -- 172.21.15.59:0/2735628829 learned_addr learned my addr 172.21.15.59:0/2735628829 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:58.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a1c00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 0x7f819c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.015+0000 7f81a1200640 1 --2- 172.21.15.59:0/2735628829 >> v2:172.21.15.66:3300/0 conn(0x7f819c06ddf0 0x7f819c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162008793 0 0) 0x7f8194001490 con 0x7f819c072680 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8184003520 con 0x7f819c072680 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2679650533 0 0) 0x7f8184003520 con 0x7f819c072680 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 msgr2=0x7f819c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 0x7f819c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 >> v2:172.21.15.66:3300/0 conn(0x7f819c06ddf0 msgr2=0x7f819c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 --2- 172.21.15.59:0/2735628829 >> v2:172.21.15.66:3300/0 conn(0x7f819c06ddf0 0x7f819c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0cdb20 con 0x7f819c072680 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8194001bc0 con 0x7f819c072680 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 >> v1:172.21.15.59:6789/0 conn(0x7f819c072680 legacy=0x7f819c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81840039b0 0x7f8184003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8184004800 0x7f8184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 0x7f81840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0cd880 con 0x7f81840039b0 2024-04-24T06:29:58.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f819c071ec0 con 0x7f8184004800 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f818400bac0 con 0x7f819c0cf410 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.016+0000 7f81a1200640 1 --2- 172.21.15.59:0/2735628829 >> v2:172.21.15.66:3300/0 conn(0x7f819c06ddf0 0x7f819c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81840039b0 0x7f8184003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 -- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 msgr2=0x7f81840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c0cf410 0x7f81840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 -- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8184004800 msgr2=0x7f8184004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8184004800 0x7f8184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.017+0000 7f81a1c00640 1 -- 172.21.15.59:0/2735628829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8184003520 con 0x7f81840039b0 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.018+0000 7f81a1c00640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81840039b0 0x7f8184003dd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ac0 tx=0x7f8184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab77fbcff6c040f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.018+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190036070 con 0x7f81840039b0 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8190031400 con 0x7f81840039b0 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f819be00640 1 -- 172.21.15.59:0/2735628829 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190009690 con 0x7f81840039b0 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a4600640 1 -- 172.21.15.59:0/2735628829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81840039b0 msgr2=0x7f8184003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a4600640 1 --2- 172.21.15.59:0/2735628829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81840039b0 0x7f8184003dd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ac0 tx=0x7f8184005830 comp rx=0 tx=0).stop 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a2600640 1 -- 172.21.15.59:0/2735628829 reap_dead start 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a4600640 1 -- 172.21.15.59:0/2735628829 shutdown_connections 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a4600640 1 -- 172.21.15.59:0/2735628829 >> 172.21.15.59:0/2735628829 conn(0x7f819c0b2130 msgr2=0x7f819c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.019+0000 7f81a4600640 1 -- 172.21.15.59:0/2735628829 shutdown_connections 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.020+0000 7f81a4600640 1 -- 172.21.15.59:0/2735628829 wait complete. 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.021+0000 7f81a4600640 1 Processor -- start 2024-04-24T06:29:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.021+0000 7f81a4600640 1 -- start start 2024-04-24T06:29:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81840039b0 0x7f819c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c1559e0 0x7f819c1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f819c15a6a0 con 0x7f819c153810 2024-04-24T06:29:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0cde00 con 0x7f81840039b0 2024-04-24T06:29:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.022+0000 7f81a4600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f819c15a820 con 0x7f819c1559e0 2024-04-24T06:29:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52094/0 (socket says 172.21.15.59:52094) 2024-04-24T06:29:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 -- 172.21.15.59:0/3749255208 learned_addr learned my addr 172.21.15.59:0/3749255208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a1200640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c1559e0 0x7f819c1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a1c00640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81840039b0 0x7f819c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 -- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c1559e0 msgr2=0x7f819c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c1559e0 0x7f819c1532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 -- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81840039b0 msgr2=0x7f819c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81840039b0 0x7f819c0ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.023+0000 7f81a2600640 1 -- 172.21.15.59:0/3749255208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0adf70 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f81a2600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f819c0507b0 tx=0x7f819400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194004d60 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819400a610 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819400aa00 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c0ae250 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.024+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c0ae810 con 0x7f819c153810 2024-04-24T06:29:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.025+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c068a80 con 0x7f819c153810 2024-04-24T06:29:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.026+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8194011070 con 0x7f819c153810 2024-04-24T06:29:58.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.027+0000 7f8199600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 0x7f817c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.027+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f819408b150 con 0x7f819c153810 2024-04-24T06:29:58.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.028+0000 7f81a1c00640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 0x7f817c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.028+0000 7f81a1c00640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 0x7f817c06ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8190033000 tx=0x7f8190008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.032+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f819c15dda0 con 0x7f819c153810 2024-04-24T06:29:58.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.298+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f819c0700b0 con 0x7f819c153810 2024-04-24T06:29:58.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.304+0000 7f8199600640 1 -- 172.21.15.59:0/3749255208 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f8194059830 con 0x7f819c153810 2024-04-24T06:29:58.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 msgr2=0x7f817c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 0x7f817c06ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8190033000 tx=0x7f8190008030 comp rx=0 tx=0).stop 2024-04-24T06:29:58.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 msgr2=0x7f819c153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f819c0507b0 tx=0x7f819400cb80 comp rx=0 tx=0).stop 2024-04-24T06:29:58.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 shutdown_connections 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f817c06c920 0x7f817c06ede0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819c153810 0x7f819c153c90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f819c1559e0 0x7f819c1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 --2- 172.21.15.59:0/3749255208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81840039b0 0x7f819c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 >> 172.21.15.59:0/3749255208 conn(0x7f819c0b2130 msgr2=0x7f819c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 shutdown_connections 2024-04-24T06:29:58.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.306+0000 7f81a4600640 1 -- 172.21.15.59:0/3749255208 wait complete. 2024-04-24T06:29:58.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-24T06:29:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.524+0000 7fa8c1800640 1 Processor -- start 2024-04-24T06:29:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.524+0000 7fa8c1800640 1 -- start start 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.525+0000 7fa8c1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa8bc1416e0 0x7fa8bc15f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.525+0000 7fa8c1800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8bc15c250 0x7fa8bc1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.525+0000 7fa8c1800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc15ba70 con 0x7fa8bc0d05c0 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.525+0000 7fa8c1800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc15bdc0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.525+0000 7fa8c1800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc15f5a0 con 0x7fa8bc15c250 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa8bc0d05c0 0x7fa8bc0d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48950/0 (socket says 172.21.15.59:48950) 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bbe00640 1 -- 172.21.15.59:0/1650629670 learned_addr learned my addr 172.21.15.59:0/1650629670 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bb400640 1 --2- 172.21.15.59:0/1650629670 >> v2:172.21.15.59:3300/0 conn(0x7fa8bc1416e0 0x7fa8bc15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8c0e00640 1 --2- 172.21.15.59:0/1650629670 >> v2:172.21.15.66:3300/0 conn(0x7fa8bc15c250 0x7fa8bc1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bb400640 1 -- 172.21.15.59:0/1650629670 >> v2:172.21.15.66:3300/0 conn(0x7fa8bc15c250 msgr2=0x7fa8bc1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:58.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bb400640 1 --2- 172.21.15.59:0/1650629670 >> v2:172.21.15.66:3300/0 conn(0x7fa8bc15c250 0x7fa8bc1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8baa00640 1 -- 172.21.15.59:0/1650629670 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968408058 0 0) 0x7fa8bc15ba70 con 0x7fa8bc0d05c0 2024-04-24T06:29:58.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bb400640 1 -- 172.21.15.59:0/1650629670 >> v1:172.21.15.59:6789/0 conn(0x7fa8bc0d05c0 legacy=0x7fa8bc0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.526+0000 7fa8bb400640 1 -- 172.21.15.59:0/1650629670 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc15f800 con 0x7fa8bc1416e0 2024-04-24T06:29:58.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8bb400640 1 --2- 172.21.15.59:0/1650629670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc15f060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097a0 tx=0x7fa8a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a749d54a237f3e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8baa00640 1 -- 172.21.15.59:0/1650629670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4037070 con 0x7fa8bc1416e0 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8baa00640 1 -- 172.21.15.59:0/1650629670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4032440 con 0x7fa8bc1416e0 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8baa00640 1 -- 172.21.15.59:0/1650629670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4032810 con 0x7fa8bc1416e0 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8c1800640 1 -- 172.21.15.59:0/1650629670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 msgr2=0x7fa8bc15f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.528+0000 7fa8c1800640 1 --2- 172.21.15.59:0/1650629670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc15f060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097a0 tx=0x7fa8a402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 -- 172.21.15.59:0/1650629670 shutdown_connections 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 --2- 172.21.15.59:0/1650629670 >> v2:172.21.15.66:3300/0 conn(0x7fa8bc15c250 0x7fa8bc1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 --2- 172.21.15.59:0/1650629670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc15f060 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 -- 172.21.15.59:0/1650629670 >> 172.21.15.59:0/1650629670 conn(0x7fa8bc06bdf0 msgr2=0x7fa8bc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 -- 172.21.15.59:0/1650629670 shutdown_connections 2024-04-24T06:29:58.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.529+0000 7fa8c1800640 1 -- 172.21.15.59:0/1650629670 wait complete. 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.530+0000 7fa8c1800640 1 Processor -- start 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 -- start start 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8bc0d05c0 0x7fa8bc15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8bc15c250 0x7fa8bc1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc141b70 con 0x7fa8bc1416e0 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc15fdc0 con 0x7fa8bc15c250 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.531+0000 7fa8c1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc15c660 con 0x7fa8bc0d05c0 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52114/0 (socket says 172.21.15.59:52114) 2024-04-24T06:29:58.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 -- 172.21.15.59:0/99980106 learned_addr learned my addr 172.21.15.59:0/99980106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 -- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8bc0d05c0 msgr2=0x7fa8bc15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8bc0d05c0 0x7fa8bc15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 -- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8bc15c250 msgr2=0x7fa8bc1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8bc15c250 0x7fa8bc1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 -- 172.21.15.59:0/99980106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc14f640 con 0x7fa8bc1416e0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.532+0000 7fa8bb400640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097a0 tx=0x7fa8a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.533+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4037040 con 0x7fa8bc1416e0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.533+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4004030 con 0x7fa8bc1416e0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.533+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4006e10 con 0x7fa8bc1416e0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.533+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc14dfb0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.533+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc14fb90 con 0x7fa8bc1416e0 2024-04-24T06:29:58.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.534+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc15dfd0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.535+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8a4030770 con 0x7fa8bc1416e0 2024-04-24T06:29:58.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.536+0000 7fa8b3e00640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 0x7fa89806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:58.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.536+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa8a40acaa0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.536+0000 7fa8bbe00640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 0x7fa89806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:58.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.537+0000 7fa8bbe00640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 0x7fa89806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac007fd0 tx=0x7fa8ac007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:58.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.541+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8a407b2a0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.807+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa8bc04fd90 con 0x7fa8bc1416e0 2024-04-24T06:29:58.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.813+0000 7fa8b3e00640 1 -- 172.21.15.59:0/99980106 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fa8a407aba0 con 0x7fa8bc1416e0 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.815+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 msgr2=0x7fa89806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.815+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 0x7fa89806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac007fd0 tx=0x7fa8ac007920 comp rx=0 tx=0).stop 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.815+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 msgr2=0x7fa8bc096e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.815+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097a0 tx=0x7fa8a40046c0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 shutdown_connections 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa89806c8f0 0x7fa89806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8bc15c250 0x7fa8bc1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8bc1416e0 0x7fa8bc096e70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 --2- 172.21.15.59:0/99980106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8bc0d05c0 0x7fa8bc15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 >> 172.21.15.59:0/99980106 conn(0x7fa8bc06bdf0 msgr2=0x7fa8bc153c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 shutdown_connections 2024-04-24T06:29:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:58.816+0000 7fa8c1800640 1 -- 172.21.15.59:0/99980106 wait complete. 2024-04-24T06:29:58.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-24T06:29:59.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.035+0000 7f333ae00640 1 Processor -- start 2024-04-24T06:29:59.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.035+0000 7f333ae00640 1 -- start start 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33340cf410 0x7f333415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3334072680 0x7f3334072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f333415dda0 con 0x7f33340cf410 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f33340cd880 con 0x7f3334072680 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f333ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3334071ec0 con 0x7f333406ddf0 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f3333e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33340cf410 0x7f333415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f3333e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33340cf410 0x7f333415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53012/0 (socket says 172.21.15.59:53012) 2024-04-24T06:29:59.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.037+0000 7f3333e00640 1 -- 172.21.15.59:0/4034427401 learned_addr learned my addr 172.21.15.59:0/4034427401 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:59.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f332b400640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3338e00640 1 --2- 172.21.15.59:0/4034427401 >> v2:172.21.15.66:3300/0 conn(0x7f3334072680 0x7f3334072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3333e00640 1 -- 172.21.15.59:0/4034427401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 msgr2=0x7f333406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:59.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3333e00640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3333e00640 1 -- 172.21.15.59:0/4034427401 >> v2:172.21.15.66:3300/0 conn(0x7f3334072680 msgr2=0x7f3334072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:59.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3333e00640 1 --2- 172.21.15.59:0/4034427401 >> v2:172.21.15.66:3300/0 conn(0x7f3334072680 0x7f3334072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.038+0000 7f3333e00640 1 -- 172.21.15.59:0/4034427401 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33340cdb20 con 0x7f33340cf410 2024-04-24T06:29:59.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.039+0000 7f3333e00640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33340cf410 0x7f333415d860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3324009870 tx=0x7f332402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ddf30c63c8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f3333400640 1 -- 172.21.15.59:0/4034427401 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324037070 con 0x7f33340cf410 2024-04-24T06:29:59.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f3333400640 1 -- 172.21.15.59:0/4034427401 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3324032440 con 0x7f33340cf410 2024-04-24T06:29:59.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f3333400640 1 -- 172.21.15.59:0/4034427401 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324032810 con 0x7f33340cf410 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 -- 172.21.15.59:0/4034427401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33340cf410 msgr2=0x7f333415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33340cf410 0x7f333415d860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3324009870 tx=0x7f332402a180 comp rx=0 tx=0).stop 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 -- 172.21.15.59:0/4034427401 shutdown_connections 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 --2- 172.21.15.59:0/4034427401 >> v2:172.21.15.66:3300/0 conn(0x7f3334072680 0x7f3334072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 --2- 172.21.15.59:0/4034427401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33340cf410 0x7f333415d860 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.040+0000 7f333ae00640 1 -- 172.21.15.59:0/4034427401 >> 172.21.15.59:0/4034427401 conn(0x7f33340b2130 msgr2=0x7f33340b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.041+0000 7f333ae00640 1 -- 172.21.15.59:0/4034427401 shutdown_connections 2024-04-24T06:29:59.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.041+0000 7f333ae00640 1 -- 172.21.15.59:0/4034427401 wait complete. 2024-04-24T06:29:59.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.042+0000 7f333ae00640 1 Processor -- start 2024-04-24T06:29:59.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 -- start start 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33340cf410 0x7f333414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f333415a6a0 con 0x7f3334072680 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33340cde00 con 0x7f33340cf410 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.043+0000 7f333ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f333415a820 con 0x7f333406ddf0 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53018/0 (socket says 172.21.15.59:53018) 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 -- 172.21.15.59:0/1870448991 learned_addr learned my addr 172.21.15.59:0/1870448991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f3333e00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f3338e00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33340cf410 0x7f333414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 -- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 msgr2=0x7f333414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 -- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33340cf410 msgr2=0x7f333414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33340cf410 0x7f333414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.044+0000 7f332b400640 1 -- 172.21.15.59:0/1870448991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33341524f0 con 0x7f3334072680 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f332b400640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d800 tx=0x7f331c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c018030 con 0x7f3334072680 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c0096a0 con 0x7f3334072680 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c009870 con 0x7f3334072680 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3334150f90 con 0x7f3334072680 2024-04-24T06:29:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.045+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3334151550 con 0x7f3334072680 2024-04-24T06:29:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.046+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3334072b10 con 0x7f3334072680 2024-04-24T06:29:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.047+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f331c01b070 con 0x7f3334072680 2024-04-24T06:29:59.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.048+0000 7f3330c00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 0x7f331006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.048+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f331c08baf0 con 0x7f3334072680 2024-04-24T06:29:59.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.049+0000 7f3333e00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 0x7f331006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.049+0000 7f3333e00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 0x7f331006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3324034000 tx=0x7f332402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.053+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3334072b10 con 0x7f3334072680 2024-04-24T06:29:59.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.319+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3334068a80 con 0x7f3334072680 2024-04-24T06:29:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.325+0000 7f3330c00640 1 -- 172.21.15.59:0/1870448991 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f33341524f0 con 0x7f3334072680 2024-04-24T06:29:59.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 msgr2=0x7f331006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 0x7f331006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3324034000 tx=0x7f332402e000 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 msgr2=0x7f3334151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d800 tx=0x7f331c00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 shutdown_connections 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f331006c8f0 0x7f331006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33340cf410 0x7f333414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3334072680 0x7f3334151fb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 --2- 172.21.15.59:0/1870448991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f333406ddf0 0x7f333414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.327+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 >> 172.21.15.59:0/1870448991 conn(0x7f33340b2130 msgr2=0x7f33340b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.328+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 shutdown_connections 2024-04-24T06:29:59.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.328+0000 7f333ae00640 1 -- 172.21.15.59:0/1870448991 wait complete. 2024-04-24T06:29:59.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-24T06:29:59.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.557+0000 7f57cfe00640 1 Processor -- start 2024-04-24T06:29:59.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.558+0000 7f57cfe00640 1 -- start start 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57d00d0f80 0x7f57d00cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 0x7f57d00d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d015fd30 con 0x7f57d015ea40 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f57d00cf930 con 0x7f57d00d0f80 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cfe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f57d00cfc10 con 0x7f57d015f880 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.559+0000 7f57cea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57d00d0f80 0x7f57d00cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57d00d0f80 0x7f57d00cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53032/0 (socket says 172.21.15.59:53032) 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 -- 172.21.15.59:0/1977793181 learned_addr learned my addr 172.21.15.59:0/1977793181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57ce000640 1 --1- 172.21.15.59:0/1977793181 >> v1:172.21.15.59:6789/0 conn(0x7f57d015ea40 0x7f57d00d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57812/0 (socket says 172.21.15.59:57812) 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cf400640 1 --2- 172.21.15.59:0/1977793181 >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 0x7f57d00d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 -- 172.21.15.59:0/1977793181 >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 msgr2=0x7f57d00d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 --2- 172.21.15.59:0/1977793181 >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 0x7f57d00d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 -- 172.21.15.59:0/1977793181 >> v1:172.21.15.59:6789/0 conn(0x7f57d015ea40 legacy=0x7f57d00d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cea00640 1 -- 172.21.15.59:0/1977793181 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d00cfef0 con 0x7f57d00d0f80 2024-04-24T06:29:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.560+0000 7f57cf400640 1 --2- 172.21.15.59:0/1977793181 >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 0x7f57d00d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:29:59.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.561+0000 7f57cea00640 1 --2- 172.21.15.59:0/1977793181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d00d0f80 0x7f57d00cf250 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f57d00ceb70 tx=0x7f57c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac7b4236ee899e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.561+0000 7f57cd600640 1 -- 172.21.15.59:0/1977793181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0036070 con 0x7f57d00d0f80 2024-04-24T06:29:59.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.561+0000 7f57cd600640 1 -- 172.21.15.59:0/1977793181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0031600 con 0x7f57d00d0f80 2024-04-24T06:29:59.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.561+0000 7f57cd600640 1 -- 172.21.15.59:0/1977793181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00319d0 con 0x7f57d00d0f80 2024-04-24T06:29:59.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 -- 172.21.15.59:0/1977793181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d00d0f80 msgr2=0x7f57d00cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 --2- 172.21.15.59:0/1977793181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d00d0f80 0x7f57d00cf250 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f57d00ceb70 tx=0x7f57c002a6a0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 -- 172.21.15.59:0/1977793181 shutdown_connections 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 --2- 172.21.15.59:0/1977793181 >> v2:172.21.15.66:3300/0 conn(0x7f57d015f880 0x7f57d00d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 --2- 172.21.15.59:0/1977793181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d00d0f80 0x7f57d00cf250 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 -- 172.21.15.59:0/1977793181 >> 172.21.15.59:0/1977793181 conn(0x7f57d008bf90 msgr2=0x7f57d00d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 -- 172.21.15.59:0/1977793181 shutdown_connections 2024-04-24T06:29:59.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.562+0000 7f57cfe00640 1 -- 172.21.15.59:0/1977793181 wait complete. 2024-04-24T06:29:59.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.564+0000 7f57cfe00640 1 Processor -- start 2024-04-24T06:29:59.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.564+0000 7f57cfe00640 1 -- start start 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57d00d0f80 0x7f57d0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d015ea40 0x7f57d006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d2850 con 0x7f57d015f880 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d04b0 con 0x7f57d015ea40 2024-04-24T06:29:59.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57d00d29d0 con 0x7f57d00d0f80 2024-04-24T06:29:59.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53048/0 (socket says 172.21.15.59:53048) 2024-04-24T06:29:59.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.565+0000 7f57cf400640 1 -- 172.21.15.59:0/123874303 learned_addr learned my addr 172.21.15.59:0/123874303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57ce000640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d015ea40 0x7f57d006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 -- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57d00d0f80 msgr2=0x7f57d0158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57d00d0f80 0x7f57d0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 -- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d015ea40 msgr2=0x7f57d006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d015ea40 0x7f57d006d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 -- 172.21.15.59:0/123874303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d0159340 con 0x7f57d015f880 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cf400640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c7c0 tx=0x7f57c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8015020 con 0x7f57d015f880 2024-04-24T06:29:59.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c8004230 con 0x7f57d015f880 2024-04-24T06:29:59.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8007620 con 0x7f57d015f880 2024-04-24T06:29:59.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.566+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d00b3da0 con 0x7f57d015f880 2024-04-24T06:29:59.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.567+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d00ccc10 con 0x7f57d015f880 2024-04-24T06:29:59.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.568+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d004fd90 con 0x7f57d015f880 2024-04-24T06:29:59.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.569+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57c8018020 con 0x7f57d015f880 2024-04-24T06:29:59.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.570+0000 7f57c6a00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:29:59.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.570+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f57c808b860 con 0x7f57d015f880 2024-04-24T06:29:59.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.570+0000 7f57cea00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:29:59.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.571+0000 7f57cea00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 0x7f57ac06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57c0033000 tx=0x7f57c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:29:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.578+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57c801f020 con 0x7f57d015f880 2024-04-24T06:29:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.862+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f57d006ead0 con 0x7f57d015f880 2024-04-24T06:29:59.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.868+0000 7f57c6a00640 1 -- 172.21.15.59:0/123874303 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f57c805a090 con 0x7f57d015f880 2024-04-24T06:29:59.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 msgr2=0x7f57ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 0x7f57ac06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57c0033000 tx=0x7f57c002e000 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 msgr2=0x7f57d006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c7c0 tx=0x7f57c800cc80 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 shutdown_connections 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d015f880 0x7f57d006db40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d015ea40 0x7f57d006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 --2- 172.21.15.59:0/123874303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57d00d0f80 0x7f57d0158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:29:59.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 >> 172.21.15.59:0/123874303 conn(0x7f57d008bf90 msgr2=0x7f57d00b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:29:59.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 shutdown_connections 2024-04-24T06:29:59.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:29:59.871+0000 7f57cfe00640 1 -- 172.21.15.59:0/123874303 wait complete. 2024-04-24T06:29:59.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-24T06:30:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.105+0000 7ff1f8a00640 1 Processor -- start 2024-04-24T06:30:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.106+0000 7ff1f8a00640 1 -- start start 2024-04-24T06:30:00.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.107+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.107+0000 7ff1f8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1f40b4230 0x7ff1f40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.107+0000 7ff1f8a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f40b4cf0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.107+0000 7ff1f8a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f415ca50 con 0x7ff1f40b4230 2024-04-24T06:30:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.107+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40ccb80 con 0x7ff1f415c670 2024-04-24T06:30:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1f3200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff1f40d05c0 0x7ff1f4160160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57818/0 (socket says 172.21.15.59:57818) 2024-04-24T06:30:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1f3200640 1 -- 172.21.15.59:0/843578717 learned_addr learned my addr 172.21.15.59:0/843578717 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/843578717 >> v2:172.21.15.59:3300/0 conn(0x7ff1f40b4230 0x7ff1f40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1f2800640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1f1e00640 1 -- 172.21.15.59:0/843578717 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914255436 0 0) 0x7ff1f40b4cf0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1f1e00640 1 -- 172.21.15.59:0/843578717 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1d8003520 con 0x7ff1f40d05c0 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1e9e00640 1 -- 172.21.15.59:0/843578717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 msgr2=0x7ff1f40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.108+0000 7ff1e9e00640 1 -- 172.21.15.59:0/843578717 >> v1:172.21.15.59:6789/0 conn(0x7ff1f40d05c0 legacy=0x7ff1f4160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.109+0000 7ff1e9e00640 1 -- 172.21.15.59:0/843578717 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40ccdd0 con 0x7ff1f40b4230 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40b4230 0x7ff1f40b4690 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0097a0 tx=0x7ff1dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9b82b964d96520 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1f1e00640 1 -- 172.21.15.59:0/843578717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc037070 con 0x7ff1f40b4230 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1f1e00640 1 -- 172.21.15.59:0/843578717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc032440 con 0x7ff1f40b4230 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1f1e00640 1 -- 172.21.15.59:0/843578717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc032810 con 0x7ff1f40b4230 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1f8a00640 1 -- 172.21.15.59:0/843578717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40b4230 msgr2=0x7ff1f40b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.110+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40b4230 0x7ff1f40b4690 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0097a0 tx=0x7ff1dc02a180 comp rx=0 tx=0).stop 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 -- 172.21.15.59:0/843578717 shutdown_connections 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40b4230 0x7ff1f40b4690 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/843578717 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 -- 172.21.15.59:0/843578717 >> 172.21.15.59:0/843578717 conn(0x7ff1f406bdf0 msgr2=0x7ff1f406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 -- 172.21.15.59:0/843578717 shutdown_connections 2024-04-24T06:30:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.111+0000 7ff1f8a00640 1 -- 172.21.15.59:0/843578717 wait complete. 2024-04-24T06:30:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.113+0000 7ff1f8a00640 1 Processor -- start 2024-04-24T06:30:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.113+0000 7ff1f8a00640 1 -- start start 2024-04-24T06:30:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f40b4230 0x7ff1f414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40b3e60 con 0x7ff1f40d05c0 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40cd390 con 0x7ff1f40b4230 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40b3fe0 con 0x7ff1f415c670 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53076/0 (socket says 172.21.15.59:53076) 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 -- 172.21.15.59:0/3076890020 learned_addr learned my addr 172.21.15.59:0/3076890020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 -- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 msgr2=0x7ff1f415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 -- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f40b4230 msgr2=0x7ff1f414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f40b4230 0x7ff1f414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.114+0000 7ff1e9e00640 1 -- 172.21.15.59:0/3076890020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40b04a0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1e9e00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc02a770 tx=0x7ff1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc037050 con 0x7ff1f40d05c0 2024-04-24T06:30:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc043070 con 0x7ff1f40d05c0 2024-04-24T06:30:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc03cb30 con 0x7ff1f40d05c0 2024-04-24T06:30:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f40b0780 con 0x7ff1f40d05c0 2024-04-24T06:30:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.115+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f40b0d40 con 0x7ff1f40d05c0 2024-04-24T06:30:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.117+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f404fd90 con 0x7ff1f40d05c0 2024-04-24T06:30:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.117+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1dc03ccd0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.118+0000 7ff1eb400640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 0x7ff1d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.119+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff1dc0ac4c0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.119+0000 7ff1f2800640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 0x7ff1d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.120+0000 7ff1f2800640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 0x7ff1d006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4006a90 tx=0x7ff1e4006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.123+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1dc046360 con 0x7ff1f40d05c0 2024-04-24T06:30:00.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.405+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7ff1f40ceab0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.409+0000 7ff1eb400640 1 -- 172.21.15.59:0/3076890020 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7ff1dc07acc0 con 0x7ff1f40d05c0 2024-04-24T06:30:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 msgr2=0x7ff1d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 0x7ff1d006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4006a90 tx=0x7ff1e4006a20 comp rx=0 tx=0).stop 2024-04-24T06:30:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 msgr2=0x7ff1f40c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc02a770 tx=0x7ff1dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 shutdown_connections 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff1d006c920 0x7ff1d006ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff1f415c670 0x7ff1f415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1f40d05c0 0x7ff1f40c9de0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 --2- 172.21.15.59:0/3076890020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f40b4230 0x7ff1f414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 >> 172.21.15.59:0/3076890020 conn(0x7ff1f406bdf0 msgr2=0x7ff1f40acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 shutdown_connections 2024-04-24T06:30:00.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.412+0000 7ff1f8a00640 1 -- 172.21.15.59:0/3076890020 wait complete. 2024-04-24T06:30:00.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.644+0000 7f5fd3e00640 1 Processor -- start 2024-04-24T06:30:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.645+0000 7f5fd3e00640 1 -- start start 2024-04-24T06:30:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.646+0000 7f5fd3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fd415c3d0 0x7f5fd40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.646+0000 7f5fd3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd40cd590 0x7f5fd40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.646+0000 7f5fd3e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd4160210 con 0x7f5fd40b4690 2024-04-24T06:30:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.646+0000 7f5fd3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd41604f0 con 0x7f5fd40cd590 2024-04-24T06:30:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.646+0000 7f5fd3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd41607d0 con 0x7f5fd415c3d0 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd40cd590 0x7f5fd40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fd40cd590 0x7f5fd40b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53090/0 (socket says 172.21.15.59:53090) 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 -- 172.21.15.59:0/889149495 learned_addr learned my addr 172.21.15.59:0/889149495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fd2a00640 1 --2- 172.21.15.59:0/889149495 >> v2:172.21.15.66:3300/0 conn(0x7f5fd415c3d0 0x7f5fd40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 -- 172.21.15.59:0/889149495 >> v2:172.21.15.66:3300/0 conn(0x7f5fd415c3d0 msgr2=0x7f5fd40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 --2- 172.21.15.59:0/889149495 >> v2:172.21.15.66:3300/0 conn(0x7f5fd415c3d0 0x7f5fd40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 -- 172.21.15.59:0/889149495 >> v1:172.21.15.66:6789/0 conn(0x7f5fd40b4690 legacy=0x7f5fd406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.647+0000 7f5fca000640 1 -- 172.21.15.59:0/889149495 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd4160ab0 con 0x7f5fd40cd590 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.648+0000 7f5fca000640 1 --2- 172.21.15.59:0/889149495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40cd590 0x7f5fd40b4150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5fd406d9b0 tx=0x7f5fbc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd5ff54cd9b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd2000640 1 -- 172.21.15.59:0/889149495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc037070 con 0x7f5fd40cd590 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd2000640 1 -- 172.21.15.59:0/889149495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fbc032440 con 0x7f5fd40cd590 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd2000640 1 -- 172.21.15.59:0/889149495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc032810 con 0x7f5fd40cd590 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 -- 172.21.15.59:0/889149495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40cd590 msgr2=0x7f5fd40b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/889149495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40cd590 0x7f5fd40b4150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5fd406d9b0 tx=0x7f5fbc02a230 comp rx=0 tx=0).stop 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 -- 172.21.15.59:0/889149495 shutdown_connections 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/889149495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40cd590 0x7f5fd40b4150 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/889149495 >> v2:172.21.15.66:3300/0 conn(0x7f5fd415c3d0 0x7f5fd40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 -- 172.21.15.59:0/889149495 >> 172.21.15.59:0/889149495 conn(0x7f5fd40b21a0 msgr2=0x7f5fd40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:00.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.649+0000 7f5fd3e00640 1 -- 172.21.15.59:0/889149495 shutdown_connections 2024-04-24T06:30:00.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.650+0000 7f5fd3e00640 1 -- 172.21.15.59:0/889149495 wait complete. 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.651+0000 7f5fd3e00640 1 Processor -- start 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.651+0000 7f5fd3e00640 1 -- start start 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40cd590 0x7f5fd4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd415c3d0 0x7f5fd40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd415c7b0 con 0x7f5fd40b4690 2024-04-24T06:30:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd40b4ac0 con 0x7f5fd40cd590 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.652+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd415c930 con 0x7f5fd415c3d0 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53098/0 (socket says 172.21.15.59:53098) 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 -- 172.21.15.59:0/2347141357 learned_addr learned my addr 172.21.15.59:0/2347141357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 -- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd415c3d0 msgr2=0x7f5fd40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd415c3d0 0x7f5fd40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 -- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40cd590 msgr2=0x7f5fd4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40cd590 0x7f5fd4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 -- 172.21.15.59:0/2347141357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd41596c0 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.653+0000 7f5fd2a00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400d620 tx=0x7f5fc400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.654+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400b620 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.654+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc4019070 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.654+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400ba00 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.654+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd40ad8b0 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.654+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd40ade70 con 0x7f5fd40b4690 2024-04-24T06:30:00.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.655+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd404fd90 con 0x7f5fd40b4690 2024-04-24T06:30:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.656+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fc400bba0 con 0x7f5fd40b4690 2024-04-24T06:30:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.657+0000 7f5fcb400640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 0x7f5fb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.658+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5fc408bc90 con 0x7f5fd40b4690 2024-04-24T06:30:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.658+0000 7f5fca000640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 0x7f5fb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:00.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.659+0000 7f5fca000640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 0x7f5fb006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc034000 tx=0x7f5fbc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:00.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.661+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fd41596c0 con 0x7f5fd40b4690 2024-04-24T06:30:00.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.874+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fd40ad5a0 con 0x7f5fd40b4690 2024-04-24T06:30:00.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.876+0000 7f5fcb400640 1 -- 172.21.15.59:0/2347141357 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12807 (secure 0 0 0) 0x7f5fc4060070 con 0x7f5fd40b4690 2024-04-24T06:30:00.878 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:00.878 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:29:56.719295+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:00.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 msgr2=0x7f5fb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 0x7f5fb006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc034000 tx=0x7f5fbc02c000 comp rx=0 tx=0).stop 2024-04-24T06:30:00.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 msgr2=0x7f5fd40c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400d620 tx=0x7f5fc400dae0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 shutdown_connections 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fb006c8f0 0x7f5fb006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fd415c3d0 0x7f5fd40ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fd40cd590 0x7f5fd4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 --2- 172.21.15.59:0/2347141357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd40b4690 0x7f5fd40c9d30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 >> 172.21.15.59:0/2347141357 conn(0x7f5fd40b21a0 msgr2=0x7f5fd4071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 shutdown_connections 2024-04-24T06:30:00.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:00.879+0000 7f5fd3e00640 1 -- 172.21.15.59:0/2347141357 wait complete. 2024-04-24T06:30:00.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:00.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:01.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.115+0000 7f23e7e00640 1 Processor -- start 2024-04-24T06:30:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.115+0000 7f23e7e00640 1 -- start start 2024-04-24T06:30:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.116+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.116+0000 7f23e7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23e80d0600 0x7f23e80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.116+0000 7f23e7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e8163320 con 0x7f23e8167180 2024-04-24T06:30:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.116+0000 7f23e7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f23e8163650 con 0x7f23e80d0600 2024-04-24T06:30:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.116+0000 7f23e7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23e8072380 con 0x7f23e815d300 2024-04-24T06:30:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23e80d0600 0x7f23e80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23e80d0600 0x7f23e80d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53106/0 (socket says 172.21.15.59:53106) 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 -- 172.21.15.59:0/1675413573 learned_addr learned my addr 172.21.15.59:0/1675413573 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e6a00640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 -- 172.21.15.59:0/1675413573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 msgr2=0x7f23e8071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 -- 172.21.15.59:0/1675413573 >> v1:172.21.15.59:6789/0 conn(0x7f23e8167180 legacy=0x7f23e8163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.117+0000 7f23e7400640 1 -- 172.21.15.59:0/1675413573 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e8072660 con 0x7f23e80d0600 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.118+0000 7f23e6a00640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:30:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.119+0000 7f23e7400640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e80d0600 0x7f23e80d0a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23e81637d0 tx=0x7f23e002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c431c3aa2d85e6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.119+0000 7f23e6000640 1 -- 172.21.15.59:0/1675413573 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0036070 con 0x7f23e80d0600 2024-04-24T06:30:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.119+0000 7f23e6000640 1 -- 172.21.15.59:0/1675413573 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e0031600 con 0x7f23e80d0600 2024-04-24T06:30:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.119+0000 7f23e6000640 1 -- 172.21.15.59:0/1675413573 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00319d0 con 0x7f23e80d0600 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 -- 172.21.15.59:0/1675413573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e80d0600 msgr2=0x7f23e80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e80d0600 0x7f23e80d0a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23e81637d0 tx=0x7f23e002a730 comp rx=0 tx=0).stop 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 -- 172.21.15.59:0/1675413573 shutdown_connections 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e80d0600 0x7f23e80d0a90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 --2- 172.21.15.59:0/1675413573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 -- 172.21.15.59:0/1675413573 >> 172.21.15.59:0/1675413573 conn(0x7f23e806bdf0 msgr2=0x7f23e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 -- 172.21.15.59:0/1675413573 shutdown_connections 2024-04-24T06:30:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.120+0000 7f23e7e00640 1 -- 172.21.15.59:0/1675413573 wait complete. 2024-04-24T06:30:01.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.122+0000 7f23e7e00640 1 Processor -- start 2024-04-24T06:30:01.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.122+0000 7f23e7e00640 1 -- start start 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23e80d0600 0x7f23e80992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23e806e1e0 con 0x7f23e8167180 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81675e0 con 0x7f23e80d0600 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23e806e360 con 0x7f23e815d300 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53112/0 (socket says 172.21.15.59:53112) 2024-04-24T06:30:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.123+0000 7f23e7400640 1 -- 172.21.15.59:0/2893432647 learned_addr learned my addr 172.21.15.59:0/2893432647 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 -- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 msgr2=0x7f23e8158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 -- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23e80d0600 msgr2=0x7f23e80992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e6a00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23e80d0600 0x7f23e80992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23e80d0600 0x7f23e80992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 -- 172.21.15.59:0/2893432647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e809b370 con 0x7f23e8167180 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.124+0000 7f23e7400640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23e0034040 tx=0x7f23e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.125+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0036050 con 0x7f23e8167180 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.125+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e0042070 con 0x7f23e8167180 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.125+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0009510 con 0x7f23e8167180 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.125+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e8158d40 con 0x7f23e8167180 2024-04-24T06:30:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.125+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e80b2080 con 0x7f23e8167180 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.126+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e804fd90 con 0x7f23e8167180 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.127+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23e0007c50 con 0x7f23e8167180 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.128+0000 7f23dea00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 0x7f23c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.128+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f23e00ac770 con 0x7f23e8167180 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.128+0000 7f23e6a00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 0x7f23c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.129+0000 7f23e6a00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 0x7f23c406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d8b0 tx=0x7f23d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.132+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23e8158d40 con 0x7f23e8167180 2024-04-24T06:30:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.335+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23e80cff50 con 0x7f23e8167180 2024-04-24T06:30:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.336+0000 7f23dea00640 1 -- 172.21.15.59:0/2893432647 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f23e007ae50 con 0x7f23e8167180 2024-04-24T06:30:01.337 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 msgr2=0x7f23c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 0x7f23c406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d8b0 tx=0x7f23d8008040 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 msgr2=0x7f23e81f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23e0034040 tx=0x7f23e0004a40 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 shutdown_connections 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23c406c8f0 0x7f23c406edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e8167180 0x7f23e81f86c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23e815d300 0x7f23e8158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 --2- 172.21.15.59:0/2893432647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23e80d0600 0x7f23e80992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.338+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 >> 172.21.15.59:0/2893432647 conn(0x7f23e806bdf0 msgr2=0x7f23e806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:01.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.339+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 shutdown_connections 2024-04-24T06:30:01.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.339+0000 7f23e7e00640 1 -- 172.21.15.59:0/2893432647 wait complete. 2024-04-24T06:30:01.358 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:30:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.567+0000 7fd120800640 1 Processor -- start 2024-04-24T06:30:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.567+0000 7fd120800640 1 -- start start 2024-04-24T06:30:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.569+0000 7fd120800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.569+0000 7fd120800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd118160910 0x7fd118160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.569+0000 7fd120800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1180cd520 con 0x7fd1180cd020 2024-04-24T06:30:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.569+0000 7fd120800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd1180cddc0 con 0x7fd118160910 2024-04-24T06:30:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.569+0000 7fd120800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1180cd800 con 0x7fd1181454e0 2024-04-24T06:30:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.570+0000 7fd11de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.570+0000 7fd11de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:35582/0 (socket says 172.21.15.59:35582) 2024-04-24T06:30:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.570+0000 7fd11de00640 1 -- 172.21.15.59:0/3300335681 learned_addr learned my addr 172.21.15.59:0/3300335681 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.570+0000 7fd11e800640 1 --2- 172.21.15.59:0/3300335681 >> v2:172.21.15.66:3300/0 conn(0x7fd118160910 0x7fd118160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.571+0000 7fd11de00640 1 -- 172.21.15.59:0/3300335681 >> v2:172.21.15.66:3300/0 conn(0x7fd118160910 msgr2=0x7fd118160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.571+0000 7fd11de00640 1 --2- 172.21.15.59:0/3300335681 >> v2:172.21.15.66:3300/0 conn(0x7fd118160910 0x7fd118160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.571+0000 7fd11de00640 1 -- 172.21.15.59:0/3300335681 >> v1:172.21.15.66:6789/0 conn(0x7fd1180cd020 legacy=0x7fd1180b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.571+0000 7fd11de00640 1 -- 172.21.15.59:0/3300335681 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1180cdae0 con 0x7fd1181454e0 2024-04-24T06:30:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.572+0000 7fd11de00640 1 --2- 172.21.15.59:0/3300335681 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009870 tx=0x7fd10c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea110fb363fb269e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.572+0000 7fd11ca00640 1 -- 172.21.15.59:0/3300335681 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c037070 con 0x7fd1181454e0 2024-04-24T06:30:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.572+0000 7fd11ca00640 1 -- 172.21.15.59:0/3300335681 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10c032440 con 0x7fd1181454e0 2024-04-24T06:30:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.572+0000 7fd11ca00640 1 -- 172.21.15.59:0/3300335681 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c032810 con 0x7fd1181454e0 2024-04-24T06:30:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 -- 172.21.15.59:0/3300335681 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 msgr2=0x7fd118145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 --2- 172.21.15.59:0/3300335681 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009870 tx=0x7fd10c02a1c0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 -- 172.21.15.59:0/3300335681 shutdown_connections 2024-04-24T06:30:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 --2- 172.21.15.59:0/3300335681 >> v2:172.21.15.66:3300/0 conn(0x7fd118160910 0x7fd118160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 --2- 172.21.15.59:0/3300335681 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1181454e0 0x7fd118145bb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 -- 172.21.15.59:0/3300335681 >> 172.21.15.59:0/3300335681 conn(0x7fd118067ea0 msgr2=0x7fd11806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 -- 172.21.15.59:0/3300335681 shutdown_connections 2024-04-24T06:30:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.573+0000 7fd120800640 1 -- 172.21.15.59:0/3300335681 wait complete. 2024-04-24T06:30:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.576+0000 7fd120800640 1 Processor -- start 2024-04-24T06:30:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.576+0000 7fd120800640 1 -- start start 2024-04-24T06:30:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1180cd020 0x7fd11815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 0x7fd11815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1180b4010 con 0x7fd118160910 2024-04-24T06:30:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181460f0 con 0x7fd1181454e0 2024-04-24T06:30:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd120800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1181f8ed0 con 0x7fd1180cd020 2024-04-24T06:30:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd11e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd11e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53122/0 (socket says 172.21.15.59:53122) 2024-04-24T06:30:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.577+0000 7fd11e800640 1 -- 172.21.15.59:0/2849610803 learned_addr learned my addr 172.21.15.59:0/2849610803 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11de00640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1180cd020 0x7fd11815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11d400640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 0x7fd11815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 -- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1180cd020 msgr2=0x7fd11815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1180cd020 0x7fd11815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 -- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 msgr2=0x7fd11815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 0x7fd11815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 -- 172.21.15.59:0/2849610803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11815df80 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11d400640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 0x7fd11815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.578+0000 7fd11e800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd11400efc0 tx=0x7fd11400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.579+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114018020 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.579+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd114014480 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.579+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114014870 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.579+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11815ecc0 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.579+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11815f280 con 0x7fd118160910 2024-04-24T06:30:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.580+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd118068a80 con 0x7fd118160910 2024-04-24T06:30:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.581+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd114005d10 con 0x7fd118160910 2024-04-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.582+0000 7fd112000640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 0x7fd0fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.582+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd11408c4c0 con 0x7fd118160910 2024-04-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.582+0000 7fd11de00640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 0x7fd0fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.583+0000 7fd11de00640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 0x7fd0fc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd10c034000 tx=0x7fd10c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:01.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.587+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd114022030 con 0x7fd118160910 2024-04-24T06:30:01.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.799+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd118160030 con 0x7fd118160910 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.800+0000 7fd112000640 1 -- 172.21.15.59:0/2849610803 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7fd114010040 con 0x7fd118160910 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:30:01.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:01.802 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.802+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 msgr2=0x7fd0fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.802+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 0x7fd0fc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd10c034000 tx=0x7fd10c02e000 comp rx=0 tx=0).stop 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.802+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 msgr2=0x7fd118143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.802+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd11400efc0 tx=0x7fd11400c490 comp rx=0 tx=0).stop 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 shutdown_connections 2024-04-24T06:30:01.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0fc06c8f0 0x7fd0fc06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd118160910 0x7fd118143f50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1181454e0 0x7fd11815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 --2- 172.21.15.59:0/2849610803 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1180cd020 0x7fd11815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 >> 172.21.15.59:0/2849610803 conn(0x7fd118067ea0 msgr2=0x7fd118140ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 shutdown_connections 2024-04-24T06:30:01.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:01.803+0000 7fd120800640 1 -- 172.21.15.59:0/2849610803 wait complete. 2024-04-24T06:30:01.825 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:30:01.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:30:02.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.045+0000 7f017e600640 1 Processor -- start 2024-04-24T06:30:02.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.046+0000 7f017e600640 1 -- start start 2024-04-24T06:30:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.047+0000 7f017e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01780cf770 0x7f0178169390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.047+0000 7f017e600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01780cfc20 con 0x7f01780d0fd0 2024-04-24T06:30:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.047+0000 7f017e600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01780cff00 con 0x7f01780b4510 2024-04-24T06:30:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.047+0000 7f017e600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f01780d04c0 con 0x7f01780cf770 2024-04-24T06:30:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.047+0000 7f017d200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f01780d0fd0 0x7f01780d13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57844/0 (socket says 172.21.15.59:57844) 2024-04-24T06:30:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f017d200640 1 -- 172.21.15.59:0/2348369325 learned_addr learned my addr 172.21.15.59:0/2348369325 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f017dc00640 1 --2- 172.21.15.59:0/2348369325 >> v2:172.21.15.59:3300/0 conn(0x7f01780cf770 0x7f0178169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:02.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f0177400640 1 -- 172.21.15.59:0/2348369325 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652758093 0 0) 0x7f01780cfc20 con 0x7f01780d0fd0 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f0177400640 1 -- 172.21.15.59:0/2348369325 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f015c003520 con 0x7f01780d0fd0 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f017dc00640 1 -- 172.21.15.59:0/2348369325 >> v1:172.21.15.66:6789/0 conn(0x7f01780b4510 legacy=0x7f01780cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f017dc00640 1 -- 172.21.15.59:0/2348369325 >> v1:172.21.15.59:6789/0 conn(0x7f01780d0fd0 legacy=0x7f01780d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.048+0000 7f017dc00640 1 -- 172.21.15.59:0/2348369325 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780d01e0 con 0x7f01780cf770 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.049+0000 7f017dc00640 1 --2- 172.21.15.59:0/2348369325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780cf770 0x7f0178169390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d90 tx=0x7f017002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293776e3cbdf75d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:02.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f0177400640 1 -- 172.21.15.59:0/2348369325 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170007740 con 0x7f01780cf770 2024-04-24T06:30:02.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f0177400640 1 -- 172.21.15.59:0/2348369325 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01700078e0 con 0x7f01780cf770 2024-04-24T06:30:02.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f0177400640 1 -- 172.21.15.59:0/2348369325 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170007cb0 con 0x7f01780cf770 2024-04-24T06:30:02.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 -- 172.21.15.59:0/2348369325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780cf770 msgr2=0x7f0178169390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 --2- 172.21.15.59:0/2348369325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780cf770 0x7f0178169390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d90 tx=0x7f017002a660 comp rx=0 tx=0).stop 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 -- 172.21.15.59:0/2348369325 shutdown_connections 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 --2- 172.21.15.59:0/2348369325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780cf770 0x7f0178169390 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 -- 172.21.15.59:0/2348369325 >> 172.21.15.59:0/2348369325 conn(0x7f017808e030 msgr2=0x7f017809d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 -- 172.21.15.59:0/2348369325 shutdown_connections 2024-04-24T06:30:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.050+0000 7f017e600640 1 -- 172.21.15.59:0/2348369325 wait complete. 2024-04-24T06:30:02.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.052+0000 7f017e600640 1 Processor -- start 2024-04-24T06:30:02.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.052+0000 7f017e600640 1 -- start start 2024-04-24T06:30:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01780cf770 0x7f01780ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01780d0fd0 0x7f017806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:02.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f017815ea90 con 0x7f01780b4510 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f01780d22e0 con 0x7f01780d0fd0 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.053+0000 7f017e600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f017815ec10 con 0x7f01780cf770 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53138/0 (socket says 172.21.15.59:53138) 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 -- 172.21.15.59:0/917268461 learned_addr learned my addr 172.21.15.59:0/917268461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f0177e00640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01780cf770 0x7f01780ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017dc00640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01780d0fd0 0x7f017806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 -- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01780cf770 msgr2=0x7f01780ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01780cf770 0x7f01780ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 -- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01780d0fd0 msgr2=0x7f017806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01780d0fd0 0x7f017806d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.054+0000 7f017d200640 1 -- 172.21.15.59:0/917268461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017814bf50 con 0x7f01780b4510 2024-04-24T06:30:02.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f017d200640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f016800c410 tx=0x7f016800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168017070 con 0x7f01780b4510 2024-04-24T06:30:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0168005ce0 con 0x7f01780b4510 2024-04-24T06:30:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680072f0 con 0x7f01780b4510 2024-04-24T06:30:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01780b8b10 con 0x7f01780b4510 2024-04-24T06:30:02.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.055+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01780b90d0 con 0x7f01780b4510 2024-04-24T06:30:02.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.056+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017806e1d0 con 0x7f01780b4510 2024-04-24T06:30:02.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.057+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f016801a070 con 0x7f01780b4510 2024-04-24T06:30:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.058+0000 7f0174c00640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.058+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f016808aca0 con 0x7f01780b4510 2024-04-24T06:30:02.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.059+0000 7f0177e00640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:02.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.059+0000 7f0177e00640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 0x7f015806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01781f7a40 tx=0x7f0164009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:02.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.063+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f017806e1d0 con 0x7f01780b4510 2024-04-24T06:30:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.264+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f017804fd90 con 0x7f01780b4510 2024-04-24T06:30:02.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.839+0000 7f0174c00640 1 -- 172.21.15.59:0/917268461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f01680594d0 con 0x7f01780b4510 2024-04-24T06:30:02.841 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 msgr2=0x7f015806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 0x7f015806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01781f7a40 tx=0x7f0164009040 comp rx=0 tx=0).stop 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 msgr2=0x7f017814ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f016800c410 tx=0x7f016800c8d0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 shutdown_connections 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f015806c8f0 0x7f015806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01780d0fd0 0x7f017806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01780cf770 0x7f01780ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 --2- 172.21.15.59:0/917268461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01780b4510 0x7f017814ba10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:02.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 >> 172.21.15.59:0/917268461 conn(0x7f017808e030 msgr2=0x7f01780d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:02.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.843+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 shutdown_connections 2024-04-24T06:30:02.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:02.844+0000 7f017e600640 1 -- 172.21.15.59:0/917268461 wait complete. 2024-04-24T06:30:02.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:30:03.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.081+0000 7fc45ae00640 1 Processor -- start 2024-04-24T06:30:03.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.082+0000 7fc45ae00640 1 -- start start 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc45ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc454160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc45ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc4540ce880 0x7fc4540b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc45ae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc45406dd40 con 0x7fc4540ccb90 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc45ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc4540b3fe0 con 0x7fc4540ce880 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc45ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc4541607e0 con 0x7fc45406dfa0 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc459a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc4540ccb90 0x7fc4540ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57846/0 (socket says 172.21.15.59:57846) 2024-04-24T06:30:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.083+0000 7fc459a00640 1 -- 172.21.15.59:0/1571893886 learned_addr learned my addr 172.21.15.59:0/1571893886 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc459000640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc454160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095584184 0 0) 0x7fc45406dd40 con 0x7fc4540ccb90 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc45a400640 1 --2- 172.21.15.59:0/1571893886 >> v2:172.21.15.66:3300/0 conn(0x7fc4540ce880 0x7fc4540b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc43c003520 con 0x7fc4540ccb90 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3770707837 0 0) 0x7fc43c003520 con 0x7fc4540ccb90 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 msgr2=0x7fc454160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc454160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 >> v2:172.21.15.66:3300/0 conn(0x7fc4540ce880 msgr2=0x7fc4540b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 --2- 172.21.15.59:0/1571893886 >> v2:172.21.15.66:3300/0 conn(0x7fc4540ce880 0x7fc4540b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454160ac0 con 0x7fc4540ccb90 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc448001670 con 0x7fc4540ccb90 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 >> v1:172.21.15.59:6789/0 conn(0x7fc4540ccb90 legacy=0x7fc4540ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc43c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c003f80 0x7fc43c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc43c004d40 0x7fc43c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4540b3fe0 con 0x7fc43c003f80 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541607e0 con 0x7fc43c004d40 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.084+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c00b9c0 con 0x7fc45406dfa0 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c003f80 0x7fc43c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459a00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc43c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc45a400640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc43c004d40 0x7fc43c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 -- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 msgr2=0x7fc43c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc45406dfa0 0x7fc43c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 -- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc43c004d40 msgr2=0x7fc43c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc43c004d40 0x7fc43c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.085+0000 7fc459000640 1 -- 172.21.15.59:0/1571893886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c003520 con 0x7fc43c003f80 2024-04-24T06:30:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc459000640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c003f80 0x7fc43c004800 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc440009af0 tx=0x7fc43c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9519edd15e67249 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440036070 con 0x7fc43c003f80 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440031400 con 0x7fc43c003f80 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc453e00640 1 -- 172.21.15.59:0/1571893886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440009690 con 0x7fc43c003f80 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc45ae00640 1 -- 172.21.15.59:0/1571893886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c003f80 msgr2=0x7fc43c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.087+0000 7fc45ae00640 1 --2- 172.21.15.59:0/1571893886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c003f80 0x7fc43c004800 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc440009af0 tx=0x7fc43c005830 comp rx=0 tx=0).stop 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.088+0000 7fc45a400640 1 -- 172.21.15.59:0/1571893886 reap_dead start 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.088+0000 7fc45ae00640 1 -- 172.21.15.59:0/1571893886 shutdown_connections 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.088+0000 7fc45ae00640 1 -- 172.21.15.59:0/1571893886 >> 172.21.15.59:0/1571893886 conn(0x7fc45406bbb0 msgr2=0x7fc45406bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.088+0000 7fc45ae00640 1 -- 172.21.15.59:0/1571893886 shutdown_connections 2024-04-24T06:30:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.088+0000 7fc45ae00640 1 -- 172.21.15.59:0/1571893886 wait complete. 2024-04-24T06:30:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.090+0000 7fc45ae00640 1 Processor -- start 2024-04-24T06:30:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.090+0000 7fc45ae00640 1 -- start start 2024-04-24T06:30:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc43c003f80 0x7fc4540ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc45415be80 0x7fc454158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc45415c910 con 0x7fc45415ba20 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc454161080 con 0x7fc45415be80 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc45415ca90 con 0x7fc43c003f80 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc459000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc459000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53146/0 (socket says 172.21.15.59:53146) 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc459000640 1 -- 172.21.15.59:0/3455211685 learned_addr learned my addr 172.21.15.59:0/3455211685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc459a00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc43c003f80 0x7fc4540ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.091+0000 7fc45a400640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc45415be80 0x7fc454158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 -- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc43c003f80 msgr2=0x7fc4540ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc43c003f80 0x7fc4540ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 -- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc45415be80 msgr2=0x7fc454158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc45415be80 0x7fc454158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 -- 172.21.15.59:0/3455211685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4540c85d0 con 0x7fc45415ba20 2024-04-24T06:30:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc459000640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc440034040 tx=0x7fc440004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.092+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440040020 con 0x7fc45415ba20 2024-04-24T06:30:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.093+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440003c40 con 0x7fc45415ba20 2024-04-24T06:30:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.093+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440036040 con 0x7fc45415ba20 2024-04-24T06:30:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.093+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45415d750 con 0x7fc45415ba20 2024-04-24T06:30:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.093+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4540c8870 con 0x7fc45415ba20 2024-04-24T06:30:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.094+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45404fd90 con 0x7fc45415ba20 2024-04-24T06:30:03.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.095+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc440043020 con 0x7fc45415ba20 2024-04-24T06:30:03.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.096+0000 7fc451600640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:03.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.096+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fc4400ac290 con 0x7fc45415ba20 2024-04-24T06:30:03.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.097+0000 7fc459a00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:03.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.097+0000 7fc459a00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 0x7fc43406edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc448001050 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:03.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.100+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc44004a070 con 0x7fc45415ba20 2024-04-24T06:30:03.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.307+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc4540ce660 con 0x7fc45415ba20 2024-04-24T06:30:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.847+0000 7fc451600640 1 -- 172.21.15.59:0/3455211685 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fc4400044e0 con 0x7fc45415ba20 2024-04-24T06:30:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 msgr2=0x7fc43406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 0x7fc43406edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc448001050 tx=0x7fc448009040 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 msgr2=0x7fc454157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc440034040 tx=0x7fc440004730 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 shutdown_connections 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.850+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc45415be80 0x7fc454158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc45415ba20 0x7fc454157cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 --2- 172.21.15.59:0/3455211685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc43c003f80 0x7fc4540ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 >> 172.21.15.59:0/3455211685 conn(0x7fc45406bbb0 msgr2=0x7fc4540b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 shutdown_connections 2024-04-24T06:30:03.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:03.851+0000 7fc45ae00640 1 -- 172.21.15.59:0/3455211685 wait complete. 2024-04-24T06:30:03.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:30:04.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.067+0000 7f46d1e00640 1 Processor -- start 2024-04-24T06:30:04.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.067+0000 7f46d1e00640 1 -- start start 2024-04-24T06:30:04.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.068+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46cc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.068+0000 7f46d1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f46cc071d70 0x7f46cc072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.068+0000 7f46d1e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc0cfc20 con 0x7f46cc145e10 2024-04-24T06:30:04.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.068+0000 7f46d1e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc0cff00 con 0x7f46cc071d70 2024-04-24T06:30:04.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.068+0000 7f46d1e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc0d01e0 con 0x7f46cc0d0cc0 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46caa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f46cc145e10 0x7f46cc0cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57848/0 (socket says 172.21.15.59:57848) 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46caa00640 1 -- 172.21.15.59:0/994964057 learned_addr learned my addr 172.21.15.59:0/994964057 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46cb400640 1 --2- 172.21.15.59:0/994964057 >> v2:172.21.15.66:3300/0 conn(0x7f46cc071d70 0x7f46cc072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46c2000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46cc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41360259 0 0) 0x7f46cc0cfc20 con 0x7f46cc145e10 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.069+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b0003520 con 0x7f46cc145e10 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3493169144 0 0) 0x7f46b0003520 con 0x7f46cc145e10 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 msgr2=0x7f46cc06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:04.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46cc06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 >> v2:172.21.15.66:3300/0 conn(0x7f46cc071d70 msgr2=0x7f46cc072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 --2- 172.21.15.59:0/994964057 >> v2:172.21.15.66:3300/0 conn(0x7f46cc071d70 0x7f46cc072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc0d04c0 con 0x7f46cc145e10 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f46bc001670 con 0x7f46cc145e10 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 >> v1:172.21.15.59:6789/0 conn(0x7f46cc145e10 legacy=0x7f46cc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46b0004d40 0x7f46b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0cff00 con 0x7f46b00039b0 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0d01e0 con 0x7f46b0004d40 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46b000b9c0 con 0x7f46cc0d0cc0 2024-04-24T06:30:04.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.070+0000 7f46caa00640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46c2000640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46caa00640 1 -- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 msgr2=0x7f46b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46caa00640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc0d0cc0 0x7f46b0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46cb400640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46b0004d40 0x7f46b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46caa00640 1 -- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46b0004d40 msgr2=0x7f46b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46caa00640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46b0004d40 0x7f46b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46caa00640 1 -- 172.21.15.59:0/994964057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b0003520 con 0x7f46b00039b0 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.071+0000 7f46cb400640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46b0004d40 0x7f46b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.072+0000 7f46caa00640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c110 tx=0x7f46b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214071c216d7cd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.072+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc007c20 con 0x7f46b00039b0 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.072+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc005af0 con 0x7f46b00039b0 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.072+0000 7f46ca000640 1 -- 172.21.15.59:0/994964057 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc007070 con 0x7f46b00039b0 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 -- 172.21.15.59:0/994964057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 msgr2=0x7f46b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 --2- 172.21.15.59:0/994964057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c110 tx=0x7f46b0005830 comp rx=0 tx=0).stop 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46cb400640 1 -- 172.21.15.59:0/994964057 reap_dead start 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 -- 172.21.15.59:0/994964057 shutdown_connections 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 -- 172.21.15.59:0/994964057 >> 172.21.15.59:0/994964057 conn(0x7f46cc067d90 msgr2=0x7f46cc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 -- 172.21.15.59:0/994964057 shutdown_connections 2024-04-24T06:30:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.073+0000 7f46d1e00640 1 -- 172.21.15.59:0/994964057 wait complete. 2024-04-24T06:30:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.075+0000 7f46d1e00640 1 Processor -- start 2024-04-24T06:30:04.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.075+0000 7f46d1e00640 1 -- start start 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46cc15b280 0x7f46cc15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc157160 0x7f46cc1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1461f0 con 0x7f46b00039b0 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc06f130 con 0x7f46cc15b280 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46d1e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc157b20 con 0x7f46cc157160 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53154/0 (socket says 172.21.15.59:53154) 2024-04-24T06:30:04.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46caa00640 1 -- 172.21.15.59:0/3329252201 learned_addr learned my addr 172.21.15.59:0/3329252201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.076+0000 7f46c2000640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46cc15b280 0x7f46cc15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46cb400640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc157160 0x7f46cc1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 -- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc157160 msgr2=0x7f46cc1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc157160 0x7f46cc1575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 -- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46cc15b280 msgr2=0x7f46cc15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46cc15b280 0x7f46cc15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 -- 172.21.15.59:0/3329252201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc15c590 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46caa00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46cc0d0690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc043070 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc038040 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0072b0 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.077+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc15bcf0 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.078+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc15c2b0 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.079+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc068a80 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.080+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46bc007450 con 0x7f46b00039b0 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.081+0000 7f46c3e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 0x7f46a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:04.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.081+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f46bc0abab0 con 0x7f46b00039b0 2024-04-24T06:30:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.081+0000 7f46c2000640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 0x7f46a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.082+0000 7f46c2000640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 0x7f46a806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46cc145770 tx=0x7f46b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.085+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46bc0798d0 con 0x7f46b00039b0 2024-04-24T06:30:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.332+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f46cc0d1190 con 0x7f46b00039b0 2024-04-24T06:30:04.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.910+0000 7f46c3e00640 1 -- 172.21.15.59:0/3329252201 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f46bc07dbd0 con 0x7f46b00039b0 2024-04-24T06:30:04.912 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 4 and data pool 5 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.919+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 msgr2=0x7f46a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.919+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 0x7f46a806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46cc145770 tx=0x7f46b4008040 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.919+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 msgr2=0x7f46cc13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.919+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46cc0d0690 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 shutdown_connections 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46a806c920 0x7f46a806ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46cc157160 0x7f46cc1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46cc15b280 0x7f46cc15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 --2- 172.21.15.59:0/3329252201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b00039b0 0x7f46cc13e0c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:04.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.920+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 >> 172.21.15.59:0/3329252201 conn(0x7f46cc067d90 msgr2=0x7f46cc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:04.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.923+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 shutdown_connections 2024-04-24T06:30:04.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:04.923+0000 7f46d1e00640 1 -- 172.21.15.59:0/3329252201 wait complete. 2024-04-24T06:30:04.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:05.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.129+0000 7f93eea00640 1 Processor -- start 2024-04-24T06:30:05.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.130+0000 7f93eea00640 1 -- start start 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93eea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f93e80cd760 0x7f93e80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93eea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8160210 con 0x7f93e815c3d0 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93eea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e81604f0 con 0x7f93e80b4690 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93eea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f93e81607d0 con 0x7f93e80cd760 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93ed600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f93e815c3d0 0x7f93e80cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57862/0 (socket says 172.21.15.59:57862) 2024-04-24T06:30:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.131+0000 7f93ed600640 1 -- 172.21.15.59:0/1911872659 learned_addr learned my addr 172.21.15.59:0/1911872659 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1183793378 0 0) 0x7f93e8160210 con 0x7f93e815c3d0 2024-04-24T06:30:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d0003520 con 0x7f93e815c3d0 2024-04-24T06:30:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93ecc00640 1 --2- 172.21.15.59:0/1911872659 >> v2:172.21.15.66:3300/0 conn(0x7f93e80cd760 0x7f93e80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2086222679 0 0) 0x7f93d0003520 con 0x7f93e815c3d0 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 >> v2:172.21.15.66:3300/0 conn(0x7f93e80cd760 msgr2=0x7f93e80b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 --2- 172.21.15.59:0/1911872659 >> v2:172.21.15.66:3300/0 conn(0x7f93e80cd760 0x7f93e80b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 >> v1:172.21.15.66:6789/0 conn(0x7f93e80b4690 legacy=0x7f93e80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8160ab0 con 0x7f93e815c3d0 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f93dc001670 con 0x7f93e815c3d0 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 >> v1:172.21.15.59:6789/0 conn(0x7f93e815c3d0 legacy=0x7f93e80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 0x7f93d0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00051c0 0x7f93d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81607d0 con 0x7f93d00039b0 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93d000c0f0 con 0x7f93d0004800 2024-04-24T06:30:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.132+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93d000c2f0 con 0x7f93d00051c0 2024-04-24T06:30:05.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 0x7f93d0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ee000640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00051c0 0x7f93d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 -- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00051c0 msgr2=0x7f93d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00051c0 0x7f93d000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ecc00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 -- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93d0004800 msgr2=0x7f93d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ed600640 1 -- 172.21.15.59:0/1911872659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d0003520 con 0x7f93d00039b0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.133+0000 7f93ee000640 1 -- 172.21.15.59:0/1911872659 reap_dead start 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.134+0000 7f93ed600640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 0x7f93d0003db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00c210 tx=0x7f93d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc03ac70da627d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc007d30 con 0x7f93d00039b0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc0058d0 con 0x7f93d00039b0 2024-04-24T06:30:05.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93e7e00640 1 -- 172.21.15.59:0/1911872659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc005d20 con 0x7f93d00039b0 2024-04-24T06:30:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 -- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 msgr2=0x7f93d0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 0x7f93d0003db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00c210 tx=0x7f93d0005830 comp rx=0 tx=0).stop 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 -- 172.21.15.59:0/1911872659 shutdown_connections 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 --2- 172.21.15.59:0/1911872659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93d00039b0 0x7f93d0003db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 -- 172.21.15.59:0/1911872659 >> 172.21.15.59:0/1911872659 conn(0x7f93e80b21a0 msgr2=0x7f93e80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 -- 172.21.15.59:0/1911872659 shutdown_connections 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.135+0000 7f93eea00640 1 -- 172.21.15.59:0/1911872659 wait complete. 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.137+0000 7f93eea00640 1 Processor -- start 2024-04-24T06:30:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.137+0000 7f93eea00640 1 -- start start 2024-04-24T06:30:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00039b0 0x7f93e80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e815cb90 0x7f93e815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93e815c7b0 con 0x7f93e80ae0b0 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80b4ae0 con 0x7f93e815cb90 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.138+0000 7f93eea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93e815c930 con 0x7f93d00039b0 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53158/0 (socket says 172.21.15.59:53158) 2024-04-24T06:30:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 -- 172.21.15.59:0/355387198 learned_addr learned my addr 172.21.15.59:0/355387198 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ed600640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00039b0 0x7f93e80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 -- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00039b0 msgr2=0x7f93e80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00039b0 0x7f93e80cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 -- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e815cb90 msgr2=0x7f93e815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e815cb90 0x7f93e815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 -- 172.21.15.59:0/355387198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e80ae870 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.139+0000 7f93ecc00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93d400e810 tx=0x7f93d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.140+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4017070 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.140+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d400c6d0 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.140+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d400f040 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.140+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e80aeb50 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.140+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e80af110 con 0x7f93e80ae0b0 2024-04-24T06:30:05.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.141+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e804fd90 con 0x7f93e80ae0b0 2024-04-24T06:30:05.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.142+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93d401a070 con 0x7f93e80ae0b0 2024-04-24T06:30:05.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.144+0000 7f93e5600640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 0x7f93c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.144+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f93d408b760 con 0x7f93e80ae0b0 2024-04-24T06:30:05.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.144+0000 7f93ed600640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 0x7f93c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.145+0000 7f93ed600640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 0x7f93c806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93dc033000 tx=0x7f93dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.148+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93d4059af0 con 0x7f93e80ae0b0 2024-04-24T06:30:05.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:30:05.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:30:05.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.356+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93e806e400 con 0x7f93e80ae0b0 2024-04-24T06:30:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.357+0000 7f93e5600640 1 -- 172.21.15.59:0/355387198 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16598 (secure 0 0 0) 0x7f93d40593f0 con 0x7f93e80ae0b0 2024-04-24T06:30:05.359 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:05.359 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:04.333301+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 msgr2=0x7f93c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 0x7f93c806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93dc033000 tx=0x7f93dc02e000 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 msgr2=0x7f93e81f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93d400e810 tx=0x7f93d400ecd0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 shutdown_connections 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c806c8f0 0x7f93c806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e815cb90 0x7f93e815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80ae0b0 0x7f93e81f7450 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 --2- 172.21.15.59:0/355387198 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93d00039b0 0x7f93e80cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 >> 172.21.15.59:0/355387198 conn(0x7f93e80b21a0 msgr2=0x7f93e806f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:05.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 shutdown_connections 2024-04-24T06:30:05.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.360+0000 7f93eea00640 1 -- 172.21.15.59:0/355387198 wait complete. 2024-04-24T06:30:05.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:05.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.567+0000 7f19ada00640 1 Processor -- start 2024-04-24T06:30:05.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.567+0000 7f19ada00640 1 -- start start 2024-04-24T06:30:05.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19ada00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19a80b4130 0x7f19a806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19ada00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a806ed40 con 0x7f19a8162df0 2024-04-24T06:30:05.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19ada00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a80b3220 con 0x7f19a80cfb90 2024-04-24T06:30:05.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19ada00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f19a806eaf0 con 0x7f19a80b4130 2024-04-24T06:30:05.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19a7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f19a8162df0 0x7f19a80b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57870/0 (socket says 172.21.15.59:57870) 2024-04-24T06:30:05.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f19a7e00640 1 -- 172.21.15.59:0/3863324278 learned_addr learned my addr 172.21.15.59:0/3863324278 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:05.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.569+0000 7f199fe00640 1 --2- 172.21.15.59:0/3863324278 >> v2:172.21.15.59:3300/0 conn(0x7f19a80b4130 0x7f19a806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.570+0000 7f19a7400640 1 -- 172.21.15.59:0/3863324278 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778954550 0 0) 0x7f19a806ed40 con 0x7f19a8162df0 2024-04-24T06:30:05.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.570+0000 7f19a7400640 1 -- 172.21.15.59:0/3863324278 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c003520 con 0x7f19a8162df0 2024-04-24T06:30:05.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.570+0000 7f199fe00640 1 -- 172.21.15.59:0/3863324278 >> v1:172.21.15.66:6789/0 conn(0x7f19a80cfb90 legacy=0x7f19a80c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:05.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.570+0000 7f199fe00640 1 -- 172.21.15.59:0/3863324278 >> v1:172.21.15.59:6789/0 conn(0x7f19a8162df0 legacy=0x7f19a80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.570+0000 7f199fe00640 1 -- 172.21.15.59:0/3863324278 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a80af4c0 con 0x7f19a80b4130 2024-04-24T06:30:05.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.571+0000 7f199fe00640 1 --2- 172.21.15.59:0/3863324278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80b4130 0x7f19a806f680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f19900097a0 tx=0x7f199002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa49d3ec21809d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.571+0000 7f19a7400640 1 -- 172.21.15.59:0/3863324278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990037070 con 0x7f19a80b4130 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.571+0000 7f19a7400640 1 -- 172.21.15.59:0/3863324278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990032440 con 0x7f19a80b4130 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19a7400640 1 -- 172.21.15.59:0/3863324278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990032810 con 0x7f19a80b4130 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 -- 172.21.15.59:0/3863324278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80b4130 msgr2=0x7f19a806f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 --2- 172.21.15.59:0/3863324278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80b4130 0x7f19a806f680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f19900097a0 tx=0x7f199002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 -- 172.21.15.59:0/3863324278 shutdown_connections 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 --2- 172.21.15.59:0/3863324278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80b4130 0x7f19a806f680 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 -- 172.21.15.59:0/3863324278 >> 172.21.15.59:0/3863324278 conn(0x7f19a80b21a0 msgr2=0x7f19a80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 -- 172.21.15.59:0/3863324278 shutdown_connections 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.572+0000 7f19ada00640 1 -- 172.21.15.59:0/3863324278 wait complete. 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.574+0000 7f19ada00640 1 Processor -- start 2024-04-24T06:30:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.574+0000 7f19ada00640 1 -- start start 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19a80b4130 0x7f19a8155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f19a8162df0 0x7f19a80ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80ce200 con 0x7f19a80cfb90 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80b3590 con 0x7f19a80b4130 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f19ada00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f19a80ce380 con 0x7f19a8162df0 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f199fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f199fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53174/0 (socket says 172.21.15.59:53174) 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.575+0000 7f199fe00640 1 -- 172.21.15.59:0/3768447800 learned_addr learned my addr 172.21.15.59:0/3768447800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f19a7e00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19a80b4130 0x7f19a8155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 -- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f19a8162df0 msgr2=0x7f19a80ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f19a8162df0 0x7f19a80ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 -- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19a80b4130 msgr2=0x7f19a8155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19a80b4130 0x7f19a8155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 -- 172.21.15.59:0/3768447800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a8164bd0 con 0x7f19a80cfb90 2024-04-24T06:30:05.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.576+0000 7f199fe00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f19900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.577+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990037040 con 0x7f19a80cfb90 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.577+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990042070 con 0x7f19a80cfb90 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.577+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990030940 con 0x7f19a80cfb90 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.577+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a8164eb0 con 0x7f19a80cfb90 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.577+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a8165470 con 0x7f19a80cfb90 2024-04-24T06:30:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.578+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a804fd90 con 0x7f19a80cfb90 2024-04-24T06:30:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.579+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1990030ae0 con 0x7f19a80cfb90 2024-04-24T06:30:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.581+0000 7f19a4c00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.581+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f19900ac4c0 con 0x7f19a80cfb90 2024-04-24T06:30:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.581+0000 7f19a7e00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.582+0000 7f19a7e00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 0x7f198406ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19a81f6140 tx=0x7f1998009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.585+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1990045360 con 0x7f19a80cfb90 2024-04-24T06:30:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.788+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19a80b27d0 con 0x7f19a80cfb90 2024-04-24T06:30:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.790+0000 7f19a4c00640 1 -- 172.21.15.59:0/3768447800 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16598 (secure 0 0 0) 0x7f199007a8e0 con 0x7f19a80cfb90 2024-04-24T06:30:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:05.792 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:04.333301+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 msgr2=0x7f198406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 0x7f198406ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19a81f6140 tx=0x7f1998009040 comp rx=0 tx=0).stop 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 msgr2=0x7f19a81fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f19900046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 shutdown_connections 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f198406c920 0x7f198406ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f19a8162df0 0x7f19a80ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a80cfb90 0x7f19a81fc280 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 --2- 172.21.15.59:0/3768447800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19a80b4130 0x7f19a8155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:05.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 >> 172.21.15.59:0/3768447800 conn(0x7f19a80b21a0 msgr2=0x7f19a80b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:05.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 shutdown_connections 2024-04-24T06:30:05.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:05.793+0000 7f19ada00640 1 -- 172.21.15.59:0/3768447800 wait complete. 2024-04-24T06:30:05.813 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:30:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.034+0000 7fd105a00640 1 Processor -- start 2024-04-24T06:30:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.035+0000 7fd105a00640 1 -- start start 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd105a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1000d0600 0x7fd1000ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd105a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd10006dcb0 0x7fd100162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd105a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd100163240 con 0x7fd1000d09e0 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd105a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd100072a20 con 0x7fd1000d0600 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd105a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd100071e10 con 0x7fd10006dcb0 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd0ffe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1000d0600 0x7fd1000ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd0ffe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1000d0600 0x7fd1000ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53180/0 (socket says 172.21.15.59:53180) 2024-04-24T06:30:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd0ffe00640 1 -- 172.21.15.59:0/37102442 learned_addr learned my addr 172.21.15.59:0/37102442 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.037+0000 7fd0f7e00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd10006dcb0 0x7fd100162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.038+0000 7fd0ffe00640 1 -- 172.21.15.59:0/37102442 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd10006dcb0 msgr2=0x7fd100162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.038+0000 7fd0ffe00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd10006dcb0 0x7fd100162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.038+0000 7fd0ffe00640 1 -- 172.21.15.59:0/37102442 >> v1:172.21.15.66:6789/0 conn(0x7fd1000d09e0 legacy=0x7fd10006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.038+0000 7fd0ffe00640 1 -- 172.21.15.59:0/37102442 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1000720f0 con 0x7fd1000d0600 2024-04-24T06:30:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.039+0000 7fd0ffe00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1000d0600 0x7fd1000ad170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009870 tx=0x7fd0f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf5c153acfb5fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.039+0000 7fd0ff400640 1 -- 172.21.15.59:0/37102442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037070 con 0x7fd1000d0600 2024-04-24T06:30:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.039+0000 7fd0ff400640 1 -- 172.21.15.59:0/37102442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f0032440 con 0x7fd1000d0600 2024-04-24T06:30:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.039+0000 7fd0ff400640 1 -- 172.21.15.59:0/37102442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0032810 con 0x7fd1000d0600 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 -- 172.21.15.59:0/37102442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1000d0600 msgr2=0x7fd1000ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1000d0600 0x7fd1000ad170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009870 tx=0x7fd0f002a270 comp rx=0 tx=0).stop 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 -- 172.21.15.59:0/37102442 shutdown_connections 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd10006dcb0 0x7fd100162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 --2- 172.21.15.59:0/37102442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1000d0600 0x7fd1000ad170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 -- 172.21.15.59:0/37102442 >> 172.21.15.59:0/37102442 conn(0x7fd10006bdf0 msgr2=0x7fd10006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 -- 172.21.15.59:0/37102442 shutdown_connections 2024-04-24T06:30:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.040+0000 7fd105a00640 1 -- 172.21.15.59:0/37102442 wait complete. 2024-04-24T06:30:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.042+0000 7fd105a00640 1 Processor -- start 2024-04-24T06:30:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.042+0000 7fd105a00640 1 -- start start 2024-04-24T06:30:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1000d0600 0x7fd10009bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1000d09e0 0x7fd10015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd10015eaf0 con 0x7fd10006dcb0 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1000726b0 con 0x7fd1000d0600 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1000d0e70 con 0x7fd1000d09e0 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53190/0 (socket says 172.21.15.59:53190) 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 -- 172.21.15.59:0/2572498293 learned_addr learned my addr 172.21.15.59:0/2572498293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0f7e00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1000d0600 0x7fd10009bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd105000640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1000d09e0 0x7fd10015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 -- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1000d09e0 msgr2=0x7fd10015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1000d09e0 0x7fd10015b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 -- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1000d0600 msgr2=0x7fd10009bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.044+0000 7fd0ffe00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1000d0600 0x7fd10009bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd0ffe00640 1 -- 172.21.15.59:0/2572498293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10009df40 con 0x7fd10006dcb0 2024-04-24T06:30:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd0ffe00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0000c00 tx=0x7fd0f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0041020 con 0x7fd10006dcb0 2024-04-24T06:30:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f0030780 con 0x7fd10006dcb0 2024-04-24T06:30:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037040 con 0x7fd10006dcb0 2024-04-24T06:30:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10009c9c0 con 0x7fd10006dcb0 2024-04-24T06:30:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.045+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10009d200 con 0x7fd10006dcb0 2024-04-24T06:30:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.047+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10004fd90 con 0x7fd10006dcb0 2024-04-24T06:30:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.047+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0f0044020 con 0x7fd10006dcb0 2024-04-24T06:30:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.048+0000 7fd0fcc00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 0x7fd0dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.049+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd0f00ac2b0 con 0x7fd10006dcb0 2024-04-24T06:30:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.049+0000 7fd0f7e00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 0x7fd0dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.049+0000 7fd0f7e00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 0x7fd0dc06ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd10015b1c0 tx=0x7fd0e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:06.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.053+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0f004b030 con 0x7fd10006dcb0 2024-04-24T06:30:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.299+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd1000cff50 con 0x7fd10006dcb0 2024-04-24T06:30:06.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.922+0000 7fd0fcc00640 1 -- 172.21.15.59:0/2572498293 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fd10009c9c0 con 0x7fd10006dcb0 2024-04-24T06:30:06.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.925+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 msgr2=0x7fd0dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:06.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.925+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 0x7fd0dc06ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd10015b1c0 tx=0x7fd0e8008040 comp rx=0 tx=0).stop 2024-04-24T06:30:06.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.925+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 msgr2=0x7fd10009da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.925+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0000c00 tx=0x7fd0f00046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 shutdown_connections 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0dc06c920 0x7fd0dc06ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1000d09e0 0x7fd10015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1000d0600 0x7fd10009bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 --2- 172.21.15.59:0/2572498293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10006dcb0 0x7fd10009da00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 >> 172.21.15.59:0/2572498293 conn(0x7fd10006bdf0 msgr2=0x7fd1000ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:06.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 shutdown_connections 2024-04-24T06:30:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:06.926+0000 7fd105a00640 1 -- 172.21.15.59:0/2572498293 wait complete. 2024-04-24T06:30:06.952 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:30:06.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:30:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.196+0000 7f6206600640 1 Processor -- start 2024-04-24T06:30:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.196+0000 7f6206600640 1 -- start start 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f6206600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62000b56f0 0x7f620006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f6206600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f620015bf60 con 0x7f62000d0a60 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f6206600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f620015c240 con 0x7f620006f340 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f6206600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f620015c520 con 0x7f62000b56f0 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f61ff400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62000b56f0 0x7f620006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f61ffe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f62000d0a60 0x7f62000b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57876/0 (socket says 172.21.15.59:57876) 2024-04-24T06:30:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.198+0000 7f61ffe00640 1 -- 172.21.15.59:0/270972647 learned_addr learned my addr 172.21.15.59:0/270972647 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:07.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.199+0000 7f61ff400640 1 -- 172.21.15.59:0/270972647 >> v1:172.21.15.66:6789/0 conn(0x7f620006f340 legacy=0x7f62000d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:07.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.199+0000 7f61ff400640 1 -- 172.21.15.59:0/270972647 >> v1:172.21.15.59:6789/0 conn(0x7f62000d0a60 legacy=0x7f62000b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.199+0000 7f61ff400640 1 -- 172.21.15.59:0/270972647 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620015c800 con 0x7f62000b56f0 2024-04-24T06:30:07.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.199+0000 7f61fe000640 1 -- 172.21.15.59:0/270972647 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3620746372 0 0) 0x7f620015bf60 con 0x7f62000d0a60 2024-04-24T06:30:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.200+0000 7f61ff400640 1 --2- 172.21.15.59:0/270972647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62000b56f0 0x7f620006ddd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f61f00097a0 tx=0x7f61f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd1d2cebd88a624 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.200+0000 7f61fe000640 1 -- 172.21.15.59:0/270972647 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0037070 con 0x7f62000b56f0 2024-04-24T06:30:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.200+0000 7f61fe000640 1 -- 172.21.15.59:0/270972647 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0032440 con 0x7f62000b56f0 2024-04-24T06:30:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.200+0000 7f61fe000640 1 -- 172.21.15.59:0/270972647 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0032810 con 0x7f62000b56f0 2024-04-24T06:30:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 -- 172.21.15.59:0/270972647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62000b56f0 msgr2=0x7f620006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 --2- 172.21.15.59:0/270972647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62000b56f0 0x7f620006ddd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f61f00097a0 tx=0x7f61f002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 -- 172.21.15.59:0/270972647 shutdown_connections 2024-04-24T06:30:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 --2- 172.21.15.59:0/270972647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62000b56f0 0x7f620006ddd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 -- 172.21.15.59:0/270972647 >> 172.21.15.59:0/270972647 conn(0x7f6200067d90 msgr2=0x7f620006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:07.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 -- 172.21.15.59:0/270972647 shutdown_connections 2024-04-24T06:30:07.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.201+0000 7f6206600640 1 -- 172.21.15.59:0/270972647 wait complete. 2024-04-24T06:30:07.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.203+0000 7f6206600640 1 Processor -- start 2024-04-24T06:30:07.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.203+0000 7f6206600640 1 -- start start 2024-04-24T06:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 0x7f620014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62000d0a60 0x7f620014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f620006f7c0 con 0x7f620006f340 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f620015bc10 con 0x7f62000b56f0 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f6206600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f620006f940 con 0x7f62000d0a60 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53208/0 (socket says 172.21.15.59:53208) 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 -- 172.21.15.59:0/1045546832 learned_addr learned my addr 172.21.15.59:0/1045546832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 -- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62000d0a60 msgr2=0x7f620014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62000d0a60 0x7f620014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 -- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 msgr2=0x7f620014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61fea00640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 0x7f620014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 0x7f620014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61ff400640 1 -- 172.21.15.59:0/1045546832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6200153460 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.204+0000 7f61fea00640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 0x7f620014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.205+0000 7f61ff400640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f61f0035040 tx=0x7f61f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.205+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0037050 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.205+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0038040 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.205+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0004470 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.205+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6200153740 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.206+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6200153d00 con 0x7f620006f340 2024-04-24T06:30:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.207+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6200068a80 con 0x7f620006f340 2024-04-24T06:30:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.208+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61f0030980 con 0x7f620006f340 2024-04-24T06:30:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.209+0000 7f61f7400640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 0x7f61e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.209+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f61f00ac760 con 0x7f620006f340 2024-04-24T06:30:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.209+0000 7f61fea00640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 0x7f61e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.210+0000 7f61fea00640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 0x7f61e006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6200152d40 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.214+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61f007ab80 con 0x7f620006f340 2024-04-24T06:30:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.465+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f620006ef20 con 0x7f620006f340 2024-04-24T06:30:07.728 INFO:tasks.ceph.mds.d.smithi066.stderr:2024-04-24T06:30:07.726+0000 7f732f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:30:07.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.935+0000 7f61f7400640 1 -- 172.21.15.59:0/1045546832 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f61f007a480 con 0x7f620006f340 2024-04-24T06:30:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 msgr2=0x7f61e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 0x7f61e006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6200152d40 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 msgr2=0x7f62001482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f61f0035040 tx=0x7f61f00046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 shutdown_connections 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61e006c920 0x7f61e006ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62000d0a60 0x7f620014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62000b56f0 0x7f620014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 --2- 172.21.15.59:0/1045546832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620006f340 0x7f62001482e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.939+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 >> 172.21.15.59:0/1045546832 conn(0x7f6200067d90 msgr2=0x7f62000b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.940+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 shutdown_connections 2024-04-24T06:30:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:07.940+0000 7f6206600640 1 -- 172.21.15.59:0/1045546832 wait complete. 2024-04-24T06:30:07.965 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:30:07.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.175+0000 7ff971a00640 1 Processor -- start 2024-04-24T06:30:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.176+0000 7ff971a00640 1 -- start start 2024-04-24T06:30:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.177+0000 7ff971a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff96c071de0 0x7ff96c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.177+0000 7ff971a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff96c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.177+0000 7ff971a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c0cd8e0 con 0x7ff96c06dde0 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff971a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c070180 con 0x7ff96c071de0 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff971a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c072700 con 0x7ff96c072e30 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff96b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff96c06dde0 0x7ff96c06e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57888/0 (socket says 172.21.15.59:57888) 2024-04-24T06:30:08.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff96b400640 1 -- 172.21.15.59:0/3842393080 learned_addr learned my addr 172.21.15.59:0/3842393080 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff96aa00640 1 --2- 172.21.15.59:0/3842393080 >> v2:172.21.15.66:3300/0 conn(0x7ff96c071de0 0x7ff96c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff96a000640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff96c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229179818 0 0) 0x7ff96c0cd8e0 con 0x7ff96c06dde0 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.178+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff950003520 con 0x7ff96c06dde0 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2413640003 0 0) 0x7ff950003520 con 0x7ff96c06dde0 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 msgr2=0x7ff96c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff96c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 >> v2:172.21.15.66:3300/0 conn(0x7ff96c071de0 msgr2=0x7ff96c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 --2- 172.21.15.59:0/3842393080 >> v2:172.21.15.66:3300/0 conn(0x7ff96c071de0 0x7ff96c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c06fbc0 con 0x7ff96c06dde0 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff9640019f0 con 0x7ff96c06dde0 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 >> v1:172.21.15.59:6789/0 conn(0x7ff96c06dde0 legacy=0x7ff96c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff9500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff950003f80 0x7ff950004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950004d40 0x7ff9500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c070180 con 0x7ff950003f80 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c072700 con 0x7ff950004d40 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff95000b9c0 con 0x7ff96c072e30 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.179+0000 7ff96a000640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff950003f80 0x7ff950004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96a000640 1 -- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 msgr2=0x7ff9500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96aa00640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff9500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96a000640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c072e30 0x7ff9500039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96b400640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950004d40 0x7ff9500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96a000640 1 -- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950004d40 msgr2=0x7ff9500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96a000640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950004d40 0x7ff9500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.180+0000 7ff96a000640 1 -- 172.21.15.59:0/3842393080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff950003520 con 0x7ff950003f80 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.181+0000 7ff96a000640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff950003f80 0x7ff950004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff954009af0 tx=0x7ff950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584e1355fa77fa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.181+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954036070 con 0x7ff950003f80 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.181+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954031400 con 0x7ff950003f80 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.181+0000 7ff969600640 1 -- 172.21.15.59:0/3842393080 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954009690 con 0x7ff950003f80 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 -- 172.21.15.59:0/3842393080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff950003f80 msgr2=0x7ff950004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 --2- 172.21.15.59:0/3842393080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff950003f80 0x7ff950004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff954009af0 tx=0x7ff950005830 comp rx=0 tx=0).stop 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff96b400640 1 -- 172.21.15.59:0/3842393080 reap_dead start 2024-04-24T06:30:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 -- 172.21.15.59:0/3842393080 shutdown_connections 2024-04-24T06:30:08.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 -- 172.21.15.59:0/3842393080 >> 172.21.15.59:0/3842393080 conn(0x7ff96c067ea0 msgr2=0x7ff96c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:08.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 -- 172.21.15.59:0/3842393080 shutdown_connections 2024-04-24T06:30:08.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.182+0000 7ff971a00640 1 -- 172.21.15.59:0/3842393080 wait complete. 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.184+0000 7ff971a00640 1 Processor -- start 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.184+0000 7ff971a00640 1 -- start start 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 0x7ff96c1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 0x7ff96c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0cde60 con 0x7ff96c1e90c0 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0cdb80 con 0x7ff950003f80 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff971a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c15e4e0 con 0x7ff96c0ad870 2024-04-24T06:30:08.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff96a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff96a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53222/0 (socket says 172.21.15.59:53222) 2024-04-24T06:30:08.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff96a000640 1 -- 172.21.15.59:0/1984551555 learned_addr learned my addr 172.21.15.59:0/1984551555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff96b400640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 0x7ff96c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.185+0000 7ff96aa00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 0x7ff96c1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 -- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 msgr2=0x7ff96c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 0x7ff96c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 -- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 msgr2=0x7ff96c1f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 0x7ff96c1f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 -- 172.21.15.59:0/1984551555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c154eb0 con 0x7ff96c1e90c0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96b400640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 0x7ff96c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96aa00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 0x7ff96c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff96a000640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff954000c00 tx=0x7ff954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954040020 con 0x7ff96c1e90c0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954003a40 con 0x7ff96c1e90c0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954036040 con 0x7ff96c1e90c0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.186+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c155190 con 0x7ff96c1e90c0 2024-04-24T06:30:08.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.187+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c155750 con 0x7ff96c1e90c0 2024-04-24T06:30:08.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.188+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c068a80 con 0x7ff96c1e90c0 2024-04-24T06:30:08.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.189+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff954043020 con 0x7ff96c1e90c0 2024-04-24T06:30:08.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.190+0000 7ff962a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.190+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff9540abc20 con 0x7ff96c1e90c0 2024-04-24T06:30:08.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.190+0000 7ff96aa00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.191+0000 7ff96aa00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009a60 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.195+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff95404a070 con 0x7ff96c1e90c0 2024-04-24T06:30:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.445+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff96c0cfca0 con 0x7ff96c1e90c0 2024-04-24T06:30:08.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.447+0000 7ff962a00640 1 -- 172.21.15.59:0/1984551555 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7ff95407a040 con 0x7ff96c1e90c0 2024-04-24T06:30:08.448 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:08.448 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:08.448 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.449+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 msgr2=0x7ff94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.449+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009a60 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.449+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 msgr2=0x7ff96c1e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.449+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff954000c00 tx=0x7ff954004730 comp rx=0 tx=0).stop 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 shutdown_connections 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff96c0ad870 0x7ff96c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c1e90c0 0x7ff96c1e9a60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 --2- 172.21.15.59:0/1984551555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff950003f80 0x7ff96c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 >> 172.21.15.59:0/1984551555 conn(0x7ff96c067ea0 msgr2=0x7ff96c0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 shutdown_connections 2024-04-24T06:30:08.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.450+0000 7ff971a00640 1 -- 172.21.15.59:0/1984551555 wait complete. 2024-04-24T06:30:08.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:08.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.686+0000 7f5e16000640 1 Processor -- start 2024-04-24T06:30:08.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.686+0000 7f5e16000640 1 -- start start 2024-04-24T06:30:08.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.687+0000 7f5e16000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5e100b3820 0x7f5e1009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.687+0000 7f5e16000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e100cd2b0 con 0x7f5e1009ba00 2024-04-24T06:30:08.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.687+0000 7f5e16000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e1015ee90 con 0x7f5e101625c0 2024-04-24T06:30:08.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.687+0000 7f5e16000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5e100d06e0 con 0x7f5e100b3820 2024-04-24T06:30:08.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.688+0000 7f5e0fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5e1009ba00 0x7f5e1015e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57904/0 (socket says 172.21.15.59:57904) 2024-04-24T06:30:08.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.688+0000 7f5e0fe00640 1 -- 172.21.15.59:0/3383106344 learned_addr learned my addr 172.21.15.59:0/3383106344 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:08.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.688+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> v2:172.21.15.66:3300/0 conn(0x7f5e100b3820 0x7f5e1009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.688+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464608773 0 0) 0x7f5e100cd2b0 con 0x7f5e1009ba00 2024-04-24T06:30:08.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.688+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df4003520 con 0x7f5e1009ba00 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 121950882 0 0) 0x7f5df4003520 con 0x7f5e1009ba00 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 >> v2:172.21.15.66:3300/0 conn(0x7f5e100b3820 msgr2=0x7f5e1009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 --2- 172.21.15.59:0/3383106344 >> v2:172.21.15.66:3300/0 conn(0x7f5e100b3820 0x7f5e1009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 >> v1:172.21.15.66:6789/0 conn(0x7f5e101625c0 legacy=0x7f5e1015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e100cd550 con 0x7f5e1009ba00 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5dfc0019f0 con 0x7f5e1009ba00 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 >> v1:172.21.15.59:6789/0 conn(0x7f5e1009ba00 legacy=0x7f5e1015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 0x7f5df4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5df4004800 0x7f5df4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df40052e0 0x7f5df400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5df400c1d0 con 0x7f5df4003a00 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5df400c380 con 0x7f5df4004800 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5df400c580 con 0x7f5df40052e0 2024-04-24T06:30:08.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> v2:172.21.15.66:3300/0 conn(0x7f5e100b3820 0x7f5e1009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:30:08.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.689+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 0x7f5df4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e0fe00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5df4004800 0x7f5df4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e14c00640 1 -- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df40052e0 msgr2=0x7f5df400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e15600640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df40052e0 0x7f5df400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df40052e0 0x7f5df400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e14c00640 1 -- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5df4004800 msgr2=0x7f5df4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5df4004800 0x7f5df4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e14c00640 1 -- 172.21.15.59:0/3383106344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df4003520 con 0x7f5df4003a00 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e0fe00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5df4004800 0x7f5df4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e15600640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df40052e0 0x7f5df400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.690+0000 7f5e15600640 1 -- 172.21.15.59:0/3383106344 reap_dead start 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.691+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 0x7f5df4003e20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098a0 tx=0x7f5df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842a8806f7e93a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.691+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00037070 con 0x7f5df4003a00 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.691+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e0002aca0 con 0x7f5df4003a00 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.691+0000 7f5e0f400640 1 -- 172.21.15.59:0/3383106344 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000f040 con 0x7f5df4003a00 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 -- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 msgr2=0x7f5df4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 0x7f5df4003e20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098a0 tx=0x7f5df4005830 comp rx=0 tx=0).stop 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 -- 172.21.15.59:0/3383106344 shutdown_connections 2024-04-24T06:30:08.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 --2- 172.21.15.59:0/3383106344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df4003a00 0x7f5df4003e20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 -- 172.21.15.59:0/3383106344 >> 172.21.15.59:0/3383106344 conn(0x7f5e100b3ef0 msgr2=0x7f5e100b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:08.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 -- 172.21.15.59:0/3383106344 shutdown_connections 2024-04-24T06:30:08.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.692+0000 7f5e16000640 1 -- 172.21.15.59:0/3383106344 wait complete. 2024-04-24T06:30:08.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.694+0000 7f5e16000640 1 Processor -- start 2024-04-24T06:30:08.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.694+0000 7f5e16000640 1 -- start start 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 0x7f5e100c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e100b74e0 0x7f5e100b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cddf0 con 0x7f5e10153b60 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cdb10 con 0x7f5e100b74e0 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e16000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5e100ce0d0 con 0x7f5df4003a00 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53236/0 (socket says 172.21.15.59:53236) 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.695+0000 7f5e0fe00640 1 -- 172.21.15.59:0/3889876616 learned_addr learned my addr 172.21.15.59:0/3889876616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e15600640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e100b74e0 0x7f5e100b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 0x7f5e100c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 -- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 msgr2=0x7f5e100c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 0x7f5e100c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 -- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e100b74e0 msgr2=0x7f5e100b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e100b74e0 0x7f5e100b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 -- 172.21.15.59:0/3889876616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e100b5b60 con 0x7f5e10153b60 2024-04-24T06:30:08.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 0x7f5e100c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0fe00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc002170 tx=0x7f5dfc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.696+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc019070 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.697+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc004d60 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.697+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0072d0 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.697+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e100b5e40 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.697+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e100c7d10 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.698+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1004fd90 con 0x7f5e10153b60 2024-04-24T06:30:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.699+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dfc01c070 con 0x7f5e10153b60 2024-04-24T06:30:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.700+0000 7f5e0cc00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 0x7f5df006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.700+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5dfc08bd90 con 0x7f5e10153b60 2024-04-24T06:30:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.700+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 0x7f5df006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.701+0000 7f5e14c00640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 0x7f5df006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e00034000 tx=0x7f5e0002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:08.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.704+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dfc05a1b0 con 0x7f5e10153b60 2024-04-24T06:30:08.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.917+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e100cffd0 con 0x7f5e10153b60 2024-04-24T06:30:08.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.919+0000 7f5e0cc00640 1 -- 172.21.15.59:0/3889876616 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f5dfc059ab0 con 0x7f5e10153b60 2024-04-24T06:30:08.920 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:08.920 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 msgr2=0x7f5df006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 0x7f5df006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e00034000 tx=0x7f5e0002e000 comp rx=0 tx=0).stop 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 msgr2=0x7f5e100c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc002170 tx=0x7f5dfc00b4a0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 shutdown_connections 2024-04-24T06:30:08.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5df006c920 0x7f5df006ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e100b74e0 0x7f5e100b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e10153b60 0x7f5e100c9ef0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 --2- 172.21.15.59:0/3889876616 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5df4003a00 0x7f5e100c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 >> 172.21.15.59:0/3889876616 conn(0x7f5e100b3ef0 msgr2=0x7f5e100d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 shutdown_connections 2024-04-24T06:30:08.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:08.922+0000 7f5e16000640 1 -- 172.21.15.59:0/3889876616 wait complete. 2024-04-24T06:30:08.943 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:30:08.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:30:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.156+0000 7ffbc9e00640 1 Processor -- start 2024-04-24T06:30:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.157+0000 7ffbc9e00640 1 -- start start 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.158+0000 7ffbc9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffbc40b5ff0 0x7ffbc40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.158+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc40d0d90 0x7ffbc40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.158+0000 7ffbc9e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40cf030 con 0x7ffbc415e7c0 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.158+0000 7ffbc9e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc40b7000 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.158+0000 7ffbc9e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc40cf620 con 0x7ffbc40d0d90 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffbc40b5ff0 0x7ffbc40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffbc40b5ff0 0x7ffbc40b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40060/0 (socket says 172.21.15.59:40060) 2024-04-24T06:30:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 -- 172.21.15.59:0/73076308 learned_addr learned my addr 172.21.15.59:0/73076308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc9400640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc40d0d90 0x7ffbc40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 -- 172.21.15.59:0/73076308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc40d0d90 msgr2=0x7ffbc40d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc40d0d90 0x7ffbc40d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 -- 172.21.15.59:0/73076308 >> v1:172.21.15.66:6789/0 conn(0x7ffbc415e7c0 legacy=0x7ffbc40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.159+0000 7ffbc3e00640 1 -- 172.21.15.59:0/73076308 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc40b6a40 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.160+0000 7ffbc3e00640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc40b6450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009870 tx=0x7ffbb002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2682484fccae191 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc3400640 1 -- 172.21.15.59:0/73076308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0037070 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc3400640 1 -- 172.21.15.59:0/73076308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb0032440 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc3400640 1 -- 172.21.15.59:0/73076308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0032810 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 -- 172.21.15.59:0/73076308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40b5ff0 msgr2=0x7ffbc40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc40b6450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009870 tx=0x7ffbb002a250 comp rx=0 tx=0).stop 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 -- 172.21.15.59:0/73076308 shutdown_connections 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc40d0d90 0x7ffbc40d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/73076308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc40b6450 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.161+0000 7ffbc9e00640 1 -- 172.21.15.59:0/73076308 >> 172.21.15.59:0/73076308 conn(0x7ffbc409d920 msgr2=0x7ffbc408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.162+0000 7ffbc9e00640 1 -- 172.21.15.59:0/73076308 shutdown_connections 2024-04-24T06:30:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.162+0000 7ffbc9e00640 1 -- 172.21.15.59:0/73076308 wait complete. 2024-04-24T06:30:09.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.163+0000 7ffbc9e00640 1 Processor -- start 2024-04-24T06:30:09.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.163+0000 7ffbc9e00640 1 -- start start 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc4158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc415e7c0 0x7ffbc415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40d2800 con 0x7ffbc40d0d90 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40cf2b0 con 0x7ffbc40b5ff0 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.164+0000 7ffbc9e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc40d1a80 con 0x7ffbc415e7c0 2024-04-24T06:30:09.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40076/0 (socket says 172.21.15.59:40076) 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 -- 172.21.15.59:0/1540887279 learned_addr learned my addr 172.21.15.59:0/1540887279 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc8a00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc4158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 -- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc415e7c0 msgr2=0x7ffbc415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc415e7c0 0x7ffbc415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 -- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 msgr2=0x7ffbc4158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc4158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc3e00640 1 -- 172.21.15.59:0/1540887279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc41e8df0 con 0x7ffbc40d0d90 2024-04-24T06:30:09.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.165+0000 7ffbc8a00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc4158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:09.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc3e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0041020 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb00306c0 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0037040 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc40b12f0 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.166+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc40b18b0 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.167+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc404fd90 con 0x7ffbc40d0d90 2024-04-24T06:30:09.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.168+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbb0044020 con 0x7ffbc40d0d90 2024-04-24T06:30:09.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.169+0000 7ffbc0c00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 0x7ffba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.169+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb00ac2a0 con 0x7ffbc40d0d90 2024-04-24T06:30:09.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.170+0000 7ffbc8a00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 0x7ffba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.171+0000 7ffbc8a00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 0x7ffba406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8000f60 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.175+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbb004b030 con 0x7ffbc40d0d90 2024-04-24T06:30:09.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.375+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbc40d1eb0 con 0x7ffba406c8f0 2024-04-24T06:30:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.396+0000 7ffbc0c00640 1 -- 172.21.15.59:0/1540887279 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7ffbc40d1eb0 con 0x7ffba406c8f0 2024-04-24T06:30:09.401 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:30:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.402+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 msgr2=0x7ffba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.402+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 0x7ffba406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8000f60 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2024-04-24T06:30:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.402+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 msgr2=0x7ffbc415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.402+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb00046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 shutdown_connections 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffba406c8f0 0x7ffba406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbc415e7c0 0x7ffbc415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc40d0d90 0x7ffbc415a450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 --2- 172.21.15.59:0/1540887279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbc40b5ff0 0x7ffbc4158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 >> 172.21.15.59:0/1540887279 conn(0x7ffbc409d920 msgr2=0x7ffbc415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 shutdown_connections 2024-04-24T06:30:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.403+0000 7ffbc9e00640 1 -- 172.21.15.59:0/1540887279 wait complete. 2024-04-24T06:30:09.425 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-24T06:30:09.212452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001389"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884852+0000","last_change":"2024-04-24T06:30:04.932359+0000","last_active":"2024-04-24T06:30:06.884852+0000","last_peered":"2024-04-24T06:30:06.884852+0000","last_clean":"2024-04-24T06:30:06.884852+0000","last_became_active":"2024-04-24T06:30:04.932078+0000","last_became_peered":"2024-04-24T06:30:04.932078+0000","last_unstale":"2024-04-24T06:30:06.884852+0000","last_undegraded":"2024-04-24T06:30:06.884852+0000","last_fullsized":"2024-04-24T06:30:06.884852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:12:18.353421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885910+0000","last_change":"2024-04-24T06:30:04.933665+0000","last_active":"2024-04-24T06:30:06.885910+0000","last_peered":"2024-04-24T06:30:06.885910+0000","last_clean":"2024-04-24T06:30:06.885910+0000","last_became_active":"2024-04-24T06:30:04.933103+0000","last_became_peered":"2024-04-24T06:30:04.933103+0000","last_unstale":"2024-04-24T06:30:06.885910+0000","last_undegraded":"2024-04-24T06:30:06.885910+0000","last_fullsized":"2024-04-24T06:30:06.885910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:07.580317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887437+0000","last_change":"2024-04-24T06:30:04.920923+0000","last_active":"2024-04-24T06:30:06.887437+0000","last_peered":"2024-04-24T06:30:06.887437+0000","last_clean":"2024-04-24T06:30:06.887437+0000","last_became_active":"2024-04-24T06:30:04.920605+0000","last_became_peered":"2024-04-24T06:30:04.920605+0000","last_unstale":"2024-04-24T06:30:06.887437+0000","last_undegraded":"2024-04-24T06:30:06.887437+0000","last_fullsized":"2024-04-24T06:30:06.887437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951509+0000","last_change":"2024-04-24T06:30:04.927487+0000","last_active":"2024-04-24T06:30:05.951509+0000","last_peered":"2024-04-24T06:30:05.951509+0000","last_clean":"2024-04-24T06:30:05.951509+0000","last_became_active":"2024-04-24T06:30:04.927159+0000","last_became_peered":"2024-04-24T06:30:04.927159+0000","last_unstale":"2024-04-24T06:30:05.951509+0000","last_undegraded":"2024-04-24T06:30:05.951509+0000","last_fullsized":"2024-04-24T06:30:05.951509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:11:24.883801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.924350+0000","last_change":"2024-04-24T06:30:04.929287+0000","last_active":"2024-04-24T06:30:05.924350+0000","last_peered":"2024-04-24T06:30:05.924350+0000","last_clean":"2024-04-24T06:30:05.924350+0000","last_became_active":"2024-04-24T06:30:04.929003+0000","last_became_peered":"2024-04-24T06:30:04.929003+0000","last_unstale":"2024-04-24T06:30:05.924350+0000","last_undegraded":"2024-04-24T06:30:05.924350+0000","last_fullsized":"2024-04-24T06:30:05.924350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:32:48.606034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884347+0000","last_change":"2024-04-24T06:30:04.931548+0000","last_active":"2024-04-24T06:30:06.884347+0000","last_peered":"2024-04-24T06:30:06.884347+0000","last_clean":"2024-04-24T06:30:06.884347+0000","last_became_active":"2024-04-24T06:30:04.930678+0000","last_became_peered":"2024-04-24T06:30:04.930678+0000","last_unstale":"2024-04-24T06:30:06.884347+0000","last_undegraded":"2024-04-24T06:30:06.884347+0000","last_fullsized":"2024-04-24T06:30:06.884347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299653+0000","last_change":"2024-04-24T06:30:04.934042+0000","last_active":"2024-04-24T06:30:06.299653+0000","last_peered":"2024-04-24T06:30:06.299653+0000","last_clean":"2024-04-24T06:30:06.299653+0000","last_became_active":"2024-04-24T06:30:04.933765+0000","last_became_peered":"2024-04-24T06:30:04.933765+0000","last_unstale":"2024-04-24T06:30:06.299653+0000","last_undegraded":"2024-04-24T06:30:06.299653+0000","last_fullsized":"2024-04-24T06:30:06.299653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299051+0000","last_change":"2024-04-24T06:30:04.928489+0000","last_active":"2024-04-24T06:30:06.299051+0000","last_peered":"2024-04-24T06:30:06.299051+0000","last_clean":"2024-04-24T06:30:06.299051+0000","last_became_active":"2024-04-24T06:30:04.928131+0000","last_became_peered":"2024-04-24T06:30:04.928131+0000","last_unstale":"2024-04-24T06:30:06.299051+0000","last_undegraded":"2024-04-24T06:30:06.299051+0000","last_fullsized":"2024-04-24T06:30:06.299051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:57.553390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927308+0000","last_change":"2024-04-24T06:30:04.921142+0000","last_active":"2024-04-24T06:30:05.927308+0000","last_peered":"2024-04-24T06:30:05.927308+0000","last_clean":"2024-04-24T06:30:05.927308+0000","last_became_active":"2024-04-24T06:30:04.919968+0000","last_became_peered":"2024-04-24T06:30:04.919968+0000","last_unstale":"2024-04-24T06:30:05.927308+0000","last_undegraded":"2024-04-24T06:30:05.927308+0000","last_fullsized":"2024-04-24T06:30:05.927308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:11.545726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952119+0000","last_change":"2024-04-24T06:30:04.932332+0000","last_active":"2024-04-24T06:30:05.952119+0000","last_peered":"2024-04-24T06:30:05.952119+0000","last_clean":"2024-04-24T06:30:05.952119+0000","last_became_active":"2024-04-24T06:30:04.932026+0000","last_became_peered":"2024-04-24T06:30:04.932026+0000","last_unstale":"2024-04-24T06:30:05.952119+0000","last_undegraded":"2024-04-24T06:30:05.952119+0000","last_fullsized":"2024-04-24T06:30:05.952119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:11.802512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299922+0000","last_change":"2024-04-24T06:30:04.931124+0000","last_active":"2024-04-24T06:30:06.299922+0000","last_peered":"2024-04-24T06:30:06.299922+0000","last_clean":"2024-04-24T06:30:06.299922+0000","last_became_active":"2024-04-24T06:30:04.930821+0000","last_became_peered":"2024-04-24T06:30:04.930821+0000","last_unstale":"2024-04-24T06:30:06.299922+0000","last_undegraded":"2024-04-24T06:30:06.299922+0000","last_fullsized":"2024-04-24T06:30:06.299922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887181+0000","last_change":"2024-04-24T06:30:04.930563+0000","last_active":"2024-04-24T06:30:06.887181+0000","last_peered":"2024-04-24T06:30:06.887181+0000","last_clean":"2024-04-24T06:30:06.887181+0000","last_became_active":"2024-04-24T06:30:04.930220+0000","last_became_peered":"2024-04-24T06:30:04.930220+0000","last_unstale":"2024-04-24T06:30:06.887181+0000","last_undegraded":"2024-04-24T06:30:06.887181+0000","last_fullsized":"2024-04-24T06:30:06.887181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923514+0000","last_change":"2024-04-24T06:30:04.935008+0000","last_active":"2024-04-24T06:30:05.923514+0000","last_peered":"2024-04-24T06:30:05.923514+0000","last_clean":"2024-04-24T06:30:05.923514+0000","last_became_active":"2024-04-24T06:30:04.934732+0000","last_became_peered":"2024-04-24T06:30:04.934732+0000","last_unstale":"2024-04-24T06:30:05.923514+0000","last_undegraded":"2024-04-24T06:30:05.923514+0000","last_fullsized":"2024-04-24T06:30:05.923514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:14.417846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884610+0000","last_change":"2024-04-24T06:30:04.924737+0000","last_active":"2024-04-24T06:30:06.884610+0000","last_peered":"2024-04-24T06:30:06.884610+0000","last_clean":"2024-04-24T06:30:06.884610+0000","last_became_active":"2024-04-24T06:30:04.924378+0000","last_became_peered":"2024-04-24T06:30:04.924378+0000","last_unstale":"2024-04-24T06:30:06.884610+0000","last_undegraded":"2024-04-24T06:30:06.884610+0000","last_fullsized":"2024-04-24T06:30:06.884610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886264+0000","last_change":"2024-04-24T06:30:04.938541+0000","last_active":"2024-04-24T06:30:06.886264+0000","last_peered":"2024-04-24T06:30:06.886264+0000","last_clean":"2024-04-24T06:30:06.886264+0000","last_became_active":"2024-04-24T06:30:04.937269+0000","last_became_peered":"2024-04-24T06:30:04.937269+0000","last_unstale":"2024-04-24T06:30:06.886264+0000","last_undegraded":"2024-04-24T06:30:06.886264+0000","last_fullsized":"2024-04-24T06:30:06.886264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:57.553390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299994+0000","last_change":"2024-04-24T06:30:04.937522+0000","last_active":"2024-04-24T06:30:06.299994+0000","last_peered":"2024-04-24T06:30:06.299994+0000","last_clean":"2024-04-24T06:30:06.299994+0000","last_became_active":"2024-04-24T06:30:04.937192+0000","last_became_peered":"2024-04-24T06:30:04.937192+0000","last_unstale":"2024-04-24T06:30:06.299994+0000","last_undegraded":"2024-04-24T06:30:06.299994+0000","last_fullsized":"2024-04-24T06:30:06.299994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:26:45.435912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.300546+0000","last_change":"2024-04-24T06:30:04.933293+0000","last_active":"2024-04-24T06:30:06.300546+0000","last_peered":"2024-04-24T06:30:06.300546+0000","last_clean":"2024-04-24T06:30:06.300546+0000","last_became_active":"2024-04-24T06:30:04.932955+0000","last_became_peered":"2024-04-24T06:30:04.932955+0000","last_unstale":"2024-04-24T06:30:06.300546+0000","last_undegraded":"2024-04-24T06:30:06.300546+0000","last_fullsized":"2024-04-24T06:30:06.300546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:03.454047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885903+0000","last_change":"2024-04-24T06:30:04.934031+0000","last_active":"2024-04-24T06:30:06.885903+0000","last_peered":"2024-04-24T06:30:06.885903+0000","last_clean":"2024-04-24T06:30:06.885903+0000","last_became_active":"2024-04-24T06:30:04.932760+0000","last_became_peered":"2024-04-24T06:30:04.932760+0000","last_unstale":"2024-04-24T06:30:06.885903+0000","last_undegraded":"2024-04-24T06:30:06.885903+0000","last_fullsized":"2024-04-24T06:30:06.885903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:24:15.105485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885379+0000","last_change":"2024-04-24T06:30:04.922954+0000","last_active":"2024-04-24T06:30:06.885379+0000","last_peered":"2024-04-24T06:30:06.885379+0000","last_clean":"2024-04-24T06:30:06.885379+0000","last_became_active":"2024-04-24T06:30:04.922646+0000","last_became_peered":"2024-04-24T06:30:04.922646+0000","last_unstale":"2024-04-24T06:30:06.885379+0000","last_undegraded":"2024-04-24T06:30:06.885379+0000","last_fullsized":"2024-04-24T06:30:06.885379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:26:45.435912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923759+0000","last_change":"2024-04-24T06:30:04.932542+0000","last_active":"2024-04-24T06:30:05.923759+0000","last_peered":"2024-04-24T06:30:05.923759+0000","last_clean":"2024-04-24T06:30:05.923759+0000","last_became_active":"2024-04-24T06:30:04.932243+0000","last_became_peered":"2024-04-24T06:30:04.932243+0000","last_unstale":"2024-04-24T06:30:05.923759+0000","last_undegraded":"2024-04-24T06:30:05.923759+0000","last_fullsized":"2024-04-24T06:30:05.923759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:02.444324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926328+0000","last_change":"2024-04-24T06:30:04.919485+0000","last_active":"2024-04-24T06:30:05.926328+0000","last_peered":"2024-04-24T06:30:05.926328+0000","last_clean":"2024-04-24T06:30:05.926328+0000","last_became_active":"2024-04-24T06:30:04.918226+0000","last_became_peered":"2024-04-24T06:30:04.918226+0000","last_unstale":"2024-04-24T06:30:05.926328+0000","last_undegraded":"2024-04-24T06:30:05.926328+0000","last_fullsized":"2024-04-24T06:30:05.926328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:53.692627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925857+0000","last_change":"2024-04-24T06:30:04.924384+0000","last_active":"2024-04-24T06:30:05.925857+0000","last_peered":"2024-04-24T06:30:05.925857+0000","last_clean":"2024-04-24T06:30:05.925857+0000","last_became_active":"2024-04-24T06:30:04.924038+0000","last_became_peered":"2024-04-24T06:30:04.924038+0000","last_unstale":"2024-04-24T06:30:05.925857+0000","last_undegraded":"2024-04-24T06:30:05.925857+0000","last_fullsized":"2024-04-24T06:30:05.925857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:01.909581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.944867+0000","last_change":"2024-04-24T06:30:04.927729+0000","last_active":"2024-04-24T06:30:05.944867+0000","last_peered":"2024-04-24T06:30:05.944867+0000","last_clean":"2024-04-24T06:30:05.944867+0000","last_became_active":"2024-04-24T06:30:04.926332+0000","last_became_peered":"2024-04-24T06:30:04.926332+0000","last_unstale":"2024-04-24T06:30:05.944867+0000","last_undegraded":"2024-04-24T06:30:05.944867+0000","last_fullsized":"2024-04-24T06:30:05.944867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:50.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885485+0000","last_change":"2024-04-24T06:30:04.923428+0000","last_active":"2024-04-24T06:30:06.885485+0000","last_peered":"2024-04-24T06:30:06.885485+0000","last_clean":"2024-04-24T06:30:06.885485+0000","last_became_active":"2024-04-24T06:30:04.922142+0000","last_became_peered":"2024-04-24T06:30:04.922142+0000","last_unstale":"2024-04-24T06:30:06.885485+0000","last_undegraded":"2024-04-24T06:30:06.885485+0000","last_fullsized":"2024-04-24T06:30:06.885485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:44:35.669397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952561+0000","last_change":"2024-04-24T06:30:04.924722+0000","last_active":"2024-04-24T06:30:05.952561+0000","last_peered":"2024-04-24T06:30:05.952561+0000","last_clean":"2024-04-24T06:30:05.952561+0000","last_became_active":"2024-04-24T06:30:04.922766+0000","last_became_peered":"2024-04-24T06:30:04.922766+0000","last_unstale":"2024-04-24T06:30:05.952561+0000","last_undegraded":"2024-04-24T06:30:05.952561+0000","last_fullsized":"2024-04-24T06:30:05.952561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:21.832041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886202+0000","last_change":"2024-04-24T06:30:04.928803+0000","last_active":"2024-04-24T06:30:06.886202+0000","last_peered":"2024-04-24T06:30:06.886202+0000","last_clean":"2024-04-24T06:30:06.886202+0000","last_became_active":"2024-04-24T06:30:04.927236+0000","last_became_peered":"2024-04-24T06:30:04.927236+0000","last_unstale":"2024-04-24T06:30:06.886202+0000","last_undegraded":"2024-04-24T06:30:06.886202+0000","last_fullsized":"2024-04-24T06:30:06.886202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:18:14.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952433+0000","last_change":"2024-04-24T06:30:04.930571+0000","last_active":"2024-04-24T06:30:05.952433+0000","last_peered":"2024-04-24T06:30:05.952433+0000","last_clean":"2024-04-24T06:30:05.952433+0000","last_became_active":"2024-04-24T06:30:04.929847+0000","last_became_peered":"2024-04-24T06:30:04.929847+0000","last_unstale":"2024-04-24T06:30:05.952433+0000","last_undegraded":"2024-04-24T06:30:05.952433+0000","last_fullsized":"2024-04-24T06:30:05.952433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:23.743368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886642+0000","last_change":"2024-04-24T06:30:04.920854+0000","last_active":"2024-04-24T06:30:06.886642+0000","last_peered":"2024-04-24T06:30:06.886642+0000","last_clean":"2024-04-24T06:30:06.886642+0000","last_became_active":"2024-04-24T06:30:04.919700+0000","last_became_peered":"2024-04-24T06:30:04.919700+0000","last_unstale":"2024-04-24T06:30:06.886642+0000","last_undegraded":"2024-04-24T06:30:06.886642+0000","last_fullsized":"2024-04-24T06:30:06.886642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:11.097944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925565+0000","last_change":"2024-04-24T06:30:04.925556+0000","last_active":"2024-04-24T06:30:05.925565+0000","last_peered":"2024-04-24T06:30:05.925565+0000","last_clean":"2024-04-24T06:30:05.925565+0000","last_became_active":"2024-04-24T06:30:04.923525+0000","last_became_peered":"2024-04-24T06:30:04.923525+0000","last_unstale":"2024-04-24T06:30:05.925565+0000","last_undegraded":"2024-04-24T06:30:05.925565+0000","last_fullsized":"2024-04-24T06:30:05.925565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299226+0000","last_change":"2024-04-24T06:30:04.934051+0000","last_active":"2024-04-24T06:30:06.299226+0000","last_peered":"2024-04-24T06:30:06.299226+0000","last_clean":"2024-04-24T06:30:06.299226+0000","last_became_active":"2024-04-24T06:30:04.933719+0000","last_became_peered":"2024-04-24T06:30:04.933719+0000","last_unstale":"2024-04-24T06:30:06.299226+0000","last_undegraded":"2024-04-24T06:30:06.299226+0000","last_fullsized":"2024-04-24T06:30:06.299226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:11.097944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885179+0000","last_change":"2024-04-24T06:30:04.928722+0000","last_active":"2024-04-24T06:30:06.885179+0000","last_peered":"2024-04-24T06:30:06.885179+0000","last_clean":"2024-04-24T06:30:06.885179+0000","last_became_active":"2024-04-24T06:30:04.928199+0000","last_became_peered":"2024-04-24T06:30:04.928199+0000","last_unstale":"2024-04-24T06:30:06.885179+0000","last_undegraded":"2024-04-24T06:30:06.885179+0000","last_fullsized":"2024-04-24T06:30:06.885179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299324+0000","last_change":"2024-04-24T06:30:04.928050+0000","last_active":"2024-04-24T06:30:06.299324+0000","last_peered":"2024-04-24T06:30:06.299324+0000","last_clean":"2024-04-24T06:30:06.299324+0000","last_became_active":"2024-04-24T06:30:04.927705+0000","last_became_peered":"2024-04-24T06:30:04.927705+0000","last_unstale":"2024-04-24T06:30:06.299324+0000","last_undegraded":"2024-04-24T06:30:06.299324+0000","last_fullsized":"2024-04-24T06:30:06.299324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945380+0000","last_change":"2024-04-24T06:30:04.926443+0000","last_active":"2024-04-24T06:30:05.945380+0000","last_peered":"2024-04-24T06:30:05.945380+0000","last_clean":"2024-04-24T06:30:05.945380+0000","last_became_active":"2024-04-24T06:30:04.926091+0000","last_became_peered":"2024-04-24T06:30:04.926091+0000","last_unstale":"2024-04-24T06:30:05.945380+0000","last_undegraded":"2024-04-24T06:30:05.945380+0000","last_fullsized":"2024-04-24T06:30:05.945380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885118+0000","last_change":"2024-04-24T06:30:03.888487+0000","last_active":"2024-04-24T06:30:06.885118+0000","last_peered":"2024-04-24T06:30:06.885118+0000","last_clean":"2024-04-24T06:30:06.885118+0000","last_became_active":"2024-04-24T06:30:03.887912+0000","last_became_peered":"2024-04-24T06:30:03.887912+0000","last_unstale":"2024-04-24T06:30:06.885118+0000","last_undegraded":"2024-04-24T06:30:06.885118+0000","last_fullsized":"2024-04-24T06:30:06.885118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884955+0000","last_change":"2024-04-24T06:30:04.924476+0000","last_active":"2024-04-24T06:30:06.884955+0000","last_peered":"2024-04-24T06:30:06.884955+0000","last_clean":"2024-04-24T06:30:06.884955+0000","last_became_active":"2024-04-24T06:30:04.924023+0000","last_became_peered":"2024-04-24T06:30:04.924023+0000","last_unstale":"2024-04-24T06:30:06.884955+0000","last_undegraded":"2024-04-24T06:30:06.884955+0000","last_fullsized":"2024-04-24T06:30:06.884955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:50.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945409+0000","last_change":"2024-04-24T06:30:03.889451+0000","last_active":"2024-04-24T06:30:05.945409+0000","last_peered":"2024-04-24T06:30:05.945409+0000","last_clean":"2024-04-24T06:30:05.945409+0000","last_became_active":"2024-04-24T06:30:03.889142+0000","last_became_peered":"2024-04-24T06:30:03.889142+0000","last_unstale":"2024-04-24T06:30:05.945409+0000","last_undegraded":"2024-04-24T06:30:05.945409+0000","last_fullsized":"2024-04-24T06:30:05.945409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926659+0000","last_change":"2024-04-24T06:30:04.929261+0000","last_active":"2024-04-24T06:30:05.926659+0000","last_peered":"2024-04-24T06:30:05.926659+0000","last_clean":"2024-04-24T06:30:05.926659+0000","last_became_active":"2024-04-24T06:30:04.928977+0000","last_became_peered":"2024-04-24T06:30:04.928977+0000","last_unstale":"2024-04-24T06:30:05.926659+0000","last_undegraded":"2024-04-24T06:30:05.926659+0000","last_fullsized":"2024-04-24T06:30:05.926659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:11:24.883801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885034+0000","last_change":"2024-04-24T06:30:03.887130+0000","last_active":"2024-04-24T06:30:06.885034+0000","last_peered":"2024-04-24T06:30:06.885034+0000","last_clean":"2024-04-24T06:30:06.885034+0000","last_became_active":"2024-04-24T06:30:03.886744+0000","last_became_peered":"2024-04-24T06:30:03.886744+0000","last_unstale":"2024-04-24T06:30:06.885034+0000","last_undegraded":"2024-04-24T06:30:06.885034+0000","last_fullsized":"2024-04-24T06:30:06.885034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:49.241036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952342+0000","last_change":"2024-04-24T06:30:04.930226+0000","last_active":"2024-04-24T06:30:05.952342+0000","last_peered":"2024-04-24T06:30:05.952342+0000","last_clean":"2024-04-24T06:30:05.952342+0000","last_became_active":"2024-04-24T06:30:04.928571+0000","last_became_peered":"2024-04-24T06:30:04.928571+0000","last_unstale":"2024-04-24T06:30:05.952342+0000","last_undegraded":"2024-04-24T06:30:05.952342+0000","last_fullsized":"2024-04-24T06:30:05.952342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:11.545726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923667+0000","last_change":"2024-04-24T06:30:03.879647+0000","last_active":"2024-04-24T06:30:05.923667+0000","last_peered":"2024-04-24T06:30:05.923667+0000","last_clean":"2024-04-24T06:30:05.923667+0000","last_became_active":"2024-04-24T06:30:03.879323+0000","last_became_peered":"2024-04-24T06:30:03.879323+0000","last_unstale":"2024-04-24T06:30:05.923667+0000","last_undegraded":"2024-04-24T06:30:05.923667+0000","last_fullsized":"2024-04-24T06:30:05.923667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:12.863252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299286+0000","last_change":"2024-04-24T06:30:04.929545+0000","last_active":"2024-04-24T06:30:06.299286+0000","last_peered":"2024-04-24T06:30:06.299286+0000","last_clean":"2024-04-24T06:30:06.299286+0000","last_became_active":"2024-04-24T06:30:04.929116+0000","last_became_peered":"2024-04-24T06:30:04.929116+0000","last_unstale":"2024-04-24T06:30:06.299286+0000","last_undegraded":"2024-04-24T06:30:06.299286+0000","last_fullsized":"2024-04-24T06:30:06.299286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:27:59.666126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926085+0000","last_change":"2024-04-24T06:30:03.886347+0000","last_active":"2024-04-24T06:30:05.926085+0000","last_peered":"2024-04-24T06:30:05.926085+0000","last_clean":"2024-04-24T06:30:05.926085+0000","last_became_active":"2024-04-24T06:30:03.886010+0000","last_became_peered":"2024-04-24T06:30:03.886010+0000","last_unstale":"2024-04-24T06:30:05.926085+0000","last_undegraded":"2024-04-24T06:30:05.926085+0000","last_fullsized":"2024-04-24T06:30:05.926085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:17.786539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886624+0000","last_change":"2024-04-24T06:30:04.918786+0000","last_active":"2024-04-24T06:30:06.886624+0000","last_peered":"2024-04-24T06:30:06.886624+0000","last_clean":"2024-04-24T06:30:06.886624+0000","last_became_active":"2024-04-24T06:30:04.918376+0000","last_became_peered":"2024-04-24T06:30:04.918376+0000","last_unstale":"2024-04-24T06:30:06.886624+0000","last_undegraded":"2024-04-24T06:30:06.886624+0000","last_fullsized":"2024-04-24T06:30:06.886624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:28.603915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925920+0000","last_change":"2024-04-24T06:30:03.891043+0000","last_active":"2024-04-24T06:30:05.925920+0000","last_peered":"2024-04-24T06:30:05.925920+0000","last_clean":"2024-04-24T06:30:05.925920+0000","last_became_active":"2024-04-24T06:30:03.887606+0000","last_became_peered":"2024-04-24T06:30:03.887606+0000","last_unstale":"2024-04-24T06:30:05.925920+0000","last_undegraded":"2024-04-24T06:30:05.925920+0000","last_fullsized":"2024-04-24T06:30:05.925920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:17:21.342442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925499+0000","last_change":"2024-04-24T06:30:04.932105+0000","last_active":"2024-04-24T06:30:05.925499+0000","last_peered":"2024-04-24T06:30:05.925499+0000","last_clean":"2024-04-24T06:30:05.925499+0000","last_became_active":"2024-04-24T06:30:04.931829+0000","last_became_peered":"2024-04-24T06:30:04.931829+0000","last_unstale":"2024-04-24T06:30:05.925499+0000","last_undegraded":"2024-04-24T06:30:05.925499+0000","last_fullsized":"2024-04-24T06:30:05.925499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884056+0000","last_change":"2024-04-24T06:30:03.875487+0000","last_active":"2024-04-24T06:30:06.884056+0000","last_peered":"2024-04-24T06:30:06.884056+0000","last_clean":"2024-04-24T06:30:06.884056+0000","last_became_active":"2024-04-24T06:30:03.875122+0000","last_became_peered":"2024-04-24T06:30:03.875122+0000","last_unstale":"2024-04-24T06:30:06.884056+0000","last_undegraded":"2024-04-24T06:30:06.884056+0000","last_fullsized":"2024-04-24T06:30:06.884056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:12:17.347720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950833+0000","last_change":"2024-04-24T06:30:04.928526+0000","last_active":"2024-04-24T06:30:05.950833+0000","last_peered":"2024-04-24T06:30:05.950833+0000","last_clean":"2024-04-24T06:30:05.950833+0000","last_became_active":"2024-04-24T06:30:04.928221+0000","last_became_peered":"2024-04-24T06:30:04.928221+0000","last_unstale":"2024-04-24T06:30:05.950833+0000","last_undegraded":"2024-04-24T06:30:05.950833+0000","last_fullsized":"2024-04-24T06:30:05.950833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925287+0000","last_change":"2024-04-24T06:30:03.881647+0000","last_active":"2024-04-24T06:30:05.925287+0000","last_peered":"2024-04-24T06:30:05.925287+0000","last_clean":"2024-04-24T06:30:05.925287+0000","last_became_active":"2024-04-24T06:30:03.881317+0000","last_became_peered":"2024-04-24T06:30:03.881317+0000","last_unstale":"2024-04-24T06:30:05.925287+0000","last_undegraded":"2024-04-24T06:30:05.925287+0000","last_fullsized":"2024-04-24T06:30:05.925287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:04.946499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886846+0000","last_change":"2024-04-24T06:30:04.929548+0000","last_active":"2024-04-24T06:30:06.886846+0000","last_peered":"2024-04-24T06:30:06.886846+0000","last_clean":"2024-04-24T06:30:06.886846+0000","last_became_active":"2024-04-24T06:30:04.928673+0000","last_became_peered":"2024-04-24T06:30:04.928673+0000","last_unstale":"2024-04-24T06:30:06.886846+0000","last_undegraded":"2024-04-24T06:30:06.886846+0000","last_fullsized":"2024-04-24T06:30:06.886846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:27:59.666126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885646+0000","last_change":"2024-04-24T06:30:03.888302+0000","last_active":"2024-04-24T06:30:06.885646+0000","last_peered":"2024-04-24T06:30:06.885646+0000","last_clean":"2024-04-24T06:30:06.885646+0000","last_became_active":"2024-04-24T06:30:03.887740+0000","last_became_peered":"2024-04-24T06:30:03.887740+0000","last_unstale":"2024-04-24T06:30:06.885646+0000","last_undegraded":"2024-04-24T06:30:06.885646+0000","last_fullsized":"2024-04-24T06:30:06.885646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:44.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885782+0000","last_change":"2024-04-24T06:30:04.933384+0000","last_active":"2024-04-24T06:30:06.885782+0000","last_peered":"2024-04-24T06:30:06.885782+0000","last_clean":"2024-04-24T06:30:06.885782+0000","last_became_active":"2024-04-24T06:30:04.932460+0000","last_became_peered":"2024-04-24T06:30:04.932460+0000","last_unstale":"2024-04-24T06:30:06.885782+0000","last_undegraded":"2024-04-24T06:30:06.885782+0000","last_fullsized":"2024-04-24T06:30:06.885782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:03.454047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945592+0000","last_change":"2024-04-24T06:30:03.883718+0000","last_active":"2024-04-24T06:30:05.945592+0000","last_peered":"2024-04-24T06:30:05.945592+0000","last_clean":"2024-04-24T06:30:05.945592+0000","last_became_active":"2024-04-24T06:30:03.883341+0000","last_became_peered":"2024-04-24T06:30:03.883341+0000","last_unstale":"2024-04-24T06:30:05.945592+0000","last_undegraded":"2024-04-24T06:30:05.945592+0000","last_fullsized":"2024-04-24T06:30:05.945592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:13.477762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885346+0000","last_change":"2024-04-24T06:30:04.927340+0000","last_active":"2024-04-24T06:30:06.885346+0000","last_peered":"2024-04-24T06:30:06.885346+0000","last_clean":"2024-04-24T06:30:06.885346+0000","last_became_active":"2024-04-24T06:30:04.922409+0000","last_became_peered":"2024-04-24T06:30:04.922409+0000","last_unstale":"2024-04-24T06:30:06.885346+0000","last_undegraded":"2024-04-24T06:30:06.885346+0000","last_fullsized":"2024-04-24T06:30:06.885346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887337+0000","last_change":"2024-04-24T06:30:03.890088+0000","last_active":"2024-04-24T06:30:06.887337+0000","last_peered":"2024-04-24T06:30:06.887337+0000","last_clean":"2024-04-24T06:30:06.887337+0000","last_became_active":"2024-04-24T06:30:03.889759+0000","last_became_peered":"2024-04-24T06:30:03.889759+0000","last_unstale":"2024-04-24T06:30:06.887337+0000","last_undegraded":"2024-04-24T06:30:06.887337+0000","last_fullsized":"2024-04-24T06:30:06.887337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:46.580010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887133+0000","last_change":"2024-04-24T06:30:04.919211+0000","last_active":"2024-04-24T06:30:06.887133+0000","last_peered":"2024-04-24T06:30:06.887133+0000","last_clean":"2024-04-24T06:30:06.887133+0000","last_became_active":"2024-04-24T06:30:04.917110+0000","last_became_peered":"2024-04-24T06:30:04.917110+0000","last_unstale":"2024-04-24T06:30:06.887133+0000","last_undegraded":"2024-04-24T06:30:06.887133+0000","last_fullsized":"2024-04-24T06:30:06.887133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885205+0000","last_change":"2024-04-24T06:30:03.886204+0000","last_active":"2024-04-24T06:30:06.885205+0000","last_peered":"2024-04-24T06:30:06.885205+0000","last_clean":"2024-04-24T06:30:06.885205+0000","last_became_active":"2024-04-24T06:30:03.885275+0000","last_became_peered":"2024-04-24T06:30:03.885275+0000","last_unstale":"2024-04-24T06:30:06.885205+0000","last_undegraded":"2024-04-24T06:30:06.885205+0000","last_fullsized":"2024-04-24T06:30:06.885205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:00.903880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950945+0000","last_change":"2024-04-24T06:30:04.915707+0000","last_active":"2024-04-24T06:30:05.950945+0000","last_peered":"2024-04-24T06:30:05.950945+0000","last_clean":"2024-04-24T06:30:05.950945+0000","last_became_active":"2024-04-24T06:30:04.915397+0000","last_became_peered":"2024-04-24T06:30:04.915397+0000","last_unstale":"2024-04-24T06:30:05.950945+0000","last_undegraded":"2024-04-24T06:30:05.950945+0000","last_fullsized":"2024-04-24T06:30:05.950945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:01.909581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926605+0000","last_change":"2024-04-24T06:30:03.882040+0000","last_active":"2024-04-24T06:30:05.926605+0000","last_peered":"2024-04-24T06:30:05.926605+0000","last_clean":"2024-04-24T06:30:05.926605+0000","last_became_active":"2024-04-24T06:30:03.881706+0000","last_became_peered":"2024-04-24T06:30:03.881706+0000","last_unstale":"2024-04-24T06:30:05.926605+0000","last_undegraded":"2024-04-24T06:30:05.926605+0000","last_fullsized":"2024-04-24T06:30:05.926605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:49.727727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885096+0000","last_change":"2024-04-24T06:30:04.921973+0000","last_active":"2024-04-24T06:30:06.885096+0000","last_peered":"2024-04-24T06:30:06.885096+0000","last_clean":"2024-04-24T06:30:06.885096+0000","last_became_active":"2024-04-24T06:30:04.921571+0000","last_became_peered":"2024-04-24T06:30:04.921571+0000","last_unstale":"2024-04-24T06:30:06.885096+0000","last_undegraded":"2024-04-24T06:30:06.885096+0000","last_fullsized":"2024-04-24T06:30:06.885096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950918+0000","last_change":"2024-04-24T06:30:03.886083+0000","last_active":"2024-04-24T06:30:05.950918+0000","last_peered":"2024-04-24T06:30:05.950918+0000","last_clean":"2024-04-24T06:30:05.950918+0000","last_became_active":"2024-04-24T06:30:03.885767+0000","last_became_peered":"2024-04-24T06:30:03.885767+0000","last_unstale":"2024-04-24T06:30:05.950918+0000","last_undegraded":"2024-04-24T06:30:05.950918+0000","last_fullsized":"2024-04-24T06:30:05.950918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299543+0000","last_change":"2024-04-24T06:30:04.935400+0000","last_active":"2024-04-24T06:30:06.299543+0000","last_peered":"2024-04-24T06:30:06.299543+0000","last_clean":"2024-04-24T06:30:06.299543+0000","last_became_active":"2024-04-24T06:30:04.935078+0000","last_became_peered":"2024-04-24T06:30:04.935078+0000","last_unstale":"2024-04-24T06:30:06.299543+0000","last_undegraded":"2024-04-24T06:30:06.299543+0000","last_fullsized":"2024-04-24T06:30:06.299543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884234+0000","last_change":"2024-04-24T06:30:03.887884+0000","last_active":"2024-04-24T06:30:06.884234+0000","last_peered":"2024-04-24T06:30:06.884234+0000","last_clean":"2024-04-24T06:30:06.884234+0000","last_became_active":"2024-04-24T06:30:03.887605+0000","last_became_peered":"2024-04-24T06:30:03.887605+0000","last_unstale":"2024-04-24T06:30:06.884234+0000","last_undegraded":"2024-04-24T06:30:06.884234+0000","last_fullsized":"2024-04-24T06:30:06.884234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:41.040181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885445+0000","last_change":"2024-04-24T06:30:04.919119+0000","last_active":"2024-04-24T06:30:06.885445+0000","last_peered":"2024-04-24T06:30:06.885445+0000","last_clean":"2024-04-24T06:30:06.885445+0000","last_became_active":"2024-04-24T06:30:04.918800+0000","last_became_peered":"2024-04-24T06:30:04.918800+0000","last_unstale":"2024-04-24T06:30:06.885445+0000","last_undegraded":"2024-04-24T06:30:06.885445+0000","last_fullsized":"2024-04-24T06:30:06.885445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884223+0000","last_change":"2024-04-24T06:30:03.884365+0000","last_active":"2024-04-24T06:30:06.884223+0000","last_peered":"2024-04-24T06:30:06.884223+0000","last_clean":"2024-04-24T06:30:06.884223+0000","last_became_active":"2024-04-24T06:30:03.884014+0000","last_became_peered":"2024-04-24T06:30:03.884014+0000","last_unstale":"2024-04-24T06:30:06.884223+0000","last_undegraded":"2024-04-24T06:30:06.884223+0000","last_fullsized":"2024-04-24T06:30:06.884223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:00.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927014+0000","last_change":"2024-04-24T06:30:04.915566+0000","last_active":"2024-04-24T06:30:05.927014+0000","last_peered":"2024-04-24T06:30:05.927014+0000","last_clean":"2024-04-24T06:30:05.927014+0000","last_became_active":"2024-04-24T06:30:04.915269+0000","last_became_peered":"2024-04-24T06:30:04.915269+0000","last_unstale":"2024-04-24T06:30:05.927014+0000","last_undegraded":"2024-04-24T06:30:05.927014+0000","last_fullsized":"2024-04-24T06:30:05.927014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:11.802512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952504+0000","last_change":"2024-04-24T06:30:03.883575+0000","last_active":"2024-04-24T06:30:05.952504+0000","last_peered":"2024-04-24T06:30:05.952504+0000","last_clean":"2024-04-24T06:30:05.952504+0000","last_became_active":"2024-04-24T06:30:03.883205+0000","last_became_peered":"2024-04-24T06:30:03.883205+0000","last_unstale":"2024-04-24T06:30:05.952504+0000","last_undegraded":"2024-04-24T06:30:05.952504+0000","last_fullsized":"2024-04-24T06:30:05.952504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:13.477762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887212+0000","last_change":"2024-04-24T06:30:04.911788+0000","last_active":"2024-04-24T06:30:06.887212+0000","last_peered":"2024-04-24T06:30:06.887212+0000","last_clean":"2024-04-24T06:30:06.887212+0000","last_became_active":"2024-04-24T06:30:04.911173+0000","last_became_peered":"2024-04-24T06:30:04.911173+0000","last_unstale":"2024-04-24T06:30:06.887212+0000","last_undegraded":"2024-04-24T06:30:06.887212+0000","last_fullsized":"2024-04-24T06:30:06.887212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927016+0000","last_change":"2024-04-24T06:30:03.887139+0000","last_active":"2024-04-24T06:30:05.927016+0000","last_peered":"2024-04-24T06:30:05.927016+0000","last_clean":"2024-04-24T06:30:05.927016+0000","last_became_active":"2024-04-24T06:30:03.886778+0000","last_became_peered":"2024-04-24T06:30:03.886778+0000","last_unstale":"2024-04-24T06:30:05.927016+0000","last_undegraded":"2024-04-24T06:30:05.927016+0000","last_fullsized":"2024-04-24T06:30:05.927016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951394+0000","last_change":"2024-04-24T06:30:04.916986+0000","last_active":"2024-04-24T06:30:05.951394+0000","last_peered":"2024-04-24T06:30:05.951394+0000","last_clean":"2024-04-24T06:30:05.951394+0000","last_became_active":"2024-04-24T06:30:04.916640+0000","last_became_peered":"2024-04-24T06:30:04.916640+0000","last_unstale":"2024-04-24T06:30:05.951394+0000","last_undegraded":"2024-04-24T06:30:05.951394+0000","last_fullsized":"2024-04-24T06:30:05.951394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:53.692627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926784+0000","last_change":"2024-04-24T06:30:03.875124+0000","last_active":"2024-04-24T06:30:05.926784+0000","last_peered":"2024-04-24T06:30:05.926784+0000","last_clean":"2024-04-24T06:30:05.926784+0000","last_became_active":"2024-04-24T06:30:03.874834+0000","last_became_peered":"2024-04-24T06:30:03.874834+0000","last_unstale":"2024-04-24T06:30:05.926784+0000","last_undegraded":"2024-04-24T06:30:05.926784+0000","last_fullsized":"2024-04-24T06:30:05.926784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.924572+0000","last_change":"2024-04-24T06:30:04.924613+0000","last_active":"2024-04-24T06:30:05.924572+0000","last_peered":"2024-04-24T06:30:05.924572+0000","last_clean":"2024-04-24T06:30:05.924572+0000","last_became_active":"2024-04-24T06:30:04.924121+0000","last_became_peered":"2024-04-24T06:30:04.924121+0000","last_unstale":"2024-04-24T06:30:05.924572+0000","last_undegraded":"2024-04-24T06:30:05.924572+0000","last_fullsized":"2024-04-24T06:30:05.924572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:57:07.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926529+0000","last_change":"2024-04-24T06:30:03.891071+0000","last_active":"2024-04-24T06:30:05.926529+0000","last_peered":"2024-04-24T06:30:05.926529+0000","last_clean":"2024-04-24T06:30:05.926529+0000","last_became_active":"2024-04-24T06:30:03.889238+0000","last_became_peered":"2024-04-24T06:30:03.889238+0000","last_unstale":"2024-04-24T06:30:05.926529+0000","last_undegraded":"2024-04-24T06:30:05.926529+0000","last_fullsized":"2024-04-24T06:30:05.926529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923166+0000","last_change":"2024-04-24T06:30:04.928264+0000","last_active":"2024-04-24T06:30:05.923166+0000","last_peered":"2024-04-24T06:30:05.923166+0000","last_clean":"2024-04-24T06:30:05.923166+0000","last_became_active":"2024-04-24T06:30:04.927986+0000","last_became_peered":"2024-04-24T06:30:04.927986+0000","last_unstale":"2024-04-24T06:30:05.923166+0000","last_undegraded":"2024-04-24T06:30:05.923166+0000","last_fullsized":"2024-04-24T06:30:05.923166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:50.733428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945359+0000","last_change":"2024-04-24T06:30:03.876785+0000","last_active":"2024-04-24T06:30:05.945359+0000","last_peered":"2024-04-24T06:30:05.945359+0000","last_clean":"2024-04-24T06:30:05.945359+0000","last_became_active":"2024-04-24T06:30:03.876105+0000","last_became_peered":"2024-04-24T06:30:03.876105+0000","last_unstale":"2024-04-24T06:30:05.945359+0000","last_undegraded":"2024-04-24T06:30:05.945359+0000","last_fullsized":"2024-04-24T06:30:05.945359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923760+0000","last_change":"2024-04-24T06:30:04.919412+0000","last_active":"2024-04-24T06:30:05.923760+0000","last_peered":"2024-04-24T06:30:05.923760+0000","last_clean":"2024-04-24T06:30:05.923760+0000","last_became_active":"2024-04-24T06:30:04.919114+0000","last_became_peered":"2024-04-24T06:30:04.919114+0000","last_unstale":"2024-04-24T06:30:05.923760+0000","last_undegraded":"2024-04-24T06:30:05.923760+0000","last_fullsized":"2024-04-24T06:30:05.923760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:25.983849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885290+0000","last_change":"2024-04-24T06:30:03.889395+0000","last_active":"2024-04-24T06:30:06.885290+0000","last_peered":"2024-04-24T06:30:06.885290+0000","last_clean":"2024-04-24T06:30:06.885290+0000","last_became_active":"2024-04-24T06:30:03.888579+0000","last_became_peered":"2024-04-24T06:30:03.888579+0000","last_unstale":"2024-04-24T06:30:06.885290+0000","last_undegraded":"2024-04-24T06:30:06.885290+0000","last_fullsized":"2024-04-24T06:30:06.885290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923410+0000","last_change":"2024-04-24T06:30:04.920396+0000","last_active":"2024-04-24T06:30:05.923410+0000","last_peered":"2024-04-24T06:30:05.923410+0000","last_clean":"2024-04-24T06:30:05.923410+0000","last_became_active":"2024-04-24T06:30:04.920127+0000","last_became_peered":"2024-04-24T06:30:04.920127+0000","last_unstale":"2024-04-24T06:30:05.923410+0000","last_undegraded":"2024-04-24T06:30:05.923410+0000","last_fullsized":"2024-04-24T06:30:05.923410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:01.696611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925494+0000","last_change":"2024-04-24T06:30:03.876388+0000","last_active":"2024-04-24T06:30:05.925494+0000","last_peered":"2024-04-24T06:30:05.925494+0000","last_clean":"2024-04-24T06:30:05.925494+0000","last_became_active":"2024-04-24T06:30:03.876057+0000","last_became_peered":"2024-04-24T06:30:03.876057+0000","last_unstale":"2024-04-24T06:30:05.925494+0000","last_undegraded":"2024-04-24T06:30:05.925494+0000","last_fullsized":"2024-04-24T06:30:05.925494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:12.152699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299229+0000","last_change":"2024-04-24T06:30:04.933750+0000","last_active":"2024-04-24T06:30:06.299229+0000","last_peered":"2024-04-24T06:30:06.299229+0000","last_clean":"2024-04-24T06:30:06.299229+0000","last_became_active":"2024-04-24T06:30:04.933380+0000","last_became_peered":"2024-04-24T06:30:04.933380+0000","last_unstale":"2024-04-24T06:30:06.299229+0000","last_undegraded":"2024-04-24T06:30:06.299229+0000","last_fullsized":"2024-04-24T06:30:06.299229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:28.603915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885201+0000","last_change":"2024-04-24T06:30:03.884607+0000","last_active":"2024-04-24T06:30:06.885201+0000","last_peered":"2024-04-24T06:30:06.885201+0000","last_clean":"2024-04-24T06:30:06.885201+0000","last_became_active":"2024-04-24T06:30:03.884246+0000","last_became_peered":"2024-04-24T06:30:03.884246+0000","last_unstale":"2024-04-24T06:30:06.885201+0000","last_undegraded":"2024-04-24T06:30:06.885201+0000","last_fullsized":"2024-04-24T06:30:06.885201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:49.241036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923932+0000","last_change":"2024-04-24T06:30:04.928815+0000","last_active":"2024-04-24T06:30:05.923932+0000","last_peered":"2024-04-24T06:30:05.923932+0000","last_clean":"2024-04-24T06:30:05.923932+0000","last_became_active":"2024-04-24T06:30:04.927732+0000","last_became_peered":"2024-04-24T06:30:04.927732+0000","last_unstale":"2024-04-24T06:30:05.923932+0000","last_undegraded":"2024-04-24T06:30:05.923932+0000","last_fullsized":"2024-04-24T06:30:05.923932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:14.483463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952631+0000","last_change":"2024-04-24T06:30:03.884499+0000","last_active":"2024-04-24T06:30:05.952631+0000","last_peered":"2024-04-24T06:30:05.952631+0000","last_clean":"2024-04-24T06:30:05.952631+0000","last_became_active":"2024-04-24T06:30:03.884171+0000","last_became_peered":"2024-04-24T06:30:03.884171+0000","last_unstale":"2024-04-24T06:30:05.952631+0000","last_undegraded":"2024-04-24T06:30:05.952631+0000","last_fullsized":"2024-04-24T06:30:05.952631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:24.978148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887435+0000","last_change":"2024-04-24T06:30:04.931777+0000","last_active":"2024-04-24T06:30:06.887435+0000","last_peered":"2024-04-24T06:30:06.887435+0000","last_clean":"2024-04-24T06:30:06.887435+0000","last_became_active":"2024-04-24T06:30:04.931384+0000","last_became_peered":"2024-04-24T06:30:04.931384+0000","last_unstale":"2024-04-24T06:30:06.887435+0000","last_undegraded":"2024-04-24T06:30:06.887435+0000","last_fullsized":"2024-04-24T06:30:06.887435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945507+0000","last_change":"2024-04-24T06:30:03.879401+0000","last_active":"2024-04-24T06:30:05.945507+0000","last_peered":"2024-04-24T06:30:05.945507+0000","last_clean":"2024-04-24T06:30:05.945507+0000","last_became_active":"2024-04-24T06:30:03.879127+0000","last_became_peered":"2024-04-24T06:30:03.879127+0000","last_unstale":"2024-04-24T06:30:05.945507+0000","last_undegraded":"2024-04-24T06:30:05.945507+0000","last_fullsized":"2024-04-24T06:30:05.945507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:24.978148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945386+0000","last_change":"2024-04-24T06:30:04.922693+0000","last_active":"2024-04-24T06:30:05.945386+0000","last_peered":"2024-04-24T06:30:05.945386+0000","last_clean":"2024-04-24T06:30:05.945386+0000","last_became_active":"2024-04-24T06:30:04.922352+0000","last_became_peered":"2024-04-24T06:30:04.922352+0000","last_unstale":"2024-04-24T06:30:05.945386+0000","last_undegraded":"2024-04-24T06:30:05.945386+0000","last_fullsized":"2024-04-24T06:30:05.945386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:47.585711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951451+0000","last_change":"2024-04-24T06:30:03.876216+0000","last_active":"2024-04-24T06:30:05.951451+0000","last_peered":"2024-04-24T06:30:05.951451+0000","last_clean":"2024-04-24T06:30:05.951451+0000","last_became_active":"2024-04-24T06:30:03.875888+0000","last_became_peered":"2024-04-24T06:30:03.875888+0000","last_unstale":"2024-04-24T06:30:05.951451+0000","last_undegraded":"2024-04-24T06:30:05.951451+0000","last_fullsized":"2024-04-24T06:30:05.951451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885126+0000","last_change":"2024-04-24T06:30:04.922166+0000","last_active":"2024-04-24T06:30:06.885126+0000","last_peered":"2024-04-24T06:30:06.885126+0000","last_clean":"2024-04-24T06:30:06.885126+0000","last_became_active":"2024-04-24T06:30:04.921823+0000","last_became_peered":"2024-04-24T06:30:04.921823+0000","last_unstale":"2024-04-24T06:30:06.885126+0000","last_undegraded":"2024-04-24T06:30:06.885126+0000","last_fullsized":"2024-04-24T06:30:06.885126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:47.585711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945757+0000","last_change":"2024-04-24T06:30:03.878948+0000","last_active":"2024-04-24T06:30:05.945757+0000","last_peered":"2024-04-24T06:30:05.945757+0000","last_clean":"2024-04-24T06:30:05.945757+0000","last_became_active":"2024-04-24T06:30:03.878652+0000","last_became_peered":"2024-04-24T06:30:03.878652+0000","last_unstale":"2024-04-24T06:30:05.945757+0000","last_undegraded":"2024-04-24T06:30:05.945757+0000","last_fullsized":"2024-04-24T06:30:05.945757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945600+0000","last_change":"2024-04-24T06:30:04.923459+0000","last_active":"2024-04-24T06:30:05.945600+0000","last_peered":"2024-04-24T06:30:05.945600+0000","last_clean":"2024-04-24T06:30:05.945600+0000","last_became_active":"2024-04-24T06:30:04.923148+0000","last_became_peered":"2024-04-24T06:30:04.923148+0000","last_unstale":"2024-04-24T06:30:05.945600+0000","last_undegraded":"2024-04-24T06:30:05.945600+0000","last_fullsized":"2024-04-24T06:30:05.945600+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951057+0000","last_change":"2024-04-24T06:30:03.885358+0000","last_active":"2024-04-24T06:30:05.951057+0000","last_peered":"2024-04-24T06:30:05.951057+0000","last_clean":"2024-04-24T06:30:05.951057+0000","last_became_active":"2024-04-24T06:30:03.884950+0000","last_became_peered":"2024-04-24T06:30:03.884950+0000","last_unstale":"2024-04-24T06:30:05.951057+0000","last_undegraded":"2024-04-24T06:30:05.951057+0000","last_fullsized":"2024-04-24T06:30:05.951057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884315+0000","last_change":"2024-04-24T06:30:04.922987+0000","last_active":"2024-04-24T06:30:06.884315+0000","last_peered":"2024-04-24T06:30:06.884315+0000","last_clean":"2024-04-24T06:30:06.884315+0000","last_became_active":"2024-04-24T06:30:04.922689+0000","last_became_peered":"2024-04-24T06:30:04.922689+0000","last_unstale":"2024-04-24T06:30:06.884315+0000","last_undegraded":"2024-04-24T06:30:06.884315+0000","last_fullsized":"2024-04-24T06:30:06.884315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925834+0000","last_change":"2024-04-24T06:30:03.881474+0000","last_active":"2024-04-24T06:30:05.925834+0000","last_peered":"2024-04-24T06:30:05.925834+0000","last_clean":"2024-04-24T06:30:05.925834+0000","last_became_active":"2024-04-24T06:30:03.881098+0000","last_became_peered":"2024-04-24T06:30:03.881098+0000","last_unstale":"2024-04-24T06:30:05.925834+0000","last_undegraded":"2024-04-24T06:30:05.925834+0000","last_fullsized":"2024-04-24T06:30:05.925834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:50.746364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925781+0000","last_change":"2024-04-24T06:30:04.926464+0000","last_active":"2024-04-24T06:30:05.925781+0000","last_peered":"2024-04-24T06:30:05.925781+0000","last_clean":"2024-04-24T06:30:05.925781+0000","last_became_active":"2024-04-24T06:30:04.923665+0000","last_became_peered":"2024-04-24T06:30:04.923665+0000","last_unstale":"2024-04-24T06:30:05.925781+0000","last_undegraded":"2024-04-24T06:30:05.925781+0000","last_fullsized":"2024-04-24T06:30:05.925781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884450+0000","last_change":"2024-04-24T06:30:03.887654+0000","last_active":"2024-04-24T06:30:06.884450+0000","last_peered":"2024-04-24T06:30:06.884450+0000","last_clean":"2024-04-24T06:30:06.884450+0000","last_became_active":"2024-04-24T06:30:03.887345+0000","last_became_peered":"2024-04-24T06:30:03.887345+0000","last_unstale":"2024-04-24T06:30:06.884450+0000","last_undegraded":"2024-04-24T06:30:06.884450+0000","last_fullsized":"2024-04-24T06:30:06.884450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:46.580010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299839+0000","last_change":"2024-04-24T06:30:04.937326+0000","last_active":"2024-04-24T06:30:06.299839+0000","last_peered":"2024-04-24T06:30:06.299839+0000","last_clean":"2024-04-24T06:30:06.299839+0000","last_became_active":"2024-04-24T06:30:04.936959+0000","last_became_peered":"2024-04-24T06:30:04.936959+0000","last_unstale":"2024-04-24T06:30:06.299839+0000","last_undegraded":"2024-04-24T06:30:06.299839+0000","last_fullsized":"2024-04-24T06:30:06.299839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884324+0000","last_change":"2024-04-24T06:30:03.890720+0000","last_active":"2024-04-24T06:30:06.884324+0000","last_peered":"2024-04-24T06:30:06.884324+0000","last_clean":"2024-04-24T06:30:06.884324+0000","last_became_active":"2024-04-24T06:30:03.890376+0000","last_became_peered":"2024-04-24T06:30:03.890376+0000","last_unstale":"2024-04-24T06:30:06.884324+0000","last_undegraded":"2024-04-24T06:30:06.884324+0000","last_fullsized":"2024-04-24T06:30:06.884324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.300116+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:30:06.300116+0000","last_peered":"2024-04-24T06:30:06.300116+0000","last_clean":"2024-04-24T06:30:06.300116+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:30:06.300116+0000","last_undegraded":"2024-04-24T06:30:06.300116+0000","last_fullsized":"2024-04-24T06:30:06.300116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:30:09.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:30:09.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.673+0000 7fa3ba000640 1 Processor -- start 2024-04-24T06:30:09.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.674+0000 7fa3ba000640 1 -- start start 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3b40cd590 0x7fa3b40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3b40b4690 0x7fa3b406b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b40cd9f0 con 0x7fa3b40b4690 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b4160210 con 0x7fa3b40cd590 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.675+0000 7fa3ba000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b41604f0 con 0x7fa3b415c3d0 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3b40b4690 0x7fa3b406b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3b40b4690 0x7fa3b406b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40080/0 (socket says 172.21.15.59:40080) 2024-04-24T06:30:09.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 -- 172.21.15.59:0/2209646267 learned_addr learned my addr 172.21.15.59:0/2209646267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:09.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b8c00640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b3e00640 1 --2- 172.21.15.59:0/2209646267 >> v2:172.21.15.66:3300/0 conn(0x7fa3b40cd590 0x7fa3b40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 -- 172.21.15.59:0/2209646267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 msgr2=0x7fa3b40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 -- 172.21.15.59:0/2209646267 >> v2:172.21.15.66:3300/0 conn(0x7fa3b40cd590 msgr2=0x7fa3b40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 --2- 172.21.15.59:0/2209646267 >> v2:172.21.15.66:3300/0 conn(0x7fa3b40cd590 0x7fa3b40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.676+0000 7fa3b9600640 1 -- 172.21.15.59:0/2209646267 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41607d0 con 0x7fa3b40b4690 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.677+0000 7fa3b9600640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b406b9d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40b2cc0 tx=0x7fa3ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f3c02babce92cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.678+0000 7fa3b3400640 1 -- 172.21.15.59:0/2209646267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac036070 con 0x7fa3b40b4690 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.678+0000 7fa3b3400640 1 -- 172.21.15.59:0/2209646267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ac031600 con 0x7fa3b40b4690 2024-04-24T06:30:09.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.678+0000 7fa3b3400640 1 -- 172.21.15.59:0/2209646267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0319d0 con 0x7fa3b40b4690 2024-04-24T06:30:09.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 -- 172.21.15.59:0/2209646267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 msgr2=0x7fa3b406b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b406b9d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40b2cc0 tx=0x7fa3ac02a6a0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 -- 172.21.15.59:0/2209646267 shutdown_connections 2024-04-24T06:30:09.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b406b9d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 --2- 172.21.15.59:0/2209646267 >> v2:172.21.15.66:3300/0 conn(0x7fa3b40cd590 0x7fa3b40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 --2- 172.21.15.59:0/2209646267 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 -- 172.21.15.59:0/2209646267 >> 172.21.15.59:0/2209646267 conn(0x7fa3b40b21a0 msgr2=0x7fa3b40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:09.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 -- 172.21.15.59:0/2209646267 shutdown_connections 2024-04-24T06:30:09.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.679+0000 7fa3ba000640 1 -- 172.21.15.59:0/2209646267 wait complete. 2024-04-24T06:30:09.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.681+0000 7fa3ba000640 1 Processor -- start 2024-04-24T06:30:09.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.681+0000 7fa3ba000640 1 -- start start 2024-04-24T06:30:09.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b40cd590 0x7fa3b415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b41f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b40b4ae0 con 0x7fa3b40b4690 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4160d90 con 0x7fa3b40cd590 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3ba000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b415c7b0 con 0x7fa3b415c3d0 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3b8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3b8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40096/0 (socket says 172.21.15.59:40096) 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.682+0000 7fa3b8c00640 1 -- 172.21.15.59:0/4155623335 learned_addr learned my addr 172.21.15.59:0/4155623335 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b3e00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b40cd590 0x7fa3b415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b9600640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b41f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 -- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 msgr2=0x7fa3b41f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b41f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 -- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b40cd590 msgr2=0x7fa3b415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b40cd590 0x7fa3b415f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 -- 172.21.15.59:0/4155623335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b40ad760 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b8c00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800d830 tx=0x7fa3a800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.683+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800b660 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.684+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a8019070 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.684+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800ba20 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.684+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b40ada40 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.684+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b40ae000 con 0x7fa3b40b4690 2024-04-24T06:30:09.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.685+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b404fd90 con 0x7fa3b40b4690 2024-04-24T06:30:09.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.686+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3a801e460 con 0x7fa3b40b4690 2024-04-24T06:30:09.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.687+0000 7fa3b0c00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 0x7fa39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:09.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.687+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3a808c180 con 0x7fa3b40b4690 2024-04-24T06:30:09.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.687+0000 7fa3b3e00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 0x7fa39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:09.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.688+0000 7fa3b3e00640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 0x7fa39406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40d0bb0 tx=0x7fa3a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:09.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.693+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3a805a5b0 con 0x7fa3b40b4690 2024-04-24T06:30:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.894+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3b406e400 con 0x7fa39406c8f0 2024-04-24T06:30:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.902+0000 7fa3b0c00640 1 -- 172.21.15.59:0/4155623335 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fa3b406e400 con 0x7fa39406c8f0 2024-04-24T06:30:09.907 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:30:09.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 msgr2=0x7fa39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 0x7fa39406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40d0bb0 tx=0x7fa3a0008040 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 msgr2=0x7fa3b40ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800d830 tx=0x7fa3a800dcf0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 shutdown_connections 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa39406c8f0 0x7fa39406edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3b415c3d0 0x7fa3b41f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b40cd590 0x7fa3b415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 --2- 172.21.15.59:0/4155623335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3b40b4690 0x7fa3b40ad060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 >> 172.21.15.59:0/4155623335 conn(0x7fa3b40b21a0 msgr2=0x7fa3b4157990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 shutdown_connections 2024-04-24T06:30:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:09.908+0000 7fa3ba000640 1 -- 172.21.15.59:0/4155623335 wait complete. 2024-04-24T06:30:09.929 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-24T06:30:09.212452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001389"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884852+0000","last_change":"2024-04-24T06:30:04.932359+0000","last_active":"2024-04-24T06:30:06.884852+0000","last_peered":"2024-04-24T06:30:06.884852+0000","last_clean":"2024-04-24T06:30:06.884852+0000","last_became_active":"2024-04-24T06:30:04.932078+0000","last_became_peered":"2024-04-24T06:30:04.932078+0000","last_unstale":"2024-04-24T06:30:06.884852+0000","last_undegraded":"2024-04-24T06:30:06.884852+0000","last_fullsized":"2024-04-24T06:30:06.884852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:12:18.353421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885910+0000","last_change":"2024-04-24T06:30:04.933665+0000","last_active":"2024-04-24T06:30:06.885910+0000","last_peered":"2024-04-24T06:30:06.885910+0000","last_clean":"2024-04-24T06:30:06.885910+0000","last_became_active":"2024-04-24T06:30:04.933103+0000","last_became_peered":"2024-04-24T06:30:04.933103+0000","last_unstale":"2024-04-24T06:30:06.885910+0000","last_undegraded":"2024-04-24T06:30:06.885910+0000","last_fullsized":"2024-04-24T06:30:06.885910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:07.580317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887437+0000","last_change":"2024-04-24T06:30:04.920923+0000","last_active":"2024-04-24T06:30:06.887437+0000","last_peered":"2024-04-24T06:30:06.887437+0000","last_clean":"2024-04-24T06:30:06.887437+0000","last_became_active":"2024-04-24T06:30:04.920605+0000","last_became_peered":"2024-04-24T06:30:04.920605+0000","last_unstale":"2024-04-24T06:30:06.887437+0000","last_undegraded":"2024-04-24T06:30:06.887437+0000","last_fullsized":"2024-04-24T06:30:06.887437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951509+0000","last_change":"2024-04-24T06:30:04.927487+0000","last_active":"2024-04-24T06:30:05.951509+0000","last_peered":"2024-04-24T06:30:05.951509+0000","last_clean":"2024-04-24T06:30:05.951509+0000","last_became_active":"2024-04-24T06:30:04.927159+0000","last_became_peered":"2024-04-24T06:30:04.927159+0000","last_unstale":"2024-04-24T06:30:05.951509+0000","last_undegraded":"2024-04-24T06:30:05.951509+0000","last_fullsized":"2024-04-24T06:30:05.951509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:11:24.883801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.924350+0000","last_change":"2024-04-24T06:30:04.929287+0000","last_active":"2024-04-24T06:30:05.924350+0000","last_peered":"2024-04-24T06:30:05.924350+0000","last_clean":"2024-04-24T06:30:05.924350+0000","last_became_active":"2024-04-24T06:30:04.929003+0000","last_became_peered":"2024-04-24T06:30:04.929003+0000","last_unstale":"2024-04-24T06:30:05.924350+0000","last_undegraded":"2024-04-24T06:30:05.924350+0000","last_fullsized":"2024-04-24T06:30:05.924350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:32:48.606034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884347+0000","last_change":"2024-04-24T06:30:04.931548+0000","last_active":"2024-04-24T06:30:06.884347+0000","last_peered":"2024-04-24T06:30:06.884347+0000","last_clean":"2024-04-24T06:30:06.884347+0000","last_became_active":"2024-04-24T06:30:04.930678+0000","last_became_peered":"2024-04-24T06:30:04.930678+0000","last_unstale":"2024-04-24T06:30:06.884347+0000","last_undegraded":"2024-04-24T06:30:06.884347+0000","last_fullsized":"2024-04-24T06:30:06.884347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299653+0000","last_change":"2024-04-24T06:30:04.934042+0000","last_active":"2024-04-24T06:30:06.299653+0000","last_peered":"2024-04-24T06:30:06.299653+0000","last_clean":"2024-04-24T06:30:06.299653+0000","last_became_active":"2024-04-24T06:30:04.933765+0000","last_became_peered":"2024-04-24T06:30:04.933765+0000","last_unstale":"2024-04-24T06:30:06.299653+0000","last_undegraded":"2024-04-24T06:30:06.299653+0000","last_fullsized":"2024-04-24T06:30:06.299653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299051+0000","last_change":"2024-04-24T06:30:04.928489+0000","last_active":"2024-04-24T06:30:06.299051+0000","last_peered":"2024-04-24T06:30:06.299051+0000","last_clean":"2024-04-24T06:30:06.299051+0000","last_became_active":"2024-04-24T06:30:04.928131+0000","last_became_peered":"2024-04-24T06:30:04.928131+0000","last_unstale":"2024-04-24T06:30:06.299051+0000","last_undegraded":"2024-04-24T06:30:06.299051+0000","last_fullsized":"2024-04-24T06:30:06.299051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:57.553390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927308+0000","last_change":"2024-04-24T06:30:04.921142+0000","last_active":"2024-04-24T06:30:05.927308+0000","last_peered":"2024-04-24T06:30:05.927308+0000","last_clean":"2024-04-24T06:30:05.927308+0000","last_became_active":"2024-04-24T06:30:04.919968+0000","last_became_peered":"2024-04-24T06:30:04.919968+0000","last_unstale":"2024-04-24T06:30:05.927308+0000","last_undegraded":"2024-04-24T06:30:05.927308+0000","last_fullsized":"2024-04-24T06:30:05.927308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:11.545726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952119+0000","last_change":"2024-04-24T06:30:04.932332+0000","last_active":"2024-04-24T06:30:05.952119+0000","last_peered":"2024-04-24T06:30:05.952119+0000","last_clean":"2024-04-24T06:30:05.952119+0000","last_became_active":"2024-04-24T06:30:04.932026+0000","last_became_peered":"2024-04-24T06:30:04.932026+0000","last_unstale":"2024-04-24T06:30:05.952119+0000","last_undegraded":"2024-04-24T06:30:05.952119+0000","last_fullsized":"2024-04-24T06:30:05.952119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:11.802512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299922+0000","last_change":"2024-04-24T06:30:04.931124+0000","last_active":"2024-04-24T06:30:06.299922+0000","last_peered":"2024-04-24T06:30:06.299922+0000","last_clean":"2024-04-24T06:30:06.299922+0000","last_became_active":"2024-04-24T06:30:04.930821+0000","last_became_peered":"2024-04-24T06:30:04.930821+0000","last_unstale":"2024-04-24T06:30:06.299922+0000","last_undegraded":"2024-04-24T06:30:06.299922+0000","last_fullsized":"2024-04-24T06:30:06.299922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887181+0000","last_change":"2024-04-24T06:30:04.930563+0000","last_active":"2024-04-24T06:30:06.887181+0000","last_peered":"2024-04-24T06:30:06.887181+0000","last_clean":"2024-04-24T06:30:06.887181+0000","last_became_active":"2024-04-24T06:30:04.930220+0000","last_became_peered":"2024-04-24T06:30:04.930220+0000","last_unstale":"2024-04-24T06:30:06.887181+0000","last_undegraded":"2024-04-24T06:30:06.887181+0000","last_fullsized":"2024-04-24T06:30:06.887181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923514+0000","last_change":"2024-04-24T06:30:04.935008+0000","last_active":"2024-04-24T06:30:05.923514+0000","last_peered":"2024-04-24T06:30:05.923514+0000","last_clean":"2024-04-24T06:30:05.923514+0000","last_became_active":"2024-04-24T06:30:04.934732+0000","last_became_peered":"2024-04-24T06:30:04.934732+0000","last_unstale":"2024-04-24T06:30:05.923514+0000","last_undegraded":"2024-04-24T06:30:05.923514+0000","last_fullsized":"2024-04-24T06:30:05.923514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:14.417846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884610+0000","last_change":"2024-04-24T06:30:04.924737+0000","last_active":"2024-04-24T06:30:06.884610+0000","last_peered":"2024-04-24T06:30:06.884610+0000","last_clean":"2024-04-24T06:30:06.884610+0000","last_became_active":"2024-04-24T06:30:04.924378+0000","last_became_peered":"2024-04-24T06:30:04.924378+0000","last_unstale":"2024-04-24T06:30:06.884610+0000","last_undegraded":"2024-04-24T06:30:06.884610+0000","last_fullsized":"2024-04-24T06:30:06.884610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886264+0000","last_change":"2024-04-24T06:30:04.938541+0000","last_active":"2024-04-24T06:30:06.886264+0000","last_peered":"2024-04-24T06:30:06.886264+0000","last_clean":"2024-04-24T06:30:06.886264+0000","last_became_active":"2024-04-24T06:30:04.937269+0000","last_became_peered":"2024-04-24T06:30:04.937269+0000","last_unstale":"2024-04-24T06:30:06.886264+0000","last_undegraded":"2024-04-24T06:30:06.886264+0000","last_fullsized":"2024-04-24T06:30:06.886264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:57.553390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299994+0000","last_change":"2024-04-24T06:30:04.937522+0000","last_active":"2024-04-24T06:30:06.299994+0000","last_peered":"2024-04-24T06:30:06.299994+0000","last_clean":"2024-04-24T06:30:06.299994+0000","last_became_active":"2024-04-24T06:30:04.937192+0000","last_became_peered":"2024-04-24T06:30:04.937192+0000","last_unstale":"2024-04-24T06:30:06.299994+0000","last_undegraded":"2024-04-24T06:30:06.299994+0000","last_fullsized":"2024-04-24T06:30:06.299994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:26:45.435912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.300546+0000","last_change":"2024-04-24T06:30:04.933293+0000","last_active":"2024-04-24T06:30:06.300546+0000","last_peered":"2024-04-24T06:30:06.300546+0000","last_clean":"2024-04-24T06:30:06.300546+0000","last_became_active":"2024-04-24T06:30:04.932955+0000","last_became_peered":"2024-04-24T06:30:04.932955+0000","last_unstale":"2024-04-24T06:30:06.300546+0000","last_undegraded":"2024-04-24T06:30:06.300546+0000","last_fullsized":"2024-04-24T06:30:06.300546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:03.454047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885903+0000","last_change":"2024-04-24T06:30:04.934031+0000","last_active":"2024-04-24T06:30:06.885903+0000","last_peered":"2024-04-24T06:30:06.885903+0000","last_clean":"2024-04-24T06:30:06.885903+0000","last_became_active":"2024-04-24T06:30:04.932760+0000","last_became_peered":"2024-04-24T06:30:04.932760+0000","last_unstale":"2024-04-24T06:30:06.885903+0000","last_undegraded":"2024-04-24T06:30:06.885903+0000","last_fullsized":"2024-04-24T06:30:06.885903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:24:15.105485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885379+0000","last_change":"2024-04-24T06:30:04.922954+0000","last_active":"2024-04-24T06:30:06.885379+0000","last_peered":"2024-04-24T06:30:06.885379+0000","last_clean":"2024-04-24T06:30:06.885379+0000","last_became_active":"2024-04-24T06:30:04.922646+0000","last_became_peered":"2024-04-24T06:30:04.922646+0000","last_unstale":"2024-04-24T06:30:06.885379+0000","last_undegraded":"2024-04-24T06:30:06.885379+0000","last_fullsized":"2024-04-24T06:30:06.885379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:26:45.435912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923759+0000","last_change":"2024-04-24T06:30:04.932542+0000","last_active":"2024-04-24T06:30:05.923759+0000","last_peered":"2024-04-24T06:30:05.923759+0000","last_clean":"2024-04-24T06:30:05.923759+0000","last_became_active":"2024-04-24T06:30:04.932243+0000","last_became_peered":"2024-04-24T06:30:04.932243+0000","last_unstale":"2024-04-24T06:30:05.923759+0000","last_undegraded":"2024-04-24T06:30:05.923759+0000","last_fullsized":"2024-04-24T06:30:05.923759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:02.444324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926328+0000","last_change":"2024-04-24T06:30:04.919485+0000","last_active":"2024-04-24T06:30:05.926328+0000","last_peered":"2024-04-24T06:30:05.926328+0000","last_clean":"2024-04-24T06:30:05.926328+0000","last_became_active":"2024-04-24T06:30:04.918226+0000","last_became_peered":"2024-04-24T06:30:04.918226+0000","last_unstale":"2024-04-24T06:30:05.926328+0000","last_undegraded":"2024-04-24T06:30:05.926328+0000","last_fullsized":"2024-04-24T06:30:05.926328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:53.692627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925857+0000","last_change":"2024-04-24T06:30:04.924384+0000","last_active":"2024-04-24T06:30:05.925857+0000","last_peered":"2024-04-24T06:30:05.925857+0000","last_clean":"2024-04-24T06:30:05.925857+0000","last_became_active":"2024-04-24T06:30:04.924038+0000","last_became_peered":"2024-04-24T06:30:04.924038+0000","last_unstale":"2024-04-24T06:30:05.925857+0000","last_undegraded":"2024-04-24T06:30:05.925857+0000","last_fullsized":"2024-04-24T06:30:05.925857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:01.909581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.944867+0000","last_change":"2024-04-24T06:30:04.927729+0000","last_active":"2024-04-24T06:30:05.944867+0000","last_peered":"2024-04-24T06:30:05.944867+0000","last_clean":"2024-04-24T06:30:05.944867+0000","last_became_active":"2024-04-24T06:30:04.926332+0000","last_became_peered":"2024-04-24T06:30:04.926332+0000","last_unstale":"2024-04-24T06:30:05.944867+0000","last_undegraded":"2024-04-24T06:30:05.944867+0000","last_fullsized":"2024-04-24T06:30:05.944867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:50.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885485+0000","last_change":"2024-04-24T06:30:04.923428+0000","last_active":"2024-04-24T06:30:06.885485+0000","last_peered":"2024-04-24T06:30:06.885485+0000","last_clean":"2024-04-24T06:30:06.885485+0000","last_became_active":"2024-04-24T06:30:04.922142+0000","last_became_peered":"2024-04-24T06:30:04.922142+0000","last_unstale":"2024-04-24T06:30:06.885485+0000","last_undegraded":"2024-04-24T06:30:06.885485+0000","last_fullsized":"2024-04-24T06:30:06.885485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:44:35.669397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952561+0000","last_change":"2024-04-24T06:30:04.924722+0000","last_active":"2024-04-24T06:30:05.952561+0000","last_peered":"2024-04-24T06:30:05.952561+0000","last_clean":"2024-04-24T06:30:05.952561+0000","last_became_active":"2024-04-24T06:30:04.922766+0000","last_became_peered":"2024-04-24T06:30:04.922766+0000","last_unstale":"2024-04-24T06:30:05.952561+0000","last_undegraded":"2024-04-24T06:30:05.952561+0000","last_fullsized":"2024-04-24T06:30:05.952561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:21.832041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886202+0000","last_change":"2024-04-24T06:30:04.928803+0000","last_active":"2024-04-24T06:30:06.886202+0000","last_peered":"2024-04-24T06:30:06.886202+0000","last_clean":"2024-04-24T06:30:06.886202+0000","last_became_active":"2024-04-24T06:30:04.927236+0000","last_became_peered":"2024-04-24T06:30:04.927236+0000","last_unstale":"2024-04-24T06:30:06.886202+0000","last_undegraded":"2024-04-24T06:30:06.886202+0000","last_fullsized":"2024-04-24T06:30:06.886202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:18:14.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952433+0000","last_change":"2024-04-24T06:30:04.930571+0000","last_active":"2024-04-24T06:30:05.952433+0000","last_peered":"2024-04-24T06:30:05.952433+0000","last_clean":"2024-04-24T06:30:05.952433+0000","last_became_active":"2024-04-24T06:30:04.929847+0000","last_became_peered":"2024-04-24T06:30:04.929847+0000","last_unstale":"2024-04-24T06:30:05.952433+0000","last_undegraded":"2024-04-24T06:30:05.952433+0000","last_fullsized":"2024-04-24T06:30:05.952433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:23.743368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886642+0000","last_change":"2024-04-24T06:30:04.920854+0000","last_active":"2024-04-24T06:30:06.886642+0000","last_peered":"2024-04-24T06:30:06.886642+0000","last_clean":"2024-04-24T06:30:06.886642+0000","last_became_active":"2024-04-24T06:30:04.919700+0000","last_became_peered":"2024-04-24T06:30:04.919700+0000","last_unstale":"2024-04-24T06:30:06.886642+0000","last_undegraded":"2024-04-24T06:30:06.886642+0000","last_fullsized":"2024-04-24T06:30:06.886642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:11.097944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925565+0000","last_change":"2024-04-24T06:30:04.925556+0000","last_active":"2024-04-24T06:30:05.925565+0000","last_peered":"2024-04-24T06:30:05.925565+0000","last_clean":"2024-04-24T06:30:05.925565+0000","last_became_active":"2024-04-24T06:30:04.923525+0000","last_became_peered":"2024-04-24T06:30:04.923525+0000","last_unstale":"2024-04-24T06:30:05.925565+0000","last_undegraded":"2024-04-24T06:30:05.925565+0000","last_fullsized":"2024-04-24T06:30:05.925565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299226+0000","last_change":"2024-04-24T06:30:04.934051+0000","last_active":"2024-04-24T06:30:06.299226+0000","last_peered":"2024-04-24T06:30:06.299226+0000","last_clean":"2024-04-24T06:30:06.299226+0000","last_became_active":"2024-04-24T06:30:04.933719+0000","last_became_peered":"2024-04-24T06:30:04.933719+0000","last_unstale":"2024-04-24T06:30:06.299226+0000","last_undegraded":"2024-04-24T06:30:06.299226+0000","last_fullsized":"2024-04-24T06:30:06.299226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:11.097944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885179+0000","last_change":"2024-04-24T06:30:04.928722+0000","last_active":"2024-04-24T06:30:06.885179+0000","last_peered":"2024-04-24T06:30:06.885179+0000","last_clean":"2024-04-24T06:30:06.885179+0000","last_became_active":"2024-04-24T06:30:04.928199+0000","last_became_peered":"2024-04-24T06:30:04.928199+0000","last_unstale":"2024-04-24T06:30:06.885179+0000","last_undegraded":"2024-04-24T06:30:06.885179+0000","last_fullsized":"2024-04-24T06:30:06.885179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299324+0000","last_change":"2024-04-24T06:30:04.928050+0000","last_active":"2024-04-24T06:30:06.299324+0000","last_peered":"2024-04-24T06:30:06.299324+0000","last_clean":"2024-04-24T06:30:06.299324+0000","last_became_active":"2024-04-24T06:30:04.927705+0000","last_became_peered":"2024-04-24T06:30:04.927705+0000","last_unstale":"2024-04-24T06:30:06.299324+0000","last_undegraded":"2024-04-24T06:30:06.299324+0000","last_fullsized":"2024-04-24T06:30:06.299324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945380+0000","last_change":"2024-04-24T06:30:04.926443+0000","last_active":"2024-04-24T06:30:05.945380+0000","last_peered":"2024-04-24T06:30:05.945380+0000","last_clean":"2024-04-24T06:30:05.945380+0000","last_became_active":"2024-04-24T06:30:04.926091+0000","last_became_peered":"2024-04-24T06:30:04.926091+0000","last_unstale":"2024-04-24T06:30:05.945380+0000","last_undegraded":"2024-04-24T06:30:05.945380+0000","last_fullsized":"2024-04-24T06:30:05.945380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885118+0000","last_change":"2024-04-24T06:30:03.888487+0000","last_active":"2024-04-24T06:30:06.885118+0000","last_peered":"2024-04-24T06:30:06.885118+0000","last_clean":"2024-04-24T06:30:06.885118+0000","last_became_active":"2024-04-24T06:30:03.887912+0000","last_became_peered":"2024-04-24T06:30:03.887912+0000","last_unstale":"2024-04-24T06:30:06.885118+0000","last_undegraded":"2024-04-24T06:30:06.885118+0000","last_fullsized":"2024-04-24T06:30:06.885118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884955+0000","last_change":"2024-04-24T06:30:04.924476+0000","last_active":"2024-04-24T06:30:06.884955+0000","last_peered":"2024-04-24T06:30:06.884955+0000","last_clean":"2024-04-24T06:30:06.884955+0000","last_became_active":"2024-04-24T06:30:04.924023+0000","last_became_peered":"2024-04-24T06:30:04.924023+0000","last_unstale":"2024-04-24T06:30:06.884955+0000","last_undegraded":"2024-04-24T06:30:06.884955+0000","last_fullsized":"2024-04-24T06:30:06.884955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:50.246737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945409+0000","last_change":"2024-04-24T06:30:03.889451+0000","last_active":"2024-04-24T06:30:05.945409+0000","last_peered":"2024-04-24T06:30:05.945409+0000","last_clean":"2024-04-24T06:30:05.945409+0000","last_became_active":"2024-04-24T06:30:03.889142+0000","last_became_peered":"2024-04-24T06:30:03.889142+0000","last_unstale":"2024-04-24T06:30:05.945409+0000","last_undegraded":"2024-04-24T06:30:05.945409+0000","last_fullsized":"2024-04-24T06:30:05.945409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926659+0000","last_change":"2024-04-24T06:30:04.929261+0000","last_active":"2024-04-24T06:30:05.926659+0000","last_peered":"2024-04-24T06:30:05.926659+0000","last_clean":"2024-04-24T06:30:05.926659+0000","last_became_active":"2024-04-24T06:30:04.928977+0000","last_became_peered":"2024-04-24T06:30:04.928977+0000","last_unstale":"2024-04-24T06:30:05.926659+0000","last_undegraded":"2024-04-24T06:30:05.926659+0000","last_fullsized":"2024-04-24T06:30:05.926659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:11:24.883801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885034+0000","last_change":"2024-04-24T06:30:03.887130+0000","last_active":"2024-04-24T06:30:06.885034+0000","last_peered":"2024-04-24T06:30:06.885034+0000","last_clean":"2024-04-24T06:30:06.885034+0000","last_became_active":"2024-04-24T06:30:03.886744+0000","last_became_peered":"2024-04-24T06:30:03.886744+0000","last_unstale":"2024-04-24T06:30:06.885034+0000","last_undegraded":"2024-04-24T06:30:06.885034+0000","last_fullsized":"2024-04-24T06:30:06.885034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:49.241036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952342+0000","last_change":"2024-04-24T06:30:04.930226+0000","last_active":"2024-04-24T06:30:05.952342+0000","last_peered":"2024-04-24T06:30:05.952342+0000","last_clean":"2024-04-24T06:30:05.952342+0000","last_became_active":"2024-04-24T06:30:04.928571+0000","last_became_peered":"2024-04-24T06:30:04.928571+0000","last_unstale":"2024-04-24T06:30:05.952342+0000","last_undegraded":"2024-04-24T06:30:05.952342+0000","last_fullsized":"2024-04-24T06:30:05.952342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:11.545726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923667+0000","last_change":"2024-04-24T06:30:03.879647+0000","last_active":"2024-04-24T06:30:05.923667+0000","last_peered":"2024-04-24T06:30:05.923667+0000","last_clean":"2024-04-24T06:30:05.923667+0000","last_became_active":"2024-04-24T06:30:03.879323+0000","last_became_peered":"2024-04-24T06:30:03.879323+0000","last_unstale":"2024-04-24T06:30:05.923667+0000","last_undegraded":"2024-04-24T06:30:05.923667+0000","last_fullsized":"2024-04-24T06:30:05.923667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:12.863252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299286+0000","last_change":"2024-04-24T06:30:04.929545+0000","last_active":"2024-04-24T06:30:06.299286+0000","last_peered":"2024-04-24T06:30:06.299286+0000","last_clean":"2024-04-24T06:30:06.299286+0000","last_became_active":"2024-04-24T06:30:04.929116+0000","last_became_peered":"2024-04-24T06:30:04.929116+0000","last_unstale":"2024-04-24T06:30:06.299286+0000","last_undegraded":"2024-04-24T06:30:06.299286+0000","last_fullsized":"2024-04-24T06:30:06.299286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:27:59.666126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926085+0000","last_change":"2024-04-24T06:30:03.886347+0000","last_active":"2024-04-24T06:30:05.926085+0000","last_peered":"2024-04-24T06:30:05.926085+0000","last_clean":"2024-04-24T06:30:05.926085+0000","last_became_active":"2024-04-24T06:30:03.886010+0000","last_became_peered":"2024-04-24T06:30:03.886010+0000","last_unstale":"2024-04-24T06:30:05.926085+0000","last_undegraded":"2024-04-24T06:30:05.926085+0000","last_fullsized":"2024-04-24T06:30:05.926085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:17.786539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886624+0000","last_change":"2024-04-24T06:30:04.918786+0000","last_active":"2024-04-24T06:30:06.886624+0000","last_peered":"2024-04-24T06:30:06.886624+0000","last_clean":"2024-04-24T06:30:06.886624+0000","last_became_active":"2024-04-24T06:30:04.918376+0000","last_became_peered":"2024-04-24T06:30:04.918376+0000","last_unstale":"2024-04-24T06:30:06.886624+0000","last_undegraded":"2024-04-24T06:30:06.886624+0000","last_fullsized":"2024-04-24T06:30:06.886624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:28.603915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925920+0000","last_change":"2024-04-24T06:30:03.891043+0000","last_active":"2024-04-24T06:30:05.925920+0000","last_peered":"2024-04-24T06:30:05.925920+0000","last_clean":"2024-04-24T06:30:05.925920+0000","last_became_active":"2024-04-24T06:30:03.887606+0000","last_became_peered":"2024-04-24T06:30:03.887606+0000","last_unstale":"2024-04-24T06:30:05.925920+0000","last_undegraded":"2024-04-24T06:30:05.925920+0000","last_fullsized":"2024-04-24T06:30:05.925920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:17:21.342442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925499+0000","last_change":"2024-04-24T06:30:04.932105+0000","last_active":"2024-04-24T06:30:05.925499+0000","last_peered":"2024-04-24T06:30:05.925499+0000","last_clean":"2024-04-24T06:30:05.925499+0000","last_became_active":"2024-04-24T06:30:04.931829+0000","last_became_peered":"2024-04-24T06:30:04.931829+0000","last_unstale":"2024-04-24T06:30:05.925499+0000","last_undegraded":"2024-04-24T06:30:05.925499+0000","last_fullsized":"2024-04-24T06:30:05.925499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:45.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884056+0000","last_change":"2024-04-24T06:30:03.875487+0000","last_active":"2024-04-24T06:30:06.884056+0000","last_peered":"2024-04-24T06:30:06.884056+0000","last_clean":"2024-04-24T06:30:06.884056+0000","last_became_active":"2024-04-24T06:30:03.875122+0000","last_became_peered":"2024-04-24T06:30:03.875122+0000","last_unstale":"2024-04-24T06:30:06.884056+0000","last_undegraded":"2024-04-24T06:30:06.884056+0000","last_fullsized":"2024-04-24T06:30:06.884056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:12:17.347720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950833+0000","last_change":"2024-04-24T06:30:04.928526+0000","last_active":"2024-04-24T06:30:05.950833+0000","last_peered":"2024-04-24T06:30:05.950833+0000","last_clean":"2024-04-24T06:30:05.950833+0000","last_became_active":"2024-04-24T06:30:04.928221+0000","last_became_peered":"2024-04-24T06:30:04.928221+0000","last_unstale":"2024-04-24T06:30:05.950833+0000","last_undegraded":"2024-04-24T06:30:05.950833+0000","last_fullsized":"2024-04-24T06:30:05.950833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925287+0000","last_change":"2024-04-24T06:30:03.881647+0000","last_active":"2024-04-24T06:30:05.925287+0000","last_peered":"2024-04-24T06:30:05.925287+0000","last_clean":"2024-04-24T06:30:05.925287+0000","last_became_active":"2024-04-24T06:30:03.881317+0000","last_became_peered":"2024-04-24T06:30:03.881317+0000","last_unstale":"2024-04-24T06:30:05.925287+0000","last_undegraded":"2024-04-24T06:30:05.925287+0000","last_fullsized":"2024-04-24T06:30:05.925287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:48:04.946499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.886846+0000","last_change":"2024-04-24T06:30:04.929548+0000","last_active":"2024-04-24T06:30:06.886846+0000","last_peered":"2024-04-24T06:30:06.886846+0000","last_clean":"2024-04-24T06:30:06.886846+0000","last_became_active":"2024-04-24T06:30:04.928673+0000","last_became_peered":"2024-04-24T06:30:04.928673+0000","last_unstale":"2024-04-24T06:30:06.886846+0000","last_undegraded":"2024-04-24T06:30:06.886846+0000","last_fullsized":"2024-04-24T06:30:06.886846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:27:59.666126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885646+0000","last_change":"2024-04-24T06:30:03.888302+0000","last_active":"2024-04-24T06:30:06.885646+0000","last_peered":"2024-04-24T06:30:06.885646+0000","last_clean":"2024-04-24T06:30:06.885646+0000","last_became_active":"2024-04-24T06:30:03.887740+0000","last_became_peered":"2024-04-24T06:30:03.887740+0000","last_unstale":"2024-04-24T06:30:06.885646+0000","last_undegraded":"2024-04-24T06:30:06.885646+0000","last_fullsized":"2024-04-24T06:30:06.885646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:44.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885782+0000","last_change":"2024-04-24T06:30:04.933384+0000","last_active":"2024-04-24T06:30:06.885782+0000","last_peered":"2024-04-24T06:30:06.885782+0000","last_clean":"2024-04-24T06:30:06.885782+0000","last_became_active":"2024-04-24T06:30:04.932460+0000","last_became_peered":"2024-04-24T06:30:04.932460+0000","last_unstale":"2024-04-24T06:30:06.885782+0000","last_undegraded":"2024-04-24T06:30:06.885782+0000","last_fullsized":"2024-04-24T06:30:06.885782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:03.454047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945592+0000","last_change":"2024-04-24T06:30:03.883718+0000","last_active":"2024-04-24T06:30:05.945592+0000","last_peered":"2024-04-24T06:30:05.945592+0000","last_clean":"2024-04-24T06:30:05.945592+0000","last_became_active":"2024-04-24T06:30:03.883341+0000","last_became_peered":"2024-04-24T06:30:03.883341+0000","last_unstale":"2024-04-24T06:30:05.945592+0000","last_undegraded":"2024-04-24T06:30:05.945592+0000","last_fullsized":"2024-04-24T06:30:05.945592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:13.477762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885346+0000","last_change":"2024-04-24T06:30:04.927340+0000","last_active":"2024-04-24T06:30:06.885346+0000","last_peered":"2024-04-24T06:30:06.885346+0000","last_clean":"2024-04-24T06:30:06.885346+0000","last_became_active":"2024-04-24T06:30:04.922409+0000","last_became_peered":"2024-04-24T06:30:04.922409+0000","last_unstale":"2024-04-24T06:30:06.885346+0000","last_undegraded":"2024-04-24T06:30:06.885346+0000","last_fullsized":"2024-04-24T06:30:06.885346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:07:16.500626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887337+0000","last_change":"2024-04-24T06:30:03.890088+0000","last_active":"2024-04-24T06:30:06.887337+0000","last_peered":"2024-04-24T06:30:06.887337+0000","last_clean":"2024-04-24T06:30:06.887337+0000","last_became_active":"2024-04-24T06:30:03.889759+0000","last_became_peered":"2024-04-24T06:30:03.889759+0000","last_unstale":"2024-04-24T06:30:06.887337+0000","last_undegraded":"2024-04-24T06:30:06.887337+0000","last_fullsized":"2024-04-24T06:30:06.887337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:46.580010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887133+0000","last_change":"2024-04-24T06:30:04.919211+0000","last_active":"2024-04-24T06:30:06.887133+0000","last_peered":"2024-04-24T06:30:06.887133+0000","last_clean":"2024-04-24T06:30:06.887133+0000","last_became_active":"2024-04-24T06:30:04.917110+0000","last_became_peered":"2024-04-24T06:30:04.917110+0000","last_unstale":"2024-04-24T06:30:06.887133+0000","last_undegraded":"2024-04-24T06:30:06.887133+0000","last_fullsized":"2024-04-24T06:30:06.887133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885205+0000","last_change":"2024-04-24T06:30:03.886204+0000","last_active":"2024-04-24T06:30:06.885205+0000","last_peered":"2024-04-24T06:30:06.885205+0000","last_clean":"2024-04-24T06:30:06.885205+0000","last_became_active":"2024-04-24T06:30:03.885275+0000","last_became_peered":"2024-04-24T06:30:03.885275+0000","last_unstale":"2024-04-24T06:30:06.885205+0000","last_undegraded":"2024-04-24T06:30:06.885205+0000","last_fullsized":"2024-04-24T06:30:06.885205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:00.903880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950945+0000","last_change":"2024-04-24T06:30:04.915707+0000","last_active":"2024-04-24T06:30:05.950945+0000","last_peered":"2024-04-24T06:30:05.950945+0000","last_clean":"2024-04-24T06:30:05.950945+0000","last_became_active":"2024-04-24T06:30:04.915397+0000","last_became_peered":"2024-04-24T06:30:04.915397+0000","last_unstale":"2024-04-24T06:30:05.950945+0000","last_undegraded":"2024-04-24T06:30:05.950945+0000","last_fullsized":"2024-04-24T06:30:05.950945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:07:01.909581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926605+0000","last_change":"2024-04-24T06:30:03.882040+0000","last_active":"2024-04-24T06:30:05.926605+0000","last_peered":"2024-04-24T06:30:05.926605+0000","last_clean":"2024-04-24T06:30:05.926605+0000","last_became_active":"2024-04-24T06:30:03.881706+0000","last_became_peered":"2024-04-24T06:30:03.881706+0000","last_unstale":"2024-04-24T06:30:05.926605+0000","last_undegraded":"2024-04-24T06:30:05.926605+0000","last_fullsized":"2024-04-24T06:30:05.926605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:49.727727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885096+0000","last_change":"2024-04-24T06:30:04.921973+0000","last_active":"2024-04-24T06:30:06.885096+0000","last_peered":"2024-04-24T06:30:06.885096+0000","last_clean":"2024-04-24T06:30:06.885096+0000","last_became_active":"2024-04-24T06:30:04.921571+0000","last_became_peered":"2024-04-24T06:30:04.921571+0000","last_unstale":"2024-04-24T06:30:06.885096+0000","last_undegraded":"2024-04-24T06:30:06.885096+0000","last_fullsized":"2024-04-24T06:30:06.885096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.950918+0000","last_change":"2024-04-24T06:30:03.886083+0000","last_active":"2024-04-24T06:30:05.950918+0000","last_peered":"2024-04-24T06:30:05.950918+0000","last_clean":"2024-04-24T06:30:05.950918+0000","last_became_active":"2024-04-24T06:30:03.885767+0000","last_became_peered":"2024-04-24T06:30:03.885767+0000","last_unstale":"2024-04-24T06:30:05.950918+0000","last_undegraded":"2024-04-24T06:30:05.950918+0000","last_fullsized":"2024-04-24T06:30:05.950918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299543+0000","last_change":"2024-04-24T06:30:04.935400+0000","last_active":"2024-04-24T06:30:06.299543+0000","last_peered":"2024-04-24T06:30:06.299543+0000","last_clean":"2024-04-24T06:30:06.299543+0000","last_became_active":"2024-04-24T06:30:04.935078+0000","last_became_peered":"2024-04-24T06:30:04.935078+0000","last_unstale":"2024-04-24T06:30:06.299543+0000","last_undegraded":"2024-04-24T06:30:06.299543+0000","last_fullsized":"2024-04-24T06:30:06.299543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:29.606642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884234+0000","last_change":"2024-04-24T06:30:03.887884+0000","last_active":"2024-04-24T06:30:06.884234+0000","last_peered":"2024-04-24T06:30:06.884234+0000","last_clean":"2024-04-24T06:30:06.884234+0000","last_became_active":"2024-04-24T06:30:03.887605+0000","last_became_peered":"2024-04-24T06:30:03.887605+0000","last_unstale":"2024-04-24T06:30:06.884234+0000","last_undegraded":"2024-04-24T06:30:06.884234+0000","last_fullsized":"2024-04-24T06:30:06.884234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:41.040181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885445+0000","last_change":"2024-04-24T06:30:04.919119+0000","last_active":"2024-04-24T06:30:06.885445+0000","last_peered":"2024-04-24T06:30:06.885445+0000","last_clean":"2024-04-24T06:30:06.885445+0000","last_became_active":"2024-04-24T06:30:04.918800+0000","last_became_peered":"2024-04-24T06:30:04.918800+0000","last_unstale":"2024-04-24T06:30:06.885445+0000","last_undegraded":"2024-04-24T06:30:06.885445+0000","last_fullsized":"2024-04-24T06:30:06.885445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884223+0000","last_change":"2024-04-24T06:30:03.884365+0000","last_active":"2024-04-24T06:30:06.884223+0000","last_peered":"2024-04-24T06:30:06.884223+0000","last_clean":"2024-04-24T06:30:06.884223+0000","last_became_active":"2024-04-24T06:30:03.884014+0000","last_became_peered":"2024-04-24T06:30:03.884014+0000","last_unstale":"2024-04-24T06:30:06.884223+0000","last_undegraded":"2024-04-24T06:30:06.884223+0000","last_fullsized":"2024-04-24T06:30:06.884223+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:00.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927014+0000","last_change":"2024-04-24T06:30:04.915566+0000","last_active":"2024-04-24T06:30:05.927014+0000","last_peered":"2024-04-24T06:30:05.927014+0000","last_clean":"2024-04-24T06:30:05.927014+0000","last_became_active":"2024-04-24T06:30:04.915269+0000","last_became_peered":"2024-04-24T06:30:04.915269+0000","last_unstale":"2024-04-24T06:30:05.927014+0000","last_undegraded":"2024-04-24T06:30:05.927014+0000","last_fullsized":"2024-04-24T06:30:05.927014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:49:11.802512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952504+0000","last_change":"2024-04-24T06:30:03.883575+0000","last_active":"2024-04-24T06:30:05.952504+0000","last_peered":"2024-04-24T06:30:05.952504+0000","last_clean":"2024-04-24T06:30:05.952504+0000","last_became_active":"2024-04-24T06:30:03.883205+0000","last_became_peered":"2024-04-24T06:30:03.883205+0000","last_unstale":"2024-04-24T06:30:05.952504+0000","last_undegraded":"2024-04-24T06:30:05.952504+0000","last_fullsized":"2024-04-24T06:30:05.952504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:13.477762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887212+0000","last_change":"2024-04-24T06:30:04.911788+0000","last_active":"2024-04-24T06:30:06.887212+0000","last_peered":"2024-04-24T06:30:06.887212+0000","last_clean":"2024-04-24T06:30:06.887212+0000","last_became_active":"2024-04-24T06:30:04.911173+0000","last_became_peered":"2024-04-24T06:30:04.911173+0000","last_unstale":"2024-04-24T06:30:06.887212+0000","last_undegraded":"2024-04-24T06:30:06.887212+0000","last_fullsized":"2024-04-24T06:30:06.887212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:24:47.578726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.927016+0000","last_change":"2024-04-24T06:30:03.887139+0000","last_active":"2024-04-24T06:30:05.927016+0000","last_peered":"2024-04-24T06:30:05.927016+0000","last_clean":"2024-04-24T06:30:05.927016+0000","last_became_active":"2024-04-24T06:30:03.886778+0000","last_became_peered":"2024-04-24T06:30:03.886778+0000","last_unstale":"2024-04-24T06:30:05.927016+0000","last_undegraded":"2024-04-24T06:30:05.927016+0000","last_fullsized":"2024-04-24T06:30:05.927016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951394+0000","last_change":"2024-04-24T06:30:04.916986+0000","last_active":"2024-04-24T06:30:05.951394+0000","last_peered":"2024-04-24T06:30:05.951394+0000","last_clean":"2024-04-24T06:30:05.951394+0000","last_became_active":"2024-04-24T06:30:04.916640+0000","last_became_peered":"2024-04-24T06:30:04.916640+0000","last_unstale":"2024-04-24T06:30:05.951394+0000","last_undegraded":"2024-04-24T06:30:05.951394+0000","last_fullsized":"2024-04-24T06:30:05.951394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:53.692627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926784+0000","last_change":"2024-04-24T06:30:03.875124+0000","last_active":"2024-04-24T06:30:05.926784+0000","last_peered":"2024-04-24T06:30:05.926784+0000","last_clean":"2024-04-24T06:30:05.926784+0000","last_became_active":"2024-04-24T06:30:03.874834+0000","last_became_peered":"2024-04-24T06:30:03.874834+0000","last_unstale":"2024-04-24T06:30:05.926784+0000","last_undegraded":"2024-04-24T06:30:05.926784+0000","last_fullsized":"2024-04-24T06:30:05.926784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.924572+0000","last_change":"2024-04-24T06:30:04.924613+0000","last_active":"2024-04-24T06:30:05.924572+0000","last_peered":"2024-04-24T06:30:05.924572+0000","last_clean":"2024-04-24T06:30:05.924572+0000","last_became_active":"2024-04-24T06:30:04.924121+0000","last_became_peered":"2024-04-24T06:30:04.924121+0000","last_unstale":"2024-04-24T06:30:05.924572+0000","last_undegraded":"2024-04-24T06:30:05.924572+0000","last_fullsized":"2024-04-24T06:30:05.924572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:57:07.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.926529+0000","last_change":"2024-04-24T06:30:03.891071+0000","last_active":"2024-04-24T06:30:05.926529+0000","last_peered":"2024-04-24T06:30:05.926529+0000","last_clean":"2024-04-24T06:30:05.926529+0000","last_became_active":"2024-04-24T06:30:03.889238+0000","last_became_peered":"2024-04-24T06:30:03.889238+0000","last_unstale":"2024-04-24T06:30:05.926529+0000","last_undegraded":"2024-04-24T06:30:05.926529+0000","last_fullsized":"2024-04-24T06:30:05.926529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923166+0000","last_change":"2024-04-24T06:30:04.928264+0000","last_active":"2024-04-24T06:30:05.923166+0000","last_peered":"2024-04-24T06:30:05.923166+0000","last_clean":"2024-04-24T06:30:05.923166+0000","last_became_active":"2024-04-24T06:30:04.927986+0000","last_became_peered":"2024-04-24T06:30:04.927986+0000","last_unstale":"2024-04-24T06:30:05.923166+0000","last_undegraded":"2024-04-24T06:30:05.923166+0000","last_fullsized":"2024-04-24T06:30:05.923166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:50.733428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945359+0000","last_change":"2024-04-24T06:30:03.876785+0000","last_active":"2024-04-24T06:30:05.945359+0000","last_peered":"2024-04-24T06:30:05.945359+0000","last_clean":"2024-04-24T06:30:05.945359+0000","last_became_active":"2024-04-24T06:30:03.876105+0000","last_became_peered":"2024-04-24T06:30:03.876105+0000","last_unstale":"2024-04-24T06:30:05.945359+0000","last_undegraded":"2024-04-24T06:30:05.945359+0000","last_fullsized":"2024-04-24T06:30:05.945359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923760+0000","last_change":"2024-04-24T06:30:04.919412+0000","last_active":"2024-04-24T06:30:05.923760+0000","last_peered":"2024-04-24T06:30:05.923760+0000","last_clean":"2024-04-24T06:30:05.923760+0000","last_became_active":"2024-04-24T06:30:04.919114+0000","last_became_peered":"2024-04-24T06:30:04.919114+0000","last_unstale":"2024-04-24T06:30:05.923760+0000","last_undegraded":"2024-04-24T06:30:05.923760+0000","last_fullsized":"2024-04-24T06:30:05.923760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:25.983849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885290+0000","last_change":"2024-04-24T06:30:03.889395+0000","last_active":"2024-04-24T06:30:06.885290+0000","last_peered":"2024-04-24T06:30:06.885290+0000","last_clean":"2024-04-24T06:30:06.885290+0000","last_became_active":"2024-04-24T06:30:03.888579+0000","last_became_peered":"2024-04-24T06:30:03.888579+0000","last_unstale":"2024-04-24T06:30:06.885290+0000","last_undegraded":"2024-04-24T06:30:06.885290+0000","last_fullsized":"2024-04-24T06:30:06.885290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923410+0000","last_change":"2024-04-24T06:30:04.920396+0000","last_active":"2024-04-24T06:30:05.923410+0000","last_peered":"2024-04-24T06:30:05.923410+0000","last_clean":"2024-04-24T06:30:05.923410+0000","last_became_active":"2024-04-24T06:30:04.920127+0000","last_became_peered":"2024-04-24T06:30:04.920127+0000","last_unstale":"2024-04-24T06:30:05.923410+0000","last_undegraded":"2024-04-24T06:30:05.923410+0000","last_fullsized":"2024-04-24T06:30:05.923410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:01.696611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925494+0000","last_change":"2024-04-24T06:30:03.876388+0000","last_active":"2024-04-24T06:30:05.925494+0000","last_peered":"2024-04-24T06:30:05.925494+0000","last_clean":"2024-04-24T06:30:05.925494+0000","last_became_active":"2024-04-24T06:30:03.876057+0000","last_became_peered":"2024-04-24T06:30:03.876057+0000","last_unstale":"2024-04-24T06:30:05.925494+0000","last_undegraded":"2024-04-24T06:30:05.925494+0000","last_fullsized":"2024-04-24T06:30:05.925494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:12.152699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299229+0000","last_change":"2024-04-24T06:30:04.933750+0000","last_active":"2024-04-24T06:30:06.299229+0000","last_peered":"2024-04-24T06:30:06.299229+0000","last_clean":"2024-04-24T06:30:06.299229+0000","last_became_active":"2024-04-24T06:30:04.933380+0000","last_became_peered":"2024-04-24T06:30:04.933380+0000","last_unstale":"2024-04-24T06:30:06.299229+0000","last_undegraded":"2024-04-24T06:30:06.299229+0000","last_fullsized":"2024-04-24T06:30:06.299229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:28.603915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885201+0000","last_change":"2024-04-24T06:30:03.884607+0000","last_active":"2024-04-24T06:30:06.885201+0000","last_peered":"2024-04-24T06:30:06.885201+0000","last_clean":"2024-04-24T06:30:06.885201+0000","last_became_active":"2024-04-24T06:30:03.884246+0000","last_became_peered":"2024-04-24T06:30:03.884246+0000","last_unstale":"2024-04-24T06:30:06.885201+0000","last_undegraded":"2024-04-24T06:30:06.885201+0000","last_fullsized":"2024-04-24T06:30:06.885201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:49.241036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.923932+0000","last_change":"2024-04-24T06:30:04.928815+0000","last_active":"2024-04-24T06:30:05.923932+0000","last_peered":"2024-04-24T06:30:05.923932+0000","last_clean":"2024-04-24T06:30:05.923932+0000","last_became_active":"2024-04-24T06:30:04.927732+0000","last_became_peered":"2024-04-24T06:30:04.927732+0000","last_unstale":"2024-04-24T06:30:05.923932+0000","last_undegraded":"2024-04-24T06:30:05.923932+0000","last_fullsized":"2024-04-24T06:30:05.923932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:14.483463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.952631+0000","last_change":"2024-04-24T06:30:03.884499+0000","last_active":"2024-04-24T06:30:05.952631+0000","last_peered":"2024-04-24T06:30:05.952631+0000","last_clean":"2024-04-24T06:30:05.952631+0000","last_became_active":"2024-04-24T06:30:03.884171+0000","last_became_peered":"2024-04-24T06:30:03.884171+0000","last_unstale":"2024-04-24T06:30:05.952631+0000","last_undegraded":"2024-04-24T06:30:05.952631+0000","last_fullsized":"2024-04-24T06:30:05.952631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:24.978148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.887435+0000","last_change":"2024-04-24T06:30:04.931777+0000","last_active":"2024-04-24T06:30:06.887435+0000","last_peered":"2024-04-24T06:30:06.887435+0000","last_clean":"2024-04-24T06:30:06.887435+0000","last_became_active":"2024-04-24T06:30:04.931384+0000","last_became_peered":"2024-04-24T06:30:04.931384+0000","last_unstale":"2024-04-24T06:30:06.887435+0000","last_undegraded":"2024-04-24T06:30:06.887435+0000","last_fullsized":"2024-04-24T06:30:06.887435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945507+0000","last_change":"2024-04-24T06:30:03.879401+0000","last_active":"2024-04-24T06:30:05.945507+0000","last_peered":"2024-04-24T06:30:05.945507+0000","last_clean":"2024-04-24T06:30:05.945507+0000","last_became_active":"2024-04-24T06:30:03.879127+0000","last_became_peered":"2024-04-24T06:30:03.879127+0000","last_unstale":"2024-04-24T06:30:05.945507+0000","last_undegraded":"2024-04-24T06:30:05.945507+0000","last_fullsized":"2024-04-24T06:30:05.945507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:24.978148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945386+0000","last_change":"2024-04-24T06:30:04.922693+0000","last_active":"2024-04-24T06:30:05.945386+0000","last_peered":"2024-04-24T06:30:05.945386+0000","last_clean":"2024-04-24T06:30:05.945386+0000","last_became_active":"2024-04-24T06:30:04.922352+0000","last_became_peered":"2024-04-24T06:30:04.922352+0000","last_unstale":"2024-04-24T06:30:05.945386+0000","last_undegraded":"2024-04-24T06:30:05.945386+0000","last_fullsized":"2024-04-24T06:30:05.945386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:47.585711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951451+0000","last_change":"2024-04-24T06:30:03.876216+0000","last_active":"2024-04-24T06:30:05.951451+0000","last_peered":"2024-04-24T06:30:05.951451+0000","last_clean":"2024-04-24T06:30:05.951451+0000","last_became_active":"2024-04-24T06:30:03.875888+0000","last_became_peered":"2024-04-24T06:30:03.875888+0000","last_unstale":"2024-04-24T06:30:05.951451+0000","last_undegraded":"2024-04-24T06:30:05.951451+0000","last_fullsized":"2024-04-24T06:30:05.951451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:01.438623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.885126+0000","last_change":"2024-04-24T06:30:04.922166+0000","last_active":"2024-04-24T06:30:06.885126+0000","last_peered":"2024-04-24T06:30:06.885126+0000","last_clean":"2024-04-24T06:30:06.885126+0000","last_became_active":"2024-04-24T06:30:04.921823+0000","last_became_peered":"2024-04-24T06:30:04.921823+0000","last_unstale":"2024-04-24T06:30:06.885126+0000","last_undegraded":"2024-04-24T06:30:06.885126+0000","last_fullsized":"2024-04-24T06:30:06.885126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:47.585711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945757+0000","last_change":"2024-04-24T06:30:03.878948+0000","last_active":"2024-04-24T06:30:05.945757+0000","last_peered":"2024-04-24T06:30:05.945757+0000","last_clean":"2024-04-24T06:30:05.945757+0000","last_became_active":"2024-04-24T06:30:03.878652+0000","last_became_peered":"2024-04-24T06:30:03.878652+0000","last_unstale":"2024-04-24T06:30:05.945757+0000","last_undegraded":"2024-04-24T06:30:05.945757+0000","last_fullsized":"2024-04-24T06:30:05.945757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.945600+0000","last_change":"2024-04-24T06:30:04.923459+0000","last_active":"2024-04-24T06:30:05.945600+0000","last_peered":"2024-04-24T06:30:05.945600+0000","last_clean":"2024-04-24T06:30:05.945600+0000","last_became_active":"2024-04-24T06:30:04.923148+0000","last_became_peered":"2024-04-24T06:30:04.923148+0000","last_unstale":"2024-04-24T06:30:05.945600+0000","last_undegraded":"2024-04-24T06:30:05.945600+0000","last_fullsized":"2024-04-24T06:30:05.945600+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:01.625129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.951057+0000","last_change":"2024-04-24T06:30:03.885358+0000","last_active":"2024-04-24T06:30:05.951057+0000","last_peered":"2024-04-24T06:30:05.951057+0000","last_clean":"2024-04-24T06:30:05.951057+0000","last_became_active":"2024-04-24T06:30:03.884950+0000","last_became_peered":"2024-04-24T06:30:03.884950+0000","last_unstale":"2024-04-24T06:30:05.951057+0000","last_undegraded":"2024-04-24T06:30:05.951057+0000","last_fullsized":"2024-04-24T06:30:05.951057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:13.412145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884315+0000","last_change":"2024-04-24T06:30:04.922987+0000","last_active":"2024-04-24T06:30:06.884315+0000","last_peered":"2024-04-24T06:30:06.884315+0000","last_clean":"2024-04-24T06:30:06.884315+0000","last_became_active":"2024-04-24T06:30:04.922689+0000","last_became_peered":"2024-04-24T06:30:04.922689+0000","last_unstale":"2024-04-24T06:30:06.884315+0000","last_undegraded":"2024-04-24T06:30:06.884315+0000","last_fullsized":"2024-04-24T06:30:06.884315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:37.922871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925834+0000","last_change":"2024-04-24T06:30:03.881474+0000","last_active":"2024-04-24T06:30:05.925834+0000","last_peered":"2024-04-24T06:30:05.925834+0000","last_clean":"2024-04-24T06:30:05.925834+0000","last_became_active":"2024-04-24T06:30:03.881098+0000","last_became_peered":"2024-04-24T06:30:03.881098+0000","last_unstale":"2024-04-24T06:30:05.925834+0000","last_undegraded":"2024-04-24T06:30:05.925834+0000","last_fullsized":"2024-04-24T06:30:05.925834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:50.746364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:05.925781+0000","last_change":"2024-04-24T06:30:04.926464+0000","last_active":"2024-04-24T06:30:05.925781+0000","last_peered":"2024-04-24T06:30:05.925781+0000","last_clean":"2024-04-24T06:30:05.925781+0000","last_became_active":"2024-04-24T06:30:04.923665+0000","last_became_peered":"2024-04-24T06:30:04.923665+0000","last_unstale":"2024-04-24T06:30:05.925781+0000","last_undegraded":"2024-04-24T06:30:05.925781+0000","last_fullsized":"2024-04-24T06:30:05.925781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884450+0000","last_change":"2024-04-24T06:30:03.887654+0000","last_active":"2024-04-24T06:30:06.884450+0000","last_peered":"2024-04-24T06:30:06.884450+0000","last_clean":"2024-04-24T06:30:06.884450+0000","last_became_active":"2024-04-24T06:30:03.887345+0000","last_became_peered":"2024-04-24T06:30:03.887345+0000","last_unstale":"2024-04-24T06:30:06.884450+0000","last_undegraded":"2024-04-24T06:30:06.884450+0000","last_fullsized":"2024-04-24T06:30:06.884450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:46.580010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.299839+0000","last_change":"2024-04-24T06:30:04.937326+0000","last_active":"2024-04-24T06:30:06.299839+0000","last_peered":"2024-04-24T06:30:06.299839+0000","last_clean":"2024-04-24T06:30:06.299839+0000","last_became_active":"2024-04-24T06:30:04.936959+0000","last_became_peered":"2024-04-24T06:30:04.936959+0000","last_unstale":"2024-04-24T06:30:06.299839+0000","last_undegraded":"2024-04-24T06:30:06.299839+0000","last_fullsized":"2024-04-24T06:30:06.299839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:03.839445+0000","last_clean_scrub_stamp":"2024-04-24T06:30:03.839445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:19:52.875666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.884324+0000","last_change":"2024-04-24T06:30:03.890720+0000","last_active":"2024-04-24T06:30:06.884324+0000","last_peered":"2024-04-24T06:30:06.884324+0000","last_clean":"2024-04-24T06:30:06.884324+0000","last_became_active":"2024-04-24T06:30:03.890376+0000","last_became_peered":"2024-04-24T06:30:03.890376+0000","last_unstale":"2024-04-24T06:30:06.884324+0000","last_undegraded":"2024-04-24T06:30:06.884324+0000","last_fullsized":"2024-04-24T06:30:06.884324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:02.833744+0000","last_clean_scrub_stamp":"2024-04-24T06:30:02.833744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:00.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-24T06:30:06.300116+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:30:06.300116+0000","last_peered":"2024-04-24T06:30:06.300116+0000","last_clean":"2024-04-24T06:30:06.300116+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:30:06.300116+0000","last_undegraded":"2024-04-24T06:30:06.300116+0000","last_fullsized":"2024-04-24T06:30:06.300116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:30:09.933 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:30:09.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:30:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.194+0000 7f5ff7600640 1 Processor -- start 2024-04-24T06:30:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.194+0000 7f5ff7600640 1 -- start start 2024-04-24T06:30:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5ff7600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ff00b56f0 0x7f5ff006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5ff7600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5ff7600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff015bf60 con 0x7f5ff00d0350 2024-04-24T06:30:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5ff7600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff015c240 con 0x7f5ff00b56f0 2024-04-24T06:30:10.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5ff7600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff015c520 con 0x7f5ff006f340 2024-04-24T06:30:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.197+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.198+0000 7f5ff4c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ff00b56f0 0x7f5ff006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.198+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:49530/0 (socket says 172.21.15.59:49530) 2024-04-24T06:30:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.198+0000 7f5fefe00640 1 -- 172.21.15.59:0/45003736 learned_addr learned my addr 172.21.15.59:0/45003736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.198+0000 7f5ff5600640 1 --1- 172.21.15.59:0/45003736 >> v1:172.21.15.66:6789/0 conn(0x7f5ff00d0350 0x7f5ff00b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:42210/0 (socket says 172.21.15.59:42210) 2024-04-24T06:30:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.198+0000 7f5ff4c00640 1 -- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 msgr2=0x7f5ff006f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.199+0000 7f5ff4c00640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.199+0000 7f5ff4c00640 1 -- 172.21.15.59:0/45003736 >> v1:172.21.15.66:6789/0 conn(0x7f5ff00d0350 legacy=0x7f5ff00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.199+0000 7f5ff4c00640 1 -- 172.21.15.59:0/45003736 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff015c800 con 0x7f5ff00b56f0 2024-04-24T06:30:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.199+0000 7f5fefe00640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:30:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.200+0000 7f5ff4c00640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff00b56f0 0x7f5ff006ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8418aa82cf201b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5fef400640 1 -- 172.21.15.59:0/45003736 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007740 con 0x7f5ff00b56f0 2024-04-24T06:30:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5fef400640 1 -- 172.21.15.59:0/45003736 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe00078e0 con 0x7f5ff00b56f0 2024-04-24T06:30:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5fef400640 1 -- 172.21.15.59:0/45003736 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007cb0 con 0x7f5ff00b56f0 2024-04-24T06:30:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 -- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff00b56f0 msgr2=0x7f5ff006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff00b56f0 0x7f5ff006ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009d90 tx=0x7f5fe002a660 comp rx=0 tx=0).stop 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 -- 172.21.15.59:0/45003736 shutdown_connections 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff006f340 0x7f5ff006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 --2- 172.21.15.59:0/45003736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff00b56f0 0x7f5ff006ddd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 -- 172.21.15.59:0/45003736 >> 172.21.15.59:0/45003736 conn(0x7f5ff0067d90 msgr2=0x7f5ff006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.201+0000 7f5ff7600640 1 -- 172.21.15.59:0/45003736 shutdown_connections 2024-04-24T06:30:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.202+0000 7f5ff7600640 1 -- 172.21.15.59:0/45003736 wait complete. 2024-04-24T06:30:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.203+0000 7f5ff7600640 1 Processor -- start 2024-04-24T06:30:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.204+0000 7f5ff7600640 1 -- start start 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff006f340 0x7f5ff0148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 0x7f5ff014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff00d07e0 con 0x7f5ff00b56f0 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff015bc10 con 0x7f5ff006f340 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff7600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff00d0960 con 0x7f5ff00d0350 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40106/0 (socket says 172.21.15.59:40106) 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5fefe00640 1 -- 172.21.15.59:0/3369787002 learned_addr learned my addr 172.21.15.59:0/3369787002 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff4c00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff006f340 0x7f5ff0148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.205+0000 7f5ff5600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 0x7f5ff014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 -- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 msgr2=0x7f5ff014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 0x7f5ff014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 -- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff006f340 msgr2=0x7f5ff0148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff006f340 0x7f5ff0148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 -- 172.21.15.59:0/3369787002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff0148fc0 con 0x7f5ff00b56f0 2024-04-24T06:30:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5ff5600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 0x7f5ff014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fefe00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9d0 tx=0x7f5fdc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x7f5ff00b56f0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00ca60 con 0x7f5ff00b56f0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ce10 con 0x7f5ff00b56f0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.206+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff015cd40 con 0x7f5ff00b56f0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.207+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff015d300 con 0x7f5ff00b56f0 2024-04-24T06:30:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.208+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff0068a80 con 0x7f5ff00b56f0 2024-04-24T06:30:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.209+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fdc01a070 con 0x7f5ff00b56f0 2024-04-24T06:30:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.210+0000 7f5fecc00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.210+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5fdc08bb70 con 0x7f5ff00b56f0 2024-04-24T06:30:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.210+0000 7f5ff4c00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.211+0000 7f5ff4c00640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 0x7f5fd006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0034000 tx=0x7f5fe002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.215+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fdc059f00 con 0x7f5ff00b56f0 2024-04-24T06:30:10.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.503+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5ff006ef20 con 0x7f5ff00b56f0 2024-04-24T06:30:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.507+0000 7f5fecc00640 1 -- 172.21.15.59:0/3369787002 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f5fdc0600d0 con 0x7f5ff00b56f0 2024-04-24T06:30:10.508 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:30:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:30:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:30:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:30:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:30:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 msgr2=0x7f5fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 0x7f5fd006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0034000 tx=0x7f5fe002f000 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 msgr2=0x7f5ff014c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9d0 tx=0x7f5fdc00ee90 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 shutdown_connections 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fd006c8f0 0x7f5fd006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.510+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ff00d0350 0x7f5ff014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.511+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff00b56f0 0x7f5ff014c400 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.511+0000 7f5ff7600640 1 --2- 172.21.15.59:0/3369787002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff006f340 0x7f5ff0148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.511+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 >> 172.21.15.59:0/3369787002 conn(0x7f5ff0067d90 msgr2=0x7f5ff00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.511+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 shutdown_connections 2024-04-24T06:30:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.511+0000 7f5ff7600640 1 -- 172.21.15.59:0/3369787002 wait complete. 2024-04-24T06:30:10.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:10.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.726+0000 7f5157e00640 1 Processor -- start 2024-04-24T06:30:10.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.727+0000 7f5157e00640 1 -- start start 2024-04-24T06:30:10.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.728+0000 7f5157e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515815e7c0 0x7f51580d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.728+0000 7f5157e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d0d90 0x7f51580d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.728+0000 7f5157e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51580cf030 con 0x7f51580d2630 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.728+0000 7f5157e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f51580d1740 con 0x7f515815e7c0 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.728+0000 7f5157e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f51580b7000 con 0x7f51580d0d90 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515815e7c0 0x7f51580d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515815e7c0 0x7f51580d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40118/0 (socket says 172.21.15.59:40118) 2024-04-24T06:30:10.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 -- 172.21.15.59:0/2297354736 learned_addr learned my addr 172.21.15.59:0/2297354736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:10.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 -- 172.21.15.59:0/2297354736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d0d90 msgr2=0x7f51580d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:10.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5157400640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d0d90 0x7f51580d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d0d90 0x7f51580d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 -- 172.21.15.59:0/2297354736 >> v1:172.21.15.66:6789/0 conn(0x7f51580d2630 legacy=0x7f51580b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:10.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.729+0000 7f5156a00640 1 -- 172.21.15.59:0/2297354736 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51580b6a40 con 0x7f515815e7c0 2024-04-24T06:30:10.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.730+0000 7f5156a00640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580d20f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5148009870 tx=0x7f514802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa3e6c452790858 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5155600640 1 -- 172.21.15.59:0/2297354736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148037070 con 0x7f515815e7c0 2024-04-24T06:30:10.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5155600640 1 -- 172.21.15.59:0/2297354736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5148032440 con 0x7f515815e7c0 2024-04-24T06:30:10.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5155600640 1 -- 172.21.15.59:0/2297354736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148032810 con 0x7f515815e7c0 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 -- 172.21.15.59:0/2297354736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 msgr2=0x7f51580d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580d20f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5148009870 tx=0x7f514802a210 comp rx=0 tx=0).stop 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 -- 172.21.15.59:0/2297354736 shutdown_connections 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d0d90 0x7f51580d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 --2- 172.21.15.59:0/2297354736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580d20f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 -- 172.21.15.59:0/2297354736 >> 172.21.15.59:0/2297354736 conn(0x7f515809d920 msgr2=0x7f515808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.731+0000 7f5157e00640 1 -- 172.21.15.59:0/2297354736 shutdown_connections 2024-04-24T06:30:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.732+0000 7f5157e00640 1 -- 172.21.15.59:0/2297354736 wait complete. 2024-04-24T06:30:10.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.733+0000 7f5157e00640 1 Processor -- start 2024-04-24T06:30:10.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.733+0000 7f5157e00640 1 -- start start 2024-04-24T06:30:10.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51580d0d90 0x7f51581539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 0x7f51580cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51580cf2b0 con 0x7f515815e7c0 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51580cf620 con 0x7f51580d0d90 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.734+0000 7f5157e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f51580b6700 con 0x7f51580d2630 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40124/0 (socket says 172.21.15.59:40124) 2024-04-24T06:30:10.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 -- 172.21.15.59:0/979642599 learned_addr learned my addr 172.21.15.59:0/979642599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:10.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 -- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 msgr2=0x7f51580cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5156a00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51580d0d90 0x7f51581539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5156000640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 0x7f51580cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 0x7f51580cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 -- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51580d0d90 msgr2=0x7f51581539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51580d0d90 0x7f51581539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 -- 172.21.15.59:0/979642599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51580c8550 con 0x7f515815e7c0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5156000640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 0x7f51580cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.735+0000 7f5157400640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f515000c7f0 tx=0x7f515000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.736+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150015070 con 0x7f515815e7c0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.736+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5150004220 con 0x7f515815e7c0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.736+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150007620 con 0x7f515815e7c0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.736+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51580c8830 con 0x7f515815e7c0 2024-04-24T06:30:10.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.736+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51580c8df0 con 0x7f515815e7c0 2024-04-24T06:30:10.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.737+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515804fd90 con 0x7f515815e7c0 2024-04-24T06:30:10.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.738+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51500079d0 con 0x7f515815e7c0 2024-04-24T06:30:10.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.739+0000 7f514ea00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 0x7f513806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:10.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.739+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f515008bd40 con 0x7f515815e7c0 2024-04-24T06:30:10.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.740+0000 7f5156a00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 0x7f513806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:10.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.740+0000 7f5156a00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 0x7f513806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f514802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:10.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.744+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f515005a040 con 0x7f515815e7c0 2024-04-24T06:30:10.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.992+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51580d1eb0 con 0x7f515815e7c0 2024-04-24T06:30:10.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.993+0000 7f514ea00640 1 -- 172.21.15.59:0/979642599 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f5150059940 con 0x7f515815e7c0 2024-04-24T06:30:10.994 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:10.994 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:10.994 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 msgr2=0x7f513806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 0x7f513806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f514802e000 comp rx=0 tx=0).stop 2024-04-24T06:30:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 msgr2=0x7f51580b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f515000c7f0 tx=0x7f515000ccb0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 shutdown_connections 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513806c8f0 0x7f513806edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515815e7c0 0x7f51580b59d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f51580d2630 0x7f51580cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 --2- 172.21.15.59:0/979642599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51580d0d90 0x7f51581539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 >> 172.21.15.59:0/979642599 conn(0x7f515809d920 msgr2=0x7f5158160e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.996+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 shutdown_connections 2024-04-24T06:30:10.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:10.997+0000 7f5157e00640 1 -- 172.21.15.59:0/979642599 wait complete. 2024-04-24T06:30:11.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:30:04.333230+0000', 'modified': '2024-04-24T06:30:07.467202+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5205': {'gid': 5205, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.66:6833/2735852616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 2735852616}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 2735852616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5205, 'qdb_cluster': [5205]} 2024-04-24T06:30:11.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:30:11.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T06:30:11.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.641+0000 7fa46f200640 1 Processor -- start 2024-04-24T06:30:11.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.642+0000 7fa46f200640 1 -- start start 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.643+0000 7fa46f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4700d0c10 0x7fa4700cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.643+0000 7fa46f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa4700af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.643+0000 7fa46f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4701676a0 con 0x7fa4700d0830 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.643+0000 7fa46f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa470166b20 con 0x7fa4700d0c10 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.643+0000 7fa46f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa470166e00 con 0x7fa4700af010 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa46de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa4700d0830 0x7fa470163bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60006/0 (socket says 172.21.15.59:60006) 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa46de00640 1 -- 172.21.15.59:0/1326248211 learned_addr learned my addr 172.21.15.59:0/1326248211 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa467e00640 1 --2- 172.21.15.59:0/1326248211 >> v2:172.21.15.59:3300/0 conn(0x7fa4700d0c10 0x7fa4700cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa46e800640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa4700af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa467e00640 1 -- 172.21.15.59:0/1326248211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 msgr2=0x7fa4700af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa46d400640 1 -- 172.21.15.59:0/1326248211 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965862236 0 0) 0x7fa4701676a0 con 0x7fa4700d0830 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa467e00640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa4700af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa467e00640 1 -- 172.21.15.59:0/1326248211 >> v1:172.21.15.59:6789/0 conn(0x7fa4700d0830 legacy=0x7fa470163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa467e00640 1 -- 172.21.15.59:0/1326248211 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4701670e0 con 0x7fa4700d0c10 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.644+0000 7fa46e800640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa4700af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.645+0000 7fa467e00640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0c10 0x7fa4700cefd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426670039cef7638 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.646+0000 7fa46d400640 1 -- 172.21.15.59:0/1326248211 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458037070 con 0x7fa4700d0c10 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.646+0000 7fa46d400640 1 -- 172.21.15.59:0/1326248211 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45802ae50 con 0x7fa4700d0c10 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.646+0000 7fa46d400640 1 -- 172.21.15.59:0/1326248211 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458032660 con 0x7fa4700d0c10 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.646+0000 7fa46f200640 1 -- 172.21.15.59:0/1326248211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0c10 msgr2=0x7fa4700cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.646+0000 7fa46f200640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0c10 0x7fa4700cefd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802a100 comp rx=0 tx=0).stop 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 -- 172.21.15.59:0/1326248211 shutdown_connections 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa4700af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 --2- 172.21.15.59:0/1326248211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0c10 0x7fa4700cefd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 -- 172.21.15.59:0/1326248211 >> 172.21.15.59:0/1326248211 conn(0x7fa47006bdf0 msgr2=0x7fa47006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 -- 172.21.15.59:0/1326248211 shutdown_connections 2024-04-24T06:30:11.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.647+0000 7fa46f200640 1 -- 172.21.15.59:0/1326248211 wait complete. 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.648+0000 7fa46f200640 1 Processor -- start 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.649+0000 7fa46f200640 1 -- start start 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.649+0000 7fa46f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa47009a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.649+0000 7fa46f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa46f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4700d0c10 0x7fa4701552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa46f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700b45f0 con 0x7fa4700d0830 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa46f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700af990 con 0x7fa4700d0c10 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa46f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa470166740 con 0x7fa4700af010 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40146/0 (socket says 172.21.15.59:40146) 2024-04-24T06:30:11.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 -- 172.21.15.59:0/1060865859 learned_addr learned my addr 172.21.15.59:0/1060865859 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 -- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 msgr2=0x7fa47009a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa47009a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 -- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4700d0c10 msgr2=0x7fa4701552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4700d0c10 0x7fa4701552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.650+0000 7fa467e00640 1 -- 172.21.15.59:0/1060865859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47009d3d0 con 0x7fa4700d0830 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa467e00640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa4580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa466000640 1 -- 172.21.15.59:0/1060865859 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458037040 con 0x7fa4700d0830 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa466000640 1 -- 172.21.15.59:0/1060865859 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa458004250 con 0x7fa4700d0830 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa466000640 1 -- 172.21.15.59:0/1060865859 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458003610 con 0x7fa4700d0830 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47009d6b0 con 0x7fa4700d0830 2024-04-24T06:30:11.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.651+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47009dc70 con 0x7fa4700d0830 2024-04-24T06:30:11.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.653+0000 7fa466000640 1 -- 172.21.15.59:0/1060865859 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4580037b0 con 0x7fa4700d0830 2024-04-24T06:30:11.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.655+0000 7fa466000640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 0x7fa44c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.655+0000 7fa466000640 1 -- 172.21.15.59:0/1060865859 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa4580acb00 con 0x7fa4700d0830 2024-04-24T06:30:11.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.655+0000 7fa46de00640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 0x7fa44c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.656+0000 7fa4779e1740 1 Processor -- start 2024-04-24T06:30:11.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.656+0000 7fa4779e1740 1 -- start start 2024-04-24T06:30:11.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.656+0000 7fa46de00640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 0x7fa44c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa460007fd0 tx=0x7fa460007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b40610de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b406048b10 con 0x55b40610e360 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b405fef2b0 con 0x55b40612b980 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b406044ad0 con 0x55b40610eaa0 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40162/0 (socket says 172.21.15.59:40162) 2024-04-24T06:30:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.657+0000 7fa46e800640 1 -- 172.21.15.59:0/2231655372 learned_addr learned my addr 172.21.15.59:0/2231655372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa467e00640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b40610de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46de00640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 -- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 msgr2=0x55b40610de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b40610de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 -- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 msgr2=0x55b4060de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 -- 172.21.15.59:0/2231655372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b405d5b2d0 con 0x55b40610e360 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46de00640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa46e800640 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa468008e60 tx=0x7fa46800d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc13ebec8f8f2de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa45e600640 1 -- 172.21.15.59:0/2231655372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800f800 con 0x55b40610e360 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa45e600640 1 -- 172.21.15.59:0/2231655372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46800f9a0 con 0x55b40610e360 2024-04-24T06:30:11.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.658+0000 7fa45e600640 1 -- 172.21.15.59:0/2231655372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800fd70 con 0x55b40610e360 2024-04-24T06:30:11.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 -- 172.21.15.59:0/2231655372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 msgr2=0x55b4060e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa468008e60 tx=0x7fa46800d6e0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 -- 172.21.15.59:0/2231655372 shutdown_connections 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610e360 0x55b4060e3170 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b40610de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 --2- 172.21.15.59:0/2231655372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 -- 172.21.15.59:0/2231655372 >> 172.21.15.59:0/2231655372 conn(0x55b4060e0580 msgr2=0x55b406128e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 -- 172.21.15.59:0/2231655372 shutdown_connections 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.659+0000 7fa4779e1740 1 -- 172.21.15.59:0/2231655372 wait complete. 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.660+0000 7fa4779e1740 1 -- start start 2024-04-24T06:30:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.660+0000 7fa4779e1740 1 Processor -- start 2024-04-24T06:30:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40610eaa0 0x55b4060fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40612b980 0x55b40613cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4060e2730 con 0x55b4061398e0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b406002590 con 0x55b40610eaa0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b4060e1b20 con 0x55b40612b980 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.661+0000 7fa4779e1740 10 client.0 _ll_register_callbacks cb 0x55b405fe53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40168/0 (socket says 172.21.15.59:40168) 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 -- 172.21.15.59:0/1835464582 learned_addr learned my addr 172.21.15.59:0/1835464582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 -- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40612b980 msgr2=0x55b40613cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa467e00640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40612b980 0x55b40613cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46de00640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40610eaa0 0x55b4060fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40612b980 0x55b40613cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 -- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40610eaa0 msgr2=0x55b4060fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40610eaa0 0x55b4060fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 -- 172.21.15.59:0/1835464582 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b405e256b0 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa46e800640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa46800dbe0 tx=0x7fa468005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58247a1e2540653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.662+0000 7fa452a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468028060 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa468018680 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46800f9a0 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa46800fe60 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa4779e1740 10 client.5382 fetch_fsmap learned FSMap version 17 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b40613d480 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46800f9a0 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fa468003990 con 0x55b4061398e0 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa4779e1740 10 client.5382 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.663+0000 7fa4779e1740 10 client.5382 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:30:11.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.664+0000 7fa4779e1740 20 client.5382 populate_metadata read hostname 'smithi059' 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.664+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b406129370 0x55b406129750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.664+0000 7fa4779e1740 4 client.5382 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.664+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 --> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b406056530 con 0x55b406129370 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.664+0000 7fa46de00640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b406129370 0x55b406129750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.665+0000 7fa46de00640 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b406129370 0x55b406129750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588382ee620545d5 server_cookie=33754d90669a155a in_seq=0 out_seq=0 2024-04-24T06:30:11.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.665+0000 7fa452a00640 10 client.5382 ms_handle_connect on v2:172.21.15.66:6832/2735852616 2024-04-24T06:30:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa452a00640 1 -- 172.21.15.59:0/1835464582 <== mds.0 v2:172.21.15.66:6832/2735852616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b406056530 con 0x55b406129370 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa452a00640 10 client.5382 handle_command_reply: tid=0 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 client.5382 shutdown 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 msgr2=0x55b40610f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa46800dbe0 tx=0x7fa468005d10 comp rx=0 tx=0).stop 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 shutdown_connections 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b406129370 0x55b406129750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40610f450 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40612b980 0x55b40613cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1835464582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40610eaa0 0x55b4060fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa452a00640 0 client.5382 ms_handle_reset on v2:172.21.15.66:6832/2735852616 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.668+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 >> 172.21.15.59:0/1835464582 conn(0x55b40610e360 msgr2=0x55b40612c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.669+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 shutdown_connections 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.669+0000 7fa4779e1740 1 -- 172.21.15.59:0/1835464582 wait complete. 2024-04-24T06:30:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.669+0000 7fa4779e1740 20 client.5382 trim_cache size 0 max 16384 2024-04-24T06:30:11.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.689+0000 7fa4779e1740 1 Processor -- start 2024-04-24T06:30:11.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.689+0000 7fa4779e1740 1 -- start start 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b4061398e0 0x55b40612c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa470204ff0 con 0x55b40610eaa0 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa470204130 con 0x55b40612b980 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa47006f9d0 con 0x55b4061398e0 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40184/0 (socket says 172.21.15.59:40184) 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 -- 172.21.15.59:0/1693135349 learned_addr learned my addr 172.21.15.59:0/1693135349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 -- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b4061398e0 msgr2=0x55b40612c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b4061398e0 0x55b40612c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 -- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 msgr2=0x55b4060e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.690+0000 7fa46de00640 1 -- 172.21.15.59:0/1693135349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b406138670 con 0x55b40610eaa0 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa46de00640 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa4600011e0 tx=0x7fa460010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57439afa51da5c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa45dc00640 1 -- 172.21.15.59:0/1693135349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460022050 con 0x55b40610eaa0 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa45dc00640 1 -- 172.21.15.59:0/1693135349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4600221f0 con 0x55b40610eaa0 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa45dc00640 1 -- 172.21.15.59:0/1693135349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600225a0 con 0x55b40610eaa0 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa4779e1740 1 -- 172.21.15.59:0/1693135349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 msgr2=0x55b4060e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.691+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa4600011e0 tx=0x7fa460010000 comp rx=0 tx=0).stop 2024-04-24T06:30:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 -- 172.21.15.59:0/1693135349 shutdown_connections 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b4061398e0 0x55b40612c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1693135349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b40610eaa0 0x55b4060e3e10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 -- 172.21.15.59:0/1693135349 >> 172.21.15.59:0/1693135349 conn(0x55b40610e360 msgr2=0x55b406129750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 -- 172.21.15.59:0/1693135349 shutdown_connections 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 -- 172.21.15.59:0/1693135349 wait complete. 2024-04-24T06:30:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.692+0000 7fa4779e1740 1 -- start start 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.693+0000 7fa4779e1740 1 Processor -- start 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b4061a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4061390e0 con 0x55b4061398e0 2024-04-24T06:30:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa470161e30 con 0x55b40612b980 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b40610fd80 con 0x55b40610eaa0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa4779e1740 10 client.0 _ll_register_callbacks cb 0x7fa46800deb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40198/0 (socket says 172.21.15.59:40198) 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 -- 172.21.15.59:0/1313632450 learned_addr learned my addr 172.21.15.59:0/1313632450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46de00640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b4061a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa467e00640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 -- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 msgr2=0x55b4061a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b4061a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 -- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 msgr2=0x55b4060fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.694+0000 7fa46e800640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa46e800640 1 -- 172.21.15.59:0/1313632450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4061a2950 con 0x55b4061398e0 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa467e00640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:11.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa46e800640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa4680186b0 tx=0x7fa4680086f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1c8a95392b01b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa452a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468037060 con 0x55b4061398e0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa468018ce0 con 0x55b4061398e0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4600221f0 con 0x55b4061398e0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa468032450 con 0x55b4061398e0 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.695+0000 7fa4779e1740 10 client.5388 fetch_fsmap learned FSMap version 17 2024-04-24T06:30:11.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b406111960 con 0x55b4061398e0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4600221f0 con 0x55b4061398e0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fa4680325f0 con 0x55b4061398e0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 10 client.5388 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 10 client.5388 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 20 client.5388 populate_metadata read hostname 'smithi059' 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b4060e42e0 0x55b40611e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 4 client.5388 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.696+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 --> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] -- command(tid 0: {"prefix": "status"}) -- 0x55b4060503a0 con 0x55b4060e42e0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.697+0000 7fa46de00640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b4060e42e0 0x55b40611e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.698+0000 7fa46de00640 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b4060e42e0 0x55b40611e7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0df72bfd851dc0f server_cookie=56483443639f051d in_seq=0 out_seq=0 2024-04-24T06:30:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.698+0000 7fa452a00640 10 client.5388 ms_handle_connect on v2:172.21.15.66:6832/2735852616 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.698+0000 7fa452a00640 1 -- 172.21.15.59:0/1313632450 <== mds.0 v2:172.21.15.66:6832/2735852616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55b4060503a0 con 0x55b4060e42e0 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.698+0000 7fa452a00640 10 client.5388 handle_command_reply: tid=0 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 client.5388 shutdown 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 msgr2=0x55b40614b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa4680186b0 tx=0x7fa4680086f0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 shutdown_connections 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:6832/2735852616,v1:172.21.15.66:6833/2735852616] conn(0x55b4060e42e0 0x55b40611e7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4061398e0 0x55b40614b9e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b40612b980 0x55b4060fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 --2- 172.21.15.59:0/1313632450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b40610eaa0 0x55b4061a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 >> 172.21.15.59:0/1313632450 conn(0x55b40610e360 msgr2=0x55b4060d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa452a00640 0 client.5388 ms_handle_reset on v2:172.21.15.66:6832/2735852616 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 shutdown_connections 2024-04-24T06:30:11.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 1 -- 172.21.15.59:0/1313632450 wait complete. 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.699+0000 7fa4779e1740 20 client.5388 trim_cache size 0 max 16384 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5205, 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:30:11.702 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.783699371, 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 17, 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 33, 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 33, 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 15.745780457 2024-04-24T06:30:11.703 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 msgr2=0x7fa44c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 0x7fa44c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa460007fd0 tx=0x7fa460007920 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 msgr2=0x7fa4700b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa4580046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 shutdown_connections 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa44c06c8f0 0x7fa44c06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4700d0c10 0x7fa4701552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700d0830 0x7fa4700b1f60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 --2- 172.21.15.59:0/1060865859 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4700af010 0x7fa47009a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 >> 172.21.15.59:0/1060865859 conn(0x7fa47006bdf0 msgr2=0x7fa4700ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 shutdown_connections 2024-04-24T06:30:11.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.700+0000 7fa46f200640 1 -- 172.21.15.59:0/1060865859 wait complete. 2024-04-24T06:30:11.719 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:30:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.925+0000 7fcb0ae00640 1 Processor -- start 2024-04-24T06:30:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.925+0000 7fcb0ae00640 1 -- start start 2024-04-24T06:30:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.927+0000 7fcb0ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 0x7fcb040ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.927+0000 7fcb0ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcb0409bad0 0x7fcb040cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.927+0000 7fcb0ae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb0409aaa0 con 0x7fcb0415c670 2024-04-24T06:30:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.927+0000 7fcb0ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0409b640 con 0x7fcb0409bad0 2024-04-24T06:30:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.927+0000 7fcb0ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0415ca50 con 0x7fcb040d0b80 2024-04-24T06:30:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.928+0000 7fcb09000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 0x7fcb040ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.928+0000 7fcb0a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcb0409bad0 0x7fcb040cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.928+0000 7fcb09a00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fcb0415c670 0x7fcb040d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:42220/0 (socket says 172.21.15.59:42220) 2024-04-24T06:30:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.928+0000 7fcb09a00640 1 -- 172.21.15.59:0/1153208974 learned_addr learned my addr 172.21.15.59:0/1153208974 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.929+0000 7fcb09000640 1 -- 172.21.15.59:0/1153208974 >> v2:172.21.15.66:3300/0 conn(0x7fcb0409bad0 msgr2=0x7fcb040cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.929+0000 7fcb09000640 1 --2- 172.21.15.59:0/1153208974 >> v2:172.21.15.66:3300/0 conn(0x7fcb0409bad0 0x7fcb040cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.929+0000 7fcb09000640 1 -- 172.21.15.59:0/1153208974 >> v1:172.21.15.66:6789/0 conn(0x7fcb0415c670 legacy=0x7fcb040d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.929+0000 7fcb09000640 1 -- 172.21.15.59:0/1153208974 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0409ad10 con 0x7fcb040d0b80 2024-04-24T06:30:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.930+0000 7fcb09000640 1 --2- 172.21.15.59:0/1153208974 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 0x7fcb040ccf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00097a0 tx=0x7fcaf002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cab55dff6a3a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb03e00640 1 -- 172.21.15.59:0/1153208974 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0037070 con 0x7fcb040d0b80 2024-04-24T06:30:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb03e00640 1 -- 172.21.15.59:0/1153208974 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0032440 con 0x7fcb040d0b80 2024-04-24T06:30:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb03e00640 1 -- 172.21.15.59:0/1153208974 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0032810 con 0x7fcb040d0b80 2024-04-24T06:30:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1153208974 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 msgr2=0x7fcb040ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1153208974 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 0x7fcb040ccf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00097a0 tx=0x7fcaf002a180 comp rx=0 tx=0).stop 2024-04-24T06:30:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1153208974 shutdown_connections 2024-04-24T06:30:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1153208974 >> v2:172.21.15.66:3300/0 conn(0x7fcb0409bad0 0x7fcb040cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1153208974 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb040d0b80 0x7fcb040ccf00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1153208974 >> 172.21.15.59:0/1153208974 conn(0x7fcb040b3ee0 msgr2=0x7fcb040b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1153208974 shutdown_connections 2024-04-24T06:30:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.931+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1153208974 wait complete. 2024-04-24T06:30:11.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.933+0000 7fcb0ae00640 1 Processor -- start 2024-04-24T06:30:11.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.933+0000 7fcb0ae00640 1 -- start start 2024-04-24T06:30:11.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 0x7fcb040c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 0x7fcb040c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d1190 con 0x7fcb040d0b80 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0409b2d0 con 0x7fcb0415c670 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.934+0000 7fcb0ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040d1310 con 0x7fcb0409bad0 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40208/0 (socket says 172.21.15.59:40208) 2024-04-24T06:30:11.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 -- 172.21.15.59:0/1817336067 learned_addr learned my addr 172.21.15.59:0/1817336067 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 -- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 msgr2=0x7fcb040c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09a00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 0x7fcb040c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb0a400640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 0x7fcb040c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 0x7fcb040c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 -- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 msgr2=0x7fcb040c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:11.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 0x7fcb040c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09000640 1 -- 172.21.15.59:0/1817336067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb040b5220 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb0a400640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 0x7fcb040c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.935+0000 7fcb09a00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 0x7fcb040c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb09000640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0035040 tx=0x7fcaf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0041020 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0030700 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb040b5500 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0037040 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.936+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb040b5ac0 con 0x7fcb040d0b80 2024-04-24T06:30:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.937+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040b4f90 con 0x7fcb040d0b80 2024-04-24T06:30:11.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.938+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaf0044020 con 0x7fcb040d0b80 2024-04-24T06:30:11.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.939+0000 7fcb01600640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 0x7fcae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:11.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.939+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcaf00ac3f0 con 0x7fcb040d0b80 2024-04-24T06:30:11.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.939+0000 7fcb09a00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 0x7fcae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:11.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.940+0000 7fcb09a00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 0x7fcae406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcb040b7630 tx=0x7fcaf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:11.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:11.944+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcaf004b030 con 0x7fcb040d0b80 2024-04-24T06:30:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.196+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb040b5c90 con 0x7fcb040d0b80 2024-04-24T06:30:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.197+0000 7fcb01600640 1 -- 172.21.15.59:0/1817336067 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fcaf007a810 con 0x7fcb040d0b80 2024-04-24T06:30:12.198 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 msgr2=0x7fcae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 0x7fcae406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcb040b7630 tx=0x7fcaf8009040 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 msgr2=0x7fcb04159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0035040 tx=0x7fcaf00046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 shutdown_connections 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcae406c8f0 0x7fcae406edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb0415c670 0x7fcb040c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb040d0b80 0x7fcb04159c00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 --2- 172.21.15.59:0/1817336067 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb0409bad0 0x7fcb040c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.200+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 >> 172.21.15.59:0/1817336067 conn(0x7fcb040b3ee0 msgr2=0x7fcb04071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:12.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.201+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 shutdown_connections 2024-04-24T06:30:12.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12.201+0000 7fcb0ae00640 1 -- 172.21.15.59:0/1817336067 wait complete. 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:30:12.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:30:12.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:30:12.221 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:30:12.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:12.221 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:30:12.272 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42532sec preferred_lft 42532sec 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:30:12.273 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.274 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:30:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:30:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:30:12.275 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:30:12.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:12.276 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:30:12.361 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:30:12.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:12.365 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:30:12.421 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:30:12.421 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:30:12.421 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:30:12.422 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:30:12.422 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:30:12.423 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:30:12.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:30:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:12.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:12.650 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:30:12.706 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:30:12.707 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:30:12.707 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:30:12.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:12.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:30:12.789 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:30:12.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:30:12.790 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:12.845 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:30:12.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:30:12.846 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:12.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:30:12.995 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:30:13.051 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:30:13.051 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:30:13.051 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:30:13.052 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:30:13.053 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:30:13.054 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:30:13.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.055 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:30:13.107 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:30:13.107 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:30:13.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:30:13.183 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:30:13.225 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:30:13.225 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:30:13.225 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:30:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:30:13.227 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:30:13.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:30:13.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.229 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:30:13.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:30:13.285+0000 7f0a5b078480 -1 init, newargv = 0x558ff69daa70 newargc=15 2024-04-24T06:30:13.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47305]: starting ceph client 2024-04-24T06:30:13.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47305]: starting fuse 2024-04-24T06:30:13.316 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:30:13.316 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:30:13.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:13.317 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:30:13.318 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:13.514 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.47305.asok 2024-04-24T06:30:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:13.531 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47305.asok 2024-04-24T06:30:13.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47305.asok status 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "47305", 2024-04-24T06:30:13.766 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5397, 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 5397 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2653653901 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:13.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2653653901 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.5397 192.168.0.1:0/2653653901", 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2653653901", 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 17, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 33, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:30:13.768 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:30:13.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.781 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:30:13.781 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:13.840 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:30:13.840 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:13.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:13.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:13.932 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-24T06:30:14.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:14.008 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-24T06:30:14.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:14.074 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-24T06:30:14.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:14.142 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-24T06:30:14.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:14.215 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-24T06:30:14.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:14.280 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-24T06:30:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-24T06:30:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:./rfile 2024-04-24T06:30:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir 2024-04-24T06:30:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/pfile 2024-04-24T06:30:14.367 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/childdir 2024-04-24T06:30:14.368 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/childdir/cfile 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-24T06:30:14.369 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:14.519 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:30:14.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:14.532 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:14.533 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:14.533 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:14.533 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:14.533 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-24T06:30:14.533 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:14.725 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-24T06:30:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:14.739 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:14.740 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:14.740 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:14.740 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:14.740 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-24T06:30:14.740 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:14.934 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:30:14.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-24T06:30:14.951 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:15.147 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-24T06:30:15.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:15.162 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:15.162 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:15.162 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:15.162 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:15.162 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-24T06:30:15.163 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:15.358 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:30:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-24T06:30:15.372 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:30:15.567 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-24T06:30:15.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:30:15.583 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:15.583 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:15.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:15.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.895+0000 7f575ea00640 1 Processor -- start 2024-04-24T06:30:15.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.895+0000 7f575ea00640 1 -- start start 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f575ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f57580b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f575ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57580cd520 con 0x7f57581454e0 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f575ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57580cd800 con 0x7f57580b3e40 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f575ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57580cddc0 con 0x7f57580cd020 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f5757e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f57581454e0 0x7f5758145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60008/0 (socket says 172.21.15.59:60008) 2024-04-24T06:30:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.897+0000 7f5757e00640 1 -- 172.21.15.59:0/3484415075 learned_addr learned my addr 172.21.15.59:0/3484415075 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:15.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f574fe00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f57580b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106156029 0 0) 0x7f57580cd520 con 0x7f57581454e0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f573c003520 con 0x7f57581454e0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3320345324 0 0) 0x7f573c003520 con 0x7f57581454e0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 msgr2=0x7f57580b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f57580b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 >> v1:172.21.15.66:6789/0 conn(0x7f57580b3e40 legacy=0x7f5758160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57580cdae0 con 0x7f57581454e0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5748001670 con 0x7f57581454e0 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 >> v1:172.21.15.59:6789/0 conn(0x7f57581454e0 legacy=0x7f5758145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c004800 0x7f573c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f573c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57580cddc0 con 0x7f573c004800 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f573c00baf0 con 0x7f573c0039e0 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.898+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f573c00bcf0 con 0x7f57580cd020 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c004800 0x7f573c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f5757e00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 -- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 msgr2=0x7f573c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f575ca00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f573c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f573c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 -- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573c0039e0 msgr2=0x7f573c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f574fe00640 1 -- 172.21.15.59:0/3484415075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c003520 con 0x7f573c004800 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f575ca00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57580cd020 0x7f573c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.899+0000 7f5757e00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f573c0039e0 0x7f573c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.900+0000 7f574fe00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c004800 0x7f573c004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ac0 tx=0x7f573c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6202b290b6293c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.900+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740036070 con 0x7f573c004800 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.900+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740031400 con 0x7f573c004800 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f5757400640 1 -- 172.21.15.59:0/3484415075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740009690 con 0x7f573c004800 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 -- 172.21.15.59:0/3484415075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c004800 msgr2=0x7f573c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 --2- 172.21.15.59:0/3484415075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c004800 0x7f573c004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ac0 tx=0x7f573c005830 comp rx=0 tx=0).stop 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ca00640 1 -- 172.21.15.59:0/3484415075 reap_dead start 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 -- 172.21.15.59:0/3484415075 shutdown_connections 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 -- 172.21.15.59:0/3484415075 >> 172.21.15.59:0/3484415075 conn(0x7f5758067ea0 msgr2=0x7f575806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 -- 172.21.15.59:0/3484415075 shutdown_connections 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.901+0000 7f575ea00640 1 -- 172.21.15.59:0/3484415075 wait complete. 2024-04-24T06:30:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.903+0000 7f575ea00640 1 Processor -- start 2024-04-24T06:30:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.903+0000 7f575ea00640 1 -- start start 2024-04-24T06:30:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f573c004800 0x7f57581f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5758142df0 0x7f5758143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5758161020 con 0x7f575813fab0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57581462c0 con 0x7f5758142df0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.904+0000 7f575ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57581611a0 con 0x7f573c004800 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40224/0 (socket says 172.21.15.59:40224) 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 -- 172.21.15.59:0/3250340759 learned_addr learned my addr 172.21.15.59:0/3250340759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 -- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f573c004800 msgr2=0x7f57581f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f575ca00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5758142df0 0x7f5758143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f5757e00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f573c004800 0x7f57581f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f573c004800 0x7f57581f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 -- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5758142df0 msgr2=0x7f5758143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5758142df0 0x7f5758143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.905+0000 7f574fe00640 1 -- 172.21.15.59:0/3250340759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5758143bc0 con 0x7f575813fab0 2024-04-24T06:30:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f574fe00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ac0 tx=0x7f5740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740040020 con 0x7f575813fab0 2024-04-24T06:30:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740003bc0 con 0x7f575813fab0 2024-04-24T06:30:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5758143ea0 con 0x7f575813fab0 2024-04-24T06:30:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740036040 con 0x7f575813fab0 2024-04-24T06:30:15.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.906+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5758144460 con 0x7f575813fab0 2024-04-24T06:30:15.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.907+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5758140470 con 0x7f575813fab0 2024-04-24T06:30:15.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.908+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5740043020 con 0x7f575813fab0 2024-04-24T06:30:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.910+0000 7f5754c00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.910+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f57400abd30 con 0x7f575813fab0 2024-04-24T06:30:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.910+0000 7f5757e00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.911+0000 7f5757e00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 0x7f573406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57480017e0 tx=0x7f5748009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:15.916+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f574004a070 con 0x7f575813fab0 2024-04-24T06:30:16.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.123+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f575815c6f0 con 0x7f575813fab0 2024-04-24T06:30:16.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.126+0000 7f5754c00640 1 -- 172.21.15.59:0/3250340759 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f57400044e0 con 0x7f575813fab0 2024-04-24T06:30:16.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:16.128 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 msgr2=0x7f573406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 0x7f573406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57480017e0 tx=0x7f5748009040 comp rx=0 tx=0).stop 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 msgr2=0x7f575813ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5740009ac0 tx=0x7f5740004730 comp rx=0 tx=0).stop 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 shutdown_connections 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5758142df0 0x7f5758143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.129+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f575813fab0 0x7f575813ff30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.130+0000 7f575ea00640 1 --2- 172.21.15.59:0/3250340759 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f573c004800 0x7f57581f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.130+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 >> 172.21.15.59:0/3250340759 conn(0x7f5758067ea0 msgr2=0x7f5758140c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.130+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 shutdown_connections 2024-04-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.130+0000 7f575ea00640 1 -- 172.21.15.59:0/3250340759 wait complete. 2024-04-24T06:30:16.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:30:02.277562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:30:03.324428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:16.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:16.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.371+0000 7f8ddd400640 1 Processor -- start 2024-04-24T06:30:16.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.371+0000 7f8ddd400640 1 -- start start 2024-04-24T06:30:16.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dd81567e0 0x7f8dd8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 0x7f8dd80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8dd80d0b40 0x7f8dd8160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd80d0f70 con 0x7f8dd81567e0 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd815cab0 con 0x7f8dd80d0b40 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.372+0000 7f8ddd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd8160e40 con 0x7f8dd815d810 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dd81567e0 0x7f8dd8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dd81567e0 0x7f8dd8156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40228/0 (socket says 172.21.15.59:40228) 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 -- 172.21.15.59:0/1541504753 learned_addr learned my addr 172.21.15.59:0/1541504753 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8ddca00640 1 --2- 172.21.15.59:0/1541504753 >> v2:172.21.15.66:3300/0 conn(0x7f8dd80d0b40 0x7f8dd8160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7400640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 0x7f8dd80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 -- 172.21.15.59:0/1541504753 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 msgr2=0x7f8dd80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 0x7f8dd80d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 -- 172.21.15.59:0/1541504753 >> v2:172.21.15.66:3300/0 conn(0x7f8dd80d0b40 msgr2=0x7f8dd8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/1541504753 >> v2:172.21.15.66:3300/0 conn(0x7f8dd80d0b40 0x7f8dd8160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7e00640 1 -- 172.21.15.59:0/1541504753 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd815cd30 con 0x7f8dd81567e0 2024-04-24T06:30:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.373+0000 7f8dd7400640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 0x7f8dd80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.374+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81567e0 0x7f8dd8156bc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8142d50 tx=0x7f8dc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7dbaae92e0e90f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.375+0000 7f8dd6a00640 1 -- 172.21.15.59:0/1541504753 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8037070 con 0x7f8dd81567e0 2024-04-24T06:30:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.375+0000 7f8dd6a00640 1 -- 172.21.15.59:0/1541504753 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc8032440 con 0x7f8dd81567e0 2024-04-24T06:30:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.375+0000 7f8dd6a00640 1 -- 172.21.15.59:0/1541504753 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8032810 con 0x7f8dd81567e0 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.375+0000 7f8ddd400640 1 -- 172.21.15.59:0/1541504753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81567e0 msgr2=0x7f8dd8156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.375+0000 7f8ddd400640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81567e0 0x7f8dd8156bc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8142d50 tx=0x7f8dc802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 -- 172.21.15.59:0/1541504753 shutdown_connections 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 --2- 172.21.15.59:0/1541504753 >> v2:172.21.15.66:3300/0 conn(0x7f8dd80d0b40 0x7f8dd8160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd815d810 0x7f8dd80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 --2- 172.21.15.59:0/1541504753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81567e0 0x7f8dd8156bc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 -- 172.21.15.59:0/1541504753 >> 172.21.15.59:0/1541504753 conn(0x7f8dd806bdf0 msgr2=0x7f8dd806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:16.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 -- 172.21.15.59:0/1541504753 shutdown_connections 2024-04-24T06:30:16.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.376+0000 7f8ddd400640 1 -- 172.21.15.59:0/1541504753 wait complete. 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.377+0000 7f8ddd400640 1 Processor -- start 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.378+0000 7f8ddd400640 1 -- start start 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd80d0b40 0x7f8dd8154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd81567e0 0x7f8dd8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8160650 con 0x7f8dd815d810 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd815d2f0 con 0x7f8dd81567e0 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd81607d0 con 0x7f8dd80d0b40 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40230/0 (socket says 172.21.15.59:40230) 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 -- 172.21.15.59:0/4246145549 learned_addr learned my addr 172.21.15.59:0/4246145549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd80d0b40 0x7f8dd8154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8dd7400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd81567e0 0x7f8dd8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 -- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd80d0b40 msgr2=0x7f8dd8154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd80d0b40 0x7f8dd8154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 -- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd81567e0 msgr2=0x7f8dd8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd81567e0 0x7f8dd8151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.379+0000 7f8ddca00640 1 -- 172.21.15.59:0/4246145549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd815f1a0 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8ddca00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0007e60 tx=0x7f8dd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0014420 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd0019070 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0014820 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd8153860 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.380+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd8153e20 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.382+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd804fd90 con 0x7f8dd815d810 2024-04-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.383+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dd00149c0 con 0x7f8dd815d810 2024-04-24T06:30:16.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.383+0000 7f8dcfe00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 0x7f8db806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.384+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8dd008c2f0 con 0x7f8dd815d810 2024-04-24T06:30:16.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.384+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 0x7f8db806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.385+0000 7f8dd7e00640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 0x7f8db806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8034000 tx=0x7f8dc802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.388+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dd005a720 con 0x7f8dd815d810 2024-04-24T06:30:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.591+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8dd80b5150 con 0x7f8dd815d810 2024-04-24T06:30:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.592+0000 7f8dcfe00640 1 -- 172.21.15.59:0/4246145549 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f8dd005a020 con 0x7f8dd815d810 2024-04-24T06:30:16.593 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:16.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 msgr2=0x7f8db806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 0x7f8db806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8034000 tx=0x7f8dc802e000 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 msgr2=0x7f8dd814f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0007e60 tx=0x7f8dd000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 shutdown_connections 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8db806c920 0x7f8db806ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd815d810 0x7f8dd814f080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd81567e0 0x7f8dd8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 --2- 172.21.15.59:0/4246145549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8dd80d0b40 0x7f8dd8154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 >> 172.21.15.59:0/4246145549 conn(0x7f8dd806bdf0 msgr2=0x7f8dd809d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 shutdown_connections 2024-04-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.595+0000 7f8ddd400640 1 -- 172.21.15.59:0/4246145549 wait complete. 2024-04-24T06:30:16.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:30:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.819+0000 7f2486200640 1 Processor -- start 2024-04-24T06:30:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.819+0000 7f2486200640 1 -- start start 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2486200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 0x7f248009ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2486200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24800d15c0 con 0x7f24800b1850 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2486200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24800d00f0 con 0x7f248009b7d0 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2486200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f24800d0a20 con 0x7f248009aa00 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2484e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f24800b1850 0x7f24800cdc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60018/0 (socket says 172.21.15.59:60018) 2024-04-24T06:30:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2484e00640 1 -- 172.21.15.59:0/4179441816 learned_addr learned my addr 172.21.15.59:0/4179441816 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.821+0000 7f2485800640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 0x7f248009ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432383577 0 0) 0x7f24800d15c0 con 0x7f24800b1850 2024-04-24T06:30:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2468003520 con 0x7f24800b1850 2024-04-24T06:30:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1925126636 0 0) 0x7f2468003520 con 0x7f24800b1850 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 msgr2=0x7f248009ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 0x7f248009ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 >> v1:172.21.15.66:6789/0 conn(0x7f248009b7d0 legacy=0x7f24800d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24800d03d0 con 0x7f24800b1850 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2474001670 con 0x7f24800b1850 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 >> v1:172.21.15.59:6789/0 conn(0x7f24800b1850 legacy=0x7f24800cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24680039e0 0x7f2468003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 0x7f24680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24800d0a20 con 0x7f24680039e0 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f246800bad0 con 0x7f2468004800 2024-04-24T06:30:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f246800bcd0 con 0x7f248009aa00 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.822+0000 7f2484e00640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24680039e0 0x7f2468003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f2484e00640 1 -- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 msgr2=0x7f24680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f2484e00640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f248009aa00 0x7f24680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f2484e00640 1 -- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2468004800 msgr2=0x7f2468004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f247fe00640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f2484e00640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2468004800 0x7f2468004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.823+0000 7f2484e00640 1 -- 172.21.15.59:0/4179441816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2468003520 con 0x7f24680039e0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.824+0000 7f2484e00640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24680039e0 0x7f2468003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f247400c160 tx=0x7f2468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cfb616803af1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.824+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474007c60 con 0x7f24680039e0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474005ad0 con 0x7f24680039e0 2024-04-24T06:30:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f247f400640 1 -- 172.21.15.59:0/4179441816 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474007070 con 0x7f24680039e0 2024-04-24T06:30:16.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 -- 172.21.15.59:0/4179441816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24680039e0 msgr2=0x7f2468003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 --2- 172.21.15.59:0/4179441816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24680039e0 0x7f2468003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f247400c160 tx=0x7f2468005830 comp rx=0 tx=0).stop 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2485800640 1 -- 172.21.15.59:0/4179441816 reap_dead start 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 -- 172.21.15.59:0/4179441816 shutdown_connections 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 -- 172.21.15.59:0/4179441816 >> 172.21.15.59:0/4179441816 conn(0x7f24800b3f70 msgr2=0x7f24800b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 -- 172.21.15.59:0/4179441816 shutdown_connections 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.825+0000 7f2486200640 1 -- 172.21.15.59:0/4179441816 wait complete. 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.827+0000 7f2486200640 1 Processor -- start 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.827+0000 7f2486200640 1 -- start start 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24680039e0 0x7f2480156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24800cac80 0x7f24800cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f248009ae90 con 0x7f24800b01c0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24800ce480 con 0x7f24800cac80 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.828+0000 7f2486200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f248009b010 con 0x7f24680039e0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40262/0 (socket says 172.21.15.59:40262) 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 -- 172.21.15.59:0/4046695910 learned_addr learned my addr 172.21.15.59:0/4046695910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f2484e00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24680039e0 0x7f2480156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 -- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24680039e0 msgr2=0x7f2480156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24680039e0 0x7f2480156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f2485800640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24800cac80 0x7f24800cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 -- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24800cac80 msgr2=0x7f24800cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24800cac80 0x7f24800cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 -- 172.21.15.59:0/4046695910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24800c9ba0 con 0x7f24800b01c0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.829+0000 7f247fe00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f246c00d5a0 tx=0x7f246c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.830+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c005480 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.830+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c017070 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.830+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c003810 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.830+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24800c9e80 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.830+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24800ca440 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.831+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248004fd90 con 0x7f24800b01c0 2024-04-24T06:30:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.832+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f246c0039b0 con 0x7f24800b01c0 2024-04-24T06:30:16.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.833+0000 7f247cc00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 0x7f246006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:16.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.833+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f246c08ba10 con 0x7f24800b01c0 2024-04-24T06:30:16.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.833+0000 7f2484e00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 0x7f246006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.834+0000 7f2484e00640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 0x7f246006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24680057e0 tx=0x7f247402d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:16.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:16.838+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f246c059da0 con 0x7f24800b01c0 2024-04-24T06:30:17.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.042+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24800b4e80 con 0x7f24800b01c0 2024-04-24T06:30:17.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.043+0000 7f247cc00640 1 -- 172.21.15.59:0/4046695910 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f246c01ae40 con 0x7f24800b01c0 2024-04-24T06:30:17.044 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 msgr2=0x7f246006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 0x7f246006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24680057e0 tx=0x7f247402d000 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 msgr2=0x7f24800b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f246c00d5a0 tx=0x7f246c00da60 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 shutdown_connections 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f246006c920 0x7f246006ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24800cac80 0x7f24800cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24800b01c0 0x7f24800b0640 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 --2- 172.21.15.59:0/4046695910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24680039e0 0x7f2480156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.046+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 >> 172.21.15.59:0/4046695910 conn(0x7f24800b3f70 msgr2=0x7f248006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:17.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.047+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 shutdown_connections 2024-04-24T06:30:17.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.047+0000 7f2486200640 1 -- 172.21.15.59:0/4046695910 wait complete. 2024-04-24T06:30:17.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:30:17.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.281+0000 7f2439800640 1 Processor -- start 2024-04-24T06:30:17.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.281+0000 7f2439800640 1 -- start start 2024-04-24T06:30:17.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.283+0000 7f2439800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f2434156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.283+0000 7f2439800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243415cab0 con 0x7f243415d810 2024-04-24T06:30:17.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.283+0000 7f2439800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2434160e40 con 0x7f24341567e0 2024-04-24T06:30:17.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.283+0000 7f2439800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f243415cd30 con 0x7f24340d0d10 2024-04-24T06:30:17.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.284+0000 7f2438e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f243415d810 0x7f2434160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60024/0 (socket says 172.21.15.59:60024) 2024-04-24T06:30:17.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.284+0000 7f2438e00640 1 -- 172.21.15.59:0/203050593 learned_addr learned my addr 172.21.15.59:0/203050593 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:17.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.284+0000 7f242b400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f2434156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.284+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327034452 0 0) 0x7f242c001490 con 0x7f243415d810 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.284+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2418003520 con 0x7f243415d810 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185530840 0 0) 0x7f2434160e40 con 0x7f24341567e0 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f242c001490 con 0x7f24341567e0 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1518134229 0 0) 0x7f2418003520 con 0x7f243415d810 2024-04-24T06:30:17.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 msgr2=0x7f2434156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f2434156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 >> v1:172.21.15.66:6789/0 conn(0x7f24341567e0 legacy=0x7f24340d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243415d010 con 0x7f243415d810 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f242c001bc0 con 0x7f243415d810 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 >> v1:172.21.15.59:6789/0 conn(0x7f243415d810 legacy=0x7f2434160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418003b70 0x7f2418003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2418004800 0x7f2418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f24180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f243415cd30 con 0x7f2418004800 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f241800ba70 con 0x7f2418003b70 2024-04-24T06:30:17.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f241800bc70 con 0x7f24340d0d10 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.285+0000 7f242b400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2418004800 0x7f2418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f2438e00640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f24180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f242b400640 1 -- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 msgr2=0x7f24180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f242b400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f24180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f242b400640 1 -- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418003b70 msgr2=0x7f2418003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f242b400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418003b70 0x7f2418003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f242b400640 1 -- 172.21.15.59:0/203050593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2418003520 con 0x7f2418004800 2024-04-24T06:30:17.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.286+0000 7f2438e00640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24340d0d10 0x7f24180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.287+0000 7f242b400640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2418004800 0x7f2418004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f241c009ac0 tx=0x7f2418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43890948a599512 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.287+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c036070 con 0x7f2418004800 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.287+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c031400 con 0x7f2418004800 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.287+0000 7f2433400640 1 -- 172.21.15.59:0/203050593 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c009690 con 0x7f2418004800 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 -- 172.21.15.59:0/203050593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2418004800 msgr2=0x7f2418004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 --2- 172.21.15.59:0/203050593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2418004800 0x7f2418004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f241c009ac0 tx=0x7f2418005830 comp rx=0 tx=0).stop 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2438e00640 1 -- 172.21.15.59:0/203050593 reap_dead start 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 -- 172.21.15.59:0/203050593 shutdown_connections 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 -- 172.21.15.59:0/203050593 >> 172.21.15.59:0/203050593 conn(0x7f243406bdf0 msgr2=0x7f243406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:17.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 -- 172.21.15.59:0/203050593 shutdown_connections 2024-04-24T06:30:17.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.288+0000 7f2439800640 1 -- 172.21.15.59:0/203050593 wait complete. 2024-04-24T06:30:17.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.290+0000 7f2439800640 1 Processor -- start 2024-04-24T06:30:17.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.290+0000 7f2439800640 1 -- start start 2024-04-24T06:30:17.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 0x7f2434072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2434151e10 0x7f2434152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2434156d40 con 0x7f24341f5350 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2434156bc0 con 0x7f2418004800 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2439800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2434156ec0 con 0x7f2434151e10 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2438e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2438e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40278/0 (socket says 172.21.15.59:40278) 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.291+0000 7f2438e00640 1 -- 172.21.15.59:0/2506506515 learned_addr learned my addr 172.21.15.59:0/2506506515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f242b400640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2434151e10 0x7f2434152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2433e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 0x7f2434072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 -- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2434151e10 msgr2=0x7f2434152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2434151e10 0x7f2434152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 -- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 msgr2=0x7f2434072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 0x7f2434072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 -- 172.21.15.59:0/2506506515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24341f5d10 con 0x7f24341f5350 2024-04-24T06:30:17.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2433e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 0x7f2434072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2438e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f243415dc20 tx=0x7f242c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c00bb80 con 0x7f24341f5350 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c00bd20 con 0x7f24341f5350 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c0072b0 con 0x7f24341f5350 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.292+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24341571f0 con 0x7f24341f5350 2024-04-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.293+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24341577b0 con 0x7f24341f5350 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.294+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243404fd90 con 0x7f24341f5350 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.295+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f242c01b020 con 0x7f24341f5350 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.296+0000 7f2430c00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.296+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f242c08be60 con 0x7f24341f5350 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.296+0000 7f2433e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.297+0000 7f2433e00640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 0x7f241006ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2424000f60 tx=0x7f2424009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.300+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f243415cab0 con 0x7f24341f5350 2024-04-24T06:30:17.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.505+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24340b4760 con 0x7f24341f5350 2024-04-24T06:30:17.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.506+0000 7f2430c00640 1 -- 172.21.15.59:0/2506506515 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f242c05a290 con 0x7f24341f5350 2024-04-24T06:30:17.507 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:30:17.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 msgr2=0x7f241006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 0x7f241006ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2424000f60 tx=0x7f2424009040 comp rx=0 tx=0).stop 2024-04-24T06:30:17.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 msgr2=0x7f24341f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f243415dc20 tx=0x7f242c00b4a0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 shutdown_connections 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24341f5350 0x7f24341f57d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2434151e10 0x7f2434152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 --2- 172.21.15.59:0/2506506515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2418004800 0x7f2434072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 >> 172.21.15.59:0/2506506515 conn(0x7f243406bdf0 msgr2=0x7f24340af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 shutdown_connections 2024-04-24T06:30:17.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.509+0000 7f2439800640 1 -- 172.21.15.59:0/2506506515 wait complete. 2024-04-24T06:30:17.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:30:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.748+0000 7f8397e00640 1 Processor -- start 2024-04-24T06:30:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.749+0000 7f8397e00640 1 -- start start 2024-04-24T06:30:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.750+0000 7f8397e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f83980af890 0x7f839815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.750+0000 7f8397e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83980b3720 con 0x7f8398144a20 2024-04-24T06:30:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.750+0000 7f8397e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8398144ed0 con 0x7f83980af4b0 2024-04-24T06:30:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.750+0000 7f8397e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f839815f3f0 con 0x7f83980af890 2024-04-24T06:30:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.751+0000 7f8397400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8398144a20 0x7f83980d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60030/0 (socket says 172.21.15.59:60030) 2024-04-24T06:30:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.751+0000 7f8397400640 1 -- 172.21.15.59:0/3582012828 learned_addr learned my addr 172.21.15.59:0/3582012828 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.751+0000 7f838fe00640 1 --2- 172.21.15.59:0/3582012828 >> v2:172.21.15.66:3300/0 conn(0x7f83980af890 0x7f839815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.751+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581788009 0 0) 0x7f83980b3720 con 0x7f8398144a20 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.751+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f837c003520 con 0x7f8398144a20 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1339708012 0 0) 0x7f837c003520 con 0x7f8398144a20 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 >> v2:172.21.15.66:3300/0 conn(0x7f83980af890 msgr2=0x7f839815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 --2- 172.21.15.59:0/3582012828 >> v2:172.21.15.66:3300/0 conn(0x7f83980af890 0x7f839815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 >> v1:172.21.15.66:6789/0 conn(0x7f83980af4b0 legacy=0x7f83980b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839815f6d0 con 0x7f8398144a20 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f83900019f0 con 0x7f8398144a20 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 >> v1:172.21.15.59:6789/0 conn(0x7f8398144a20 legacy=0x7f83980d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f837c003a00 0x7f837c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 0x7f837c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c0051c0 0x7f837c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f839815f3f0 con 0x7f837c004800 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f837c00c0c0 con 0x7f837c0051c0 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f837c00c2c0 con 0x7f837c003a00 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.752+0000 7f838fe00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 0x7f837c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f8396a00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f837c003a00 0x7f837c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f8397400640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c0051c0 0x7f837c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f838fe00640 1 -- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f837c003a00 msgr2=0x7f837c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f838fe00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f837c003a00 0x7f837c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f838fe00640 1 -- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c0051c0 msgr2=0x7f837c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f838fe00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c0051c0 0x7f837c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f838fe00640 1 -- 172.21.15.59:0/3582012828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c003520 con 0x7f837c004800 2024-04-24T06:30:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.753+0000 7f8397400640 1 -- 172.21.15.59:0/3582012828 reap_dead start 2024-04-24T06:30:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.754+0000 7f838fe00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 0x7f837c004c80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8380009ac0 tx=0x7f837c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b39584196e13b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.754+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380036070 con 0x7f837c004800 2024-04-24T06:30:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.754+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380031400 con 0x7f837c004800 2024-04-24T06:30:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.754+0000 7f8396000640 1 -- 172.21.15.59:0/3582012828 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380009690 con 0x7f837c004800 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 -- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 msgr2=0x7f837c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 0x7f837c004c80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8380009ac0 tx=0x7f837c005830 comp rx=0 tx=0).stop 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 -- 172.21.15.59:0/3582012828 shutdown_connections 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 --2- 172.21.15.59:0/3582012828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f837c004800 0x7f837c004c80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 -- 172.21.15.59:0/3582012828 >> 172.21.15.59:0/3582012828 conn(0x7f839806bdf0 msgr2=0x7f839806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 -- 172.21.15.59:0/3582012828 shutdown_connections 2024-04-24T06:30:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.755+0000 7f8397e00640 1 -- 172.21.15.59:0/3582012828 wait complete. 2024-04-24T06:30:17.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.757+0000 7f8397e00640 1 Processor -- start 2024-04-24T06:30:17.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.757+0000 7f8397e00640 1 -- start start 2024-04-24T06:30:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c004800 0x7f839809a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839815c9b0 0x7f8398145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f839815ed50 con 0x7f839815ce30 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f839815fc90 con 0x7f837c004800 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.758+0000 7f8397e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f839815eed0 con 0x7f839815c9b0 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40288/0 (socket says 172.21.15.59:40288) 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 -- 172.21.15.59:0/1907120236 learned_addr learned my addr 172.21.15.59:0/1907120236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f838fe00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839815c9b0 0x7f8398145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8396a00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c004800 0x7f839809a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 -- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839815c9b0 msgr2=0x7f8398145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839815c9b0 0x7f8398145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 -- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c004800 msgr2=0x7f839809a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c004800 0x7f839809a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 -- 172.21.15.59:0/1907120236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980ad190 con 0x7f839815ce30 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.759+0000 7f8397400640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83981622a0 tx=0x7f8390007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.760+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839000eaa0 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.760+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390018070 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.760+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83980ad470 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.760+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839000ee80 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.760+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83980ada30 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.761+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839804fd90 con 0x7f839815ce30 2024-04-24T06:30:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.762+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f839001d460 con 0x7f839815ce30 2024-04-24T06:30:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.763+0000 7f838ea00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.764+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f839008bef0 con 0x7f839815ce30 2024-04-24T06:30:17.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.764+0000 7f8396a00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:17.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.764+0000 7f8396a00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 0x7f837406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f839806a7b0 tx=0x7f8388009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.767+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f839005a310 con 0x7f839815ce30 2024-04-24T06:30:17.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.990+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f839806fba0 con 0x7f839815ce30 2024-04-24T06:30:17.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.990+0000 7f838ea00640 1 -- 172.21.15.59:0/1907120236 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f8390059c10 con 0x7f839815ce30 2024-04-24T06:30:17.992 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.993+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 msgr2=0x7f837406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.993+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 0x7f837406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f839806a7b0 tx=0x7f8388009040 comp rx=0 tx=0).stop 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.993+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 msgr2=0x7f83981471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.993+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83981622a0 tx=0x7f8390007b50 comp rx=0 tx=0).stop 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 shutdown_connections 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837406c8f0 0x7f837406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839815ce30 0x7f83981471c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839815c9b0 0x7f8398145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 --2- 172.21.15.59:0/1907120236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f837c004800 0x7f839809a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 >> 172.21.15.59:0/1907120236 conn(0x7f839806bdf0 msgr2=0x7f83980cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 shutdown_connections 2024-04-24T06:30:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:17.994+0000 7f8397e00640 1 -- 172.21.15.59:0/1907120236 wait complete. 2024-04-24T06:30:18.015 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:18.015 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:18.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.280+0000 7f34e1a00640 1 Processor -- start 2024-04-24T06:30:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.280+0000 7f34e1a00640 1 -- start start 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34e1a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f34dc1454e0 0x7f34dc145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34dc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34e1a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34dc0cd520 con 0x7f34dc0b3e40 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34e1a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc0cddc0 con 0x7f34dc1454e0 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34e1a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc0cd800 con 0x7f34dc0cd020 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34db400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f34dc0b3e40 0x7f34dc160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60042/0 (socket says 172.21.15.59:60042) 2024-04-24T06:30:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.282+0000 7f34db400640 1 -- 172.21.15.59:0/1055219985 learned_addr learned my addr 172.21.15.59:0/1055219985 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:18.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34d3e00640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34dc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34daa00640 1 --2- 172.21.15.59:0/1055219985 >> v2:172.21.15.66:3300/0 conn(0x7f34dc1454e0 0x7f34dc145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168464802 0 0) 0x7f34dc0cd520 con 0x7f34dc0b3e40 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c0003520 con 0x7f34dc0b3e40 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3479557003 0 0) 0x7f34c0003520 con 0x7f34dc0b3e40 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 msgr2=0x7f34dc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34dc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 >> v2:172.21.15.66:3300/0 conn(0x7f34dc1454e0 msgr2=0x7f34dc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 --2- 172.21.15.59:0/1055219985 >> v2:172.21.15.66:3300/0 conn(0x7f34dc1454e0 0x7f34dc145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc0cdae0 con 0x7f34dc0b3e40 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f34d40019f0 con 0x7f34dc0b3e40 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.283+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 >> v1:172.21.15.59:6789/0 conn(0x7f34dc0b3e40 legacy=0x7f34dc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c0003f30 0x7f34c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0cddc0 con 0x7f34c0004d40 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0cd800 con 0x7f34c0003f30 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f34c000ba10 con 0x7f34dc0cd020 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34daa00640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34d3e00640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c0003f30 0x7f34c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 -- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 msgr2=0x7f34c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc0cd020 0x7f34c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 -- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c0003f30 msgr2=0x7f34c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c0003f30 0x7f34c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34db400640 1 -- 172.21.15.59:0/1055219985 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c0003520 con 0x7f34c0004d40 2024-04-24T06:30:18.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.284+0000 7f34d3e00640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34c0003f30 0x7f34c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.285+0000 7f34db400640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f34d400a8d0 tx=0x7f34c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a7487c527362cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400a4c0 con 0x7f34c0004d40 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d402f070 con 0x7f34c0004d40 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34da000640 1 -- 172.21.15.59:0/1055219985 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d402bd80 con 0x7f34c0004d40 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34e1a00640 1 -- 172.21.15.59:0/1055219985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 msgr2=0x7f34c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34e1a00640 1 --2- 172.21.15.59:0/1055219985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34c00051c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f34d400a8d0 tx=0x7f34c0005830 comp rx=0 tx=0).stop 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34db400640 1 -- 172.21.15.59:0/1055219985 reap_dead start 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34e1a00640 1 -- 172.21.15.59:0/1055219985 shutdown_connections 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.286+0000 7f34e1a00640 1 -- 172.21.15.59:0/1055219985 >> 172.21.15.59:0/1055219985 conn(0x7f34dc067ea0 msgr2=0x7f34dc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.287+0000 7f34e1a00640 1 -- 172.21.15.59:0/1055219985 shutdown_connections 2024-04-24T06:30:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.287+0000 7f34e1a00640 1 -- 172.21.15.59:0/1055219985 wait complete. 2024-04-24T06:30:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.288+0000 7f34e1a00640 1 Processor -- start 2024-04-24T06:30:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.289+0000 7f34e1a00640 1 -- start start 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc1f4380 0x7f34dc1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc13f830 0x7f34dc13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc146270 con 0x7f34c0004d40 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1460f0 con 0x7f34dc13f830 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34e1a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc161020 con 0x7f34dc1f4380 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40306/0 (socket says 172.21.15.59:40306) 2024-04-24T06:30:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 -- 172.21.15.59:0/2521336123 learned_addr learned my addr 172.21.15.59:0/2521336123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34d3e00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc1f4380 0x7f34dc1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 -- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc1f4380 msgr2=0x7f34dc1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34db400640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc13f830 0x7f34dc13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc1f4380 0x7f34dc1f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 -- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc13f830 msgr2=0x7f34dc13fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc13f830 0x7f34dc13fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.290+0000 7f34daa00640 1 -- 172.21.15.59:0/2521336123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1405f0 con 0x7f34c0004d40 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34daa00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f34cc011700 tx=0x7f34cc011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00f9c0 con 0x7f34c0004d40 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc00fb60 con 0x7f34c0004d40 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc015460 con 0x7f34c0004d40 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc15e000 con 0x7f34c0004d40 2024-04-24T06:30:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.291+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc150970 con 0x7f34c0004d40 2024-04-24T06:30:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.292+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc145950 con 0x7f34c0004d40 2024-04-24T06:30:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.294+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34cc015600 con 0x7f34c0004d40 2024-04-24T06:30:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.294+0000 7f34d2a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 0x7f34b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.295+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f34cc08c540 con 0x7f34c0004d40 2024-04-24T06:30:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.295+0000 7f34d3e00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 0x7f34b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.296+0000 7f34d3e00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 0x7f34b806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b54b0 tx=0x7f34c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.299+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34dc145950 con 0x7f34c0004d40 2024-04-24T06:30:18.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.501+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34dc068a80 con 0x7f34c0004d40 2024-04-24T06:30:18.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.504+0000 7f34d2a00640 1 -- 172.21.15.59:0/2521336123 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f34cc019030 con 0x7f34c0004d40 2024-04-24T06:30:18.506 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:18.506 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:18.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 msgr2=0x7f34b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 0x7f34b806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b54b0 tx=0x7f34c4008040 comp rx=0 tx=0).stop 2024-04-24T06:30:18.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 msgr2=0x7f34dc150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f34cc011700 tx=0x7f34cc011bc0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 shutdown_connections 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f34b806c8f0 0x7f34b806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34dc13f830 0x7f34dc13fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f34dc1f4380 0x7f34dc1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 --2- 172.21.15.59:0/2521336123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34c0004d40 0x7f34dc150430 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 >> 172.21.15.59:0/2521336123 conn(0x7f34dc067ea0 msgr2=0x7f34dc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 shutdown_connections 2024-04-24T06:30:18.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.507+0000 7f34e1a00640 1 -- 172.21.15.59:0/2521336123 wait complete. 2024-04-24T06:30:18.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:30:02.277562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:30:03.324428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:18.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.748+0000 7fd9e1800640 1 Processor -- start 2024-04-24T06:30:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.749+0000 7fd9e1800640 1 -- start start 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.750+0000 7fd9e1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.750+0000 7fd9e1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.750+0000 7fd9e1800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc0cfc20 con 0x7fd9dc145e10 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.750+0000 7fd9e1800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd9dc0cff00 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.750+0000 7fd9e1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc0d01e0 con 0x7fd9dc071d70 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40322/0 (socket says 172.21.15.59:40322) 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 -- 172.21.15.59:0/3163005354 learned_addr learned my addr 172.21.15.59:0/3163005354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9db400640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 -- 172.21.15.59:0/3163005354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 msgr2=0x7fd9dc072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 -- 172.21.15.59:0/3163005354 >> v1:172.21.15.66:6789/0 conn(0x7fd9dc145e10 legacy=0x7fd9dc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.751+0000 7fd9d2000640 1 -- 172.21.15.59:0/3163005354 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc0d04c0 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.752+0000 7fd9d2000640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009e50 tx=0x7fd9c402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7166a0a54be0dce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.753+0000 7fd9da000640 1 -- 172.21.15.59:0/3163005354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4036070 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.753+0000 7fd9da000640 1 -- 172.21.15.59:0/3163005354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4031600 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.753+0000 7fd9da000640 1 -- 172.21.15.59:0/3163005354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40319d0 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.753+0000 7fd9e1800640 1 -- 172.21.15.59:0/3163005354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 msgr2=0x7fd9dc06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.753+0000 7fd9e1800640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009e50 tx=0x7fd9c402a710 comp rx=0 tx=0).stop 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 -- 172.21.15.59:0/3163005354 shutdown_connections 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 --2- 172.21.15.59:0/3163005354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc06f130 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 -- 172.21.15.59:0/3163005354 >> 172.21.15.59:0/3163005354 conn(0x7fd9dc067d90 msgr2=0x7fd9dc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 -- 172.21.15.59:0/3163005354 shutdown_connections 2024-04-24T06:30:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.754+0000 7fd9e1800640 1 -- 172.21.15.59:0/3163005354 wait complete. 2024-04-24T06:30:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.756+0000 7fd9e1800640 1 Processor -- start 2024-04-24T06:30:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.756+0000 7fd9e1800640 1 -- start start 2024-04-24T06:30:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9dc145e10 0x7fd9dc0cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1461f0 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc15cdd0 con 0x7fd9dc145e10 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9e1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc0cab40 con 0x7fd9dc071d70 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9d2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9d2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40324/0 (socket says 172.21.15.59:40324) 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9d2000640 1 -- 172.21.15.59:0/648811915 learned_addr learned my addr 172.21.15.59:0/648811915 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.757+0000 7fd9daa00640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 -- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 msgr2=0x7fd9dc0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc0ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9db400640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9dc145e10 0x7fd9dc0cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 -- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9dc145e10 msgr2=0x7fd9dc0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9dc145e10 0x7fd9dc0cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 -- 172.21.15.59:0/648811915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc0ccec0 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9daa00640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d2000640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402ad00 tx=0x7fd9c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4041020 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4008790 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4036040 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.758+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc0cd160 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.759+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc0cd720 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.760+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc068a80 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.761+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9c4044020 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.762+0000 7fd9d3400640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 0x7fd9b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.762+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd9c40ac610 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.762+0000 7fd9daa00640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 0x7fd9b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:18.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.763+0000 7fd9daa00640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 0x7fd9b806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0015e0 tx=0x7fd9cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:18.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.767+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9c404b070 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.970+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9dc0d1190 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.971+0000 7fd9d3400640 1 -- 172.21.15.59:0/648811915 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fd9c407a9a0 con 0x7fd9dc0d0cc0 2024-04-24T06:30:18.972 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.973+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 msgr2=0x7fd9b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.973+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 0x7fd9b806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0015e0 tx=0x7fd9cc009040 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.973+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 msgr2=0x7fd9dc0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.973+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402ad00 tx=0x7fd9c4004a40 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 shutdown_connections 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd9b806c8f0 0x7fd9b806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9dc145e10 0x7fd9dc0cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9dc0d0cc0 0x7fd9dc0cae70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 --2- 172.21.15.59:0/648811915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9dc071d70 0x7fd9dc0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 >> 172.21.15.59:0/648811915 conn(0x7fd9dc067d90 msgr2=0x7fd9dc0c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 shutdown_connections 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:18.974+0000 7fd9e1800640 1 -- 172.21.15.59:0/648811915 wait complete. 2024-04-24T06:30:18.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:30:19.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.198+0000 7f9dcb200640 1 Processor -- start 2024-04-24T06:30:19.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.198+0000 7f9dcb200640 1 -- start start 2024-04-24T06:30:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dcb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9dc41567e0 0x7f9dc4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dcb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dc415d810 0x7f9dc40d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dcb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc415cab0 con 0x7f9dc40d0b40 2024-04-24T06:30:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dcb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc40d0f90 con 0x7f9dc415d810 2024-04-24T06:30:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dcb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc4160e40 con 0x7f9dc41567e0 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dc9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dc415d810 0x7f9dc40d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dc9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dc415d810 0x7f9dc40d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53562/0 (socket says 172.21.15.59:53562) 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dc9400640 1 -- 172.21.15.59:0/1016318051 learned_addr learned my addr 172.21.15.59:0/1016318051 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dca800640 1 --1- 172.21.15.59:0/1016318051 >> v1:172.21.15.59:6789/0 conn(0x7f9dc40d0b40 0x7f9dc4160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53500/0 (socket says 172.21.15.59:53500) 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.200+0000 7f9dc9e00640 1 --2- 172.21.15.59:0/1016318051 >> v2:172.21.15.66:3300/0 conn(0x7f9dc41567e0 0x7f9dc4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.201+0000 7f9dc9400640 1 -- 172.21.15.59:0/1016318051 >> v2:172.21.15.66:3300/0 conn(0x7f9dc41567e0 msgr2=0x7f9dc4156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.201+0000 7f9dc9400640 1 --2- 172.21.15.59:0/1016318051 >> v2:172.21.15.66:3300/0 conn(0x7f9dc41567e0 0x7f9dc4156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.201+0000 7f9dc9400640 1 -- 172.21.15.59:0/1016318051 >> v1:172.21.15.59:6789/0 conn(0x7f9dc40d0b40 legacy=0x7f9dc4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.201+0000 7f9dc9400640 1 -- 172.21.15.59:0/1016318051 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc415cd30 con 0x7f9dc415d810 2024-04-24T06:30:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.202+0000 7f9dc9400640 1 --2- 172.21.15.59:0/1016318051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc415d810 0x7f9dc40d0600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b6360 tx=0x7f9db402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53006fec3767bb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.202+0000 7f9dc8a00640 1 -- 172.21.15.59:0/1016318051 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4037070 con 0x7f9dc415d810 2024-04-24T06:30:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.202+0000 7f9dc8a00640 1 -- 172.21.15.59:0/1016318051 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4032440 con 0x7f9dc415d810 2024-04-24T06:30:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.202+0000 7f9dc8a00640 1 -- 172.21.15.59:0/1016318051 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4032810 con 0x7f9dc415d810 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 -- 172.21.15.59:0/1016318051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc415d810 msgr2=0x7f9dc40d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 --2- 172.21.15.59:0/1016318051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc415d810 0x7f9dc40d0600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b6360 tx=0x7f9db402a270 comp rx=0 tx=0).stop 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 -- 172.21.15.59:0/1016318051 shutdown_connections 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 --2- 172.21.15.59:0/1016318051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc415d810 0x7f9dc40d0600 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 --2- 172.21.15.59:0/1016318051 >> v2:172.21.15.66:3300/0 conn(0x7f9dc41567e0 0x7f9dc4156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 -- 172.21.15.59:0/1016318051 >> 172.21.15.59:0/1016318051 conn(0x7f9dc406bdf0 msgr2=0x7f9dc406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 -- 172.21.15.59:0/1016318051 shutdown_connections 2024-04-24T06:30:19.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.203+0000 7f9dcb200640 1 -- 172.21.15.59:0/1016318051 wait complete. 2024-04-24T06:30:19.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.205+0000 7f9dcb200640 1 Processor -- start 2024-04-24T06:30:19.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.205+0000 7f9dcb200640 1 -- start start 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.206+0000 7f9dcb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dc40d0b40 0x7f9dc41541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.206+0000 7f9dcb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.206+0000 7f9dcb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc415d810 0x7f9dc41501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dcb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4160820 con 0x7f9dc41567e0 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dcb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc415d2f0 con 0x7f9dc415d810 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dcb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc41609a0 con 0x7f9dc40d0b40 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53568/0 (socket says 172.21.15.59:53568) 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 -- 172.21.15.59:0/143504881 learned_addr learned my addr 172.21.15.59:0/143504881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 -- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dc40d0b40 msgr2=0x7f9dc41541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dc40d0b40 0x7f9dc41541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 -- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc415d810 msgr2=0x7f9dc41501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc415d810 0x7f9dc41501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.207+0000 7f9dc9400640 1 -- 172.21.15.59:0/143504881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc41547d0 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dc9400640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9db402a2b0 tx=0x7f9db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4041020 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db40306c0 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4037040 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc41f5ca0 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.208+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc41f6260 con 0x7f9dc41567e0 2024-04-24T06:30:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.210+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc404fd90 con 0x7f9dc41567e0 2024-04-24T06:30:19.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.211+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9db4044020 con 0x7f9dc41567e0 2024-04-24T06:30:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.212+0000 7f9dba000640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 0x7f9da806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.212+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9db40ac390 con 0x7f9dc41567e0 2024-04-24T06:30:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.212+0000 7f9dc9e00640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 0x7f9da806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.213+0000 7f9dc9e00640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 0x7f9da806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098f0 tx=0x7f9dbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.217+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9db404b030 con 0x7f9dc41567e0 2024-04-24T06:30:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.423+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9dc40b5150 con 0x7f9dc41567e0 2024-04-24T06:30:19.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.424+0000 7f9dba000640 1 -- 172.21.15.59:0/143504881 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f9db407a7b0 con 0x7f9dc41567e0 2024-04-24T06:30:19.425 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:30:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 msgr2=0x7f9da806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 0x7f9da806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098f0 tx=0x7f9dbc008040 comp rx=0 tx=0).stop 2024-04-24T06:30:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 msgr2=0x7f9dc4151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9db402a2b0 tx=0x7f9db40046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 shutdown_connections 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9da806c920 0x7f9da806ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc415d810 0x7f9dc41501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dc41567e0 0x7f9dc4151c20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 --2- 172.21.15.59:0/143504881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dc40d0b40 0x7f9dc41541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 >> 172.21.15.59:0/143504881 conn(0x7f9dc406bdf0 msgr2=0x7f9dc40af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 shutdown_connections 2024-04-24T06:30:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.427+0000 7f9dcb200640 1 -- 172.21.15.59:0/143504881 wait complete. 2024-04-24T06:30:19.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:30:19.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.665+0000 7f0e76200640 1 Processor -- start 2024-04-24T06:30:19.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.665+0000 7f0e76200640 1 -- start start 2024-04-24T06:30:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e76200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0e700d0600 0x7f0e700d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e76200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e700ae290 0x7f0e700ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e76200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7015ef20 con 0x7f0e70155a80 2024-04-24T06:30:19.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e76200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7015f1c0 con 0x7f0e700d0600 2024-04-24T06:30:19.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e76200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7015f4a0 con 0x7f0e700ae290 2024-04-24T06:30:19.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e6fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0e700d0600 0x7f0e700d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e74e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0e70155a80 0x7f0e7006f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53514/0 (socket says 172.21.15.59:53514) 2024-04-24T06:30:19.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.667+0000 7f0e74e00640 1 -- 172.21.15.59:0/3986643371 learned_addr learned my addr 172.21.15.59:0/3986643371 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.668+0000 7f0e6fe00640 1 -- 172.21.15.59:0/3986643371 >> v2:172.21.15.66:3300/0 conn(0x7f0e700ae290 msgr2=0x7f0e700ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.668+0000 7f0e75800640 1 --2- 172.21.15.59:0/3986643371 >> v2:172.21.15.66:3300/0 conn(0x7f0e700ae290 0x7f0e700ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.668+0000 7f0e6fe00640 1 --2- 172.21.15.59:0/3986643371 >> v2:172.21.15.66:3300/0 conn(0x7f0e700ae290 0x7f0e700ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.668+0000 7f0e6fe00640 1 -- 172.21.15.59:0/3986643371 >> v1:172.21.15.59:6789/0 conn(0x7f0e70155a80 legacy=0x7f0e7006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.668+0000 7f0e6fe00640 1 -- 172.21.15.59:0/3986643371 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7015f780 con 0x7f0e700d0600 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.669+0000 7f0e6fe00640 1 --2- 172.21.15.59:0/3986643371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700d0600 0x7f0e700d0a90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e7006df60 tx=0x7f0e5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a7028364cf0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.669+0000 7f0e6f400640 1 -- 172.21.15.59:0/3986643371 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c037070 con 0x7f0e700d0600 2024-04-24T06:30:19.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.669+0000 7f0e6f400640 1 -- 172.21.15.59:0/3986643371 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c032440 con 0x7f0e700d0600 2024-04-24T06:30:19.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.669+0000 7f0e6f400640 1 -- 172.21.15.59:0/3986643371 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c032810 con 0x7f0e700d0600 2024-04-24T06:30:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 -- 172.21.15.59:0/3986643371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700d0600 msgr2=0x7f0e700d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 --2- 172.21.15.59:0/3986643371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700d0600 0x7f0e700d0a90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e7006df60 tx=0x7f0e5c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 -- 172.21.15.59:0/3986643371 shutdown_connections 2024-04-24T06:30:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 --2- 172.21.15.59:0/3986643371 >> v2:172.21.15.66:3300/0 conn(0x7f0e700ae290 0x7f0e700ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 --2- 172.21.15.59:0/3986643371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700d0600 0x7f0e700d0a90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 -- 172.21.15.59:0/3986643371 >> 172.21.15.59:0/3986643371 conn(0x7f0e7006bdf0 msgr2=0x7f0e7006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 -- 172.21.15.59:0/3986643371 shutdown_connections 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.670+0000 7f0e76200640 1 -- 172.21.15.59:0/3986643371 wait complete. 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.672+0000 7f0e76200640 1 Processor -- start 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.672+0000 7f0e76200640 1 -- start start 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e700d0600 0x7f0e701542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e70155a80 0x7f0e701547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7006ef80 con 0x7f0e700ae290 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7015fd40 con 0x7f0e70155a80 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e76200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7006f100 con 0x7f0e700d0600 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e74e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e74e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53576/0 (socket says 172.21.15.59:53576) 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.673+0000 7f0e74e00640 1 -- 172.21.15.59:0/3687027363 learned_addr learned my addr 172.21.15.59:0/3687027363 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 -- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e700d0600 msgr2=0x7f0e701542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e700d0600 0x7f0e701542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 -- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e70155a80 msgr2=0x7f0e701547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e70155a80 0x7f0e701547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 -- 172.21.15.59:0/3687027363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e700ce0b0 con 0x7f0e700ae290 2024-04-24T06:30:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e74e00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3c0 tx=0x7f0e6400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64017070 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e64005ce0 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e640072f0 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e700ce390 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.674+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e700ce950 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.676+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7004fd90 con 0x7f0e700ae290 2024-04-24T06:30:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.677+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e6401a070 con 0x7f0e700ae290 2024-04-24T06:30:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.678+0000 7f0e6cc00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 0x7f0e5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.678+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0e6408af80 con 0x7f0e700ae290 2024-04-24T06:30:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.678+0000 7f0e6fe00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 0x7f0e5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.679+0000 7f0e6fe00640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 0x7f0e5006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c034000 tx=0x7f0e5c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.683+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e64059310 con 0x7f0e700ae290 2024-04-24T06:30:19.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.890+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e7006eba0 con 0x7f0e700ae290 2024-04-24T06:30:19.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.891+0000 7f0e6cc00640 1 -- 172.21.15.59:0/3687027363 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f0e64058c10 con 0x7f0e700ae290 2024-04-24T06:30:19.892 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.893+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 msgr2=0x7f0e5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 0x7f0e5006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c034000 tx=0x7f0e5c02c000 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 msgr2=0x7f0e701fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c3c0 tx=0x7f0e6400c880 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 shutdown_connections 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e5006c8f0 0x7f0e5006edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e70155a80 0x7f0e701547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e700d0600 0x7f0e701542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 --2- 172.21.15.59:0/3687027363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e700ae290 0x7f0e701fb850 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 >> 172.21.15.59:0/3687027363 conn(0x7f0e7006bdf0 msgr2=0x7f0e7006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 shutdown_connections 2024-04-24T06:30:19.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:19.894+0000 7f0e76200640 1 -- 172.21.15.59:0/3687027363 wait complete. 2024-04-24T06:30:19.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:30:20.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.122+0000 7f8586400640 1 Processor -- start 2024-04-24T06:30:20.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.123+0000 7f8586400640 1 -- start start 2024-04-24T06:30:20.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.124+0000 7f8586400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85800cd8b0 0x7f85800d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.124+0000 7f8586400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800b4cf0 con 0x7f85800d0b00 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.124+0000 7f8586400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800ccdd0 con 0x7f858015c670 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.124+0000 7f8586400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f858015ca50 con 0x7f85800cd8b0 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85800cd8b0 0x7f85800d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85800cd8b0 0x7f85800d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53590/0 (socket says 172.21.15.59:53590) 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 -- 172.21.15.59:0/2282018701 learned_addr learned my addr 172.21.15.59:0/2282018701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 -- 172.21.15.59:0/2282018701 >> v1:172.21.15.66:6789/0 conn(0x7f858015c670 legacy=0x7f85800b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 -- 172.21.15.59:0/2282018701 >> v1:172.21.15.59:6789/0 conn(0x7f85800d0b00 legacy=0x7f858006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.125+0000 7f857fe00640 1 -- 172.21.15.59:0/2282018701 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800cd0b0 con 0x7f85800cd8b0 2024-04-24T06:30:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.126+0000 7f857fe00640 1 --2- 172.21.15.59:0/2282018701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd8b0 0x7f85800d05c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f856c009870 tx=0x7f856c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ccc553f62d5b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f857f400640 1 -- 172.21.15.59:0/2282018701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c037070 con 0x7f85800cd8b0 2024-04-24T06:30:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f857f400640 1 -- 172.21.15.59:0/2282018701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856c032440 con 0x7f85800cd8b0 2024-04-24T06:30:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f857f400640 1 -- 172.21.15.59:0/2282018701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c032810 con 0x7f85800cd8b0 2024-04-24T06:30:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f8586400640 1 -- 172.21.15.59:0/2282018701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd8b0 msgr2=0x7f85800d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f8586400640 1 --2- 172.21.15.59:0/2282018701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd8b0 0x7f85800d05c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f856c009870 tx=0x7f856c02a2a0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f8586400640 1 -- 172.21.15.59:0/2282018701 shutdown_connections 2024-04-24T06:30:20.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.127+0000 7f8586400640 1 --2- 172.21.15.59:0/2282018701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd8b0 0x7f85800d05c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.128+0000 7f8586400640 1 -- 172.21.15.59:0/2282018701 >> 172.21.15.59:0/2282018701 conn(0x7f858006bdf0 msgr2=0x7f858006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:20.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.128+0000 7f8586400640 1 -- 172.21.15.59:0/2282018701 shutdown_connections 2024-04-24T06:30:20.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.128+0000 7f8586400640 1 -- 172.21.15.59:0/2282018701 wait complete. 2024-04-24T06:30:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.130+0000 7f8586400640 1 Processor -- start 2024-04-24T06:30:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.130+0000 7f8586400640 1 -- start start 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85800cd8b0 0x7f8580156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85800d0b00 0x7f85801581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85800b3e60 con 0x7f858015c670 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85800ccb80 con 0x7f85800d0b00 2024-04-24T06:30:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.131+0000 7f8586400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f85800b3fe0 con 0x7f85800cd8b0 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53600/0 (socket says 172.21.15.59:53600) 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 -- 172.21.15.59:0/2723316296 learned_addr learned my addr 172.21.15.59:0/2723316296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f857fe00640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85800d0b00 0x7f85801581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585000640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85800cd8b0 0x7f8580156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 -- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85800cd8b0 msgr2=0x7f8580156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85800cd8b0 0x7f8580156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 -- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85800d0b00 msgr2=0x7f85801581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85800d0b00 0x7f85801581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.132+0000 7f8585a00640 1 -- 172.21.15.59:0/2723316296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8580157930 con 0x7f858015c670 2024-04-24T06:30:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f8585a00640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f857800c710 tx=0x7f857800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578016020 con 0x7f858015c670 2024-04-24T06:30:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8578006d50 con 0x7f858015c670 2024-04-24T06:30:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578003bb0 con 0x7f858015c670 2024-04-24T06:30:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8580157c10 con 0x7f858015c670 2024-04-24T06:30:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.133+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85800ca9f0 con 0x7f858015c670 2024-04-24T06:30:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.134+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858004fd90 con 0x7f858015c670 2024-04-24T06:30:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.135+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f857800a850 con 0x7f858015c670 2024-04-24T06:30:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.136+0000 7f857cc00640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:20.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.136+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f857808b830 con 0x7f858015c670 2024-04-24T06:30:20.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.137+0000 7f8585000640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:20.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.137+0000 7f8585000640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 0x7f856006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8574006fd0 tx=0x7f8574008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:20.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.141+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8578020020 con 0x7f858015c670 2024-04-24T06:30:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.357+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85800ceab0 con 0x7f858015c670 2024-04-24T06:30:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.358+0000 7f857cc00640 1 -- 172.21.15.59:0/2723316296 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f8578019310 con 0x7f858015c670 2024-04-24T06:30:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:30:20.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 msgr2=0x7f856006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 0x7f856006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8574006fd0 tx=0x7f8574008040 comp rx=0 tx=0).stop 2024-04-24T06:30:20.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 msgr2=0x7f85800b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:20.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f857800c710 tx=0x7f857800cbd0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 shutdown_connections 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858015c670 0x7f85800b1af0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85800d0b00 0x7f85801581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 --2- 172.21.15.59:0/2723316296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85800cd8b0 0x7f8580156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 >> 172.21.15.59:0/2723316296 conn(0x7f858006bdf0 msgr2=0x7f858015ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 shutdown_connections 2024-04-24T06:30:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:20.361+0000 7f8586400640 1 -- 172.21.15.59:0/2723316296 wait complete. 2024-04-24T06:30:20.381 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:30:20.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:20.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:30:20.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47305]: fuse finished with error 0 and tester_r 0 2024-04-24T06:30:20.469 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:30:26.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:30:26.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:26.571 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:30:26.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.895+0000 7fab12600640 1 Processor -- start 2024-04-24T06:30:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.895+0000 7fab12600640 1 -- start start 2024-04-24T06:30:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.896+0000 7fab12600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7fab0c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.896+0000 7fab12600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fab0c0d2630 0x7fab0c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.896+0000 7fab12600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0c0cf030 con 0x7fab0c0b6530 2024-04-24T06:30:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.896+0000 7fab12600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fab0c0b7000 con 0x7fab0c0d2630 2024-04-24T06:30:26.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.896+0000 7fab12600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c0cf620 con 0x7fab0c15e7c0 2024-04-24T06:30:26.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab11c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fab0c0b6530 0x7fab0c0d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53534/0 (socket says 172.21.15.59:53534) 2024-04-24T06:30:26.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab11c00640 1 -- 172.21.15.59:0/1259719878 learned_addr learned my addr 172.21.15.59:0/1259719878 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab11200640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7fab0c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab0be00640 1 --2- 172.21.15.59:0/1259719878 >> v2:172.21.15.66:3300/0 conn(0x7fab0c0d2630 0x7fab0c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061339817 0 0) 0x7fab04001110 con 0x7fab0c0b6530 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.897+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf0003520 con 0x7fab0c0b6530 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1888820380 0 0) 0x7faaf0003520 con 0x7fab0c0b6530 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 msgr2=0x7fab0c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7fab0c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 >> v2:172.21.15.66:3300/0 conn(0x7fab0c0d2630 msgr2=0x7fab0c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 --2- 172.21.15.59:0/1259719878 >> v2:172.21.15.66:3300/0 conn(0x7fab0c0d2630 0x7fab0c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c0b6a40 con 0x7fab0c0b6530 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fab04001840 con 0x7fab0c0b6530 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 >> v1:172.21.15.59:6789/0 conn(0x7fab0c0b6530 legacy=0x7fab0c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7faaf00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7faaf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faaf0004d40 0x7faaf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0b7000 con 0x7faaf0003f80 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0cf620 con 0x7faaf0004d40 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faaf000b9c0 con 0x7fab0c15e7c0 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab0be00640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7faaf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.898+0000 7fab11200640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7faaf00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab0be00640 1 -- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 msgr2=0x7faaf00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab0be00640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c15e7c0 0x7faaf00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab11c00640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faaf0004d40 0x7faaf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab0be00640 1 -- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faaf0004d40 msgr2=0x7faaf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab0be00640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faaf0004d40 0x7faaf00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.899+0000 7fab0be00640 1 -- 172.21.15.59:0/1259719878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf0003520 con 0x7faaf0003f80 2024-04-24T06:30:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.900+0000 7fab0be00640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7faaf0004800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faafc009fc0 tx=0x7faaf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8517dbaa2e4fd134 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.900+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc009c10 con 0x7faaf0003f80 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.900+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc036070 con 0x7faaf0003f80 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.900+0000 7fab0b400640 1 -- 172.21.15.59:0/1259719878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc02d070 con 0x7faaf0003f80 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 -- 172.21.15.59:0/1259719878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 msgr2=0x7faaf0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 --2- 172.21.15.59:0/1259719878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7faaf0004800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faafc009fc0 tx=0x7faaf0005830 comp rx=0 tx=0).stop 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab11c00640 1 -- 172.21.15.59:0/1259719878 reap_dead start 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 -- 172.21.15.59:0/1259719878 shutdown_connections 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 -- 172.21.15.59:0/1259719878 >> 172.21.15.59:0/1259719878 conn(0x7fab0c09d920 msgr2=0x7fab0c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 -- 172.21.15.59:0/1259719878 shutdown_connections 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.901+0000 7fab12600640 1 -- 172.21.15.59:0/1259719878 wait complete. 2024-04-24T06:30:26.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.903+0000 7fab12600640 1 Processor -- start 2024-04-24T06:30:26.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.903+0000 7fab12600640 1 -- start start 2024-04-24T06:30:26.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c0cbf90 0x7fab0c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0cfcf0 0x7fab0c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0d1a80 con 0x7faaf0003f80 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0cf2b0 con 0x7fab0c0cfcf0 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab12600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c0d1c00 con 0x7fab0c0cbf90 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab11200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab11200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53616/0 (socket says 172.21.15.59:53616) 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab11200640 1 -- 172.21.15.59:0/2056426298 learned_addr learned my addr 172.21.15.59:0/2056426298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.904+0000 7fab0be00640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c0cbf90 0x7fab0c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 -- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c0cbf90 msgr2=0x7fab0c159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c0cbf90 0x7fab0c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 -- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0cfcf0 msgr2=0x7fab0c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:26.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0cfcf0 0x7fab0c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 -- 172.21.15.59:0/2056426298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c160f40 con 0x7faaf0003f80 2024-04-24T06:30:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab11200640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faaf4011630 tx=0x7faaf4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf400f920 con 0x7faaf0003f80 2024-04-24T06:30:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf400fac0 con 0x7faaf0003f80 2024-04-24T06:30:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf400fe70 con 0x7faaf0003f80 2024-04-24T06:30:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c161220 con 0x7faaf0003f80 2024-04-24T06:30:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.905+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1617e0 con 0x7faaf0003f80 2024-04-24T06:30:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.906+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c04fd90 con 0x7faaf0003f80 2024-04-24T06:30:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.908+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaf4015460 con 0x7faaf0003f80 2024-04-24T06:30:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.908+0000 7fab08c00640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 0x7faae806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.909+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faaf408c550 con 0x7faaf0003f80 2024-04-24T06:30:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.909+0000 7fab0be00640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 0x7faae806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:26.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.909+0000 7fab0be00640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 0x7faae806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faafc035000 tx=0x7faafc02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:26.913+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faaf4019030 con 0x7faaf0003f80 2024-04-24T06:30:27.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.156+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab0c0b3bd0 con 0x7faaf0003f80 2024-04-24T06:30:27.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.158+0000 7fab08c00640 1 -- 172.21.15.59:0/2056426298 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7faaf405a850 con 0x7faaf0003f80 2024-04-24T06:30:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:27.160 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:27.160 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 msgr2=0x7faae806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 0x7faae806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faafc035000 tx=0x7faafc02f000 comp rx=0 tx=0).stop 2024-04-24T06:30:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 msgr2=0x7fab0c160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faaf4011630 tx=0x7faaf4011af0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 shutdown_connections 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae806c920 0x7faae806ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab0c0cfcf0 0x7fab0c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab0c0cbf90 0x7fab0c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 --2- 172.21.15.59:0/2056426298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaf0003f80 0x7fab0c160880 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.160+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 >> 172.21.15.59:0/2056426298 conn(0x7fab0c09d920 msgr2=0x7fab0c162200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.161+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 shutdown_connections 2024-04-24T06:30:27.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.161+0000 7fab12600640 1 -- 172.21.15.59:0/2056426298 wait complete. 2024-04-24T06:30:27.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-24T06:30:27.448 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:30:27.449 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:30:27.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.661+0000 7ff4b3e00640 1 Processor -- start 2024-04-24T06:30:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.661+0000 7ff4b3e00640 1 -- start start 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 0x7ff4b40d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4b409ba00 0x7ff4b40d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b40cd2b0 con 0x7ff4b40b3820 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b40d06e0 con 0x7ff4b409ba00 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b40cd550 con 0x7ff4b409b110 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4b409ba00 0x7ff4b40d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4b409ba00 0x7ff4b40d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53626/0 (socket says 172.21.15.59:53626) 2024-04-24T06:30:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 -- 172.21.15.59:0/388946596 learned_addr learned my addr 172.21.15.59:0/388946596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b2000640 1 --2- 172.21.15.59:0/388946596 >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 0x7ff4b40d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 -- 172.21.15.59:0/388946596 >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 msgr2=0x7ff4b40d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 --2- 172.21.15.59:0/388946596 >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 0x7ff4b40d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 -- 172.21.15.59:0/388946596 >> v1:172.21.15.59:6789/0 conn(0x7ff4b40b3820 legacy=0x7ff4b409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.663+0000 7ff4b3400640 1 -- 172.21.15.59:0/388946596 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b40cd830 con 0x7ff4b409ba00 2024-04-24T06:30:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.664+0000 7ff4b2000640 1 --2- 172.21.15.59:0/388946596 >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 0x7ff4b40d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.665+0000 7ff4b3400640 1 --2- 172.21.15.59:0/388946596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409ba00 0x7ff4b40d0e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4163600 tx=0x7ff4ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4a0b041d0ae9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.665+0000 7ff4b1600640 1 -- 172.21.15.59:0/388946596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac036070 con 0x7ff4b409ba00 2024-04-24T06:30:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.665+0000 7ff4b1600640 1 -- 172.21.15.59:0/388946596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac031600 con 0x7ff4b409ba00 2024-04-24T06:30:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.665+0000 7ff4b1600640 1 -- 172.21.15.59:0/388946596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0319d0 con 0x7ff4b409ba00 2024-04-24T06:30:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 -- 172.21.15.59:0/388946596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409ba00 msgr2=0x7ff4b40d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/388946596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409ba00 0x7ff4b40d0e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4163600 tx=0x7ff4ac02a6a0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 -- 172.21.15.59:0/388946596 shutdown_connections 2024-04-24T06:30:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/388946596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409ba00 0x7ff4b40d0e00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/388946596 >> v2:172.21.15.66:3300/0 conn(0x7ff4b409b110 0x7ff4b40d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 -- 172.21.15.59:0/388946596 >> 172.21.15.59:0/388946596 conn(0x7ff4b40b3ef0 msgr2=0x7ff4b40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 -- 172.21.15.59:0/388946596 shutdown_connections 2024-04-24T06:30:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.666+0000 7ff4b3e00640 1 -- 172.21.15.59:0/388946596 wait complete. 2024-04-24T06:30:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.668+0000 7ff4b3e00640 1 Processor -- start 2024-04-24T06:30:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.668+0000 7ff4b3e00640 1 -- start start 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4b409ba00 0x7ff4b40c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4b40b3820 0x7ff4b415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b40ce0d0 con 0x7ff4b409b110 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b40cddf0 con 0x7ff4b409ba00 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b40ce5f0 con 0x7ff4b40b3820 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53634/0 (socket says 172.21.15.59:53634) 2024-04-24T06:30:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.669+0000 7ff4b2a00640 1 -- 172.21.15.59:0/974118965 learned_addr learned my addr 172.21.15.59:0/974118965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 -- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4b40b3820 msgr2=0x7ff4b415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4b40b3820 0x7ff4b415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2000640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4b409ba00 0x7ff4b40c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 -- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4b409ba00 msgr2=0x7ff4b40c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4b409ba00 0x7ff4b40c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 -- 172.21.15.59:0/974118965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b40b5f70 con 0x7ff4b409b110 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b2a00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c3c0 tx=0x7ff4a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40052d0 con 0x7ff4b409b110 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a4005490 con 0x7ff4b409b110 2024-04-24T06:30:27.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4007260 con 0x7ff4b409b110 2024-04-24T06:30:27.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b40b6210 con 0x7ff4b409b110 2024-04-24T06:30:27.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.670+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b40c97b0 con 0x7ff4b409b110 2024-04-24T06:30:27.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.672+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b404fd90 con 0x7ff4b409b110 2024-04-24T06:30:27.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.673+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4a4007400 con 0x7ff4b409b110 2024-04-24T06:30:27.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.673+0000 7ff4aaa00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 0x7ff49406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:27.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.674+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a408b0f0 con 0x7ff4b409b110 2024-04-24T06:30:27.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.674+0000 7ff4b2000640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 0x7ff49406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:27.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.675+0000 7ff4b2000640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 0x7ff49406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40694c0 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:27.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.678+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4a4059480 con 0x7ff4b409b110 2024-04-24T06:30:27.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.921+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4b40b4cb0 con 0x7ff4b409b110 2024-04-24T06:30:27.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.923+0000 7ff4aaa00640 1 -- 172.21.15.59:0/974118965 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7ff4a4058d80 con 0x7ff4b409b110 2024-04-24T06:30:27.925 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:27.925 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:27.925 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:27.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 msgr2=0x7ff49406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 0x7ff49406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40694c0 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2024-04-24T06:30:27.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 msgr2=0x7ff4b40b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:27.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c3c0 tx=0x7ff4a400c880 comp rx=0 tx=0).stop 2024-04-24T06:30:27.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 shutdown_connections 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff49406c8f0 0x7ff49406edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4b40b3820 0x7ff4b415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4b409ba00 0x7ff4b40c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 --2- 172.21.15.59:0/974118965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4b409b110 0x7ff4b40b5a30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 >> 172.21.15.59:0/974118965 conn(0x7ff4b40b3ef0 msgr2=0x7ff4b40ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 shutdown_connections 2024-04-24T06:30:27.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:27.925+0000 7ff4b3e00640 1 -- 172.21.15.59:0/974118965 wait complete. 2024-04-24T06:30:27.945 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2024-04-24T06:30:28.193 INFO:teuthology.orchestra.run.smithi066.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-24T06:30:28.194 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-24T06:30:35.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:30:35.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:30:38.195 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-24T06:30:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.314+0000 7fa3c64d1f40 1 Processor -- start 2024-04-24T06:30:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.314+0000 7fa3c64d1f40 1 -- start start 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a8c750 0x561515b115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561515a88590 con 0x561515a86160 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5615159e8cc0 con 0x561515b12cd0 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.315+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561515a88830 con 0x561515a8c750 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52538/0 (socket says 172.21.15.59:52538) 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 -- 172.21.15.59:0/1463372016 learned_addr learned my addr 172.21.15.59:0/1463372016 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3bac00640 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a8c750 0x561515b115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 -- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a8c750 msgr2=0x561515b115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a8c750 0x561515b115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c4000640 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 -- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 msgr2=0x561515b11b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b11b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.316+0000 7fa3c3600640 1 -- 172.21.15.59:0/1463372016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561515b113b0 con 0x561515a86160 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c3600640 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400a7b0 tx=0x7fa3b400ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747ebcac07304d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c2200640 1 -- 172.21.15.59:0/1463372016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4016020 con 0x561515a86160 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c2200640 1 -- 172.21.15.59:0/1463372016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b400d960 con 0x561515a86160 2024-04-24T06:30:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c2200640 1 -- 172.21.15.59:0/1463372016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400dd30 con 0x561515a86160 2024-04-24T06:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/1463372016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 msgr2=0x561515b0d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.317+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400a7b0 tx=0x7fa3b400ac70 comp rx=0 tx=0).stop 2024-04-24T06:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/1463372016 shutdown_connections 2024-04-24T06:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b11b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a8c750 0x561515b115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/1463372016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a86160 0x561515b0d0c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/1463372016 >> 172.21.15.59:0/1463372016 conn(0x561515a8d290 msgr2=0x561515a8bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/1463372016 shutdown_connections 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.318+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/1463372016 wait complete. 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.319+0000 7fa3c64d1f40 1 Processor -- start 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.319+0000 7fa3c64d1f40 1 -- start start 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a86160 0x561515b0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 0x561515a40460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561515b190f0 con 0x561515a8c750 2024-04-24T06:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561515b0ace0 con 0x561515b12cd0 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3c64d1f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561515a8b840 con 0x561515a86160 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3bac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 0x561515a40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3bac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 0x561515a40460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52544/0 (socket says 172.21.15.59:52544) 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.320+0000 7fa3bac00640 1 -- 172.21.15.59:0/4000437148 learned_addr learned my addr 172.21.15.59:0/4000437148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 -- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a86160 msgr2=0x561515b0c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c3600640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a86160 0x561515b0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c4000640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561515a86160 0x561515b0c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 -- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 msgr2=0x561515b074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561515b12cd0 0x561515b074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 -- 172.21.15.59:0/4000437148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561515b1bc60 con 0x561515a8c750 2024-04-24T06:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3bac00640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 0x561515a40460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00cde0 tx=0x7fa3ac00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c0e00640 1 -- 172.21.15.59:0/4000437148 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00f620 con 0x561515a8c750 2024-04-24T06:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c0e00640 1 -- 172.21.15.59:0/4000437148 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ac006070 con 0x561515a8c750 2024-04-24T06:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c0e00640 1 -- 172.21.15.59:0/4000437148 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00f9e0 con 0x561515a8c750 2024-04-24T06:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.321+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561515b0b910 con 0x561515a8c750 2024-04-24T06:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.322+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561515b0bc50 con 0x561515a8c750 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.324+0000 7fa3c0e00640 1 -- 172.21.15.59:0/4000437148 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3ac00fb80 con 0x561515a8c750 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.324+0000 7fa3c0e00640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3a006b2d0 0x7fa3a006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.325+0000 7fa3c0e00640 1 -- 172.21.15.59:0/4000437148 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3ac08aa90 con 0x561515a8c750 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.325+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x561515a409a0 0x561515a421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.325+0000 7fa3c3600640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3a006b2d0 0x7fa3a006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.325+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x561515b0cc10 con 0x561515a409a0 2024-04-24T06:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.325+0000 7fa3c4000640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x561515a409a0 0x561515a421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.326+0000 7fa3c3600640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3a006b2d0 0x7fa3a006d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000f80 tx=0x7fa3b4007030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.326+0000 7fa3c4000640 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x561515a409a0 0x561515a421f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.326+0000 7fa3c4000640 1 -- 172.21.15.59:0/4000437148 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa3bc00a0c0 con 0x561515a409a0 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.326+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x561515a409a0 msgr2=0x561515a421f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.326+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x561515a409a0 0x561515a421f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3a006b2d0 msgr2=0x7fa3a006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3a006b2d0 0x7fa3a006d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000f80 tx=0x7fa3b4007030 comp rx=0 tx=0).stop 2024-04-24T06:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 msgr2=0x561515a40460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 --2- 172.21.15.59:0/4000437148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561515a8c750 0x561515a40460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00cde0 tx=0x7fa3ac00a5f0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c4000640 1 -- 172.21.15.59:0/4000437148 reap_dead start 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 shutdown_connections 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 >> 172.21.15.59:0/4000437148 conn(0x561515a8d290 msgr2=0x561515b11580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 shutdown_connections 2024-04-24T06:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.327+0000 7fa3c64d1f40 1 -- 172.21.15.59:0/4000437148 wait complete. 2024-04-24T06:30:38.334 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-24T06:30:38.334 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-24T06:30:38.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.424+0000 7f60bc1e9f40 1 Processor -- start 2024-04-24T06:30:38.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.424+0000 7f60bc1e9f40 1 -- start start 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17e32ac0 0x563a17e4e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e4dad0 0x563a17e4ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a17d759c0 con 0x563a17d731d0 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563a17d1ccc0 con 0x563a17e32ac0 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x563a17d73b20 con 0x563a17e4dad0 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52558/0 (socket says 172.21.15.59:52558) 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.425+0000 7f60b9400640 1 -- 172.21.15.59:0/1414562373 learned_addr learned my addr 172.21.15.59:0/1414562373 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 -- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e4dad0 msgr2=0x563a17e4ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e4dad0 0x563a17e4ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 -- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17e32ac0 msgr2=0x563a17e4e9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T06:30:38.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17e32ac0 0x563a17e4e9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 -- 172.21.15.59:0/1414562373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a17e49af0 con 0x563a17d731d0 2024-04-24T06:30:38.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b9400640 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00a7e0 tx=0x7f60ac005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce070ee72b6fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b3400640 1 -- 172.21.15.59:0/1414562373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac017020 con 0x563a17d731d0 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b3400640 1 -- 172.21.15.59:0/1414562373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac00d9c0 con 0x563a17d731d0 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.426+0000 7f60b3400640 1 -- 172.21.15.59:0/1414562373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00dd90 con 0x563a17d731d0 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/1414562373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 msgr2=0x563a17dc1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00a7e0 tx=0x7f60ac005d80 comp rx=0 tx=0).stop 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/1414562373 shutdown_connections 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e4dad0 0x563a17e4ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17e32ac0 0x563a17e4e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/1414562373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17d731d0 0x563a17dc1d50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/1414562373 >> 172.21.15.59:0/1414562373 conn(0x563a17dbfcc0 msgr2=0x563a17da2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/1414562373 shutdown_connections 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.427+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/1414562373 wait complete. 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.428+0000 7f60bc1e9f40 1 Processor -- start 2024-04-24T06:30:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.428+0000 7f60bc1e9f40 1 -- start start 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17d731d0 0x563a17ee3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e32ac0 0x563a17e47a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 0x563a17e2e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a17e4e230 con 0x563a17e4dad0 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563a17da4790 con 0x563a17d731d0 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60bc1e9f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x563a17da44b0 con 0x563a17e32ac0 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 0x563a17e2e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 0x563a17e2e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52562/0 (socket says 172.21.15.59:52562) 2024-04-24T06:30:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 -- 172.21.15.59:0/224251253 learned_addr learned my addr 172.21.15.59:0/224251253 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b8a00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e32ac0 0x563a17e47a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9400640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17d731d0 0x563a17ee3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 -- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e32ac0 msgr2=0x563a17e47a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e32ac0 0x563a17e47a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.429+0000 7f60b9e00640 1 -- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17d731d0 msgr2=0x563a17ee3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b9e00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a17d731d0 0x563a17ee3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b9e00640 1 -- 172.21.15.59:0/224251253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a17ee3f50 con 0x563a17e4dad0 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b8a00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563a17e32ac0 0x563a17e47a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b9e00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 0x563a17e2e860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f60b400a5a0 tx=0x7f60b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b2000640 1 -- 172.21.15.59:0/224251253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4053780 con 0x563a17e4dad0 2024-04-24T06:30:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b2000640 1 -- 172.21.15.59:0/224251253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b4053920 con 0x563a17e4dad0 2024-04-24T06:30:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60b2000640 1 -- 172.21.15.59:0/224251253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4053cf0 con 0x563a17e4dad0 2024-04-24T06:30:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a17e47fd0 con 0x563a17e4dad0 2024-04-24T06:30:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.430+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a17dc07b0 con 0x563a17e4dad0 2024-04-24T06:30:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.432+0000 7f60b2000640 1 -- 172.21.15.59:0/224251253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60b405d410 con 0x563a17e4dad0 2024-04-24T06:30:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.433+0000 7f60b2000640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806b300 0x7f609806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.433+0000 7f60b2000640 1 -- 172.21.15.59:0/224251253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f60b40cef80 con 0x563a17e4dad0 2024-04-24T06:30:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.434+0000 7f60b9400640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806b300 0x7f609806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.434+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x563a17e43c40 0x563a17e44050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.434+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563a17e44590 con 0x563a17e43c40 2024-04-24T06:30:38.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.434+0000 7f60b8a00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x563a17e43c40 0x563a17e44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.434+0000 7f60b9400640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806b300 0x7f609806d7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00a7e0 tx=0x7f60ac012000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60b8a00640 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x563a17e43c40 0x563a17e44050 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60b8a00640 1 -- 172.21.15.59:0/224251253 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f60a400e330 con 0x563a17e43c40 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x563a17e43c40 msgr2=0x563a17e44050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x563a17e43c40 0x563a17e44050 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806b300 msgr2=0x7f609806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806b300 0x7f609806d7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00a7e0 tx=0x7f60ac012000 comp rx=0 tx=0).stop 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 msgr2=0x563a17e2e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60bc1e9f40 1 --2- 172.21.15.59:0/224251253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a17e4dad0 0x563a17e2e860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f60b400a5a0 tx=0x7f60b4002fb0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.435+0000 7f60b9e00640 1 -- 172.21.15.59:0/224251253 reap_dead start 2024-04-24T06:30:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.436+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 shutdown_connections 2024-04-24T06:30:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.436+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 >> 172.21.15.59:0/224251253 conn(0x563a17dbfcc0 msgr2=0x563a17e35d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.436+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 shutdown_connections 2024-04-24T06:30:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.436+0000 7f60bc1e9f40 1 -- 172.21.15.59:0/224251253 wait complete. 2024-04-24T06:30:38.442 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-24T06:30:38.443 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-24T06:30:38.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.533+0000 7fc36b937f40 1 Processor -- start 2024-04-24T06:30:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.533+0000 7fc36b937f40 1 -- start start 2024-04-24T06:30:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f4474da10 0x559f4474de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559f446aa200 0x559f446aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559f44620cc0 con 0x7fc34c0022a0 2024-04-24T06:30:38.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559f44624d00 con 0x559f4474da10 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.534+0000 7fc36b937f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559f4474f720 con 0x559f446aa200 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52582/0 (socket says 172.21.15.59:52582) 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 -- 172.21.15.59:0/795101007 learned_addr learned my addr 172.21.15.59:0/795101007 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 -- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559f446aa200 msgr2=0x559f446aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc369600640 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559f446aa200 0x559f446aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559f446aa200 0x559f446aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 -- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f4474da10 msgr2=0x559f4474de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f4474da10 0x559f4474de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.535+0000 7fc368c00640 1 -- 172.21.15.59:0/795101007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f446a74b0 con 0x7fc34c0022a0 2024-04-24T06:30:38.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc368c00640 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc358025e40 tx=0x7fc35804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd83339cab59b617 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc363400640 1 -- 172.21.15.59:0/795101007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358005050 con 0x7fc34c0022a0 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc363400640 1 -- 172.21.15.59:0/795101007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358005210 con 0x7fc34c0022a0 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc363400640 1 -- 172.21.15.59:0/795101007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35804d910 con 0x7fc34c0022a0 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc36b937f40 1 -- 172.21.15.59:0/795101007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 msgr2=0x559f4474d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.536+0000 7fc36b937f40 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc358025e40 tx=0x7fc35804f000 comp rx=0 tx=0).stop 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 -- 172.21.15.59:0/795101007 shutdown_connections 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559f446aa200 0x559f446aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f4474da10 0x559f4474de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 --2- 172.21.15.59:0/795101007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0022a0 0x559f4474d4d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 -- 172.21.15.59:0/795101007 >> 172.21.15.59:0/795101007 conn(0x559f446c3cc0 msgr2=0x559f446a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 -- 172.21.15.59:0/795101007 shutdown_connections 2024-04-24T06:30:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 -- 172.21.15.59:0/795101007 wait complete. 2024-04-24T06:30:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.537+0000 7fc36b937f40 1 Processor -- start 2024-04-24T06:30:38.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.538+0000 7fc36b937f40 1 -- start start 2024-04-24T06:30:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.538+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f446aa200 0x559f4474cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.538+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 0x559f4467b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.538+0000 7fc36b937f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc34c0022a0 0x559f4474e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc36b937f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559f446a7730 con 0x559f4474da10 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc36b937f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559f446c4f70 con 0x559f446aa200 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc36b937f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559f44755c60 con 0x7fc34c0022a0 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 0x559f4467b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 0x559f4467b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52596/0 (socket says 172.21.15.59:52596) 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 -- 172.21.15.59:0/3755865550 learned_addr learned my addr 172.21.15.59:0/3755865550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 -- 172.21.15.59:0/3755865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc34c0022a0 msgr2=0x559f4474e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc34c0022a0 0x559f4474e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 -- 172.21.15.59:0/3755865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f446aa200 msgr2=0x559f4474cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559f446aa200 0x559f4474cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 -- 172.21.15.59:0/3755865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f4474d160 con 0x559f4474da10 2024-04-24T06:30:38.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.539+0000 7fc360200640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 0x559f4467b750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc35000bb40 tx=0x7fc350009160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.540+0000 7fc362000640 1 -- 172.21.15.59:0/3755865550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350013070 con 0x559f4474da10 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.540+0000 7fc362000640 1 -- 172.21.15.59:0/3755865550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc350009dc0 con 0x559f4474da10 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.540+0000 7fc362000640 1 -- 172.21.15.59:0/3755865550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35000f640 con 0x559f4474da10 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.540+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f44757860 con 0x559f4474da10 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.540+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f44750b90 con 0x559f4474da10 2024-04-24T06:30:38.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.542+0000 7fc362000640 1 -- 172.21.15.59:0/3755865550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc350016070 con 0x559f4474da10 2024-04-24T06:30:38.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.543+0000 7fc362000640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc34406b2d0 0x7fc34406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.543+0000 7fc362000640 1 -- 172.21.15.59:0/3755865550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc35008a180 con 0x559f4474da10 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.543+0000 7fc368c00640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc34406b2d0 0x7fc34406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.543+0000 7fc36b937f40 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x559f4467bc90 0x559f44751390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.543+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559f4474e560 con 0x559f4467bc90 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.544+0000 7fc369600640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x559f4467bc90 0x559f44751390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.544+0000 7fc368c00640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc34406b2d0 0x7fc34406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x559f446a7d30 tx=0x7fc358007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.544+0000 7fc369600640 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x559f4467bc90 0x559f44751390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.544+0000 7fc369600640 1 -- 172.21.15.59:0/3755865550 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fc36400e360 con 0x559f4467bc90 2024-04-24T06:30:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x559f4467bc90 msgr2=0x559f44751390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x559f4467bc90 0x559f44751390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc34406b2d0 msgr2=0x7fc34406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc34406b2d0 0x7fc34406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x559f446a7d30 tx=0x7fc358007040 comp rx=0 tx=0).stop 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 msgr2=0x559f4467b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 --2- 172.21.15.59:0/3755865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559f4474da10 0x559f4467b750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc35000bb40 tx=0x7fc350009160 comp rx=0 tx=0).stop 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc369600640 1 -- 172.21.15.59:0/3755865550 reap_dead start 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 shutdown_connections 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 >> 172.21.15.59:0/3755865550 conn(0x559f446c3cc0 msgr2=0x559f446a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 shutdown_connections 2024-04-24T06:30:38.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.545+0000 7fc36b937f40 1 -- 172.21.15.59:0/3755865550 wait complete. 2024-04-24T06:30:38.551 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-24T06:30:38.551 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-24T06:30:38.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.643+0000 7fd31e423f40 1 Processor -- start 2024-04-24T06:30:38.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.643+0000 7fd31e423f40 1 -- start start 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfdc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cfd5850 0x55db0cfdcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55db0cfdf5d0 con 0x55db0cf39330 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db0cf54750 con 0x55db0cfd5470 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31e423f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55db0cfdf750 con 0x55db0cfd5850 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52626/0 (socket says 172.21.15.59:52626) 2024-04-24T06:30:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.644+0000 7fd31b600640 1 -- 172.21.15.59:0/2199226860 learned_addr learned my addr 172.21.15.59:0/2199226860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31c000640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cfd5850 0x55db0cfdcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 -- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cfd5850 msgr2=0x55db0cfdcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd312c00640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfdc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cfd5850 0x55db0cfdcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 -- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 msgr2=0x55db0cfdc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfdc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 -- 172.21.15.59:0/2199226860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db0ce34b30 con 0x55db0cf39330 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd312c00640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfdc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31b600640 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd30c045ed0 tx=0x7fd30c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7e915d34fc62a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31a200640 1 -- 172.21.15.59:0/2199226860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c062070 con 0x55db0cf39330 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31a200640 1 -- 172.21.15.59:0/2199226860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c054970 con 0x55db0cf39330 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.645+0000 7fd31a200640 1 -- 172.21.15.59:0/2199226860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c054d60 con 0x55db0cf39330 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 -- 172.21.15.59:0/2199226860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 msgr2=0x55db0cfd0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd30c045ed0 tx=0x7fd30c051000 comp rx=0 tx=0).stop 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 -- 172.21.15.59:0/2199226860 shutdown_connections 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cfd5850 0x55db0cfdcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfdc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 --2- 172.21.15.59:0/2199226860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 -- 172.21.15.59:0/2199226860 >> 172.21.15.59:0/2199226860 conn(0x55db0cf55290 msgr2=0x55db0cf38340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 -- 172.21.15.59:0/2199226860 shutdown_connections 2024-04-24T06:30:38.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.646+0000 7fd31e423f40 1 -- 172.21.15.59:0/2199226860 wait complete. 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.647+0000 7fd31e423f40 1 Processor -- start 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.647+0000 7fd31e423f40 1 -- start start 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 0x55db0cfdb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55db0cfe30f0 con 0x55db0cfd5850 2024-04-24T06:30:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db0cfd44f0 con 0x55db0cfd5470 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.648+0000 7fd31e423f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55db0cfe2e50 con 0x55db0cf39330 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 0x55db0cfdb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 0x55db0cfdb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52628/0 (socket says 172.21.15.59:52628) 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 -- 172.21.15.59:0/3380607439 learned_addr learned my addr 172.21.15.59:0/3380607439 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31b600640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd312c00640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 -- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cf39330 msgr2=0x55db0cfd0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55db0cf39330 0x55db0cfd0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 -- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 msgr2=0x55db0cfca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 -- 172.21.15.59:0/3380607439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db0cf09c20 con 0x55db0cfd5850 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd312c00640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db0cfd5470 0x55db0cfca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.649+0000 7fd31c000640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 0x55db0cfdb1c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd314005f50 tx=0x7fd3140085b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.650+0000 7fd318e00640 1 -- 172.21.15.59:0/3380607439 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31400b070 con 0x55db0cfd5850 2024-04-24T06:30:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.650+0000 7fd318e00640 1 -- 172.21.15.59:0/3380607439 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd314017070 con 0x55db0cfd5850 2024-04-24T06:30:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.650+0000 7fd318e00640 1 -- 172.21.15.59:0/3380607439 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31400e040 con 0x55db0cfd5850 2024-04-24T06:30:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.650+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db0cf09ec0 con 0x55db0cfd5850 2024-04-24T06:30:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.650+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db0cfdaca0 con 0x55db0cfd5850 2024-04-24T06:30:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.652+0000 7fd318e00640 1 -- 172.21.15.59:0/3380607439 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd314012650 con 0x55db0cfd5850 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.653+0000 7fd318e00640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd2f806b2d0 0x7fd2f806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.653+0000 7fd318e00640 1 -- 172.21.15.59:0/3380607439 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd31408a420 con 0x55db0cfd5850 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.653+0000 7fd31b600640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd2f806b2d0 0x7fd2f806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.653+0000 7fd31e423f40 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55db0cf082e0 0x55db0cfcaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.653+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55db0cfdb700 con 0x55db0cf082e0 2024-04-24T06:30:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.654+0000 7fd312c00640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55db0cf082e0 0x55db0cfcaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.654+0000 7fd31b600640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd2f806b2d0 0x7fd2f806d790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd30c066000 tx=0x7fd30c053000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.654+0000 7fd312c00640 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55db0cf082e0 0x55db0cfcaf20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd312c00640 1 -- 172.21.15.59:0/3380607439 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fd304007730 con 0x55db0cf082e0 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55db0cf082e0 msgr2=0x55db0cfcaf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55db0cf082e0 0x55db0cfcaf20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd2f806b2d0 msgr2=0x7fd2f806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd2f806b2d0 0x7fd2f806d790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd30c066000 tx=0x7fd30c053000 comp rx=0 tx=0).stop 2024-04-24T06:30:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 msgr2=0x55db0cfdb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 --2- 172.21.15.59:0/3380607439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55db0cfd5850 0x55db0cfdb1c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd314005f50 tx=0x7fd3140085b0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31c000640 1 -- 172.21.15.59:0/3380607439 reap_dead start 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 shutdown_connections 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 >> 172.21.15.59:0/3380607439 conn(0x55db0cf55290 msgr2=0x55db0cfdd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 shutdown_connections 2024-04-24T06:30:38.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.655+0000 7fd31e423f40 1 -- 172.21.15.59:0/3380607439 wait complete. 2024-04-24T06:30:38.661 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-24T06:30:38.662 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-24T06:30:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.760+0000 7f9d778f2f40 1 Processor -- start 2024-04-24T06:30:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.760+0000 7f9d778f2f40 1 -- start start 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 0x5566b9918e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 0x5566b98ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566b991b740 con 0x5566b99c6bb0 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5566b9892cc0 con 0x5566b99c7370 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5566b99c8ac0 con 0x5566b991adf0 2024-04-24T06:30:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d6c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d6c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52652/0 (socket says 172.21.15.59:52652) 2024-04-24T06:30:38.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.761+0000 7f9d6c000640 1 -- 172.21.15.59:0/859278251 learned_addr learned my addr 172.21.15.59:0/859278251 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d74a00640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 0x5566b9918e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 -- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 msgr2=0x5566b9918e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d75400640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 0x5566b98ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 0x5566b9918e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 -- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 msgr2=0x5566b98ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 0x5566b98ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 -- 172.21.15.59:0/859278251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b9936490 con 0x5566b99c6bb0 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d74a00640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 0x5566b9918e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d75400640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 0x5566b98ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6c000640 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009760 tx=0x7f9d5c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54fed810fa71595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.762+0000 7f9d6f400640 1 -- 172.21.15.59:0/859278251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c017070 con 0x5566b99c6bb0 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d6f400640 1 -- 172.21.15.59:0/859278251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0090d0 con 0x5566b99c6bb0 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d6f400640 1 -- 172.21.15.59:0/859278251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c006b60 con 0x5566b99c6bb0 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 -- 172.21.15.59:0/859278251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 msgr2=0x5566b99c0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009760 tx=0x7f9d5c009c20 comp rx=0 tx=0).stop 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 -- 172.21.15.59:0/859278251 shutdown_connections 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c7370 0x5566b98ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b99c6bb0 0x5566b99c0490 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/859278251 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b991adf0 0x5566b9918e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 -- 172.21.15.59:0/859278251 >> 172.21.15.59:0/859278251 conn(0x5566b9937290 msgr2=0x5566b9916920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 -- 172.21.15.59:0/859278251 shutdown_connections 2024-04-24T06:30:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.763+0000 7f9d778f2f40 1 -- 172.21.15.59:0/859278251 wait complete. 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.764+0000 7f9d778f2f40 1 Processor -- start 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.764+0000 7f9d778f2f40 1 -- start start 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 0x5566b99a9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c6bb0 0x5566b99c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b99c7370 0x5566b99c0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566b98ed170 con 0x5566b991adf0 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5566b99c3c00 con 0x5566b99c6bb0 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d778f2f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5566b98ed450 con 0x5566b99c7370 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d74a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 0x5566b99a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d74a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 0x5566b99a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52654/0 (socket says 172.21.15.59:52654) 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.765+0000 7f9d74a00640 1 -- 172.21.15.59:0/2032452219 learned_addr learned my addr 172.21.15.59:0/2032452219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d75400640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b99c7370 0x5566b99c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d6c000640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c6bb0 0x5566b99c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 -- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b99c7370 msgr2=0x5566b99c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566b99c7370 0x5566b99c0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 -- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c6bb0 msgr2=0x5566b99c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c6bb0 0x5566b99c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 -- 172.21.15.59:0/2032452219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b98ee9e0 con 0x5566b991adf0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d6c000640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566b99c6bb0 0x5566b99c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d74a00640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 0x5566b99a9090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9d64001eb0 tx=0x7f9d6400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.766+0000 7f9d6e000640 1 -- 172.21.15.59:0/2032452219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400f420 con 0x5566b991adf0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.767+0000 7f9d6e000640 1 -- 172.21.15.59:0/2032452219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d64014020 con 0x5566b991adf0 2024-04-24T06:30:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.767+0000 7f9d6e000640 1 -- 172.21.15.59:0/2032452219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400f7e0 con 0x5566b991adf0 2024-04-24T06:30:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.767+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566b9917c90 con 0x5566b991adf0 2024-04-24T06:30:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.767+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b99173f0 con 0x5566b991adf0 2024-04-24T06:30:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.769+0000 7f9d6e000640 1 -- 172.21.15.59:0/2032452219 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d6400f980 con 0x5566b991adf0 2024-04-24T06:30:38.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d6e000640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d5006b2d0 0x7f9d5006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d6e000640 1 -- 172.21.15.59:0/2032452219 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9d6408add0 con 0x5566b991adf0 2024-04-24T06:30:38.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d6c000640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d5006b2d0 0x7f9d5006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566b99c2270 0x5566b99ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:38.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5566b98ee0d0 con 0x5566b99c2270 2024-04-24T06:30:38.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.770+0000 7f9d75400640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566b99c2270 0x5566b99ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:38.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.771+0000 7f9d6c000640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d5006b2d0 0x7f9d5006d790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009c60 tx=0x7f9d5c00f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.771+0000 7f9d75400640 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566b99c2270 0x5566b99ad740 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d75400640 1 -- 172.21.15.59:0/2032452219 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f9d7004f5c0 con 0x5566b99c2270 2024-04-24T06:30:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-24T06:30:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566b99c2270 msgr2=0x5566b99ad740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566b99c2270 0x5566b99ad740 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d5006b2d0 msgr2=0x7f9d5006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d5006b2d0 0x7f9d5006d790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009c60 tx=0x7f9d5c00f000 comp rx=0 tx=0).stop 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 msgr2=0x5566b99a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 --2- 172.21.15.59:0/2032452219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566b991adf0 0x5566b99a9090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9d64001eb0 tx=0x7f9d6400a5f0 comp rx=0 tx=0).stop 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d75400640 1 -- 172.21.15.59:0/2032452219 reap_dead start 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 shutdown_connections 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 >> 172.21.15.59:0/2032452219 conn(0x5566b9937290 msgr2=0x5566b99bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 shutdown_connections 2024-04-24T06:30:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:38.772+0000 7f9d778f2f40 1 -- 172.21.15.59:0/2032452219 wait complete. 2024-04-24T06:30:38.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:30:38.779 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:30:38.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:30:38.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:30:39.001 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:30:39.001 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:30:39.002 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:39.002 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:39.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:30:39.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:30:39.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:39.064 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:30:39.120 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-24T06:30:39.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:30:39.122 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:30:39.122 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:30:39.122 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:39.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:30:39.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:30:39.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:39.183 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:30:39.244 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:30:39.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:30:39.245 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:30:39.245 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:39.245 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:39.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:30:39.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:30:39.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:39.307 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:30:39.363 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:30:39.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:30:39.364 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:30:39.365 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:30:39.365 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:39.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:30:39.427 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:30:39.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:30:39.428 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:30:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:30:39.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:30:39.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.701+0000 7fbd5e000640 1 Processor -- start 2024-04-24T06:30:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.701+0000 7fbd5e000640 1 -- start start 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd5815c3d0 0x7fbd580cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd580cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5e000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd58160210 con 0x7fbd580cd590 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5e000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbd581604f0 con 0x7fbd5815c3d0 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd581607d0 con 0x7fbd580b4860 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd5815c3d0 0x7fbd580cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd57e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbd580cd590 0x7fbd580b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45442/0 (socket says 172.21.15.59:45442) 2024-04-24T06:30:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd5cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd5815c3d0 0x7fbd580cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59578/0 (socket says 172.21.15.59:59578) 2024-04-24T06:30:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.703+0000 7fbd57e00640 1 -- 172.21.15.59:0/299567324 learned_addr learned my addr 172.21.15.59:0/299567324 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5cc00640 1 -- 172.21.15.59:0/299567324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 msgr2=0x7fbd580cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5d600640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd580cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd57400640 1 -- 172.21.15.59:0/299567324 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291769265 0 0) 0x7fbd58160210 con 0x7fbd580cd590 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5cc00640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd580cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5cc00640 1 -- 172.21.15.59:0/299567324 >> v1:172.21.15.59:6789/0 conn(0x7fbd580cd590 legacy=0x7fbd580b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5cc00640 1 -- 172.21.15.59:0/299567324 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd58160ab0 con 0x7fbd5815c3d0 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.704+0000 7fbd5d600640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd580cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.705+0000 7fbd5cc00640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd5815c3d0 0x7fbd580cd050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0097a0 tx=0x7fbd4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22693cddccc398a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.706+0000 7fbd57400640 1 -- 172.21.15.59:0/299567324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c037070 con 0x7fbd5815c3d0 2024-04-24T06:30:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.706+0000 7fbd57400640 1 -- 172.21.15.59:0/299567324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4c032440 con 0x7fbd5815c3d0 2024-04-24T06:30:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.706+0000 7fbd57400640 1 -- 172.21.15.59:0/299567324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c032810 con 0x7fbd5815c3d0 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 -- 172.21.15.59:0/299567324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd5815c3d0 msgr2=0x7fbd580cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd5815c3d0 0x7fbd580cd050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0097a0 tx=0x7fbd4c02a180 comp rx=0 tx=0).stop 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 -- 172.21.15.59:0/299567324 shutdown_connections 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd580cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 --2- 172.21.15.59:0/299567324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd5815c3d0 0x7fbd580cd050 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 -- 172.21.15.59:0/299567324 >> 172.21.15.59:0/299567324 conn(0x7fbd580b21a0 msgr2=0x7fbd580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 -- 172.21.15.59:0/299567324 shutdown_connections 2024-04-24T06:30:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.707+0000 7fbd5e000640 1 -- 172.21.15.59:0/299567324 wait complete. 2024-04-24T06:30:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.709+0000 7fbd5e000640 1 Processor -- start 2024-04-24T06:30:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.709+0000 7fbd5e000640 1 -- start start 2024-04-24T06:30:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd58159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd5815c3d0 0x7fbd5815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5815c930 con 0x7fbd580cd590 2024-04-24T06:30:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5815c7b0 con 0x7fbd5815c3d0 2024-04-24T06:30:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd5e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd580cbd10 con 0x7fbd580b4860 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59586/0 (socket says 172.21.15.59:59586) 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.710+0000 7fbd57e00640 1 -- 172.21.15.59:0/3991985846 learned_addr learned my addr 172.21.15.59:0/3991985846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd5cc00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd58159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 -- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 msgr2=0x7fbd58159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd58159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 -- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd5815c3d0 msgr2=0x7fbd5815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd5d600640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd5815c3d0 0x7fbd5815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd5815c3d0 0x7fbd5815cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 -- 172.21.15.59:0/3991985846 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5815b490 con 0x7fbd580cd590 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd57e00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c4d0 tx=0x7fbd4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44018070 con 0x7fbd580cd590 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd44007500 con 0x7fbd580cd590 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44004a20 con 0x7fbd580cd590 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.711+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5815abf0 con 0x7fbd580cd590 2024-04-24T06:30:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.712+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5815b1b0 con 0x7fbd580cd590 2024-04-24T06:30:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.713+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5804fd90 con 0x7fbd580cd590 2024-04-24T06:30:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.714+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd4401b070 con 0x7fbd580cd590 2024-04-24T06:30:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.715+0000 7fbd54c00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 0x7fbd3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.715+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbd4408ae80 con 0x7fbd580cd590 2024-04-24T06:30:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.715+0000 7fbd5cc00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 0x7fbd3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.716+0000 7fbd5cc00640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 0x7fbd3806edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c034000 tx=0x7fbd4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.719+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd5815b490 con 0x7fbd580cd590 2024-04-24T06:30:39.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.963+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd5806e400 con 0x7fbd580cd590 2024-04-24T06:30:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.964+0000 7fbd54c00640 1 -- 172.21.15.59:0/3991985846 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fbd440592a0 con 0x7fbd580cd590 2024-04-24T06:30:39.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:39.966 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:39.967 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 msgr2=0x7fbd3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 0x7fbd3806edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c034000 tx=0x7fbd4c02e000 comp rx=0 tx=0).stop 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 msgr2=0x7fbd5815a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c4d0 tx=0x7fbd4400c990 comp rx=0 tx=0).stop 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 shutdown_connections 2024-04-24T06:30:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd3806c8f0 0x7fbd3806edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd5815c3d0 0x7fbd5815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd580cd590 0x7fbd5815a130 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 --2- 172.21.15.59:0/3991985846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd580b4860 0x7fbd58159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 >> 172.21.15.59:0/3991985846 conn(0x7fbd580b21a0 msgr2=0x7fbd580b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 shutdown_connections 2024-04-24T06:30:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:39.967+0000 7fbd5e000640 1 -- 172.21.15.59:0/3991985846 wait complete. 2024-04-24T06:30:39.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:40.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.193+0000 7f29a8c00640 1 Processor -- start 2024-04-24T06:30:40.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.194+0000 7f29a8c00640 1 -- start start 2024-04-24T06:30:40.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.195+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40b5ff0 0x7f29a40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.195+0000 7f29a8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29a40d0d90 0x7f29a40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.195+0000 7f29a8c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a40cf030 con 0x7f29a415e7c0 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.195+0000 7f29a8c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f29a40b7000 con 0x7f29a40d0d90 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.195+0000 7f29a8c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29a40cf620 con 0x7f29a40b5ff0 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f29a415e7c0 0x7f29a40d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45448/0 (socket says 172.21.15.59:45448) 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3400640 1 -- 172.21.15.59:0/2221802373 learned_addr learned my addr 172.21.15.59:0/2221802373 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3e00640 1 --2- 172.21.15.59:0/2221802373 >> v2:172.21.15.59:3300/0 conn(0x7f29a40d0d90 0x7f29a40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a2a00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40b5ff0 0x7f29a40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a2000640 1 -- 172.21.15.59:0/2221802373 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060403523 0 0) 0x7f2994001490 con 0x7f29a415e7c0 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a2000640 1 -- 172.21.15.59:0/2221802373 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2988003520 con 0x7f29a415e7c0 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3e00640 1 -- 172.21.15.59:0/2221802373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40b5ff0 msgr2=0x7f29a40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3e00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40b5ff0 0x7f29a40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3e00640 1 -- 172.21.15.59:0/2221802373 >> v1:172.21.15.59:6789/0 conn(0x7f29a415e7c0 legacy=0x7f29a40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.196+0000 7f29a3e00640 1 -- 172.21.15.59:0/2221802373 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a40b6a40 con 0x7f29a40d0d90 2024-04-24T06:30:40.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.197+0000 7f29a3e00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a40d0d90 0x7f29a40d1220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f299c0097a0 tx=0x7f299c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcf717dc50aa8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.198+0000 7f29a2000640 1 -- 172.21.15.59:0/2221802373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c037070 con 0x7f29a40d0d90 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.198+0000 7f29a2000640 1 -- 172.21.15.59:0/2221802373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299c032440 con 0x7f29a40d0d90 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.198+0000 7f29a2000640 1 -- 172.21.15.59:0/2221802373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0327f0 con 0x7f29a40d0d90 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.198+0000 7f29a8c00640 1 -- 172.21.15.59:0/2221802373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a40d0d90 msgr2=0x7f29a40d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.198+0000 7f29a8c00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a40d0d90 0x7f29a40d1220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f299c0097a0 tx=0x7f299c02a180 comp rx=0 tx=0).stop 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 -- 172.21.15.59:0/2221802373 shutdown_connections 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a40d0d90 0x7f29a40d1220 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 --2- 172.21.15.59:0/2221802373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40b5ff0 0x7f29a40b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 -- 172.21.15.59:0/2221802373 >> 172.21.15.59:0/2221802373 conn(0x7f29a409d920 msgr2=0x7f29a408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 -- 172.21.15.59:0/2221802373 shutdown_connections 2024-04-24T06:30:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.199+0000 7f29a8c00640 1 -- 172.21.15.59:0/2221802373 wait complete. 2024-04-24T06:30:40.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.200+0000 7f29a8c00640 1 Processor -- start 2024-04-24T06:30:40.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.201+0000 7f29a8c00640 1 -- start start 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40b5ff0 0x7f29a4159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40d0d90 0x7f29a40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29a40d2800 con 0x7f29a415e7c0 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29a40cf2b0 con 0x7f29a40b5ff0 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a8c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29a40d1a80 con 0x7f29a40d0d90 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59600/0 (socket says 172.21.15.59:59600) 2024-04-24T06:30:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 -- 172.21.15.59:0/1027726280 learned_addr learned my addr 172.21.15.59:0/1027726280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a2a00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40d0d90 0x7f29a40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3400640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40b5ff0 0x7f29a4159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 -- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40d0d90 msgr2=0x7f29a40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40d0d90 0x7f29a40cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 -- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40b5ff0 msgr2=0x7f29a4159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40b5ff0 0x7f29a4159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.202+0000 7f29a3e00640 1 -- 172.21.15.59:0/1027726280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a4159fd0 con 0x7f29a415e7c0 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f29a3e00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f299c035040 tx=0x7f299c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c037050 con 0x7f29a415e7c0 2024-04-24T06:30:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299c043070 con 0x7f29a415e7c0 2024-04-24T06:30:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c032a30 con 0x7f29a415e7c0 2024-04-24T06:30:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a4159b90 con 0x7f29a415e7c0 2024-04-24T06:30:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.203+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a40cd540 con 0x7f29a415e7c0 2024-04-24T06:30:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.204+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a404fd90 con 0x7f29a415e7c0 2024-04-24T06:30:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.205+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f299c032bd0 con 0x7f29a415e7c0 2024-04-24T06:30:40.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.207+0000 7f299b400640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 0x7f298406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.207+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f299c0ac9d0 con 0x7f29a415e7c0 2024-04-24T06:30:40.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.207+0000 7f29a3400640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 0x7f298406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.208+0000 7f29a3400640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 0x7f298406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2994001630 tx=0x7f2994009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.211+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f299c030340 con 0x7f29a415e7c0 2024-04-24T06:30:40.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.418+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29a40d1eb0 con 0x7f29a415e7c0 2024-04-24T06:30:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.421+0000 7f299b400640 1 -- 172.21.15.59:0/1027726280 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f299c0044a0 con 0x7f29a415e7c0 2024-04-24T06:30:40.423 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:40.424 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.423+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 msgr2=0x7f298406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.423+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 0x7f298406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2994001630 tx=0x7f2994009040 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.423+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 msgr2=0x7f29a40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.423+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f299c035040 tx=0x7f299c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 shutdown_connections 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f298406c8f0 0x7f298406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a415e7c0 0x7f29a40b2320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29a40d0d90 0x7f29a40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 --2- 172.21.15.59:0/1027726280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29a40b5ff0 0x7f29a4159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 >> 172.21.15.59:0/1027726280 conn(0x7f29a409d920 msgr2=0x7f29a406cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:40.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 shutdown_connections 2024-04-24T06:30:40.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.424+0000 7f29a8c00640 1 -- 172.21.15.59:0/1027726280 wait complete. 2024-04-24T06:30:40.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:30:02.277562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:30:03.324428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:40.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.663+0000 7f693f800640 1 Processor -- start 2024-04-24T06:30:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.664+0000 7f693f800640 1 -- start start 2024-04-24T06:30:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693f800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6938145e10 0x7f693806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 0x7f6938072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693f800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69380cfc20 con 0x7f69380d0cc0 2024-04-24T06:30:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693f800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f69380cff00 con 0x7f6938145e10 2024-04-24T06:30:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69380d01e0 con 0x7f6938071d70 2024-04-24T06:30:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6938145e10 0x7f693806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6938145e10 0x7f693806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59610/0 (socket says 172.21.15.59:59610) 2024-04-24T06:30:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.665+0000 7f693ce00640 1 -- 172.21.15.59:0/2293456822 learned_addr learned my addr 172.21.15.59:0/2293456822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693d800640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 0x7f6938072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693ce00640 1 -- 172.21.15.59:0/2293456822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 msgr2=0x7f6938072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693ce00640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 0x7f6938072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693ce00640 1 -- 172.21.15.59:0/2293456822 >> v1:172.21.15.59:6789/0 conn(0x7f69380d0cc0 legacy=0x7f69380cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693ce00640 1 -- 172.21.15.59:0/2293456822 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69380d04c0 con 0x7f6938145e10 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.666+0000 7f693d800640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 0x7f6938072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.667+0000 7f693ce00640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6938145e10 0x7f693806f130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6928009870 tx=0x7f692802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae535e770d7583f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.667+0000 7f6937e00640 1 -- 172.21.15.59:0/2293456822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928037070 con 0x7f6938145e10 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.667+0000 7f6937e00640 1 -- 172.21.15.59:0/2293456822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6928032440 con 0x7f6938145e10 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.667+0000 7f6937e00640 1 -- 172.21.15.59:0/2293456822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928032810 con 0x7f6938145e10 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 -- 172.21.15.59:0/2293456822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6938145e10 msgr2=0x7f693806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6938145e10 0x7f693806f130 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6928009870 tx=0x7f692802a250 comp rx=0 tx=0).stop 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 -- 172.21.15.59:0/2293456822 shutdown_connections 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938071d70 0x7f6938072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 --2- 172.21.15.59:0/2293456822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6938145e10 0x7f693806f130 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 -- 172.21.15.59:0/2293456822 >> 172.21.15.59:0/2293456822 conn(0x7f6938067d90 msgr2=0x7f693806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 -- 172.21.15.59:0/2293456822 shutdown_connections 2024-04-24T06:30:40.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.668+0000 7f693f800640 1 -- 172.21.15.59:0/2293456822 wait complete. 2024-04-24T06:30:40.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.670+0000 7f693f800640 1 Processor -- start 2024-04-24T06:30:40.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.670+0000 7f693f800640 1 -- start start 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938071d70 0x7f69380b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938145e10 0x7f693815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69381461f0 con 0x7f69380d0cc0 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f693815cdd0 con 0x7f6938071d70 2024-04-24T06:30:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.671+0000 7f693f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69381319e0 con 0x7f6938145e10 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59612/0 (socket says 172.21.15.59:59612) 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 -- 172.21.15.59:0/1322953933 learned_addr learned my addr 172.21.15.59:0/1322953933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f693d800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938145e10 0x7f693815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f693ce00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938071d70 0x7f69380b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 -- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938145e10 msgr2=0x7f693815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938145e10 0x7f693815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 -- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938071d70 msgr2=0x7f69380b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938071d70 0x7f69380b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 -- 172.21.15.59:0/1322953933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69380ad420 con 0x7f69380d0cc0 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.672+0000 7f692fe00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f692000c400 tx=0x7f692000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.673+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200052d0 con 0x7f69380d0cc0 2024-04-24T06:30:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.673+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6920005490 con 0x7f69380d0cc0 2024-04-24T06:30:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.673+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920007260 con 0x7f69380d0cc0 2024-04-24T06:30:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.673+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6938131c80 con 0x7f69380d0cc0 2024-04-24T06:30:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.673+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6938132240 con 0x7f69380d0cc0 2024-04-24T06:30:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.674+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6938068a80 con 0x7f69380d0cc0 2024-04-24T06:30:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.675+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6920007400 con 0x7f69380d0cc0 2024-04-24T06:30:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.676+0000 7f6935600640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 0x7f691406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.676+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f692008b210 con 0x7f69380d0cc0 2024-04-24T06:30:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.676+0000 7f693ce00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 0x7f691406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.677+0000 7f693ce00640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 0x7f691406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6928034000 tx=0x7f692802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.681+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6920059510 con 0x7f69380d0cc0 2024-04-24T06:30:40.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.885+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69380d1190 con 0x7f69380d0cc0 2024-04-24T06:30:40.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.886+0000 7f6935600640 1 -- 172.21.15.59:0/1322953933 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f6920058e10 con 0x7f69380d0cc0 2024-04-24T06:30:40.888 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 msgr2=0x7f691406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 0x7f691406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6928034000 tx=0x7f692802e000 comp rx=0 tx=0).stop 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 msgr2=0x7f693815bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f692000c400 tx=0x7f692000c8c0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 shutdown_connections 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f691406c8f0 0x7f691406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6938145e10 0x7f693815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69380d0cc0 0x7f693815bd70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 --2- 172.21.15.59:0/1322953933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6938071d70 0x7f69380b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 >> 172.21.15.59:0/1322953933 conn(0x7f6938067d90 msgr2=0x7f69380a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 shutdown_connections 2024-04-24T06:30:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:40.889+0000 7f693f800640 1 -- 172.21.15.59:0/1322953933 wait complete. 2024-04-24T06:30:40.911 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:30:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.116+0000 7f3a6f200640 1 Processor -- start 2024-04-24T06:30:41.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.117+0000 7f3a6f200640 1 -- start start 2024-04-24T06:30:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.118+0000 7f3a6f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.118+0000 7f3a6f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a700d13e0 0x7f3a700d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.118+0000 7f3a6f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a700b5da0 con 0x7f3a700b3600 2024-04-24T06:30:41.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.118+0000 7f3a6f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7015f100 con 0x7f3a700d13e0 2024-04-24T06:30:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.118+0000 7f3a6f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3a700d07d0 con 0x7f3a7015ed20 2024-04-24T06:30:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.119+0000 7f3a6de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.119+0000 7f3a6de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:44820/0 (socket says 172.21.15.59:44820) 2024-04-24T06:30:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.119+0000 7f3a6de00640 1 -- 172.21.15.59:0/3473374726 learned_addr learned my addr 172.21.15.59:0/3473374726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:41.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.119+0000 7f3a6e800640 1 --2- 172.21.15.59:0/3473374726 >> v2:172.21.15.66:3300/0 conn(0x7f3a700d13e0 0x7f3a700d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.120+0000 7f3a6de00640 1 -- 172.21.15.59:0/3473374726 >> v2:172.21.15.66:3300/0 conn(0x7f3a700d13e0 msgr2=0x7f3a700d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.120+0000 7f3a6de00640 1 --2- 172.21.15.59:0/3473374726 >> v2:172.21.15.66:3300/0 conn(0x7f3a700d13e0 0x7f3a700d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.120+0000 7f3a6de00640 1 -- 172.21.15.59:0/3473374726 >> v1:172.21.15.66:6789/0 conn(0x7f3a700b3600 legacy=0x7f3a700b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.120+0000 7f3a6de00640 1 -- 172.21.15.59:0/3473374726 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a700d0a20 con 0x7f3a7015ed20 2024-04-24T06:30:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.122+0000 7f3a6de00640 1 --2- 172.21.15.59:0/3473374726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca56e3f8f3c9b104 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.122+0000 7f3a6d400640 1 -- 172.21.15.59:0/3473374726 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60037070 con 0x7f3a7015ed20 2024-04-24T06:30:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.122+0000 7f3a6d400640 1 -- 172.21.15.59:0/3473374726 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60032440 con 0x7f3a7015ed20 2024-04-24T06:30:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.122+0000 7f3a6d400640 1 -- 172.21.15.59:0/3473374726 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60032810 con 0x7f3a7015ed20 2024-04-24T06:30:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 -- 172.21.15.59:0/3473374726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 msgr2=0x7f3a700d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 --2- 172.21.15.59:0/3473374726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6002a130 comp rx=0 tx=0).stop 2024-04-24T06:30:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 -- 172.21.15.59:0/3473374726 shutdown_connections 2024-04-24T06:30:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 --2- 172.21.15.59:0/3473374726 >> v2:172.21.15.66:3300/0 conn(0x7f3a700d13e0 0x7f3a700d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 --2- 172.21.15.59:0/3473374726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a7015ed20 0x7f3a700d0ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.123+0000 7f3a6f200640 1 -- 172.21.15.59:0/3473374726 >> 172.21.15.59:0/3473374726 conn(0x7f3a7008bf90 msgr2=0x7f3a700d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.124+0000 7f3a6f200640 1 -- 172.21.15.59:0/3473374726 shutdown_connections 2024-04-24T06:30:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.124+0000 7f3a6f200640 1 -- 172.21.15.59:0/3473374726 wait complete. 2024-04-24T06:30:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.125+0000 7f3a6f200640 1 Processor -- start 2024-04-24T06:30:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.125+0000 7f3a6f200640 1 -- start start 2024-04-24T06:30:41.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a700b3600 0x7f3a701529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a700d13e0 0x7f3a700b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a700d3490 con 0x7f3a7015ed20 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3a700d31b0 con 0x7f3a700b3600 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.126+0000 7f3a6f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3a700b4620 con 0x7f3a700d13e0 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59622/0 (socket says 172.21.15.59:59622) 2024-04-24T06:30:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 -- 172.21.15.59:0/2979122534 learned_addr learned my addr 172.21.15.59:0/2979122534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a67e00640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a700d13e0 0x7f3a700b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6de00640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a700b3600 0x7f3a701529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 -- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a700d13e0 msgr2=0x7f3a700b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a700d13e0 0x7f3a700b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 -- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a700b3600 msgr2=0x7f3a701529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a700b3600 0x7f3a701529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.127+0000 7f3a6e800640 1 -- 172.21.15.59:0/2979122534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a701f8c30 con 0x7f3a7015ed20 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a6e800640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800efc0 tx=0x7f3a6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68018020 con 0x7f3a7015ed20 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a68014480 con 0x7f3a7015ed20 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68014870 con 0x7f3a7015ed20 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a700ca110 con 0x7f3a7015ed20 2024-04-24T06:30:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.128+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a700ca6d0 con 0x7f3a7015ed20 2024-04-24T06:30:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.129+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7004fd90 con 0x7f3a7015ed20 2024-04-24T06:30:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.130+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a68005d10 con 0x7f3a7015ed20 2024-04-24T06:30:41.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.131+0000 7f3a66000640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.131+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3a6808c660 con 0x7f3a7015ed20 2024-04-24T06:30:41.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.132+0000 7f3a6de00640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.132+0000 7f3a6de00640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3a60034000 tx=0x7f3a6002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.136+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a68022070 con 0x7f3a7015ed20 2024-04-24T06:30:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.340+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a7006d120 con 0x7f3a7015ed20 2024-04-24T06:30:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.341+0000 7f3a66000640 1 -- 172.21.15.59:0/2979122534 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f3a68010040 con 0x7f3a7015ed20 2024-04-24T06:30:41.343 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.343+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 msgr2=0x7f3a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.343+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3a60034000 tx=0x7f3a6002e000 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.343+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 msgr2=0x7f3a700b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.343+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800efc0 tx=0x7f3a6800c490 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 shutdown_connections 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a4c06c8f0 0x7f3a4c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a7015ed20 0x7f3a700b2bb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a700d13e0 0x7f3a700b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 --2- 172.21.15.59:0/2979122534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a700b3600 0x7f3a701529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 >> 172.21.15.59:0/2979122534 conn(0x7f3a7008bf90 msgr2=0x7f3a700b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 shutdown_connections 2024-04-24T06:30:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.344+0000 7f3a6f200640 1 -- 172.21.15.59:0/2979122534 wait complete. 2024-04-24T06:30:41.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:30:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.584+0000 7fe2e7c00640 1 Processor -- start 2024-04-24T06:30:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.584+0000 7fe2e7c00640 1 -- start start 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e7c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2e00b56f0 0x7fe2e006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e7c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 0x7fe2e006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e7c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e015bf60 con 0x7fe2e00d0350 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e7c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e015c240 con 0x7fe2e00b56f0 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e7c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e015c520 con 0x7fe2e006f340 2024-04-24T06:30:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2e00b56f0 0x7fe2e006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e5c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe2e00d0350 0x7fe2e00b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45458/0 (socket says 172.21.15.59:45458) 2024-04-24T06:30:41.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2e00b56f0 0x7fe2e006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59636/0 (socket says 172.21.15.59:59636) 2024-04-24T06:30:41.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.586+0000 7fe2e5c00640 1 -- 172.21.15.59:0/2010769819 learned_addr learned my addr 172.21.15.59:0/2010769819 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2e5200640 1 -- 172.21.15.59:0/2010769819 >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 msgr2=0x7fe2e006f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2dfe00640 1 --2- 172.21.15.59:0/2010769819 >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 0x7fe2e006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2df400640 1 -- 172.21.15.59:0/2010769819 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002555805 0 0) 0x7fe2e015bf60 con 0x7fe2e00d0350 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2e5200640 1 --2- 172.21.15.59:0/2010769819 >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 0x7fe2e006f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2e5200640 1 -- 172.21.15.59:0/2010769819 >> v1:172.21.15.59:6789/0 conn(0x7fe2e00d0350 legacy=0x7fe2e00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2e5200640 1 -- 172.21.15.59:0/2010769819 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e015c800 con 0x7fe2e00b56f0 2024-04-24T06:30:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.587+0000 7fe2dfe00640 1 --2- 172.21.15.59:0/2010769819 >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 0x7fe2e006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.588+0000 7fe2e5200640 1 --2- 172.21.15.59:0/2010769819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00b56f0 0x7fe2e006ddd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00d1360 tx=0x7fe2d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6e57955ab790f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.588+0000 7fe2df400640 1 -- 172.21.15.59:0/2010769819 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0037070 con 0x7fe2e00b56f0 2024-04-24T06:30:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.588+0000 7fe2df400640 1 -- 172.21.15.59:0/2010769819 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d0032440 con 0x7fe2e00b56f0 2024-04-24T06:30:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.588+0000 7fe2df400640 1 -- 172.21.15.59:0/2010769819 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0032810 con 0x7fe2e00b56f0 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 -- 172.21.15.59:0/2010769819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00b56f0 msgr2=0x7fe2e006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/2010769819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00b56f0 0x7fe2e006ddd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00d1360 tx=0x7fe2d002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 -- 172.21.15.59:0/2010769819 shutdown_connections 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/2010769819 >> v2:172.21.15.66:3300/0 conn(0x7fe2e006f340 0x7fe2e006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/2010769819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00b56f0 0x7fe2e006ddd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 -- 172.21.15.59:0/2010769819 >> 172.21.15.59:0/2010769819 conn(0x7fe2e0067d90 msgr2=0x7fe2e006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 -- 172.21.15.59:0/2010769819 shutdown_connections 2024-04-24T06:30:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.589+0000 7fe2e7c00640 1 -- 172.21.15.59:0/2010769819 wait complete. 2024-04-24T06:30:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.591+0000 7fe2e7c00640 1 Processor -- start 2024-04-24T06:30:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.591+0000 7fe2e7c00640 1 -- start start 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 0x7fe2e00c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 0x7fe2e00c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e00d07e0 con 0x7fe2e00d0350 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e015bc10 con 0x7fe2e006f340 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e7c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e00d0960 con 0x7fe2e00b56f0 2024-04-24T06:30:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59644/0 (socket says 172.21.15.59:59644) 2024-04-24T06:30:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.592+0000 7fe2e5c00640 1 -- 172.21.15.59:0/175355374 learned_addr learned my addr 172.21.15.59:0/175355374 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5200640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 0x7fe2e00c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2dfe00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 0x7fe2e00c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 -- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 msgr2=0x7fe2e00c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 0x7fe2e00c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 -- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 msgr2=0x7fe2e00c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 0x7fe2e00c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 -- 172.21.15.59:0/175355374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e00c5520 con 0x7fe2e00d0350 2024-04-24T06:30:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5200640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 0x7fe2e00c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2dfe00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 0x7fe2e00c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e5c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c700 tx=0x7fe2d800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8018070 con 0x7fe2e00d0350 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d800ce10 con 0x7fe2e00d0350 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800a860 con 0x7fe2e00d0350 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.593+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e00c5800 con 0x7fe2e00d0350 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.594+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e00c5dc0 con 0x7fe2e00d0350 2024-04-24T06:30:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.595+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e0068a80 con 0x7fe2e00d0350 2024-04-24T06:30:41.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.596+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2d800aa00 con 0x7fe2e00d0350 2024-04-24T06:30:41.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.597+0000 7fe2dcc00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:41.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.597+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe2d808b1f0 con 0x7fe2e00d0350 2024-04-24T06:30:41.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.597+0000 7fe2e5200640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:41.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.598+0000 7fe2e5200640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.602+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2d8059580 con 0x7fe2e00d0350 2024-04-24T06:30:41.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.813+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2e006ef20 con 0x7fe2e00d0350 2024-04-24T06:30:41.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.814+0000 7fe2dcc00640 1 -- 172.21.15.59:0/175355374 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fe2d8058e80 con 0x7fe2e00d0350 2024-04-24T06:30:41.816 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 msgr2=0x7fe2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002e000 comp rx=0 tx=0).stop 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 msgr2=0x7fe2e015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c700 tx=0x7fe2d800cbc0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 shutdown_connections 2024-04-24T06:30:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e00d0350 0x7fe2e015ac70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe2e00b56f0 0x7fe2e00c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 --2- 172.21.15.59:0/175355374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e006f340 0x7fe2e00c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 >> 172.21.15.59:0/175355374 conn(0x7fe2e0067d90 msgr2=0x7fe2e00b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 shutdown_connections 2024-04-24T06:30:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:41.817+0000 7fe2e7c00640 1 -- 172.21.15.59:0/175355374 wait complete. 2024-04-24T06:30:41.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:42.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.037+0000 7f1448000640 1 Processor -- start 2024-04-24T06:30:42.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.038+0000 7f1448000640 1 -- start start 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14401454e0 0x7f1440145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14400cd020 0x7f14400b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14400b3e40 0x7f1440160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f14400cddc0 con 0x7f14400cd020 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f14400cd520 con 0x7f14400b3e40 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.039+0000 7f1448000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f14400cd800 con 0x7f14401454e0 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14400cd020 0x7f14400b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14400cd020 0x7f14400b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59660/0 (socket says 172.21.15.59:59660) 2024-04-24T06:30:42.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 -- 172.21.15.59:0/3631263340 learned_addr learned my addr 172.21.15.59:0/3631263340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1445600640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14401454e0 0x7f1440145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1446000640 1 --2- 172.21.15.59:0/3631263340 >> v2:172.21.15.66:3300/0 conn(0x7f14400b3e40 0x7f1440160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 -- 172.21.15.59:0/3631263340 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14401454e0 msgr2=0x7f1440145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14401454e0 0x7f1440145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 -- 172.21.15.59:0/3631263340 >> v2:172.21.15.66:3300/0 conn(0x7f14400b3e40 msgr2=0x7f1440160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 --2- 172.21.15.59:0/3631263340 >> v2:172.21.15.66:3300/0 conn(0x7f14400b3e40 0x7f1440160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.040+0000 7f1444c00640 1 -- 172.21.15.59:0/3631263340 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14400cdae0 con 0x7f14400cd020 2024-04-24T06:30:42.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.041+0000 7f1444c00640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14400cd020 0x7f14400b3900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f14400d0270 tx=0x7f142c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa58c9a2acf7a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f143fe00640 1 -- 172.21.15.59:0/3631263340 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c037070 con 0x7f14400cd020 2024-04-24T06:30:42.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f143fe00640 1 -- 172.21.15.59:0/3631263340 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c032440 con 0x7f14400cd020 2024-04-24T06:30:42.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f143fe00640 1 -- 172.21.15.59:0/3631263340 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c032810 con 0x7f14400cd020 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 -- 172.21.15.59:0/3631263340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14400cd020 msgr2=0x7f14400b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14400cd020 0x7f14400b3900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f14400d0270 tx=0x7f142c02a140 comp rx=0 tx=0).stop 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 -- 172.21.15.59:0/3631263340 shutdown_connections 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 --2- 172.21.15.59:0/3631263340 >> v2:172.21.15.66:3300/0 conn(0x7f14400b3e40 0x7f1440160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14400cd020 0x7f14400b3900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 --2- 172.21.15.59:0/3631263340 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14401454e0 0x7f1440145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 -- 172.21.15.59:0/3631263340 >> 172.21.15.59:0/3631263340 conn(0x7f1440067ea0 msgr2=0x7f144006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 -- 172.21.15.59:0/3631263340 shutdown_connections 2024-04-24T06:30:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.042+0000 7f1448000640 1 -- 172.21.15.59:0/3631263340 wait complete. 2024-04-24T06:30:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.044+0000 7f1448000640 1 Processor -- start 2024-04-24T06:30:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.044+0000 7f1448000640 1 -- start start 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 0x7f144015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 0x7f144015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1440146270 con 0x7f14401454e0 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14401460f0 con 0x7f14400b3e40 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.045+0000 7f1448000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1440160e50 con 0x7f14400cd020 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59672/0 (socket says 172.21.15.59:59672) 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 -- 172.21.15.59:0/896011739 learned_addr learned my addr 172.21.15.59:0/896011739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1445600640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 0x7f144015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1444c00640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 0x7f144015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 -- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 msgr2=0x7f144015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 0x7f144015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 -- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 msgr2=0x7f144015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 0x7f144015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 -- 172.21.15.59:0/896011739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144006e510 con 0x7f14401454e0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1445600640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 0x7f144015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1444c00640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 0x7f144015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.046+0000 7f1446000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1438007e70 tx=0x7f143800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.047+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438014620 con 0x7f14401454e0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.047+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1438019070 con 0x7f14401454e0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.047+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438014a20 con 0x7f14401454e0 2024-04-24T06:30:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.047+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144006e7f0 con 0x7f14401454e0 2024-04-24T06:30:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.047+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144006edb0 con 0x7f14401454e0 2024-04-24T06:30:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.048+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1440068a80 con 0x7f14401454e0 2024-04-24T06:30:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.049+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1438014bc0 con 0x7f14401454e0 2024-04-24T06:30:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.050+0000 7f143d600640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 0x7f142006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.050+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f143808c4a0 con 0x7f14401454e0 2024-04-24T06:30:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.050+0000 7f1445600640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 0x7f142006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.051+0000 7f1445600640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 0x7f142006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.055+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1438011260 con 0x7f14401454e0 2024-04-24T06:30:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.303+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f144006f4b0 con 0x7f14401454e0 2024-04-24T06:30:42.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.304+0000 7f143d600640 1 -- 172.21.15.59:0/896011739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f143805a8d0 con 0x7f14401454e0 2024-04-24T06:30:42.306 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:42.306 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:42.307 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 msgr2=0x7f142006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 0x7f142006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1430006fd0 tx=0x7f1430008040 comp rx=0 tx=0).stop 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 msgr2=0x7f144006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1438007e70 tx=0x7f143800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 shutdown_connections 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f142006c8f0 0x7f142006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14401454e0 0x7f144006dce0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14400cd020 0x7f144015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 --2- 172.21.15.59:0/896011739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14400b3e40 0x7f144015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 >> 172.21.15.59:0/896011739 conn(0x7f1440067ea0 msgr2=0x7f14400b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:42.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 shutdown_connections 2024-04-24T06:30:42.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.307+0000 7f1448000640 1 -- 172.21.15.59:0/896011739 wait complete. 2024-04-24T06:30:42.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.538+0000 7f7cd6600640 1 Processor -- start 2024-04-24T06:30:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.538+0000 7f7cd6600640 1 -- start start 2024-04-24T06:30:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.539+0000 7f7cd6600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7cd00cd760 0x7f7cd00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.539+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.539+0000 7f7cd6600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd0160210 con 0x7f7cd015c3d0 2024-04-24T06:30:42.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.539+0000 7f7cd6600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd01604f0 con 0x7f7cd00cd760 2024-04-24T06:30:42.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.539+0000 7f7cd6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd01607d0 con 0x7f7cd00b4690 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.540+0000 7f7cd5200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7cd015c3d0 0x7f7cd00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45474/0 (socket says 172.21.15.59:45474) 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.540+0000 7f7cd5200640 1 -- 172.21.15.59:0/447944326 learned_addr learned my addr 172.21.15.59:0/447944326 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.540+0000 7f7ccfe00640 1 --2- 172.21.15.59:0/447944326 >> v2:172.21.15.66:3300/0 conn(0x7f7cd00cd760 0x7f7cd00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.540+0000 7f7cd5c00640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.540+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3520434868 0 0) 0x7f7cd0160210 con 0x7f7cd015c3d0 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb4003520 con 0x7f7cd015c3d0 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccfe00640 1 -- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 msgr2=0x7f7cd00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccfe00640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd00cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccfe00640 1 -- 172.21.15.59:0/447944326 >> v1:172.21.15.59:6789/0 conn(0x7f7cd015c3d0 legacy=0x7f7cd00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355045895 0 0) 0x7f7cb4003520 con 0x7f7cd015c3d0 2024-04-24T06:30:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.541+0000 7f7ccfe00640 1 -- 172.21.15.59:0/447944326 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0160ab0 con 0x7f7cd00cd760 2024-04-24T06:30:42.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.542+0000 7f7ccfe00640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00b4150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3ff9ea4563ac2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7cd00cd760 2024-04-24T06:30:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb802aba0 con 0x7f7cd00cd760 2024-04-24T06:30:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7ccf400640 1 -- 172.21.15.59:0/447944326 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0001670 con 0x7f7cd00cd760 2024-04-24T06:30:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 -- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 msgr2=0x7f7cd00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00b4150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a110 comp rx=0 tx=0).stop 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 -- 172.21.15.59:0/447944326 shutdown_connections 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 --2- 172.21.15.59:0/447944326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00b4150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 -- 172.21.15.59:0/447944326 >> 172.21.15.59:0/447944326 conn(0x7f7cd00b21a0 msgr2=0x7f7cd00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 -- 172.21.15.59:0/447944326 shutdown_connections 2024-04-24T06:30:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.543+0000 7f7cd6600640 1 -- 172.21.15.59:0/447944326 wait complete. 2024-04-24T06:30:42.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.545+0000 7f7cd6600640 1 Processor -- start 2024-04-24T06:30:42.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.545+0000 7f7cd6600640 1 -- start start 2024-04-24T06:30:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd015c7b0 con 0x7f7cd015c3d0 2024-04-24T06:30:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00b4b00 con 0x7f7cd00cd760 2024-04-24T06:30:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.546+0000 7f7cd6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd015c930 con 0x7f7cd00b4690 2024-04-24T06:30:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59676/0 (socket says 172.21.15.59:59676) 2024-04-24T06:30:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 -- 172.21.15.59:0/417025200 learned_addr learned my addr 172.21.15.59:0/417025200 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5200640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 -- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 msgr2=0x7f7cd015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7ccfe00640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 -- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 msgr2=0x7f7cd00c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5c00640 1 -- 172.21.15.59:0/417025200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0156fc0 con 0x7f7cd015c3d0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.547+0000 7f7cd5200640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cd5c00640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800efc0 tx=0x7f7cc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8018020 con 0x7f7cd015c3d0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8014480 con 0x7f7cd015c3d0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8014870 con 0x7f7cd015c3d0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd0157260 con 0x7f7cd015c3d0 2024-04-24T06:30:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.548+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd0157820 con 0x7f7cd015c3d0 2024-04-24T06:30:42.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.549+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd00cdb40 con 0x7f7cd015c3d0 2024-04-24T06:30:42.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.550+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cc8005d10 con 0x7f7cd015c3d0 2024-04-24T06:30:42.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.551+0000 7f7cccc00640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 0x7f7cac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:42.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.551+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7cc808c710 con 0x7f7cd015c3d0 2024-04-24T06:30:42.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.551+0000 7f7cd5200640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 0x7f7cac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:42.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.552+0000 7f7cd5200640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 0x7f7cac06ee00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0001380 tx=0x7f7cc0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:42.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.556+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cd00cdb40 con 0x7f7cd015c3d0 2024-04-24T06:30:42.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.763+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cd004fd90 con 0x7f7cd015c3d0 2024-04-24T06:30:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.765+0000 7f7cccc00640 1 -- 172.21.15.59:0/417025200 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f7cc8022070 con 0x7f7cd015c3d0 2024-04-24T06:30:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:42.768 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 msgr2=0x7f7cac06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 0x7f7cac06ee00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0001380 tx=0x7f7cc0009040 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 msgr2=0x7f7cd00cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800efc0 tx=0x7f7cc800c490 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 shutdown_connections 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7cac06c940 0x7f7cac06ee00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cd015c3d0 0x7f7cd00cdd80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd00cd760 0x7f7cd00c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 --2- 172.21.15.59:0/417025200 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cd00b4690 0x7f7cd015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 >> 172.21.15.59:0/417025200 conn(0x7f7cd00b21a0 msgr2=0x7f7cd00cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 shutdown_connections 2024-04-24T06:30:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.768+0000 7f7cd6600640 1 -- 172.21.15.59:0/417025200 wait complete. 2024-04-24T06:30:42.789 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:30:42.789 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.998+0000 7f7cb0e00640 1 Processor -- start 2024-04-24T06:30:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:42.999+0000 7f7cb0e00640 1 -- start start 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7cb0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7cac0afaf0 0x7f7cac0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7cb0e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cac15f0f0 con 0x7f7cac0d06e0 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7cb0e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cac15f3d0 con 0x7f7cac15c5e0 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7cb0e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7cac15fa20 con 0x7f7cac0afaf0 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7ca9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7cac0afaf0 0x7f7cac0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7ca9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7cac0afaf0 0x7f7cac0d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59688/0 (socket says 172.21.15.59:59688) 2024-04-24T06:30:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7ca9400640 1 -- 172.21.15.59:0/2595589013 learned_addr learned my addr 172.21.15.59:0/2595589013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.000+0000 7f7caa800640 1 --1- 172.21.15.59:0/2595589013 >> v1:172.21.15.59:6789/0 conn(0x7f7cac0d06e0 0x7f7cac1624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45476/0 (socket says 172.21.15.59:45476) 2024-04-24T06:30:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.001+0000 7f7ca9400640 1 -- 172.21.15.59:0/2595589013 >> v1:172.21.15.66:6789/0 conn(0x7f7cac15c5e0 legacy=0x7f7cac0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.001+0000 7f7ca9400640 1 -- 172.21.15.59:0/2595589013 >> v1:172.21.15.59:6789/0 conn(0x7f7cac0d06e0 legacy=0x7f7cac1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.001+0000 7f7ca9400640 1 -- 172.21.15.59:0/2595589013 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac15f6b0 con 0x7f7cac0afaf0 2024-04-24T06:30:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.002+0000 7f7ca9400640 1 --2- 172.21.15.59:0/2595589013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac0d01a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009870 tx=0x7f7c9802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40038fb253cb5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.002+0000 7f7ca8a00640 1 -- 172.21.15.59:0/2595589013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98037070 con 0x7f7cac0afaf0 2024-04-24T06:30:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.002+0000 7f7ca8a00640 1 -- 172.21.15.59:0/2595589013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c98032440 con 0x7f7cac0afaf0 2024-04-24T06:30:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.002+0000 7f7ca8a00640 1 -- 172.21.15.59:0/2595589013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98032810 con 0x7f7cac0afaf0 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 -- 172.21.15.59:0/2595589013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 msgr2=0x7f7cac0d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/2595589013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac0d01a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009870 tx=0x7f7c9802a2a0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 -- 172.21.15.59:0/2595589013 shutdown_connections 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/2595589013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac0d01a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 -- 172.21.15.59:0/2595589013 >> 172.21.15.59:0/2595589013 conn(0x7f7cac067ea0 msgr2=0x7f7cac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 -- 172.21.15.59:0/2595589013 shutdown_connections 2024-04-24T06:30:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.003+0000 7f7cb0e00640 1 -- 172.21.15.59:0/2595589013 wait complete. 2024-04-24T06:30:43.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.005+0000 7f7cb0e00640 1 Processor -- start 2024-04-24T06:30:43.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.005+0000 7f7cb0e00640 1 -- start start 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cac0d06e0 0x7f7cac15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cac15c5e0 0x7f7cac15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac06e600 con 0x7f7cac0afaf0 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac06e480 con 0x7f7cac0d06e0 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7cb0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7cac162be0 con 0x7f7cac15c5e0 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59700/0 (socket says 172.21.15.59:59700) 2024-04-24T06:30:43.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.006+0000 7f7ca9e00640 1 -- 172.21.15.59:0/4075279861 learned_addr learned my addr 172.21.15.59:0/4075279861 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:43.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 -- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cac15c5e0 msgr2=0x7f7cac15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cac15c5e0 0x7f7cac15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 -- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cac0d06e0 msgr2=0x7f7cac15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cac0d06e0 0x7f7cac15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 -- 172.21.15.59:0/4075279861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac0cd600 con 0x7f7cac0afaf0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca9e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b3f0 tx=0x7f7c9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c015020 con 0x7f7cac0afaf0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9c011830 con 0x7f7cac0afaf0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011c20 con 0x7f7cac0afaf0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac15a270 con 0x7f7cac0afaf0 2024-04-24T06:30:43.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.007+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac15a830 con 0x7f7cac0afaf0 2024-04-24T06:30:43.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.008+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac068a80 con 0x7f7cac0afaf0 2024-04-24T06:30:43.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.010+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c9c018020 con 0x7f7cac0afaf0 2024-04-24T06:30:43.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.011+0000 7f7ca2000640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 0x7f7c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.011+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c08b850 con 0x7f7cac0afaf0 2024-04-24T06:30:43.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.011+0000 7f7ca9400640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 0x7f7c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.012+0000 7f7ca9400640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 0x7f7c8c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034000 tx=0x7f7c9802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.016+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c9c01f030 con 0x7f7cac0afaf0 2024-04-24T06:30:43.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.260+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cac1ea720 con 0x7f7cac0afaf0 2024-04-24T06:30:43.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.262+0000 7f7ca2000640 1 -- 172.21.15.59:0/4075279861 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f7c9c0172c0 con 0x7f7cac0afaf0 2024-04-24T06:30:43.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:43.264 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:43.264 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 msgr2=0x7f7c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 0x7f7c8c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034000 tx=0x7f7c9802c000 comp rx=0 tx=0).stop 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 msgr2=0x7f7cac148050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b3f0 tx=0x7f7c9c00b8b0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 shutdown_connections 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7c8c06c8f0 0x7f7c8c06edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7cac15c5e0 0x7f7cac15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cac0d06e0 0x7f7cac15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 --2- 172.21.15.59:0/4075279861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cac0afaf0 0x7f7cac148050 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 >> 172.21.15.59:0/4075279861 conn(0x7f7cac067ea0 msgr2=0x7f7cac0b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 shutdown_connections 2024-04-24T06:30:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.265+0000 7f7cb0e00640 1 -- 172.21.15.59:0/4075279861 wait complete. 2024-04-24T06:30:43.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.489+0000 7fb97da00640 1 Processor -- start 2024-04-24T06:30:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.490+0000 7fb97da00640 1 -- start start 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.491+0000 7fb97da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb97815e7c0 0x7fb9780d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.491+0000 7fb97da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb9780d0d90 0x7fb9780d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.491+0000 7fb97da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9780cf030 con 0x7fb9780d2630 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.491+0000 7fb97da00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb9780d1740 con 0x7fb97815e7c0 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.491+0000 7fb97da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780b7000 con 0x7fb9780d0d90 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb97815e7c0 0x7fb9780d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb9780d2630 0x7fb9780b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45490/0 (socket says 172.21.15.59:45490) 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977400640 1 -- 172.21.15.59:0/682669585 learned_addr learned my addr 172.21.15.59:0/682669585 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb97d000640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb9780d0d90 0x7fb9780d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977e00640 1 -- 172.21.15.59:0/682669585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb9780d0d90 msgr2=0x7fb9780d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977e00640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb9780d0d90 0x7fb9780d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977e00640 1 -- 172.21.15.59:0/682669585 >> v1:172.21.15.59:6789/0 conn(0x7fb9780d2630 legacy=0x7fb9780b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.492+0000 7fb977e00640 1 -- 172.21.15.59:0/682669585 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9780b6a40 con 0x7fb97815e7c0 2024-04-24T06:30:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.493+0000 7fb977e00640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb97815e7c0 0x7fb9780d20f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9600097a0 tx=0x7fb96002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f47cc5968700a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb976a00640 1 -- 172.21.15.59:0/682669585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960037070 con 0x7fb97815e7c0 2024-04-24T06:30:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb976a00640 1 -- 172.21.15.59:0/682669585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960032440 con 0x7fb97815e7c0 2024-04-24T06:30:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb976a00640 1 -- 172.21.15.59:0/682669585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960032810 con 0x7fb97815e7c0 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 -- 172.21.15.59:0/682669585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb97815e7c0 msgr2=0x7fb9780d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb97815e7c0 0x7fb9780d20f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9600097a0 tx=0x7fb96002a180 comp rx=0 tx=0).stop 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 -- 172.21.15.59:0/682669585 shutdown_connections 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb9780d0d90 0x7fb9780d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 --2- 172.21.15.59:0/682669585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb97815e7c0 0x7fb9780d20f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.494+0000 7fb97da00640 1 -- 172.21.15.59:0/682669585 >> 172.21.15.59:0/682669585 conn(0x7fb97809d920 msgr2=0x7fb97808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.495+0000 7fb97da00640 1 -- 172.21.15.59:0/682669585 shutdown_connections 2024-04-24T06:30:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.495+0000 7fb97da00640 1 -- 172.21.15.59:0/682669585 wait complete. 2024-04-24T06:30:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.496+0000 7fb97da00640 1 Processor -- start 2024-04-24T06:30:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.496+0000 7fb97da00640 1 -- start start 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9780d0d90 0x7fb97815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb97815e7c0 0x7fb9780c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780cf2b0 con 0x7fb9780d2630 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780cf620 con 0x7fb9780d0d90 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb97da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780b6700 con 0x7fb97815e7c0 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.497+0000 7fb977400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59716/0 (socket says 172.21.15.59:59716) 2024-04-24T06:30:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 -- 172.21.15.59:0/318257318 learned_addr learned my addr 172.21.15.59:0/318257318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb97d000640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb97815e7c0 0x7fb9780c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 -- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb97815e7c0 msgr2=0x7fb9780c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb97815e7c0 0x7fb9780c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 -- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9780d0d90 msgr2=0x7fb97815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9780d0d90 0x7fb97815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 -- 172.21.15.59:0/318257318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97815c220 con 0x7fb9780d2630 2024-04-24T06:30:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb977400640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb96800cae0 tx=0x7fb96800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.498+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968017070 con 0x7fb9780d2630 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.499+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96800ac40 con 0x7fb9780d2630 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.499+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968012620 con 0x7fb9780d2630 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.499+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97815c4c0 con 0x7fb9780d2630 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.499+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97815ca80 con 0x7fb9780d2630 2024-04-24T06:30:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.500+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb978161af0 con 0x7fb9780d2630 2024-04-24T06:30:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.501+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb96801a070 con 0x7fb9780d2630 2024-04-24T06:30:43.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.502+0000 7fb96fe00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 0x7fb95406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.502+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb96808b830 con 0x7fb9780d2630 2024-04-24T06:30:43.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.502+0000 7fb977e00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 0x7fb95406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.503+0000 7fb977e00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 0x7fb95406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb960034000 tx=0x7fb96002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.506+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb978161af0 con 0x7fb9780d2630 2024-04-24T06:30:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.739+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9780d2c90 con 0x7fb9780d2630 2024-04-24T06:30:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.741+0000 7fb96fe00640 1 -- 172.21.15.59:0/318257318 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fb9780d2c90 con 0x7fb9780d2630 2024-04-24T06:30:43.743 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:43.743 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:04.333230+0000","modified":"2024-04-24T06:30:07.467202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.66:6833/2735852616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2735852616},{"type":"v1","addr":"172.21.15.66:6833","nonce":2735852616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-24T06:30:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-04-24T06:30:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 msgr2=0x7fb95406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 0x7fb95406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb960034000 tx=0x7fb96002c000 comp rx=0 tx=0).stop 2024-04-24T06:30:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 msgr2=0x7fb9780ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb96800cae0 tx=0x7fb96800a060 comp rx=0 tx=0).stop 2024-04-24T06:30:43.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 shutdown_connections 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb95406c8f0 0x7fb95406edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb97815e7c0 0x7fb9780c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9780d2630 0x7fb9780ca7c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 --2- 172.21.15.59:0/318257318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9780d0d90 0x7fb97815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 >> 172.21.15.59:0/318257318 conn(0x7fb97809d920 msgr2=0x7fb97806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 shutdown_connections 2024-04-24T06:30:43.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.743+0000 7fb97da00640 1 -- 172.21.15.59:0/318257318 wait complete. 2024-04-24T06:30:43.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.972+0000 7f5b34a00640 1 Processor -- start 2024-04-24T06:30:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.972+0000 7f5b34a00640 1 -- start start 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.973+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b34a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b300b4690 0x7f5b300cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b34a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b30160210 con 0x7f5b3015c3d0 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b34a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5b301604f0 con 0x7f5b300b4690 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b34a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5b301607d0 con 0x7f5b300cd760 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b300b4690 0x7f5b300cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5b3015c3d0 0x7f5b300cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45496/0 (socket says 172.21.15.59:45496) 2024-04-24T06:30:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2e800640 1 -- 172.21.15.59:0/2528147914 learned_addr learned my addr 172.21.15.59:0/2528147914 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2de00640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2f200640 1 -- 172.21.15.59:0/2528147914 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 msgr2=0x7f5b300b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2f200640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2f200640 1 -- 172.21.15.59:0/2528147914 >> v1:172.21.15.59:6789/0 conn(0x7f5b3015c3d0 legacy=0x7f5b300cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2f200640 1 -- 172.21.15.59:0/2528147914 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b30160ab0 con 0x7f5b300b4690 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2de00640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.974+0000 7f5b2d400640 1 -- 172.21.15.59:0/2528147914 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650222547 0 0) 0x7f5b30160210 con 0x7f5b3015c3d0 2024-04-24T06:30:43.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b2f200640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b300cb7d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009d90 tx=0x7f5b2802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90efdb81a2ab1825 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b2d400640 1 -- 172.21.15.59:0/2528147914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28007740 con 0x7f5b300b4690 2024-04-24T06:30:43.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b2d400640 1 -- 172.21.15.59:0/2528147914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b280078e0 con 0x7f5b300b4690 2024-04-24T06:30:43.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b2d400640 1 -- 172.21.15.59:0/2528147914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28007cb0 con 0x7f5b300b4690 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b34a00640 1 -- 172.21.15.59:0/2528147914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 msgr2=0x7f5b300cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.976+0000 7f5b34a00640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b300cb7d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009d90 tx=0x7f5b2802a660 comp rx=0 tx=0).stop 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 -- 172.21.15.59:0/2528147914 shutdown_connections 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b300cb7d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 --2- 172.21.15.59:0/2528147914 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 -- 172.21.15.59:0/2528147914 >> 172.21.15.59:0/2528147914 conn(0x7f5b300b21a0 msgr2=0x7f5b300b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 -- 172.21.15.59:0/2528147914 shutdown_connections 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.977+0000 7f5b34a00640 1 -- 172.21.15.59:0/2528147914 wait complete. 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.979+0000 7f5b34a00640 1 Processor -- start 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.979+0000 7f5b34a00640 1 -- start start 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b3015c3d0 0x7f5b300d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3015c7b0 con 0x7f5b300b4690 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b300b4ac0 con 0x7f5b3015c3d0 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b34a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3015c930 con 0x7f5b300cd760 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b2e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b2e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59726/0 (socket says 172.21.15.59:59726) 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.980+0000 7f5b2e800640 1 -- 172.21.15.59:0/3395255888 learned_addr learned my addr 172.21.15.59:0/3395255888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 -- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 msgr2=0x7f5b300cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 -- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b3015c3d0 msgr2=0x7f5b300d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b3015c3d0 0x7f5b300d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 -- 172.21.15.59:0/3395255888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b300ce6f0 con 0x7f5b300b4690 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b2e800640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c430 tx=0x7f5b2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200074e0 con 0x7f5b300b4690 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b200047e0 con 0x7f5b300b4690 2024-04-24T06:30:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004bd0 con 0x7f5b300b4690 2024-04-24T06:30:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b300ce9d0 con 0x7f5b300b4690 2024-04-24T06:30:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.981+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b300cef90 con 0x7f5b300b4690 2024-04-24T06:30:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.982+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3004fd90 con 0x7f5b300b4690 2024-04-24T06:30:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.983+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b20011070 con 0x7f5b300b4690 2024-04-24T06:30:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.984+0000 7f5b26a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 0x7f5b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.984+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5b2008b820 con 0x7f5b300b4690 2024-04-24T06:30:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.985+0000 7f5b2de00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 0x7f5b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.985+0000 7f5b2de00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 0x7f5b1006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c006fd0 tx=0x7f5b1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:43.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:43.989+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b2000e280 con 0x7f5b300b4690 2024-04-24T06:30:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.186+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b3006e400 con 0x7f5b300b4690 2024-04-24T06:30:44.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.188+0000 7f5b26a00640 1 -- 172.21.15.59:0/3395255888 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f5b20059bb0 con 0x7f5b300b4690 2024-04-24T06:30:44.190 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:44.190 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:05.908925+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:02.277562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:03.324428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 msgr2=0x7f5b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 0x7f5b1006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c006fd0 tx=0x7f5b1c008040 comp rx=0 tx=0).stop 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 msgr2=0x7f5b3009dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c430 tx=0x7f5b2000c8f0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 shutdown_connections 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5b1006c8f0 0x7f5b1006edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b3015c3d0 0x7f5b300d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b300cd760 0x7f5b300cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 --2- 172.21.15.59:0/3395255888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b300b4690 0x7f5b3009dd20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 >> 172.21.15.59:0/3395255888 conn(0x7f5b300b21a0 msgr2=0x7f5b3006e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 shutdown_connections 2024-04-24T06:30:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.191+0000 7f5b34a00640 1 -- 172.21.15.59:0/3395255888 wait complete. 2024-04-24T06:30:44.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:30:44.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.420+0000 7fd19a400640 1 Processor -- start 2024-04-24T06:30:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.421+0000 7fd19a400640 1 -- start start 2024-04-24T06:30:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.422+0000 7fd19a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd1940cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.422+0000 7fd19a400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1940cf930 con 0x7fd1940d0f80 2024-04-24T06:30:44.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.422+0000 7fd19a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1940cfc10 con 0x7fd19415f880 2024-04-24T06:30:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.422+0000 7fd19a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1940cfef0 con 0x7fd19415ea40 2024-04-24T06:30:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.423+0000 7fd199000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd1940d0f80 0x7fd1940d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45508/0 (socket says 172.21.15.59:45508) 2024-04-24T06:30:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.423+0000 7fd199000640 1 -- 172.21.15.59:0/3786946628 learned_addr learned my addr 172.21.15.59:0/3786946628 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.423+0000 7fd193e00640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd1940cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.423+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2240085141 0 0) 0x7fd1940cf930 con 0x7fd1940d0f80 2024-04-24T06:30:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.423+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd178003520 con 0x7fd1940d0f80 2024-04-24T06:30:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3697342879 0 0) 0x7fd178003520 con 0x7fd1940d0f80 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 msgr2=0x7fd1940cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd1940cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 >> v1:172.21.15.66:6789/0 conn(0x7fd19415f880 legacy=0x7fd1940d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1940d01d0 con 0x7fd1940d0f80 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd184001670 con 0x7fd1940d0f80 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 >> v1:172.21.15.59:6789/0 conn(0x7fd1940d0f80 legacy=0x7fd1940d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd178003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd178004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd178004d40 0x7fd1780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cfef0 con 0x7fd1780039e0 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd17800bb50 con 0x7fd178004d40 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd17800bd50 con 0x7fd19415ea40 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd199000640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd178003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd193e00640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd178004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd199000640 1 -- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 msgr2=0x7fd178004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.424+0000 7fd199000640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415ea40 0x7fd178004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.425+0000 7fd199000640 1 -- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd178004d40 msgr2=0x7fd1780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.425+0000 7fd199000640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd178004d40 0x7fd1780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.425+0000 7fd199000640 1 -- 172.21.15.59:0/3786946628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd178003520 con 0x7fd1780039e0 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd199000640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd178003e00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c190 tx=0x7fd178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6883a63196ad9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007c60 con 0x7fd1780039e0 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd184005af0 con 0x7fd1780039e0 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd193400640 1 -- 172.21.15.59:0/3786946628 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007070 con 0x7fd1780039e0 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd19a400640 1 -- 172.21.15.59:0/3786946628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 msgr2=0x7fd178003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd19a400640 1 --2- 172.21.15.59:0/3786946628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd178003e00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c190 tx=0x7fd178005830 comp rx=0 tx=0).stop 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.426+0000 7fd199a00640 1 -- 172.21.15.59:0/3786946628 reap_dead start 2024-04-24T06:30:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.427+0000 7fd19a400640 1 -- 172.21.15.59:0/3786946628 shutdown_connections 2024-04-24T06:30:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.427+0000 7fd19a400640 1 -- 172.21.15.59:0/3786946628 >> 172.21.15.59:0/3786946628 conn(0x7fd19408bf90 msgr2=0x7fd1940d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.427+0000 7fd19a400640 1 -- 172.21.15.59:0/3786946628 shutdown_connections 2024-04-24T06:30:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.427+0000 7fd19a400640 1 -- 172.21.15.59:0/3786946628 wait complete. 2024-04-24T06:30:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.428+0000 7fd19a400640 1 Processor -- start 2024-04-24T06:30:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 -- start start 2024-04-24T06:30:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 0x7fd194158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415b7d0 0x7fd194158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cf250 con 0x7fd1780039e0 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd19415fd00 con 0x7fd19415b350 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.429+0000 7fd19a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1940cf3d0 con 0x7fd19415b7d0 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59752/0 (socket says 172.21.15.59:59752) 2024-04-24T06:30:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 -- 172.21.15.59:0/1923392628 learned_addr learned my addr 172.21.15.59:0/1923392628 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:44.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199a00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415b7d0 0x7fd194158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd193e00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 0x7fd194158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 -- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415b7d0 msgr2=0x7fd194158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415b7d0 0x7fd194158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 -- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 msgr2=0x7fd194158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 0x7fd194158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd199000640 1 -- 172.21.15.59:0/1923392628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19415bda0 con 0x7fd1780039e0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.430+0000 7fd193e00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 0x7fd194158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd199000640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c190 tx=0x7fd184001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184044070 con 0x7fd1780039e0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd184038040 con 0x7fd1780039e0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184004c20 con 0x7fd1780039e0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19415c080 con 0x7fd1780039e0 2024-04-24T06:30:44.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.431+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19415c640 con 0x7fd1780039e0 2024-04-24T06:30:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.432+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19404fd90 con 0x7fd1780039e0 2024-04-24T06:30:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.433+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd184007070 con 0x7fd1780039e0 2024-04-24T06:30:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.434+0000 7fd190c00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.434+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd1840ab630 con 0x7fd1780039e0 2024-04-24T06:30:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.434+0000 7fd193e00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.435+0000 7fd193e00640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd1940cb610 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.439+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1840799d0 con 0x7fd1780039e0 2024-04-24T06:30:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.689+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd19406ead0 con 0x7fd1780039e0 2024-04-24T06:30:44.692 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:30:44.690+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:30:44.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.694+0000 7fd190c00640 1 -- 172.21.15.59:0/1923392628 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fd1840792d0 con 0x7fd1780039e0 2024-04-24T06:30:44.696 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.697+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 msgr2=0x7fd17406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.697+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd1940cb610 tx=0x7fd180008040 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.697+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 msgr2=0x7fd1940cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.697+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c190 tx=0x7fd184001970 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.697+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 shutdown_connections 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd19415b7d0 0x7fd194158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd19415b350 0x7fd194158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 --2- 172.21.15.59:0/1923392628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1780039e0 0x7fd1940cde80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 >> 172.21.15.59:0/1923392628 conn(0x7fd19408bf90 msgr2=0x7fd1940d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 shutdown_connections 2024-04-24T06:30:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.698+0000 7fd19a400640 1 -- 172.21.15.59:0/1923392628 wait complete. 2024-04-24T06:30:44.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:30:44.736 INFO:tasks.ceph.mds.d.smithi066.stderr: -6247> 2024-04-24T06:30:07.726+0000 7f732f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:30:44.826 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:30:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.936+0000 7f47c6e00640 1 Processor -- start 2024-04-24T06:30:44.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.937+0000 7f47c6e00640 1 -- start start 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c6e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f47c006c0e0 0x7f47c00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c6e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c00b2ca0 con 0x7f47c00ce860 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c6e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c00cdaa0 con 0x7f47c015c940 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c6e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f47c00cdd80 con 0x7f47c006c0e0 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c5000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f47c00ce860 0x7f47c006b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45510/0 (socket says 172.21.15.59:45510) 2024-04-24T06:30:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.938+0000 7f47c5000640 1 -- 172.21.15.59:0/2921128435 learned_addr learned my addr 172.21.15.59:0/2921128435 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362498933 0 0) 0x7f47c00b2ca0 con 0x7f47c00ce860 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47c6400640 1 --2- 172.21.15.59:0/2921128435 >> v2:172.21.15.66:3300/0 conn(0x7f47c006c0e0 0x7f47c00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47a8003520 con 0x7f47c00ce860 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2377260083 0 0) 0x7f47a8003520 con 0x7f47c00ce860 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 >> v2:172.21.15.66:3300/0 conn(0x7f47c006c0e0 msgr2=0x7f47c00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 --2- 172.21.15.59:0/2921128435 >> v2:172.21.15.66:3300/0 conn(0x7f47c006c0e0 0x7f47c00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 >> v1:172.21.15.66:6789/0 conn(0x7f47c015c940 legacy=0x7f47c00b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00ce060 con 0x7f47c00ce860 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f47ac001670 con 0x7f47c00ce860 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 >> v1:172.21.15.59:6789/0 conn(0x7f47c00ce860 legacy=0x7f47c006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47a80039b0 0x7f47a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47a80051c0 0x7f47a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00cdd80 con 0x7f47a8004800 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47a800c110 con 0x7f47a80051c0 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.939+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f47a800c310 con 0x7f47a80039b0 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5a00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47a80039b0 0x7f47a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c6400640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47a80051c0 0x7f47a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 -- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47a80039b0 msgr2=0x7f47a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47a80039b0 0x7f47a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 -- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47a80051c0 msgr2=0x7f47a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47a80051c0 0x7f47a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c5000640 1 -- 172.21.15.59:0/2921128435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a8003520 con 0x7f47a8004800 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c6400640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47a80051c0 0x7f47a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.940+0000 7f47c6400640 1 -- 172.21.15.59:0/2921128435 reap_dead start 2024-04-24T06:30:44.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.941+0000 7f47c5000640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47a8004c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c210 tx=0x7f47a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511a6f7cf8b28aec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.942+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac007d30 con 0x7f47a8004800 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.942+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac005ad0 con 0x7f47a8004800 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.942+0000 7f47bfe00640 1 -- 172.21.15.59:0/2921128435 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac007070 con 0x7f47a8004800 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.942+0000 7f47c6e00640 1 -- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 msgr2=0x7f47a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.942+0000 7f47c6e00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47a8004c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c210 tx=0x7f47a8005830 comp rx=0 tx=0).stop 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.943+0000 7f47c6e00640 1 -- 172.21.15.59:0/2921128435 shutdown_connections 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.943+0000 7f47c6e00640 1 --2- 172.21.15.59:0/2921128435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.943+0000 7f47c6e00640 1 -- 172.21.15.59:0/2921128435 >> 172.21.15.59:0/2921128435 conn(0x7f47c00b2f10 msgr2=0x7f47c00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.943+0000 7f47c6e00640 1 -- 172.21.15.59:0/2921128435 shutdown_connections 2024-04-24T06:30:44.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.943+0000 7f47c6e00640 1 -- 172.21.15.59:0/2921128435 wait complete. 2024-04-24T06:30:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.944+0000 7f47c6e00640 1 Processor -- start 2024-04-24T06:30:44.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.945+0000 7f47c6e00640 1 -- start start 2024-04-24T06:30:44.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.945+0000 7f47c6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 0x7f47c015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47c015f430 0x7f47c00aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47c006b9d0 con 0x7f47a8004800 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00c4ba0 con 0x7f47c015b3a0 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f47c006bb50 con 0x7f47c015f430 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59782/0 (socket says 172.21.15.59:59782) 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 -- 172.21.15.59:0/454328133 learned_addr learned my addr 172.21.15.59:0/454328133 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c6400640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47c015f430 0x7f47c00aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5000640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 0x7f47c015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 -- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47c015f430 msgr2=0x7f47c00aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47c015f430 0x7f47c00aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 -- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 msgr2=0x7f47c015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 0x7f47c015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:44.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5a00640 1 -- 172.21.15.59:0/454328133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00af8a0 con 0x7f47a8004800 2024-04-24T06:30:44.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.946+0000 7f47c5000640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 0x7f47c015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:44.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47c5a00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e500 tx=0x7f47b400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40053a0 con 0x7f47a8004800 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b4018070 con 0x7f47a8004800 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4003810 con 0x7f47a8004800 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c00afb80 con 0x7f47a8004800 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.947+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c00b0140 con 0x7f47a8004800 2024-04-24T06:30:44.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.948+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c004fd90 con 0x7f47a8004800 2024-04-24T06:30:44.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.949+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b40039b0 con 0x7f47a8004800 2024-04-24T06:30:44.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.950+0000 7f47bd600640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:44.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.951+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f47b408b980 con 0x7f47a8004800 2024-04-24T06:30:44.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.951+0000 7f47c5000640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.951+0000 7f47c5000640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 0x7f47a006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47ac033000 tx=0x7f47ac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:44.955+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47b4059c30 con 0x7f47a8004800 2024-04-24T06:30:45.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.207+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f47c00c8070 con 0x7f47a8004800 2024-04-24T06:30:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.705+0000 7f47bd600640 1 -- 172.21.15.59:0/454328133 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f47b4059530 con 0x7f47a8004800 2024-04-24T06:30:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 msgr2=0x7f47a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 0x7f47a006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47ac033000 tx=0x7f47ac02e000 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 msgr2=0x7f47c00b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e500 tx=0x7f47b400e9c0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 shutdown_connections 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47a006c8f0 0x7f47a006edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f47c015f430 0x7f47c00aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47c015b3a0 0x7f47c015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 --2- 172.21.15.59:0/454328133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47a8004800 0x7f47c00b0940 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.708+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 >> 172.21.15.59:0/454328133 conn(0x7f47c00b2f10 msgr2=0x7f47c0074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.709+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 shutdown_connections 2024-04-24T06:30:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.709+0000 7f47c6e00640 1 -- 172.21.15.59:0/454328133 wait complete. 2024-04-24T06:30:45.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:30:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.956+0000 7f4ac1600640 1 Processor -- start 2024-04-24T06:30:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.956+0000 7f4ac1600640 1 -- start start 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4ac1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4abc0d4210 0x7f4abc0d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4ac1600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc0cf070 con 0x7f4abc15e8e0 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4ac1600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4abc0d4c00 con 0x7f4abc06cbb0 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4ac1600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4abc0d04c0 con 0x7f4abc0d4210 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4abbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4abc15e8e0 0x7f4abc0cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45514/0 (socket says 172.21.15.59:45514) 2024-04-24T06:30:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.958+0000 7f4abbe00640 1 -- 172.21.15.59:0/3301947689 learned_addr learned my addr 172.21.15.59:0/3301947689 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abb400640 1 --2- 172.21.15.59:0/3301947689 >> v2:172.21.15.66:3300/0 conn(0x7f4abc0d4210 0x7f4abc0d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405013482 0 0) 0x7f4aac001110 con 0x7f4abc15e8e0 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a9c003520 con 0x7f4abc15e8e0 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 953840992 0 0) 0x7f4a9c003520 con 0x7f4abc15e8e0 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 >> v2:172.21.15.66:3300/0 conn(0x7f4abc0d4210 msgr2=0x7f4abc0d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 --2- 172.21.15.59:0/3301947689 >> v2:172.21.15.66:3300/0 conn(0x7f4abc0d4210 0x7f4abc0d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 >> v1:172.21.15.66:6789/0 conn(0x7f4abc06cbb0 legacy=0x7f4abc06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc0cf2f0 con 0x7f4abc15e8e0 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4aac001840 con 0x7f4abc15e8e0 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 >> v1:172.21.15.59:6789/0 conn(0x7f4abc15e8e0 legacy=0x7f4abc0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c003a00 0x7f4a9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 0x7f4a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4a9c005210 0x7f4a9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0d04c0 con 0x7f4a9c004800 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c00c140 con 0x7f4a9c003a00 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c00c340 con 0x7f4a9c005210 2024-04-24T06:30:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.959+0000 7f4abb400640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 0x7f4a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4ac0c00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4a9c005210 0x7f4a9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abbe00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c003a00 0x7f4a9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abb400640 1 -- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4a9c005210 msgr2=0x7f4a9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abb400640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4a9c005210 0x7f4a9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abb400640 1 -- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c003a00 msgr2=0x7f4a9c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abb400640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c003a00 0x7f4a9c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abb400640 1 -- 172.21.15.59:0/3301947689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c003520 con 0x7f4a9c004800 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4abbe00640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c003a00 0x7f4a9c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.960+0000 7f4ac0c00640 1 -- 172.21.15.59:0/3301947689 reap_dead start 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.961+0000 7f4abb400640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 0x7f4a9c004c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400b9e0 tx=0x7f4a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f914abea13f64145 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.961+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400b5e0 con 0x7f4a9c004800 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.961+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa402ad70 con 0x7f4a9c004800 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.961+0000 7f4abaa00640 1 -- 172.21.15.59:0/3301947689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4005bb0 con 0x7f4a9c004800 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 -- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 msgr2=0x7f4a9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 0x7f4a9c004c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400b9e0 tx=0x7f4a9c005830 comp rx=0 tx=0).stop 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 -- 172.21.15.59:0/3301947689 shutdown_connections 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 --2- 172.21.15.59:0/3301947689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a9c004800 0x7f4a9c004c80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 -- 172.21.15.59:0/3301947689 >> 172.21.15.59:0/3301947689 conn(0x7f4abc08e060 msgr2=0x7f4abc09d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 -- 172.21.15.59:0/3301947689 shutdown_connections 2024-04-24T06:30:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.962+0000 7f4ac1600640 1 -- 172.21.15.59:0/3301947689 wait complete. 2024-04-24T06:30:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.964+0000 7f4ac1600640 1 Processor -- start 2024-04-24T06:30:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.964+0000 7f4ac1600640 1 -- start start 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c004800 0x7f4abc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4abc158a20 0x7f4abc15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0cf890 con 0x7f4abc15d1f0 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc0cf5b0 con 0x7f4a9c004800 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc15ecc0 con 0x7f4abc158a20 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59808/0 (socket says 172.21.15.59:59808) 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.965+0000 7f4ac0c00640 1 -- 172.21.15.59:0/2472662237 learned_addr learned my addr 172.21.15.59:0/2472662237 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4abb400640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4abc158a20 0x7f4abc15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 -- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4abc158a20 msgr2=0x7f4abc15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4abc158a20 0x7f4abc15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 -- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c004800 msgr2=0x7f4abc0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c004800 0x7f4abc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 -- 172.21.15.59:0/2472662237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc0cc620 con 0x7f4abc15d1f0 2024-04-24T06:30:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac0c00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0d20c0 tx=0x7f4ab400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4003970 con 0x7f4abc15d1f0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4018070 con 0x7f4abc15d1f0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4003d70 con 0x7f4abc15d1f0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.966+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc0cc900 con 0x7f4abc15d1f0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.967+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc0ccec0 con 0x7f4abc15d1f0 2024-04-24T06:30:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.968+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4abc04fd90 con 0x7f4abc15d1f0 2024-04-24T06:30:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.969+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ab40050d0 con 0x7f4abc15d1f0 2024-04-24T06:30:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.970+0000 7f4ab3e00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 0x7f4a9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.971+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4ab408b930 con 0x7f4abc15d1f0 2024-04-24T06:30:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.971+0000 7f4abbe00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 0x7f4a9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.971+0000 7f4abbe00640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 0x7f4a9806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4abc162210 tx=0x7f4aac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:45.976+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ab4059cf0 con 0x7f4abc15d1f0 2024-04-24T06:30:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.186+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4abc06c990 con 0x7f4abc15d1f0 2024-04-24T06:30:46.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.709+0000 7f4ab3e00640 1 -- 172.21.15.59:0/2472662237 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f4ab401be40 con 0x7f4abc15d1f0 2024-04-24T06:30:46.711 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:30:46.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 msgr2=0x7f4a9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:46.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 0x7f4a9806ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4abc162210 tx=0x7f4aac009040 comp rx=0 tx=0).stop 2024-04-24T06:30:46.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 msgr2=0x7f4abc0cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:46.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0d20c0 tx=0x7f4ab400ec10 comp rx=0 tx=0).stop 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 shutdown_connections 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a9806c920 0x7f4a9806ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4abc15d1f0 0x7f4abc0cb8d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4abc158a20 0x7f4abc15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 --2- 172.21.15.59:0/2472662237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a9c004800 0x7f4abc0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 >> 172.21.15.59:0/2472662237 conn(0x7f4abc08e060 msgr2=0x7f4abc06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 shutdown_connections 2024-04-24T06:30:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.713+0000 7f4ac1600640 1 -- 172.21.15.59:0/2472662237 wait complete. 2024-04-24T06:30:46.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:30:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.954+0000 7f65abe00640 1 Processor -- start 2024-04-24T06:30:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.954+0000 7f65abe00640 1 -- start start 2024-04-24T06:30:46.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.955+0000 7f65abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65ac15acf0 0x7f65ac073810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.955+0000 7f65abe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f65ac071cb0 0x7f65ac15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.955+0000 7f65abe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ac15b0d0 con 0x7f65ac15ed30 2024-04-24T06:30:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.955+0000 7f65abe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f65ac0b5830 con 0x7f65ac15acf0 2024-04-24T06:30:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.955+0000 7f65abe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f65ac073e10 con 0x7f65ac071cb0 2024-04-24T06:30:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65aaa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65ac15acf0 0x7f65ac073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65aaa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65ac15acf0 0x7f65ac073810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59814/0 (socket says 172.21.15.59:59814) 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65aaa00640 1 -- 172.21.15.59:0/177488090 learned_addr learned my addr 172.21.15.59:0/177488090 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65a3e00640 1 --2- 172.21.15.59:0/177488090 >> v2:172.21.15.66:3300/0 conn(0x7f65ac071cb0 0x7f65ac15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65aaa00640 1 -- 172.21.15.59:0/177488090 >> v2:172.21.15.66:3300/0 conn(0x7f65ac071cb0 msgr2=0x7f65ac15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.956+0000 7f65aaa00640 1 --2- 172.21.15.59:0/177488090 >> v2:172.21.15.66:3300/0 conn(0x7f65ac071cb0 0x7f65ac15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.957+0000 7f65aaa00640 1 -- 172.21.15.59:0/177488090 >> v1:172.21.15.66:6789/0 conn(0x7f65ac15ed30 legacy=0x7f65ac0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.957+0000 7f65aaa00640 1 -- 172.21.15.59:0/177488090 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac0b51e0 con 0x7f65ac15acf0 2024-04-24T06:30:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.958+0000 7f65aaa00640 1 --2- 172.21.15.59:0/177488090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac073810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f65ac06d3f0 tx=0x7f659c02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce509e0b6430f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.958+0000 7f65aa000640 1 -- 172.21.15.59:0/177488090 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c00b040 con 0x7f65ac15acf0 2024-04-24T06:30:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.958+0000 7f65aa000640 1 -- 172.21.15.59:0/177488090 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c031710 con 0x7f65ac15acf0 2024-04-24T06:30:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.958+0000 7f65aa000640 1 -- 172.21.15.59:0/177488090 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c031ae0 con 0x7f65ac15acf0 2024-04-24T06:30:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 -- 172.21.15.59:0/177488090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 msgr2=0x7f65ac073810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 --2- 172.21.15.59:0/177488090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac073810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f65ac06d3f0 tx=0x7f659c02c660 comp rx=0 tx=0).stop 2024-04-24T06:30:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 -- 172.21.15.59:0/177488090 shutdown_connections 2024-04-24T06:30:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 --2- 172.21.15.59:0/177488090 >> v2:172.21.15.66:3300/0 conn(0x7f65ac071cb0 0x7f65ac15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 --2- 172.21.15.59:0/177488090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac073810 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 -- 172.21.15.59:0/177488090 >> 172.21.15.59:0/177488090 conn(0x7f65ac0b1f90 msgr2=0x7f65ac0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 -- 172.21.15.59:0/177488090 shutdown_connections 2024-04-24T06:30:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.959+0000 7f65abe00640 1 -- 172.21.15.59:0/177488090 wait complete. 2024-04-24T06:30:46.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.962+0000 7f65abe00640 1 Processor -- start 2024-04-24T06:30:46.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.962+0000 7f65abe00640 1 -- start start 2024-04-24T06:30:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65ac071cb0 0x7f65ac13ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ac15ed30 0x7f65ac14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac072270 con 0x7f65ac15acf0 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac0720f0 con 0x7f65ac15ed30 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65abe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65ac0b4610 con 0x7f65ac071cb0 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.963+0000 7f65a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59822/0 (socket says 172.21.15.59:59822) 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 -- 172.21.15.59:0/981160312 learned_addr learned my addr 172.21.15.59:0/981160312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65ab400640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ac15ed30 0x7f65ac14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65aaa00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65ac071cb0 0x7f65ac13ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 -- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65ac071cb0 msgr2=0x7f65ac13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65ac071cb0 0x7f65ac13ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 -- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ac15ed30 msgr2=0x7f65ac14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ac15ed30 0x7f65ac14c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 -- 172.21.15.59:0/981160312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac15cf30 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.964+0000 7f65a3e00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f659400d830 tx=0x7f659400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.965+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659400b6c0 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.965+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594019070 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.965+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659400ba80 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.965+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac15d210 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.965+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac15d7d0 con 0x7f65ac15acf0 2024-04-24T06:30:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.966+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ac04fd90 con 0x7f65ac15acf0 2024-04-24T06:30:46.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.967+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f659401e460 con 0x7f65ac15acf0 2024-04-24T06:30:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.969+0000 7f65a2a00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:46.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.969+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f659408c140 con 0x7f65ac15acf0 2024-04-24T06:30:46.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.969+0000 7f65aaa00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:46.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.970+0000 7f65aaa00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 0x7f658806ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f659c036000 tx=0x7f659c00c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:46.973+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f659405a630 con 0x7f65ac15acf0 2024-04-24T06:30:47.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.176+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65ac158790 con 0x7f65ac15acf0 2024-04-24T06:30:47.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.712+0000 7f65a2a00640 1 -- 172.21.15.59:0/981160312 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f6594059f30 con 0x7f65ac15acf0 2024-04-24T06:30:47.715 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.716+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 msgr2=0x7f658806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 0x7f658806ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f659c036000 tx=0x7f659c00c040 comp rx=0 tx=0).stop 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 msgr2=0x7f65ac14c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f659400d830 tx=0x7f659400dcf0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 shutdown_connections 2024-04-24T06:30:47.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f658806c920 0x7f658806ede0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ac15ed30 0x7f65ac14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65ac15acf0 0x7f65ac14c340 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 --2- 172.21.15.59:0/981160312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65ac071cb0 0x7f65ac13ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 >> 172.21.15.59:0/981160312 conn(0x7f65ac0b1f90 msgr2=0x7f65ac154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 shutdown_connections 2024-04-24T06:30:47.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.717+0000 7f65abe00640 1 -- 172.21.15.59:0/981160312 wait complete. 2024-04-24T06:30:47.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:30:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.949+0000 7f50bea00640 1 Processor -- start 2024-04-24T06:30:47.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.950+0000 7f50bea00640 1 -- start start 2024-04-24T06:30:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50bea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50b80b4670 0x7f50b8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50bea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 0x7f50b80722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50bea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b80b35d0 con 0x7f50b8162c10 2024-04-24T06:30:47.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50bea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f50b80b38b0 con 0x7f50b80b4670 2024-04-24T06:30:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50bea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f50b80b3b90 con 0x7f50b80cfba0 2024-04-24T06:30:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50b7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50b80b4670 0x7f50b8071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50b7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50b8162c10 0x7f50b80d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45526/0 (socket says 172.21.15.59:45526) 2024-04-24T06:30:47.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.951+0000 7f50b7e00640 1 -- 172.21.15.59:0/1867305874 learned_addr learned my addr 172.21.15.59:0/1867305874 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:47.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50bca00640 1 --2- 172.21.15.59:0/1867305874 >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 0x7f50b80722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50b7400640 1 -- 172.21.15.59:0/1867305874 >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 msgr2=0x7f50b80722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:47.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50b7400640 1 --2- 172.21.15.59:0/1867305874 >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 0x7f50b80722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50b7400640 1 -- 172.21.15.59:0/1867305874 >> v1:172.21.15.59:6789/0 conn(0x7f50b8162c10 legacy=0x7f50b80d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50b7400640 1 -- 172.21.15.59:0/1867305874 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b80b3e70 con 0x7f50b80b4670 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50bca00640 1 --2- 172.21.15.59:0/1867305874 >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 0x7f50b80722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.952+0000 7f50b6a00640 1 -- 172.21.15.59:0/1867305874 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789299561 0 0) 0x7f50b80b35d0 con 0x7f50b8162c10 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.953+0000 7f50b7400640 1 --2- 172.21.15.59:0/1867305874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b80b4670 0x7f50b8071d70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f50b8070070 tx=0x7f50a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd64625a1d5b8d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.953+0000 7f50b6a00640 1 -- 172.21.15.59:0/1867305874 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4037070 con 0x7f50b80b4670 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.953+0000 7f50b6a00640 1 -- 172.21.15.59:0/1867305874 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a4032440 con 0x7f50b80b4670 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.953+0000 7f50b6a00640 1 -- 172.21.15.59:0/1867305874 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4032810 con 0x7f50b80b4670 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 -- 172.21.15.59:0/1867305874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b80b4670 msgr2=0x7f50b8071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 --2- 172.21.15.59:0/1867305874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b80b4670 0x7f50b8071d70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f50b8070070 tx=0x7f50a402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 -- 172.21.15.59:0/1867305874 shutdown_connections 2024-04-24T06:30:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 --2- 172.21.15.59:0/1867305874 >> v2:172.21.15.66:3300/0 conn(0x7f50b80cfba0 0x7f50b80722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 --2- 172.21.15.59:0/1867305874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b80b4670 0x7f50b8071d70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 -- 172.21.15.59:0/1867305874 >> 172.21.15.59:0/1867305874 conn(0x7f50b8067d90 msgr2=0x7f50b806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 -- 172.21.15.59:0/1867305874 shutdown_connections 2024-04-24T06:30:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.954+0000 7f50bea00640 1 -- 172.21.15.59:0/1867305874 wait complete. 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.956+0000 7f50bea00640 1 Processor -- start 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.956+0000 7f50bea00640 1 -- start start 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50b80b4670 0x7f50b8163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b80cfba0 0x7f50b8148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8163170 con 0x7f50b8162c10 2024-04-24T06:30:47.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8162ff0 con 0x7f50b80cfba0 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50b806ddd0 con 0x7f50b80b4670 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59846/0 (socket says 172.21.15.59:59846) 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.957+0000 7f50bca00640 1 -- 172.21.15.59:0/3364728439 learned_addr learned my addr 172.21.15.59:0/3364728439 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50b7e00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50b80b4670 0x7f50b8163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50b7400640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b80cfba0 0x7f50b8148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 -- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50b80b4670 msgr2=0x7f50b8163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50b80b4670 0x7f50b8163ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 -- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b80cfba0 msgr2=0x7f50b8148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b80cfba0 0x7f50b8148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 -- 172.21.15.59:0/3364728439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b81652c0 con 0x7f50b8162c10 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50bca00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f50b000ced0 tx=0x7f50b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.958+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0003b90 con 0x7f50b8162c10 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.959+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b0003d50 con 0x7f50b8162c10 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.959+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004260 con 0x7f50b8162c10 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.959+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b81655a0 con 0x7f50b8162c10 2024-04-24T06:30:47.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.959+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b8165b60 con 0x7f50b8162c10 2024-04-24T06:30:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.960+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b8068a80 con 0x7f50b8162c10 2024-04-24T06:30:47.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.961+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50b0004400 con 0x7f50b8162c10 2024-04-24T06:30:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.962+0000 7f50afe00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 0x7f509806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.963+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f50b008c200 con 0x7f50b8162c10 2024-04-24T06:30:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.963+0000 7f50b7e00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 0x7f509806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:47.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.963+0000 7f50b7e00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 0x7f509806ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000f60 tx=0x7f50a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:47.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:47.967+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50b005a9e0 con 0x7f50b8162c10 2024-04-24T06:30:48.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.168+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f50b806eb00 con 0x7f50b8162c10 2024-04-24T06:30:48.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.175+0000 7f50afe00640 1 -- 172.21.15.59:0/3364728439 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f50b005a2e0 con 0x7f50b8162c10 2024-04-24T06:30:48.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 msgr2=0x7f509806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 0x7f509806ede0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000f60 tx=0x7f50a8009040 comp rx=0 tx=0).stop 2024-04-24T06:30:48.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 msgr2=0x7f50b81649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f50b000ced0 tx=0x7f50b000a710 comp rx=0 tx=0).stop 2024-04-24T06:30:48.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 shutdown_connections 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f509806c920 0x7f509806ede0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b8162c10 0x7f50b81649c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b80cfba0 0x7f50b8148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 --2- 172.21.15.59:0/3364728439 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50b80b4670 0x7f50b8163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 >> 172.21.15.59:0/3364728439 conn(0x7f50b8067d90 msgr2=0x7f50b8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 shutdown_connections 2024-04-24T06:30:48.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.178+0000 7f50bea00640 1 -- 172.21.15.59:0/3364728439 wait complete. 2024-04-24T06:30:48.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-24T06:30:48.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.401+0000 7f7e91800640 1 Processor -- start 2024-04-24T06:30:48.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.401+0000 7f7e91800640 1 -- start start 2024-04-24T06:30:48.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e91800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e8c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e91800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c0cd520 con 0x7f7e8c0cd020 2024-04-24T06:30:48.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e91800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c0cd800 con 0x7f7e8c160910 2024-04-24T06:30:48.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e91800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c0cddc0 con 0x7f7e8c1454e0 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e82000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7e8c0cd020 0x7f7e8c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45534/0 (socket says 172.21.15.59:45534) 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.403+0000 7f7e82000640 1 -- 172.21.15.59:0/1017539460 learned_addr learned my addr 172.21.15.59:0/1017539460 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3137409087 0 0) 0x7f7e8c0cd520 con 0x7f7e8c0cd020 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e70003520 con 0x7f7e8c0cd020 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e8c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1625222396 0 0) 0x7f7e70003520 con 0x7f7e8c0cd020 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 msgr2=0x7f7e8c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e8c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 >> v1:172.21.15.66:6789/0 conn(0x7f7e8c160910 legacy=0x7f7e8c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c0cdae0 con 0x7f7e8c0cd020 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7e74001670 con 0x7f7e8c0cd020 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 >> v1:172.21.15.59:6789/0 conn(0x7f7e8c0cd020 legacy=0x7f7e8c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.404+0000 7f7e8a000640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e700039e0 0x7f7e70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8a000640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e70004800 0x7f7e70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8a000640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0cddc0 con 0x7f7e700039e0 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e700054f0 con 0x7f7e70004800 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7000bf00 con 0x7f7e8c1454e0 2024-04-24T06:30:48.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e700039e0 0x7f7e70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e82000640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e70004800 0x7f7e70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8b400640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 -- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 msgr2=0x7f7e7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e8c1454e0 0x7f7e7000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 -- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e70004800 msgr2=0x7f7e70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e70004800 0x7f7e70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.405+0000 7f7e8aa00640 1 -- 172.21.15.59:0/1017539460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e70003520 con 0x7f7e700039e0 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.406+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e700039e0 0x7f7e70003e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ba30 tx=0x7f7e70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8965eea060957460 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c030070 con 0x7f7e700039e0 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c02ade0 con 0x7f7e700039e0 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e8a000640 1 -- 172.21.15.59:0/1017539460 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00b600 con 0x7f7e700039e0 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e91800640 1 -- 172.21.15.59:0/1017539460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e700039e0 msgr2=0x7f7e70003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e91800640 1 --2- 172.21.15.59:0/1017539460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e700039e0 0x7f7e70003e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ba30 tx=0x7f7e70005830 comp rx=0 tx=0).stop 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e8b400640 1 -- 172.21.15.59:0/1017539460 reap_dead start 2024-04-24T06:30:48.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e91800640 1 -- 172.21.15.59:0/1017539460 shutdown_connections 2024-04-24T06:30:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.407+0000 7f7e91800640 1 -- 172.21.15.59:0/1017539460 >> 172.21.15.59:0/1017539460 conn(0x7f7e8c067ea0 msgr2=0x7f7e8c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.408+0000 7f7e91800640 1 -- 172.21.15.59:0/1017539460 shutdown_connections 2024-04-24T06:30:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.408+0000 7f7e91800640 1 -- 172.21.15.59:0/1017539460 wait complete. 2024-04-24T06:30:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.409+0000 7f7e91800640 1 Processor -- start 2024-04-24T06:30:48.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.410+0000 7f7e91800640 1 -- start start 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 0x7f7e8c1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e8c13ff40 0x7f7e8c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0b4010 con 0x7f7e8c143e50 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0d0a30 con 0x7f7e8c13ff40 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e91800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c1f8850 con 0x7f7e700039e0 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e82000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e82000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59866/0 (socket says 172.21.15.59:59866) 2024-04-24T06:30:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e82000640 1 -- 172.21.15.59:0/1295639481 learned_addr learned my addr 172.21.15.59:0/1295639481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 0x7f7e8c1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e8b400640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e8c13ff40 0x7f7e8c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e82000640 1 -- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 msgr2=0x7f7e8c1f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.411+0000 7f7e82000640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 0x7f7e8c1f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e82000640 1 -- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e8c13ff40 msgr2=0x7f7e8c0ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e82000640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e8c13ff40 0x7f7e8c0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e82000640 1 -- 172.21.15.59:0/1295639481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1f8b30 con 0x7f7e8c143e50 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 0x7f7e8c1f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e82000640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e74001050 tx=0x7f7e7400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400c870 con 0x7f7e8c143e50 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e74018070 con 0x7f7e8c143e50 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400cc70 con 0x7f7e8c143e50 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1426f0 con 0x7f7e8c143e50 2024-04-24T06:30:48.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.412+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1f8560 con 0x7f7e8c143e50 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.414+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c068a80 con 0x7f7e8c143e50 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.414+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e7401d460 con 0x7f7e8c143e50 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.415+0000 7f7e83400640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 0x7f7e6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.415+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f7e7408b7d0 con 0x7f7e8c143e50 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.416+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 0x7f7e6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.416+0000 7f7e8aa00640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 0x7f7e6806ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ba00 tx=0x7f7e7c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.420+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e74059e60 con 0x7f7e8c143e50 2024-04-24T06:30:48.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.696+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f7e8c160030 con 0x7f7e8c143e50 2024-04-24T06:30:48.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.726+0000 7f7e83400640 1 -- 172.21.15.59:0/1295639481 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v133) ==== 142+0+0 (secure 0 0 0) 0x7f7e74059760 con 0x7f7e8c143e50 2024-04-24T06:30:48.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 msgr2=0x7f7e6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 0x7f7e6806ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ba00 tx=0x7f7e7c030000 comp rx=0 tx=0).stop 2024-04-24T06:30:48.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 msgr2=0x7f7e8c142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e74001050 tx=0x7f7e7400ec80 comp rx=0 tx=0).stop 2024-04-24T06:30:48.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 shutdown_connections 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e6806c920 0x7f7e6806ede0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e8c13ff40 0x7f7e8c0ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e8c143e50 0x7f7e8c142ef0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 --2- 172.21.15.59:0/1295639481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e700039e0 0x7f7e8c1f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 >> 172.21.15.59:0/1295639481 conn(0x7f7e8c067ea0 msgr2=0x7f7e8c071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 shutdown_connections 2024-04-24T06:30:48.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.729+0000 7f7e91800640 1 -- 172.21.15.59:0/1295639481 wait complete. 2024-04-24T06:30:48.750 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-24T06:30:48.750 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-24T06:30:48.751 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:30:48.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.947+0000 7fc79cc00640 1 Processor -- start 2024-04-24T06:30:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.948+0000 7fc79cc00640 1 -- start start 2024-04-24T06:30:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.949+0000 7fc79cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc79815e7c0 0x7fc7980d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.949+0000 7fc79cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7980cf030 con 0x7fc7980d0d90 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.949+0000 7fc79cc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7980b6a40 con 0x7fc7980d2630 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.949+0000 7fc79cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc7980b7000 con 0x7fc79815e7c0 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc79815e7c0 0x7fc7980d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7980d0d90 0x7fc7980cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37782/0 (socket says 172.21.15.59:37782) 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797e00640 1 -- 172.21.15.59:0/1773289273 learned_addr learned my addr 172.21.15.59:0/1773289273 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797400640 1 -- 172.21.15.59:0/1773289273 >> v1:172.21.15.66:6789/0 conn(0x7fc7980d2630 legacy=0x7fc7980b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797400640 1 -- 172.21.15.59:0/1773289273 >> v1:172.21.15.59:6789/0 conn(0x7fc7980d0d90 legacy=0x7fc7980cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc797400640 1 -- 172.21.15.59:0/1773289273 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980b6d20 con 0x7fc79815e7c0 2024-04-24T06:30:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.950+0000 7fc796000640 1 -- 172.21.15.59:0/1773289273 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257669165 0 0) 0x7fc790001490 con 0x7fc7980d0d90 2024-04-24T06:30:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.951+0000 7fc797400640 1 --2- 172.21.15.59:0/1773289273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7980d20f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097a0 tx=0x7fc78802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3055107180045c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc796000640 1 -- 172.21.15.59:0/1773289273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788037070 con 0x7fc79815e7c0 2024-04-24T06:30:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc796000640 1 -- 172.21.15.59:0/1773289273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788032440 con 0x7fc79815e7c0 2024-04-24T06:30:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc796000640 1 -- 172.21.15.59:0/1773289273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788032810 con 0x7fc79815e7c0 2024-04-24T06:30:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 -- 172.21.15.59:0/1773289273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 msgr2=0x7fc7980d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 --2- 172.21.15.59:0/1773289273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7980d20f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc7880097a0 tx=0x7fc78802a230 comp rx=0 tx=0).stop 2024-04-24T06:30:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 -- 172.21.15.59:0/1773289273 shutdown_connections 2024-04-24T06:30:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 --2- 172.21.15.59:0/1773289273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7980d20f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 -- 172.21.15.59:0/1773289273 >> 172.21.15.59:0/1773289273 conn(0x7fc79809d920 msgr2=0x7fc79808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 -- 172.21.15.59:0/1773289273 shutdown_connections 2024-04-24T06:30:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.952+0000 7fc79cc00640 1 -- 172.21.15.59:0/1773289273 wait complete. 2024-04-24T06:30:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.954+0000 7fc79cc00640 1 Processor -- start 2024-04-24T06:30:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.954+0000 7fc79cc00640 1 -- start start 2024-04-24T06:30:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7980d0d90 0x7fc7980b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7980d2630 0x7fc79815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980b6700 con 0x7fc79815e7c0 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980cf620 con 0x7fc7980d0d90 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.955+0000 7fc79cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7980d1a80 con 0x7fc7980d2630 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58514/0 (socket says 172.21.15.59:58514) 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 -- 172.21.15.59:0/3243864778 learned_addr learned my addr 172.21.15.59:0/3243864778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc796a00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7980d2630 0x7fc79815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797400640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7980d0d90 0x7fc7980b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 -- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7980d2630 msgr2=0x7fc79815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7980d2630 0x7fc79815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 -- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7980d0d90 msgr2=0x7fc7980b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7980d0d90 0x7fc7980b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.956+0000 7fc797e00640 1 -- 172.21.15.59:0/3243864778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79815b700 con 0x7fc79815e7c0 2024-04-24T06:30:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc797e00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c7f0 tx=0x7fc79000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790018070 con 0x7fc79815e7c0 2024-04-24T06:30:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79000a5e0 con 0x7fc79815e7c0 2024-04-24T06:30:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79000aa10 con 0x7fc79815e7c0 2024-04-24T06:30:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79815b9e0 con 0x7fc79815e7c0 2024-04-24T06:30:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.957+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79815bfa0 con 0x7fc79815e7c0 2024-04-24T06:30:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.958+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79804fd90 con 0x7fc79815e7c0 2024-04-24T06:30:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.960+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc79000abb0 con 0x7fc79815e7c0 2024-04-24T06:30:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.960+0000 7fc78f400640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 0x7fc77806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.961+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc79008add0 con 0x7fc79815e7c0 2024-04-24T06:30:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.961+0000 7fc797400640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 0x7fc77806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.961+0000 7fc797400640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 0x7fc77806ede0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc788034000 tx=0x7fc78802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:48.965+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7980cf030 con 0x7fc79815e7c0 2024-04-24T06:30:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.171+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7980d1eb0 con 0x7fc79815e7c0 2024-04-24T06:30:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.172+0000 7fc78f400640 1 -- 172.21.15.59:0/3243864778 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12936 (secure 0 0 0) 0x7fc7900594f0 con 0x7fc79815e7c0 2024-04-24T06:30:49.174 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:49.175 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:47.709896+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6833/2735852616":"2024-04-25T06:30:44.691385+0000","172.21.15.66:6832/2735852616":"2024-04-25T06:30:44.691385+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 msgr2=0x7fc77806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 0x7fc77806ede0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc788034000 tx=0x7fc78802e000 comp rx=0 tx=0).stop 2024-04-24T06:30:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 msgr2=0x7fc7981598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c7f0 tx=0x7fc79000ccb0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 shutdown_connections 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc77806c920 0x7fc77806ede0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79815e7c0 0x7fc7981598a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7980d2630 0x7fc79815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 --2- 172.21.15.59:0/3243864778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7980d0d90 0x7fc7980b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 >> 172.21.15.59:0/3243864778 conn(0x7fc79809d920 msgr2=0x7fc798160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 shutdown_connections 2024-04-24T06:30:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.175+0000 7fc79cc00640 1 -- 172.21.15.59:0/3243864778 wait complete. 2024-04-24T06:30:49.196 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:49.197 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:49.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.401+0000 7fcf0e200640 1 Processor -- start 2024-04-24T06:30:49.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.401+0000 7fcf0e200640 1 -- start start 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0e200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf081567e0 0x7fcf08156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0e200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0815cab0 con 0x7fcf080d0d10 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0e200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf08160e40 con 0x7fcf0815d810 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0e200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0815cd30 con 0x7fcf081567e0 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0d800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcf080d0d10 0x7fcf08160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37792/0 (socket says 172.21.15.59:37792) 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.402+0000 7fcf0d800640 1 -- 172.21.15.59:0/1687670614 learned_addr learned my addr 172.21.15.59:0/1687670614 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/1687670614 >> v2:172.21.15.66:3300/0 conn(0x7fcf081567e0 0x7fcf08156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1739713490 0 0) 0x7fcf00001490 con 0x7fcf080d0d10 2024-04-24T06:30:49.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceec003520 con 0x7fcf080d0d10 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1018394642 0 0) 0x7fceec003520 con 0x7fcf080d0d10 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 >> v2:172.21.15.66:3300/0 conn(0x7fcf081567e0 msgr2=0x7fcf08156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 --2- 172.21.15.59:0/1687670614 >> v2:172.21.15.66:3300/0 conn(0x7fcf081567e0 0x7fcf08156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 >> v1:172.21.15.66:6789/0 conn(0x7fcf0815d810 legacy=0x7fcf080d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0815d010 con 0x7fcf080d0d10 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fcf00001bc0 con 0x7fcf080d0d10 2024-04-24T06:30:49.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 >> v1:172.21.15.59:6789/0 conn(0x7fcf080d0d10 legacy=0x7fcf08160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceec0039b0 0x7fceec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fceec004800 0x7fceec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.403+0000 7fcf07400640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fceec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0815cd30 con 0x7fceec0051c0 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fceec00c110 con 0x7fceec0039b0 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fceec00c310 con 0x7fceec004800 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/1687670614 >> v2:172.21.15.66:3300/0 conn(0x7fcf081567e0 0x7fcf08156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fceec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf07e00640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fceec004800 0x7fceec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceec0039b0 0x7fceec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 -- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fceec004800 msgr2=0x7fceec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fceec004800 0x7fceec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 -- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceec0039b0 msgr2=0x7fceec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fceec0039b0 0x7fceec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.404+0000 7fcf0d800640 1 -- 172.21.15.59:0/1687670614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec003520 con 0x7fceec0051c0 2024-04-24T06:30:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.405+0000 7fcf0d800640 1 -- 172.21.15.59:0/1687670614 reap_dead start 2024-04-24T06:30:49.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0d800640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fceec00b9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000aa30 tx=0x7fceec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f86905eb8cb790 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000a650 con 0x7fceec0051c0 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0002f070 con 0x7fceec0051c0 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf07400640 1 -- 172.21.15.59:0/1687670614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00034470 con 0x7fceec0051c0 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0e200640 1 -- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 msgr2=0x7fceec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0e200640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fceec00b9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000aa30 tx=0x7fceec005830 comp rx=0 tx=0).stop 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0e200640 1 -- 172.21.15.59:0/1687670614 shutdown_connections 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0e200640 1 --2- 172.21.15.59:0/1687670614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fceec00b9c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.406+0000 7fcf0e200640 1 -- 172.21.15.59:0/1687670614 >> 172.21.15.59:0/1687670614 conn(0x7fcf0806bdf0 msgr2=0x7fcf0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.407+0000 7fcf0e200640 1 -- 172.21.15.59:0/1687670614 shutdown_connections 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.407+0000 7fcf0e200640 1 -- 172.21.15.59:0/1687670614 wait complete. 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.408+0000 7fcf0e200640 1 Processor -- start 2024-04-24T06:30:49.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.409+0000 7fcf0e200640 1 -- start start 2024-04-24T06:30:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.409+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.409+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf080ce8b0 0x7fcf081f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.409+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 0x7fcf080b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08156d40 con 0x7fceec0051c0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0e200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08156bc0 con 0x7fcf080b20a0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0e200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcf08156ec0 con 0x7fcf080ce8b0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58530/0 (socket says 172.21.15.59:58530) 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 -- 172.21.15.59:0/3686112143 learned_addr learned my addr 172.21.15.59:0/3686112143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf07e00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf080ce8b0 0x7fcf081f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 -- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf080ce8b0 msgr2=0x7fcf081f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0d800640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 0x7fcf080b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf080ce8b0 0x7fcf081f7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 -- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 msgr2=0x7fcf080b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 0x7fcf080b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0ce00640 1 -- 172.21.15.59:0/3686112143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf08159870 con 0x7fceec0051c0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.410+0000 7fcf0d800640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 0x7fcf080b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf0ce00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcef8010890 tx=0x7fcef8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef801a070 con 0x7fceec0051c0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef800cae0 con 0x7fceec0051c0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf08159b50 con 0x7fceec0051c0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef800f040 con 0x7fceec0051c0 2024-04-24T06:30:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.411+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0815a110 con 0x7fceec0051c0 2024-04-24T06:30:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.412+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf080d0b30 con 0x7fceec0051c0 2024-04-24T06:30:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.413+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcef801d070 con 0x7fceec0051c0 2024-04-24T06:30:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.414+0000 7fcf04c00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 0x7fcee406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.414+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fcef808bd10 con 0x7fceec0051c0 2024-04-24T06:30:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.415+0000 7fcf07e00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 0x7fcee406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.415+0000 7fcf07e00640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 0x7fcee406ee00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007fd0 tx=0x7fcef0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.419+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf080d0b30 con 0x7fceec0051c0 2024-04-24T06:30:49.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.629+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf0804fd90 con 0x7fceec0051c0 2024-04-24T06:30:49.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.630+0000 7fcf04c00640 1 -- 172.21.15.59:0/3686112143 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fcef8091020 con 0x7fceec0051c0 2024-04-24T06:30:49.633 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.633+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 msgr2=0x7fcee406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.633+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 0x7fcee406ee00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007fd0 tx=0x7fcef0009040 comp rx=0 tx=0).stop 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.633+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 msgr2=0x7fcf081590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.633+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcef8010890 tx=0x7fcef8010d50 comp rx=0 tx=0).stop 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 shutdown_connections 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcee406c940 0x7fcee406ee00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf080b20a0 0x7fcf080b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf080ce8b0 0x7fcf081f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 --2- 172.21.15.59:0/3686112143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fceec0051c0 0x7fcf081590f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 >> 172.21.15.59:0/3686112143 conn(0x7fcf0806bdf0 msgr2=0x7fcf08073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:49.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 shutdown_connections 2024-04-24T06:30:49.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.634+0000 7fcf0e200640 1 -- 172.21.15.59:0/3686112143 wait complete. 2024-04-24T06:30:49.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-24T06:30:49.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.870+0000 7fc163200640 1 Processor -- start 2024-04-24T06:30:49.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.870+0000 7fc163200640 1 -- start start 2024-04-24T06:30:49.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc15c15e7c0 0x7fc15c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc15c0b6530 0x7fc15c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c0b7000 con 0x7fc15c15e7c0 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c0cf620 con 0x7fc15c0b6530 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.871+0000 7fc163200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0cf030 con 0x7fc15c0d2630 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc15c15e7c0 0x7fc15c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc15c15e7c0 0x7fc15c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58534/0 (socket says 172.21.15.59:58534) 2024-04-24T06:30:49.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 -- 172.21.15.59:0/531093041 learned_addr learned my addr 172.21.15.59:0/531093041 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161400640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 -- 172.21.15.59:0/531093041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 msgr2=0x7fc15c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 -- 172.21.15.59:0/531093041 >> v2:172.21.15.66:3300/0 conn(0x7fc15c0b6530 msgr2=0x7fc15c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 --2- 172.21.15.59:0/531093041 >> v2:172.21.15.66:3300/0 conn(0x7fc15c0b6530 0x7fc15c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161e00640 1 -- 172.21.15.59:0/531093041 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c0b6a40 con 0x7fc15c15e7c0 2024-04-24T06:30:49.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.872+0000 7fc161400640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:49.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.873+0000 7fc161e00640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0d20f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0cf590 tx=0x7fc15002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ecbe6bf9e41047 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.874+0000 7fc160a00640 1 -- 172.21.15.59:0/531093041 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150037070 con 0x7fc15c15e7c0 2024-04-24T06:30:49.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.874+0000 7fc160a00640 1 -- 172.21.15.59:0/531093041 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc150032440 con 0x7fc15c15e7c0 2024-04-24T06:30:49.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.874+0000 7fc160a00640 1 -- 172.21.15.59:0/531093041 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150032810 con 0x7fc15c15e7c0 2024-04-24T06:30:49.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.874+0000 7fc163200640 1 -- 172.21.15.59:0/531093041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 msgr2=0x7fc15c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.874+0000 7fc163200640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0d20f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0cf590 tx=0x7fc15002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 -- 172.21.15.59:0/531093041 shutdown_connections 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 --2- 172.21.15.59:0/531093041 >> v2:172.21.15.66:3300/0 conn(0x7fc15c0b6530 0x7fc15c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 --2- 172.21.15.59:0/531093041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0d20f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 -- 172.21.15.59:0/531093041 >> 172.21.15.59:0/531093041 conn(0x7fc15c09d920 msgr2=0x7fc15c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 -- 172.21.15.59:0/531093041 shutdown_connections 2024-04-24T06:30:49.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.875+0000 7fc163200640 1 -- 172.21.15.59:0/531093041 wait complete. 2024-04-24T06:30:49.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.877+0000 7fc163200640 1 Processor -- start 2024-04-24T06:30:49.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.877+0000 7fc163200640 1 -- start start 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc15c0b6530 0x7fc15c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0d1a80 con 0x7fc15c15e7c0 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0cf2b0 con 0x7fc15c0b6530 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc163200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0d1c00 con 0x7fc15c0d2630 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc162800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc162800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58542/0 (socket says 172.21.15.59:58542) 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.878+0000 7fc162800640 1 -- 172.21.15.59:0/1285401551 learned_addr learned my addr 172.21.15.59:0/1285401551 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc161e00640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc15c0b6530 0x7fc15c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc161400640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 -- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 msgr2=0x7fc15c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 -- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc15c0b6530 msgr2=0x7fc15c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc15c0b6530 0x7fc15c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 -- 172.21.15.59:0/1285401551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c0c8610 con 0x7fc15c15e7c0 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc161400640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:49.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc162800640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc15800ba10 tx=0x7fc15800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158019070 con 0x7fc15c15e7c0 2024-04-24T06:30:49.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15800cb40 con 0x7fc15c15e7c0 2024-04-24T06:30:49.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158014440 con 0x7fc15c15e7c0 2024-04-24T06:30:49.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.879+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c0c88f0 con 0x7fc15c15e7c0 2024-04-24T06:30:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.880+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c0c8eb0 con 0x7fc15c15e7c0 2024-04-24T06:30:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.881+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15c04fd90 con 0x7fc15c15e7c0 2024-04-24T06:30:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.882+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc15801c070 con 0x7fc15c15e7c0 2024-04-24T06:30:49.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.883+0000 7fc156000640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 0x7fc13c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:49.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.883+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc15808b650 con 0x7fc15c15e7c0 2024-04-24T06:30:49.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.883+0000 7fc161e00640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 0x7fc13c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.884+0000 7fc161e00640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 0x7fc13c06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc150034000 tx=0x7fc15002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:49.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:49.888+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc158059c50 con 0x7fc15c15e7c0 2024-04-24T06:30:50.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.170+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fc15c0d1eb0 con 0x7fc15c15e7c0 2024-04-24T06:30:50.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.741+0000 7fc156000640 1 -- 172.21.15.59:0/1285401551 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v135) ==== 152+0+0 (secure 0 0 0) 0x7fc158059550 con 0x7fc15c15e7c0 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.744+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 msgr2=0x7fc13c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.744+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 0x7fc13c06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc150034000 tx=0x7fc15002e000 comp rx=0 tx=0).stop 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 msgr2=0x7fc15c0b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc15800ba10 tx=0x7fc15800bed0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 shutdown_connections 2024-04-24T06:30:50.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13c06c8f0 0x7fc13c06edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc15c15e7c0 0x7fc15c0b5500 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15c0d2630 0x7fc15c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 --2- 172.21.15.59:0/1285401551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc15c0b6530 0x7fc15c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 >> 172.21.15.59:0/1285401551 conn(0x7fc15c09d920 msgr2=0x7fc15c0d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 shutdown_connections 2024-04-24T06:30:50.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.745+0000 7fc163200640 1 -- 172.21.15.59:0/1285401551 wait complete. 2024-04-24T06:30:50.771 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:30:50.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.984+0000 7fcd6fe00640 1 Processor -- start 2024-04-24T06:30:50.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.984+0000 7fcd6fe00640 1 -- start start 2024-04-24T06:30:50.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.985+0000 7fcd6fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 0x7fcd700b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd70148330 0x7fcd7006bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd700b5e10 con 0x7fcd700b43f0 2024-04-24T06:30:50.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7015ff50 con 0x7fcd70148330 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700b66c0 con 0x7fcd7006b9d0 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd70148330 0x7fcd7006bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd70148330 0x7fcd7006bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58544/0 (socket says 172.21.15.59:58544) 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 -- 172.21.15.59:0/1435274328 learned_addr learned my addr 172.21.15.59:0/1435274328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1435274328 >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 0x7fcd700b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 -- 172.21.15.59:0/1435274328 >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 msgr2=0x7fcd700b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 --2- 172.21.15.59:0/1435274328 >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 0x7fcd700b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 -- 172.21.15.59:0/1435274328 >> v1:172.21.15.66:6789/0 conn(0x7fcd700b43f0 legacy=0x7fcd701600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6e000640 1 -- 172.21.15.59:0/1435274328 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700b60b0 con 0x7fcd70148330 2024-04-24T06:30:50.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.986+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1435274328 >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 0x7fcd700b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:50.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.988+0000 7fcd6e000640 1 --2- 172.21.15.59:0/1435274328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd70148330 0x7fcd7006bdb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcd70073da0 tx=0x7fcd5802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4faed1137d86c278 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:50.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.988+0000 7fcd6d600640 1 -- 172.21.15.59:0/1435274328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58036070 con 0x7fcd70148330 2024-04-24T06:30:50.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.988+0000 7fcd6d600640 1 -- 172.21.15.59:0/1435274328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd58031600 con 0x7fcd70148330 2024-04-24T06:30:50.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.988+0000 7fcd6d600640 1 -- 172.21.15.59:0/1435274328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd580319d0 con 0x7fcd70148330 2024-04-24T06:30:50.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1435274328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd70148330 msgr2=0x7fcd7006bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:50.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1435274328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd70148330 0x7fcd7006bdb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcd70073da0 tx=0x7fcd5802a660 comp rx=0 tx=0).stop 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1435274328 shutdown_connections 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1435274328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd70148330 0x7fcd7006bdb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1435274328 >> v2:172.21.15.66:3300/0 conn(0x7fcd7006b9d0 0x7fcd700b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1435274328 >> 172.21.15.59:0/1435274328 conn(0x7fcd700b6910 msgr2=0x7fcd700b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1435274328 shutdown_connections 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.989+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1435274328 wait complete. 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.991+0000 7fcd6fe00640 1 Processor -- start 2024-04-24T06:30:50.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.991+0000 7fcd6fe00640 1 -- start start 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 0x7fcd7015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd70148330 0x7fcd701f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b6350 con 0x7fcd700b43f0 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7015fc70 con 0x7fcd70148330 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7015f980 con 0x7fcd7006b9d0 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58552/0 (socket says 172.21.15.59:58552) 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6e000640 1 -- 172.21.15.59:0/1570226195 learned_addr learned my addr 172.21.15.59:0/1570226195 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6f400640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd70148330 0x7fcd701f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.992+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 0x7fcd7015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 -- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 msgr2=0x7fcd7015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 0x7fcd7015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 -- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd70148330 msgr2=0x7fcd701f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd70148330 0x7fcd701f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 -- 172.21.15.59:0/1570226195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70140500 con 0x7fcd700b43f0 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 0x7fcd7015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:50.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6e000640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcd58000f80 tx=0x7fcd58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:50.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58041020 con 0x7fcd700b43f0 2024-04-24T06:30:50.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd58007820 con 0x7fcd700b43f0 2024-04-24T06:30:50.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58036040 con 0x7fcd700b43f0 2024-04-24T06:30:50.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.993+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd701407e0 con 0x7fcd700b43f0 2024-04-24T06:30:50.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.994+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd70140fe0 con 0x7fcd700b43f0 2024-04-24T06:30:50.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.995+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7004fd90 con 0x7fcd700b43f0 2024-04-24T06:30:50.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.995+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd58044020 con 0x7fcd700b43f0 2024-04-24T06:30:50.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.996+0000 7fcd66a00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 0x7fcd4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:50.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.997+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fcd580ac680 con 0x7fcd700b43f0 2024-04-24T06:30:50.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.997+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 0x7fcd4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:50.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:50.997+0000 7fcd6ea00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 0x7fcd4c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006fd0 tx=0x7fcd60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:51.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.001+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd5804b030 con 0x7fcd700b43f0 2024-04-24T06:30:51.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.213+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcd70161d00 con 0x7fcd700b43f0 2024-04-24T06:30:51.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.218+0000 7fcd66a00640 1 -- 172.21.15.59:0/1570226195 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fcd5807ada0 con 0x7fcd700b43f0 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 msgr2=0x7fcd4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 0x7fcd4c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006fd0 tx=0x7fcd60008040 comp rx=0 tx=0).stop 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 msgr2=0x7fcd7013fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcd58000f80 tx=0x7fcd58004a40 comp rx=0 tx=0).stop 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 shutdown_connections 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcd4c06c8f0 0x7fcd4c06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd70148330 0x7fcd701f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd700b43f0 0x7fcd7013fcb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 --2- 172.21.15.59:0/1570226195 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd7006b9d0 0x7fcd7015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 >> 172.21.15.59:0/1570226195 conn(0x7fcd700b6910 msgr2=0x7fcd7015ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:51.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 shutdown_connections 2024-04-24T06:30:51.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.222+0000 7fcd6fe00640 1 -- 172.21.15.59:0/1570226195 wait complete. 2024-04-24T06:30:51.244 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:30:51.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:30:51.449 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:30:51.450 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:51.450 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:51.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:30:51.506 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:30:51.507 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:51.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:30:51.570 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:30:51.570 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:51.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:30:51.629 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:30:51.630 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:30:51.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:30:51.690 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:30:51.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:30:51.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.905+0000 7f80dd400640 1 Processor -- start 2024-04-24T06:30:51.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.905+0000 7f80dd400640 1 -- start start 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80dd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80d81644b0 0x7f80d8163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80dd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80d806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80dd400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d8160d40 con 0x7f80d8160960 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80dd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f80d8164270 con 0x7f80d81644b0 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80dd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f80d8166de0 con 0x7f80d80b4550 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80d6800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f80d8160960 0x7f80d80b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37804/0 (socket says 172.21.15.59:37804) 2024-04-24T06:30:51.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.907+0000 7f80d6800640 1 -- 172.21.15.59:0/3849984021 learned_addr learned my addr 172.21.15.59:0/3849984021 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d7200640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80d806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5e00640 1 --2- 172.21.15.59:0/3849984021 >> v2:172.21.15.66:3300/0 conn(0x7f80d81644b0 0x7f80d8163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095143272 0 0) 0x7f80d8160d40 con 0x7f80d8160960 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80bc003520 con 0x7f80d8160960 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1910265159 0 0) 0x7f80bc003520 con 0x7f80d8160960 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 msgr2=0x7f80d806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80d806e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 >> v2:172.21.15.66:3300/0 conn(0x7f80d81644b0 msgr2=0x7f80d8163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 --2- 172.21.15.59:0/3849984021 >> v2:172.21.15.66:3300/0 conn(0x7f80d81644b0 0x7f80d8163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d81670c0 con 0x7f80d8160960 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f80c8001670 con 0x7f80d8160960 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 >> v1:172.21.15.59:6789/0 conn(0x7f80d8160960 legacy=0x7f80d80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0039b0 0x7f80bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc004d40 0x7f80bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8164270 con 0x7f80bc004d40 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8166de0 con 0x7f80bc0039b0 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.908+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc00b9c0 con 0x7f80d80b4550 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc004d40 0x7f80bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d6800640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0039b0 0x7f80bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d5e00640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 -- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 msgr2=0x7f80bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 -- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0039b0 msgr2=0x7f80bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80bc0039b0 0x7f80bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d7200640 1 -- 172.21.15.59:0/3849984021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc003520 con 0x7f80bc004d40 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.909+0000 7f80d5e00640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80b4550 0x7f80bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:51.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.910+0000 7f80d7200640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc004d40 0x7f80bc0051c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009fc0 tx=0x7f80bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645dd9d18998da41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:51.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0009c10 con 0x7f80bc004d40 2024-04-24T06:30:51.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d0036070 con 0x7f80bc004d40 2024-04-24T06:30:51.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80d5400640 1 -- 172.21.15.59:0/3849984021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0031840 con 0x7f80bc004d40 2024-04-24T06:30:51.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 -- 172.21.15.59:0/3849984021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc004d40 msgr2=0x7f80bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 --2- 172.21.15.59:0/3849984021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc004d40 0x7f80bc0051c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009fc0 tx=0x7f80bc005830 comp rx=0 tx=0).stop 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80d7200640 1 -- 172.21.15.59:0/3849984021 reap_dead start 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 -- 172.21.15.59:0/3849984021 shutdown_connections 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 -- 172.21.15.59:0/3849984021 >> 172.21.15.59:0/3849984021 conn(0x7f80d8067ea0 msgr2=0x7f80d806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 -- 172.21.15.59:0/3849984021 shutdown_connections 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.911+0000 7f80dd400640 1 -- 172.21.15.59:0/3849984021 wait complete. 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.913+0000 7f80dd400640 1 Processor -- start 2024-04-24T06:30:51.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.913+0000 7f80dd400640 1 -- start start 2024-04-24T06:30:51.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d815d2c0 0x7f80d815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80cd040 0x7f80d80cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80d81668e0 con 0x7f80d81ee840 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8167680 con 0x7f80d815d2c0 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.914+0000 7f80dd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f80d8166a60 con 0x7f80d80cd040 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58562/0 (socket says 172.21.15.59:58562) 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 -- 172.21.15.59:0/1167692980 learned_addr learned my addr 172.21.15.59:0/1167692980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d6800640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d815d2c0 0x7f80d815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d7200640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80cd040 0x7f80d80cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 -- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80cd040 msgr2=0x7f80d80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80cd040 0x7f80d80cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 -- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d815d2c0 msgr2=0x7f80d815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d815d2c0 0x7f80d815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.915+0000 7f80d5e00640 1 -- 172.21.15.59:0/1167692980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d80ce330 con 0x7f80d81ee840 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80d5e00640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f80c4010a00 tx=0x7f80c4010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400f040 con 0x7f80d81ee840 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c400ec20 con 0x7f80d81ee840 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c401c400 con 0x7f80d81ee840 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d80ce610 con 0x7f80d81ee840 2024-04-24T06:30:51.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.916+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d80cebd0 con 0x7f80d81ee840 2024-04-24T06:30:51.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.917+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d8068a80 con 0x7f80d81ee840 2024-04-24T06:30:51.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.918+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80c401c5a0 con 0x7f80d81ee840 2024-04-24T06:30:51.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.920+0000 7f80cea00640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 0x7f80b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:51.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.920+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f80c408bf90 con 0x7f80d81ee840 2024-04-24T06:30:51.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.920+0000 7f80d6800640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 0x7f80b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:51.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.921+0000 7f80d6800640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 0x7f80b806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004840 tx=0x7f80c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:51.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:51.925+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80c4018030 con 0x7f80d81ee840 2024-04-24T06:30:52.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.170+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80d806e260 con 0x7f80d81ee840 2024-04-24T06:30:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.171+0000 7f80cea00640 1 -- 172.21.15.59:0/1167692980 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3866 (secure 0 0 0) 0x7f80c405a6b0 con 0x7f80d81ee840 2024-04-24T06:30:52.173 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:52.173 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2200393013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2200393013},{"type":"v1","addr":"172.21.15.66:6835","nonce":2200393013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/516761758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":516761758},{"type":"v1","addr":"172.21.15.59:6837","nonce":516761758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2482857265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2482857265},{"type":"v1","addr":"172.21.15.66:6833","nonce":2482857265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5193,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/4286438839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4286438839},{"type":"v1","addr":"172.21.15.59:6836","nonce":4286438839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-24T06:30:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 19 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.173+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 msgr2=0x7f80b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.173+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 0x7f80b806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004840 tx=0x7f80c8009040 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.173+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 msgr2=0x7f80d81eecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.173+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f80c4010a00 tx=0x7f80c4010ec0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 shutdown_connections 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80b806c920 0x7f80b806ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f80d80cd040 0x7f80d80cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d81ee840 0x7f80d81eecc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 --2- 172.21.15.59:0/1167692980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d815d2c0 0x7f80d815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 >> 172.21.15.59:0/1167692980 conn(0x7f80d8067ea0 msgr2=0x7f80d81637d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 shutdown_connections 2024-04-24T06:30:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.174+0000 7f80dd400640 1 -- 172.21.15.59:0/1167692980 wait complete. 2024-04-24T06:30:52.198 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:30:52.198 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:30:52.198 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:30:52.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:30:52.199 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:30:52.199 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:30:52.199 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:30:52.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:30:52.199 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:30:52.200 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:30:52.200 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:30:52.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:30:52.200 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:30:52.200 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:30:52.200 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:30:52.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:30:52.220 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:30:52.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:30:52.221 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:30:52.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:30:52.231 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:30:52.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:30:52.235 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:30:52.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:30:52.268 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:30:52.269 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:52.275 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:30:52.275 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:52.282 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:30:52.283 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:52.290 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:30:52.290 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:52.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:30:52.421 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:30:52.421 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:30:52.438 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:30:52.438 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:30:52.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.476+0000 7f62db000640 1 Processor -- start 2024-04-24T06:30:52.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.476+0000 7f62db000640 1 -- start start 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.477+0000 7f62db000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d40d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.477+0000 7f62db000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62d40d1c50 0x7f62d40b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.477+0000 7f62db000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d40b91e0 con 0x7f62d40ba470 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62db000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f62d40d2080 con 0x7f62d40d1c50 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62db000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f62d40b94c0 con 0x7f62d415e950 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62d40d1c50 0x7f62d40b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62d40d1c50 0x7f62d40b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58604/0 (socket says 172.21.15.59:58604) 2024-04-24T06:30:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 -- 172.21.15.59:0/1689211732 learned_addr learned my addr 172.21.15.59:0/1689211732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d3e00640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d40d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 -- 172.21.15.59:0/1689211732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 msgr2=0x7f62d40d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d40d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 -- 172.21.15.59:0/1689211732 >> v1:172.21.15.66:6789/0 conn(0x7f62d40ba470 legacy=0x7f62d40d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.478+0000 7f62d9000640 1 -- 172.21.15.59:0/1689211732 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d40b97a0 con 0x7f62d40d1c50 2024-04-24T06:30:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.479+0000 7f62d9000640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40d1c50 0x7f62d40b87e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009e50 tx=0x7f62cc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f685190c5e4e721e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.480+0000 7f62d3400640 1 -- 172.21.15.59:0/1689211732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc036070 con 0x7f62d40d1c50 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.480+0000 7f62d3400640 1 -- 172.21.15.59:0/1689211732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc031600 con 0x7f62d40d1c50 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.480+0000 7f62d3400640 1 -- 172.21.15.59:0/1689211732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0319d0 con 0x7f62d40d1c50 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.480+0000 7f62db000640 1 -- 172.21.15.59:0/1689211732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40d1c50 msgr2=0x7f62d40b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.480+0000 7f62db000640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40d1c50 0x7f62d40b87e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009e50 tx=0x7f62cc02a6c0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 -- 172.21.15.59:0/1689211732 shutdown_connections 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40d1c50 0x7f62d40b87e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 --2- 172.21.15.59:0/1689211732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d40d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 -- 172.21.15.59:0/1689211732 >> 172.21.15.59:0/1689211732 conn(0x7f62d4067b80 msgr2=0x7f62d4067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 -- 172.21.15.59:0/1689211732 shutdown_connections 2024-04-24T06:30:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.481+0000 7f62db000640 1 -- 172.21.15.59:0/1689211732 wait complete. 2024-04-24T06:30:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.482+0000 7f62db000640 1 Processor -- start 2024-04-24T06:30:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.482+0000 7f62db000640 1 -- start start 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d40d1c50 0x7f62d41603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d41f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62d415ed30 con 0x7f62d40ba470 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f62d40d27a0 con 0x7f62d40d1c50 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.483+0000 7f62db000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f62d40b8d20 con 0x7f62d415e950 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58616/0 (socket says 172.21.15.59:58616) 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 -- 172.21.15.59:0/3654655485 learned_addr learned my addr 172.21.15.59:0/3654655485 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d9000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d41f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62cbe00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d40d1c50 0x7f62d41603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 -- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 msgr2=0x7f62d41f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d41f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 -- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d40d1c50 msgr2=0x7f62d41603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d40d1c50 0x7f62d41603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 -- 172.21.15.59:0/3654655485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d4160ac0 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.484+0000 7f62d3e00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d5f0 tx=0x7f62c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.485+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40125f0 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.485+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4019070 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.485+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40129d0 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.485+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d4160da0 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.485+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d4161360 con 0x7f62d40ba470 2024-04-24T06:30:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.486+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d4068a80 con 0x7f62d40ba470 2024-04-24T06:30:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.487+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62c4012b70 con 0x7f62d40ba470 2024-04-24T06:30:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.488+0000 7f62d0c00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.488+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f62c408bba0 con 0x7f62d40ba470 2024-04-24T06:30:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.488+0000 7f62cbe00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.489+0000 7f62cbe00640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 0x7f62b006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f62d4162280 tx=0x7f62bc007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:52.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.492+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62c4059be0 con 0x7f62d40ba470 2024-04-24T06:30:52.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:52.711+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f62d40ba250 con 0x7f62d40ba470 2024-04-24T06:30:53.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.777+0000 7f62d0c00640 1 -- 172.21.15.59:0/3654655485 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7f62c405dee0 con 0x7f62d40ba470 2024-04-24T06:30:53.779 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 msgr2=0x7f62b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 0x7f62b006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f62d4162280 tx=0x7f62bc007e30 comp rx=0 tx=0).stop 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 msgr2=0x7f62d40ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d5f0 tx=0x7f62c400dab0 comp rx=0 tx=0).stop 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 shutdown_connections 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f62b006c8f0 0x7f62b006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:53.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f62d415e950 0x7f62d41f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:53.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d40d1c50 0x7f62d41603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:53.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 --2- 172.21.15.59:0/3654655485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62d40ba470 0x7f62d40ce830 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:53.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 >> 172.21.15.59:0/3654655485 conn(0x7f62d4067b80 msgr2=0x7f62d40d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:53.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.781+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 shutdown_connections 2024-04-24T06:30:53.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:53.782+0000 7f62db000640 1 -- 172.21.15.59:0/3654655485 wait complete. 2024-04-24T06:30:53.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:30:54.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.003+0000 7f1b13200640 1 Processor -- start 2024-04-24T06:30:54.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.004+0000 7f1b13200640 1 -- start start 2024-04-24T06:30:54.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b13200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1b0c0b4130 0x7f1b0c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b13200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b13200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0c06ed40 con 0x7f1b0c162df0 2024-04-24T06:30:54.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b13200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1b0c06eaf0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b13200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1b0c0b3590 con 0x7f1b0c0b4130 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.005+0000 7f1b12800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58622/0 (socket says 172.21.15.59:58622) 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 -- 172.21.15.59:0/2575576192 learned_addr learned my addr 172.21.15.59:0/2575576192 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 -- 172.21.15.59:0/2575576192 >> v2:172.21.15.66:3300/0 conn(0x7f1b0c0b4130 msgr2=0x7f1b0c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 --2- 172.21.15.59:0/2575576192 >> v2:172.21.15.66:3300/0 conn(0x7f1b0c0b4130 0x7f1b0c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 -- 172.21.15.59:0/2575576192 >> v1:172.21.15.59:6789/0 conn(0x7f1b0c162df0 legacy=0x7f1b0c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.006+0000 7f1b12800640 1 -- 172.21.15.59:0/2575576192 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0b3220 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.007+0000 7f1b12800640 1 --2- 172.21.15.59:0/2575576192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c074090 tx=0x7f1b0802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b03352db33ab6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.007+0000 7f1b10a00640 1 -- 172.21.15.59:0/2575576192 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08036070 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.007+0000 7f1b10a00640 1 -- 172.21.15.59:0/2575576192 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b08031600 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.007+0000 7f1b10a00640 1 -- 172.21.15.59:0/2575576192 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b080319d0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 -- 172.21.15.59:0/2575576192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 msgr2=0x7f1b0c0c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 --2- 172.21.15.59:0/2575576192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c074090 tx=0x7f1b0802a730 comp rx=0 tx=0).stop 2024-04-24T06:30:54.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 -- 172.21.15.59:0/2575576192 shutdown_connections 2024-04-24T06:30:54.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 --2- 172.21.15.59:0/2575576192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c31a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 --2- 172.21.15.59:0/2575576192 >> v2:172.21.15.66:3300/0 conn(0x7f1b0c0b4130 0x7f1b0c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 -- 172.21.15.59:0/2575576192 >> 172.21.15.59:0/2575576192 conn(0x7f1b0c0b21a0 msgr2=0x7f1b0c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:54.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 -- 172.21.15.59:0/2575576192 shutdown_connections 2024-04-24T06:30:54.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.008+0000 7f1b13200640 1 -- 172.21.15.59:0/2575576192 wait complete. 2024-04-24T06:30:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.010+0000 7f1b13200640 1 Processor -- start 2024-04-24T06:30:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.010+0000 7f1b13200640 1 -- start start 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1b0c0b4130 0x7f1b0c163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 0x7f1b0c0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c0c3860 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c0c36e0 con 0x7f1b0c162df0 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b13200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c0c39e0 con 0x7f1b0c0b4130 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b11400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b11400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58626/0 (socket says 172.21.15.59:58626) 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.011+0000 7f1b11400640 1 -- 172.21.15.59:0/653547156 learned_addr learned my addr 172.21.15.59:0/653547156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:54.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11e00640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1b0c0b4130 0x7f1b0c163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b12800640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 0x7f1b0c0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 -- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1b0c0b4130 msgr2=0x7f1b0c163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1b0c0b4130 0x7f1b0c163750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 -- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 msgr2=0x7f1b0c0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 0x7f1b0c0ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 -- 172.21.15.59:0/653547156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0aeb40 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b12800640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 0x7f1b0c0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.012+0000 7f1b11400640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b4d0 tx=0x7f1afc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.013+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc014070 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.013+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc010870 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.013+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc010c60 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.013+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c0ae2a0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.013+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c0ae860 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.014+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c04fd90 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.015+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1afc017070 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.016+0000 7f1b06000640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 0x7f1af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.016+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f1afc08b5f0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.016+0000 7f1b11e00640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 0x7f1af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.017+0000 7f1b11e00640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 0x7f1af006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b00007fd0 tx=0x7f1b00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.020+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1afc059cd0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.305+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b0c0b3ea0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.306+0000 7f1b06000640 1 -- 172.21.15.59:0/653547156 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f1afc0595d0 con 0x7f1b0c0cfb90 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.311 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.312 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.313 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:30:54.314 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.316 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.318 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:30:54.320 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:30:54.321 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:30:54.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:54.321 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:30:54.321 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 msgr2=0x7f1af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 0x7f1af006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b00007fd0 tx=0x7f1b00009040 comp rx=0 tx=0).stop 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 msgr2=0x7f1b0c0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b4d0 tx=0x7f1afc00b990 comp rx=0 tx=0).stop 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 shutdown_connections 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1af006c8f0 0x7f1af006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b0c162df0 0x7f1b0c0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c0cfb90 0x7f1b0c0c9bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 --2- 172.21.15.59:0/653547156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1b0c0b4130 0x7f1b0c163750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 >> 172.21.15.59:0/653547156 conn(0x7f1b0c0b21a0 msgr2=0x7f1b0c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 shutdown_connections 2024-04-24T06:30:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.309+0000 7f1b13200640 1 -- 172.21.15.59:0/653547156 wait complete. 2024-04-24T06:30:54.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:54.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.539+0000 7f06a9400640 1 Processor -- start 2024-04-24T06:30:54.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.540+0000 7f06a9400640 1 -- start start 2024-04-24T06:30:54.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06a41454e0 0x7f06a4145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a40b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a9400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a40cd520 con 0x7f06a40b3e40 2024-04-24T06:30:54.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a9400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f06a40cddc0 con 0x7f06a41454e0 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a9400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40cd800 con 0x7f06a40cd020 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06a41454e0 0x7f06a4145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06a41454e0 0x7f06a4145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58638/0 (socket says 172.21.15.59:58638) 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.541+0000 7f06a2800640 1 -- 172.21.15.59:0/2874568688 learned_addr learned my addr 172.21.15.59:0/2874568688 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.542+0000 7f0699e00640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a40b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.542+0000 7f06a2800640 1 -- 172.21.15.59:0/2874568688 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 msgr2=0x7f06a40b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:54.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.542+0000 7f06a2800640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a40b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.542+0000 7f06a2800640 1 -- 172.21.15.59:0/2874568688 >> v1:172.21.15.66:6789/0 conn(0x7f06a40b3e40 legacy=0x7f06a4160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.542+0000 7f06a2800640 1 -- 172.21.15.59:0/2874568688 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40cdae0 con 0x7f06a41454e0 2024-04-24T06:30:54.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.544+0000 7f06a2800640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a4145bb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8647d976950ed509 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.544+0000 7f06a1e00640 1 -- 172.21.15.59:0/2874568688 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694037070 con 0x7f06a41454e0 2024-04-24T06:30:54.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.544+0000 7f06a1e00640 1 -- 172.21.15.59:0/2874568688 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694032440 con 0x7f06a41454e0 2024-04-24T06:30:54.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.544+0000 7f06a1e00640 1 -- 172.21.15.59:0/2874568688 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694032810 con 0x7f06a41454e0 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 -- 172.21.15.59:0/2874568688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 msgr2=0x7f06a4145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a4145bb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069402a1c0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 -- 172.21.15.59:0/2874568688 shutdown_connections 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a40b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 --2- 172.21.15.59:0/2874568688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a4145bb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 -- 172.21.15.59:0/2874568688 >> 172.21.15.59:0/2874568688 conn(0x7f06a4067ea0 msgr2=0x7f06a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 -- 172.21.15.59:0/2874568688 shutdown_connections 2024-04-24T06:30:54.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.545+0000 7f06a9400640 1 -- 172.21.15.59:0/2874568688 wait complete. 2024-04-24T06:30:54.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.547+0000 7f06a9400640 1 Processor -- start 2024-04-24T06:30:54.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.547+0000 7f06a9400640 1 -- start start 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a40b3e40 0x7f06a40abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a41f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4146270 con 0x7f06a41454e0 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41460f0 con 0x7f06a40b3e40 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a9400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f06a4161020 con 0x7f06a40cd020 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58650/0 (socket says 172.21.15.59:58650) 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.548+0000 7f06a3200640 1 -- 172.21.15.59:0/426366713 learned_addr learned my addr 172.21.15.59:0/426366713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:54.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f0699e00640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a41f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a2800640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a40b3e40 0x7f06a40abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 -- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 msgr2=0x7f06a41f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a41f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 -- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a40b3e40 msgr2=0x7f06a40abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a40b3e40 0x7f06a40abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 -- 172.21.15.59:0/426366713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40ac530 con 0x7f06a41454e0 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f0699e00640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:54.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f06a3200640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c7d0 tx=0x7f069c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c003b70 con 0x7f06a41454e0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c003d30 con 0x7f06a41454e0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.549+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c00aa90 con 0x7f06a41454e0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.550+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a40a9900 con 0x7f06a41454e0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.550+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a40a9ec0 con 0x7f06a41454e0 2024-04-24T06:30:54.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.551+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a406fdf0 con 0x7f06a41454e0 2024-04-24T06:30:54.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.552+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f069c00ac30 con 0x7f06a41454e0 2024-04-24T06:30:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.553+0000 7f069b400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 0x7f068006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.553+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f069c08b800 con 0x7f06a41454e0 2024-04-24T06:30:54.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.554+0000 7f06a2800640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 0x7f068006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:54.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.554+0000 7f06a2800640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 0x7f068006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0694034000 tx=0x7f069402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:54.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.558+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06a406fdf0 con 0x7f06a41454e0 2024-04-24T06:30:54.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.768+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06a40ce8e0 con 0x7f06a41454e0 2024-04-24T06:30:54.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.769+0000 7f069b400640 1 -- 172.21.15.59:0/426366713 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12807 (secure 0 0 0) 0x7f06a40ce8e0 con 0x7f06a41454e0 2024-04-24T06:30:54.772 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:54.772 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":41,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:53.774686+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.772+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 msgr2=0x7f068006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.772+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 0x7f068006edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0694034000 tx=0x7f069402e000 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.772+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 msgr2=0x7f06a41f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.772+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c7d0 tx=0x7f069c00cc90 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 shutdown_connections 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f068006c8f0 0x7f068006edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06a41454e0 0x7f06a41f4ce0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06a40cd020 0x7f06a41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 --2- 172.21.15.59:0/426366713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a40b3e40 0x7f06a40abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 >> 172.21.15.59:0/426366713 conn(0x7f06a4067ea0 msgr2=0x7f06a40b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 shutdown_connections 2024-04-24T06:30:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:54.773+0000 7f06a9400640 1 -- 172.21.15.59:0/426366713 wait complete. 2024-04-24T06:30:54.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:30:54.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:30:55.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.011+0000 7f3e5e800640 1 Processor -- start 2024-04-24T06:30:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.012+0000 7f3e5e800640 1 -- start start 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3e580d15b0 0x7f3e580cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5e800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e580b5da0 con 0x7f3e580b4f20 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5e800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e580d0a20 con 0x7f3e5815ed20 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5815f100 con 0x7f3e580d15b0 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e580b4f20 0x7f3e580cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37838/0 (socket says 172.21.15.59:37838) 2024-04-24T06:30:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5de00640 1 -- 172.21.15.59:0/1673532529 learned_addr learned my addr 172.21.15.59:0/1673532529 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.013+0000 7f3e5ca00640 1 --2- 172.21.15.59:0/1673532529 >> v2:172.21.15.66:3300/0 conn(0x7f3e580d15b0 0x7f3e580cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745571466 0 0) 0x7f3e580b5da0 con 0x7f3e580b4f20 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e3c003520 con 0x7f3e580b4f20 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3112839823 0 0) 0x7f3e3c003520 con 0x7f3e580b4f20 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 >> v2:172.21.15.66:3300/0 conn(0x7f3e580d15b0 msgr2=0x7f3e580cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 --2- 172.21.15.59:0/1673532529 >> v2:172.21.15.66:3300/0 conn(0x7f3e580d15b0 0x7f3e580cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 >> v1:172.21.15.66:6789/0 conn(0x7f3e5815ed20 legacy=0x7f3e580d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580d2ed0 con 0x7f3e580b4f20 2024-04-24T06:30:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f3e50001670 con 0x7f3e580b4f20 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 >> v1:172.21.15.59:6789/0 conn(0x7f3e580b4f20 legacy=0x7f3e580cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 0x7f3e3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e3c004800 0x7f3e3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c0052e0 0x7f3e3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c00c1b0 con 0x7f3e3c003a00 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c00c360 con 0x7f3e3c0052e0 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.014+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c00c560 con 0x7f3e3c004800 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 0x7f3e3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5ca00640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e3c004800 0x7f3e3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 -- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e3c004800 msgr2=0x7f3e3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e3c004800 0x7f3e3c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 -- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c0052e0 msgr2=0x7f3e3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c0052e0 0x7f3e3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5d400640 1 -- 172.21.15.59:0/1673532529 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c003520 con 0x7f3e3c003a00 2024-04-24T06:30:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.015+0000 7f3e5de00640 1 -- 172.21.15.59:0/1673532529 reap_dead start 2024-04-24T06:30:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.016+0000 7f3e5d400640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 0x7f3e3c003e20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3e48001750 tx=0x7f3e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae42473049bccaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.016+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48001370 con 0x7f3e3c003a00 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.016+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e480052d0 con 0x7f3e3c003a00 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.016+0000 7f3e57e00640 1 -- 172.21.15.59:0/1673532529 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4802abe0 con 0x7f3e3c003a00 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 -- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 msgr2=0x7f3e3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 0x7f3e3c003e20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3e48001750 tx=0x7f3e3c005830 comp rx=0 tx=0).stop 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 -- 172.21.15.59:0/1673532529 shutdown_connections 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 --2- 172.21.15.59:0/1673532529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c003a00 0x7f3e3c003e20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 -- 172.21.15.59:0/1673532529 >> 172.21.15.59:0/1673532529 conn(0x7f3e5808bf90 msgr2=0x7f3e580d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 -- 172.21.15.59:0/1673532529 shutdown_connections 2024-04-24T06:30:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.017+0000 7f3e5e800640 1 -- 172.21.15.59:0/1673532529 wait complete. 2024-04-24T06:30:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.019+0000 7f3e5e800640 1 Processor -- start 2024-04-24T06:30:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.019+0000 7f3e5e800640 1 -- start start 2024-04-24T06:30:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 0x7f3e580b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e58158c30 0x7f3e58160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580b3600 con 0x7f3e58159090 2024-04-24T06:30:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580d3490 con 0x7f3e3c003a00 2024-04-24T06:30:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.020+0000 7f3e5e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3e580b3780 con 0x7f3e58158c30 2024-04-24T06:30:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58658/0 (socket says 172.21.15.59:58658) 2024-04-24T06:30:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 -- 172.21.15.59:0/876451481 learned_addr learned my addr 172.21.15.59:0/876451481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5d400640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 0x7f3e580b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5ca00640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e58158c30 0x7f3e58160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 -- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e58158c30 msgr2=0x7f3e58160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e58158c30 0x7f3e58160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 -- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 msgr2=0x7f3e580b8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 0x7f3e580b8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 -- 172.21.15.59:0/876451481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580b7c90 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5d400640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 0x7f3e580b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.021+0000 7f3e5de00640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3e5808e600 tx=0x7f3e5000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.022+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004bf0 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.022+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e50004db0 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.022+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50007270 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.022+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e580b7f30 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.022+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e580b84f0 con 0x7f3e58159090 2024-04-24T06:30:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.023+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5804fd90 con 0x7f3e58159090 2024-04-24T06:30:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.024+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e50007410 con 0x7f3e58159090 2024-04-24T06:30:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.025+0000 7f3e55600640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 0x7f3e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.025+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f3e5008adc0 con 0x7f3e58159090 2024-04-24T06:30:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.026+0000 7f3e5d400640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 0x7f3e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.026+0000 7f3e5d400640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 0x7f3e3806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3e48032000 tx=0x7f3e4802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.030+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e500192a0 con 0x7f3e58159090 2024-04-24T06:30:55.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.235+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e5806ba80 con 0x7f3e58159090 2024-04-24T06:30:55.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.236+0000 7f3e55600640 1 -- 172.21.15.59:0/876451481 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f3e50059410 con 0x7f3e58159090 2024-04-24T06:30:55.238 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:30:55.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.239+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 msgr2=0x7f3e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.239+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 0x7f3e3806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3e48032000 tx=0x7f3e4802e000 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.239+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 msgr2=0x7f3e580cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.239+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3e5808e600 tx=0x7f3e5000c7a0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 shutdown_connections 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3e3806c8f0 0x7f3e3806edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e58159090 0x7f3e580cf280 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3e58158c30 0x7f3e58160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 --2- 172.21.15.59:0/876451481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e3c003a00 0x7f3e580b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 >> 172.21.15.59:0/876451481 conn(0x7f3e5808bf90 msgr2=0x7f3e580d4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 shutdown_connections 2024-04-24T06:30:55.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.240+0000 7f3e5e800640 1 -- 172.21.15.59:0/876451481 wait complete. 2024-04-24T06:30:55.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:30:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.476+0000 7f364a800640 1 Processor -- start 2024-04-24T06:30:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.477+0000 7f364a800640 1 -- start start 2024-04-24T06:30:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.478+0000 7f364a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f36440d0b70 0x7f36440d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.478+0000 7f364a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f36440cd250 0x7f36440d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.478+0000 7f364a800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440d1490 con 0x7f364415e200 2024-04-24T06:30:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.478+0000 7f364a800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f364409b6f0 con 0x7f36440cd250 2024-04-24T06:30:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.478+0000 7f364a800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f36440cd9f0 con 0x7f36440d0b70 2024-04-24T06:30:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f36440cd250 0x7f36440d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f36440cd250 0x7f36440d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58668/0 (socket says 172.21.15.59:58668) 2024-04-24T06:30:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 -- 172.21.15.59:0/1213829037 learned_addr learned my addr 172.21.15.59:0/1213829037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3643400640 1 --2- 172.21.15.59:0/1213829037 >> v2:172.21.15.66:3300/0 conn(0x7f36440d0b70 0x7f36440d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 -- 172.21.15.59:0/1213829037 >> v2:172.21.15.66:3300/0 conn(0x7f36440d0b70 msgr2=0x7f36440d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 --2- 172.21.15.59:0/1213829037 >> v2:172.21.15.66:3300/0 conn(0x7f36440d0b70 0x7f36440d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 -- 172.21.15.59:0/1213829037 >> v1:172.21.15.66:6789/0 conn(0x7f364415e200 legacy=0x7f364415e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.479+0000 7f3642a00640 1 -- 172.21.15.59:0/1213829037 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440cdcb0 con 0x7f36440cd250 2024-04-24T06:30:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.480+0000 7f3642a00640 1 --2- 172.21.15.59:0/1213829037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440cd250 0x7f36440d0f50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3644073910 tx=0x7f362c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6514559c9fee8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.481+0000 7f3642000640 1 -- 172.21.15.59:0/1213829037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c037070 con 0x7f36440cd250 2024-04-24T06:30:55.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.481+0000 7f3642000640 1 -- 172.21.15.59:0/1213829037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c032440 con 0x7f36440cd250 2024-04-24T06:30:55.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.481+0000 7f3642000640 1 -- 172.21.15.59:0/1213829037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c032810 con 0x7f36440cd250 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.481+0000 7f364a800640 1 -- 172.21.15.59:0/1213829037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440cd250 msgr2=0x7f36440d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.481+0000 7f364a800640 1 --2- 172.21.15.59:0/1213829037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440cd250 0x7f36440d0f50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3644073910 tx=0x7f362c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 -- 172.21.15.59:0/1213829037 shutdown_connections 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 --2- 172.21.15.59:0/1213829037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440cd250 0x7f36440d0f50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 --2- 172.21.15.59:0/1213829037 >> v2:172.21.15.66:3300/0 conn(0x7f36440d0b70 0x7f36440d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 -- 172.21.15.59:0/1213829037 >> 172.21.15.59:0/1213829037 conn(0x7f36440b4080 msgr2=0x7f36440b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 -- 172.21.15.59:0/1213829037 shutdown_connections 2024-04-24T06:30:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.482+0000 7f364a800640 1 -- 172.21.15.59:0/1213829037 wait complete. 2024-04-24T06:30:55.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.483+0000 7f364a800640 1 Processor -- start 2024-04-24T06:30:55.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 -- start start 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440cd250 0x7f364415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f364415e200 0x7f36440c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36440cd6c0 con 0x7f36440d0b70 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.484+0000 7f364a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36440ce270 con 0x7f36440cd250 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f364a800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f364415ed80 con 0x7f364415e200 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58676/0 (socket says 172.21.15.59:58676) 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 -- 172.21.15.59:0/1651850276 learned_addr learned my addr 172.21.15.59:0/1651850276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3643400640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440cd250 0x7f364415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 -- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f364415e200 msgr2=0x7f36440c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f364415e200 0x7f36440c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 -- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440cd250 msgr2=0x7f364415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440cd250 0x7f364415bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.485+0000 7f3642a00640 1 -- 172.21.15.59:0/1651850276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364415c190 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f3642a00640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f362c02a1e0 tx=0x7f362c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c041020 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c0306c0 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c037040 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440c7e40 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.486+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440c8400 con 0x7f36440d0b70 2024-04-24T06:30:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.487+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644068a80 con 0x7f36440d0b70 2024-04-24T06:30:55.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.488+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f362c044020 con 0x7f36440d0b70 2024-04-24T06:30:55.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.489+0000 7f363b400640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 0x7f362006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.489+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f362c0ac0d0 con 0x7f36440d0b70 2024-04-24T06:30:55.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.489+0000 7f3643400640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 0x7f362006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.490+0000 7f3643400640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 0x7f362006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e90 tx=0x7f363400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.494+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f362c04b030 con 0x7f36440d0b70 2024-04-24T06:30:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.699+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f364415f8a0 con 0x7f36440d0b70 2024-04-24T06:30:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.700+0000 7f363b400640 1 -- 172.21.15.59:0/1651850276 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f362c07a7b0 con 0x7f36440d0b70 2024-04-24T06:30:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:55.702 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:30:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:30:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:30:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:30:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:30:55.703 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.702+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 msgr2=0x7f362006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.702+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 0x7f362006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e90 tx=0x7f363400b040 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.702+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 msgr2=0x7f36440c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.702+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f362c02a1e0 tx=0x7f362c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 shutdown_connections 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f362006c8f0 0x7f362006edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f364415e200 0x7f36440c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36440d0b70 0x7f36440c9890 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 --2- 172.21.15.59:0/1651850276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440cd250 0x7f364415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 >> 172.21.15.59:0/1651850276 conn(0x7f36440b4080 msgr2=0x7f364415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:55.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 shutdown_connections 2024-04-24T06:30:55.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.703+0000 7f364a800640 1 -- 172.21.15.59:0/1651850276 wait complete. 2024-04-24T06:30:55.724 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:30:55.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:30:55.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.943+0000 7ff71ca00640 1 Processor -- start 2024-04-24T06:30:55.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.944+0000 7ff71ca00640 1 -- start start 2024-04-24T06:30:55.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.945+0000 7ff71ca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff71806c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.945+0000 7ff71ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff718160100 0x7ff718160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.945+0000 7ff71ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff718157000 con 0x7ff718156c20 2024-04-24T06:30:55.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.945+0000 7ff71ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff7180afb00 con 0x7ff718160100 2024-04-24T06:30:55.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.945+0000 7ff71ca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff71815cc20 con 0x7ff71806bdf0 2024-04-24T06:30:55.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff716800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff718156c20 0x7ff7180aef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37844/0 (socket says 172.21.15.59:37844) 2024-04-24T06:30:55.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff716800640 1 -- 172.21.15.59:0/1409578104 learned_addr learned my addr 172.21.15.59:0/1409578104 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:30:55.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363570689 0 0) 0x7ff718157000 con 0x7ff718156c20 2024-04-24T06:30:55.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6fc003520 con 0x7ff718156c20 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff70de00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff71806c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff717200640 1 --2- 172.21.15.59:0/1409578104 >> v2:172.21.15.66:3300/0 conn(0x7ff718160100 0x7ff718160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 961957690 0 0) 0x7ff6fc003520 con 0x7ff718156c20 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 msgr2=0x7ff71806c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff71806c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 >> v2:172.21.15.66:3300/0 conn(0x7ff718160100 msgr2=0x7ff718160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 --2- 172.21.15.59:0/1409578104 >> v2:172.21.15.66:3300/0 conn(0x7ff718160100 0x7ff718160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71815cf00 con 0x7ff718156c20 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff7080019f0 con 0x7ff718156c20 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.946+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 >> v1:172.21.15.59:6789/0 conn(0x7ff718156c20 legacy=0x7ff7180aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff6fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff6fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6fc004d40 0x7ff6fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7180afb00 con 0x7ff6fc004030 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff71815cc20 con 0x7ff6fc004d40 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff6fc00bb20 con 0x7ff71806bdf0 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff6fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff717200640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6fc004d40 0x7ff6fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff716800640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff6fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 -- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 msgr2=0x7ff6fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff6fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 -- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6fc004d40 msgr2=0x7ff6fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6fc004d40 0x7ff6fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff70de00640 1 -- 172.21.15.59:0/1409578104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc003520 con 0x7ff6fc004030 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff717200640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6fc004d40 0x7ff6fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:55.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.947+0000 7ff716800640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71806bdf0 0x7ff6fc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:55.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff70de00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff6fc004800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ba30 tx=0x7ff6fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d5358df60bda4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700030070 con 0x7ff6fc004030 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70002add0 con 0x7ff6fc004030 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff715e00640 1 -- 172.21.15.59:0/1409578104 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70000b600 con 0x7ff6fc004030 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff71ca00640 1 -- 172.21.15.59:0/1409578104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 msgr2=0x7ff6fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1409578104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff6fc004800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ba30 tx=0x7ff6fc005830 comp rx=0 tx=0).stop 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.949+0000 7ff717200640 1 -- 172.21.15.59:0/1409578104 reap_dead start 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.950+0000 7ff71ca00640 1 -- 172.21.15.59:0/1409578104 shutdown_connections 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.950+0000 7ff71ca00640 1 -- 172.21.15.59:0/1409578104 >> 172.21.15.59:0/1409578104 conn(0x7ff7180b2200 msgr2=0x7ff7180b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.950+0000 7ff71ca00640 1 -- 172.21.15.59:0/1409578104 shutdown_connections 2024-04-24T06:30:55.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.950+0000 7ff71ca00640 1 -- 172.21.15.59:0/1409578104 wait complete. 2024-04-24T06:30:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.951+0000 7ff71ca00640 1 Processor -- start 2024-04-24T06:30:55.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.952+0000 7ff71ca00640 1 -- start start 2024-04-24T06:30:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71814e270 0x7ff71814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 0x7ff718152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff71815d9e0 con 0x7ff6fc004030 2024-04-24T06:30:55.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff71815d4c0 con 0x7ff71815b140 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff71ca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff718073ba0 con 0x7ff71814e270 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58686/0 (socket says 172.21.15.59:58686) 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 -- 172.21.15.59:0/1404362921 learned_addr learned my addr 172.21.15.59:0/1404362921 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff717200640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 0x7ff718152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff70de00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71814e270 0x7ff71814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 -- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71814e270 msgr2=0x7ff71814e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71814e270 0x7ff71814e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 -- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 msgr2=0x7ff718152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 0x7ff718152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff716800640 1 -- 172.21.15.59:0/1404362921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181f5220 con 0x7ff6fc004030 2024-04-24T06:30:55.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.953+0000 7ff717200640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 0x7ff718152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff716800640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff708002130 tx=0x7ff70800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708019070 con 0x7ff6fc004030 2024-04-24T06:30:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff708004d60 con 0x7ff6fc004030 2024-04-24T06:30:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7080072d0 con 0x7ff6fc004030 2024-04-24T06:30:55.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181f4cc0 con 0x7ff6fc004030 2024-04-24T06:30:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.954+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181f7950 con 0x7ff6fc004030 2024-04-24T06:30:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.955+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71804fd90 con 0x7ff6fc004030 2024-04-24T06:30:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.957+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff708007470 con 0x7ff6fc004030 2024-04-24T06:30:55.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.958+0000 7ff70f400640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 0x7ff6f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:55.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.958+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7ff70808b9a0 con 0x7ff6fc004030 2024-04-24T06:30:55.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.958+0000 7ff70de00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 0x7ff6f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:55.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.959+0000 7ff70de00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 0x7ff6f406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ba00 tx=0x7ff700030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:55.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:55.962+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7181f4cc0 con 0x7ff6fc004030 2024-04-24T06:30:56.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.171+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff71815b550 con 0x7ff6fc004030 2024-04-24T06:30:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.794+0000 7ff70f400640 1 -- 172.21.15.59:0/1404362921 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7ff70805a080 con 0x7ff6fc004030 2024-04-24T06:30:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:30:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 msgr2=0x7ff6f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 0x7ff6f406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff70000ba00 tx=0x7ff700030000 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 msgr2=0x7ff7181f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff708002130 tx=0x7ff70800b4a0 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 shutdown_connections 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff6f406c920 0x7ff6f406ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff71815b140 0x7ff718152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff71814e270 0x7ff71814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 --2- 172.21.15.59:0/1404362921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6fc004030 0x7ff7181f8430 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.798+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 >> 172.21.15.59:0/1404362921 conn(0x7ff7180b2200 msgr2=0x7ff71806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.799+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 shutdown_connections 2024-04-24T06:30:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:56.799+0000 7ff71ca00640 1 -- 172.21.15.59:0/1404362921 wait complete. 2024-04-24T06:30:56.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:30:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.040+0000 7f0296200640 1 Processor -- start 2024-04-24T06:30:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.040+0000 7f0296200640 1 -- start start 2024-04-24T06:30:57.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0296200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f029015c3e0 0x7f029015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0296200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f02900b2930 0x7f0290162f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0296200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02900d1ea0 con 0x7f02900d0810 2024-04-24T06:30:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0296200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02900d0d90 con 0x7f029015c3e0 2024-04-24T06:30:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0296200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f02900d1070 con 0x7f02900b2930 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0294e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f029015c3e0 0x7f029015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0294e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f029015c3e0 0x7f029015fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58702/0 (socket says 172.21.15.59:58702) 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0294e00640 1 -- 172.21.15.59:0/1678350910 learned_addr learned my addr 172.21.15.59:0/1678350910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.042+0000 7f0295800640 1 --2- 172.21.15.59:0/1678350910 >> v2:172.21.15.66:3300/0 conn(0x7f02900b2930 0x7f0290162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.043+0000 7f0294e00640 1 -- 172.21.15.59:0/1678350910 >> v2:172.21.15.66:3300/0 conn(0x7f02900b2930 msgr2=0x7f0290162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.043+0000 7f0294e00640 1 --2- 172.21.15.59:0/1678350910 >> v2:172.21.15.66:3300/0 conn(0x7f02900b2930 0x7f0290162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.043+0000 7f0294e00640 1 -- 172.21.15.59:0/1678350910 >> v1:172.21.15.66:6789/0 conn(0x7f02900d0810 legacy=0x7f02900b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.043+0000 7f0294e00640 1 -- 172.21.15.59:0/1678350910 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029015f040 con 0x7f029015c3e0 2024-04-24T06:30:57.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.044+0000 7f0294e00640 1 --2- 172.21.15.59:0/1678350910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029015c3e0 0x7f029015fb20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f02900b7210 tx=0x7f028002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88d83780331d74e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:57.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.044+0000 7f028f400640 1 -- 172.21.15.59:0/1678350910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280037070 con 0x7f029015c3e0 2024-04-24T06:30:57.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.044+0000 7f028f400640 1 -- 172.21.15.59:0/1678350910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280032440 con 0x7f029015c3e0 2024-04-24T06:30:57.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.044+0000 7f028f400640 1 -- 172.21.15.59:0/1678350910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280032810 con 0x7f029015c3e0 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 -- 172.21.15.59:0/1678350910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029015c3e0 msgr2=0x7f029015fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 --2- 172.21.15.59:0/1678350910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029015c3e0 0x7f029015fb20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f02900b7210 tx=0x7f028002a230 comp rx=0 tx=0).stop 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 -- 172.21.15.59:0/1678350910 shutdown_connections 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 --2- 172.21.15.59:0/1678350910 >> v2:172.21.15.66:3300/0 conn(0x7f02900b2930 0x7f0290162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 --2- 172.21.15.59:0/1678350910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029015c3e0 0x7f029015fb20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 -- 172.21.15.59:0/1678350910 >> 172.21.15.59:0/1678350910 conn(0x7f029009aa00 msgr2=0x7f02900d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 -- 172.21.15.59:0/1678350910 shutdown_connections 2024-04-24T06:30:57.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.045+0000 7f0296200640 1 -- 172.21.15.59:0/1678350910 wait complete. 2024-04-24T06:30:57.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.047+0000 7f0296200640 1 Processor -- start 2024-04-24T06:30:57.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.047+0000 7f0296200640 1 -- start start 2024-04-24T06:30:57.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02900d0810 0x7f02901e8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f029015c3e0 0x7f02901e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f029015c7c0 con 0x7f02900b2930 2024-04-24T06:30:57.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f029015f600 con 0x7f029015c3e0 2024-04-24T06:30:57.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0296200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f029015c940 con 0x7f02900d0810 2024-04-24T06:30:57.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:57.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58714/0 (socket says 172.21.15.59:58714) 2024-04-24T06:30:57.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.048+0000 7f0294e00640 1 -- 172.21.15.59:0/3855202752 learned_addr learned my addr 172.21.15.59:0/3855202752 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0295800640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f029015c3e0 0x7f02901e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 -- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02900d0810 msgr2=0x7f02901e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02900d0810 0x7f02901e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 -- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f029015c3e0 msgr2=0x7f02901e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f029015c3e0 0x7f02901e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 -- 172.21.15.59:0/3855202752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901484e0 con 0x7f02900b2930 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.049+0000 7f0294e00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f028002a270 tx=0x7f02800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.050+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280041070 con 0x7f02900b2930 2024-04-24T06:30:57.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.050+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028003c870 con 0x7f02900b2930 2024-04-24T06:30:57.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.050+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280037040 con 0x7f02900b2930 2024-04-24T06:30:57.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.050+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901487c0 con 0x7f02900b2930 2024-04-24T06:30:57.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.050+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0290148d80 con 0x7f02900b2930 2024-04-24T06:30:57.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.051+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029004fd90 con 0x7f02900b2930 2024-04-24T06:30:57.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.052+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0280044070 con 0x7f02900b2930 2024-04-24T06:30:57.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.053+0000 7f028cc00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:57.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.054+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f02800ac830 con 0x7f02900b2930 2024-04-24T06:30:57.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.054+0000 7f028fe00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:57.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.055+0000 7f028fe00640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 0x7f026c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0290070a50 tx=0x7f0278009230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.057+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f028007ae20 con 0x7f02900b2930 2024-04-24T06:30:57.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.264+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f02900b14e0 con 0x7f02900b2930 2024-04-24T06:30:57.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.800+0000 7f028cc00640 1 -- 172.21.15.59:0/3855202752 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f028007a720 con 0x7f02900b2930 2024-04-24T06:30:57.802 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:30:57.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 msgr2=0x7f026c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 0x7f026c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0290070a50 tx=0x7f0278009230 comp rx=0 tx=0).stop 2024-04-24T06:30:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 msgr2=0x7f02900b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f028002a270 tx=0x7f02800046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 shutdown_connections 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f029015c3e0 0x7f02901e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02900d0810 0x7f02901e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 --2- 172.21.15.59:0/3855202752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02900b2930 0x7f02900b46e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 >> 172.21.15.59:0/3855202752 conn(0x7f029009aa00 msgr2=0x7f029015d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 shutdown_connections 2024-04-24T06:30:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:57.804+0000 7f0296200640 1 -- 172.21.15.59:0/3855202752 wait complete. 2024-04-24T06:30:57.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:30:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.025+0000 7f750e400640 1 Processor -- start 2024-04-24T06:30:58.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.025+0000 7f750e400640 1 -- start start 2024-04-24T06:30:58.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f750e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75080d0cc0 0x7f750806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f750e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7508071d70 0x7f7508072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f750e400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75080cfc20 con 0x7f7508145e10 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f750e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f75080cff00 con 0x7f7508071d70 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f750e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f75080d01e0 con 0x7f75080d0cc0 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f7507e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7508071d70 0x7f7508072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f7507e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7508071d70 0x7f7508072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58722/0 (socket says 172.21.15.59:58722) 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.027+0000 7f7507e00640 1 -- 172.21.15.59:0/3528158010 learned_addr learned my addr 172.21.15.59:0/3528158010 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.028+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3528158010 >> v2:172.21.15.66:3300/0 conn(0x7f75080d0cc0 0x7f750806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.028+0000 7f7507e00640 1 -- 172.21.15.59:0/3528158010 >> v2:172.21.15.66:3300/0 conn(0x7f75080d0cc0 msgr2=0x7f750806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:58.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.028+0000 7f7507e00640 1 --2- 172.21.15.59:0/3528158010 >> v2:172.21.15.66:3300/0 conn(0x7f75080d0cc0 0x7f750806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.028+0000 7f7507e00640 1 -- 172.21.15.59:0/3528158010 >> v1:172.21.15.66:6789/0 conn(0x7f7508145e10 legacy=0x7f75080cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.028+0000 7f7507e00640 1 -- 172.21.15.59:0/3528158010 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080d04c0 con 0x7f7508071d70 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.029+0000 7f7507e00640 1 --2- 172.21.15.59:0/3528158010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f7508072200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7500009e50 tx=0x7f750002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f762485cd0ec5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.029+0000 7f7506a00640 1 -- 172.21.15.59:0/3528158010 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500036070 con 0x7f7508071d70 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.029+0000 7f7506a00640 1 -- 172.21.15.59:0/3528158010 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7500031600 con 0x7f7508071d70 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.029+0000 7f7506a00640 1 -- 172.21.15.59:0/3528158010 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75000319d0 con 0x7f7508071d70 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 -- 172.21.15.59:0/3528158010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 msgr2=0x7f7508072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:58.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 --2- 172.21.15.59:0/3528158010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f7508072200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7500009e50 tx=0x7f750002a710 comp rx=0 tx=0).stop 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 -- 172.21.15.59:0/3528158010 shutdown_connections 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 --2- 172.21.15.59:0/3528158010 >> v2:172.21.15.66:3300/0 conn(0x7f75080d0cc0 0x7f750806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 --2- 172.21.15.59:0/3528158010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f7508072200 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 -- 172.21.15.59:0/3528158010 >> 172.21.15.59:0/3528158010 conn(0x7f7508067d90 msgr2=0x7f750806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.030+0000 7f750e400640 1 -- 172.21.15.59:0/3528158010 shutdown_connections 2024-04-24T06:30:58.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.031+0000 7f750e400640 1 -- 172.21.15.59:0/3528158010 wait complete. 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.032+0000 7f750e400640 1 Processor -- start 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.033+0000 7f750e400640 1 -- start start 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75080d0cc0 0x7f75080b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7508145e10 0x7f75081f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75081461f0 con 0x7f7508071d70 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f750815cdd0 con 0x7f75080d0cc0 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f750e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7508142350 con 0x7f7508145e10 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f7507400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f7507400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58728/0 (socket says 172.21.15.59:58728) 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.034+0000 7f7507400640 1 -- 172.21.15.59:0/3472568481 learned_addr learned my addr 172.21.15.59:0/3472568481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75080d0cc0 0x7f75080b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507e00640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7508145e10 0x7f75081f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 -- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7508145e10 msgr2=0x7f75081f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7508145e10 0x7f75081f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 -- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75080d0cc0 msgr2=0x7f75080b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75080d0cc0 0x7f75080b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 -- 172.21.15.59:0/3472568481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080b4e50 con 0x7f7508071d70 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f7507400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b3e0 tx=0x7f74f800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8015020 con 0x7f7508071d70 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f8011850 con 0x7f7508071d70 2024-04-24T06:30:58.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011c40 con 0x7f7508071d70 2024-04-24T06:30:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.035+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75080b45b0 con 0x7f7508071d70 2024-04-24T06:30:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.036+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75080b4b70 con 0x7f7508071d70 2024-04-24T06:30:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.037+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750806ebc0 con 0x7f7508071d70 2024-04-24T06:30:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.038+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74f8018020 con 0x7f7508071d70 2024-04-24T06:30:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.039+0000 7f74ff400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 0x7f74e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.039+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7f74f808bac0 con 0x7f7508071d70 2024-04-24T06:30:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.039+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 0x7f74e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.040+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 0x7f74e406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.044+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f750806ebc0 con 0x7f7508071d70 2024-04-24T06:30:58.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.289+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7508068a80 con 0x7f7508071d70 2024-04-24T06:30:58.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.855+0000 7f74ff400640 1 -- 172.21.15.59:0/3472568481 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7f74f801f020 con 0x7f7508071d70 2024-04-24T06:30:58.859 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 6 and data pool 7 2024-04-24T06:30:58.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.864+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 msgr2=0x7f74e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.864+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 0x7f74e406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f0008040 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.864+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 msgr2=0x7f75080b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.864+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b3e0 tx=0x7f74f800b8a0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 shutdown_connections 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f74e406c8f0 0x7f74e406edb0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f0008040 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7508145e10 0x7f75081f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75080d0cc0 0x7f75080b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 --2- 172.21.15.59:0/3472568481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7508071d70 0x7f75080b3e70 secure :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b3e0 tx=0x7f74f800b8a0 comp rx=0 tx=0).stop 2024-04-24T06:30:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.865+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 >> 172.21.15.59:0/3472568481 conn(0x7f7508067d90 msgr2=0x7f750815d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.866+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 shutdown_connections 2024-04-24T06:30:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:58.867+0000 7f750e400640 1 -- 172.21.15.59:0/3472568481 wait complete. 2024-04-24T06:30:58.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.078+0000 7f0467200640 1 Processor -- start 2024-04-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.078+0000 7f0467200640 1 -- start start 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.079+0000 7f0467200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04680cd050 0x7f046806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.079+0000 7f0467200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f046815ece0 0x7f04680cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.079+0000 7f0467200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680b3080 0x7f04680cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.079+0000 7f0467200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f04680ced20 con 0x7f04680cd050 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0467200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f046815fcd0 con 0x7f046815ece0 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0467200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04680cd970 con 0x7f04680b3080 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04680cd050 0x7f046806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04680cd050 0x7f046806f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46956/0 (socket says 172.21.15.59:46956) 2024-04-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 -- 172.21.15.59:0/3571436089 learned_addr learned my addr 172.21.15.59:0/3571436089 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465400640 1 --2- 172.21.15.59:0/3571436089 >> v2:172.21.15.66:3300/0 conn(0x7f046815ece0 0x7f04680cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 -- 172.21.15.59:0/3571436089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680b3080 msgr2=0x7f04680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 --2- 172.21.15.59:0/3571436089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680b3080 0x7f04680cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 -- 172.21.15.59:0/3571436089 >> v2:172.21.15.66:3300/0 conn(0x7f046815ece0 msgr2=0x7f04680cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 --2- 172.21.15.59:0/3571436089 >> v2:172.21.15.66:3300/0 conn(0x7f046815ece0 0x7f04680cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.080+0000 7f0465e00640 1 -- 172.21.15.59:0/3571436089 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046815f160 con 0x7f04680cd050 2024-04-24T06:30:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.081+0000 7f0465e00640 1 --2- 172.21.15.59:0/3571436089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680cd050 0x7f046806f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f046806e210 tx=0x7f045802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec663ad1a5a9f9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.082+0000 7f0464a00640 1 -- 172.21.15.59:0/3571436089 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458037070 con 0x7f04680cd050 2024-04-24T06:30:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.082+0000 7f0464a00640 1 -- 172.21.15.59:0/3571436089 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0458032440 con 0x7f04680cd050 2024-04-24T06:30:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.082+0000 7f0464a00640 1 -- 172.21.15.59:0/3571436089 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458032810 con 0x7f04680cd050 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.082+0000 7f0467200640 1 -- 172.21.15.59:0/3571436089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680cd050 msgr2=0x7f046806f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.082+0000 7f0467200640 1 --2- 172.21.15.59:0/3571436089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680cd050 0x7f046806f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f046806e210 tx=0x7f045802a140 comp rx=0 tx=0).stop 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 -- 172.21.15.59:0/3571436089 shutdown_connections 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 --2- 172.21.15.59:0/3571436089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680b3080 0x7f04680cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 --2- 172.21.15.59:0/3571436089 >> v2:172.21.15.66:3300/0 conn(0x7f046815ece0 0x7f04680cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 --2- 172.21.15.59:0/3571436089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680cd050 0x7f046806f620 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 -- 172.21.15.59:0/3571436089 >> 172.21.15.59:0/3571436089 conn(0x7f04680b21a0 msgr2=0x7f04680b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 -- 172.21.15.59:0/3571436089 shutdown_connections 2024-04-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.083+0000 7f0467200640 1 -- 172.21.15.59:0/3571436089 wait complete. 2024-04-24T06:30:59.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.084+0000 7f0467200640 1 Processor -- start 2024-04-24T06:30:59.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.085+0000 7f0467200640 1 -- start start 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680cd050 0x7f04681f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046815ece0 0x7f046815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f046815f9f0 con 0x7f04680b3080 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f046815f710 con 0x7f046815ece0 2024-04-24T06:30:59.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0467200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04680cef50 con 0x7f04680cd050 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46970/0 (socket says 172.21.15.59:46970) 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 -- 172.21.15.59:0/1587298721 learned_addr learned my addr 172.21.15.59:0/1587298721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 -- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680cd050 msgr2=0x7f04681f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680cd050 0x7f04681f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 -- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046815ece0 msgr2=0x7f046815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046815ece0 0x7f046815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.086+0000 7f0465e00640 1 -- 172.21.15.59:0/1587298721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04681f8cf0 con 0x7f04680b3080 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f0465e00640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0458000c00 tx=0x7f04580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458041020 con 0x7f04680b3080 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0458030820 con 0x7f04680b3080 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458037040 con 0x7f04680b3080 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04681f4de0 con 0x7f04680b3080 2024-04-24T06:30:59.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.087+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681f53a0 con 0x7f04680b3080 2024-04-24T06:30:59.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.089+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046804fd90 con 0x7f04680b3080 2024-04-24T06:30:59.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.089+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0458044020 con 0x7f04680b3080 2024-04-24T06:30:59.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.091+0000 7f045e000640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.091+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f04580ac4f0 con 0x7f04680b3080 2024-04-24T06:30:59.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.091+0000 7f0465400640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.092+0000 7f0465400640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 0x7f044806edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0454006fd0 tx=0x7f0454008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.095+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f045804b030 con 0x7f04680b3080 2024-04-24T06:30:59.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.307+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04680d03e0 con 0x7f04680b3080 2024-04-24T06:30:59.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.308+0000 7f045e000640 1 -- 172.21.15.59:0/1587298721 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16542 (secure 0 0 0) 0x7f04681f8cf0 con 0x7f04680b3080 2024-04-24T06:30:59.311 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:59.311 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:58.292077+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 msgr2=0x7f044806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 0x7f044806edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0454006fd0 tx=0x7f0454008040 comp rx=0 tx=0).stop 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 msgr2=0x7f04681f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0458000c00 tx=0x7f04580046c0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 shutdown_connections 2024-04-24T06:30:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f046815ece0 0x7f046815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04680cd050 0x7f04681f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 --2- 172.21.15.59:0/1587298721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04680b3080 0x7f04681f4780 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 >> 172.21.15.59:0/1587298721 conn(0x7f04680b21a0 msgr2=0x7f04680cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 shutdown_connections 2024-04-24T06:30:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.311+0000 7f0467200640 1 -- 172.21.15.59:0/1587298721 wait complete. 2024-04-24T06:30:59.333 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.538+0000 7f3315000640 1 Processor -- start 2024-04-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.538+0000 7f3315000640 1 -- start start 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.539+0000 7f3315000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f331015c3d0 0x7f33100cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.539+0000 7f3315000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100cd590 0x7f33100b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f3315000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3310160210 con 0x7f33100b4690 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f3315000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f33101604f0 con 0x7f331015c3d0 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f3315000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f33101607d0 con 0x7f33100cd590 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f331015c3d0 0x7f33100cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f331015c3d0 0x7f33100cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46982/0 (socket says 172.21.15.59:46982) 2024-04-24T06:30:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 -- 172.21.15.59:0/3033196949 learned_addr learned my addr 172.21.15.59:0/3033196949 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:59.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f3307e00640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100cd590 0x7f33100b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 -- 172.21.15.59:0/3033196949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100cd590 msgr2=0x7f33100b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100cd590 0x7f33100b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.540+0000 7f330f400640 1 -- 172.21.15.59:0/3033196949 >> v1:172.21.15.59:6789/0 conn(0x7f33100b4690 legacy=0x7f331006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.541+0000 7f330ea00640 1 -- 172.21.15.59:0/3033196949 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138010490 0 0) 0x7f3310160210 con 0x7f33100b4690 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.541+0000 7f330f400640 1 -- 172.21.15.59:0/3033196949 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310160ab0 con 0x7f331015c3d0 2024-04-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f330f400640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015c3d0 0x7f33100cd050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3300009870 tx=0x7f330002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c38d8d65f6628c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f330ea00640 1 -- 172.21.15.59:0/3033196949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300037070 con 0x7f331015c3d0 2024-04-24T06:30:59.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f330ea00640 1 -- 172.21.15.59:0/3033196949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3300032440 con 0x7f331015c3d0 2024-04-24T06:30:59.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f330ea00640 1 -- 172.21.15.59:0/3033196949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300032810 con 0x7f331015c3d0 2024-04-24T06:30:59.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f3315000640 1 -- 172.21.15.59:0/3033196949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015c3d0 msgr2=0x7f33100cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.542+0000 7f3315000640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015c3d0 0x7f33100cd050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3300009870 tx=0x7f330002a250 comp rx=0 tx=0).stop 2024-04-24T06:30:59.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 -- 172.21.15.59:0/3033196949 shutdown_connections 2024-04-24T06:30:59.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100cd590 0x7f33100b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 --2- 172.21.15.59:0/3033196949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f331015c3d0 0x7f33100cd050 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 -- 172.21.15.59:0/3033196949 >> 172.21.15.59:0/3033196949 conn(0x7f33100b21a0 msgr2=0x7f33100b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:59.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 -- 172.21.15.59:0/3033196949 shutdown_connections 2024-04-24T06:30:59.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.543+0000 7f3315000640 1 -- 172.21.15.59:0/3033196949 wait complete. 2024-04-24T06:30:59.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.544+0000 7f3315000640 1 Processor -- start 2024-04-24T06:30:59.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 -- start start 2024-04-24T06:30:59.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100b4690 0x7f33100c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 0x7f33100c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f331015c7b0 con 0x7f33100cd590 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33100b4b00 con 0x7f331015c3d0 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.545+0000 7f3315000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f331015c930 con 0x7f33100b4690 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46990/0 (socket says 172.21.15.59:46990) 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 -- 172.21.15.59:0/2950821947 learned_addr learned my addr 172.21.15.59:0/2950821947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f330f400640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100b4690 0x7f33100c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f330fe00640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 0x7f33100c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 -- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100b4690 msgr2=0x7f33100c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100b4690 0x7f33100c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 -- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 msgr2=0x7f33100c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 0x7f33100c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f3307e00640 1 -- 172.21.15.59:0/2950821947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101f5ae0 con 0x7f33100cd590 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.546+0000 7f330fe00640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 0x7f33100c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3307e00640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32f800d830 tx=0x7f32f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800b6c0 con 0x7f33100cd590 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f8019070 con 0x7f33100cd590 2024-04-24T06:30:59.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800ba80 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33100cde20 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.547+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33100ce3e0 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.548+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331004fd90 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.549+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32f801e460 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.550+0000 7f3307400640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.550+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f32f808c2d0 con 0x7f33100cd590 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.550+0000 7f330f400640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:30:59.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.551+0000 7f330f400640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 0x7f32ec06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3300034000 tx=0x7f330002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:30:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.555+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32f805a660 con 0x7f33100cd590 2024-04-24T06:30:59.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.764+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f331006e400 con 0x7f33100cd590 2024-04-24T06:30:59.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.767+0000 7f3307400640 1 -- 172.21.15.59:0/2950821947 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16542 (secure 0 0 0) 0x7f32f8059f60 con 0x7f33100cd590 2024-04-24T06:30:59.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:30:59.769 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:58.292077+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:30:59.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 msgr2=0x7f32ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 0x7f32ec06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3300034000 tx=0x7f330002e000 comp rx=0 tx=0).stop 2024-04-24T06:30:59.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 msgr2=0x7f33100998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:30:59.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32f800d830 tx=0x7f32f800dcf0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 shutdown_connections 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32ec06c8f0 0x7f32ec06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f331015c3d0 0x7f33100c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33100cd590 0x7f33100998d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 --2- 172.21.15.59:0/2950821947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f33100b4690 0x7f33100c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.769+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 >> 172.21.15.59:0/2950821947 conn(0x7f33100b21a0 msgr2=0x7f33100d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:30:59.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.770+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 shutdown_connections 2024-04-24T06:30:59.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:30:59.770+0000 7f3315000640 1 -- 172.21.15.59:0/2950821947 wait complete. 2024-04-24T06:30:59.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:31:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.005+0000 7f417ae00640 1 Processor -- start 2024-04-24T06:31:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.005+0000 7f417ae00640 1 -- start start 2024-04-24T06:31:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f417415ed20 0x7f41740d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41740b3600 0x7f41740d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417ae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41740b5da0 con 0x7f41740b3f10 2024-04-24T06:31:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f417415f100 con 0x7f41740b3600 2024-04-24T06:31:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f41740d0a20 con 0x7f417415ed20 2024-04-24T06:31:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f4179000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41740b3600 0x7f41740d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417a400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f41740b3f10 0x7f41740b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48248/0 (socket says 172.21.15.59:48248) 2024-04-24T06:31:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.006+0000 7f417a400640 1 -- 172.21.15.59:0/1591211996 learned_addr learned my addr 172.21.15.59:0/1591211996 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4179a00640 1 --2- 172.21.15.59:0/1591211996 >> v2:172.21.15.66:3300/0 conn(0x7f417415ed20 0x7f41740d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4173e00640 1 -- 172.21.15.59:0/1591211996 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942311312 0 0) 0x7f41740b5da0 con 0x7f41740b3f10 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4173e00640 1 -- 172.21.15.59:0/1591211996 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4158003520 con 0x7f41740b3f10 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4179000640 1 -- 172.21.15.59:0/1591211996 >> v2:172.21.15.66:3300/0 conn(0x7f417415ed20 msgr2=0x7f41740d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4179000640 1 --2- 172.21.15.59:0/1591211996 >> v2:172.21.15.66:3300/0 conn(0x7f417415ed20 0x7f41740d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4179000640 1 -- 172.21.15.59:0/1591211996 >> v1:172.21.15.59:6789/0 conn(0x7f41740b3f10 legacy=0x7f41740b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.007+0000 7f4179000640 1 -- 172.21.15.59:0/1591211996 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41740d2ed0 con 0x7f41740b3600 2024-04-24T06:31:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.009+0000 7f4179000640 1 --2- 172.21.15.59:0/1591211996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3600 0x7f41740d13e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f417408e2a0 tx=0x7f416002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c88647d2b12607 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.009+0000 7f4173e00640 1 -- 172.21.15.59:0/1591211996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160037070 con 0x7f41740b3600 2024-04-24T06:31:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.009+0000 7f4173e00640 1 -- 172.21.15.59:0/1591211996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416002add0 con 0x7f41740b3600 2024-04-24T06:31:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.009+0000 7f4173e00640 1 -- 172.21.15.59:0/1591211996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160032640 con 0x7f41740b3600 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 -- 172.21.15.59:0/1591211996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3600 msgr2=0x7f41740d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 --2- 172.21.15.59:0/1591211996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3600 0x7f41740d13e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f417408e2a0 tx=0x7f416002a080 comp rx=0 tx=0).stop 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 -- 172.21.15.59:0/1591211996 shutdown_connections 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 --2- 172.21.15.59:0/1591211996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3600 0x7f41740d13e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 --2- 172.21.15.59:0/1591211996 >> v2:172.21.15.66:3300/0 conn(0x7f417415ed20 0x7f41740d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 -- 172.21.15.59:0/1591211996 >> 172.21.15.59:0/1591211996 conn(0x7f417408bf90 msgr2=0x7f41740d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 -- 172.21.15.59:0/1591211996 shutdown_connections 2024-04-24T06:31:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.010+0000 7f417ae00640 1 -- 172.21.15.59:0/1591211996 wait complete. 2024-04-24T06:31:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.012+0000 7f417ae00640 1 Processor -- start 2024-04-24T06:31:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.012+0000 7f417ae00640 1 -- start start 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41740b3600 0x7f41741ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f417415ed20 0x7f41741e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41740d0290 con 0x7f41740b3f10 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41740d3490 con 0x7f41740b3600 2024-04-24T06:31:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f417ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f41740d0410 con 0x7f417415ed20 2024-04-24T06:31:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f4179000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f4179000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47008/0 (socket says 172.21.15.59:47008) 2024-04-24T06:31:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.013+0000 7f4179000640 1 -- 172.21.15.59:0/1254349689 learned_addr learned my addr 172.21.15.59:0/1254349689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 -- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f417415ed20 msgr2=0x7f41741e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f417415ed20 0x7f41741e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 -- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41740b3600 msgr2=0x7f41741ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41740b3600 0x7f41741ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 -- 172.21.15.59:0/1254349689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41741f5860 con 0x7f41740b3f10 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4179000640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160041030 con 0x7f41740b3f10 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416002abd0 con 0x7f41740b3f10 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.014+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160037040 con 0x7f41740b3f10 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.015+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41740cbc00 con 0x7f41740b3f10 2024-04-24T06:31:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.015+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41740cc1c0 con 0x7f41740b3f10 2024-04-24T06:31:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.016+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417404fd90 con 0x7f41740b3f10 2024-04-24T06:31:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.017+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4160044070 con 0x7f41740b3f10 2024-04-24T06:31:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.018+0000 7f4171600640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 0x7f415406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.018+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f41600abff0 con 0x7f41740b3f10 2024-04-24T06:31:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.018+0000 7f4179a00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 0x7f415406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:00.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.019+0000 7f4179a00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 0x7f415406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:00.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.023+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41741f5860 con 0x7f41740b3f10 2024-04-24T06:31:00.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.272+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f417406e3f0 con 0x7f41740b3f10 2024-04-24T06:31:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.866+0000 7f4171600640 1 -- 172.21.15.59:0/1254349689 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f416007a430 con 0x7f41740b3f10 2024-04-24T06:31:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.869+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 msgr2=0x7f415406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.869+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 0x7f415406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164008040 comp rx=0 tx=0).stop 2024-04-24T06:31:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.869+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 msgr2=0x7f41741f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.869+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 shutdown_connections 2024-04-24T06:31:00.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415406c8f0 0x7f415406edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f417415ed20 0x7f41741e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b3f10 0x7f41741f5320 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 --2- 172.21.15.59:0/1254349689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41740b3600 0x7f41741ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:00.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 >> 172.21.15.59:0/1254349689 conn(0x7f417408bf90 msgr2=0x7f4174156070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:00.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 shutdown_connections 2024-04-24T06:31:00.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:00.870+0000 7f417ae00640 1 -- 172.21.15.59:0/1254349689 wait complete. 2024-04-24T06:31:00.894 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:31:00.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:31:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.109+0000 7f6f5dc00640 1 Processor -- start 2024-04-24T06:31:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.109+0000 7f6f5dc00640 1 -- start start 2024-04-24T06:31:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.110+0000 7f6f5dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f581567e0 0x7f6f58156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.110+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f58160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f5dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f5815cab0 con 0x7f6f5815d810 2024-04-24T06:31:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f5dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6f58160e40 con 0x7f6f581567e0 2024-04-24T06:31:01.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f5dc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5815cd30 con 0x7f6f580d0d10 2024-04-24T06:31:01.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f57e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f581567e0 0x7f6f58156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f57e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f581567e0 0x7f6f58156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47014/0 (socket says 172.21.15.59:47014) 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f57e00640 1 -- 172.21.15.59:0/1437046850 learned_addr learned my addr 172.21.15.59:0/1437046850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f57e00640 1 -- 172.21.15.59:0/1437046850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 msgr2=0x7f6f58160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f5d200640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f58160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.111+0000 7f6f57e00640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f58160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.112+0000 7f6f57e00640 1 -- 172.21.15.59:0/1437046850 >> v1:172.21.15.59:6789/0 conn(0x7f6f5815d810 legacy=0x7f6f580d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.112+0000 7f6f57e00640 1 -- 172.21.15.59:0/1437046850 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5815d010 con 0x7f6f581567e0 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.112+0000 7f6f5d200640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f58160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.112+0000 7f6f56a00640 1 -- 172.21.15.59:0/1437046850 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428156901 0 0) 0x7f6f40001110 con 0x7f6f5815d810 2024-04-24T06:31:01.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f57e00640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f58156bc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009e50 tx=0x7f6f4802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d6988d8a1c4052 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f56a00640 1 -- 172.21.15.59:0/1437046850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48036070 con 0x7f6f581567e0 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f56a00640 1 -- 172.21.15.59:0/1437046850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f48031600 con 0x7f6f581567e0 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f56a00640 1 -- 172.21.15.59:0/1437046850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480319d0 con 0x7f6f581567e0 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f5dc00640 1 -- 172.21.15.59:0/1437046850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 msgr2=0x7f6f58156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.113+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f58156bc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009e50 tx=0x7f6f4802a750 comp rx=0 tx=0).stop 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 -- 172.21.15.59:0/1437046850 shutdown_connections 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f58160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/1437046850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f58156bc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 -- 172.21.15.59:0/1437046850 >> 172.21.15.59:0/1437046850 conn(0x7f6f5806bdf0 msgr2=0x7f6f5806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 -- 172.21.15.59:0/1437046850 shutdown_connections 2024-04-24T06:31:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.114+0000 7f6f5dc00640 1 -- 172.21.15.59:0/1437046850 wait complete. 2024-04-24T06:31:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.115+0000 7f6f5dc00640 1 Processor -- start 2024-04-24T06:31:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.116+0000 7f6f5dc00640 1 -- start start 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.116+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f5814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.116+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.116+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815d810 0x7f6f5815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.116+0000 7f6f5dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f581607d0 con 0x7f6f581567e0 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f5dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f58160650 con 0x7f6f5815d810 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f5dc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6f58160950 con 0x7f6f580d0d10 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47028/0 (socket says 172.21.15.59:47028) 2024-04-24T06:31:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 -- 172.21.15.59:0/3419646401 learned_addr learned my addr 172.21.15.59:0/3419646401 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57e00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f5814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 -- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 msgr2=0x7f6f5814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f5814fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 -- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815d810 msgr2=0x7f6f5815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815d810 0x7f6f5815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.117+0000 7f6f57400640 1 -- 172.21.15.59:0/3419646401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f58158340 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f57400640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000c780 tx=0x7f6f4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40017070 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f40005070 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000a640 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f58158620 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.118+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f58158be0 con 0x7f6f581567e0 2024-04-24T06:31:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.119+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5804fd90 con 0x7f6f581567e0 2024-04-24T06:31:01.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.120+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f4001a070 con 0x7f6f581567e0 2024-04-24T06:31:01.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.121+0000 7f6f4fe00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 0x7f6f3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:01.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.121+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6f4008ae60 con 0x7f6f581567e0 2024-04-24T06:31:01.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.121+0000 7f6f57e00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 0x7f6f3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:01.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.122+0000 7f6f57e00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 0x7f6f3406ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f48033000 tx=0x7f6f48009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:01.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.125+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f5815cab0 con 0x7f6f581567e0 2024-04-24T06:31:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.370+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6f580b5150 con 0x7f6f581567e0 2024-04-24T06:31:01.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.881+0000 7f6f4fe00640 1 -- 172.21.15.59:0/3419646401 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f6f40059280 con 0x7f6f581567e0 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.885+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 msgr2=0x7f6f3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.885+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 0x7f6f3406ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f48033000 tx=0x7f6f48009000 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.885+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 msgr2=0x7f6f5815f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.885+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000c780 tx=0x7f6f4000cc40 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.885+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 shutdown_connections 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f3406c920 0x7f6f3406ede0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f5815d810 0x7f6f5815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f581567e0 0x7f6f5815f200 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 --2- 172.21.15.59:0/3419646401 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f580d0d10 0x7f6f5814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 >> 172.21.15.59:0/3419646401 conn(0x7f6f5806bdf0 msgr2=0x7f6f58071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 shutdown_connections 2024-04-24T06:31:01.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:01.886+0000 7f6f5dc00640 1 -- 172.21.15.59:0/3419646401 wait complete. 2024-04-24T06:31:01.911 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:31:01.911 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.124+0000 7fc30f200640 1 Processor -- start 2024-04-24T06:31:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.124+0000 7fc30f200640 1 -- start start 2024-04-24T06:31:02.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.125+0000 7fc30f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc3080b4130 0x7fc30806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.125+0000 7fc30f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3080cfb90 0x7fc3080c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.125+0000 7fc30f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc30806ed40 con 0x7fc308162df0 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.125+0000 7fc30f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc30806eaf0 con 0x7fc3080cfb90 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc3080b3590 con 0x7fc3080b4130 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3080cfb90 0x7fc3080c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3080cfb90 0x7fc3080c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47036/0 (socket says 172.21.15.59:47036) 2024-04-24T06:31:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 -- 172.21.15.59:0/63445886 learned_addr learned my addr 172.21.15.59:0/63445886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30d400640 1 --2- 172.21.15.59:0/63445886 >> v2:172.21.15.66:3300/0 conn(0x7fc3080b4130 0x7fc30806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 -- 172.21.15.59:0/63445886 >> v2:172.21.15.66:3300/0 conn(0x7fc3080b4130 msgr2=0x7fc30806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 --2- 172.21.15.59:0/63445886 >> v2:172.21.15.66:3300/0 conn(0x7fc3080b4130 0x7fc30806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 -- 172.21.15.59:0/63445886 >> v1:172.21.15.66:6789/0 conn(0x7fc308162df0 legacy=0x7fc3080b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.126+0000 7fc30e800640 1 -- 172.21.15.59:0/63445886 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3080b3220 con 0x7fc3080cfb90 2024-04-24T06:31:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.127+0000 7fc30e800640 1 --2- 172.21.15.59:0/63445886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc3080c31a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc308074090 tx=0x7fc30402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238a0da7eae550ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30ca00640 1 -- 172.21.15.59:0/63445886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304036070 con 0x7fc3080cfb90 2024-04-24T06:31:02.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30ca00640 1 -- 172.21.15.59:0/63445886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc304031600 con 0x7fc3080cfb90 2024-04-24T06:31:02.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30ca00640 1 -- 172.21.15.59:0/63445886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040319d0 con 0x7fc3080cfb90 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 -- 172.21.15.59:0/63445886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 msgr2=0x7fc3080c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 --2- 172.21.15.59:0/63445886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc3080c31a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc308074090 tx=0x7fc30402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 -- 172.21.15.59:0/63445886 shutdown_connections 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 --2- 172.21.15.59:0/63445886 >> v2:172.21.15.66:3300/0 conn(0x7fc3080b4130 0x7fc30806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 --2- 172.21.15.59:0/63445886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc3080c31a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.128+0000 7fc30f200640 1 -- 172.21.15.59:0/63445886 >> 172.21.15.59:0/63445886 conn(0x7fc3080b21a0 msgr2=0x7fc3080b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.129+0000 7fc30f200640 1 -- 172.21.15.59:0/63445886 shutdown_connections 2024-04-24T06:31:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.129+0000 7fc30f200640 1 -- 172.21.15.59:0/63445886 wait complete. 2024-04-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.130+0000 7fc30f200640 1 Processor -- start 2024-04-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.131+0000 7fc30f200640 1 -- start start 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc3080b4130 0x7fc308163520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc308162df0 0x7fc3080c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080c3860 con 0x7fc3080cfb90 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080c36e0 con 0x7fc308162df0 2024-04-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc3080c39e0 con 0x7fc3080b4130 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47050/0 (socket says 172.21.15.59:47050) 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 -- 172.21.15.59:0/762255201 learned_addr learned my addr 172.21.15.59:0/762255201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30de00640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc3080b4130 0x7fc308163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30e800640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc308162df0 0x7fc3080c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 -- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc3080b4130 msgr2=0x7fc308163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc3080b4130 0x7fc308163520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 -- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc308162df0 msgr2=0x7fc3080c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc308162df0 0x7fc3080c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.132+0000 7fc30d400640 1 -- 172.21.15.59:0/762255201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc308166050 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc30d400640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800d830 tx=0x7fc2f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800b6c0 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f8019070 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc308165a90 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800ba80 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.133+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3080c7e40 con 0x7fc3080cfb90 2024-04-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.134+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30804fd90 con 0x7fc3080cfb90 2024-04-24T06:31:02.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.135+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2f801e460 con 0x7fc3080cfb90 2024-04-24T06:31:02.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.136+0000 7fc302000640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.137+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc2f808c350 con 0x7fc3080cfb90 2024-04-24T06:31:02.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.137+0000 7fc30de00640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.138+0000 7fc30de00640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc006fd0 tx=0x7fc2fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.141+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2f805a650 con 0x7fc3080cfb90 2024-04-24T06:31:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.386+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc30815d8a0 con 0x7fc3080cfb90 2024-04-24T06:31:02.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.388+0000 7fc302000640 1 -- 172.21.15.59:0/762255201 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7fc2f8059f50 con 0x7fc3080cfb90 2024-04-24T06:31:02.391 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:02.391 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:02.391 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:02.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 msgr2=0x7fc2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc006fd0 tx=0x7fc2fc008040 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 msgr2=0x7fc30815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800d830 tx=0x7fc2f800dcf0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 shutdown_connections 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.390+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc2ec06c8f0 0x7fc2ec06edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc308162df0 0x7fc3080c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3080cfb90 0x7fc30815d360 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 --2- 172.21.15.59:0/762255201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc3080b4130 0x7fc308163520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 >> 172.21.15.59:0/762255201 conn(0x7fc3080b21a0 msgr2=0x7fc30806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:02.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 shutdown_connections 2024-04-24T06:31:02.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.391+0000 7fc30f200640 1 -- 172.21.15.59:0/762255201 wait complete. 2024-04-24T06:31:02.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:02.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.627+0000 7f8e19600640 1 Processor -- start 2024-04-24T06:31:02.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.627+0000 7f8e19600640 1 -- start start 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e14162df0 0x7f8e1406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e140b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 0x7f8e140c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1406eaf0 con 0x7f8e14162df0 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8e140b3590 con 0x7f8e140b3f60 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.628+0000 7f8e19600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1406ed40 con 0x7f8e140cfb90 2024-04-24T06:31:02.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e14162df0 0x7f8e1406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e14162df0 0x7f8e1406f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47064/0 (socket says 172.21.15.59:47064) 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 -- 172.21.15.59:0/346337189 learned_addr learned my addr 172.21.15.59:0/346337189 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e0be00640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e140b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e18c00640 1 --2- 172.21.15.59:0/346337189 >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 0x7f8e140c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 -- 172.21.15.59:0/346337189 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 msgr2=0x7f8e140b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e140b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 -- 172.21.15.59:0/346337189 >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 msgr2=0x7f8e140c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 --2- 172.21.15.59:0/346337189 >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 0x7f8e140c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e13e00640 1 -- 172.21.15.59:0/346337189 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e140b3220 con 0x7f8e14162df0 2024-04-24T06:31:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.629+0000 7f8e18c00640 1 --2- 172.21.15.59:0/346337189 >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 0x7f8e140c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.630+0000 7f8e13e00640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e1406f680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8e14071b90 tx=0x7f8e0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49c317ca27bc9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e13400640 1 -- 172.21.15.59:0/346337189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04037070 con 0x7f8e14162df0 2024-04-24T06:31:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e13400640 1 -- 172.21.15.59:0/346337189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04032440 con 0x7f8e14162df0 2024-04-24T06:31:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e13400640 1 -- 172.21.15.59:0/346337189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04032810 con 0x7f8e14162df0 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 -- 172.21.15.59:0/346337189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 msgr2=0x7f8e1406f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e1406f680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8e14071b90 tx=0x7f8e0402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 -- 172.21.15.59:0/346337189 shutdown_connections 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 --2- 172.21.15.59:0/346337189 >> v2:172.21.15.66:3300/0 conn(0x7f8e140b3f60 0x7f8e140c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e140b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 --2- 172.21.15.59:0/346337189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e1406f680 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 -- 172.21.15.59:0/346337189 >> 172.21.15.59:0/346337189 conn(0x7f8e140b21a0 msgr2=0x7f8e140b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 -- 172.21.15.59:0/346337189 shutdown_connections 2024-04-24T06:31:02.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.631+0000 7f8e19600640 1 -- 172.21.15.59:0/346337189 wait complete. 2024-04-24T06:31:02.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.633+0000 7f8e19600640 1 Processor -- start 2024-04-24T06:31:02.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.633+0000 7f8e19600640 1 -- start start 2024-04-24T06:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e140b3f60 0x7f8e140c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e14150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e140c3860 con 0x7f8e14162df0 2024-04-24T06:31:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8e140c36e0 con 0x7f8e140b3f60 2024-04-24T06:31:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.634+0000 7f8e19600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8e140c39e0 con 0x7f8e140cfb90 2024-04-24T06:31:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47074/0 (socket says 172.21.15.59:47074) 2024-04-24T06:31:02.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 -- 172.21.15.59:0/3134462699 learned_addr learned my addr 172.21.15.59:0/3134462699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e13e00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e140b3f60 0x7f8e140c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e0be00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e14150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 -- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 msgr2=0x7f8e14150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e14150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 -- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e140b3f60 msgr2=0x7f8e140c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e140b3f60 0x7f8e140c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 -- 172.21.15.59:0/3134462699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e14151180 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e0be00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e14150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.635+0000 7f8e18c00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00cf20 tx=0x7f8e0c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.636+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c01a070 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.636+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00d840 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.636+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00dc30 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.636+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e140c6170 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.636+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e140c6730 con 0x7f8e14162df0 2024-04-24T06:31:02.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.637+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1404fd90 con 0x7f8e14162df0 2024-04-24T06:31:02.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.638+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e0c01d070 con 0x7f8e14162df0 2024-04-24T06:31:02.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.639+0000 7f8e10c00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 0x7f8df006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:02.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.639+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8e0c08c420 con 0x7f8e14162df0 2024-04-24T06:31:02.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.639+0000 7f8e13e00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 0x7f8df006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:02.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.640+0000 7f8e13e00640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 0x7f8df006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e04034000 tx=0x7f8e0402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:02.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.643+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e14151180 con 0x7f8e14162df0 2024-04-24T06:31:02.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.848+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e140cd2b0 con 0x7f8e14162df0 2024-04-24T06:31:02.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.850+0000 7f8e10c00640 1 -- 172.21.15.59:0/3134462699 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16529 (secure 0 0 0) 0x7f8e0c05a7b0 con 0x7f8e14162df0 2024-04-24T06:31:02.853 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:02.853 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:30:59.852277+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:02.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 msgr2=0x7f8df006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 0x7f8df006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e04034000 tx=0x7f8e0402e000 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 msgr2=0x7f8e140b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00cf20 tx=0x7f8e0c00a710 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 shutdown_connections 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8df006c8f0 0x7f8df006edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e14162df0 0x7f8e140b15f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8e140cfb90 0x7f8e14150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 --2- 172.21.15.59:0/3134462699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e140b3f60 0x7f8e140c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 >> 172.21.15.59:0/3134462699 conn(0x7f8e140b21a0 msgr2=0x7f8e140b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.853+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 shutdown_connections 2024-04-24T06:31:02.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:02.854+0000 7f8e19600640 1 -- 172.21.15.59:0/3134462699 wait complete. 2024-04-24T06:31:02.875 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:31:02.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:31:03.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.080+0000 7f3297e00640 1 Processor -- start 2024-04-24T06:31:03.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.080+0000 7f3297e00640 1 -- start start 2024-04-24T06:31:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f329815b450 0x7f32980b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f329806f3f0 0x7f32980cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32980b3870 con 0x7f32980b3080 2024-04-24T06:31:03.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f32980b41a0 con 0x7f329815b450 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f32980b3b50 con 0x7f329806f3f0 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f329815b450 0x7f32980b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f329815b450 0x7f32980b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47082/0 (socket says 172.21.15.59:47082) 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 -- 172.21.15.59:0/1305359834 learned_addr learned my addr 172.21.15.59:0/1305359834 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3297400640 1 --1- 172.21.15.59:0/1305359834 >> v1:172.21.15.59:6789/0 conn(0x7f32980b3080 0x7f32980ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48262/0 (socket says 172.21.15.59:48262) 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 -- 172.21.15.59:0/1305359834 >> v2:172.21.15.66:3300/0 conn(0x7f329806f3f0 msgr2=0x7f32980cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 --2- 172.21.15.59:0/1305359834 >> v2:172.21.15.66:3300/0 conn(0x7f329806f3f0 0x7f32980cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.082+0000 7f3296a00640 1 -- 172.21.15.59:0/1305359834 >> v1:172.21.15.59:6789/0 conn(0x7f32980b3080 legacy=0x7f32980ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.083+0000 7f3296a00640 1 -- 172.21.15.59:0/1305359834 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32980b3e30 con 0x7f329815b450 2024-04-24T06:31:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.084+0000 7f3296a00640 1 --2- 172.21.15.59:0/1305359834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329815b450 0x7f32980b4630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f32980af4a0 tx=0x7f328802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534f8712471cdee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.084+0000 7f3296000640 1 -- 172.21.15.59:0/1305359834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288037070 con 0x7f329815b450 2024-04-24T06:31:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.084+0000 7f3296000640 1 -- 172.21.15.59:0/1305359834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328802ae50 con 0x7f329815b450 2024-04-24T06:31:03.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.084+0000 7f3296000640 1 -- 172.21.15.59:0/1305359834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288032860 con 0x7f329815b450 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 -- 172.21.15.59:0/1305359834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329815b450 msgr2=0x7f32980b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 --2- 172.21.15.59:0/1305359834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329815b450 0x7f32980b4630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f32980af4a0 tx=0x7f328802a300 comp rx=0 tx=0).stop 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 -- 172.21.15.59:0/1305359834 shutdown_connections 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 --2- 172.21.15.59:0/1305359834 >> v2:172.21.15.66:3300/0 conn(0x7f329806f3f0 0x7f32980cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 --2- 172.21.15.59:0/1305359834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329815b450 0x7f32980b4630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 -- 172.21.15.59:0/1305359834 >> 172.21.15.59:0/1305359834 conn(0x7f32980b21a0 msgr2=0x7f32980b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 -- 172.21.15.59:0/1305359834 shutdown_connections 2024-04-24T06:31:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.085+0000 7f3297e00640 1 -- 172.21.15.59:0/1305359834 wait complete. 2024-04-24T06:31:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.086+0000 7f3297e00640 1 Processor -- start 2024-04-24T06:31:03.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.087+0000 7f3297e00640 1 -- start start 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.087+0000 7f3297e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.087+0000 7f3297e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 0x7f329815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3297e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f329815b450 0x7f329809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3297e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32980b4ff0 con 0x7f329806f3f0 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3297e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f329815b830 con 0x7f329815b450 2024-04-24T06:31:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3297e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f32980cd590 con 0x7f32980b3080 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47094/0 (socket says 172.21.15.59:47094) 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 -- 172.21.15.59:0/2420548878 learned_addr learned my addr 172.21.15.59:0/2420548878 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 -- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 msgr2=0x7f329815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f328e000640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 0x7f329815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3297400640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f329815b450 0x7f329809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 0x7f329815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 -- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f329815b450 msgr2=0x7f329809dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f329815b450 0x7f329809dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f3296a00640 1 -- 172.21.15.59:0/2420548878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329815bef0 con 0x7f329806f3f0 2024-04-24T06:31:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.088+0000 7f328e000640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 0x7f329815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f3296a00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f328802a340 tx=0x7f32880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288041070 con 0x7f329806f3f0 2024-04-24T06:31:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328803c870 con 0x7f329806f3f0 2024-04-24T06:31:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329815c190 con 0x7f329806f3f0 2024-04-24T06:31:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288037040 con 0x7f329806f3f0 2024-04-24T06:31:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.089+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329815c750 con 0x7f329806f3f0 2024-04-24T06:31:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.090+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329804fd90 con 0x7f329806f3f0 2024-04-24T06:31:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.091+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3288044070 con 0x7f329806f3f0 2024-04-24T06:31:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.092+0000 7f328f400640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 0x7f327406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.092+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f32880ac6f0 con 0x7f329806f3f0 2024-04-24T06:31:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.092+0000 7f328e000640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 0x7f327406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.093+0000 7f328e000640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 0x7f327406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f329806c060 tx=0x7f32800073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.096+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f328807ac30 con 0x7f329806f3f0 2024-04-24T06:31:03.243 INFO:tasks.ceph.mds.b.smithi066.stderr:2024-04-24T06:31:03.242+0000 7fb4fde00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:31:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.299+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f329806def0 con 0x7f327406c8f0 2024-04-24T06:31:03.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.311+0000 7f328f400640 1 -- 172.21.15.59:0/2420548878 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f329806def0 con 0x7f327406c8f0 2024-04-24T06:31:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 msgr2=0x7f327406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 0x7f327406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f329806c060 tx=0x7f32800073c0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 msgr2=0x7f32981f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f328802a340 tx=0x7f32880046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 shutdown_connections 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f327406c8f0 0x7f327406edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f329815b450 0x7f329809dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f32980b3080 0x7f329815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 --2- 172.21.15.59:0/2420548878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329806f3f0 0x7f32981f5320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 >> 172.21.15.59:0/2420548878 conn(0x7f32980b21a0 msgr2=0x7f32980d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.317+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 shutdown_connections 2024-04-24T06:31:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.318+0000 7f3297e00640 1 -- 172.21.15.59:0/2420548878 wait complete. 2024-04-24T06:31:03.338 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-24T06:31:03.230579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225392,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749328,"statfs":{"total":773094113280,"available":772863311872,"internally_reserved":0,"allocated":5181440,"data_stored":2778255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001571"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823354+0000","last_change":"2024-04-24T06:30:57.823354+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823354+0000","last_undegraded":"2024-04-24T06:30:57.823354+0000","last_fullsized":"2024-04-24T06:30:57.823354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825952+0000","last_change":"2024-04-24T06:30:57.825952+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825952+0000","last_undegraded":"2024-04-24T06:30:57.825952+0000","last_fullsized":"2024-04-24T06:30:57.825952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815572+0000","last_change":"2024-04-24T06:30:57.815572+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815572+0000","last_undegraded":"2024-04-24T06:30:57.815572+0000","last_fullsized":"2024-04-24T06:30:57.815572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817440+0000","last_change":"2024-04-24T06:30:57.817440+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817440+0000","last_undegraded":"2024-04-24T06:30:57.817440+0000","last_fullsized":"2024-04-24T06:30:57.817440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819614+0000","last_change":"2024-04-24T06:30:57.819614+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819614+0000","last_undegraded":"2024-04-24T06:30:57.819614+0000","last_fullsized":"2024-04-24T06:30:57.819614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822617+0000","last_change":"2024-04-24T06:30:57.822617+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822617+0000","last_undegraded":"2024-04-24T06:30:57.822617+0000","last_fullsized":"2024-04-24T06:30:57.822617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826625+0000","last_change":"2024-04-24T06:30:57.826625+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826625+0000","last_undegraded":"2024-04-24T06:30:57.826625+0000","last_fullsized":"2024-04-24T06:30:57.826625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815801+0000","last_change":"2024-04-24T06:30:57.815801+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815801+0000","last_undegraded":"2024-04-24T06:30:57.815801+0000","last_fullsized":"2024-04-24T06:30:57.815801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817353+0000","last_change":"2024-04-24T06:30:57.817353+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817353+0000","last_undegraded":"2024-04-24T06:30:57.817353+0000","last_fullsized":"2024-04-24T06:30:57.817353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817275+0000","last_change":"2024-04-24T06:30:57.817275+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817275+0000","last_undegraded":"2024-04-24T06:30:57.817275+0000","last_fullsized":"2024-04-24T06:30:57.817275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816768+0000","last_change":"2024-04-24T06:30:57.816768+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816768+0000","last_undegraded":"2024-04-24T06:30:57.816768+0000","last_fullsized":"2024-04-24T06:30:57.816768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.830810+0000","last_change":"2024-04-24T06:30:57.830810+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.830810+0000","last_undegraded":"2024-04-24T06:30:57.830810+0000","last_fullsized":"2024-04-24T06:30:57.830810+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815397+0000","last_change":"2024-04-24T06:30:57.815397+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815397+0000","last_undegraded":"2024-04-24T06:30:57.815397+0000","last_fullsized":"2024-04-24T06:30:57.815397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816773+0000","last_change":"2024-04-24T06:30:57.816773+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816773+0000","last_undegraded":"2024-04-24T06:30:57.816773+0000","last_fullsized":"2024-04-24T06:30:57.816773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824106+0000","last_change":"2024-04-24T06:30:57.824106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824106+0000","last_undegraded":"2024-04-24T06:30:57.824106+0000","last_fullsized":"2024-04-24T06:30:57.824106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822636+0000","last_change":"2024-04-24T06:30:57.822636+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822636+0000","last_undegraded":"2024-04-24T06:30:57.822636+0000","last_fullsized":"2024-04-24T06:30:57.822636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812675+0000","last_change":"2024-04-24T06:30:57.812675+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812675+0000","last_undegraded":"2024-04-24T06:30:57.812675+0000","last_fullsized":"2024-04-24T06:30:57.812675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814940+0000","last_change":"2024-04-24T06:30:57.814940+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814940+0000","last_undegraded":"2024-04-24T06:30:57.814940+0000","last_fullsized":"2024-04-24T06:30:57.814940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.827593+0000","last_change":"2024-04-24T06:30:57.827593+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.827593+0000","last_undegraded":"2024-04-24T06:30:57.827593+0000","last_fullsized":"2024-04-24T06:30:57.827593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816449+0000","last_change":"2024-04-24T06:30:57.816449+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816449+0000","last_undegraded":"2024-04-24T06:30:57.816449+0000","last_fullsized":"2024-04-24T06:30:57.816449+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825494+0000","last_change":"2024-04-24T06:30:57.825494+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825494+0000","last_undegraded":"2024-04-24T06:30:57.825494+0000","last_fullsized":"2024-04-24T06:30:57.825494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816080+0000","last_change":"2024-04-24T06:30:57.816080+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816080+0000","last_undegraded":"2024-04-24T06:30:57.816080+0000","last_fullsized":"2024-04-24T06:30:57.816080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823690+0000","last_change":"2024-04-24T06:30:57.823690+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823690+0000","last_undegraded":"2024-04-24T06:30:57.823690+0000","last_fullsized":"2024-04-24T06:30:57.823690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821714+0000","last_change":"2024-04-24T06:30:57.821714+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821714+0000","last_undegraded":"2024-04-24T06:30:57.821714+0000","last_fullsized":"2024-04-24T06:30:57.821714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829589+0000","last_change":"2024-04-24T06:30:57.829589+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829589+0000","last_undegraded":"2024-04-24T06:30:57.829589+0000","last_fullsized":"2024-04-24T06:30:57.829589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815784+0000","last_change":"2024-04-24T06:30:57.815784+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815784+0000","last_undegraded":"2024-04-24T06:30:57.815784+0000","last_fullsized":"2024-04-24T06:30:57.815784+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.809984+0000","last_change":"2024-04-24T06:30:57.809984+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.809984+0000","last_undegraded":"2024-04-24T06:30:57.809984+0000","last_fullsized":"2024-04-24T06:30:57.809984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826656+0000","last_change":"2024-04-24T06:30:57.826656+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826656+0000","last_undegraded":"2024-04-24T06:30:57.826656+0000","last_fullsized":"2024-04-24T06:30:57.826656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824980+0000","last_change":"2024-04-24T06:30:57.824980+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824980+0000","last_undegraded":"2024-04-24T06:30:57.824980+0000","last_fullsized":"2024-04-24T06:30:57.824980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812237+0000","last_change":"2024-04-24T06:30:57.812237+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812237+0000","last_undegraded":"2024-04-24T06:30:57.812237+0000","last_fullsized":"2024-04-24T06:30:57.812237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821611+0000","last_change":"2024-04-24T06:30:57.821611+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821611+0000","last_undegraded":"2024-04-24T06:30:57.821611+0000","last_fullsized":"2024-04-24T06:30:57.821611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817106+0000","last_change":"2024-04-24T06:30:57.817106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817106+0000","last_undegraded":"2024-04-24T06:30:57.817106+0000","last_fullsized":"2024-04-24T06:30:57.817106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820562+0000","last_change":"2024-04-24T06:30:57.820562+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820562+0000","last_undegraded":"2024-04-24T06:30:57.820562+0000","last_fullsized":"2024-04-24T06:30:57.820562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844480+0000","last_change":"2024-04-24T06:30:57.837026+0000","last_active":"2024-04-24T06:30:57.844480+0000","last_peered":"2024-04-24T06:30:57.844480+0000","last_clean":"2024-04-24T06:30:57.844480+0000","last_became_active":"2024-04-24T06:30:57.836706+0000","last_became_peered":"2024-04-24T06:30:57.836706+0000","last_unstale":"2024-04-24T06:30:57.844480+0000","last_undegraded":"2024-04-24T06:30:57.844480+0000","last_fullsized":"2024-04-24T06:30:57.844480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829026+0000","last_change":"2024-04-24T06:30:57.829026+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829026+0000","last_undegraded":"2024-04-24T06:30:57.829026+0000","last_fullsized":"2024-04-24T06:30:57.829026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842666+0000","last_change":"2024-04-24T06:30:57.838444+0000","last_active":"2024-04-24T06:30:57.842666+0000","last_peered":"2024-04-24T06:30:57.842666+0000","last_clean":"2024-04-24T06:30:57.842666+0000","last_became_active":"2024-04-24T06:30:57.838117+0000","last_became_peered":"2024-04-24T06:30:57.838117+0000","last_unstale":"2024-04-24T06:30:57.842666+0000","last_undegraded":"2024-04-24T06:30:57.842666+0000","last_fullsized":"2024-04-24T06:30:57.842666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823489+0000","last_change":"2024-04-24T06:30:57.823489+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823489+0000","last_undegraded":"2024-04-24T06:30:57.823489+0000","last_fullsized":"2024-04-24T06:30:57.823489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.846663+0000","last_change":"2024-04-24T06:30:57.844480+0000","last_active":"2024-04-24T06:30:57.846663+0000","last_peered":"2024-04-24T06:30:57.846663+0000","last_clean":"2024-04-24T06:30:57.846663+0000","last_became_active":"2024-04-24T06:30:57.844131+0000","last_became_peered":"2024-04-24T06:30:57.844131+0000","last_unstale":"2024-04-24T06:30:57.846663+0000","last_undegraded":"2024-04-24T06:30:57.846663+0000","last_fullsized":"2024-04-24T06:30:57.846663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817619+0000","last_change":"2024-04-24T06:30:57.817619+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817619+0000","last_undegraded":"2024-04-24T06:30:57.817619+0000","last_fullsized":"2024-04-24T06:30:57.817619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849607+0000","last_change":"2024-04-24T06:30:57.845981+0000","last_active":"2024-04-24T06:30:57.849607+0000","last_peered":"2024-04-24T06:30:57.849607+0000","last_clean":"2024-04-24T06:30:57.849607+0000","last_became_active":"2024-04-24T06:30:57.845675+0000","last_became_peered":"2024-04-24T06:30:57.845675+0000","last_unstale":"2024-04-24T06:30:57.849607+0000","last_undegraded":"2024-04-24T06:30:57.849607+0000","last_fullsized":"2024-04-24T06:30:57.849607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.811045+0000","last_change":"2024-04-24T06:30:57.811045+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.811045+0000","last_undegraded":"2024-04-24T06:30:57.811045+0000","last_fullsized":"2024-04-24T06:30:57.811045+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844747+0000","last_change":"2024-04-24T06:30:57.841400+0000","last_active":"2024-04-24T06:30:57.844747+0000","last_peered":"2024-04-24T06:30:57.844747+0000","last_clean":"2024-04-24T06:30:57.844747+0000","last_became_active":"2024-04-24T06:30:57.841071+0000","last_became_peered":"2024-04-24T06:30:57.841071+0000","last_unstale":"2024-04-24T06:30:57.844747+0000","last_undegraded":"2024-04-24T06:30:57.844747+0000","last_fullsized":"2024-04-24T06:30:57.844747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:26.018041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816200+0000","last_change":"2024-04-24T06:30:57.816200+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816200+0000","last_undegraded":"2024-04-24T06:30:57.816200+0000","last_fullsized":"2024-04-24T06:30:57.816200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842087+0000","last_change":"2024-04-24T06:30:57.835137+0000","last_active":"2024-04-24T06:30:57.842087+0000","last_peered":"2024-04-24T06:30:57.842087+0000","last_clean":"2024-04-24T06:30:57.842087+0000","last_became_active":"2024-04-24T06:30:57.834822+0000","last_became_peered":"2024-04-24T06:30:57.834822+0000","last_unstale":"2024-04-24T06:30:57.842087+0000","last_undegraded":"2024-04-24T06:30:57.842087+0000","last_fullsized":"2024-04-24T06:30:57.842087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814695+0000","last_change":"2024-04-24T06:30:57.814695+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814695+0000","last_undegraded":"2024-04-24T06:30:57.814695+0000","last_fullsized":"2024-04-24T06:30:57.814695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845153+0000","last_change":"2024-04-24T06:30:57.840100+0000","last_active":"2024-04-24T06:30:57.845153+0000","last_peered":"2024-04-24T06:30:57.845153+0000","last_clean":"2024-04-24T06:30:57.845153+0000","last_became_active":"2024-04-24T06:30:57.839664+0000","last_became_peered":"2024-04-24T06:30:57.839664+0000","last_unstale":"2024-04-24T06:30:57.845153+0000","last_undegraded":"2024-04-24T06:30:57.845153+0000","last_fullsized":"2024-04-24T06:30:57.845153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824821+0000","last_change":"2024-04-24T06:30:57.824821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824821+0000","last_undegraded":"2024-04-24T06:30:57.824821+0000","last_fullsized":"2024-04-24T06:30:57.824821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832582+0000","last_change":"2024-04-24T06:30:57.825727+0000","last_active":"2024-04-24T06:30:57.832582+0000","last_peered":"2024-04-24T06:30:57.832582+0000","last_clean":"2024-04-24T06:30:57.832582+0000","last_became_active":"2024-04-24T06:30:57.825361+0000","last_became_peered":"2024-04-24T06:30:57.825361+0000","last_unstale":"2024-04-24T06:30:57.832582+0000","last_undegraded":"2024-04-24T06:30:57.832582+0000","last_fullsized":"2024-04-24T06:30:57.832582+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:09.450817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812403+0000","last_change":"2024-04-24T06:30:57.812403+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812403+0000","last_undegraded":"2024-04-24T06:30:57.812403+0000","last_fullsized":"2024-04-24T06:30:57.812403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842570+0000","last_change":"2024-04-24T06:30:57.838423+0000","last_active":"2024-04-24T06:30:57.842570+0000","last_peered":"2024-04-24T06:30:57.842570+0000","last_clean":"2024-04-24T06:30:57.842570+0000","last_became_active":"2024-04-24T06:30:57.838078+0000","last_became_peered":"2024-04-24T06:30:57.838078+0000","last_unstale":"2024-04-24T06:30:57.842570+0000","last_undegraded":"2024-04-24T06:30:57.842570+0000","last_fullsized":"2024-04-24T06:30:57.842570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821943+0000","last_change":"2024-04-24T06:30:57.821943+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821943+0000","last_undegraded":"2024-04-24T06:30:57.821943+0000","last_fullsized":"2024-04-24T06:30:57.821943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845770+0000","last_change":"2024-04-24T06:30:57.840907+0000","last_active":"2024-04-24T06:30:57.845770+0000","last_peered":"2024-04-24T06:30:57.845770+0000","last_clean":"2024-04-24T06:30:57.845770+0000","last_became_active":"2024-04-24T06:30:57.840593+0000","last_became_peered":"2024-04-24T06:30:57.840593+0000","last_unstale":"2024-04-24T06:30:57.845770+0000","last_undegraded":"2024-04-24T06:30:57.845770+0000","last_fullsized":"2024-04-24T06:30:57.845770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:40:00.530508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823821+0000","last_change":"2024-04-24T06:30:57.823821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823821+0000","last_undegraded":"2024-04-24T06:30:57.823821+0000","last_fullsized":"2024-04-24T06:30:57.823821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849557+0000","last_change":"2024-04-24T06:30:57.845222+0000","last_active":"2024-04-24T06:30:57.849557+0000","last_peered":"2024-04-24T06:30:57.849557+0000","last_clean":"2024-04-24T06:30:57.849557+0000","last_became_active":"2024-04-24T06:30:57.843505+0000","last_became_peered":"2024-04-24T06:30:57.843505+0000","last_unstale":"2024-04-24T06:30:57.849557+0000","last_undegraded":"2024-04-24T06:30:57.849557+0000","last_fullsized":"2024-04-24T06:30:57.849557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826174+0000","last_change":"2024-04-24T06:30:57.826174+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826174+0000","last_undegraded":"2024-04-24T06:30:57.826174+0000","last_fullsized":"2024-04-24T06:30:57.826174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843367+0000","last_change":"2024-04-24T06:30:57.841598+0000","last_active":"2024-04-24T06:30:57.843367+0000","last_peered":"2024-04-24T06:30:57.843367+0000","last_clean":"2024-04-24T06:30:57.843367+0000","last_became_active":"2024-04-24T06:30:57.841319+0000","last_became_peered":"2024-04-24T06:30:57.841319+0000","last_unstale":"2024-04-24T06:30:57.843367+0000","last_undegraded":"2024-04-24T06:30:57.843367+0000","last_fullsized":"2024-04-24T06:30:57.843367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:52.616317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824873+0000","last_change":"2024-04-24T06:30:57.824873+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824873+0000","last_undegraded":"2024-04-24T06:30:57.824873+0000","last_fullsized":"2024-04-24T06:30:57.824873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835720+0000","last_change":"2024-04-24T06:30:57.831254+0000","last_active":"2024-04-24T06:30:57.835720+0000","last_peered":"2024-04-24T06:30:57.835720+0000","last_clean":"2024-04-24T06:30:57.835720+0000","last_became_active":"2024-04-24T06:30:57.830948+0000","last_became_peered":"2024-04-24T06:30:57.830948+0000","last_unstale":"2024-04-24T06:30:57.835720+0000","last_undegraded":"2024-04-24T06:30:57.835720+0000","last_fullsized":"2024-04-24T06:30:57.835720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814759+0000","last_change":"2024-04-24T06:30:57.814759+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814759+0000","last_undegraded":"2024-04-24T06:30:57.814759+0000","last_fullsized":"2024-04-24T06:30:57.814759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849559+0000","last_change":"2024-04-24T06:30:57.844423+0000","last_active":"2024-04-24T06:30:57.849559+0000","last_peered":"2024-04-24T06:30:57.849559+0000","last_clean":"2024-04-24T06:30:57.849559+0000","last_became_active":"2024-04-24T06:30:57.843788+0000","last_became_peered":"2024-04-24T06:30:57.843788+0000","last_unstale":"2024-04-24T06:30:57.849559+0000","last_undegraded":"2024-04-24T06:30:57.849559+0000","last_fullsized":"2024-04-24T06:30:57.849559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.818746+0000","last_change":"2024-04-24T06:30:57.818746+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.818746+0000","last_undegraded":"2024-04-24T06:30:57.818746+0000","last_fullsized":"2024-04-24T06:30:57.818746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843669+0000","last_change":"2024-04-24T06:30:57.833067+0000","last_active":"2024-04-24T06:30:57.843669+0000","last_peered":"2024-04-24T06:30:57.843669+0000","last_clean":"2024-04-24T06:30:57.843669+0000","last_became_active":"2024-04-24T06:30:57.832715+0000","last_became_peered":"2024-04-24T06:30:57.832715+0000","last_unstale":"2024-04-24T06:30:57.843669+0000","last_undegraded":"2024-04-24T06:30:57.843669+0000","last_fullsized":"2024-04-24T06:30:57.843669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824842+0000","last_change":"2024-04-24T06:30:57.824842+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824842+0000","last_undegraded":"2024-04-24T06:30:57.824842+0000","last_fullsized":"2024-04-24T06:30:57.824842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832875+0000","last_change":"2024-04-24T06:30:57.830642+0000","last_active":"2024-04-24T06:30:57.832875+0000","last_peered":"2024-04-24T06:30:57.832875+0000","last_clean":"2024-04-24T06:30:57.832875+0000","last_became_active":"2024-04-24T06:30:57.830243+0000","last_became_peered":"2024-04-24T06:30:57.830243+0000","last_unstale":"2024-04-24T06:30:57.832875+0000","last_undegraded":"2024-04-24T06:30:57.832875+0000","last_fullsized":"2024-04-24T06:30:57.832875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:56.404238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814978+0000","last_change":"2024-04-24T06:30:57.814978+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814978+0000","last_undegraded":"2024-04-24T06:30:57.814978+0000","last_fullsized":"2024-04-24T06:30:57.814978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835865+0000","last_change":"2024-04-24T06:30:57.833779+0000","last_active":"2024-04-24T06:30:57.835865+0000","last_peered":"2024-04-24T06:30:57.835865+0000","last_clean":"2024-04-24T06:30:57.835865+0000","last_became_active":"2024-04-24T06:30:57.833537+0000","last_became_peered":"2024-04-24T06:30:57.833537+0000","last_unstale":"2024-04-24T06:30:57.835865+0000","last_undegraded":"2024-04-24T06:30:57.835865+0000","last_fullsized":"2024-04-24T06:30:57.835865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:43:04.048136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812756+0000","last_change":"2024-04-24T06:30:57.812756+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812756+0000","last_undegraded":"2024-04-24T06:30:57.812756+0000","last_fullsized":"2024-04-24T06:30:57.812756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841087+0000","last_change":"2024-04-24T06:30:57.830426+0000","last_active":"2024-04-24T06:30:57.841087+0000","last_peered":"2024-04-24T06:30:57.841087+0000","last_clean":"2024-04-24T06:30:57.841087+0000","last_became_active":"2024-04-24T06:30:57.830061+0000","last_became_peered":"2024-04-24T06:30:57.830061+0000","last_unstale":"2024-04-24T06:30:57.841087+0000","last_undegraded":"2024-04-24T06:30:57.841087+0000","last_fullsized":"2024-04-24T06:30:57.841087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:49.837169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829029+0000","last_change":"2024-04-24T06:30:57.829029+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829029+0000","last_undegraded":"2024-04-24T06:30:57.829029+0000","last_fullsized":"2024-04-24T06:30:57.829029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843580+0000","last_change":"2024-04-24T06:30:57.832486+0000","last_active":"2024-04-24T06:30:57.843580+0000","last_peered":"2024-04-24T06:30:57.843580+0000","last_clean":"2024-04-24T06:30:57.843580+0000","last_became_active":"2024-04-24T06:30:57.832182+0000","last_became_peered":"2024-04-24T06:30:57.832182+0000","last_unstale":"2024-04-24T06:30:57.843580+0000","last_undegraded":"2024-04-24T06:30:57.843580+0000","last_fullsized":"2024-04-24T06:30:57.843580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:52.738153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.832279+0000","last_change":"2024-04-24T06:30:57.832279+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.832279+0000","last_undegraded":"2024-04-24T06:30:57.832279+0000","last_fullsized":"2024-04-24T06:30:57.832279+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845028+0000","last_change":"2024-04-24T06:30:57.842579+0000","last_active":"2024-04-24T06:30:57.845028+0000","last_peered":"2024-04-24T06:30:57.845028+0000","last_clean":"2024-04-24T06:30:57.845028+0000","last_became_active":"2024-04-24T06:30:57.842265+0000","last_became_peered":"2024-04-24T06:30:57.842265+0000","last_unstale":"2024-04-24T06:30:57.845028+0000","last_undegraded":"2024-04-24T06:30:57.845028+0000","last_fullsized":"2024-04-24T06:30:57.845028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:03.622481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821845+0000","last_change":"2024-04-24T06:30:57.821845+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821845+0000","last_undegraded":"2024-04-24T06:30:57.821845+0000","last_fullsized":"2024-04-24T06:30:57.821845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844541+0000","last_change":"2024-04-24T06:30:57.840860+0000","last_active":"2024-04-24T06:30:57.844541+0000","last_peered":"2024-04-24T06:30:57.844541+0000","last_clean":"2024-04-24T06:30:57.844541+0000","last_became_active":"2024-04-24T06:30:57.840543+0000","last_became_peered":"2024-04-24T06:30:57.840543+0000","last_unstale":"2024-04-24T06:30:57.844541+0000","last_undegraded":"2024-04-24T06:30:57.844541+0000","last_fullsized":"2024-04-24T06:30:57.844541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:50.961300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819015+0000","last_change":"2024-04-24T06:30:57.819015+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819015+0000","last_undegraded":"2024-04-24T06:30:57.819015+0000","last_fullsized":"2024-04-24T06:30:57.819015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843981+0000","last_change":"2024-04-24T06:30:57.837623+0000","last_active":"2024-04-24T06:30:57.843981+0000","last_peered":"2024-04-24T06:30:57.843981+0000","last_clean":"2024-04-24T06:30:57.843981+0000","last_became_active":"2024-04-24T06:30:57.837021+0000","last_became_peered":"2024-04-24T06:30:57.837021+0000","last_unstale":"2024-04-24T06:30:57.843981+0000","last_undegraded":"2024-04-24T06:30:57.843981+0000","last_fullsized":"2024-04-24T06:30:57.843981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:12.661118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814915+0000","last_change":"2024-04-24T06:30:57.814915+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814915+0000","last_undegraded":"2024-04-24T06:30:57.814915+0000","last_fullsized":"2024-04-24T06:30:57.814915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845287+0000","last_change":"2024-04-24T06:30:57.840879+0000","last_active":"2024-04-24T06:30:57.845287+0000","last_peered":"2024-04-24T06:30:57.845287+0000","last_clean":"2024-04-24T06:30:57.845287+0000","last_became_active":"2024-04-24T06:30:57.840563+0000","last_became_peered":"2024-04-24T06:30:57.840563+0000","last_unstale":"2024-04-24T06:30:57.845287+0000","last_undegraded":"2024-04-24T06:30:57.845287+0000","last_fullsized":"2024-04-24T06:30:57.845287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:08.055676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813969+0000","last_change":"2024-04-24T06:30:57.813969+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813969+0000","last_undegraded":"2024-04-24T06:30:57.813969+0000","last_fullsized":"2024-04-24T06:30:57.813969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841288+0000","last_change":"2024-04-24T06:30:57.838469+0000","last_active":"2024-04-24T06:30:57.841288+0000","last_peered":"2024-04-24T06:30:57.841288+0000","last_clean":"2024-04-24T06:30:57.841288+0000","last_became_active":"2024-04-24T06:30:57.837756+0000","last_became_peered":"2024-04-24T06:30:57.837756+0000","last_unstale":"2024-04-24T06:30:57.841288+0000","last_undegraded":"2024-04-24T06:30:57.841288+0000","last_fullsized":"2024-04-24T06:30:57.841288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.828392+0000","last_change":"2024-04-24T06:30:57.828392+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.828392+0000","last_undegraded":"2024-04-24T06:30:57.828392+0000","last_fullsized":"2024-04-24T06:30:57.828392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844768+0000","last_change":"2024-04-24T06:30:57.841819+0000","last_active":"2024-04-24T06:30:57.844768+0000","last_peered":"2024-04-24T06:30:57.844768+0000","last_clean":"2024-04-24T06:30:57.844768+0000","last_became_active":"2024-04-24T06:30:57.841562+0000","last_became_peered":"2024-04-24T06:30:57.841562+0000","last_unstale":"2024-04-24T06:30:57.844768+0000","last_undegraded":"2024-04-24T06:30:57.844768+0000","last_fullsized":"2024-04-24T06:30:57.844768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:00.545236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812705+0000","last_change":"2024-04-24T06:30:57.812705+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812705+0000","last_undegraded":"2024-04-24T06:30:57.812705+0000","last_fullsized":"2024-04-24T06:30:57.812705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849535+0000","last_change":"2024-04-24T06:30:57.844143+0000","last_active":"2024-04-24T06:30:57.849535+0000","last_peered":"2024-04-24T06:30:57.849535+0000","last_clean":"2024-04-24T06:30:57.849535+0000","last_became_active":"2024-04-24T06:30:57.843866+0000","last_became_peered":"2024-04-24T06:30:57.843866+0000","last_unstale":"2024-04-24T06:30:57.849535+0000","last_undegraded":"2024-04-24T06:30:57.849535+0000","last_fullsized":"2024-04-24T06:30:57.849535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:37.425589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825155+0000","last_change":"2024-04-24T06:30:57.825155+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825155+0000","last_undegraded":"2024-04-24T06:30:57.825155+0000","last_fullsized":"2024-04-24T06:30:57.825155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842315+0000","last_change":"2024-04-24T06:30:57.836689+0000","last_active":"2024-04-24T06:30:57.842315+0000","last_peered":"2024-04-24T06:30:57.842315+0000","last_clean":"2024-04-24T06:30:57.842315+0000","last_became_active":"2024-04-24T06:30:57.836379+0000","last_became_peered":"2024-04-24T06:30:57.836379+0000","last_unstale":"2024-04-24T06:30:57.842315+0000","last_undegraded":"2024-04-24T06:30:57.842315+0000","last_fullsized":"2024-04-24T06:30:57.842315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821398+0000","last_change":"2024-04-24T06:30:57.821398+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821398+0000","last_undegraded":"2024-04-24T06:30:57.821398+0000","last_fullsized":"2024-04-24T06:30:57.821398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.847322+0000","last_change":"2024-04-24T06:30:57.842064+0000","last_active":"2024-04-24T06:30:57.847322+0000","last_peered":"2024-04-24T06:30:57.847322+0000","last_clean":"2024-04-24T06:30:57.847322+0000","last_became_active":"2024-04-24T06:30:57.841762+0000","last_became_peered":"2024-04-24T06:30:57.841762+0000","last_unstale":"2024-04-24T06:30:57.847322+0000","last_undegraded":"2024-04-24T06:30:57.847322+0000","last_fullsized":"2024-04-24T06:30:57.847322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825006+0000","last_change":"2024-04-24T06:30:57.825006+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825006+0000","last_undegraded":"2024-04-24T06:30:57.825006+0000","last_fullsized":"2024-04-24T06:30:57.825006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.840560+0000","last_change":"2024-04-24T06:30:57.830985+0000","last_active":"2024-04-24T06:30:57.840560+0000","last_peered":"2024-04-24T06:30:57.840560+0000","last_clean":"2024-04-24T06:30:57.840560+0000","last_became_active":"2024-04-24T06:30:57.830710+0000","last_became_peered":"2024-04-24T06:30:57.830710+0000","last_unstale":"2024-04-24T06:30:57.840560+0000","last_undegraded":"2024-04-24T06:30:57.840560+0000","last_fullsized":"2024-04-24T06:30:57.840560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:07.336222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820085+0000","last_change":"2024-04-24T06:30:57.820085+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820085+0000","last_undegraded":"2024-04-24T06:30:57.820085+0000","last_fullsized":"2024-04-24T06:30:57.820085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835911+0000","last_change":"2024-04-24T06:30:57.834070+0000","last_active":"2024-04-24T06:30:57.835911+0000","last_peered":"2024-04-24T06:30:57.835911+0000","last_clean":"2024-04-24T06:30:57.835911+0000","last_became_active":"2024-04-24T06:30:57.833755+0000","last_became_peered":"2024-04-24T06:30:57.833755+0000","last_unstale":"2024-04-24T06:30:57.835911+0000","last_undegraded":"2024-04-24T06:30:57.835911+0000","last_fullsized":"2024-04-24T06:30:57.835911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.810018+0000","last_change":"2024-04-24T06:30:57.810018+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.810018+0000","last_undegraded":"2024-04-24T06:30:57.810018+0000","last_fullsized":"2024-04-24T06:30:57.810018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832606+0000","last_change":"2024-04-24T06:30:57.828293+0000","last_active":"2024-04-24T06:30:57.832606+0000","last_peered":"2024-04-24T06:30:57.832606+0000","last_clean":"2024-04-24T06:30:57.832606+0000","last_became_active":"2024-04-24T06:30:57.828011+0000","last_became_peered":"2024-04-24T06:30:57.828011+0000","last_unstale":"2024-04-24T06:30:57.832606+0000","last_undegraded":"2024-04-24T06:30:57.832606+0000","last_fullsized":"2024-04-24T06:30:57.832606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:27:38.386104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813206+0000","last_change":"2024-04-24T06:30:57.813206+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813206+0000","last_undegraded":"2024-04-24T06:30:57.813206+0000","last_fullsized":"2024-04-24T06:30:57.813206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849129+0000","last_change":"2024-04-24T06:30:57.844538+0000","last_active":"2024-04-24T06:30:57.849129+0000","last_peered":"2024-04-24T06:30:57.849129+0000","last_clean":"2024-04-24T06:30:57.849129+0000","last_became_active":"2024-04-24T06:30:57.844154+0000","last_became_peered":"2024-04-24T06:30:57.844154+0000","last_unstale":"2024-04-24T06:30:57.849129+0000","last_undegraded":"2024-04-24T06:30:57.849129+0000","last_fullsized":"2024-04-24T06:30:57.849129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.823129+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:30:57.823129+0000","last_peered":"2024-04-24T06:30:57.823129+0000","last_clean":"2024-04-24T06:30:57.823129+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:30:57.823129+0000","last_undegraded":"2024-04-24T06:30:57.823129+0000","last_fullsized":"2024-04-24T06:30:57.823129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":792408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:31:03.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:31:03.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.591+0000 7fe90d200640 1 Processor -- start 2024-04-24T06:31:03.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.591+0000 7fe90d200640 1 -- start start 2024-04-24T06:31:03.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe90d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9080cd050 0x7fe90806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe90d200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 0x7fe9080cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe90d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9080cd970 con 0x7fe9080b3080 2024-04-24T06:31:03.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe90d200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe9080ced20 con 0x7fe9080cd050 2024-04-24T06:31:03.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe90d200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe90815fcd0 con 0x7fe90815ece0 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe907400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9080cd050 0x7fe90806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe907e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe9080b3080 0x7fe9080cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48278/0 (socket says 172.21.15.59:48278) 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.593+0000 7fe907e00640 1 -- 172.21.15.59:0/3608022076 learned_addr learned my addr 172.21.15.59:0/3608022076 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/3608022076 >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 0x7fe9080cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe907400640 1 -- 172.21.15.59:0/3608022076 >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 msgr2=0x7fe9080cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe907400640 1 --2- 172.21.15.59:0/3608022076 >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 0x7fe9080cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe907400640 1 -- 172.21.15.59:0/3608022076 >> v1:172.21.15.59:6789/0 conn(0x7fe9080b3080 legacy=0x7fe9080cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe907400640 1 -- 172.21.15.59:0/3608022076 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90815f160 con 0x7fe9080cd050 2024-04-24T06:31:03.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.594+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/3608022076 >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 0x7fe9080cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.595+0000 7fe907400640 1 --2- 172.21.15.59:0/3608022076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9080cd050 0x7fe90806f620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe90806e210 tx=0x7fe8f802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01f7cca65f821d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.595+0000 7fe906a00640 1 -- 172.21.15.59:0/3608022076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8037070 con 0x7fe9080cd050 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.595+0000 7fe906a00640 1 -- 172.21.15.59:0/3608022076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f8032440 con 0x7fe9080cd050 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.595+0000 7fe906a00640 1 -- 172.21.15.59:0/3608022076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f802ae60 con 0x7fe9080cd050 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 -- 172.21.15.59:0/3608022076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9080cd050 msgr2=0x7fe90806f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 --2- 172.21.15.59:0/3608022076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9080cd050 0x7fe90806f620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe90806e210 tx=0x7fe8f802a110 comp rx=0 tx=0).stop 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 -- 172.21.15.59:0/3608022076 shutdown_connections 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 --2- 172.21.15.59:0/3608022076 >> v2:172.21.15.66:3300/0 conn(0x7fe90815ece0 0x7fe9080cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 --2- 172.21.15.59:0/3608022076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9080cd050 0x7fe90806f620 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 -- 172.21.15.59:0/3608022076 >> 172.21.15.59:0/3608022076 conn(0x7fe9080b21a0 msgr2=0x7fe9080b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 -- 172.21.15.59:0/3608022076 shutdown_connections 2024-04-24T06:31:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.596+0000 7fe90d200640 1 -- 172.21.15.59:0/3608022076 wait complete. 2024-04-24T06:31:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.598+0000 7fe90d200640 1 Processor -- start 2024-04-24T06:31:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.598+0000 7fe90d200640 1 -- start start 2024-04-24T06:31:03.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9080b3080 0x7fe90809bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 0x7fe9080c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe90815f9f0 con 0x7fe90815ece0 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe90815f710 con 0x7fe9080cd050 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe90d200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe9080cef50 con 0x7fe9080b3080 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe907e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe907e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47112/0 (socket says 172.21.15.59:47112) 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe907e00640 1 -- 172.21.15.59:0/4152235677 learned_addr learned my addr 172.21.15.59:0/4152235677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.599+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 0x7fe9080c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 -- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9080b3080 msgr2=0x7fe90809bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907400640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9080b3080 0x7fe90809bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9080b3080 0x7fe90809bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 -- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 msgr2=0x7fe9080c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 0x7fe9080c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 -- 172.21.15.59:0/4152235677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90809d460 con 0x7fe90815ece0 2024-04-24T06:31:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 0x7fe9080c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe907e00640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe90000cbc0 tx=0x7fe90000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900017070 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90000ad40 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900012600 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90809d740 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.600+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90809ba90 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.602+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90804fd90 con 0x7fe90815ece0 2024-04-24T06:31:03.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.603+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe90001a070 con 0x7fe90815ece0 2024-04-24T06:31:03.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.603+0000 7fe8ff400640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 0x7fe8e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:03.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.604+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe90008b830 con 0x7fe90815ece0 2024-04-24T06:31:03.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.604+0000 7fe907400640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 0x7fe8e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:03.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.605+0000 7fe907400640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 0x7fe8e406edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8034000 tx=0x7fe8f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:03.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.608+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe900059600 con 0x7fe90815ece0 2024-04-24T06:31:03.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.806+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe90815e720 con 0x7fe8e406c8f0 2024-04-24T06:31:03.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.824+0000 7fe8ff400640 1 -- 172.21.15.59:0/4152235677 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7fe90815e720 con 0x7fe8e406c8f0 2024-04-24T06:31:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:03.831 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 msgr2=0x7fe8e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 0x7fe8e406edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8034000 tx=0x7fe8f802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 msgr2=0x7fe90809ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe90000cbc0 tx=0x7fe90000a430 comp rx=0 tx=0).stop 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 shutdown_connections 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8e406c8f0 0x7fe8e406edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90815ece0 0x7fe90809ca10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9080cd050 0x7fe9080c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 --2- 172.21.15.59:0/4152235677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9080b3080 0x7fe90809bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:03.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 >> 172.21.15.59:0/4152235677 conn(0x7fe9080b21a0 msgr2=0x7fe9080ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:03.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 shutdown_connections 2024-04-24T06:31:03.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:03.830+0000 7fe90d200640 1 -- 172.21.15.59:0/4152235677 wait complete. 2024-04-24T06:31:03.851 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-24T06:31:03.230579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225392,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749328,"statfs":{"total":773094113280,"available":772863311872,"internally_reserved":0,"allocated":5181440,"data_stored":2778255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001571"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823354+0000","last_change":"2024-04-24T06:30:57.823354+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823354+0000","last_undegraded":"2024-04-24T06:30:57.823354+0000","last_fullsized":"2024-04-24T06:30:57.823354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825952+0000","last_change":"2024-04-24T06:30:57.825952+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825952+0000","last_undegraded":"2024-04-24T06:30:57.825952+0000","last_fullsized":"2024-04-24T06:30:57.825952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815572+0000","last_change":"2024-04-24T06:30:57.815572+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815572+0000","last_undegraded":"2024-04-24T06:30:57.815572+0000","last_fullsized":"2024-04-24T06:30:57.815572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817440+0000","last_change":"2024-04-24T06:30:57.817440+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817440+0000","last_undegraded":"2024-04-24T06:30:57.817440+0000","last_fullsized":"2024-04-24T06:30:57.817440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819614+0000","last_change":"2024-04-24T06:30:57.819614+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819614+0000","last_undegraded":"2024-04-24T06:30:57.819614+0000","last_fullsized":"2024-04-24T06:30:57.819614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822617+0000","last_change":"2024-04-24T06:30:57.822617+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822617+0000","last_undegraded":"2024-04-24T06:30:57.822617+0000","last_fullsized":"2024-04-24T06:30:57.822617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826625+0000","last_change":"2024-04-24T06:30:57.826625+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826625+0000","last_undegraded":"2024-04-24T06:30:57.826625+0000","last_fullsized":"2024-04-24T06:30:57.826625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815801+0000","last_change":"2024-04-24T06:30:57.815801+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815801+0000","last_undegraded":"2024-04-24T06:30:57.815801+0000","last_fullsized":"2024-04-24T06:30:57.815801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817353+0000","last_change":"2024-04-24T06:30:57.817353+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817353+0000","last_undegraded":"2024-04-24T06:30:57.817353+0000","last_fullsized":"2024-04-24T06:30:57.817353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817275+0000","last_change":"2024-04-24T06:30:57.817275+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817275+0000","last_undegraded":"2024-04-24T06:30:57.817275+0000","last_fullsized":"2024-04-24T06:30:57.817275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816768+0000","last_change":"2024-04-24T06:30:57.816768+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816768+0000","last_undegraded":"2024-04-24T06:30:57.816768+0000","last_fullsized":"2024-04-24T06:30:57.816768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.830810+0000","last_change":"2024-04-24T06:30:57.830810+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.830810+0000","last_undegraded":"2024-04-24T06:30:57.830810+0000","last_fullsized":"2024-04-24T06:30:57.830810+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815397+0000","last_change":"2024-04-24T06:30:57.815397+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815397+0000","last_undegraded":"2024-04-24T06:30:57.815397+0000","last_fullsized":"2024-04-24T06:30:57.815397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816773+0000","last_change":"2024-04-24T06:30:57.816773+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816773+0000","last_undegraded":"2024-04-24T06:30:57.816773+0000","last_fullsized":"2024-04-24T06:30:57.816773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824106+0000","last_change":"2024-04-24T06:30:57.824106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824106+0000","last_undegraded":"2024-04-24T06:30:57.824106+0000","last_fullsized":"2024-04-24T06:30:57.824106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822636+0000","last_change":"2024-04-24T06:30:57.822636+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822636+0000","last_undegraded":"2024-04-24T06:30:57.822636+0000","last_fullsized":"2024-04-24T06:30:57.822636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812675+0000","last_change":"2024-04-24T06:30:57.812675+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812675+0000","last_undegraded":"2024-04-24T06:30:57.812675+0000","last_fullsized":"2024-04-24T06:30:57.812675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814940+0000","last_change":"2024-04-24T06:30:57.814940+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814940+0000","last_undegraded":"2024-04-24T06:30:57.814940+0000","last_fullsized":"2024-04-24T06:30:57.814940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.827593+0000","last_change":"2024-04-24T06:30:57.827593+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.827593+0000","last_undegraded":"2024-04-24T06:30:57.827593+0000","last_fullsized":"2024-04-24T06:30:57.827593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816449+0000","last_change":"2024-04-24T06:30:57.816449+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816449+0000","last_undegraded":"2024-04-24T06:30:57.816449+0000","last_fullsized":"2024-04-24T06:30:57.816449+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825494+0000","last_change":"2024-04-24T06:30:57.825494+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825494+0000","last_undegraded":"2024-04-24T06:30:57.825494+0000","last_fullsized":"2024-04-24T06:30:57.825494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816080+0000","last_change":"2024-04-24T06:30:57.816080+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816080+0000","last_undegraded":"2024-04-24T06:30:57.816080+0000","last_fullsized":"2024-04-24T06:30:57.816080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823690+0000","last_change":"2024-04-24T06:30:57.823690+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823690+0000","last_undegraded":"2024-04-24T06:30:57.823690+0000","last_fullsized":"2024-04-24T06:30:57.823690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821714+0000","last_change":"2024-04-24T06:30:57.821714+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821714+0000","last_undegraded":"2024-04-24T06:30:57.821714+0000","last_fullsized":"2024-04-24T06:30:57.821714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829589+0000","last_change":"2024-04-24T06:30:57.829589+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829589+0000","last_undegraded":"2024-04-24T06:30:57.829589+0000","last_fullsized":"2024-04-24T06:30:57.829589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815784+0000","last_change":"2024-04-24T06:30:57.815784+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815784+0000","last_undegraded":"2024-04-24T06:30:57.815784+0000","last_fullsized":"2024-04-24T06:30:57.815784+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.809984+0000","last_change":"2024-04-24T06:30:57.809984+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.809984+0000","last_undegraded":"2024-04-24T06:30:57.809984+0000","last_fullsized":"2024-04-24T06:30:57.809984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826656+0000","last_change":"2024-04-24T06:30:57.826656+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826656+0000","last_undegraded":"2024-04-24T06:30:57.826656+0000","last_fullsized":"2024-04-24T06:30:57.826656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824980+0000","last_change":"2024-04-24T06:30:57.824980+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824980+0000","last_undegraded":"2024-04-24T06:30:57.824980+0000","last_fullsized":"2024-04-24T06:30:57.824980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812237+0000","last_change":"2024-04-24T06:30:57.812237+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812237+0000","last_undegraded":"2024-04-24T06:30:57.812237+0000","last_fullsized":"2024-04-24T06:30:57.812237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821611+0000","last_change":"2024-04-24T06:30:57.821611+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821611+0000","last_undegraded":"2024-04-24T06:30:57.821611+0000","last_fullsized":"2024-04-24T06:30:57.821611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817106+0000","last_change":"2024-04-24T06:30:57.817106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817106+0000","last_undegraded":"2024-04-24T06:30:57.817106+0000","last_fullsized":"2024-04-24T06:30:57.817106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820562+0000","last_change":"2024-04-24T06:30:57.820562+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820562+0000","last_undegraded":"2024-04-24T06:30:57.820562+0000","last_fullsized":"2024-04-24T06:30:57.820562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844480+0000","last_change":"2024-04-24T06:30:57.837026+0000","last_active":"2024-04-24T06:30:57.844480+0000","last_peered":"2024-04-24T06:30:57.844480+0000","last_clean":"2024-04-24T06:30:57.844480+0000","last_became_active":"2024-04-24T06:30:57.836706+0000","last_became_peered":"2024-04-24T06:30:57.836706+0000","last_unstale":"2024-04-24T06:30:57.844480+0000","last_undegraded":"2024-04-24T06:30:57.844480+0000","last_fullsized":"2024-04-24T06:30:57.844480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829026+0000","last_change":"2024-04-24T06:30:57.829026+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829026+0000","last_undegraded":"2024-04-24T06:30:57.829026+0000","last_fullsized":"2024-04-24T06:30:57.829026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842666+0000","last_change":"2024-04-24T06:30:57.838444+0000","last_active":"2024-04-24T06:30:57.842666+0000","last_peered":"2024-04-24T06:30:57.842666+0000","last_clean":"2024-04-24T06:30:57.842666+0000","last_became_active":"2024-04-24T06:30:57.838117+0000","last_became_peered":"2024-04-24T06:30:57.838117+0000","last_unstale":"2024-04-24T06:30:57.842666+0000","last_undegraded":"2024-04-24T06:30:57.842666+0000","last_fullsized":"2024-04-24T06:30:57.842666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823489+0000","last_change":"2024-04-24T06:30:57.823489+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823489+0000","last_undegraded":"2024-04-24T06:30:57.823489+0000","last_fullsized":"2024-04-24T06:30:57.823489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.846663+0000","last_change":"2024-04-24T06:30:57.844480+0000","last_active":"2024-04-24T06:30:57.846663+0000","last_peered":"2024-04-24T06:30:57.846663+0000","last_clean":"2024-04-24T06:30:57.846663+0000","last_became_active":"2024-04-24T06:30:57.844131+0000","last_became_peered":"2024-04-24T06:30:57.844131+0000","last_unstale":"2024-04-24T06:30:57.846663+0000","last_undegraded":"2024-04-24T06:30:57.846663+0000","last_fullsized":"2024-04-24T06:30:57.846663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817619+0000","last_change":"2024-04-24T06:30:57.817619+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817619+0000","last_undegraded":"2024-04-24T06:30:57.817619+0000","last_fullsized":"2024-04-24T06:30:57.817619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849607+0000","last_change":"2024-04-24T06:30:57.845981+0000","last_active":"2024-04-24T06:30:57.849607+0000","last_peered":"2024-04-24T06:30:57.849607+0000","last_clean":"2024-04-24T06:30:57.849607+0000","last_became_active":"2024-04-24T06:30:57.845675+0000","last_became_peered":"2024-04-24T06:30:57.845675+0000","last_unstale":"2024-04-24T06:30:57.849607+0000","last_undegraded":"2024-04-24T06:30:57.849607+0000","last_fullsized":"2024-04-24T06:30:57.849607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.811045+0000","last_change":"2024-04-24T06:30:57.811045+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.811045+0000","last_undegraded":"2024-04-24T06:30:57.811045+0000","last_fullsized":"2024-04-24T06:30:57.811045+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844747+0000","last_change":"2024-04-24T06:30:57.841400+0000","last_active":"2024-04-24T06:30:57.844747+0000","last_peered":"2024-04-24T06:30:57.844747+0000","last_clean":"2024-04-24T06:30:57.844747+0000","last_became_active":"2024-04-24T06:30:57.841071+0000","last_became_peered":"2024-04-24T06:30:57.841071+0000","last_unstale":"2024-04-24T06:30:57.844747+0000","last_undegraded":"2024-04-24T06:30:57.844747+0000","last_fullsized":"2024-04-24T06:30:57.844747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:26.018041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816200+0000","last_change":"2024-04-24T06:30:57.816200+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816200+0000","last_undegraded":"2024-04-24T06:30:57.816200+0000","last_fullsized":"2024-04-24T06:30:57.816200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842087+0000","last_change":"2024-04-24T06:30:57.835137+0000","last_active":"2024-04-24T06:30:57.842087+0000","last_peered":"2024-04-24T06:30:57.842087+0000","last_clean":"2024-04-24T06:30:57.842087+0000","last_became_active":"2024-04-24T06:30:57.834822+0000","last_became_peered":"2024-04-24T06:30:57.834822+0000","last_unstale":"2024-04-24T06:30:57.842087+0000","last_undegraded":"2024-04-24T06:30:57.842087+0000","last_fullsized":"2024-04-24T06:30:57.842087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814695+0000","last_change":"2024-04-24T06:30:57.814695+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814695+0000","last_undegraded":"2024-04-24T06:30:57.814695+0000","last_fullsized":"2024-04-24T06:30:57.814695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845153+0000","last_change":"2024-04-24T06:30:57.840100+0000","last_active":"2024-04-24T06:30:57.845153+0000","last_peered":"2024-04-24T06:30:57.845153+0000","last_clean":"2024-04-24T06:30:57.845153+0000","last_became_active":"2024-04-24T06:30:57.839664+0000","last_became_peered":"2024-04-24T06:30:57.839664+0000","last_unstale":"2024-04-24T06:30:57.845153+0000","last_undegraded":"2024-04-24T06:30:57.845153+0000","last_fullsized":"2024-04-24T06:30:57.845153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824821+0000","last_change":"2024-04-24T06:30:57.824821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824821+0000","last_undegraded":"2024-04-24T06:30:57.824821+0000","last_fullsized":"2024-04-24T06:30:57.824821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832582+0000","last_change":"2024-04-24T06:30:57.825727+0000","last_active":"2024-04-24T06:30:57.832582+0000","last_peered":"2024-04-24T06:30:57.832582+0000","last_clean":"2024-04-24T06:30:57.832582+0000","last_became_active":"2024-04-24T06:30:57.825361+0000","last_became_peered":"2024-04-24T06:30:57.825361+0000","last_unstale":"2024-04-24T06:30:57.832582+0000","last_undegraded":"2024-04-24T06:30:57.832582+0000","last_fullsized":"2024-04-24T06:30:57.832582+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:09.450817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812403+0000","last_change":"2024-04-24T06:30:57.812403+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812403+0000","last_undegraded":"2024-04-24T06:30:57.812403+0000","last_fullsized":"2024-04-24T06:30:57.812403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842570+0000","last_change":"2024-04-24T06:30:57.838423+0000","last_active":"2024-04-24T06:30:57.842570+0000","last_peered":"2024-04-24T06:30:57.842570+0000","last_clean":"2024-04-24T06:30:57.842570+0000","last_became_active":"2024-04-24T06:30:57.838078+0000","last_became_peered":"2024-04-24T06:30:57.838078+0000","last_unstale":"2024-04-24T06:30:57.842570+0000","last_undegraded":"2024-04-24T06:30:57.842570+0000","last_fullsized":"2024-04-24T06:30:57.842570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821943+0000","last_change":"2024-04-24T06:30:57.821943+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821943+0000","last_undegraded":"2024-04-24T06:30:57.821943+0000","last_fullsized":"2024-04-24T06:30:57.821943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845770+0000","last_change":"2024-04-24T06:30:57.840907+0000","last_active":"2024-04-24T06:30:57.845770+0000","last_peered":"2024-04-24T06:30:57.845770+0000","last_clean":"2024-04-24T06:30:57.845770+0000","last_became_active":"2024-04-24T06:30:57.840593+0000","last_became_peered":"2024-04-24T06:30:57.840593+0000","last_unstale":"2024-04-24T06:30:57.845770+0000","last_undegraded":"2024-04-24T06:30:57.845770+0000","last_fullsized":"2024-04-24T06:30:57.845770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:40:00.530508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823821+0000","last_change":"2024-04-24T06:30:57.823821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823821+0000","last_undegraded":"2024-04-24T06:30:57.823821+0000","last_fullsized":"2024-04-24T06:30:57.823821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849557+0000","last_change":"2024-04-24T06:30:57.845222+0000","last_active":"2024-04-24T06:30:57.849557+0000","last_peered":"2024-04-24T06:30:57.849557+0000","last_clean":"2024-04-24T06:30:57.849557+0000","last_became_active":"2024-04-24T06:30:57.843505+0000","last_became_peered":"2024-04-24T06:30:57.843505+0000","last_unstale":"2024-04-24T06:30:57.849557+0000","last_undegraded":"2024-04-24T06:30:57.849557+0000","last_fullsized":"2024-04-24T06:30:57.849557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826174+0000","last_change":"2024-04-24T06:30:57.826174+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826174+0000","last_undegraded":"2024-04-24T06:30:57.826174+0000","last_fullsized":"2024-04-24T06:30:57.826174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843367+0000","last_change":"2024-04-24T06:30:57.841598+0000","last_active":"2024-04-24T06:30:57.843367+0000","last_peered":"2024-04-24T06:30:57.843367+0000","last_clean":"2024-04-24T06:30:57.843367+0000","last_became_active":"2024-04-24T06:30:57.841319+0000","last_became_peered":"2024-04-24T06:30:57.841319+0000","last_unstale":"2024-04-24T06:30:57.843367+0000","last_undegraded":"2024-04-24T06:30:57.843367+0000","last_fullsized":"2024-04-24T06:30:57.843367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:52.616317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824873+0000","last_change":"2024-04-24T06:30:57.824873+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824873+0000","last_undegraded":"2024-04-24T06:30:57.824873+0000","last_fullsized":"2024-04-24T06:30:57.824873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835720+0000","last_change":"2024-04-24T06:30:57.831254+0000","last_active":"2024-04-24T06:30:57.835720+0000","last_peered":"2024-04-24T06:30:57.835720+0000","last_clean":"2024-04-24T06:30:57.835720+0000","last_became_active":"2024-04-24T06:30:57.830948+0000","last_became_peered":"2024-04-24T06:30:57.830948+0000","last_unstale":"2024-04-24T06:30:57.835720+0000","last_undegraded":"2024-04-24T06:30:57.835720+0000","last_fullsized":"2024-04-24T06:30:57.835720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814759+0000","last_change":"2024-04-24T06:30:57.814759+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814759+0000","last_undegraded":"2024-04-24T06:30:57.814759+0000","last_fullsized":"2024-04-24T06:30:57.814759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849559+0000","last_change":"2024-04-24T06:30:57.844423+0000","last_active":"2024-04-24T06:30:57.849559+0000","last_peered":"2024-04-24T06:30:57.849559+0000","last_clean":"2024-04-24T06:30:57.849559+0000","last_became_active":"2024-04-24T06:30:57.843788+0000","last_became_peered":"2024-04-24T06:30:57.843788+0000","last_unstale":"2024-04-24T06:30:57.849559+0000","last_undegraded":"2024-04-24T06:30:57.849559+0000","last_fullsized":"2024-04-24T06:30:57.849559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.818746+0000","last_change":"2024-04-24T06:30:57.818746+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.818746+0000","last_undegraded":"2024-04-24T06:30:57.818746+0000","last_fullsized":"2024-04-24T06:30:57.818746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843669+0000","last_change":"2024-04-24T06:30:57.833067+0000","last_active":"2024-04-24T06:30:57.843669+0000","last_peered":"2024-04-24T06:30:57.843669+0000","last_clean":"2024-04-24T06:30:57.843669+0000","last_became_active":"2024-04-24T06:30:57.832715+0000","last_became_peered":"2024-04-24T06:30:57.832715+0000","last_unstale":"2024-04-24T06:30:57.843669+0000","last_undegraded":"2024-04-24T06:30:57.843669+0000","last_fullsized":"2024-04-24T06:30:57.843669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824842+0000","last_change":"2024-04-24T06:30:57.824842+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824842+0000","last_undegraded":"2024-04-24T06:30:57.824842+0000","last_fullsized":"2024-04-24T06:30:57.824842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832875+0000","last_change":"2024-04-24T06:30:57.830642+0000","last_active":"2024-04-24T06:30:57.832875+0000","last_peered":"2024-04-24T06:30:57.832875+0000","last_clean":"2024-04-24T06:30:57.832875+0000","last_became_active":"2024-04-24T06:30:57.830243+0000","last_became_peered":"2024-04-24T06:30:57.830243+0000","last_unstale":"2024-04-24T06:30:57.832875+0000","last_undegraded":"2024-04-24T06:30:57.832875+0000","last_fullsized":"2024-04-24T06:30:57.832875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:56.404238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814978+0000","last_change":"2024-04-24T06:30:57.814978+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814978+0000","last_undegraded":"2024-04-24T06:30:57.814978+0000","last_fullsized":"2024-04-24T06:30:57.814978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835865+0000","last_change":"2024-04-24T06:30:57.833779+0000","last_active":"2024-04-24T06:30:57.835865+0000","last_peered":"2024-04-24T06:30:57.835865+0000","last_clean":"2024-04-24T06:30:57.835865+0000","last_became_active":"2024-04-24T06:30:57.833537+0000","last_became_peered":"2024-04-24T06:30:57.833537+0000","last_unstale":"2024-04-24T06:30:57.835865+0000","last_undegraded":"2024-04-24T06:30:57.835865+0000","last_fullsized":"2024-04-24T06:30:57.835865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:43:04.048136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812756+0000","last_change":"2024-04-24T06:30:57.812756+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812756+0000","last_undegraded":"2024-04-24T06:30:57.812756+0000","last_fullsized":"2024-04-24T06:30:57.812756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841087+0000","last_change":"2024-04-24T06:30:57.830426+0000","last_active":"2024-04-24T06:30:57.841087+0000","last_peered":"2024-04-24T06:30:57.841087+0000","last_clean":"2024-04-24T06:30:57.841087+0000","last_became_active":"2024-04-24T06:30:57.830061+0000","last_became_peered":"2024-04-24T06:30:57.830061+0000","last_unstale":"2024-04-24T06:30:57.841087+0000","last_undegraded":"2024-04-24T06:30:57.841087+0000","last_fullsized":"2024-04-24T06:30:57.841087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:49.837169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829029+0000","last_change":"2024-04-24T06:30:57.829029+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829029+0000","last_undegraded":"2024-04-24T06:30:57.829029+0000","last_fullsized":"2024-04-24T06:30:57.829029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843580+0000","last_change":"2024-04-24T06:30:57.832486+0000","last_active":"2024-04-24T06:30:57.843580+0000","last_peered":"2024-04-24T06:30:57.843580+0000","last_clean":"2024-04-24T06:30:57.843580+0000","last_became_active":"2024-04-24T06:30:57.832182+0000","last_became_peered":"2024-04-24T06:30:57.832182+0000","last_unstale":"2024-04-24T06:30:57.843580+0000","last_undegraded":"2024-04-24T06:30:57.843580+0000","last_fullsized":"2024-04-24T06:30:57.843580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:52.738153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.832279+0000","last_change":"2024-04-24T06:30:57.832279+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.832279+0000","last_undegraded":"2024-04-24T06:30:57.832279+0000","last_fullsized":"2024-04-24T06:30:57.832279+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845028+0000","last_change":"2024-04-24T06:30:57.842579+0000","last_active":"2024-04-24T06:30:57.845028+0000","last_peered":"2024-04-24T06:30:57.845028+0000","last_clean":"2024-04-24T06:30:57.845028+0000","last_became_active":"2024-04-24T06:30:57.842265+0000","last_became_peered":"2024-04-24T06:30:57.842265+0000","last_unstale":"2024-04-24T06:30:57.845028+0000","last_undegraded":"2024-04-24T06:30:57.845028+0000","last_fullsized":"2024-04-24T06:30:57.845028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:03.622481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821845+0000","last_change":"2024-04-24T06:30:57.821845+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821845+0000","last_undegraded":"2024-04-24T06:30:57.821845+0000","last_fullsized":"2024-04-24T06:30:57.821845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844541+0000","last_change":"2024-04-24T06:30:57.840860+0000","last_active":"2024-04-24T06:30:57.844541+0000","last_peered":"2024-04-24T06:30:57.844541+0000","last_clean":"2024-04-24T06:30:57.844541+0000","last_became_active":"2024-04-24T06:30:57.840543+0000","last_became_peered":"2024-04-24T06:30:57.840543+0000","last_unstale":"2024-04-24T06:30:57.844541+0000","last_undegraded":"2024-04-24T06:30:57.844541+0000","last_fullsized":"2024-04-24T06:30:57.844541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:50.961300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819015+0000","last_change":"2024-04-24T06:30:57.819015+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819015+0000","last_undegraded":"2024-04-24T06:30:57.819015+0000","last_fullsized":"2024-04-24T06:30:57.819015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843981+0000","last_change":"2024-04-24T06:30:57.837623+0000","last_active":"2024-04-24T06:30:57.843981+0000","last_peered":"2024-04-24T06:30:57.843981+0000","last_clean":"2024-04-24T06:30:57.843981+0000","last_became_active":"2024-04-24T06:30:57.837021+0000","last_became_peered":"2024-04-24T06:30:57.837021+0000","last_unstale":"2024-04-24T06:30:57.843981+0000","last_undegraded":"2024-04-24T06:30:57.843981+0000","last_fullsized":"2024-04-24T06:30:57.843981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:12.661118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814915+0000","last_change":"2024-04-24T06:30:57.814915+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814915+0000","last_undegraded":"2024-04-24T06:30:57.814915+0000","last_fullsized":"2024-04-24T06:30:57.814915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845287+0000","last_change":"2024-04-24T06:30:57.840879+0000","last_active":"2024-04-24T06:30:57.845287+0000","last_peered":"2024-04-24T06:30:57.845287+0000","last_clean":"2024-04-24T06:30:57.845287+0000","last_became_active":"2024-04-24T06:30:57.840563+0000","last_became_peered":"2024-04-24T06:30:57.840563+0000","last_unstale":"2024-04-24T06:30:57.845287+0000","last_undegraded":"2024-04-24T06:30:57.845287+0000","last_fullsized":"2024-04-24T06:30:57.845287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:08.055676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813969+0000","last_change":"2024-04-24T06:30:57.813969+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813969+0000","last_undegraded":"2024-04-24T06:30:57.813969+0000","last_fullsized":"2024-04-24T06:30:57.813969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841288+0000","last_change":"2024-04-24T06:30:57.838469+0000","last_active":"2024-04-24T06:30:57.841288+0000","last_peered":"2024-04-24T06:30:57.841288+0000","last_clean":"2024-04-24T06:30:57.841288+0000","last_became_active":"2024-04-24T06:30:57.837756+0000","last_became_peered":"2024-04-24T06:30:57.837756+0000","last_unstale":"2024-04-24T06:30:57.841288+0000","last_undegraded":"2024-04-24T06:30:57.841288+0000","last_fullsized":"2024-04-24T06:30:57.841288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.828392+0000","last_change":"2024-04-24T06:30:57.828392+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.828392+0000","last_undegraded":"2024-04-24T06:30:57.828392+0000","last_fullsized":"2024-04-24T06:30:57.828392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844768+0000","last_change":"2024-04-24T06:30:57.841819+0000","last_active":"2024-04-24T06:30:57.844768+0000","last_peered":"2024-04-24T06:30:57.844768+0000","last_clean":"2024-04-24T06:30:57.844768+0000","last_became_active":"2024-04-24T06:30:57.841562+0000","last_became_peered":"2024-04-24T06:30:57.841562+0000","last_unstale":"2024-04-24T06:30:57.844768+0000","last_undegraded":"2024-04-24T06:30:57.844768+0000","last_fullsized":"2024-04-24T06:30:57.844768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:00.545236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812705+0000","last_change":"2024-04-24T06:30:57.812705+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812705+0000","last_undegraded":"2024-04-24T06:30:57.812705+0000","last_fullsized":"2024-04-24T06:30:57.812705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849535+0000","last_change":"2024-04-24T06:30:57.844143+0000","last_active":"2024-04-24T06:30:57.849535+0000","last_peered":"2024-04-24T06:30:57.849535+0000","last_clean":"2024-04-24T06:30:57.849535+0000","last_became_active":"2024-04-24T06:30:57.843866+0000","last_became_peered":"2024-04-24T06:30:57.843866+0000","last_unstale":"2024-04-24T06:30:57.849535+0000","last_undegraded":"2024-04-24T06:30:57.849535+0000","last_fullsized":"2024-04-24T06:30:57.849535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:37.425589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825155+0000","last_change":"2024-04-24T06:30:57.825155+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825155+0000","last_undegraded":"2024-04-24T06:30:57.825155+0000","last_fullsized":"2024-04-24T06:30:57.825155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842315+0000","last_change":"2024-04-24T06:30:57.836689+0000","last_active":"2024-04-24T06:30:57.842315+0000","last_peered":"2024-04-24T06:30:57.842315+0000","last_clean":"2024-04-24T06:30:57.842315+0000","last_became_active":"2024-04-24T06:30:57.836379+0000","last_became_peered":"2024-04-24T06:30:57.836379+0000","last_unstale":"2024-04-24T06:30:57.842315+0000","last_undegraded":"2024-04-24T06:30:57.842315+0000","last_fullsized":"2024-04-24T06:30:57.842315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821398+0000","last_change":"2024-04-24T06:30:57.821398+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821398+0000","last_undegraded":"2024-04-24T06:30:57.821398+0000","last_fullsized":"2024-04-24T06:30:57.821398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.847322+0000","last_change":"2024-04-24T06:30:57.842064+0000","last_active":"2024-04-24T06:30:57.847322+0000","last_peered":"2024-04-24T06:30:57.847322+0000","last_clean":"2024-04-24T06:30:57.847322+0000","last_became_active":"2024-04-24T06:30:57.841762+0000","last_became_peered":"2024-04-24T06:30:57.841762+0000","last_unstale":"2024-04-24T06:30:57.847322+0000","last_undegraded":"2024-04-24T06:30:57.847322+0000","last_fullsized":"2024-04-24T06:30:57.847322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825006+0000","last_change":"2024-04-24T06:30:57.825006+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825006+0000","last_undegraded":"2024-04-24T06:30:57.825006+0000","last_fullsized":"2024-04-24T06:30:57.825006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.840560+0000","last_change":"2024-04-24T06:30:57.830985+0000","last_active":"2024-04-24T06:30:57.840560+0000","last_peered":"2024-04-24T06:30:57.840560+0000","last_clean":"2024-04-24T06:30:57.840560+0000","last_became_active":"2024-04-24T06:30:57.830710+0000","last_became_peered":"2024-04-24T06:30:57.830710+0000","last_unstale":"2024-04-24T06:30:57.840560+0000","last_undegraded":"2024-04-24T06:30:57.840560+0000","last_fullsized":"2024-04-24T06:30:57.840560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:07.336222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820085+0000","last_change":"2024-04-24T06:30:57.820085+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820085+0000","last_undegraded":"2024-04-24T06:30:57.820085+0000","last_fullsized":"2024-04-24T06:30:57.820085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835911+0000","last_change":"2024-04-24T06:30:57.834070+0000","last_active":"2024-04-24T06:30:57.835911+0000","last_peered":"2024-04-24T06:30:57.835911+0000","last_clean":"2024-04-24T06:30:57.835911+0000","last_became_active":"2024-04-24T06:30:57.833755+0000","last_became_peered":"2024-04-24T06:30:57.833755+0000","last_unstale":"2024-04-24T06:30:57.835911+0000","last_undegraded":"2024-04-24T06:30:57.835911+0000","last_fullsized":"2024-04-24T06:30:57.835911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.810018+0000","last_change":"2024-04-24T06:30:57.810018+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.810018+0000","last_undegraded":"2024-04-24T06:30:57.810018+0000","last_fullsized":"2024-04-24T06:30:57.810018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832606+0000","last_change":"2024-04-24T06:30:57.828293+0000","last_active":"2024-04-24T06:30:57.832606+0000","last_peered":"2024-04-24T06:30:57.832606+0000","last_clean":"2024-04-24T06:30:57.832606+0000","last_became_active":"2024-04-24T06:30:57.828011+0000","last_became_peered":"2024-04-24T06:30:57.828011+0000","last_unstale":"2024-04-24T06:30:57.832606+0000","last_undegraded":"2024-04-24T06:30:57.832606+0000","last_fullsized":"2024-04-24T06:30:57.832606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:27:38.386104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813206+0000","last_change":"2024-04-24T06:30:57.813206+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813206+0000","last_undegraded":"2024-04-24T06:30:57.813206+0000","last_fullsized":"2024-04-24T06:30:57.813206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849129+0000","last_change":"2024-04-24T06:30:57.844538+0000","last_active":"2024-04-24T06:30:57.849129+0000","last_peered":"2024-04-24T06:30:57.849129+0000","last_clean":"2024-04-24T06:30:57.849129+0000","last_became_active":"2024-04-24T06:30:57.844154+0000","last_became_peered":"2024-04-24T06:30:57.844154+0000","last_unstale":"2024-04-24T06:30:57.849129+0000","last_undegraded":"2024-04-24T06:30:57.849129+0000","last_fullsized":"2024-04-24T06:30:57.849129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.823129+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:30:57.823129+0000","last_peered":"2024-04-24T06:30:57.823129+0000","last_clean":"2024-04-24T06:30:57.823129+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:30:57.823129+0000","last_undegraded":"2024-04-24T06:30:57.823129+0000","last_fullsized":"2024-04-24T06:30:57.823129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":792408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:31:03.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:31:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.113+0000 7f1dc5800640 1 Processor -- start 2024-04-24T06:31:04.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.113+0000 7f1dc5800640 1 -- start start 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1dc00b4690 0x7f1dc00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc5800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc0160210 con 0x7f1dc015c3d0 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc5800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc01604f0 con 0x7f1dc00b4690 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc01607d0 con 0x7f1dc00cd760 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dbfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1dc015c3d0 0x7f1dc00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48288/0 (socket says 172.21.15.59:48288) 2024-04-24T06:31:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dbfe00640 1 -- 172.21.15.59:0/2557265944 learned_addr learned my addr 172.21.15.59:0/2557265944 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.115+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/2557265944 >> v2:172.21.15.59:3300/0 conn(0x7f1dc00b4690 0x7f1dc00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1db7e00640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1dc4e00640 1 -- 172.21.15.59:0/2557265944 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 msgr2=0x7f1dc00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1dbf400640 1 -- 172.21.15.59:0/2557265944 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948435250 0 0) 0x7f1dc0160210 con 0x7f1dc015c3d0 2024-04-24T06:31:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1dc4e00640 1 -- 172.21.15.59:0/2557265944 >> v1:172.21.15.59:6789/0 conn(0x7f1dc015c3d0 legacy=0x7f1dc00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1dc4e00640 1 -- 172.21.15.59:0/2557265944 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0160ab0 con 0x7f1dc00b4690 2024-04-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.116+0000 7f1db7e00640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.117+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc00b4690 0x7f1dc00cb7d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48afeb9c086c4040 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.117+0000 7f1dbf400640 1 -- 172.21.15.59:0/2557265944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007740 con 0x7f1dc00b4690 2024-04-24T06:31:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.117+0000 7f1dbf400640 1 -- 172.21.15.59:0/2557265944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db80078e0 con 0x7f1dc00b4690 2024-04-24T06:31:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.117+0000 7f1dbf400640 1 -- 172.21.15.59:0/2557265944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007cb0 con 0x7f1dc00b4690 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 -- 172.21.15.59:0/2557265944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc00b4690 msgr2=0x7f1dc00cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc00b4690 0x7f1dc00cb7d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db802a660 comp rx=0 tx=0).stop 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 -- 172.21.15.59:0/2557265944 shutdown_connections 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc00b4690 0x7f1dc00cb7d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 --2- 172.21.15.59:0/2557265944 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 -- 172.21.15.59:0/2557265944 >> 172.21.15.59:0/2557265944 conn(0x7f1dc00b21a0 msgr2=0x7f1dc00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 -- 172.21.15.59:0/2557265944 shutdown_connections 2024-04-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.118+0000 7f1dc5800640 1 -- 172.21.15.59:0/2557265944 wait complete. 2024-04-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.120+0000 7f1dc5800640 1 Processor -- start 2024-04-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.120+0000 7f1dc5800640 1 -- start start 2024-04-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 0x7f1dc009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc015c7b0 con 0x7f1dc015c3d0 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00b4ac0 con 0x7f1dc00b4690 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc015c930 con 0x7f1dc00cd760 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47126/0 (socket says 172.21.15.59:47126) 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.121+0000 7f1dc4e00640 1 -- 172.21.15.59:0/521510520 learned_addr learned my addr 172.21.15.59:0/521510520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dbfe00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 0x7f1dc009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1db7e00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 -- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 msgr2=0x7f1dc00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 -- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 msgr2=0x7f1dc009dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 0x7f1dc009dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 -- 172.21.15.59:0/521510520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00ce6f0 con 0x7f1dc015c3d0 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dbfe00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 0x7f1dc009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.122+0000 7f1dc4e00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.123+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8040070 con 0x7f1dc015c3d0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.123+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8031c40 con 0x7f1dc015c3d0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.123+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8003990 con 0x7f1dc015c3d0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.123+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc00ce9d0 con 0x7f1dc015c3d0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.123+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc00cef90 con 0x7f1dc015c3d0 2024-04-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.124+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc004fd90 con 0x7f1dc015c3d0 2024-04-24T06:31:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.125+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db8039040 con 0x7f1dc015c3d0 2024-04-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.127+0000 7f1dbcc00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.127+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1db80acb60 con 0x7f1dc015c3d0 2024-04-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.127+0000 7f1dbfe00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.128+0000 7f1dbfe00640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000f60 tx=0x7f1db0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.131+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1db807af90 con 0x7f1dc015c3d0 2024-04-24T06:31:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.330+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1dc00b3240 con 0x7f1d9c06c8f0 2024-04-24T06:31:04.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.337+0000 7f1dbcc00640 1 -- 172.21.15.59:0/521510520 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246641 (secure 0 0 0) 0x7f1dc00b3240 con 0x7f1d9c06c8f0 2024-04-24T06:31:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:31:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 msgr2=0x7f1d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000f60 tx=0x7f1db0009040 comp rx=0 tx=0).stop 2024-04-24T06:31:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 msgr2=0x7f1dc00d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db8004a40 comp rx=0 tx=0).stop 2024-04-24T06:31:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 shutdown_connections 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc015c3d0 0x7f1dc00d0120 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1dc00cd760 0x7f1dc00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 --2- 172.21.15.59:0/521510520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dc00b4690 0x7f1dc009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 >> 172.21.15.59:0/521510520 conn(0x7f1dc00b21a0 msgr2=0x7f1dc006e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 shutdown_connections 2024-04-24T06:31:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.344+0000 7f1dc5800640 1 -- 172.21.15.59:0/521510520 wait complete. 2024-04-24T06:31:04.367 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-04-24T06:31:03.230579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225392,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749328,"statfs":{"total":773094113280,"available":772863311872,"internally_reserved":0,"allocated":5181440,"data_stored":2778255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001571"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823354+0000","last_change":"2024-04-24T06:30:57.823354+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823354+0000","last_undegraded":"2024-04-24T06:30:57.823354+0000","last_fullsized":"2024-04-24T06:30:57.823354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825952+0000","last_change":"2024-04-24T06:30:57.825952+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825952+0000","last_undegraded":"2024-04-24T06:30:57.825952+0000","last_fullsized":"2024-04-24T06:30:57.825952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815572+0000","last_change":"2024-04-24T06:30:57.815572+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815572+0000","last_undegraded":"2024-04-24T06:30:57.815572+0000","last_fullsized":"2024-04-24T06:30:57.815572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817440+0000","last_change":"2024-04-24T06:30:57.817440+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817440+0000","last_undegraded":"2024-04-24T06:30:57.817440+0000","last_fullsized":"2024-04-24T06:30:57.817440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819614+0000","last_change":"2024-04-24T06:30:57.819614+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819614+0000","last_undegraded":"2024-04-24T06:30:57.819614+0000","last_fullsized":"2024-04-24T06:30:57.819614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822617+0000","last_change":"2024-04-24T06:30:57.822617+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822617+0000","last_undegraded":"2024-04-24T06:30:57.822617+0000","last_fullsized":"2024-04-24T06:30:57.822617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826625+0000","last_change":"2024-04-24T06:30:57.826625+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826625+0000","last_undegraded":"2024-04-24T06:30:57.826625+0000","last_fullsized":"2024-04-24T06:30:57.826625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815801+0000","last_change":"2024-04-24T06:30:57.815801+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815801+0000","last_undegraded":"2024-04-24T06:30:57.815801+0000","last_fullsized":"2024-04-24T06:30:57.815801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817353+0000","last_change":"2024-04-24T06:30:57.817353+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817353+0000","last_undegraded":"2024-04-24T06:30:57.817353+0000","last_fullsized":"2024-04-24T06:30:57.817353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817275+0000","last_change":"2024-04-24T06:30:57.817275+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817275+0000","last_undegraded":"2024-04-24T06:30:57.817275+0000","last_fullsized":"2024-04-24T06:30:57.817275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816768+0000","last_change":"2024-04-24T06:30:57.816768+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816768+0000","last_undegraded":"2024-04-24T06:30:57.816768+0000","last_fullsized":"2024-04-24T06:30:57.816768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.830810+0000","last_change":"2024-04-24T06:30:57.830810+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.830810+0000","last_undegraded":"2024-04-24T06:30:57.830810+0000","last_fullsized":"2024-04-24T06:30:57.830810+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815397+0000","last_change":"2024-04-24T06:30:57.815397+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815397+0000","last_undegraded":"2024-04-24T06:30:57.815397+0000","last_fullsized":"2024-04-24T06:30:57.815397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816773+0000","last_change":"2024-04-24T06:30:57.816773+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816773+0000","last_undegraded":"2024-04-24T06:30:57.816773+0000","last_fullsized":"2024-04-24T06:30:57.816773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824106+0000","last_change":"2024-04-24T06:30:57.824106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824106+0000","last_undegraded":"2024-04-24T06:30:57.824106+0000","last_fullsized":"2024-04-24T06:30:57.824106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.822636+0000","last_change":"2024-04-24T06:30:57.822636+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.822636+0000","last_undegraded":"2024-04-24T06:30:57.822636+0000","last_fullsized":"2024-04-24T06:30:57.822636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812675+0000","last_change":"2024-04-24T06:30:57.812675+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812675+0000","last_undegraded":"2024-04-24T06:30:57.812675+0000","last_fullsized":"2024-04-24T06:30:57.812675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814940+0000","last_change":"2024-04-24T06:30:57.814940+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814940+0000","last_undegraded":"2024-04-24T06:30:57.814940+0000","last_fullsized":"2024-04-24T06:30:57.814940+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.827593+0000","last_change":"2024-04-24T06:30:57.827593+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.827593+0000","last_undegraded":"2024-04-24T06:30:57.827593+0000","last_fullsized":"2024-04-24T06:30:57.827593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816449+0000","last_change":"2024-04-24T06:30:57.816449+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816449+0000","last_undegraded":"2024-04-24T06:30:57.816449+0000","last_fullsized":"2024-04-24T06:30:57.816449+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825494+0000","last_change":"2024-04-24T06:30:57.825494+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825494+0000","last_undegraded":"2024-04-24T06:30:57.825494+0000","last_fullsized":"2024-04-24T06:30:57.825494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816080+0000","last_change":"2024-04-24T06:30:57.816080+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816080+0000","last_undegraded":"2024-04-24T06:30:57.816080+0000","last_fullsized":"2024-04-24T06:30:57.816080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823690+0000","last_change":"2024-04-24T06:30:57.823690+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823690+0000","last_undegraded":"2024-04-24T06:30:57.823690+0000","last_fullsized":"2024-04-24T06:30:57.823690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821714+0000","last_change":"2024-04-24T06:30:57.821714+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821714+0000","last_undegraded":"2024-04-24T06:30:57.821714+0000","last_fullsized":"2024-04-24T06:30:57.821714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829589+0000","last_change":"2024-04-24T06:30:57.829589+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829589+0000","last_undegraded":"2024-04-24T06:30:57.829589+0000","last_fullsized":"2024-04-24T06:30:57.829589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.815784+0000","last_change":"2024-04-24T06:30:57.815784+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.815784+0000","last_undegraded":"2024-04-24T06:30:57.815784+0000","last_fullsized":"2024-04-24T06:30:57.815784+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.809984+0000","last_change":"2024-04-24T06:30:57.809984+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.809984+0000","last_undegraded":"2024-04-24T06:30:57.809984+0000","last_fullsized":"2024-04-24T06:30:57.809984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826656+0000","last_change":"2024-04-24T06:30:57.826656+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826656+0000","last_undegraded":"2024-04-24T06:30:57.826656+0000","last_fullsized":"2024-04-24T06:30:57.826656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824980+0000","last_change":"2024-04-24T06:30:57.824980+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824980+0000","last_undegraded":"2024-04-24T06:30:57.824980+0000","last_fullsized":"2024-04-24T06:30:57.824980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812237+0000","last_change":"2024-04-24T06:30:57.812237+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812237+0000","last_undegraded":"2024-04-24T06:30:57.812237+0000","last_fullsized":"2024-04-24T06:30:57.812237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821611+0000","last_change":"2024-04-24T06:30:57.821611+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821611+0000","last_undegraded":"2024-04-24T06:30:57.821611+0000","last_fullsized":"2024-04-24T06:30:57.821611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817106+0000","last_change":"2024-04-24T06:30:57.817106+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817106+0000","last_undegraded":"2024-04-24T06:30:57.817106+0000","last_fullsized":"2024-04-24T06:30:57.817106+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820562+0000","last_change":"2024-04-24T06:30:57.820562+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820562+0000","last_undegraded":"2024-04-24T06:30:57.820562+0000","last_fullsized":"2024-04-24T06:30:57.820562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844480+0000","last_change":"2024-04-24T06:30:57.837026+0000","last_active":"2024-04-24T06:30:57.844480+0000","last_peered":"2024-04-24T06:30:57.844480+0000","last_clean":"2024-04-24T06:30:57.844480+0000","last_became_active":"2024-04-24T06:30:57.836706+0000","last_became_peered":"2024-04-24T06:30:57.836706+0000","last_unstale":"2024-04-24T06:30:57.844480+0000","last_undegraded":"2024-04-24T06:30:57.844480+0000","last_fullsized":"2024-04-24T06:30:57.844480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829026+0000","last_change":"2024-04-24T06:30:57.829026+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829026+0000","last_undegraded":"2024-04-24T06:30:57.829026+0000","last_fullsized":"2024-04-24T06:30:57.829026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842666+0000","last_change":"2024-04-24T06:30:57.838444+0000","last_active":"2024-04-24T06:30:57.842666+0000","last_peered":"2024-04-24T06:30:57.842666+0000","last_clean":"2024-04-24T06:30:57.842666+0000","last_became_active":"2024-04-24T06:30:57.838117+0000","last_became_peered":"2024-04-24T06:30:57.838117+0000","last_unstale":"2024-04-24T06:30:57.842666+0000","last_undegraded":"2024-04-24T06:30:57.842666+0000","last_fullsized":"2024-04-24T06:30:57.842666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823489+0000","last_change":"2024-04-24T06:30:57.823489+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823489+0000","last_undegraded":"2024-04-24T06:30:57.823489+0000","last_fullsized":"2024-04-24T06:30:57.823489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.846663+0000","last_change":"2024-04-24T06:30:57.844480+0000","last_active":"2024-04-24T06:30:57.846663+0000","last_peered":"2024-04-24T06:30:57.846663+0000","last_clean":"2024-04-24T06:30:57.846663+0000","last_became_active":"2024-04-24T06:30:57.844131+0000","last_became_peered":"2024-04-24T06:30:57.844131+0000","last_unstale":"2024-04-24T06:30:57.846663+0000","last_undegraded":"2024-04-24T06:30:57.846663+0000","last_fullsized":"2024-04-24T06:30:57.846663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:48.635358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.817619+0000","last_change":"2024-04-24T06:30:57.817619+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.817619+0000","last_undegraded":"2024-04-24T06:30:57.817619+0000","last_fullsized":"2024-04-24T06:30:57.817619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849607+0000","last_change":"2024-04-24T06:30:57.845981+0000","last_active":"2024-04-24T06:30:57.849607+0000","last_peered":"2024-04-24T06:30:57.849607+0000","last_clean":"2024-04-24T06:30:57.849607+0000","last_became_active":"2024-04-24T06:30:57.845675+0000","last_became_peered":"2024-04-24T06:30:57.845675+0000","last_unstale":"2024-04-24T06:30:57.849607+0000","last_undegraded":"2024-04-24T06:30:57.849607+0000","last_fullsized":"2024-04-24T06:30:57.849607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.811045+0000","last_change":"2024-04-24T06:30:57.811045+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.811045+0000","last_undegraded":"2024-04-24T06:30:57.811045+0000","last_fullsized":"2024-04-24T06:30:57.811045+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844747+0000","last_change":"2024-04-24T06:30:57.841400+0000","last_active":"2024-04-24T06:30:57.844747+0000","last_peered":"2024-04-24T06:30:57.844747+0000","last_clean":"2024-04-24T06:30:57.844747+0000","last_became_active":"2024-04-24T06:30:57.841071+0000","last_became_peered":"2024-04-24T06:30:57.841071+0000","last_unstale":"2024-04-24T06:30:57.844747+0000","last_undegraded":"2024-04-24T06:30:57.844747+0000","last_fullsized":"2024-04-24T06:30:57.844747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:26.018041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.816200+0000","last_change":"2024-04-24T06:30:57.816200+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.816200+0000","last_undegraded":"2024-04-24T06:30:57.816200+0000","last_fullsized":"2024-04-24T06:30:57.816200+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842087+0000","last_change":"2024-04-24T06:30:57.835137+0000","last_active":"2024-04-24T06:30:57.842087+0000","last_peered":"2024-04-24T06:30:57.842087+0000","last_clean":"2024-04-24T06:30:57.842087+0000","last_became_active":"2024-04-24T06:30:57.834822+0000","last_became_peered":"2024-04-24T06:30:57.834822+0000","last_unstale":"2024-04-24T06:30:57.842087+0000","last_undegraded":"2024-04-24T06:30:57.842087+0000","last_fullsized":"2024-04-24T06:30:57.842087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814695+0000","last_change":"2024-04-24T06:30:57.814695+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814695+0000","last_undegraded":"2024-04-24T06:30:57.814695+0000","last_fullsized":"2024-04-24T06:30:57.814695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845153+0000","last_change":"2024-04-24T06:30:57.840100+0000","last_active":"2024-04-24T06:30:57.845153+0000","last_peered":"2024-04-24T06:30:57.845153+0000","last_clean":"2024-04-24T06:30:57.845153+0000","last_became_active":"2024-04-24T06:30:57.839664+0000","last_became_peered":"2024-04-24T06:30:57.839664+0000","last_unstale":"2024-04-24T06:30:57.845153+0000","last_undegraded":"2024-04-24T06:30:57.845153+0000","last_fullsized":"2024-04-24T06:30:57.845153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824821+0000","last_change":"2024-04-24T06:30:57.824821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824821+0000","last_undegraded":"2024-04-24T06:30:57.824821+0000","last_fullsized":"2024-04-24T06:30:57.824821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832582+0000","last_change":"2024-04-24T06:30:57.825727+0000","last_active":"2024-04-24T06:30:57.832582+0000","last_peered":"2024-04-24T06:30:57.832582+0000","last_clean":"2024-04-24T06:30:57.832582+0000","last_became_active":"2024-04-24T06:30:57.825361+0000","last_became_peered":"2024-04-24T06:30:57.825361+0000","last_unstale":"2024-04-24T06:30:57.832582+0000","last_undegraded":"2024-04-24T06:30:57.832582+0000","last_fullsized":"2024-04-24T06:30:57.832582+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:08:09.450817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812403+0000","last_change":"2024-04-24T06:30:57.812403+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812403+0000","last_undegraded":"2024-04-24T06:30:57.812403+0000","last_fullsized":"2024-04-24T06:30:57.812403+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842570+0000","last_change":"2024-04-24T06:30:57.838423+0000","last_active":"2024-04-24T06:30:57.842570+0000","last_peered":"2024-04-24T06:30:57.842570+0000","last_clean":"2024-04-24T06:30:57.842570+0000","last_became_active":"2024-04-24T06:30:57.838078+0000","last_became_peered":"2024-04-24T06:30:57.838078+0000","last_unstale":"2024-04-24T06:30:57.842570+0000","last_undegraded":"2024-04-24T06:30:57.842570+0000","last_fullsized":"2024-04-24T06:30:57.842570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821943+0000","last_change":"2024-04-24T06:30:57.821943+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821943+0000","last_undegraded":"2024-04-24T06:30:57.821943+0000","last_fullsized":"2024-04-24T06:30:57.821943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845770+0000","last_change":"2024-04-24T06:30:57.840907+0000","last_active":"2024-04-24T06:30:57.845770+0000","last_peered":"2024-04-24T06:30:57.845770+0000","last_clean":"2024-04-24T06:30:57.845770+0000","last_became_active":"2024-04-24T06:30:57.840593+0000","last_became_peered":"2024-04-24T06:30:57.840593+0000","last_unstale":"2024-04-24T06:30:57.845770+0000","last_undegraded":"2024-04-24T06:30:57.845770+0000","last_fullsized":"2024-04-24T06:30:57.845770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:40:00.530508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.823821+0000","last_change":"2024-04-24T06:30:57.823821+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.823821+0000","last_undegraded":"2024-04-24T06:30:57.823821+0000","last_fullsized":"2024-04-24T06:30:57.823821+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849557+0000","last_change":"2024-04-24T06:30:57.845222+0000","last_active":"2024-04-24T06:30:57.849557+0000","last_peered":"2024-04-24T06:30:57.849557+0000","last_clean":"2024-04-24T06:30:57.849557+0000","last_became_active":"2024-04-24T06:30:57.843505+0000","last_became_peered":"2024-04-24T06:30:57.843505+0000","last_unstale":"2024-04-24T06:30:57.849557+0000","last_undegraded":"2024-04-24T06:30:57.849557+0000","last_fullsized":"2024-04-24T06:30:57.849557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.826174+0000","last_change":"2024-04-24T06:30:57.826174+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.826174+0000","last_undegraded":"2024-04-24T06:30:57.826174+0000","last_fullsized":"2024-04-24T06:30:57.826174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843367+0000","last_change":"2024-04-24T06:30:57.841598+0000","last_active":"2024-04-24T06:30:57.843367+0000","last_peered":"2024-04-24T06:30:57.843367+0000","last_clean":"2024-04-24T06:30:57.843367+0000","last_became_active":"2024-04-24T06:30:57.841319+0000","last_became_peered":"2024-04-24T06:30:57.841319+0000","last_unstale":"2024-04-24T06:30:57.843367+0000","last_undegraded":"2024-04-24T06:30:57.843367+0000","last_fullsized":"2024-04-24T06:30:57.843367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:52.616317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824873+0000","last_change":"2024-04-24T06:30:57.824873+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824873+0000","last_undegraded":"2024-04-24T06:30:57.824873+0000","last_fullsized":"2024-04-24T06:30:57.824873+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835720+0000","last_change":"2024-04-24T06:30:57.831254+0000","last_active":"2024-04-24T06:30:57.835720+0000","last_peered":"2024-04-24T06:30:57.835720+0000","last_clean":"2024-04-24T06:30:57.835720+0000","last_became_active":"2024-04-24T06:30:57.830948+0000","last_became_peered":"2024-04-24T06:30:57.830948+0000","last_unstale":"2024-04-24T06:30:57.835720+0000","last_undegraded":"2024-04-24T06:30:57.835720+0000","last_fullsized":"2024-04-24T06:30:57.835720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814759+0000","last_change":"2024-04-24T06:30:57.814759+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814759+0000","last_undegraded":"2024-04-24T06:30:57.814759+0000","last_fullsized":"2024-04-24T06:30:57.814759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849559+0000","last_change":"2024-04-24T06:30:57.844423+0000","last_active":"2024-04-24T06:30:57.849559+0000","last_peered":"2024-04-24T06:30:57.849559+0000","last_clean":"2024-04-24T06:30:57.849559+0000","last_became_active":"2024-04-24T06:30:57.843788+0000","last_became_peered":"2024-04-24T06:30:57.843788+0000","last_unstale":"2024-04-24T06:30:57.849559+0000","last_undegraded":"2024-04-24T06:30:57.849559+0000","last_fullsized":"2024-04-24T06:30:57.849559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.818746+0000","last_change":"2024-04-24T06:30:57.818746+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.818746+0000","last_undegraded":"2024-04-24T06:30:57.818746+0000","last_fullsized":"2024-04-24T06:30:57.818746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843669+0000","last_change":"2024-04-24T06:30:57.833067+0000","last_active":"2024-04-24T06:30:57.843669+0000","last_peered":"2024-04-24T06:30:57.843669+0000","last_clean":"2024-04-24T06:30:57.843669+0000","last_became_active":"2024-04-24T06:30:57.832715+0000","last_became_peered":"2024-04-24T06:30:57.832715+0000","last_unstale":"2024-04-24T06:30:57.843669+0000","last_undegraded":"2024-04-24T06:30:57.843669+0000","last_fullsized":"2024-04-24T06:30:57.843669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.824842+0000","last_change":"2024-04-24T06:30:57.824842+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.824842+0000","last_undegraded":"2024-04-24T06:30:57.824842+0000","last_fullsized":"2024-04-24T06:30:57.824842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832875+0000","last_change":"2024-04-24T06:30:57.830642+0000","last_active":"2024-04-24T06:30:57.832875+0000","last_peered":"2024-04-24T06:30:57.832875+0000","last_clean":"2024-04-24T06:30:57.832875+0000","last_became_active":"2024-04-24T06:30:57.830243+0000","last_became_peered":"2024-04-24T06:30:57.830243+0000","last_unstale":"2024-04-24T06:30:57.832875+0000","last_undegraded":"2024-04-24T06:30:57.832875+0000","last_fullsized":"2024-04-24T06:30:57.832875+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:56.404238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814978+0000","last_change":"2024-04-24T06:30:57.814978+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814978+0000","last_undegraded":"2024-04-24T06:30:57.814978+0000","last_fullsized":"2024-04-24T06:30:57.814978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835865+0000","last_change":"2024-04-24T06:30:57.833779+0000","last_active":"2024-04-24T06:30:57.835865+0000","last_peered":"2024-04-24T06:30:57.835865+0000","last_clean":"2024-04-24T06:30:57.835865+0000","last_became_active":"2024-04-24T06:30:57.833537+0000","last_became_peered":"2024-04-24T06:30:57.833537+0000","last_unstale":"2024-04-24T06:30:57.835865+0000","last_undegraded":"2024-04-24T06:30:57.835865+0000","last_fullsized":"2024-04-24T06:30:57.835865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:43:04.048136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812756+0000","last_change":"2024-04-24T06:30:57.812756+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812756+0000","last_undegraded":"2024-04-24T06:30:57.812756+0000","last_fullsized":"2024-04-24T06:30:57.812756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841087+0000","last_change":"2024-04-24T06:30:57.830426+0000","last_active":"2024-04-24T06:30:57.841087+0000","last_peered":"2024-04-24T06:30:57.841087+0000","last_clean":"2024-04-24T06:30:57.841087+0000","last_became_active":"2024-04-24T06:30:57.830061+0000","last_became_peered":"2024-04-24T06:30:57.830061+0000","last_unstale":"2024-04-24T06:30:57.841087+0000","last_undegraded":"2024-04-24T06:30:57.841087+0000","last_fullsized":"2024-04-24T06:30:57.841087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:49.837169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.829029+0000","last_change":"2024-04-24T06:30:57.829029+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.829029+0000","last_undegraded":"2024-04-24T06:30:57.829029+0000","last_fullsized":"2024-04-24T06:30:57.829029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843580+0000","last_change":"2024-04-24T06:30:57.832486+0000","last_active":"2024-04-24T06:30:57.843580+0000","last_peered":"2024-04-24T06:30:57.843580+0000","last_clean":"2024-04-24T06:30:57.843580+0000","last_became_active":"2024-04-24T06:30:57.832182+0000","last_became_peered":"2024-04-24T06:30:57.832182+0000","last_unstale":"2024-04-24T06:30:57.843580+0000","last_undegraded":"2024-04-24T06:30:57.843580+0000","last_fullsized":"2024-04-24T06:30:57.843580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:52.738153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.832279+0000","last_change":"2024-04-24T06:30:57.832279+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.832279+0000","last_undegraded":"2024-04-24T06:30:57.832279+0000","last_fullsized":"2024-04-24T06:30:57.832279+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845028+0000","last_change":"2024-04-24T06:30:57.842579+0000","last_active":"2024-04-24T06:30:57.845028+0000","last_peered":"2024-04-24T06:30:57.845028+0000","last_clean":"2024-04-24T06:30:57.845028+0000","last_became_active":"2024-04-24T06:30:57.842265+0000","last_became_peered":"2024-04-24T06:30:57.842265+0000","last_unstale":"2024-04-24T06:30:57.845028+0000","last_undegraded":"2024-04-24T06:30:57.845028+0000","last_fullsized":"2024-04-24T06:30:57.845028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:03.622481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821845+0000","last_change":"2024-04-24T06:30:57.821845+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821845+0000","last_undegraded":"2024-04-24T06:30:57.821845+0000","last_fullsized":"2024-04-24T06:30:57.821845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844541+0000","last_change":"2024-04-24T06:30:57.840860+0000","last_active":"2024-04-24T06:30:57.844541+0000","last_peered":"2024-04-24T06:30:57.844541+0000","last_clean":"2024-04-24T06:30:57.844541+0000","last_became_active":"2024-04-24T06:30:57.840543+0000","last_became_peered":"2024-04-24T06:30:57.840543+0000","last_unstale":"2024-04-24T06:30:57.844541+0000","last_undegraded":"2024-04-24T06:30:57.844541+0000","last_fullsized":"2024-04-24T06:30:57.844541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:50.961300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.819015+0000","last_change":"2024-04-24T06:30:57.819015+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.819015+0000","last_undegraded":"2024-04-24T06:30:57.819015+0000","last_fullsized":"2024-04-24T06:30:57.819015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.843981+0000","last_change":"2024-04-24T06:30:57.837623+0000","last_active":"2024-04-24T06:30:57.843981+0000","last_peered":"2024-04-24T06:30:57.843981+0000","last_clean":"2024-04-24T06:30:57.843981+0000","last_became_active":"2024-04-24T06:30:57.837021+0000","last_became_peered":"2024-04-24T06:30:57.837021+0000","last_unstale":"2024-04-24T06:30:57.843981+0000","last_undegraded":"2024-04-24T06:30:57.843981+0000","last_fullsized":"2024-04-24T06:30:57.843981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:12.661118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.814915+0000","last_change":"2024-04-24T06:30:57.814915+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.814915+0000","last_undegraded":"2024-04-24T06:30:57.814915+0000","last_fullsized":"2024-04-24T06:30:57.814915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.845287+0000","last_change":"2024-04-24T06:30:57.840879+0000","last_active":"2024-04-24T06:30:57.845287+0000","last_peered":"2024-04-24T06:30:57.845287+0000","last_clean":"2024-04-24T06:30:57.845287+0000","last_became_active":"2024-04-24T06:30:57.840563+0000","last_became_peered":"2024-04-24T06:30:57.840563+0000","last_unstale":"2024-04-24T06:30:57.845287+0000","last_undegraded":"2024-04-24T06:30:57.845287+0000","last_fullsized":"2024-04-24T06:30:57.845287+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:25:08.055676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813969+0000","last_change":"2024-04-24T06:30:57.813969+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813969+0000","last_undegraded":"2024-04-24T06:30:57.813969+0000","last_fullsized":"2024-04-24T06:30:57.813969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.841288+0000","last_change":"2024-04-24T06:30:57.838469+0000","last_active":"2024-04-24T06:30:57.841288+0000","last_peered":"2024-04-24T06:30:57.841288+0000","last_clean":"2024-04-24T06:30:57.841288+0000","last_became_active":"2024-04-24T06:30:57.837756+0000","last_became_peered":"2024-04-24T06:30:57.837756+0000","last_unstale":"2024-04-24T06:30:57.841288+0000","last_undegraded":"2024-04-24T06:30:57.841288+0000","last_fullsized":"2024-04-24T06:30:57.841288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:17.935623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.828392+0000","last_change":"2024-04-24T06:30:57.828392+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.828392+0000","last_undegraded":"2024-04-24T06:30:57.828392+0000","last_fullsized":"2024-04-24T06:30:57.828392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.844768+0000","last_change":"2024-04-24T06:30:57.841819+0000","last_active":"2024-04-24T06:30:57.844768+0000","last_peered":"2024-04-24T06:30:57.844768+0000","last_clean":"2024-04-24T06:30:57.844768+0000","last_became_active":"2024-04-24T06:30:57.841562+0000","last_became_peered":"2024-04-24T06:30:57.841562+0000","last_unstale":"2024-04-24T06:30:57.844768+0000","last_undegraded":"2024-04-24T06:30:57.844768+0000","last_fullsized":"2024-04-24T06:30:57.844768+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:00.545236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.812705+0000","last_change":"2024-04-24T06:30:57.812705+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.812705+0000","last_undegraded":"2024-04-24T06:30:57.812705+0000","last_fullsized":"2024-04-24T06:30:57.812705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849535+0000","last_change":"2024-04-24T06:30:57.844143+0000","last_active":"2024-04-24T06:30:57.849535+0000","last_peered":"2024-04-24T06:30:57.849535+0000","last_clean":"2024-04-24T06:30:57.849535+0000","last_became_active":"2024-04-24T06:30:57.843866+0000","last_became_peered":"2024-04-24T06:30:57.843866+0000","last_unstale":"2024-04-24T06:30:57.849535+0000","last_undegraded":"2024-04-24T06:30:57.849535+0000","last_fullsized":"2024-04-24T06:30:57.849535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:37.425589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825155+0000","last_change":"2024-04-24T06:30:57.825155+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825155+0000","last_undegraded":"2024-04-24T06:30:57.825155+0000","last_fullsized":"2024-04-24T06:30:57.825155+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.842315+0000","last_change":"2024-04-24T06:30:57.836689+0000","last_active":"2024-04-24T06:30:57.842315+0000","last_peered":"2024-04-24T06:30:57.842315+0000","last_clean":"2024-04-24T06:30:57.842315+0000","last_became_active":"2024-04-24T06:30:57.836379+0000","last_became_peered":"2024-04-24T06:30:57.836379+0000","last_unstale":"2024-04-24T06:30:57.842315+0000","last_undegraded":"2024-04-24T06:30:57.842315+0000","last_fullsized":"2024-04-24T06:30:57.842315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:33.583436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.821398+0000","last_change":"2024-04-24T06:30:57.821398+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.821398+0000","last_undegraded":"2024-04-24T06:30:57.821398+0000","last_fullsized":"2024-04-24T06:30:57.821398+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.847322+0000","last_change":"2024-04-24T06:30:57.842064+0000","last_active":"2024-04-24T06:30:57.847322+0000","last_peered":"2024-04-24T06:30:57.847322+0000","last_clean":"2024-04-24T06:30:57.847322+0000","last_became_active":"2024-04-24T06:30:57.841762+0000","last_became_peered":"2024-04-24T06:30:57.841762+0000","last_unstale":"2024-04-24T06:30:57.847322+0000","last_undegraded":"2024-04-24T06:30:57.847322+0000","last_fullsized":"2024-04-24T06:30:57.847322+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.825006+0000","last_change":"2024-04-24T06:30:57.825006+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.825006+0000","last_undegraded":"2024-04-24T06:30:57.825006+0000","last_fullsized":"2024-04-24T06:30:57.825006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.840560+0000","last_change":"2024-04-24T06:30:57.830985+0000","last_active":"2024-04-24T06:30:57.840560+0000","last_peered":"2024-04-24T06:30:57.840560+0000","last_clean":"2024-04-24T06:30:57.840560+0000","last_became_active":"2024-04-24T06:30:57.830710+0000","last_became_peered":"2024-04-24T06:30:57.830710+0000","last_unstale":"2024-04-24T06:30:57.840560+0000","last_undegraded":"2024-04-24T06:30:57.840560+0000","last_fullsized":"2024-04-24T06:30:57.840560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:07.336222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.820085+0000","last_change":"2024-04-24T06:30:57.820085+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.820085+0000","last_undegraded":"2024-04-24T06:30:57.820085+0000","last_fullsized":"2024-04-24T06:30:57.820085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.835911+0000","last_change":"2024-04-24T06:30:57.834070+0000","last_active":"2024-04-24T06:30:57.835911+0000","last_peered":"2024-04-24T06:30:57.835911+0000","last_clean":"2024-04-24T06:30:57.835911+0000","last_became_active":"2024-04-24T06:30:57.833755+0000","last_became_peered":"2024-04-24T06:30:57.833755+0000","last_unstale":"2024-04-24T06:30:57.835911+0000","last_undegraded":"2024-04-24T06:30:57.835911+0000","last_fullsized":"2024-04-24T06:30:57.835911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:50.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.810018+0000","last_change":"2024-04-24T06:30:57.810018+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.810018+0000","last_undegraded":"2024-04-24T06:30:57.810018+0000","last_fullsized":"2024-04-24T06:30:57.810018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.832606+0000","last_change":"2024-04-24T06:30:57.828293+0000","last_active":"2024-04-24T06:30:57.832606+0000","last_peered":"2024-04-24T06:30:57.832606+0000","last_clean":"2024-04-24T06:30:57.832606+0000","last_became_active":"2024-04-24T06:30:57.828011+0000","last_became_peered":"2024-04-24T06:30:57.828011+0000","last_unstale":"2024-04-24T06:30:57.832606+0000","last_undegraded":"2024-04-24T06:30:57.832606+0000","last_fullsized":"2024-04-24T06:30:57.832606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:27:38.386104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-24T06:30:57.813206+0000","last_change":"2024-04-24T06:30:57.813206+0000","last_active":"2024-04-24T06:30:57.794876+0000","last_peered":"2024-04-24T06:30:57.794876+0000","last_clean":"2024-04-24T06:30:57.794876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:30:57.813206+0000","last_undegraded":"2024-04-24T06:30:57.813206+0000","last_fullsized":"2024-04-24T06:30:57.813206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.849129+0000","last_change":"2024-04-24T06:30:57.844538+0000","last_active":"2024-04-24T06:30:57.849129+0000","last_peered":"2024-04-24T06:30:57.849129+0000","last_clean":"2024-04-24T06:30:57.849129+0000","last_became_active":"2024-04-24T06:30:57.844154+0000","last_became_peered":"2024-04-24T06:30:57.844154+0000","last_unstale":"2024-04-24T06:30:57.849129+0000","last_undegraded":"2024-04-24T06:30:57.849129+0000","last_fullsized":"2024-04-24T06:30:57.849129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:46.452078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-24T06:30:57.823129+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:30:57.823129+0000","last_peered":"2024-04-24T06:30:57.823129+0000","last_clean":"2024-04-24T06:30:57.823129+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:30:57.823129+0000","last_undegraded":"2024-04-24T06:30:57.823129+0000","last_fullsized":"2024-04-24T06:30:57.823129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":202040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":792408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.823354+0000', 'last_change': '2024-04-24T06:30:57.823354+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.823354+0000', 'last_undegraded': '2024-04-24T06:30:57.823354+0000', 'last_fullsized': '2024-04-24T06:30:57.823354+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.825952+0000', 'last_change': '2024-04-24T06:30:57.825952+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.825952+0000', 'last_undegraded': '2024-04-24T06:30:57.825952+0000', 'last_fullsized': '2024-04-24T06:30:57.825952+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-24T06:31:04.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.815572+0000', 'last_change': '2024-04-24T06:30:57.815572+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.815572+0000', 'last_undegraded': '2024-04-24T06:30:57.815572+0000', 'last_fullsized': '2024-04-24T06:30:57.815572+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.817440+0000', 'last_change': '2024-04-24T06:30:57.817440+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.817440+0000', 'last_undegraded': '2024-04-24T06:30:57.817440+0000', 'last_fullsized': '2024-04-24T06:30:57.817440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.819614+0000', 'last_change': '2024-04-24T06:30:57.819614+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.819614+0000', 'last_undegraded': '2024-04-24T06:30:57.819614+0000', 'last_fullsized': '2024-04-24T06:30:57.819614+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.822617+0000', 'last_change': '2024-04-24T06:30:57.822617+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.822617+0000', 'last_undegraded': '2024-04-24T06:30:57.822617+0000', 'last_fullsized': '2024-04-24T06:30:57.822617+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.826625+0000', 'last_change': '2024-04-24T06:30:57.826625+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.826625+0000', 'last_undegraded': '2024-04-24T06:30:57.826625+0000', 'last_fullsized': '2024-04-24T06:30:57.826625+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-24T06:31:04.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.815801+0000', 'last_change': '2024-04-24T06:30:57.815801+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.815801+0000', 'last_undegraded': '2024-04-24T06:30:57.815801+0000', 'last_fullsized': '2024-04-24T06:30:57.815801+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.817353+0000', 'last_change': '2024-04-24T06:30:57.817353+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.817353+0000', 'last_undegraded': '2024-04-24T06:30:57.817353+0000', 'last_fullsized': '2024-04-24T06:30:57.817353+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.817275+0000', 'last_change': '2024-04-24T06:30:57.817275+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.817275+0000', 'last_undegraded': '2024-04-24T06:30:57.817275+0000', 'last_fullsized': '2024-04-24T06:30:57.817275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.816768+0000', 'last_change': '2024-04-24T06:30:57.816768+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.816768+0000', 'last_undegraded': '2024-04-24T06:30:57.816768+0000', 'last_fullsized': '2024-04-24T06:30:57.816768+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.830810+0000', 'last_change': '2024-04-24T06:30:57.830810+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.830810+0000', 'last_undegraded': '2024-04-24T06:30:57.830810+0000', 'last_fullsized': '2024-04-24T06:30:57.830810+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.815397+0000', 'last_change': '2024-04-24T06:30:57.815397+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.815397+0000', 'last_undegraded': '2024-04-24T06:30:57.815397+0000', 'last_fullsized': '2024-04-24T06:30:57.815397+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.816773+0000', 'last_change': '2024-04-24T06:30:57.816773+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.816773+0000', 'last_undegraded': '2024-04-24T06:30:57.816773+0000', 'last_fullsized': '2024-04-24T06:30:57.816773+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.824106+0000', 'last_change': '2024-04-24T06:30:57.824106+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.824106+0000', 'last_undegraded': '2024-04-24T06:30:57.824106+0000', 'last_fullsized': '2024-04-24T06:30:57.824106+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.822636+0000', 'last_change': '2024-04-24T06:30:57.822636+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.822636+0000', 'last_undegraded': '2024-04-24T06:30:57.822636+0000', 'last_fullsized': '2024-04-24T06:30:57.822636+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.812675+0000', 'last_change': '2024-04-24T06:30:57.812675+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.812675+0000', 'last_undegraded': '2024-04-24T06:30:57.812675+0000', 'last_fullsized': '2024-04-24T06:30:57.812675+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.814940+0000', 'last_change': '2024-04-24T06:30:57.814940+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.814940+0000', 'last_undegraded': '2024-04-24T06:30:57.814940+0000', 'last_fullsized': '2024-04-24T06:30:57.814940+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-24T06:31:04.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.827593+0000', 'last_change': '2024-04-24T06:30:57.827593+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.827593+0000', 'last_undegraded': '2024-04-24T06:30:57.827593+0000', 'last_fullsized': '2024-04-24T06:30:57.827593+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.816449+0000', 'last_change': '2024-04-24T06:30:57.816449+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.816449+0000', 'last_undegraded': '2024-04-24T06:30:57.816449+0000', 'last_fullsized': '2024-04-24T06:30:57.816449+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.825494+0000', 'last_change': '2024-04-24T06:30:57.825494+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.825494+0000', 'last_undegraded': '2024-04-24T06:30:57.825494+0000', 'last_fullsized': '2024-04-24T06:30:57.825494+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.816080+0000', 'last_change': '2024-04-24T06:30:57.816080+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.816080+0000', 'last_undegraded': '2024-04-24T06:30:57.816080+0000', 'last_fullsized': '2024-04-24T06:30:57.816080+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.823690+0000', 'last_change': '2024-04-24T06:30:57.823690+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.823690+0000', 'last_undegraded': '2024-04-24T06:30:57.823690+0000', 'last_fullsized': '2024-04-24T06:30:57.823690+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.821714+0000', 'last_change': '2024-04-24T06:30:57.821714+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.821714+0000', 'last_undegraded': '2024-04-24T06:30:57.821714+0000', 'last_fullsized': '2024-04-24T06:30:57.821714+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.829589+0000', 'last_change': '2024-04-24T06:30:57.829589+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.829589+0000', 'last_undegraded': '2024-04-24T06:30:57.829589+0000', 'last_fullsized': '2024-04-24T06:30:57.829589+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.815784+0000', 'last_change': '2024-04-24T06:30:57.815784+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.815784+0000', 'last_undegraded': '2024-04-24T06:30:57.815784+0000', 'last_fullsized': '2024-04-24T06:30:57.815784+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.809984+0000', 'last_change': '2024-04-24T06:30:57.809984+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.809984+0000', 'last_undegraded': '2024-04-24T06:30:57.809984+0000', 'last_fullsized': '2024-04-24T06:30:57.809984+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.826656+0000', 'last_change': '2024-04-24T06:30:57.826656+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.826656+0000', 'last_undegraded': '2024-04-24T06:30:57.826656+0000', 'last_fullsized': '2024-04-24T06:30:57.826656+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.824980+0000', 'last_change': '2024-04-24T06:30:57.824980+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.824980+0000', 'last_undegraded': '2024-04-24T06:30:57.824980+0000', 'last_fullsized': '2024-04-24T06:30:57.824980+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-24T06:31:04.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.812237+0000', 'last_change': '2024-04-24T06:30:57.812237+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.812237+0000', 'last_undegraded': '2024-04-24T06:30:57.812237+0000', 'last_fullsized': '2024-04-24T06:30:57.812237+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.821611+0000', 'last_change': '2024-04-24T06:30:57.821611+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.821611+0000', 'last_undegraded': '2024-04-24T06:30:57.821611+0000', 'last_fullsized': '2024-04-24T06:30:57.821611+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.817106+0000', 'last_change': '2024-04-24T06:30:57.817106+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.817106+0000', 'last_undegraded': '2024-04-24T06:30:57.817106+0000', 'last_fullsized': '2024-04-24T06:30:57.817106+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.820562+0000', 'last_change': '2024-04-24T06:30:57.820562+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.820562+0000', 'last_undegraded': '2024-04-24T06:30:57.820562+0000', 'last_fullsized': '2024-04-24T06:30:57.820562+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.829026+0000', 'last_change': '2024-04-24T06:30:57.829026+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.829026+0000', 'last_undegraded': '2024-04-24T06:30:57.829026+0000', 'last_fullsized': '2024-04-24T06:30:57.829026+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.823489+0000', 'last_change': '2024-04-24T06:30:57.823489+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.823489+0000', 'last_undegraded': '2024-04-24T06:30:57.823489+0000', 'last_fullsized': '2024-04-24T06:30:57.823489+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.817619+0000', 'last_change': '2024-04-24T06:30:57.817619+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.817619+0000', 'last_undegraded': '2024-04-24T06:30:57.817619+0000', 'last_fullsized': '2024-04-24T06:30:57.817619+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.811045+0000', 'last_change': '2024-04-24T06:30:57.811045+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.811045+0000', 'last_undegraded': '2024-04-24T06:30:57.811045+0000', 'last_fullsized': '2024-04-24T06:30:57.811045+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.816200+0000', 'last_change': '2024-04-24T06:30:57.816200+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.816200+0000', 'last_undegraded': '2024-04-24T06:30:57.816200+0000', 'last_fullsized': '2024-04-24T06:30:57.816200+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.814695+0000', 'last_change': '2024-04-24T06:30:57.814695+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.814695+0000', 'last_undegraded': '2024-04-24T06:30:57.814695+0000', 'last_fullsized': '2024-04-24T06:30:57.814695+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.824821+0000', 'last_change': '2024-04-24T06:30:57.824821+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.824821+0000', 'last_undegraded': '2024-04-24T06:30:57.824821+0000', 'last_fullsized': '2024-04-24T06:30:57.824821+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-24T06:31:04.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.812403+0000', 'last_change': '2024-04-24T06:30:57.812403+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.812403+0000', 'last_undegraded': '2024-04-24T06:30:57.812403+0000', 'last_fullsized': '2024-04-24T06:30:57.812403+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.821943+0000', 'last_change': '2024-04-24T06:30:57.821943+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.821943+0000', 'last_undegraded': '2024-04-24T06:30:57.821943+0000', 'last_fullsized': '2024-04-24T06:30:57.821943+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.823821+0000', 'last_change': '2024-04-24T06:30:57.823821+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.823821+0000', 'last_undegraded': '2024-04-24T06:30:57.823821+0000', 'last_fullsized': '2024-04-24T06:30:57.823821+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.826174+0000', 'last_change': '2024-04-24T06:30:57.826174+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.826174+0000', 'last_undegraded': '2024-04-24T06:30:57.826174+0000', 'last_fullsized': '2024-04-24T06:30:57.826174+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.824873+0000', 'last_change': '2024-04-24T06:30:57.824873+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.824873+0000', 'last_undegraded': '2024-04-24T06:30:57.824873+0000', 'last_fullsized': '2024-04-24T06:30:57.824873+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.814759+0000', 'last_change': '2024-04-24T06:30:57.814759+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.814759+0000', 'last_undegraded': '2024-04-24T06:30:57.814759+0000', 'last_fullsized': '2024-04-24T06:30:57.814759+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.818746+0000', 'last_change': '2024-04-24T06:30:57.818746+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.818746+0000', 'last_undegraded': '2024-04-24T06:30:57.818746+0000', 'last_fullsized': '2024-04-24T06:30:57.818746+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.824842+0000', 'last_change': '2024-04-24T06:30:57.824842+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.824842+0000', 'last_undegraded': '2024-04-24T06:30:57.824842+0000', 'last_fullsized': '2024-04-24T06:30:57.824842+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.814978+0000', 'last_change': '2024-04-24T06:30:57.814978+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.814978+0000', 'last_undegraded': '2024-04-24T06:30:57.814978+0000', 'last_fullsized': '2024-04-24T06:30:57.814978+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.812756+0000', 'last_change': '2024-04-24T06:30:57.812756+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.812756+0000', 'last_undegraded': '2024-04-24T06:30:57.812756+0000', 'last_fullsized': '2024-04-24T06:30:57.812756+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.829029+0000', 'last_change': '2024-04-24T06:30:57.829029+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.829029+0000', 'last_undegraded': '2024-04-24T06:30:57.829029+0000', 'last_fullsized': '2024-04-24T06:30:57.829029+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-24T06:31:04.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.832279+0000', 'last_change': '2024-04-24T06:30:57.832279+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.832279+0000', 'last_undegraded': '2024-04-24T06:30:57.832279+0000', 'last_fullsized': '2024-04-24T06:30:57.832279+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.821845+0000', 'last_change': '2024-04-24T06:30:57.821845+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.821845+0000', 'last_undegraded': '2024-04-24T06:30:57.821845+0000', 'last_fullsized': '2024-04-24T06:30:57.821845+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.819015+0000', 'last_change': '2024-04-24T06:30:57.819015+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.819015+0000', 'last_undegraded': '2024-04-24T06:30:57.819015+0000', 'last_fullsized': '2024-04-24T06:30:57.819015+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.814915+0000', 'last_change': '2024-04-24T06:30:57.814915+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.814915+0000', 'last_undegraded': '2024-04-24T06:30:57.814915+0000', 'last_fullsized': '2024-04-24T06:30:57.814915+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.813969+0000', 'last_change': '2024-04-24T06:30:57.813969+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.813969+0000', 'last_undegraded': '2024-04-24T06:30:57.813969+0000', 'last_fullsized': '2024-04-24T06:30:57.813969+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.828392+0000', 'last_change': '2024-04-24T06:30:57.828392+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.828392+0000', 'last_undegraded': '2024-04-24T06:30:57.828392+0000', 'last_fullsized': '2024-04-24T06:30:57.828392+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-24T06:31:04.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.812705+0000', 'last_change': '2024-04-24T06:30:57.812705+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.812705+0000', 'last_undegraded': '2024-04-24T06:30:57.812705+0000', 'last_fullsized': '2024-04-24T06:30:57.812705+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.825155+0000', 'last_change': '2024-04-24T06:30:57.825155+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.825155+0000', 'last_undegraded': '2024-04-24T06:30:57.825155+0000', 'last_fullsized': '2024-04-24T06:30:57.825155+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.821398+0000', 'last_change': '2024-04-24T06:30:57.821398+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.821398+0000', 'last_undegraded': '2024-04-24T06:30:57.821398+0000', 'last_fullsized': '2024-04-24T06:30:57.821398+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.825006+0000', 'last_change': '2024-04-24T06:30:57.825006+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.825006+0000', 'last_undegraded': '2024-04-24T06:30:57.825006+0000', 'last_fullsized': '2024-04-24T06:30:57.825006+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.820085+0000', 'last_change': '2024-04-24T06:30:57.820085+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.820085+0000', 'last_undegraded': '2024-04-24T06:30:57.820085+0000', 'last_fullsized': '2024-04-24T06:30:57.820085+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-24T06:31:04.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.810018+0000', 'last_change': '2024-04-24T06:30:57.810018+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.810018+0000', 'last_undegraded': '2024-04-24T06:30:57.810018+0000', 'last_fullsized': '2024-04-24T06:30:57.810018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-24T06:31:04.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-24T06:31:04.383 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-24T06:30:57.813206+0000', 'last_change': '2024-04-24T06:30:57.813206+0000', 'last_active': '2024-04-24T06:30:57.794876+0000', 'last_peered': '2024-04-24T06:30:57.794876+0000', 'last_clean': '2024-04-24T06:30:57.794876+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-24T06:30:57.813206+0000', 'last_undegraded': '2024-04-24T06:30:57.813206+0000', 'last_fullsized': '2024-04-24T06:30:57.813206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'last_clean_scrub_stamp': '2024-04-24T06:30:57.794876+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-24T06:31:04.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T06:31:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.624+0000 7fbcf5800640 1 Processor -- start 2024-04-24T06:31:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.624+0000 7fbcf5800640 1 -- start start 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcf5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf00afaf0 0x7fbcf00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcf5800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf015f0f0 con 0x7fbcf00d06e0 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcf5800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf015f3d0 con 0x7fbcf015c5e0 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcf5800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf015fa20 con 0x7fbcf00afaf0 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcee000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf00afaf0 0x7fbcf00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcef400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbcf00d06e0 0x7fbcf01624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48294/0 (socket says 172.21.15.59:48294) 2024-04-24T06:31:04.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.626+0000 7fbcef400640 1 -- 172.21.15.59:0/4071023798 learned_addr learned my addr 172.21.15.59:0/4071023798 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.627+0000 7fbcee000640 1 -- 172.21.15.59:0/4071023798 >> v1:172.21.15.66:6789/0 conn(0x7fbcf015c5e0 legacy=0x7fbcf00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.627+0000 7fbcee000640 1 -- 172.21.15.59:0/4071023798 >> v1:172.21.15.59:6789/0 conn(0x7fbcf00d06e0 legacy=0x7fbcf01624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.627+0000 7fbcee000640 1 -- 172.21.15.59:0/4071023798 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf015f6b0 con 0x7fbcf00afaf0 2024-04-24T06:31:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.627+0000 7fbced600640 1 -- 172.21.15.59:0/4071023798 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036803138 0 0) 0x7fbcf015f0f0 con 0x7fbcf00d06e0 2024-04-24T06:31:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.628+0000 7fbcee000640 1 --2- 172.21.15.59:0/4071023798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00afaf0 0x7fbcf00d01a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097a0 tx=0x7fbcd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9761a3b8390323a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.628+0000 7fbced600640 1 -- 172.21.15.59:0/4071023798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8037070 con 0x7fbcf00afaf0 2024-04-24T06:31:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.628+0000 7fbced600640 1 -- 172.21.15.59:0/4071023798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8032440 con 0x7fbcf00afaf0 2024-04-24T06:31:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.628+0000 7fbced600640 1 -- 172.21.15.59:0/4071023798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8032810 con 0x7fbcf00afaf0 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 -- 172.21.15.59:0/4071023798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00afaf0 msgr2=0x7fbcf00d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 --2- 172.21.15.59:0/4071023798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00afaf0 0x7fbcf00d01a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097a0 tx=0x7fbcd802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 -- 172.21.15.59:0/4071023798 shutdown_connections 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 --2- 172.21.15.59:0/4071023798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00afaf0 0x7fbcf00d01a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 -- 172.21.15.59:0/4071023798 >> 172.21.15.59:0/4071023798 conn(0x7fbcf0067ea0 msgr2=0x7fbcf006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 -- 172.21.15.59:0/4071023798 shutdown_connections 2024-04-24T06:31:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.629+0000 7fbcf5800640 1 -- 172.21.15.59:0/4071023798 wait complete. 2024-04-24T06:31:04.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.631+0000 7fbcf5800640 1 Processor -- start 2024-04-24T06:31:04.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.631+0000 7fbcf5800640 1 -- start start 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00afaf0 0x7fbcf00b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf015c5e0 0x7fbcf0155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf006e600 con 0x7fbcf00d06e0 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf006e480 con 0x7fbcf015c5e0 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcf5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf0162be0 con 0x7fbcf00afaf0 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcee000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcee000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47130/0 (socket says 172.21.15.59:47130) 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcee000640 1 -- 172.21.15.59:0/3283538097 learned_addr learned my addr 172.21.15.59:0/3283538097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcee000640 1 -- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00afaf0 msgr2=0x7fbcf00b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.632+0000 7fbcee000640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00afaf0 0x7fbcf00b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcee000640 1 -- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf015c5e0 msgr2=0x7fbcf0155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcee000640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf015c5e0 0x7fbcf0155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcee000640 1 -- 172.21.15.59:0/3283538097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf0147620 con 0x7fbcf00d06e0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcee000640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098c0 tx=0x7fbcd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8037040 con 0x7fbcf00d06e0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8042070 con 0x7fbcf00d06e0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf0147900 con 0x7fbcf00d06e0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030a70 con 0x7fbcf00d06e0 2024-04-24T06:31:04.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.633+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf0147ec0 con 0x7fbcf00d06e0 2024-04-24T06:31:04.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.635+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf0068a80 con 0x7fbcf00d06e0 2024-04-24T06:31:04.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.636+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd8030c10 con 0x7fbcf00d06e0 2024-04-24T06:31:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.637+0000 7fbce6a00640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.637+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fbcd80ac710 con 0x7fbcf00d06e0 2024-04-24T06:31:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.637+0000 7fbceea00640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.638+0000 7fbceea00640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbce000ab90 tx=0x7fbce0009230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:04.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.642+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcd807aa10 con 0x7fbcf00d06e0 2024-04-24T06:31:04.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.926+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbcf015f170 con 0x7fbcf00d06e0 2024-04-24T06:31:04.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.927+0000 7fbce6a00640 1 -- 172.21.15.59:0/3283538097 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7fbcd807a310 con 0x7fbcf00d06e0 2024-04-24T06:31:04.929 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:04.929 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1713940112,"num_in_osds":8,"osd_in_since":1713940103,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":230801408,"bytes_avail":772863311872,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5706}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-24T06:30:45.226328+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-24T06:31:04.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 msgr2=0x7fbccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbce000ab90 tx=0x7fbce0009230 comp rx=0 tx=0).stop 2024-04-24T06:31:04.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 msgr2=0x7fbcf00b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:04.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098c0 tx=0x7fbcd80046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 shutdown_connections 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf015c5e0 0x7fbcf0155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00d06e0 0x7fbcf00b1920 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 --2- 172.21.15.59:0/3283538097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00afaf0 0x7fbcf00b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 >> 172.21.15.59:0/3283538097 conn(0x7fbcf0067ea0 msgr2=0x7fbcf00b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.930+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 shutdown_connections 2024-04-24T06:31:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:04.931+0000 7fbcf5800640 1 -- 172.21.15.59:0/3283538097 wait complete. 2024-04-24T06:31:04.953 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-24T06:31:04.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:31:05.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.165+0000 7f5f4b200640 1 Processor -- start 2024-04-24T06:31:05.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.166+0000 7f5f4b200640 1 -- start start 2024-04-24T06:31:05.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.167+0000 7f5f4b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f4c15c3d0 0x7f5f4c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.167+0000 7f5f4b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c0b4860 0x7f5f4c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.167+0000 7f5f4b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f4c160210 con 0x7f5f4c0cd590 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.167+0000 7f5f4b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5f4c1604f0 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.167+0000 7f5f4b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4c1607d0 con 0x7f5f4c0b4860 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f4c15c3d0 0x7f5f4c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5f4c0cd590 0x7f5f4c0b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48306/0 (socket says 172.21.15.59:48306) 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49400640 1 -- 172.21.15.59:0/1451746222 learned_addr learned my addr 172.21.15.59:0/1451746222 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f4a800640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c0b4860 0x7f5f4c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49e00640 1 -- 172.21.15.59:0/1451746222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c0b4860 msgr2=0x7f5f4c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49e00640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c0b4860 0x7f5f4c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f48a00640 1 -- 172.21.15.59:0/1451746222 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288149206 0 0) 0x7f5f4c160210 con 0x7f5f4c0cd590 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49e00640 1 -- 172.21.15.59:0/1451746222 >> v1:172.21.15.59:6789/0 conn(0x7f5f4c0cd590 legacy=0x7f5f4c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.168+0000 7f5f49e00640 1 -- 172.21.15.59:0/1451746222 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c160ab0 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.170+0000 7f5f49e00640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c15c3d0 0x7f5f4c0cd050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14db948be47ceb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.170+0000 7f5f48a00640 1 -- 172.21.15.59:0/1451746222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c007740 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.170+0000 7f5f48a00640 1 -- 172.21.15.59:0/1451746222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3c0078e0 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.170+0000 7f5f48a00640 1 -- 172.21.15.59:0/1451746222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c007cb0 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 -- 172.21.15.59:0/1451746222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c15c3d0 msgr2=0x7f5f4c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c15c3d0 0x7f5f4c0cd050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c02a660 comp rx=0 tx=0).stop 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 -- 172.21.15.59:0/1451746222 shutdown_connections 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c0b4860 0x7f5f4c0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 --2- 172.21.15.59:0/1451746222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c15c3d0 0x7f5f4c0cd050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 -- 172.21.15.59:0/1451746222 >> 172.21.15.59:0/1451746222 conn(0x7f5f4c0b21a0 msgr2=0x7f5f4c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 -- 172.21.15.59:0/1451746222 shutdown_connections 2024-04-24T06:31:05.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.171+0000 7f5f4b200640 1 -- 172.21.15.59:0/1451746222 wait complete. 2024-04-24T06:31:05.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.173+0000 7f5f4b200640 1 Processor -- start 2024-04-24T06:31:05.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.173+0000 7f5f4b200640 1 -- start start 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 0x7f5f4c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c15c3d0 0x7f5f4c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c15c930 con 0x7f5f4c0b4860 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c15c7b0 con 0x7f5f4c0cd590 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f4b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4c0cbd10 con 0x7f5f4c15c3d0 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47150/0 (socket says 172.21.15.59:47150) 2024-04-24T06:31:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 -- 172.21.15.59:0/3238563596 learned_addr learned my addr 172.21.15.59:0/3238563596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:05.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49400640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 0x7f5f4c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:05.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 -- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c15c3d0 msgr2=0x7f5f4c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c15c3d0 0x7f5f4c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 -- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 msgr2=0x7f5f4c15af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 0x7f5f4c15af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49e00640 1 -- 172.21.15.59:0/3238563596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c0aecd0 con 0x7f5f4c0b4860 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.174+0000 7f5f49400640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 0x7f5f4c15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f49e00640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c03f070 con 0x7f5f4c0b4860 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3c031c40 con 0x7f5f4c0b4860 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c043410 con 0x7f5f4c0b4860 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4c0aefb0 con 0x7f5f4c0b4860 2024-04-24T06:31:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.175+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4c0c8750 con 0x7f5f4c0b4860 2024-04-24T06:31:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.176+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4c04fd90 con 0x7f5f4c0b4860 2024-04-24T06:31:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.177+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f3c0435b0 con 0x7f5f4c0b4860 2024-04-24T06:31:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.179+0000 7f5f42000640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 0x7f5f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.179+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5f3c0ad0c0 con 0x7f5f4c0b4860 2024-04-24T06:31:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.179+0000 7f5f49400640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 0x7f5f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.180+0000 7f5f49400640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 0x7f5f2806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0d0c60 tx=0x7f5f34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:05.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.184+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f3c07b450 con 0x7f5f4c0b4860 2024-04-24T06:31:05.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.388+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f4c0af180 con 0x7f5f2806c8f0 2024-04-24T06:31:05.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.397+0000 7f5f42000640 1 -- 172.21.15.59:0/3238563596 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f5f4c0af180 con 0x7f5f2806c8f0 2024-04-24T06:31:05.403 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:31:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.403+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 msgr2=0x7f5f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.403+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 0x7f5f2806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0d0c60 tx=0x7f5f34009040 comp rx=0 tx=0).stop 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 msgr2=0x7f5f4c0ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c004a40 comp rx=0 tx=0).stop 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 shutdown_connections 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5f2806c8f0 0x7f5f2806edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5f4c15c3d0 0x7f5f4c15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0cd590 0x7f5f4c15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 --2- 172.21.15.59:0/3238563596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f4c0b4860 0x7f5f4c0ae3c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 >> 172.21.15.59:0/3238563596 conn(0x7f5f4c0b21a0 msgr2=0x7f5f4c0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 shutdown_connections 2024-04-24T06:31:05.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:05.404+0000 7f5f4b200640 1 -- 172.21.15.59:0/3238563596 wait complete. 2024-04-24T06:31:05.426 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-04-24T06:31:05.231132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229864,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744856,"statfs":{"total":773094113280,"available":772858732544,"internally_reserved":0,"allocated":5480448,"data_stored":2906476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.427106"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719659+0000","last_change":"2024-04-24T06:30:58.871689+0000","last_active":"2024-04-24T06:31:00.719659+0000","last_peered":"2024-04-24T06:31:00.719659+0000","last_clean":"2024-04-24T06:31:00.719659+0000","last_became_active":"2024-04-24T06:30:58.871404+0000","last_became_peered":"2024-04-24T06:30:58.871404+0000","last_unstale":"2024-04-24T06:31:00.719659+0000","last_undegraded":"2024-04-24T06:31:00.719659+0000","last_fullsized":"2024-04-24T06:31:00.719659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:14.318658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019342+0000","last_change":"2024-04-24T06:30:58.867604+0000","last_active":"2024-04-24T06:31:00.019342+0000","last_peered":"2024-04-24T06:31:00.019342+0000","last_clean":"2024-04-24T06:31:00.019342+0000","last_became_active":"2024-04-24T06:30:58.867314+0000","last_became_peered":"2024-04-24T06:30:58.867314+0000","last_unstale":"2024-04-24T06:31:00.019342+0000","last_undegraded":"2024-04-24T06:31:00.019342+0000","last_fullsized":"2024-04-24T06:31:00.019342+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:04.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871079+0000","last_change":"2024-04-24T06:30:58.872202+0000","last_active":"2024-04-24T06:30:59.871079+0000","last_peered":"2024-04-24T06:30:59.871079+0000","last_clean":"2024-04-24T06:30:59.871079+0000","last_became_active":"2024-04-24T06:30:58.871886+0000","last_became_peered":"2024-04-24T06:30:58.871886+0000","last_unstale":"2024-04-24T06:30:59.871079+0000","last_undegraded":"2024-04-24T06:30:59.871079+0000","last_fullsized":"2024-04-24T06:30:59.871079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:01.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869436+0000","last_change":"2024-04-24T06:30:58.862197+0000","last_active":"2024-04-24T06:30:59.869436+0000","last_peered":"2024-04-24T06:30:59.869436+0000","last_clean":"2024-04-24T06:30:59.869436+0000","last_became_active":"2024-04-24T06:30:58.861844+0000","last_became_peered":"2024-04-24T06:30:58.861844+0000","last_unstale":"2024-04-24T06:30:59.869436+0000","last_undegraded":"2024-04-24T06:30:59.869436+0000","last_fullsized":"2024-04-24T06:30:59.869436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:04.627721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018895+0000","last_change":"2024-04-24T06:30:58.860625+0000","last_active":"2024-04-24T06:31:00.018895+0000","last_peered":"2024-04-24T06:31:00.018895+0000","last_clean":"2024-04-24T06:31:00.018895+0000","last_became_active":"2024-04-24T06:30:58.860255+0000","last_became_peered":"2024-04-24T06:30:58.860255+0000","last_unstale":"2024-04-24T06:31:00.018895+0000","last_undegraded":"2024-04-24T06:31:00.018895+0000","last_fullsized":"2024-04-24T06:31:00.018895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:11.926370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880415+0000","last_change":"2024-04-24T06:30:58.864319+0000","last_active":"2024-04-24T06:30:59.880415+0000","last_peered":"2024-04-24T06:30:59.880415+0000","last_clean":"2024-04-24T06:30:59.880415+0000","last_became_active":"2024-04-24T06:30:58.863968+0000","last_became_peered":"2024-04-24T06:30:58.863968+0000","last_unstale":"2024-04-24T06:30:59.880415+0000","last_undegraded":"2024-04-24T06:30:59.880415+0000","last_fullsized":"2024-04-24T06:30:59.880415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:28.258338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719450+0000","last_change":"2024-04-24T06:30:58.870026+0000","last_active":"2024-04-24T06:31:00.719450+0000","last_peered":"2024-04-24T06:31:00.719450+0000","last_clean":"2024-04-24T06:31:00.719450+0000","last_became_active":"2024-04-24T06:30:58.869087+0000","last_became_peered":"2024-04-24T06:30:58.869087+0000","last_unstale":"2024-04-24T06:31:00.719450+0000","last_undegraded":"2024-04-24T06:31:00.719450+0000","last_fullsized":"2024-04-24T06:31:00.719450+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:13.895061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868247+0000","last_change":"2024-04-24T06:30:58.865264+0000","last_active":"2024-04-24T06:30:59.868247+0000","last_peered":"2024-04-24T06:30:59.868247+0000","last_clean":"2024-04-24T06:30:59.868247+0000","last_became_active":"2024-04-24T06:30:58.862289+0000","last_became_peered":"2024-04-24T06:30:58.862289+0000","last_unstale":"2024-04-24T06:30:59.868247+0000","last_undegraded":"2024-04-24T06:30:59.868247+0000","last_fullsized":"2024-04-24T06:30:59.868247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:12.514254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869740+0000","last_change":"2024-04-24T06:30:58.864282+0000","last_active":"2024-04-24T06:30:59.869740+0000","last_peered":"2024-04-24T06:30:59.869740+0000","last_clean":"2024-04-24T06:30:59.869740+0000","last_became_active":"2024-04-24T06:30:58.863882+0000","last_became_peered":"2024-04-24T06:30:58.863882+0000","last_unstale":"2024-04-24T06:30:59.869740+0000","last_undegraded":"2024-04-24T06:30:59.869740+0000","last_fullsized":"2024-04-24T06:30:59.869740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:15.893761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869837+0000","last_change":"2024-04-24T06:30:58.865745+0000","last_active":"2024-04-24T06:30:59.869837+0000","last_peered":"2024-04-24T06:30:59.869837+0000","last_clean":"2024-04-24T06:30:59.869837+0000","last_became_active":"2024-04-24T06:30:58.864835+0000","last_became_peered":"2024-04-24T06:30:58.864835+0000","last_unstale":"2024-04-24T06:30:59.869837+0000","last_undegraded":"2024-04-24T06:30:59.869837+0000","last_fullsized":"2024-04-24T06:30:59.869837+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:12.360600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869888+0000","last_change":"2024-04-24T06:30:58.866632+0000","last_active":"2024-04-24T06:30:59.869888+0000","last_peered":"2024-04-24T06:30:59.869888+0000","last_clean":"2024-04-24T06:30:59.869888+0000","last_became_active":"2024-04-24T06:30:58.865213+0000","last_became_peered":"2024-04-24T06:30:58.865213+0000","last_unstale":"2024-04-24T06:30:59.869888+0000","last_undegraded":"2024-04-24T06:30:59.869888+0000","last_fullsized":"2024-04-24T06:30:59.869888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:11.926370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019108+0000","last_change":"2024-04-24T06:30:58.880299+0000","last_active":"2024-04-24T06:31:00.019108+0000","last_peered":"2024-04-24T06:31:00.019108+0000","last_clean":"2024-04-24T06:31:00.019108+0000","last_became_active":"2024-04-24T06:30:58.879940+0000","last_became_peered":"2024-04-24T06:30:58.879940+0000","last_unstale":"2024-04-24T06:31:00.019108+0000","last_undegraded":"2024-04-24T06:31:00.019108+0000","last_fullsized":"2024-04-24T06:31:00.019108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:19.924194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556943+0000","last_change":"2024-04-24T06:30:58.868071+0000","last_active":"2024-04-24T06:31:01.556943+0000","last_peered":"2024-04-24T06:31:01.556943+0000","last_clean":"2024-04-24T06:31:01.556943+0000","last_became_active":"2024-04-24T06:30:58.867536+0000","last_became_peered":"2024-04-24T06:30:58.867536+0000","last_unstale":"2024-04-24T06:31:01.556943+0000","last_undegraded":"2024-04-24T06:31:01.556943+0000","last_fullsized":"2024-04-24T06:31:01.556943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:50.842409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868439+0000","last_change":"2024-04-24T06:30:58.867069+0000","last_active":"2024-04-24T06:30:59.868439+0000","last_peered":"2024-04-24T06:30:59.868439+0000","last_clean":"2024-04-24T06:30:59.868439+0000","last_became_active":"2024-04-24T06:30:58.862775+0000","last_became_peered":"2024-04-24T06:30:58.862775+0000","last_unstale":"2024-04-24T06:30:59.868439+0000","last_undegraded":"2024-04-24T06:30:59.868439+0000","last_fullsized":"2024-04-24T06:30:59.868439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:01.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880337+0000","last_change":"2024-04-24T06:30:58.865306+0000","last_active":"2024-04-24T06:30:59.880337+0000","last_peered":"2024-04-24T06:30:59.880337+0000","last_clean":"2024-04-24T06:30:59.880337+0000","last_became_active":"2024-04-24T06:30:58.864360+0000","last_became_peered":"2024-04-24T06:30:58.864360+0000","last_unstale":"2024-04-24T06:30:59.880337+0000","last_undegraded":"2024-04-24T06:30:59.880337+0000","last_fullsized":"2024-04-24T06:30:59.880337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:36.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557401+0000","last_change":"2024-04-24T06:30:58.862904+0000","last_active":"2024-04-24T06:31:01.557401+0000","last_peered":"2024-04-24T06:31:01.557401+0000","last_clean":"2024-04-24T06:31:01.557401+0000","last_became_active":"2024-04-24T06:30:58.862231+0000","last_became_peered":"2024-04-24T06:30:58.862231+0000","last_unstale":"2024-04-24T06:31:01.557401+0000","last_undegraded":"2024-04-24T06:31:01.557401+0000","last_fullsized":"2024-04-24T06:31:01.557401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:12.795113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872244+0000","last_change":"2024-04-24T06:30:58.866711+0000","last_active":"2024-04-24T06:30:59.872244+0000","last_peered":"2024-04-24T06:30:59.872244+0000","last_clean":"2024-04-24T06:30:59.872244+0000","last_became_active":"2024-04-24T06:30:58.864055+0000","last_became_peered":"2024-04-24T06:30:58.864055+0000","last_unstale":"2024-04-24T06:30:59.872244+0000","last_undegraded":"2024-04-24T06:30:59.872244+0000","last_fullsized":"2024-04-24T06:30:59.872244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:12.360600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872070+0000","last_change":"2024-04-24T06:30:58.864767+0000","last_active":"2024-04-24T06:30:59.872070+0000","last_peered":"2024-04-24T06:30:59.872070+0000","last_clean":"2024-04-24T06:30:59.872070+0000","last_became_active":"2024-04-24T06:30:58.864318+0000","last_became_peered":"2024-04-24T06:30:58.864318+0000","last_unstale":"2024-04-24T06:30:59.872070+0000","last_undegraded":"2024-04-24T06:30:59.872070+0000","last_fullsized":"2024-04-24T06:30:59.872070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:01.165078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558749+0000","last_change":"2024-04-24T06:30:58.865342+0000","last_active":"2024-04-24T06:31:01.558749+0000","last_peered":"2024-04-24T06:31:01.558749+0000","last_clean":"2024-04-24T06:31:01.558749+0000","last_became_active":"2024-04-24T06:30:58.865070+0000","last_became_peered":"2024-04-24T06:30:58.865070+0000","last_unstale":"2024-04-24T06:31:01.558749+0000","last_undegraded":"2024-04-24T06:31:01.558749+0000","last_fullsized":"2024-04-24T06:31:01.558749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.430828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557389+0000","last_change":"2024-04-24T06:30:58.866451+0000","last_active":"2024-04-24T06:31:01.557389+0000","last_peered":"2024-04-24T06:31:01.557389+0000","last_clean":"2024-04-24T06:31:01.557389+0000","last_became_active":"2024-04-24T06:30:58.866157+0000","last_became_peered":"2024-04-24T06:30:58.866157+0000","last_unstale":"2024-04-24T06:31:01.557389+0000","last_undegraded":"2024-04-24T06:31:01.557389+0000","last_fullsized":"2024-04-24T06:31:01.557389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:34.588675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880583+0000","last_change":"2024-04-24T06:30:58.876631+0000","last_active":"2024-04-24T06:30:59.880583+0000","last_peered":"2024-04-24T06:30:59.880583+0000","last_clean":"2024-04-24T06:30:59.880583+0000","last_became_active":"2024-04-24T06:30:58.876311+0000","last_became_peered":"2024-04-24T06:30:58.876311+0000","last_unstale":"2024-04-24T06:30:59.880583+0000","last_undegraded":"2024-04-24T06:30:59.880583+0000","last_fullsized":"2024-04-24T06:30:59.880583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:24:57.723346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868721+0000","last_change":"2024-04-24T06:30:58.866407+0000","last_active":"2024-04-24T06:30:59.868721+0000","last_peered":"2024-04-24T06:30:59.868721+0000","last_clean":"2024-04-24T06:30:59.868721+0000","last_became_active":"2024-04-24T06:30:58.865418+0000","last_became_peered":"2024-04-24T06:30:58.865418+0000","last_unstale":"2024-04-24T06:30:59.868721+0000","last_undegraded":"2024-04-24T06:30:59.868721+0000","last_fullsized":"2024-04-24T06:30:59.868721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:27.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557499+0000","last_change":"2024-04-24T06:30:58.864816+0000","last_active":"2024-04-24T06:31:01.557499+0000","last_peered":"2024-04-24T06:31:01.557499+0000","last_clean":"2024-04-24T06:31:01.557499+0000","last_became_active":"2024-04-24T06:30:58.864454+0000","last_became_peered":"2024-04-24T06:30:58.864454+0000","last_unstale":"2024-04-24T06:31:01.557499+0000","last_undegraded":"2024-04-24T06:31:01.557499+0000","last_fullsized":"2024-04-24T06:31:01.557499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:39.404609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557431+0000","last_change":"2024-04-24T06:30:58.862606+0000","last_active":"2024-04-24T06:31:01.557431+0000","last_peered":"2024-04-24T06:31:01.557431+0000","last_clean":"2024-04-24T06:31:01.557431+0000","last_became_active":"2024-04-24T06:30:58.861854+0000","last_became_peered":"2024-04-24T06:30:58.861854+0000","last_unstale":"2024-04-24T06:31:01.557431+0000","last_undegraded":"2024-04-24T06:31:01.557431+0000","last_fullsized":"2024-04-24T06:31:01.557431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:47.710714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558469+0000","last_change":"2024-04-24T06:30:58.866056+0000","last_active":"2024-04-24T06:31:01.558469+0000","last_peered":"2024-04-24T06:31:01.558469+0000","last_clean":"2024-04-24T06:31:01.558469+0000","last_became_active":"2024-04-24T06:30:58.863311+0000","last_became_peered":"2024-04-24T06:30:58.863311+0000","last_unstale":"2024-04-24T06:31:01.558469+0000","last_undegraded":"2024-04-24T06:31:01.558469+0000","last_fullsized":"2024-04-24T06:31:01.558469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:53.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557043+0000","last_change":"2024-04-24T06:30:58.884986+0000","last_active":"2024-04-24T06:31:01.557043+0000","last_peered":"2024-04-24T06:31:01.557043+0000","last_clean":"2024-04-24T06:31:01.557043+0000","last_became_active":"2024-04-24T06:30:58.884648+0000","last_became_peered":"2024-04-24T06:30:58.884648+0000","last_unstale":"2024-04-24T06:31:01.557043+0000","last_undegraded":"2024-04-24T06:31:01.557043+0000","last_fullsized":"2024-04-24T06:31:01.557043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:51.765763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871901+0000","last_change":"2024-04-24T06:30:58.864078+0000","last_active":"2024-04-24T06:30:59.871901+0000","last_peered":"2024-04-24T06:30:59.871901+0000","last_clean":"2024-04-24T06:30:59.871901+0000","last_became_active":"2024-04-24T06:30:58.862258+0000","last_became_peered":"2024-04-24T06:30:58.862258+0000","last_unstale":"2024-04-24T06:30:59.871901+0000","last_undegraded":"2024-04-24T06:30:59.871901+0000","last_fullsized":"2024-04-24T06:30:59.871901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:08.619728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019627+0000","last_change":"2024-04-24T06:30:58.866076+0000","last_active":"2024-04-24T06:31:00.019627+0000","last_peered":"2024-04-24T06:31:00.019627+0000","last_clean":"2024-04-24T06:31:00.019627+0000","last_became_active":"2024-04-24T06:30:58.865774+0000","last_became_peered":"2024-04-24T06:30:58.865774+0000","last_unstale":"2024-04-24T06:31:00.019627+0000","last_undegraded":"2024-04-24T06:31:00.019627+0000","last_fullsized":"2024-04-24T06:31:00.019627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:22.472936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557999+0000","last_change":"2024-04-24T06:30:58.862627+0000","last_active":"2024-04-24T06:31:01.557999+0000","last_peered":"2024-04-24T06:31:01.557999+0000","last_clean":"2024-04-24T06:31:01.557999+0000","last_became_active":"2024-04-24T06:30:58.862204+0000","last_became_peered":"2024-04-24T06:30:58.862204+0000","last_unstale":"2024-04-24T06:31:01.557999+0000","last_undegraded":"2024-04-24T06:31:01.557999+0000","last_fullsized":"2024-04-24T06:31:01.557999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:55.673782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.870900+0000","last_change":"2024-04-24T06:30:58.864159+0000","last_active":"2024-04-24T06:30:59.870900+0000","last_peered":"2024-04-24T06:30:59.870900+0000","last_clean":"2024-04-24T06:30:59.870900+0000","last_became_active":"2024-04-24T06:30:58.862379+0000","last_became_peered":"2024-04-24T06:30:58.862379+0000","last_unstale":"2024-04-24T06:30:59.870900+0000","last_undegraded":"2024-04-24T06:30:59.870900+0000","last_fullsized":"2024-04-24T06:30:59.870900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:51.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880444+0000","last_change":"2024-04-24T06:30:58.864570+0000","last_active":"2024-04-24T06:30:59.880444+0000","last_peered":"2024-04-24T06:30:59.880444+0000","last_clean":"2024-04-24T06:30:59.880444+0000","last_became_active":"2024-04-24T06:30:58.863132+0000","last_became_peered":"2024-04-24T06:30:58.863132+0000","last_unstale":"2024-04-24T06:30:59.880444+0000","last_undegraded":"2024-04-24T06:30:59.880444+0000","last_fullsized":"2024-04-24T06:30:59.880444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:17.984864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556563+0000","last_change":"2024-04-24T06:30:58.870111+0000","last_active":"2024-04-24T06:31:01.556563+0000","last_peered":"2024-04-24T06:31:01.556563+0000","last_clean":"2024-04-24T06:31:01.556563+0000","last_became_active":"2024-04-24T06:30:58.869818+0000","last_became_peered":"2024-04-24T06:30:58.869818+0000","last_unstale":"2024-04-24T06:31:01.556563+0000","last_undegraded":"2024-04-24T06:31:01.556563+0000","last_fullsized":"2024-04-24T06:31:01.556563+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:27:53.827485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822587+0000","last_change":"2024-04-24T06:30:58.876187+0000","last_active":"2024-04-24T06:31:00.822587+0000","last_peered":"2024-04-24T06:31:00.822587+0000","last_clean":"2024-04-24T06:31:00.822587+0000","last_became_active":"2024-04-24T06:30:58.875909+0000","last_became_peered":"2024-04-24T06:30:58.875909+0000","last_unstale":"2024-04-24T06:31:00.822587+0000","last_undegraded":"2024-04-24T06:31:00.822587+0000","last_fullsized":"2024-04-24T06:31:00.822587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:18.940863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881545+0000","last_change":"2024-04-24T06:30:57.837026+0000","last_active":"2024-04-24T06:30:59.881545+0000","last_peered":"2024-04-24T06:30:59.881545+0000","last_clean":"2024-04-24T06:30:59.881545+0000","last_became_active":"2024-04-24T06:30:57.836706+0000","last_became_peered":"2024-04-24T06:30:57.836706+0000","last_unstale":"2024-04-24T06:30:59.881545+0000","last_undegraded":"2024-04-24T06:30:59.881545+0000","last_fullsized":"2024-04-24T06:30:59.881545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:10.921131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558746+0000","last_change":"2024-04-24T06:30:58.865805+0000","last_active":"2024-04-24T06:31:01.558746+0000","last_peered":"2024-04-24T06:31:01.558746+0000","last_clean":"2024-04-24T06:31:01.558746+0000","last_became_active":"2024-04-24T06:30:58.862885+0000","last_became_peered":"2024-04-24T06:30:58.862885+0000","last_unstale":"2024-04-24T06:31:01.558746+0000","last_undegraded":"2024-04-24T06:31:01.558746+0000","last_fullsized":"2024-04-24T06:31:01.558746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:47.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872357+0000","last_change":"2024-04-24T06:30:57.838444+0000","last_active":"2024-04-24T06:30:59.872357+0000","last_peered":"2024-04-24T06:30:59.872357+0000","last_clean":"2024-04-24T06:30:59.872357+0000","last_became_active":"2024-04-24T06:30:57.838117+0000","last_became_peered":"2024-04-24T06:30:57.838117+0000","last_unstale":"2024-04-24T06:30:59.872357+0000","last_undegraded":"2024-04-24T06:30:59.872357+0000","last_fullsized":"2024-04-24T06:30:59.872357+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:32.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019799+0000","last_change":"2024-04-24T06:30:58.858690+0000","last_active":"2024-04-24T06:31:00.019799+0000","last_peered":"2024-04-24T06:31:00.019799+0000","last_clean":"2024-04-24T06:31:00.019799+0000","last_became_active":"2024-04-24T06:30:58.858273+0000","last_became_peered":"2024-04-24T06:30:58.858273+0000","last_unstale":"2024-04-24T06:31:00.019799+0000","last_undegraded":"2024-04-24T06:31:00.019799+0000","last_fullsized":"2024-04-24T06:31:00.019799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:13.895061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822957+0000","last_change":"2024-04-24T06:30:57.844480+0000","last_active":"2024-04-24T06:31:00.822957+0000","last_peered":"2024-04-24T06:31:00.822957+0000","last_clean":"2024-04-24T06:31:00.822957+0000","last_became_active":"2024-04-24T06:30:57.844131+0000","last_became_peered":"2024-04-24T06:30:57.844131+0000","last_unstale":"2024-04-24T06:31:00.822957+0000","last_undegraded":"2024-04-24T06:31:00.822957+0000","last_fullsized":"2024-04-24T06:31:00.822957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:46.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.823021+0000","last_change":"2024-04-24T06:30:58.876295+0000","last_active":"2024-04-24T06:31:00.823021+0000","last_peered":"2024-04-24T06:31:00.823021+0000","last_clean":"2024-04-24T06:31:00.823021+0000","last_became_active":"2024-04-24T06:30:58.875942+0000","last_became_peered":"2024-04-24T06:30:58.875942+0000","last_unstale":"2024-04-24T06:31:00.823021+0000","last_undegraded":"2024-04-24T06:31:00.823021+0000","last_fullsized":"2024-04-24T06:31:00.823021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.430828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719157+0000","last_change":"2024-04-24T06:30:57.845981+0000","last_active":"2024-04-24T06:31:00.719157+0000","last_peered":"2024-04-24T06:31:00.719157+0000","last_clean":"2024-04-24T06:31:00.719157+0000","last_became_active":"2024-04-24T06:30:57.845675+0000","last_became_peered":"2024-04-24T06:30:57.845675+0000","last_unstale":"2024-04-24T06:31:00.719157+0000","last_undegraded":"2024-04-24T06:31:00.719157+0000","last_fullsized":"2024-04-24T06:31:00.719157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:50.961300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871004+0000","last_change":"2024-04-24T06:30:58.874570+0000","last_active":"2024-04-24T06:30:59.871004+0000","last_peered":"2024-04-24T06:30:59.871004+0000","last_clean":"2024-04-24T06:30:59.871004+0000","last_became_active":"2024-04-24T06:30:58.873859+0000","last_became_peered":"2024-04-24T06:30:58.873859+0000","last_unstale":"2024-04-24T06:30:59.871004+0000","last_undegraded":"2024-04-24T06:30:59.871004+0000","last_fullsized":"2024-04-24T06:30:59.871004+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:27.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880723+0000","last_change":"2024-04-24T06:30:57.841400+0000","last_active":"2024-04-24T06:30:59.880723+0000","last_peered":"2024-04-24T06:30:59.880723+0000","last_clean":"2024-04-24T06:30:59.880723+0000","last_became_active":"2024-04-24T06:30:57.841071+0000","last_became_peered":"2024-04-24T06:30:57.841071+0000","last_unstale":"2024-04-24T06:30:59.880723+0000","last_undegraded":"2024-04-24T06:30:59.880723+0000","last_fullsized":"2024-04-24T06:30:59.880723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:11.355360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869654+0000","last_change":"2024-04-24T06:30:58.866709+0000","last_active":"2024-04-24T06:30:59.869654+0000","last_peered":"2024-04-24T06:30:59.869654+0000","last_clean":"2024-04-24T06:30:59.869654+0000","last_became_active":"2024-04-24T06:30:58.866354+0000","last_became_peered":"2024-04-24T06:30:58.866354+0000","last_unstale":"2024-04-24T06:30:59.869654+0000","last_undegraded":"2024-04-24T06:30:59.869654+0000","last_fullsized":"2024-04-24T06:30:59.869654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:01.165078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871213+0000","last_change":"2024-04-24T06:30:57.835137+0000","last_active":"2024-04-24T06:30:59.871213+0000","last_peered":"2024-04-24T06:30:59.871213+0000","last_clean":"2024-04-24T06:30:59.871213+0000","last_became_active":"2024-04-24T06:30:57.834822+0000","last_became_peered":"2024-04-24T06:30:57.834822+0000","last_unstale":"2024-04-24T06:30:59.871213+0000","last_undegraded":"2024-04-24T06:30:59.871213+0000","last_fullsized":"2024-04-24T06:30:59.871213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:11.789874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868257+0000","last_change":"2024-04-24T06:30:58.866295+0000","last_active":"2024-04-24T06:30:59.868257+0000","last_peered":"2024-04-24T06:30:59.868257+0000","last_clean":"2024-04-24T06:30:59.868257+0000","last_became_active":"2024-04-24T06:30:58.865378+0000","last_became_peered":"2024-04-24T06:30:58.865378+0000","last_unstale":"2024-04-24T06:30:59.868257+0000","last_undegraded":"2024-04-24T06:30:59.868257+0000","last_fullsized":"2024-04-24T06:30:59.868257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:43.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018323+0000","last_change":"2024-04-24T06:30:57.840100+0000","last_active":"2024-04-24T06:31:00.018323+0000","last_peered":"2024-04-24T06:31:00.018323+0000","last_clean":"2024-04-24T06:31:00.018323+0000","last_became_active":"2024-04-24T06:30:57.839664+0000","last_became_peered":"2024-04-24T06:30:57.839664+0000","last_unstale":"2024-04-24T06:31:00.018323+0000","last_undegraded":"2024-04-24T06:31:00.018323+0000","last_fullsized":"2024-04-24T06:31:00.018323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557267+0000","last_change":"2024-04-24T06:30:58.868481+0000","last_active":"2024-04-24T06:31:01.557267+0000","last_peered":"2024-04-24T06:31:01.557267+0000","last_clean":"2024-04-24T06:31:01.557267+0000","last_became_active":"2024-04-24T06:30:58.866356+0000","last_became_peered":"2024-04-24T06:30:58.866356+0000","last_unstale":"2024-04-24T06:31:01.557267+0000","last_undegraded":"2024-04-24T06:31:01.557267+0000","last_fullsized":"2024-04-24T06:31:01.557267+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:49.551427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868412+0000","last_change":"2024-04-24T06:30:57.825727+0000","last_active":"2024-04-24T06:30:59.868412+0000","last_peered":"2024-04-24T06:30:59.868412+0000","last_clean":"2024-04-24T06:30:59.868412+0000","last_became_active":"2024-04-24T06:30:57.825361+0000","last_became_peered":"2024-04-24T06:30:57.825361+0000","last_unstale":"2024-04-24T06:30:59.868412+0000","last_undegraded":"2024-04-24T06:30:59.868412+0000","last_fullsized":"2024-04-24T06:30:59.868412+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:27:52.822246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872093+0000","last_change":"2024-04-24T06:30:58.866058+0000","last_active":"2024-04-24T06:30:59.872093+0000","last_peered":"2024-04-24T06:30:59.872093+0000","last_clean":"2024-04-24T06:30:59.872093+0000","last_became_active":"2024-04-24T06:30:58.865764+0000","last_became_peered":"2024-04-24T06:30:58.865764+0000","last_unstale":"2024-04-24T06:30:59.872093+0000","last_undegraded":"2024-04-24T06:30:59.872093+0000","last_fullsized":"2024-04-24T06:30:59.872093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:15.893761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872403+0000","last_change":"2024-04-24T06:30:57.838423+0000","last_active":"2024-04-24T06:30:59.872403+0000","last_peered":"2024-04-24T06:30:59.872403+0000","last_clean":"2024-04-24T06:30:59.872403+0000","last_became_active":"2024-04-24T06:30:57.838078+0000","last_became_peered":"2024-04-24T06:30:57.838078+0000","last_unstale":"2024-04-24T06:30:59.872403+0000","last_undegraded":"2024-04-24T06:30:59.872403+0000","last_fullsized":"2024-04-24T06:30:59.872403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:38.399369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719931+0000","last_change":"2024-04-24T06:30:58.868595+0000","last_active":"2024-04-24T06:31:00.719931+0000","last_peered":"2024-04-24T06:31:00.719931+0000","last_clean":"2024-04-24T06:31:00.719931+0000","last_became_active":"2024-04-24T06:30:58.868279+0000","last_became_peered":"2024-04-24T06:30:58.868279+0000","last_unstale":"2024-04-24T06:31:00.719931+0000","last_undegraded":"2024-04-24T06:31:00.719931+0000","last_fullsized":"2024-04-24T06:31:00.719931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:33:56.265248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869978+0000","last_change":"2024-04-24T06:30:57.840907+0000","last_active":"2024-04-24T06:30:59.869978+0000","last_peered":"2024-04-24T06:30:59.869978+0000","last_clean":"2024-04-24T06:30:59.869978+0000","last_became_active":"2024-04-24T06:30:57.840593+0000","last_became_peered":"2024-04-24T06:30:57.840593+0000","last_unstale":"2024-04-24T06:30:59.869978+0000","last_undegraded":"2024-04-24T06:30:59.869978+0000","last_fullsized":"2024-04-24T06:30:59.869978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:04.353231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881162+0000","last_change":"2024-04-24T06:30:58.873121+0000","last_active":"2024-04-24T06:30:59.881162+0000","last_peered":"2024-04-24T06:30:59.881162+0000","last_clean":"2024-04-24T06:30:59.881162+0000","last_became_active":"2024-04-24T06:30:58.870071+0000","last_became_peered":"2024-04-24T06:30:58.870071+0000","last_unstale":"2024-04-24T06:30:59.881162+0000","last_undegraded":"2024-04-24T06:30:59.881162+0000","last_fullsized":"2024-04-24T06:30:59.881162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:03.409884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719885+0000","last_change":"2024-04-24T06:30:57.845222+0000","last_active":"2024-04-24T06:31:00.719885+0000","last_peered":"2024-04-24T06:31:00.719885+0000","last_clean":"2024-04-24T06:31:00.719885+0000","last_became_active":"2024-04-24T06:30:57.843505+0000","last_became_peered":"2024-04-24T06:30:57.843505+0000","last_unstale":"2024-04-24T06:31:00.719885+0000","last_undegraded":"2024-04-24T06:31:00.719885+0000","last_fullsized":"2024-04-24T06:31:00.719885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:00.545236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719573+0000","last_change":"2024-04-24T06:30:58.869239+0000","last_active":"2024-04-24T06:31:00.719573+0000","last_peered":"2024-04-24T06:31:00.719573+0000","last_clean":"2024-04-24T06:31:00.719573+0000","last_became_active":"2024-04-24T06:30:58.868970+0000","last_became_peered":"2024-04-24T06:30:58.868970+0000","last_unstale":"2024-04-24T06:31:00.719573+0000","last_undegraded":"2024-04-24T06:31:00.719573+0000","last_fullsized":"2024-04-24T06:31:00.719573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:41.265011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557204+0000","last_change":"2024-04-24T06:30:57.841598+0000","last_active":"2024-04-24T06:31:01.557204+0000","last_peered":"2024-04-24T06:31:01.557204+0000","last_clean":"2024-04-24T06:31:01.557204+0000","last_became_active":"2024-04-24T06:30:57.841319+0000","last_became_peered":"2024-04-24T06:30:57.841319+0000","last_unstale":"2024-04-24T06:31:01.557204+0000","last_undegraded":"2024-04-24T06:31:01.557204+0000","last_fullsized":"2024-04-24T06:31:01.557204+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:30.873062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557776+0000","last_change":"2024-04-24T06:30:58.860453+0000","last_active":"2024-04-24T06:31:01.557776+0000","last_peered":"2024-04-24T06:31:01.557776+0000","last_clean":"2024-04-24T06:31:01.557776+0000","last_became_active":"2024-04-24T06:30:58.860112+0000","last_became_peered":"2024-04-24T06:30:58.860112+0000","last_unstale":"2024-04-24T06:31:01.557776+0000","last_undegraded":"2024-04-24T06:31:01.557776+0000","last_fullsized":"2024-04-24T06:31:01.557776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:33.839469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880834+0000","last_change":"2024-04-24T06:30:57.831254+0000","last_active":"2024-04-24T06:30:59.880834+0000","last_peered":"2024-04-24T06:30:59.880834+0000","last_clean":"2024-04-24T06:30:59.880834+0000","last_became_active":"2024-04-24T06:30:57.830948+0000","last_became_peered":"2024-04-24T06:30:57.830948+0000","last_unstale":"2024-04-24T06:30:59.880834+0000","last_undegraded":"2024-04-24T06:30:59.880834+0000","last_fullsized":"2024-04-24T06:30:59.880834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:10.418458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869597+0000","last_change":"2024-04-24T06:30:58.859846+0000","last_active":"2024-04-24T06:30:59.869597+0000","last_peered":"2024-04-24T06:30:59.869597+0000","last_clean":"2024-04-24T06:30:59.869597+0000","last_became_active":"2024-04-24T06:30:58.859511+0000","last_became_peered":"2024-04-24T06:30:58.859511+0000","last_unstale":"2024-04-24T06:30:59.869597+0000","last_undegraded":"2024-04-24T06:30:59.869597+0000","last_fullsized":"2024-04-24T06:30:59.869597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:08.619728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720171+0000","last_change":"2024-04-24T06:30:57.844423+0000","last_active":"2024-04-24T06:31:00.720171+0000","last_peered":"2024-04-24T06:31:00.720171+0000","last_clean":"2024-04-24T06:31:00.720171+0000","last_became_active":"2024-04-24T06:30:57.843788+0000","last_became_peered":"2024-04-24T06:30:57.843788+0000","last_unstale":"2024-04-24T06:31:00.720171+0000","last_undegraded":"2024-04-24T06:31:00.720171+0000","last_fullsized":"2024-04-24T06:31:00.720171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:26.018041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880240+0000","last_change":"2024-04-24T06:30:58.868452+0000","last_active":"2024-04-24T06:30:59.880240+0000","last_peered":"2024-04-24T06:30:59.880240+0000","last_clean":"2024-04-24T06:30:59.880240+0000","last_became_active":"2024-04-24T06:30:58.868152+0000","last_became_peered":"2024-04-24T06:30:58.868152+0000","last_unstale":"2024-04-24T06:30:59.880240+0000","last_undegraded":"2024-04-24T06:30:59.880240+0000","last_fullsized":"2024-04-24T06:30:59.880240+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:02.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880460+0000","last_change":"2024-04-24T06:30:57.833067+0000","last_active":"2024-04-24T06:30:59.880460+0000","last_peered":"2024-04-24T06:30:59.880460+0000","last_clean":"2024-04-24T06:30:59.880460+0000","last_became_active":"2024-04-24T06:30:57.832715+0000","last_became_peered":"2024-04-24T06:30:57.832715+0000","last_unstale":"2024-04-24T06:30:59.880460+0000","last_undegraded":"2024-04-24T06:30:59.880460+0000","last_fullsized":"2024-04-24T06:30:59.880460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:18.918955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720017+0000","last_change":"2024-04-24T06:30:58.874248+0000","last_active":"2024-04-24T06:31:00.720017+0000","last_peered":"2024-04-24T06:31:00.720017+0000","last_clean":"2024-04-24T06:31:00.720017+0000","last_became_active":"2024-04-24T06:30:58.873994+0000","last_became_peered":"2024-04-24T06:30:58.873994+0000","last_unstale":"2024-04-24T06:31:00.720017+0000","last_undegraded":"2024-04-24T06:31:00.720017+0000","last_fullsized":"2024-04-24T06:31:00.720017+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:04.234496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869024+0000","last_change":"2024-04-24T06:30:57.830642+0000","last_active":"2024-04-24T06:30:59.869024+0000","last_peered":"2024-04-24T06:30:59.869024+0000","last_clean":"2024-04-24T06:30:59.869024+0000","last_became_active":"2024-04-24T06:30:57.830243+0000","last_became_peered":"2024-04-24T06:30:57.830243+0000","last_unstale":"2024-04-24T06:30:59.869024+0000","last_undegraded":"2024-04-24T06:30:59.869024+0000","last_fullsized":"2024-04-24T06:30:59.869024+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:49.837169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557091+0000","last_change":"2024-04-24T06:30:58.872701+0000","last_active":"2024-04-24T06:31:01.557091+0000","last_peered":"2024-04-24T06:31:01.557091+0000","last_clean":"2024-04-24T06:31:01.557091+0000","last_became_active":"2024-04-24T06:30:58.872363+0000","last_became_peered":"2024-04-24T06:30:58.872363+0000","last_unstale":"2024-04-24T06:31:01.557091+0000","last_undegraded":"2024-04-24T06:31:01.557091+0000","last_fullsized":"2024-04-24T06:31:01.557091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:08.341462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556923+0000","last_change":"2024-04-24T06:30:57.833779+0000","last_active":"2024-04-24T06:31:01.556923+0000","last_peered":"2024-04-24T06:31:01.556923+0000","last_clean":"2024-04-24T06:31:01.556923+0000","last_became_active":"2024-04-24T06:30:57.833537+0000","last_became_peered":"2024-04-24T06:30:57.833537+0000","last_unstale":"2024-04-24T06:31:01.556923+0000","last_undegraded":"2024-04-24T06:31:01.556923+0000","last_fullsized":"2024-04-24T06:31:01.556923+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:45.825857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.870220+0000","last_change":"2024-04-24T06:30:58.862563+0000","last_active":"2024-04-24T06:30:59.870220+0000","last_peered":"2024-04-24T06:30:59.870220+0000","last_clean":"2024-04-24T06:30:59.870220+0000","last_became_active":"2024-04-24T06:30:58.860782+0000","last_became_peered":"2024-04-24T06:30:58.860782+0000","last_unstale":"2024-04-24T06:30:59.870220+0000","last_undegraded":"2024-04-24T06:30:59.870220+0000","last_fullsized":"2024-04-24T06:30:59.870220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:04.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872385+0000","last_change":"2024-04-24T06:30:57.830426+0000","last_active":"2024-04-24T06:30:59.872385+0000","last_peered":"2024-04-24T06:30:59.872385+0000","last_clean":"2024-04-24T06:30:59.872385+0000","last_became_active":"2024-04-24T06:30:57.830061+0000","last_became_peered":"2024-04-24T06:30:57.830061+0000","last_unstale":"2024-04-24T06:30:59.872385+0000","last_undegraded":"2024-04-24T06:30:59.872385+0000","last_fullsized":"2024-04-24T06:30:59.872385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:54.668542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558144+0000","last_change":"2024-04-24T06:30:58.865758+0000","last_active":"2024-04-24T06:31:01.558144+0000","last_peered":"2024-04-24T06:31:01.558144+0000","last_clean":"2024-04-24T06:31:01.558144+0000","last_became_active":"2024-04-24T06:30:58.865490+0000","last_became_peered":"2024-04-24T06:30:58.865490+0000","last_unstale":"2024-04-24T06:31:01.558144+0000","last_undegraded":"2024-04-24T06:31:01.558144+0000","last_fullsized":"2024-04-24T06:31:01.558144+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:47.110910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018916+0000","last_change":"2024-04-24T06:30:57.832486+0000","last_active":"2024-04-24T06:31:00.018916+0000","last_peered":"2024-04-24T06:31:00.018916+0000","last_clean":"2024-04-24T06:31:00.018916+0000","last_became_active":"2024-04-24T06:30:57.832182+0000","last_became_peered":"2024-04-24T06:30:57.832182+0000","last_unstale":"2024-04-24T06:31:00.018916+0000","last_undegraded":"2024-04-24T06:31:00.018916+0000","last_fullsized":"2024-04-24T06:31:00.018916+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558535+0000","last_change":"2024-04-24T06:30:58.870159+0000","last_active":"2024-04-24T06:31:01.558535+0000","last_peered":"2024-04-24T06:31:01.558535+0000","last_clean":"2024-04-24T06:31:01.558535+0000","last_became_active":"2024-04-24T06:30:58.869016+0000","last_became_peered":"2024-04-24T06:30:58.869016+0000","last_unstale":"2024-04-24T06:31:01.558535+0000","last_undegraded":"2024-04-24T06:31:01.558535+0000","last_fullsized":"2024-04-24T06:31:01.558535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:13.666358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881151+0000","last_change":"2024-04-24T06:30:57.842579+0000","last_active":"2024-04-24T06:30:59.881151+0000","last_peered":"2024-04-24T06:30:59.881151+0000","last_clean":"2024-04-24T06:30:59.881151+0000","last_became_active":"2024-04-24T06:30:57.842265+0000","last_became_peered":"2024-04-24T06:30:57.842265+0000","last_unstale":"2024-04-24T06:30:59.881151+0000","last_undegraded":"2024-04-24T06:30:59.881151+0000","last_fullsized":"2024-04-24T06:30:59.881151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:14.888521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880740+0000","last_change":"2024-04-24T06:30:58.869805+0000","last_active":"2024-04-24T06:30:59.880740+0000","last_peered":"2024-04-24T06:30:59.880740+0000","last_clean":"2024-04-24T06:30:59.880740+0000","last_became_active":"2024-04-24T06:30:58.869459+0000","last_became_peered":"2024-04-24T06:30:58.869459+0000","last_unstale":"2024-04-24T06:30:59.880740+0000","last_undegraded":"2024-04-24T06:30:59.880740+0000","last_fullsized":"2024-04-24T06:30:59.880740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:58.620837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880972+0000","last_change":"2024-04-24T06:30:57.840860+0000","last_active":"2024-04-24T06:30:59.880972+0000","last_peered":"2024-04-24T06:30:59.880972+0000","last_clean":"2024-04-24T06:30:59.880972+0000","last_became_active":"2024-04-24T06:30:57.840543+0000","last_became_peered":"2024-04-24T06:30:57.840543+0000","last_unstale":"2024-04-24T06:30:59.880972+0000","last_undegraded":"2024-04-24T06:30:59.880972+0000","last_fullsized":"2024-04-24T06:30:59.880972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:03.465992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822921+0000","last_change":"2024-04-24T06:30:58.870393+0000","last_active":"2024-04-24T06:31:00.822921+0000","last_peered":"2024-04-24T06:31:00.822921+0000","last_clean":"2024-04-24T06:31:00.822921+0000","last_became_active":"2024-04-24T06:30:58.869579+0000","last_became_peered":"2024-04-24T06:30:58.869579+0000","last_unstale":"2024-04-24T06:31:00.822921+0000","last_undegraded":"2024-04-24T06:31:00.822921+0000","last_fullsized":"2024-04-24T06:31:00.822921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:47.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018472+0000","last_change":"2024-04-24T06:30:57.837623+0000","last_active":"2024-04-24T06:31:00.018472+0000","last_peered":"2024-04-24T06:31:00.018472+0000","last_clean":"2024-04-24T06:31:00.018472+0000","last_became_active":"2024-04-24T06:30:57.837021+0000","last_became_peered":"2024-04-24T06:30:57.837021+0000","last_unstale":"2024-04-24T06:31:00.018472+0000","last_undegraded":"2024-04-24T06:31:00.018472+0000","last_fullsized":"2024-04-24T06:31:00.018472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:37.425589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556791+0000","last_change":"2024-04-24T06:30:58.871213+0000","last_active":"2024-04-24T06:31:01.556791+0000","last_peered":"2024-04-24T06:31:01.556791+0000","last_clean":"2024-04-24T06:31:01.556791+0000","last_became_active":"2024-04-24T06:30:58.870919+0000","last_became_peered":"2024-04-24T06:30:58.870919+0000","last_unstale":"2024-04-24T06:31:01.556791+0000","last_undegraded":"2024-04-24T06:31:01.556791+0000","last_fullsized":"2024-04-24T06:31:01.556791+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:05.358470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868824+0000","last_change":"2024-04-24T06:30:57.840879+0000","last_active":"2024-04-24T06:30:59.868824+0000","last_peered":"2024-04-24T06:30:59.868824+0000","last_clean":"2024-04-24T06:30:59.868824+0000","last_became_active":"2024-04-24T06:30:57.840563+0000","last_became_peered":"2024-04-24T06:30:57.840563+0000","last_unstale":"2024-04-24T06:30:59.868824+0000","last_undegraded":"2024-04-24T06:30:59.868824+0000","last_fullsized":"2024-04-24T06:30:59.868824+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:07.048257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868524+0000","last_change":"2024-04-24T06:30:58.865848+0000","last_active":"2024-04-24T06:30:59.868524+0000","last_peered":"2024-04-24T06:30:59.868524+0000","last_clean":"2024-04-24T06:30:59.868524+0000","last_became_active":"2024-04-24T06:30:58.864154+0000","last_became_peered":"2024-04-24T06:30:58.864154+0000","last_unstale":"2024-04-24T06:30:59.868524+0000","last_undegraded":"2024-04-24T06:30:59.868524+0000","last_fullsized":"2024-04-24T06:30:59.868524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:51.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.718797+0000","last_change":"2024-04-24T06:30:57.838469+0000","last_active":"2024-04-24T06:31:00.718797+0000","last_peered":"2024-04-24T06:31:00.718797+0000","last_clean":"2024-04-24T06:31:00.718797+0000","last_became_active":"2024-04-24T06:30:57.837756+0000","last_became_peered":"2024-04-24T06:30:57.837756+0000","last_unstale":"2024-04-24T06:31:00.718797+0000","last_undegraded":"2024-04-24T06:31:00.718797+0000","last_fullsized":"2024-04-24T06:31:00.718797+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:07.614488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558753+0000","last_change":"2024-04-24T06:30:58.866058+0000","last_active":"2024-04-24T06:31:01.558753+0000","last_peered":"2024-04-24T06:31:01.558753+0000","last_clean":"2024-04-24T06:31:01.558753+0000","last_became_active":"2024-04-24T06:30:58.862508+0000","last_became_peered":"2024-04-24T06:30:58.862508+0000","last_unstale":"2024-04-24T06:31:01.558753+0000","last_undegraded":"2024-04-24T06:31:01.558753+0000","last_fullsized":"2024-04-24T06:31:01.558753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:18.940863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881315+0000","last_change":"2024-04-24T06:30:57.841819+0000","last_active":"2024-04-24T06:30:59.881315+0000","last_peered":"2024-04-24T06:30:59.881315+0000","last_clean":"2024-04-24T06:30:59.881315+0000","last_became_active":"2024-04-24T06:30:57.841562+0000","last_became_peered":"2024-04-24T06:30:57.841562+0000","last_unstale":"2024-04-24T06:30:59.881315+0000","last_undegraded":"2024-04-24T06:30:59.881315+0000","last_fullsized":"2024-04-24T06:30:59.881315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:00.159838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869986+0000","last_change":"2024-04-24T06:30:58.862997+0000","last_active":"2024-04-24T06:30:59.869986+0000","last_peered":"2024-04-24T06:30:59.869986+0000","last_clean":"2024-04-24T06:30:59.869986+0000","last_became_active":"2024-04-24T06:30:58.862525+0000","last_became_peered":"2024-04-24T06:30:58.862525+0000","last_unstale":"2024-04-24T06:30:59.869986+0000","last_undegraded":"2024-04-24T06:30:59.869986+0000","last_fullsized":"2024-04-24T06:30:59.869986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:19.924194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720092+0000","last_change":"2024-04-24T06:30:57.844143+0000","last_active":"2024-04-24T06:31:00.720092+0000","last_peered":"2024-04-24T06:31:00.720092+0000","last_clean":"2024-04-24T06:31:00.720092+0000","last_became_active":"2024-04-24T06:30:57.843866+0000","last_became_peered":"2024-04-24T06:30:57.843866+0000","last_unstale":"2024-04-24T06:31:00.720092+0000","last_undegraded":"2024-04-24T06:31:00.720092+0000","last_fullsized":"2024-04-24T06:31:00.720092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:03.622481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719785+0000","last_change":"2024-04-24T06:30:58.880222+0000","last_active":"2024-04-24T06:31:00.719785+0000","last_peered":"2024-04-24T06:31:00.719785+0000","last_clean":"2024-04-24T06:31:00.719785+0000","last_became_active":"2024-04-24T06:30:58.879845+0000","last_became_peered":"2024-04-24T06:30:58.879845+0000","last_unstale":"2024-04-24T06:31:00.719785+0000","last_undegraded":"2024-04-24T06:31:00.719785+0000","last_fullsized":"2024-04-24T06:31:00.719785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:08:05.651612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822255+0000","last_change":"2024-04-24T06:30:57.836689+0000","last_active":"2024-04-24T06:31:00.822255+0000","last_peered":"2024-04-24T06:31:00.822255+0000","last_clean":"2024-04-24T06:31:00.822255+0000","last_became_active":"2024-04-24T06:30:57.836379+0000","last_became_peered":"2024-04-24T06:30:57.836379+0000","last_unstale":"2024-04-24T06:31:00.822255+0000","last_undegraded":"2024-04-24T06:31:00.822255+0000","last_fullsized":"2024-04-24T06:31:00.822255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:38.399369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880649+0000","last_change":"2024-04-24T06:30:58.867889+0000","last_active":"2024-04-24T06:30:59.880649+0000","last_peered":"2024-04-24T06:30:59.880649+0000","last_clean":"2024-04-24T06:30:59.880649+0000","last_became_active":"2024-04-24T06:30:58.867265+0000","last_became_peered":"2024-04-24T06:30:58.867265+0000","last_unstale":"2024-04-24T06:30:59.880649+0000","last_undegraded":"2024-04-24T06:30:59.880649+0000","last_fullsized":"2024-04-24T06:30:59.880649+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:41:17.920781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719653+0000","last_change":"2024-04-24T06:30:57.842064+0000","last_active":"2024-04-24T06:31:00.719653+0000","last_peered":"2024-04-24T06:31:00.719653+0000","last_clean":"2024-04-24T06:31:00.719653+0000","last_became_active":"2024-04-24T06:30:57.841762+0000","last_became_peered":"2024-04-24T06:30:57.841762+0000","last_unstale":"2024-04-24T06:31:00.719653+0000","last_undegraded":"2024-04-24T06:31:00.719653+0000","last_fullsized":"2024-04-24T06:31:00.719653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:00.159838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880908+0000","last_change":"2024-04-24T06:30:58.874050+0000","last_active":"2024-04-24T06:30:59.880908+0000","last_peered":"2024-04-24T06:30:59.880908+0000","last_clean":"2024-04-24T06:30:59.880908+0000","last_became_active":"2024-04-24T06:30:58.873660+0000","last_became_peered":"2024-04-24T06:30:58.873660+0000","last_unstale":"2024-04-24T06:30:59.880908+0000","last_undegraded":"2024-04-24T06:30:59.880908+0000","last_fullsized":"2024-04-24T06:30:59.880908+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:16:26.127812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871308+0000","last_change":"2024-04-24T06:30:57.830985+0000","last_active":"2024-04-24T06:30:59.871308+0000","last_peered":"2024-04-24T06:30:59.871308+0000","last_clean":"2024-04-24T06:30:59.871308+0000","last_became_active":"2024-04-24T06:30:57.830710+0000","last_became_peered":"2024-04-24T06:30:57.830710+0000","last_unstale":"2024-04-24T06:30:59.871308+0000","last_undegraded":"2024-04-24T06:30:59.871308+0000","last_fullsized":"2024-04-24T06:30:59.871308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:46.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018501+0000","last_change":"2024-04-24T06:30:58.870793+0000","last_active":"2024-04-24T06:31:00.018501+0000","last_peered":"2024-04-24T06:31:00.018501+0000","last_clean":"2024-04-24T06:31:00.018501+0000","last_became_active":"2024-04-24T06:30:58.870062+0000","last_became_peered":"2024-04-24T06:30:58.870062+0000","last_unstale":"2024-04-24T06:31:00.018501+0000","last_undegraded":"2024-04-24T06:31:00.018501+0000","last_fullsized":"2024-04-24T06:31:00.018501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:11.423698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822169+0000","last_change":"2024-04-24T06:30:57.834070+0000","last_active":"2024-04-24T06:31:00.822169+0000","last_peered":"2024-04-24T06:31:00.822169+0000","last_clean":"2024-04-24T06:31:00.822169+0000","last_became_active":"2024-04-24T06:30:57.833755+0000","last_became_peered":"2024-04-24T06:30:57.833755+0000","last_unstale":"2024-04-24T06:31:00.822169+0000","last_undegraded":"2024-04-24T06:31:00.822169+0000","last_fullsized":"2024-04-24T06:31:00.822169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:11.789874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871815+0000","last_change":"2024-04-24T06:30:58.857953+0000","last_active":"2024-04-24T06:30:59.871815+0000","last_peered":"2024-04-24T06:30:59.871815+0000","last_clean":"2024-04-24T06:30:59.871815+0000","last_became_active":"2024-04-24T06:30:58.857589+0000","last_became_peered":"2024-04-24T06:30:58.857589+0000","last_unstale":"2024-04-24T06:30:59.871815+0000","last_undegraded":"2024-04-24T06:30:59.871815+0000","last_fullsized":"2024-04-24T06:30:59.871815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:04.627721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868632+0000","last_change":"2024-04-24T06:30:57.828293+0000","last_active":"2024-04-24T06:30:59.868632+0000","last_peered":"2024-04-24T06:30:59.868632+0000","last_clean":"2024-04-24T06:30:59.868632+0000","last_became_active":"2024-04-24T06:30:57.828011+0000","last_became_peered":"2024-04-24T06:30:57.828011+0000","last_unstale":"2024-04-24T06:30:59.868632+0000","last_undegraded":"2024-04-24T06:30:59.868632+0000","last_fullsized":"2024-04-24T06:30:59.868632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:07.336222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869864+0000","last_change":"2024-04-24T06:30:58.866603+0000","last_active":"2024-04-24T06:30:59.869864+0000","last_peered":"2024-04-24T06:30:59.869864+0000","last_clean":"2024-04-24T06:30:59.869864+0000","last_became_active":"2024-04-24T06:30:58.863784+0000","last_became_peered":"2024-04-24T06:30:58.863784+0000","last_unstale":"2024-04-24T06:30:59.869864+0000","last_undegraded":"2024-04-24T06:30:59.869864+0000","last_fullsized":"2024-04-24T06:30:59.869864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:11.423698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019234+0000","last_change":"2024-04-24T06:30:57.844538+0000","last_active":"2024-04-24T06:31:00.019234+0000","last_peered":"2024-04-24T06:31:00.019234+0000","last_clean":"2024-04-24T06:31:00.019234+0000","last_became_active":"2024-04-24T06:30:57.844154+0000","last_became_peered":"2024-04-24T06:30:57.844154+0000","last_unstale":"2024-04-24T06:31:00.019234+0000","last_undegraded":"2024-04-24T06:31:00.019234+0000","last_fullsized":"2024-04-24T06:31:00.019234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.718839+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:31:00.718839+0000","last_peered":"2024-04-24T06:31:00.718839+0000","last_clean":"2024-04-24T06:31:00.718839+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:31:00.718839+0000","last_undegraded":"2024-04-24T06:31:00.718839+0000","last_fullsized":"2024-04-24T06:31:00.718839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":216727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":215753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:31:05.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:31:05.579 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:31:08.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:31:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.651+0000 7f4e43600640 1 Processor -- start 2024-04-24T06:31:08.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.651+0000 7f4e43600640 1 -- start start 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.652+0000 7f4e43600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e3c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.652+0000 7f4e43600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3c15c0a0 con 0x7f4e3c15c730 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.652+0000 7f4e43600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3c0ce7b0 con 0x7f4e3c0cf9a0 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.652+0000 7f4e43600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c0ce570 con 0x7f4e3c0cdbd0 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.653+0000 7f4e40c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4e3c15c730 0x7f4e3c0cbf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48310/0 (socket says 172.21.15.59:48310) 2024-04-24T06:31:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.653+0000 7f4e40c00640 1 -- 172.21.15.59:0/3915061993 learned_addr learned my addr 172.21.15.59:0/3915061993 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:08.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.653+0000 7f4e3be00640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e3c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.653+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297359497 0 0) 0x7f4e3c15c0a0 con 0x7f4e3c15c730 2024-04-24T06:31:08.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.653+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e20003520 con 0x7f4e3c15c730 2024-04-24T06:31:08.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2309177259 0 0) 0x7f4e20003520 con 0x7f4e3c15c730 2024-04-24T06:31:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 msgr2=0x7f4e3c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e3c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 >> v1:172.21.15.66:6789/0 conn(0x7f4e3c0cf9a0 legacy=0x7f4e3c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c0cea90 con 0x7f4e3c15c730 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4e2c001670 con 0x7f4e3c15c730 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 >> v1:172.21.15.59:6789/0 conn(0x7f4e3c15c730 legacy=0x7f4e3c0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e200039e0 0x7f4e20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e20004d40 0x7f4e200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0ce570 con 0x7f4e20004d40 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2000bb50 con 0x7f4e200039e0 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2000bd50 con 0x7f4e3c0cdbd0 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e41600640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e20004d40 0x7f4e200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.654+0000 7f4e40c00640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e200039e0 0x7f4e20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e3be00640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e41600640 1 -- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 msgr2=0x7f4e20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e41600640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c0cdbd0 0x7f4e20004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e41600640 1 -- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e200039e0 msgr2=0x7f4e20003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e41600640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e200039e0 0x7f4e20003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.655+0000 7f4e41600640 1 -- 172.21.15.59:0/3915061993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e20003520 con 0x7f4e20004d40 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.656+0000 7f4e41600640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e20004d40 0x7f4e200051c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001e50 tx=0x7f4e20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1bb8786f6cc0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.656+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34001a70 con 0x7f4e20004d40 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.656+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e34003970 con 0x7f4e20004d40 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.656+0000 7f4e3b400640 1 -- 172.21.15.59:0/3915061993 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003d60 con 0x7f4e20004d40 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 -- 172.21.15.59:0/3915061993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e20004d40 msgr2=0x7f4e200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 --2- 172.21.15.59:0/3915061993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e20004d40 0x7f4e200051c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001e50 tx=0x7f4e20005830 comp rx=0 tx=0).stop 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e41600640 1 -- 172.21.15.59:0/3915061993 reap_dead start 2024-04-24T06:31:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 -- 172.21.15.59:0/3915061993 shutdown_connections 2024-04-24T06:31:08.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 -- 172.21.15.59:0/3915061993 >> 172.21.15.59:0/3915061993 conn(0x7f4e3c073430 msgr2=0x7f4e3c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:08.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 -- 172.21.15.59:0/3915061993 shutdown_connections 2024-04-24T06:31:08.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.657+0000 7f4e43600640 1 -- 172.21.15.59:0/3915061993 wait complete. 2024-04-24T06:31:08.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.659+0000 7f4e43600640 1 Processor -- start 2024-04-24T06:31:08.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.659+0000 7f4e43600640 1 -- start start 2024-04-24T06:31:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c159e80 0x7f4e3c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e3c149e70 0x7f4e3c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0ced70 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0cf510 con 0x7f4e3c149e70 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e43600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c15cb10 con 0x7f4e3c159e80 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47160/0 (socket says 172.21.15.59:47160) 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 -- 172.21.15.59:0/2411802987 learned_addr learned my addr 172.21.15.59:0/2411802987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e40c00640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c159e80 0x7f4e3c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e3be00640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e3c149e70 0x7f4e3c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 -- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c159e80 msgr2=0x7f4e3c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c159e80 0x7f4e3c0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 -- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e3c149e70 msgr2=0x7f4e3c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e3c149e70 0x7f4e3c0c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.660+0000 7f4e41600640 1 -- 172.21.15.59:0/2411802987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c0c9ea0 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e41600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001f40 tx=0x7f4e340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34041070 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e34006d60 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34001a70 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c0ca180 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.661+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c0ca740 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.662+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c0b1a70 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.664+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e34044070 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.664+0000 7f4e38c00640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 0x7f4e1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.665+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f4e340ab060 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.665+0000 7f4e40c00640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 0x7f4e1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.665+0000 7f4e40c00640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 0x7f4e1c06ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c001050 tx=0x7f4e2c001b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.672+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e34079360 con 0x7f4e3c0b10b0 2024-04-24T06:31:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.870+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e3c0c8330 con 0x7f4e1c06c920 2024-04-24T06:31:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.888+0000 7f4e38c00640 1 -- 172.21.15.59:0/2411802987 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f4e3c0c8330 con 0x7f4e1c06c920 2024-04-24T06:31:08.892 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:31:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 msgr2=0x7f4e1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 0x7f4e1c06ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c001050 tx=0x7f4e2c001b60 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 msgr2=0x7f4e3c0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001f40 tx=0x7f4e340014a0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 shutdown_connections 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4e1c06c920 0x7f4e1c06ede0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e3c0b10b0 0x7f4e3c0b1530 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e3c149e70 0x7f4e3c0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 --2- 172.21.15.59:0/2411802987 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4e3c159e80 0x7f4e3c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 >> 172.21.15.59:0/2411802987 conn(0x7f4e3c073430 msgr2=0x7f4e3c157570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.893+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 shutdown_connections 2024-04-24T06:31:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:08.894+0000 7f4e43600640 1 -- 172.21.15.59:0/2411802987 wait complete. 2024-04-24T06:31:08.914 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-24T06:31:07.232042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229864,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744856,"statfs":{"total":773094113280,"available":772858732544,"internally_reserved":0,"allocated":5480448,"data_stored":2906476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.402122"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719659+0000","last_change":"2024-04-24T06:30:58.871689+0000","last_active":"2024-04-24T06:31:00.719659+0000","last_peered":"2024-04-24T06:31:00.719659+0000","last_clean":"2024-04-24T06:31:00.719659+0000","last_became_active":"2024-04-24T06:30:58.871404+0000","last_became_peered":"2024-04-24T06:30:58.871404+0000","last_unstale":"2024-04-24T06:31:00.719659+0000","last_undegraded":"2024-04-24T06:31:00.719659+0000","last_fullsized":"2024-04-24T06:31:00.719659+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:14.318658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019342+0000","last_change":"2024-04-24T06:30:58.867604+0000","last_active":"2024-04-24T06:31:00.019342+0000","last_peered":"2024-04-24T06:31:00.019342+0000","last_clean":"2024-04-24T06:31:00.019342+0000","last_became_active":"2024-04-24T06:30:58.867314+0000","last_became_peered":"2024-04-24T06:30:58.867314+0000","last_unstale":"2024-04-24T06:31:00.019342+0000","last_undegraded":"2024-04-24T06:31:00.019342+0000","last_fullsized":"2024-04-24T06:31:00.019342+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:04.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871079+0000","last_change":"2024-04-24T06:30:58.872202+0000","last_active":"2024-04-24T06:30:59.871079+0000","last_peered":"2024-04-24T06:30:59.871079+0000","last_clean":"2024-04-24T06:30:59.871079+0000","last_became_active":"2024-04-24T06:30:58.871886+0000","last_became_peered":"2024-04-24T06:30:58.871886+0000","last_unstale":"2024-04-24T06:30:59.871079+0000","last_undegraded":"2024-04-24T06:30:59.871079+0000","last_fullsized":"2024-04-24T06:30:59.871079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:01.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869436+0000","last_change":"2024-04-24T06:30:58.862197+0000","last_active":"2024-04-24T06:30:59.869436+0000","last_peered":"2024-04-24T06:30:59.869436+0000","last_clean":"2024-04-24T06:30:59.869436+0000","last_became_active":"2024-04-24T06:30:58.861844+0000","last_became_peered":"2024-04-24T06:30:58.861844+0000","last_unstale":"2024-04-24T06:30:59.869436+0000","last_undegraded":"2024-04-24T06:30:59.869436+0000","last_fullsized":"2024-04-24T06:30:59.869436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:04.627721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018895+0000","last_change":"2024-04-24T06:30:58.860625+0000","last_active":"2024-04-24T06:31:00.018895+0000","last_peered":"2024-04-24T06:31:00.018895+0000","last_clean":"2024-04-24T06:31:00.018895+0000","last_became_active":"2024-04-24T06:30:58.860255+0000","last_became_peered":"2024-04-24T06:30:58.860255+0000","last_unstale":"2024-04-24T06:31:00.018895+0000","last_undegraded":"2024-04-24T06:31:00.018895+0000","last_fullsized":"2024-04-24T06:31:00.018895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:11.926370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880415+0000","last_change":"2024-04-24T06:30:58.864319+0000","last_active":"2024-04-24T06:30:59.880415+0000","last_peered":"2024-04-24T06:30:59.880415+0000","last_clean":"2024-04-24T06:30:59.880415+0000","last_became_active":"2024-04-24T06:30:58.863968+0000","last_became_peered":"2024-04-24T06:30:58.863968+0000","last_unstale":"2024-04-24T06:30:59.880415+0000","last_undegraded":"2024-04-24T06:30:59.880415+0000","last_fullsized":"2024-04-24T06:30:59.880415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:28.258338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719450+0000","last_change":"2024-04-24T06:30:58.870026+0000","last_active":"2024-04-24T06:31:00.719450+0000","last_peered":"2024-04-24T06:31:00.719450+0000","last_clean":"2024-04-24T06:31:00.719450+0000","last_became_active":"2024-04-24T06:30:58.869087+0000","last_became_peered":"2024-04-24T06:30:58.869087+0000","last_unstale":"2024-04-24T06:31:00.719450+0000","last_undegraded":"2024-04-24T06:31:00.719450+0000","last_fullsized":"2024-04-24T06:31:00.719450+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:13.895061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868247+0000","last_change":"2024-04-24T06:30:58.865264+0000","last_active":"2024-04-24T06:30:59.868247+0000","last_peered":"2024-04-24T06:30:59.868247+0000","last_clean":"2024-04-24T06:30:59.868247+0000","last_became_active":"2024-04-24T06:30:58.862289+0000","last_became_peered":"2024-04-24T06:30:58.862289+0000","last_unstale":"2024-04-24T06:30:59.868247+0000","last_undegraded":"2024-04-24T06:30:59.868247+0000","last_fullsized":"2024-04-24T06:30:59.868247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:12.514254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869740+0000","last_change":"2024-04-24T06:30:58.864282+0000","last_active":"2024-04-24T06:30:59.869740+0000","last_peered":"2024-04-24T06:30:59.869740+0000","last_clean":"2024-04-24T06:30:59.869740+0000","last_became_active":"2024-04-24T06:30:58.863882+0000","last_became_peered":"2024-04-24T06:30:58.863882+0000","last_unstale":"2024-04-24T06:30:59.869740+0000","last_undegraded":"2024-04-24T06:30:59.869740+0000","last_fullsized":"2024-04-24T06:30:59.869740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:15.893761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869837+0000","last_change":"2024-04-24T06:30:58.865745+0000","last_active":"2024-04-24T06:30:59.869837+0000","last_peered":"2024-04-24T06:30:59.869837+0000","last_clean":"2024-04-24T06:30:59.869837+0000","last_became_active":"2024-04-24T06:30:58.864835+0000","last_became_peered":"2024-04-24T06:30:58.864835+0000","last_unstale":"2024-04-24T06:30:59.869837+0000","last_undegraded":"2024-04-24T06:30:59.869837+0000","last_fullsized":"2024-04-24T06:30:59.869837+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:12.360600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869888+0000","last_change":"2024-04-24T06:30:58.866632+0000","last_active":"2024-04-24T06:30:59.869888+0000","last_peered":"2024-04-24T06:30:59.869888+0000","last_clean":"2024-04-24T06:30:59.869888+0000","last_became_active":"2024-04-24T06:30:58.865213+0000","last_became_peered":"2024-04-24T06:30:58.865213+0000","last_unstale":"2024-04-24T06:30:59.869888+0000","last_undegraded":"2024-04-24T06:30:59.869888+0000","last_fullsized":"2024-04-24T06:30:59.869888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:11.926370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019108+0000","last_change":"2024-04-24T06:30:58.880299+0000","last_active":"2024-04-24T06:31:00.019108+0000","last_peered":"2024-04-24T06:31:00.019108+0000","last_clean":"2024-04-24T06:31:00.019108+0000","last_became_active":"2024-04-24T06:30:58.879940+0000","last_became_peered":"2024-04-24T06:30:58.879940+0000","last_unstale":"2024-04-24T06:31:00.019108+0000","last_undegraded":"2024-04-24T06:31:00.019108+0000","last_fullsized":"2024-04-24T06:31:00.019108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:19.924194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556943+0000","last_change":"2024-04-24T06:30:58.868071+0000","last_active":"2024-04-24T06:31:01.556943+0000","last_peered":"2024-04-24T06:31:01.556943+0000","last_clean":"2024-04-24T06:31:01.556943+0000","last_became_active":"2024-04-24T06:30:58.867536+0000","last_became_peered":"2024-04-24T06:30:58.867536+0000","last_unstale":"2024-04-24T06:31:01.556943+0000","last_undegraded":"2024-04-24T06:31:01.556943+0000","last_fullsized":"2024-04-24T06:31:01.556943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:50.842409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868439+0000","last_change":"2024-04-24T06:30:58.867069+0000","last_active":"2024-04-24T06:30:59.868439+0000","last_peered":"2024-04-24T06:30:59.868439+0000","last_clean":"2024-04-24T06:30:59.868439+0000","last_became_active":"2024-04-24T06:30:58.862775+0000","last_became_peered":"2024-04-24T06:30:58.862775+0000","last_unstale":"2024-04-24T06:30:59.868439+0000","last_undegraded":"2024-04-24T06:30:59.868439+0000","last_fullsized":"2024-04-24T06:30:59.868439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:01.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880337+0000","last_change":"2024-04-24T06:30:58.865306+0000","last_active":"2024-04-24T06:30:59.880337+0000","last_peered":"2024-04-24T06:30:59.880337+0000","last_clean":"2024-04-24T06:30:59.880337+0000","last_became_active":"2024-04-24T06:30:58.864360+0000","last_became_peered":"2024-04-24T06:30:58.864360+0000","last_unstale":"2024-04-24T06:30:59.880337+0000","last_undegraded":"2024-04-24T06:30:59.880337+0000","last_fullsized":"2024-04-24T06:30:59.880337+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:36.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557401+0000","last_change":"2024-04-24T06:30:58.862904+0000","last_active":"2024-04-24T06:31:01.557401+0000","last_peered":"2024-04-24T06:31:01.557401+0000","last_clean":"2024-04-24T06:31:01.557401+0000","last_became_active":"2024-04-24T06:30:58.862231+0000","last_became_peered":"2024-04-24T06:30:58.862231+0000","last_unstale":"2024-04-24T06:31:01.557401+0000","last_undegraded":"2024-04-24T06:31:01.557401+0000","last_fullsized":"2024-04-24T06:31:01.557401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:12.795113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872244+0000","last_change":"2024-04-24T06:30:58.866711+0000","last_active":"2024-04-24T06:30:59.872244+0000","last_peered":"2024-04-24T06:30:59.872244+0000","last_clean":"2024-04-24T06:30:59.872244+0000","last_became_active":"2024-04-24T06:30:58.864055+0000","last_became_peered":"2024-04-24T06:30:58.864055+0000","last_unstale":"2024-04-24T06:30:59.872244+0000","last_undegraded":"2024-04-24T06:30:59.872244+0000","last_fullsized":"2024-04-24T06:30:59.872244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:12.360600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872070+0000","last_change":"2024-04-24T06:30:58.864767+0000","last_active":"2024-04-24T06:30:59.872070+0000","last_peered":"2024-04-24T06:30:59.872070+0000","last_clean":"2024-04-24T06:30:59.872070+0000","last_became_active":"2024-04-24T06:30:58.864318+0000","last_became_peered":"2024-04-24T06:30:58.864318+0000","last_unstale":"2024-04-24T06:30:59.872070+0000","last_undegraded":"2024-04-24T06:30:59.872070+0000","last_fullsized":"2024-04-24T06:30:59.872070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:01.165078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558749+0000","last_change":"2024-04-24T06:30:58.865342+0000","last_active":"2024-04-24T06:31:01.558749+0000","last_peered":"2024-04-24T06:31:01.558749+0000","last_clean":"2024-04-24T06:31:01.558749+0000","last_became_active":"2024-04-24T06:30:58.865070+0000","last_became_peered":"2024-04-24T06:30:58.865070+0000","last_unstale":"2024-04-24T06:31:01.558749+0000","last_undegraded":"2024-04-24T06:31:01.558749+0000","last_fullsized":"2024-04-24T06:31:01.558749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.430828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557389+0000","last_change":"2024-04-24T06:30:58.866451+0000","last_active":"2024-04-24T06:31:01.557389+0000","last_peered":"2024-04-24T06:31:01.557389+0000","last_clean":"2024-04-24T06:31:01.557389+0000","last_became_active":"2024-04-24T06:30:58.866157+0000","last_became_peered":"2024-04-24T06:30:58.866157+0000","last_unstale":"2024-04-24T06:31:01.557389+0000","last_undegraded":"2024-04-24T06:31:01.557389+0000","last_fullsized":"2024-04-24T06:31:01.557389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:34.588675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880583+0000","last_change":"2024-04-24T06:30:58.876631+0000","last_active":"2024-04-24T06:30:59.880583+0000","last_peered":"2024-04-24T06:30:59.880583+0000","last_clean":"2024-04-24T06:30:59.880583+0000","last_became_active":"2024-04-24T06:30:58.876311+0000","last_became_peered":"2024-04-24T06:30:58.876311+0000","last_unstale":"2024-04-24T06:30:59.880583+0000","last_undegraded":"2024-04-24T06:30:59.880583+0000","last_fullsized":"2024-04-24T06:30:59.880583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:24:57.723346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868721+0000","last_change":"2024-04-24T06:30:58.866407+0000","last_active":"2024-04-24T06:30:59.868721+0000","last_peered":"2024-04-24T06:30:59.868721+0000","last_clean":"2024-04-24T06:30:59.868721+0000","last_became_active":"2024-04-24T06:30:58.865418+0000","last_became_peered":"2024-04-24T06:30:58.865418+0000","last_unstale":"2024-04-24T06:30:59.868721+0000","last_undegraded":"2024-04-24T06:30:59.868721+0000","last_fullsized":"2024-04-24T06:30:59.868721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:27.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557499+0000","last_change":"2024-04-24T06:30:58.864816+0000","last_active":"2024-04-24T06:31:01.557499+0000","last_peered":"2024-04-24T06:31:01.557499+0000","last_clean":"2024-04-24T06:31:01.557499+0000","last_became_active":"2024-04-24T06:30:58.864454+0000","last_became_peered":"2024-04-24T06:30:58.864454+0000","last_unstale":"2024-04-24T06:31:01.557499+0000","last_undegraded":"2024-04-24T06:31:01.557499+0000","last_fullsized":"2024-04-24T06:31:01.557499+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:39.404609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557431+0000","last_change":"2024-04-24T06:30:58.862606+0000","last_active":"2024-04-24T06:31:01.557431+0000","last_peered":"2024-04-24T06:31:01.557431+0000","last_clean":"2024-04-24T06:31:01.557431+0000","last_became_active":"2024-04-24T06:30:58.861854+0000","last_became_peered":"2024-04-24T06:30:58.861854+0000","last_unstale":"2024-04-24T06:31:01.557431+0000","last_undegraded":"2024-04-24T06:31:01.557431+0000","last_fullsized":"2024-04-24T06:31:01.557431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:47.710714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558469+0000","last_change":"2024-04-24T06:30:58.866056+0000","last_active":"2024-04-24T06:31:01.558469+0000","last_peered":"2024-04-24T06:31:01.558469+0000","last_clean":"2024-04-24T06:31:01.558469+0000","last_became_active":"2024-04-24T06:30:58.863311+0000","last_became_peered":"2024-04-24T06:30:58.863311+0000","last_unstale":"2024-04-24T06:31:01.558469+0000","last_undegraded":"2024-04-24T06:31:01.558469+0000","last_fullsized":"2024-04-24T06:31:01.558469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:53.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557043+0000","last_change":"2024-04-24T06:30:58.884986+0000","last_active":"2024-04-24T06:31:01.557043+0000","last_peered":"2024-04-24T06:31:01.557043+0000","last_clean":"2024-04-24T06:31:01.557043+0000","last_became_active":"2024-04-24T06:30:58.884648+0000","last_became_peered":"2024-04-24T06:30:58.884648+0000","last_unstale":"2024-04-24T06:31:01.557043+0000","last_undegraded":"2024-04-24T06:31:01.557043+0000","last_fullsized":"2024-04-24T06:31:01.557043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:51.765763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871901+0000","last_change":"2024-04-24T06:30:58.864078+0000","last_active":"2024-04-24T06:30:59.871901+0000","last_peered":"2024-04-24T06:30:59.871901+0000","last_clean":"2024-04-24T06:30:59.871901+0000","last_became_active":"2024-04-24T06:30:58.862258+0000","last_became_peered":"2024-04-24T06:30:58.862258+0000","last_unstale":"2024-04-24T06:30:59.871901+0000","last_undegraded":"2024-04-24T06:30:59.871901+0000","last_fullsized":"2024-04-24T06:30:59.871901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:08.619728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019627+0000","last_change":"2024-04-24T06:30:58.866076+0000","last_active":"2024-04-24T06:31:00.019627+0000","last_peered":"2024-04-24T06:31:00.019627+0000","last_clean":"2024-04-24T06:31:00.019627+0000","last_became_active":"2024-04-24T06:30:58.865774+0000","last_became_peered":"2024-04-24T06:30:58.865774+0000","last_unstale":"2024-04-24T06:31:00.019627+0000","last_undegraded":"2024-04-24T06:31:00.019627+0000","last_fullsized":"2024-04-24T06:31:00.019627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:22.472936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557999+0000","last_change":"2024-04-24T06:30:58.862627+0000","last_active":"2024-04-24T06:31:01.557999+0000","last_peered":"2024-04-24T06:31:01.557999+0000","last_clean":"2024-04-24T06:31:01.557999+0000","last_became_active":"2024-04-24T06:30:58.862204+0000","last_became_peered":"2024-04-24T06:30:58.862204+0000","last_unstale":"2024-04-24T06:31:01.557999+0000","last_undegraded":"2024-04-24T06:31:01.557999+0000","last_fullsized":"2024-04-24T06:31:01.557999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:55.673782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.870900+0000","last_change":"2024-04-24T06:30:58.864159+0000","last_active":"2024-04-24T06:30:59.870900+0000","last_peered":"2024-04-24T06:30:59.870900+0000","last_clean":"2024-04-24T06:30:59.870900+0000","last_became_active":"2024-04-24T06:30:58.862379+0000","last_became_peered":"2024-04-24T06:30:58.862379+0000","last_unstale":"2024-04-24T06:30:59.870900+0000","last_undegraded":"2024-04-24T06:30:59.870900+0000","last_fullsized":"2024-04-24T06:30:59.870900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:51.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880444+0000","last_change":"2024-04-24T06:30:58.864570+0000","last_active":"2024-04-24T06:30:59.880444+0000","last_peered":"2024-04-24T06:30:59.880444+0000","last_clean":"2024-04-24T06:30:59.880444+0000","last_became_active":"2024-04-24T06:30:58.863132+0000","last_became_peered":"2024-04-24T06:30:58.863132+0000","last_unstale":"2024-04-24T06:30:59.880444+0000","last_undegraded":"2024-04-24T06:30:59.880444+0000","last_fullsized":"2024-04-24T06:30:59.880444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:17.984864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556563+0000","last_change":"2024-04-24T06:30:58.870111+0000","last_active":"2024-04-24T06:31:01.556563+0000","last_peered":"2024-04-24T06:31:01.556563+0000","last_clean":"2024-04-24T06:31:01.556563+0000","last_became_active":"2024-04-24T06:30:58.869818+0000","last_became_peered":"2024-04-24T06:30:58.869818+0000","last_unstale":"2024-04-24T06:31:01.556563+0000","last_undegraded":"2024-04-24T06:31:01.556563+0000","last_fullsized":"2024-04-24T06:31:01.556563+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:27:53.827485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822587+0000","last_change":"2024-04-24T06:30:58.876187+0000","last_active":"2024-04-24T06:31:00.822587+0000","last_peered":"2024-04-24T06:31:00.822587+0000","last_clean":"2024-04-24T06:31:00.822587+0000","last_became_active":"2024-04-24T06:30:58.875909+0000","last_became_peered":"2024-04-24T06:30:58.875909+0000","last_unstale":"2024-04-24T06:31:00.822587+0000","last_undegraded":"2024-04-24T06:31:00.822587+0000","last_fullsized":"2024-04-24T06:31:00.822587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:18.940863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881545+0000","last_change":"2024-04-24T06:30:57.837026+0000","last_active":"2024-04-24T06:30:59.881545+0000","last_peered":"2024-04-24T06:30:59.881545+0000","last_clean":"2024-04-24T06:30:59.881545+0000","last_became_active":"2024-04-24T06:30:57.836706+0000","last_became_peered":"2024-04-24T06:30:57.836706+0000","last_unstale":"2024-04-24T06:30:59.881545+0000","last_undegraded":"2024-04-24T06:30:59.881545+0000","last_fullsized":"2024-04-24T06:30:59.881545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:10.921131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558746+0000","last_change":"2024-04-24T06:30:58.865805+0000","last_active":"2024-04-24T06:31:01.558746+0000","last_peered":"2024-04-24T06:31:01.558746+0000","last_clean":"2024-04-24T06:31:01.558746+0000","last_became_active":"2024-04-24T06:30:58.862885+0000","last_became_peered":"2024-04-24T06:30:58.862885+0000","last_unstale":"2024-04-24T06:31:01.558746+0000","last_undegraded":"2024-04-24T06:31:01.558746+0000","last_fullsized":"2024-04-24T06:31:01.558746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:47.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872357+0000","last_change":"2024-04-24T06:30:57.838444+0000","last_active":"2024-04-24T06:30:59.872357+0000","last_peered":"2024-04-24T06:30:59.872357+0000","last_clean":"2024-04-24T06:30:59.872357+0000","last_became_active":"2024-04-24T06:30:57.838117+0000","last_became_peered":"2024-04-24T06:30:57.838117+0000","last_unstale":"2024-04-24T06:30:59.872357+0000","last_undegraded":"2024-04-24T06:30:59.872357+0000","last_fullsized":"2024-04-24T06:30:59.872357+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:32.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019799+0000","last_change":"2024-04-24T06:30:58.858690+0000","last_active":"2024-04-24T06:31:00.019799+0000","last_peered":"2024-04-24T06:31:00.019799+0000","last_clean":"2024-04-24T06:31:00.019799+0000","last_became_active":"2024-04-24T06:30:58.858273+0000","last_became_peered":"2024-04-24T06:30:58.858273+0000","last_unstale":"2024-04-24T06:31:00.019799+0000","last_undegraded":"2024-04-24T06:31:00.019799+0000","last_fullsized":"2024-04-24T06:31:00.019799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:13.895061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822957+0000","last_change":"2024-04-24T06:30:57.844480+0000","last_active":"2024-04-24T06:31:00.822957+0000","last_peered":"2024-04-24T06:31:00.822957+0000","last_clean":"2024-04-24T06:31:00.822957+0000","last_became_active":"2024-04-24T06:30:57.844131+0000","last_became_peered":"2024-04-24T06:30:57.844131+0000","last_unstale":"2024-04-24T06:31:00.822957+0000","last_undegraded":"2024-04-24T06:31:00.822957+0000","last_fullsized":"2024-04-24T06:31:00.822957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:46.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.823021+0000","last_change":"2024-04-24T06:30:58.876295+0000","last_active":"2024-04-24T06:31:00.823021+0000","last_peered":"2024-04-24T06:31:00.823021+0000","last_clean":"2024-04-24T06:31:00.823021+0000","last_became_active":"2024-04-24T06:30:58.875942+0000","last_became_peered":"2024-04-24T06:30:58.875942+0000","last_unstale":"2024-04-24T06:31:00.823021+0000","last_undegraded":"2024-04-24T06:31:00.823021+0000","last_fullsized":"2024-04-24T06:31:00.823021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:38.430828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719157+0000","last_change":"2024-04-24T06:30:57.845981+0000","last_active":"2024-04-24T06:31:00.719157+0000","last_peered":"2024-04-24T06:31:00.719157+0000","last_clean":"2024-04-24T06:31:00.719157+0000","last_became_active":"2024-04-24T06:30:57.845675+0000","last_became_peered":"2024-04-24T06:30:57.845675+0000","last_unstale":"2024-04-24T06:31:00.719157+0000","last_undegraded":"2024-04-24T06:31:00.719157+0000","last_fullsized":"2024-04-24T06:31:00.719157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:50.961300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871004+0000","last_change":"2024-04-24T06:30:58.874570+0000","last_active":"2024-04-24T06:30:59.871004+0000","last_peered":"2024-04-24T06:30:59.871004+0000","last_clean":"2024-04-24T06:30:59.871004+0000","last_became_active":"2024-04-24T06:30:58.873859+0000","last_became_peered":"2024-04-24T06:30:58.873859+0000","last_unstale":"2024-04-24T06:30:59.871004+0000","last_undegraded":"2024-04-24T06:30:59.871004+0000","last_fullsized":"2024-04-24T06:30:59.871004+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:27.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880723+0000","last_change":"2024-04-24T06:30:57.841400+0000","last_active":"2024-04-24T06:30:59.880723+0000","last_peered":"2024-04-24T06:30:59.880723+0000","last_clean":"2024-04-24T06:30:59.880723+0000","last_became_active":"2024-04-24T06:30:57.841071+0000","last_became_peered":"2024-04-24T06:30:57.841071+0000","last_unstale":"2024-04-24T06:30:59.880723+0000","last_undegraded":"2024-04-24T06:30:59.880723+0000","last_fullsized":"2024-04-24T06:30:59.880723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:11.355360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869654+0000","last_change":"2024-04-24T06:30:58.866709+0000","last_active":"2024-04-24T06:30:59.869654+0000","last_peered":"2024-04-24T06:30:59.869654+0000","last_clean":"2024-04-24T06:30:59.869654+0000","last_became_active":"2024-04-24T06:30:58.866354+0000","last_became_peered":"2024-04-24T06:30:58.866354+0000","last_unstale":"2024-04-24T06:30:59.869654+0000","last_undegraded":"2024-04-24T06:30:59.869654+0000","last_fullsized":"2024-04-24T06:30:59.869654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:01.165078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871213+0000","last_change":"2024-04-24T06:30:57.835137+0000","last_active":"2024-04-24T06:30:59.871213+0000","last_peered":"2024-04-24T06:30:59.871213+0000","last_clean":"2024-04-24T06:30:59.871213+0000","last_became_active":"2024-04-24T06:30:57.834822+0000","last_became_peered":"2024-04-24T06:30:57.834822+0000","last_unstale":"2024-04-24T06:30:59.871213+0000","last_undegraded":"2024-04-24T06:30:59.871213+0000","last_fullsized":"2024-04-24T06:30:59.871213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:11.789874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868257+0000","last_change":"2024-04-24T06:30:58.866295+0000","last_active":"2024-04-24T06:30:59.868257+0000","last_peered":"2024-04-24T06:30:59.868257+0000","last_clean":"2024-04-24T06:30:59.868257+0000","last_became_active":"2024-04-24T06:30:58.865378+0000","last_became_peered":"2024-04-24T06:30:58.865378+0000","last_unstale":"2024-04-24T06:30:59.868257+0000","last_undegraded":"2024-04-24T06:30:59.868257+0000","last_fullsized":"2024-04-24T06:30:59.868257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:43.283999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018323+0000","last_change":"2024-04-24T06:30:57.840100+0000","last_active":"2024-04-24T06:31:00.018323+0000","last_peered":"2024-04-24T06:31:00.018323+0000","last_clean":"2024-04-24T06:31:00.018323+0000","last_became_active":"2024-04-24T06:30:57.839664+0000","last_became_peered":"2024-04-24T06:30:57.839664+0000","last_unstale":"2024-04-24T06:31:00.018323+0000","last_undegraded":"2024-04-24T06:31:00.018323+0000","last_fullsized":"2024-04-24T06:31:00.018323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:11.509015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557267+0000","last_change":"2024-04-24T06:30:58.868481+0000","last_active":"2024-04-24T06:31:01.557267+0000","last_peered":"2024-04-24T06:31:01.557267+0000","last_clean":"2024-04-24T06:31:01.557267+0000","last_became_active":"2024-04-24T06:30:58.866356+0000","last_became_peered":"2024-04-24T06:30:58.866356+0000","last_unstale":"2024-04-24T06:31:01.557267+0000","last_undegraded":"2024-04-24T06:31:01.557267+0000","last_fullsized":"2024-04-24T06:31:01.557267+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:49.551427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868412+0000","last_change":"2024-04-24T06:30:57.825727+0000","last_active":"2024-04-24T06:30:59.868412+0000","last_peered":"2024-04-24T06:30:59.868412+0000","last_clean":"2024-04-24T06:30:59.868412+0000","last_became_active":"2024-04-24T06:30:57.825361+0000","last_became_peered":"2024-04-24T06:30:57.825361+0000","last_unstale":"2024-04-24T06:30:59.868412+0000","last_undegraded":"2024-04-24T06:30:59.868412+0000","last_fullsized":"2024-04-24T06:30:59.868412+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:27:52.822246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872093+0000","last_change":"2024-04-24T06:30:58.866058+0000","last_active":"2024-04-24T06:30:59.872093+0000","last_peered":"2024-04-24T06:30:59.872093+0000","last_clean":"2024-04-24T06:30:59.872093+0000","last_became_active":"2024-04-24T06:30:58.865764+0000","last_became_peered":"2024-04-24T06:30:58.865764+0000","last_unstale":"2024-04-24T06:30:59.872093+0000","last_undegraded":"2024-04-24T06:30:59.872093+0000","last_fullsized":"2024-04-24T06:30:59.872093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:15.893761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872403+0000","last_change":"2024-04-24T06:30:57.838423+0000","last_active":"2024-04-24T06:30:59.872403+0000","last_peered":"2024-04-24T06:30:59.872403+0000","last_clean":"2024-04-24T06:30:59.872403+0000","last_became_active":"2024-04-24T06:30:57.838078+0000","last_became_peered":"2024-04-24T06:30:57.838078+0000","last_unstale":"2024-04-24T06:30:59.872403+0000","last_undegraded":"2024-04-24T06:30:59.872403+0000","last_fullsized":"2024-04-24T06:30:59.872403+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:38.399369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719931+0000","last_change":"2024-04-24T06:30:58.868595+0000","last_active":"2024-04-24T06:31:00.719931+0000","last_peered":"2024-04-24T06:31:00.719931+0000","last_clean":"2024-04-24T06:31:00.719931+0000","last_became_active":"2024-04-24T06:30:58.868279+0000","last_became_peered":"2024-04-24T06:30:58.868279+0000","last_unstale":"2024-04-24T06:31:00.719931+0000","last_undegraded":"2024-04-24T06:31:00.719931+0000","last_fullsized":"2024-04-24T06:31:00.719931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:33:56.265248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869978+0000","last_change":"2024-04-24T06:30:57.840907+0000","last_active":"2024-04-24T06:30:59.869978+0000","last_peered":"2024-04-24T06:30:59.869978+0000","last_clean":"2024-04-24T06:30:59.869978+0000","last_became_active":"2024-04-24T06:30:57.840593+0000","last_became_peered":"2024-04-24T06:30:57.840593+0000","last_unstale":"2024-04-24T06:30:59.869978+0000","last_undegraded":"2024-04-24T06:30:59.869978+0000","last_fullsized":"2024-04-24T06:30:59.869978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:04.353231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881162+0000","last_change":"2024-04-24T06:30:58.873121+0000","last_active":"2024-04-24T06:30:59.881162+0000","last_peered":"2024-04-24T06:30:59.881162+0000","last_clean":"2024-04-24T06:30:59.881162+0000","last_became_active":"2024-04-24T06:30:58.870071+0000","last_became_peered":"2024-04-24T06:30:58.870071+0000","last_unstale":"2024-04-24T06:30:59.881162+0000","last_undegraded":"2024-04-24T06:30:59.881162+0000","last_fullsized":"2024-04-24T06:30:59.881162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:03.409884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719885+0000","last_change":"2024-04-24T06:30:57.845222+0000","last_active":"2024-04-24T06:31:00.719885+0000","last_peered":"2024-04-24T06:31:00.719885+0000","last_clean":"2024-04-24T06:31:00.719885+0000","last_became_active":"2024-04-24T06:30:57.843505+0000","last_became_peered":"2024-04-24T06:30:57.843505+0000","last_unstale":"2024-04-24T06:31:00.719885+0000","last_undegraded":"2024-04-24T06:31:00.719885+0000","last_fullsized":"2024-04-24T06:31:00.719885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:00.545236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719573+0000","last_change":"2024-04-24T06:30:58.869239+0000","last_active":"2024-04-24T06:31:00.719573+0000","last_peered":"2024-04-24T06:31:00.719573+0000","last_clean":"2024-04-24T06:31:00.719573+0000","last_became_active":"2024-04-24T06:30:58.868970+0000","last_became_peered":"2024-04-24T06:30:58.868970+0000","last_unstale":"2024-04-24T06:31:00.719573+0000","last_undegraded":"2024-04-24T06:31:00.719573+0000","last_fullsized":"2024-04-24T06:31:00.719573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:41.265011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557204+0000","last_change":"2024-04-24T06:30:57.841598+0000","last_active":"2024-04-24T06:31:01.557204+0000","last_peered":"2024-04-24T06:31:01.557204+0000","last_clean":"2024-04-24T06:31:01.557204+0000","last_became_active":"2024-04-24T06:30:57.841319+0000","last_became_peered":"2024-04-24T06:30:57.841319+0000","last_unstale":"2024-04-24T06:31:01.557204+0000","last_undegraded":"2024-04-24T06:31:01.557204+0000","last_fullsized":"2024-04-24T06:31:01.557204+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:30.873062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557776+0000","last_change":"2024-04-24T06:30:58.860453+0000","last_active":"2024-04-24T06:31:01.557776+0000","last_peered":"2024-04-24T06:31:01.557776+0000","last_clean":"2024-04-24T06:31:01.557776+0000","last_became_active":"2024-04-24T06:30:58.860112+0000","last_became_peered":"2024-04-24T06:30:58.860112+0000","last_unstale":"2024-04-24T06:31:01.557776+0000","last_undegraded":"2024-04-24T06:31:01.557776+0000","last_fullsized":"2024-04-24T06:31:01.557776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:33.839469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880834+0000","last_change":"2024-04-24T06:30:57.831254+0000","last_active":"2024-04-24T06:30:59.880834+0000","last_peered":"2024-04-24T06:30:59.880834+0000","last_clean":"2024-04-24T06:30:59.880834+0000","last_became_active":"2024-04-24T06:30:57.830948+0000","last_became_peered":"2024-04-24T06:30:57.830948+0000","last_unstale":"2024-04-24T06:30:59.880834+0000","last_undegraded":"2024-04-24T06:30:59.880834+0000","last_fullsized":"2024-04-24T06:30:59.880834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:10.418458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869597+0000","last_change":"2024-04-24T06:30:58.859846+0000","last_active":"2024-04-24T06:30:59.869597+0000","last_peered":"2024-04-24T06:30:59.869597+0000","last_clean":"2024-04-24T06:30:59.869597+0000","last_became_active":"2024-04-24T06:30:58.859511+0000","last_became_peered":"2024-04-24T06:30:58.859511+0000","last_unstale":"2024-04-24T06:30:59.869597+0000","last_undegraded":"2024-04-24T06:30:59.869597+0000","last_fullsized":"2024-04-24T06:30:59.869597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:08.619728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720171+0000","last_change":"2024-04-24T06:30:57.844423+0000","last_active":"2024-04-24T06:31:00.720171+0000","last_peered":"2024-04-24T06:31:00.720171+0000","last_clean":"2024-04-24T06:31:00.720171+0000","last_became_active":"2024-04-24T06:30:57.843788+0000","last_became_peered":"2024-04-24T06:30:57.843788+0000","last_unstale":"2024-04-24T06:31:00.720171+0000","last_undegraded":"2024-04-24T06:31:00.720171+0000","last_fullsized":"2024-04-24T06:31:00.720171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:26.018041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880240+0000","last_change":"2024-04-24T06:30:58.868452+0000","last_active":"2024-04-24T06:30:59.880240+0000","last_peered":"2024-04-24T06:30:59.880240+0000","last_clean":"2024-04-24T06:30:59.880240+0000","last_became_active":"2024-04-24T06:30:58.868152+0000","last_became_peered":"2024-04-24T06:30:58.868152+0000","last_unstale":"2024-04-24T06:30:59.880240+0000","last_undegraded":"2024-04-24T06:30:59.880240+0000","last_fullsized":"2024-04-24T06:30:59.880240+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:02.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880460+0000","last_change":"2024-04-24T06:30:57.833067+0000","last_active":"2024-04-24T06:30:59.880460+0000","last_peered":"2024-04-24T06:30:59.880460+0000","last_clean":"2024-04-24T06:30:59.880460+0000","last_became_active":"2024-04-24T06:30:57.832715+0000","last_became_peered":"2024-04-24T06:30:57.832715+0000","last_unstale":"2024-04-24T06:30:59.880460+0000","last_undegraded":"2024-04-24T06:30:59.880460+0000","last_fullsized":"2024-04-24T06:30:59.880460+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:18.918955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720017+0000","last_change":"2024-04-24T06:30:58.874248+0000","last_active":"2024-04-24T06:31:00.720017+0000","last_peered":"2024-04-24T06:31:00.720017+0000","last_clean":"2024-04-24T06:31:00.720017+0000","last_became_active":"2024-04-24T06:30:58.873994+0000","last_became_peered":"2024-04-24T06:30:58.873994+0000","last_unstale":"2024-04-24T06:31:00.720017+0000","last_undegraded":"2024-04-24T06:31:00.720017+0000","last_fullsized":"2024-04-24T06:31:00.720017+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:04.234496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869024+0000","last_change":"2024-04-24T06:30:57.830642+0000","last_active":"2024-04-24T06:30:59.869024+0000","last_peered":"2024-04-24T06:30:59.869024+0000","last_clean":"2024-04-24T06:30:59.869024+0000","last_became_active":"2024-04-24T06:30:57.830243+0000","last_became_peered":"2024-04-24T06:30:57.830243+0000","last_unstale":"2024-04-24T06:30:59.869024+0000","last_undegraded":"2024-04-24T06:30:59.869024+0000","last_fullsized":"2024-04-24T06:30:59.869024+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:04:49.837169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.557091+0000","last_change":"2024-04-24T06:30:58.872701+0000","last_active":"2024-04-24T06:31:01.557091+0000","last_peered":"2024-04-24T06:31:01.557091+0000","last_clean":"2024-04-24T06:31:01.557091+0000","last_became_active":"2024-04-24T06:30:58.872363+0000","last_became_peered":"2024-04-24T06:30:58.872363+0000","last_unstale":"2024-04-24T06:31:01.557091+0000","last_undegraded":"2024-04-24T06:31:01.557091+0000","last_fullsized":"2024-04-24T06:31:01.557091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:08.341462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556923+0000","last_change":"2024-04-24T06:30:57.833779+0000","last_active":"2024-04-24T06:31:01.556923+0000","last_peered":"2024-04-24T06:31:01.556923+0000","last_clean":"2024-04-24T06:31:01.556923+0000","last_became_active":"2024-04-24T06:30:57.833537+0000","last_became_peered":"2024-04-24T06:30:57.833537+0000","last_unstale":"2024-04-24T06:31:01.556923+0000","last_undegraded":"2024-04-24T06:31:01.556923+0000","last_fullsized":"2024-04-24T06:31:01.556923+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:45.825857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.870220+0000","last_change":"2024-04-24T06:30:58.862563+0000","last_active":"2024-04-24T06:30:59.870220+0000","last_peered":"2024-04-24T06:30:59.870220+0000","last_clean":"2024-04-24T06:30:59.870220+0000","last_became_active":"2024-04-24T06:30:58.860782+0000","last_became_peered":"2024-04-24T06:30:58.860782+0000","last_unstale":"2024-04-24T06:30:59.870220+0000","last_undegraded":"2024-04-24T06:30:59.870220+0000","last_fullsized":"2024-04-24T06:30:59.870220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:04.471231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.872385+0000","last_change":"2024-04-24T06:30:57.830426+0000","last_active":"2024-04-24T06:30:59.872385+0000","last_peered":"2024-04-24T06:30:59.872385+0000","last_clean":"2024-04-24T06:30:59.872385+0000","last_became_active":"2024-04-24T06:30:57.830061+0000","last_became_peered":"2024-04-24T06:30:57.830061+0000","last_unstale":"2024-04-24T06:30:59.872385+0000","last_undegraded":"2024-04-24T06:30:59.872385+0000","last_fullsized":"2024-04-24T06:30:59.872385+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:29:54.668542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558144+0000","last_change":"2024-04-24T06:30:58.865758+0000","last_active":"2024-04-24T06:31:01.558144+0000","last_peered":"2024-04-24T06:31:01.558144+0000","last_clean":"2024-04-24T06:31:01.558144+0000","last_became_active":"2024-04-24T06:30:58.865490+0000","last_became_peered":"2024-04-24T06:30:58.865490+0000","last_unstale":"2024-04-24T06:31:01.558144+0000","last_undegraded":"2024-04-24T06:31:01.558144+0000","last_fullsized":"2024-04-24T06:31:01.558144+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:47.110910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018916+0000","last_change":"2024-04-24T06:30:57.832486+0000","last_active":"2024-04-24T06:31:00.018916+0000","last_peered":"2024-04-24T06:31:00.018916+0000","last_clean":"2024-04-24T06:31:00.018916+0000","last_became_active":"2024-04-24T06:30:57.832182+0000","last_became_peered":"2024-04-24T06:30:57.832182+0000","last_unstale":"2024-04-24T06:31:00.018916+0000","last_undegraded":"2024-04-24T06:31:00.018916+0000","last_fullsized":"2024-04-24T06:31:00.018916+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.278759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558535+0000","last_change":"2024-04-24T06:30:58.870159+0000","last_active":"2024-04-24T06:31:01.558535+0000","last_peered":"2024-04-24T06:31:01.558535+0000","last_clean":"2024-04-24T06:31:01.558535+0000","last_became_active":"2024-04-24T06:30:58.869016+0000","last_became_peered":"2024-04-24T06:30:58.869016+0000","last_unstale":"2024-04-24T06:31:01.558535+0000","last_undegraded":"2024-04-24T06:31:01.558535+0000","last_fullsized":"2024-04-24T06:31:01.558535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:13.666358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881151+0000","last_change":"2024-04-24T06:30:57.842579+0000","last_active":"2024-04-24T06:30:59.881151+0000","last_peered":"2024-04-24T06:30:59.881151+0000","last_clean":"2024-04-24T06:30:59.881151+0000","last_became_active":"2024-04-24T06:30:57.842265+0000","last_became_peered":"2024-04-24T06:30:57.842265+0000","last_unstale":"2024-04-24T06:30:59.881151+0000","last_undegraded":"2024-04-24T06:30:59.881151+0000","last_fullsized":"2024-04-24T06:30:59.881151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:53:14.888521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880740+0000","last_change":"2024-04-24T06:30:58.869805+0000","last_active":"2024-04-24T06:30:59.880740+0000","last_peered":"2024-04-24T06:30:59.880740+0000","last_clean":"2024-04-24T06:30:59.880740+0000","last_became_active":"2024-04-24T06:30:58.869459+0000","last_became_peered":"2024-04-24T06:30:58.869459+0000","last_unstale":"2024-04-24T06:30:59.880740+0000","last_undegraded":"2024-04-24T06:30:59.880740+0000","last_fullsized":"2024-04-24T06:30:59.880740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:58.620837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880972+0000","last_change":"2024-04-24T06:30:57.840860+0000","last_active":"2024-04-24T06:30:59.880972+0000","last_peered":"2024-04-24T06:30:59.880972+0000","last_clean":"2024-04-24T06:30:59.880972+0000","last_became_active":"2024-04-24T06:30:57.840543+0000","last_became_peered":"2024-04-24T06:30:57.840543+0000","last_unstale":"2024-04-24T06:30:59.880972+0000","last_undegraded":"2024-04-24T06:30:59.880972+0000","last_fullsized":"2024-04-24T06:30:59.880972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:04:03.465992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822921+0000","last_change":"2024-04-24T06:30:58.870393+0000","last_active":"2024-04-24T06:31:00.822921+0000","last_peered":"2024-04-24T06:31:00.822921+0000","last_clean":"2024-04-24T06:31:00.822921+0000","last_became_active":"2024-04-24T06:30:58.869579+0000","last_became_peered":"2024-04-24T06:30:58.869579+0000","last_unstale":"2024-04-24T06:31:00.822921+0000","last_undegraded":"2024-04-24T06:31:00.822921+0000","last_fullsized":"2024-04-24T06:31:00.822921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:47.457318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018472+0000","last_change":"2024-04-24T06:30:57.837623+0000","last_active":"2024-04-24T06:31:00.018472+0000","last_peered":"2024-04-24T06:31:00.018472+0000","last_clean":"2024-04-24T06:31:00.018472+0000","last_became_active":"2024-04-24T06:30:57.837021+0000","last_became_peered":"2024-04-24T06:30:57.837021+0000","last_unstale":"2024-04-24T06:31:00.018472+0000","last_undegraded":"2024-04-24T06:31:00.018472+0000","last_fullsized":"2024-04-24T06:31:00.018472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:37.425589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.556791+0000","last_change":"2024-04-24T06:30:58.871213+0000","last_active":"2024-04-24T06:31:01.556791+0000","last_peered":"2024-04-24T06:31:01.556791+0000","last_clean":"2024-04-24T06:31:01.556791+0000","last_became_active":"2024-04-24T06:30:58.870919+0000","last_became_peered":"2024-04-24T06:30:58.870919+0000","last_unstale":"2024-04-24T06:31:01.556791+0000","last_undegraded":"2024-04-24T06:31:01.556791+0000","last_fullsized":"2024-04-24T06:31:01.556791+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:05.358470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868824+0000","last_change":"2024-04-24T06:30:57.840879+0000","last_active":"2024-04-24T06:30:59.868824+0000","last_peered":"2024-04-24T06:30:59.868824+0000","last_clean":"2024-04-24T06:30:59.868824+0000","last_became_active":"2024-04-24T06:30:57.840563+0000","last_became_peered":"2024-04-24T06:30:57.840563+0000","last_unstale":"2024-04-24T06:30:59.868824+0000","last_undegraded":"2024-04-24T06:30:59.868824+0000","last_fullsized":"2024-04-24T06:30:59.868824+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:07.048257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868524+0000","last_change":"2024-04-24T06:30:58.865848+0000","last_active":"2024-04-24T06:30:59.868524+0000","last_peered":"2024-04-24T06:30:59.868524+0000","last_clean":"2024-04-24T06:30:59.868524+0000","last_became_active":"2024-04-24T06:30:58.864154+0000","last_became_peered":"2024-04-24T06:30:58.864154+0000","last_unstale":"2024-04-24T06:30:59.868524+0000","last_undegraded":"2024-04-24T06:30:59.868524+0000","last_fullsized":"2024-04-24T06:30:59.868524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:51.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.718797+0000","last_change":"2024-04-24T06:30:57.838469+0000","last_active":"2024-04-24T06:31:00.718797+0000","last_peered":"2024-04-24T06:31:00.718797+0000","last_clean":"2024-04-24T06:31:00.718797+0000","last_became_active":"2024-04-24T06:30:57.837756+0000","last_became_peered":"2024-04-24T06:30:57.837756+0000","last_unstale":"2024-04-24T06:31:00.718797+0000","last_undegraded":"2024-04-24T06:31:00.718797+0000","last_fullsized":"2024-04-24T06:31:00.718797+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:24:07.614488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:01.558753+0000","last_change":"2024-04-24T06:30:58.866058+0000","last_active":"2024-04-24T06:31:01.558753+0000","last_peered":"2024-04-24T06:31:01.558753+0000","last_clean":"2024-04-24T06:31:01.558753+0000","last_became_active":"2024-04-24T06:30:58.862508+0000","last_became_peered":"2024-04-24T06:30:58.862508+0000","last_unstale":"2024-04-24T06:31:01.558753+0000","last_undegraded":"2024-04-24T06:31:01.558753+0000","last_fullsized":"2024-04-24T06:31:01.558753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:18.940863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.881315+0000","last_change":"2024-04-24T06:30:57.841819+0000","last_active":"2024-04-24T06:30:59.881315+0000","last_peered":"2024-04-24T06:30:59.881315+0000","last_clean":"2024-04-24T06:30:59.881315+0000","last_became_active":"2024-04-24T06:30:57.841562+0000","last_became_peered":"2024-04-24T06:30:57.841562+0000","last_unstale":"2024-04-24T06:30:59.881315+0000","last_undegraded":"2024-04-24T06:30:59.881315+0000","last_fullsized":"2024-04-24T06:30:59.881315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:00.159838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869986+0000","last_change":"2024-04-24T06:30:58.862997+0000","last_active":"2024-04-24T06:30:59.869986+0000","last_peered":"2024-04-24T06:30:59.869986+0000","last_clean":"2024-04-24T06:30:59.869986+0000","last_became_active":"2024-04-24T06:30:58.862525+0000","last_became_peered":"2024-04-24T06:30:58.862525+0000","last_unstale":"2024-04-24T06:30:59.869986+0000","last_undegraded":"2024-04-24T06:30:59.869986+0000","last_fullsized":"2024-04-24T06:30:59.869986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:19.924194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.720092+0000","last_change":"2024-04-24T06:30:57.844143+0000","last_active":"2024-04-24T06:31:00.720092+0000","last_peered":"2024-04-24T06:31:00.720092+0000","last_clean":"2024-04-24T06:31:00.720092+0000","last_became_active":"2024-04-24T06:30:57.843866+0000","last_became_peered":"2024-04-24T06:30:57.843866+0000","last_unstale":"2024-04-24T06:31:00.720092+0000","last_undegraded":"2024-04-24T06:31:00.720092+0000","last_fullsized":"2024-04-24T06:31:00.720092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:03.622481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719785+0000","last_change":"2024-04-24T06:30:58.880222+0000","last_active":"2024-04-24T06:31:00.719785+0000","last_peered":"2024-04-24T06:31:00.719785+0000","last_clean":"2024-04-24T06:31:00.719785+0000","last_became_active":"2024-04-24T06:30:58.879845+0000","last_became_peered":"2024-04-24T06:30:58.879845+0000","last_unstale":"2024-04-24T06:31:00.719785+0000","last_undegraded":"2024-04-24T06:31:00.719785+0000","last_fullsized":"2024-04-24T06:31:00.719785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:08:05.651612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822255+0000","last_change":"2024-04-24T06:30:57.836689+0000","last_active":"2024-04-24T06:31:00.822255+0000","last_peered":"2024-04-24T06:31:00.822255+0000","last_clean":"2024-04-24T06:31:00.822255+0000","last_became_active":"2024-04-24T06:30:57.836379+0000","last_became_peered":"2024-04-24T06:30:57.836379+0000","last_unstale":"2024-04-24T06:31:00.822255+0000","last_undegraded":"2024-04-24T06:31:00.822255+0000","last_fullsized":"2024-04-24T06:31:00.822255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:38.399369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880649+0000","last_change":"2024-04-24T06:30:58.867889+0000","last_active":"2024-04-24T06:30:59.880649+0000","last_peered":"2024-04-24T06:30:59.880649+0000","last_clean":"2024-04-24T06:30:59.880649+0000","last_became_active":"2024-04-24T06:30:58.867265+0000","last_became_peered":"2024-04-24T06:30:58.867265+0000","last_unstale":"2024-04-24T06:30:59.880649+0000","last_undegraded":"2024-04-24T06:30:59.880649+0000","last_fullsized":"2024-04-24T06:30:59.880649+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:41:17.920781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.719653+0000","last_change":"2024-04-24T06:30:57.842064+0000","last_active":"2024-04-24T06:31:00.719653+0000","last_peered":"2024-04-24T06:31:00.719653+0000","last_clean":"2024-04-24T06:31:00.719653+0000","last_became_active":"2024-04-24T06:30:57.841762+0000","last_became_peered":"2024-04-24T06:30:57.841762+0000","last_unstale":"2024-04-24T06:31:00.719653+0000","last_undegraded":"2024-04-24T06:31:00.719653+0000","last_fullsized":"2024-04-24T06:31:00.719653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:00.159838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.880908+0000","last_change":"2024-04-24T06:30:58.874050+0000","last_active":"2024-04-24T06:30:59.880908+0000","last_peered":"2024-04-24T06:30:59.880908+0000","last_clean":"2024-04-24T06:30:59.880908+0000","last_became_active":"2024-04-24T06:30:58.873660+0000","last_became_peered":"2024-04-24T06:30:58.873660+0000","last_unstale":"2024-04-24T06:30:59.880908+0000","last_undegraded":"2024-04-24T06:30:59.880908+0000","last_fullsized":"2024-04-24T06:30:59.880908+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:16:26.127812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871308+0000","last_change":"2024-04-24T06:30:57.830985+0000","last_active":"2024-04-24T06:30:59.871308+0000","last_peered":"2024-04-24T06:30:59.871308+0000","last_clean":"2024-04-24T06:30:59.871308+0000","last_became_active":"2024-04-24T06:30:57.830710+0000","last_became_peered":"2024-04-24T06:30:57.830710+0000","last_unstale":"2024-04-24T06:30:59.871308+0000","last_undegraded":"2024-04-24T06:30:59.871308+0000","last_fullsized":"2024-04-24T06:30:59.871308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:10:46.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.018501+0000","last_change":"2024-04-24T06:30:58.870793+0000","last_active":"2024-04-24T06:31:00.018501+0000","last_peered":"2024-04-24T06:31:00.018501+0000","last_clean":"2024-04-24T06:31:00.018501+0000","last_became_active":"2024-04-24T06:30:58.870062+0000","last_became_peered":"2024-04-24T06:30:58.870062+0000","last_unstale":"2024-04-24T06:31:00.018501+0000","last_undegraded":"2024-04-24T06:31:00.018501+0000","last_fullsized":"2024-04-24T06:31:00.018501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:11.423698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.822169+0000","last_change":"2024-04-24T06:30:57.834070+0000","last_active":"2024-04-24T06:31:00.822169+0000","last_peered":"2024-04-24T06:31:00.822169+0000","last_clean":"2024-04-24T06:31:00.822169+0000","last_became_active":"2024-04-24T06:30:57.833755+0000","last_became_peered":"2024-04-24T06:30:57.833755+0000","last_unstale":"2024-04-24T06:31:00.822169+0000","last_undegraded":"2024-04-24T06:31:00.822169+0000","last_fullsized":"2024-04-24T06:31:00.822169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:11.789874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.871815+0000","last_change":"2024-04-24T06:30:58.857953+0000","last_active":"2024-04-24T06:30:59.871815+0000","last_peered":"2024-04-24T06:30:59.871815+0000","last_clean":"2024-04-24T06:30:59.871815+0000","last_became_active":"2024-04-24T06:30:58.857589+0000","last_became_peered":"2024-04-24T06:30:58.857589+0000","last_unstale":"2024-04-24T06:30:59.871815+0000","last_undegraded":"2024-04-24T06:30:59.871815+0000","last_fullsized":"2024-04-24T06:30:59.871815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:59:04.627721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.868632+0000","last_change":"2024-04-24T06:30:57.828293+0000","last_active":"2024-04-24T06:30:59.868632+0000","last_peered":"2024-04-24T06:30:59.868632+0000","last_clean":"2024-04-24T06:30:59.868632+0000","last_became_active":"2024-04-24T06:30:57.828011+0000","last_became_peered":"2024-04-24T06:30:57.828011+0000","last_unstale":"2024-04-24T06:30:59.868632+0000","last_undegraded":"2024-04-24T06:30:59.868632+0000","last_fullsized":"2024-04-24T06:30:59.868632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:07.336222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:30:59.869864+0000","last_change":"2024-04-24T06:30:58.866603+0000","last_active":"2024-04-24T06:30:59.869864+0000","last_peered":"2024-04-24T06:30:59.869864+0000","last_clean":"2024-04-24T06:30:59.869864+0000","last_became_active":"2024-04-24T06:30:58.863784+0000","last_became_peered":"2024-04-24T06:30:58.863784+0000","last_unstale":"2024-04-24T06:30:59.869864+0000","last_undegraded":"2024-04-24T06:30:59.869864+0000","last_fullsized":"2024-04-24T06:30:59.869864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:57.794876+0000","last_clean_scrub_stamp":"2024-04-24T06:30:57.794876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:30:11.423698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.019234+0000","last_change":"2024-04-24T06:30:57.844538+0000","last_active":"2024-04-24T06:31:00.019234+0000","last_peered":"2024-04-24T06:31:00.019234+0000","last_clean":"2024-04-24T06:31:00.019234+0000","last_became_active":"2024-04-24T06:30:57.844154+0000","last_became_peered":"2024-04-24T06:30:57.844154+0000","last_unstale":"2024-04-24T06:31:00.019234+0000","last_undegraded":"2024-04-24T06:31:00.019234+0000","last_fullsized":"2024-04-24T06:31:00.019234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:30:56.789636+0000","last_clean_scrub_stamp":"2024-04-24T06:30:56.789636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:25.194122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-24T06:31:00.718839+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:31:00.718839+0000","last_peered":"2024-04-24T06:31:00.718839+0000","last_clean":"2024-04-24T06:31:00.718839+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:31:00.718839+0000","last_undegraded":"2024-04-24T06:31:00.718839+0000","last_fullsized":"2024-04-24T06:31:00.718839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":215787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":216727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":215753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":215185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":805517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:31:08.918 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:31:08.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:31:09.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.175+0000 7f9debe00640 1 Processor -- start 2024-04-24T06:31:09.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.175+0000 7f9debe00640 1 -- start start 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9debe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dec0d5170 0x7f9dec0d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9debe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d3490 0x7f9dec0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9debe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0d0470 con 0x7f9dec15e840 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9debe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0d2220 con 0x7f9dec0d5170 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9debe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec0d2ac0 con 0x7f9dec0d3490 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dec0d5170 0x7f9dec0d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9dec0d5170 0x7f9dec0d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36012/0 (socket says 172.21.15.59:36012) 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9deaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9dec15e840 0x7f9dec0cfcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33544/0 (socket says 172.21.15.59:33544) 2024-04-24T06:31:09.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 -- 172.21.15.59:0/2895965276 learned_addr learned my addr 172.21.15.59:0/2895965276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:09.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9deb400640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d3490 0x7f9dec0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 -- 172.21.15.59:0/2895965276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d3490 msgr2=0x7f9dec0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:09.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d3490 0x7f9dec0d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 -- 172.21.15.59:0/2895965276 >> v1:172.21.15.59:6789/0 conn(0x7f9dec15e840 legacy=0x7f9dec0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.177+0000 7f9dea000640 1 -- 172.21.15.59:0/2895965276 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0d2470 con 0x7f9dec0d5170 2024-04-24T06:31:09.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9dea000640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec0d5170 0x7f9dec0d2f50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009870 tx=0x7f9dd402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26132e5ed277eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9de9600640 1 -- 172.21.15.59:0/2895965276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4037070 con 0x7f9dec0d5170 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9de9600640 1 -- 172.21.15.59:0/2895965276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4032440 con 0x7f9dec0d5170 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9de9600640 1 -- 172.21.15.59:0/2895965276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4032810 con 0x7f9dec0d5170 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9debe00640 1 -- 172.21.15.59:0/2895965276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec0d5170 msgr2=0x7f9dec0d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.179+0000 7f9debe00640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec0d5170 0x7f9dec0d2f50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009870 tx=0x7f9dd402a250 comp rx=0 tx=0).stop 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 -- 172.21.15.59:0/2895965276 shutdown_connections 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d3490 0x7f9dec0d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 --2- 172.21.15.59:0/2895965276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec0d5170 0x7f9dec0d2f50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 -- 172.21.15.59:0/2895965276 >> 172.21.15.59:0/2895965276 conn(0x7f9dec08bf90 msgr2=0x7f9dec0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 -- 172.21.15.59:0/2895965276 shutdown_connections 2024-04-24T06:31:09.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.180+0000 7f9debe00640 1 -- 172.21.15.59:0/2895965276 wait complete. 2024-04-24T06:31:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.181+0000 7f9debe00640 1 Processor -- start 2024-04-24T06:31:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.182+0000 7f9debe00640 1 -- start start 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 0x7f9dec0b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec1f4270 0x7f9dec0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0d1580 con 0x7f9dec15e840 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec15ec20 con 0x7f9dec1f4270 2024-04-24T06:31:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9debe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec0d1700 con 0x7f9dec0d5170 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36018/0 (socket says 172.21.15.59:36018) 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 -- 172.21.15.59:0/2942792324 learned_addr learned my addr 172.21.15.59:0/2942792324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 -- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 msgr2=0x7f9dec0b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9deaa00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 0x7f9dec0b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9deb400640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec1f4270 0x7f9dec0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 0x7f9dec0b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 -- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec1f4270 msgr2=0x7f9dec0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec1f4270 0x7f9dec0d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9dea000640 1 -- 172.21.15.59:0/2942792324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0cbc60 con 0x7f9dec15e840 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.183+0000 7f9deaa00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 0x7f9dec0b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9dea000640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4000c00 tx=0x7f9dd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4037040 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4042070 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4030bd0 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec14d750 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.184+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec0cc1e0 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.185+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec04fd90 con 0x7f9dec15e840 2024-04-24T06:31:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.186+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dd4047460 con 0x7f9dec15e840 2024-04-24T06:31:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.187+0000 7f9de3400640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 0x7f9dc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.187+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9dd40ac950 con 0x7f9dec15e840 2024-04-24T06:31:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.188+0000 7f9deaa00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 0x7f9dc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.188+0000 7f9deaa00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 0x7f9dc806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc007fd0 tx=0x7f9ddc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.192+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9dd407ac50 con 0x7f9dec15e840 2024-04-24T06:31:09.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.468+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9dec0d1d40 con 0x7f9dec15e840 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.473+0000 7f9de3400640 1 -- 172.21.15.59:0/2942792324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f9dd407a550 con 0x7f9dec15e840 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:31:09.475 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:31:09.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 msgr2=0x7f9dc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 0x7f9dc806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc007fd0 tx=0x7f9ddc009040 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 msgr2=0x7f9dec0cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4000c00 tx=0x7f9dd40046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 shutdown_connections 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9dc806c8f0 0x7f9dc806edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dec1f4270 0x7f9dec0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9dec15e840 0x7f9dec0cb720 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 --2- 172.21.15.59:0/2942792324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9dec0d5170 0x7f9dec0b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.476+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 >> 172.21.15.59:0/2942792324 conn(0x7f9dec08bf90 msgr2=0x7f9dec15ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.477+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 shutdown_connections 2024-04-24T06:31:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.477+0000 7f9debe00640 1 -- 172.21.15.59:0/2942792324 wait complete. 2024-04-24T06:31:09.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:09.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.709+0000 7f9e5c400640 1 Processor -- start 2024-04-24T06:31:09.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.710+0000 7f9e5c400640 1 -- start start 2024-04-24T06:31:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.711+0000 7f9e5c400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 0x7f9e54145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.711+0000 7f9e5c400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9e540cd020 0x7f9e540b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.711+0000 7f9e5c400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e540cd520 con 0x7f9e540b3e40 2024-04-24T06:31:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.711+0000 7f9e5c400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9e540cddc0 con 0x7f9e540cd020 2024-04-24T06:31:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.711+0000 7f9e5c400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e540cd800 con 0x7f9e541454e0 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e5a400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9e540b3e40 0x7f9e54160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33558/0 (socket says 172.21.15.59:33558) 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e5a400640 1 -- 172.21.15.59:0/2461597049 learned_addr learned my addr 172.21.15.59:0/2461597049 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e59000640 1 --2- 172.21.15.59:0/2461597049 >> v2:172.21.15.66:3300/0 conn(0x7f9e540cd020 0x7f9e540b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e59a00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 0x7f9e54145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975335775 0 0) 0x7f9e540cd520 con 0x7f9e540b3e40 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.712+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e3c003520 con 0x7f9e540b3e40 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1105458793 0 0) 0x7f9e3c003520 con 0x7f9e540b3e40 2024-04-24T06:31:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 msgr2=0x7f9e54145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 0x7f9e54145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 >> v2:172.21.15.66:3300/0 conn(0x7f9e540cd020 msgr2=0x7f9e540b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 --2- 172.21.15.59:0/2461597049 >> v2:172.21.15.66:3300/0 conn(0x7f9e540cd020 0x7f9e540b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e540cdae0 con 0x7f9e540b3e40 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9e4c0019f0 con 0x7f9e540b3e40 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 >> v1:172.21.15.59:6789/0 conn(0x7f9e540b3e40 legacy=0x7f9e54160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e3c004800 0x7f9e3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 0x7f9e3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e540cddc0 con 0x7f9e3c0039b0 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e540cd800 con 0x7f9e3c004800 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3c00b9c0 con 0x7f9e541454e0 2024-04-24T06:31:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.713+0000 7f9e59a00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.714+0000 7f9e59a00640 1 -- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 msgr2=0x7f9e3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.714+0000 7f9e59a00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e541454e0 0x7f9e3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.714+0000 7f9e59a00640 1 -- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e3c004800 msgr2=0x7f9e3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.714+0000 7f9e59a00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e3c004800 0x7f9e3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.714+0000 7f9e59a00640 1 -- 172.21.15.59:0/2461597049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c003520 con 0x7f9e3c0039b0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.715+0000 7f9e59a00640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009af0 tx=0x7f9e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04b9e1407c29f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.715+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48036070 con 0x7f9e3c0039b0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.715+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48031400 con 0x7f9e3c0039b0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.715+0000 7f9e53e00640 1 -- 172.21.15.59:0/2461597049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48009690 con 0x7f9e3c0039b0 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 -- 172.21.15.59:0/2461597049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 msgr2=0x7f9e3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 --2- 172.21.15.59:0/2461597049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e3c003dd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009af0 tx=0x7f9e3c005830 comp rx=0 tx=0).stop 2024-04-24T06:31:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5a400640 1 -- 172.21.15.59:0/2461597049 reap_dead start 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 -- 172.21.15.59:0/2461597049 shutdown_connections 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 -- 172.21.15.59:0/2461597049 >> 172.21.15.59:0/2461597049 conn(0x7f9e54067ea0 msgr2=0x7f9e5406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 -- 172.21.15.59:0/2461597049 shutdown_connections 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.716+0000 7f9e5c400640 1 -- 172.21.15.59:0/2461597049 wait complete. 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.718+0000 7f9e5c400640 1 Processor -- start 2024-04-24T06:31:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.718+0000 7f9e5c400640 1 -- start start 2024-04-24T06:31:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e5415d410 0x7f9e5406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e54144070 0x7f9e54144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541611a0 con 0x7f9e3c0039b0 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54161020 con 0x7f9e54144070 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5c400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e54145bb0 con 0x7f9e5415d410 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36038/0 (socket says 172.21.15.59:36038) 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 -- 172.21.15.59:0/3748461673 learned_addr learned my addr 172.21.15.59:0/3748461673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59000640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e5415d410 0x7f9e5406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e5a400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e54144070 0x7f9e54144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 -- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e5415d410 msgr2=0x7f9e5406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e5415d410 0x7f9e5406f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 -- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e54144070 msgr2=0x7f9e54144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e54144070 0x7f9e54144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.719+0000 7f9e59a00640 1 -- 172.21.15.59:0/3748461673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5406e030 con 0x7f9e3c0039b0 2024-04-24T06:31:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e59a00640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9e48034040 tx=0x7f9e48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48041020 con 0x7f9e3c0039b0 2024-04-24T06:31:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4802ade0 con 0x7f9e3c0039b0 2024-04-24T06:31:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48036040 con 0x7f9e3c0039b0 2024-04-24T06:31:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5406e310 con 0x7f9e3c0039b0 2024-04-24T06:31:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.720+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5406e8d0 con 0x7f9e3c0039b0 2024-04-24T06:31:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.722+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e54068a80 con 0x7f9e3c0039b0 2024-04-24T06:31:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.723+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e48044020 con 0x7f9e3c0039b0 2024-04-24T06:31:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.724+0000 7f9e51600640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.724+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9e480ab830 con 0x7f9e3c0039b0 2024-04-24T06:31:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.724+0000 7f9e59000640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.725+0000 7f9e59000640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 0x7f9e3806ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e540b1bd0 tx=0x7f9e44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:09.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.729+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e48079c50 con 0x7f9e3c0039b0 2024-04-24T06:31:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.977+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e54160030 con 0x7f9e3c0039b0 2024-04-24T06:31:09.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.979+0000 7f9e51600640 1 -- 172.21.15.59:0/3748461673 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f9e48079550 con 0x7f9e3c0039b0 2024-04-24T06:31:09.980 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:09.980 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.981+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 msgr2=0x7f9e3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.981+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 0x7f9e3806ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e540b1bd0 tx=0x7f9e44008040 comp rx=0 tx=0).stop 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 msgr2=0x7f9e5415c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9e48034040 tx=0x7f9e48004730 comp rx=0 tx=0).stop 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 shutdown_connections 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e3806c920 0x7f9e3806ede0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e54144070 0x7f9e54144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e5415d410 0x7f9e5406f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 --2- 172.21.15.59:0/3748461673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e3c0039b0 0x7f9e5415c050 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 >> 172.21.15.59:0/3748461673 conn(0x7f9e54067ea0 msgr2=0x7f9e540ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 shutdown_connections 2024-04-24T06:31:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:09.982+0000 7f9e5c400640 1 -- 172.21.15.59:0/3748461673 wait complete. 2024-04-24T06:31:10.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:30:58.292001+0000', 'modified': '2024-04-24T06:31:01.372687+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5706}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5706': {'gid': 5706, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/1331520600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 1331520600}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 1331520600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5706, 'qdb_cluster': [5706]} 2024-04-24T06:31:10.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:31:10.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:31:10.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.633+0000 7f744ce00640 1 Processor -- start 2024-04-24T06:31:10.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.633+0000 7f744ce00640 1 -- start start 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f744ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74480d0f80 0x7f74480cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f744ce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f744815f880 0x7f74480d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f744ce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744815fd30 con 0x7f744815ea40 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f744ce00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f74480cf930 con 0x7f74480d0f80 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f744ce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f74480cfc10 con 0x7f744815f880 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f7447400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74480d0f80 0x7f74480cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f7447400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74480d0f80 0x7f74480cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36048/0 (socket says 172.21.15.59:36048) 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.635+0000 7f7447400640 1 -- 172.21.15.59:0/569538350 learned_addr learned my addr 172.21.15.59:0/569538350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.636+0000 7f7447e00640 1 --2- 172.21.15.59:0/569538350 >> v2:172.21.15.66:3300/0 conn(0x7f744815f880 0x7f74480d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.636+0000 7f7447400640 1 -- 172.21.15.59:0/569538350 >> v2:172.21.15.66:3300/0 conn(0x7f744815f880 msgr2=0x7f74480d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.636+0000 7f7447400640 1 --2- 172.21.15.59:0/569538350 >> v2:172.21.15.66:3300/0 conn(0x7f744815f880 0x7f74480d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.636+0000 7f7447400640 1 -- 172.21.15.59:0/569538350 >> v1:172.21.15.66:6789/0 conn(0x7f744815ea40 legacy=0x7f74480d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.636+0000 7f7447400640 1 -- 172.21.15.59:0/569538350 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74480cfef0 con 0x7f74480d0f80 2024-04-24T06:31:10.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.637+0000 7f7447400640 1 --2- 172.21.15.59:0/569538350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480cf250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f74480ceb70 tx=0x7f743802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39edc55eeabb692e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.637+0000 7f7446000640 1 -- 172.21.15.59:0/569538350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438037070 con 0x7f74480d0f80 2024-04-24T06:31:10.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.637+0000 7f7446000640 1 -- 172.21.15.59:0/569538350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438032440 con 0x7f74480d0f80 2024-04-24T06:31:10.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.637+0000 7f7446000640 1 -- 172.21.15.59:0/569538350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438032810 con 0x7f74480d0f80 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 -- 172.21.15.59:0/569538350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 msgr2=0x7f74480cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 --2- 172.21.15.59:0/569538350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480cf250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f74480ceb70 tx=0x7f743802a1a0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 -- 172.21.15.59:0/569538350 shutdown_connections 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 --2- 172.21.15.59:0/569538350 >> v2:172.21.15.66:3300/0 conn(0x7f744815f880 0x7f74480d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 --2- 172.21.15.59:0/569538350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480cf250 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 -- 172.21.15.59:0/569538350 >> 172.21.15.59:0/569538350 conn(0x7f744808bf90 msgr2=0x7f74480d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 -- 172.21.15.59:0/569538350 shutdown_connections 2024-04-24T06:31:10.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.638+0000 7f744ce00640 1 -- 172.21.15.59:0/569538350 wait complete. 2024-04-24T06:31:10.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.640+0000 7f744ce00640 1 Processor -- start 2024-04-24T06:31:10.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.640+0000 7f744ce00640 1 -- start start 2024-04-24T06:31:10.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 0x7f74480cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f744815f880 0x7f74480b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74480d2850 con 0x7f74480d0f80 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74480d04b0 con 0x7f744815f880 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f744ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f74480d29d0 con 0x7f744815ea40 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36052/0 (socket says 172.21.15.59:36052) 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f7447400640 1 -- 172.21.15.59:0/257908006 learned_addr learned my addr 172.21.15.59:0/257908006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f7447400640 1 -- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 msgr2=0x7f74480cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.641+0000 7f7446a00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 0x7f74480cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447e00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f744815f880 0x7f74480b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447400640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 0x7f74480cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447400640 1 -- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f744815f880 msgr2=0x7f74480b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447400640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f744815f880 0x7f74480b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447400640 1 -- 172.21.15.59:0/257908006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481f8340 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7446a00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 0x7f74480cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f7447400640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7438000c00 tx=0x7f74380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f743f400640 1 -- 172.21.15.59:0/257908006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438041070 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f743f400640 1 -- 172.21.15.59:0/257908006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743803c870 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f743f400640 1 -- 172.21.15.59:0/257908006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438037040 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.642+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74481f8620 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.643+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74481f8be0 con 0x7f74480d0f80 2024-04-24T06:31:10.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.645+0000 7f743f400640 1 -- 172.21.15.59:0/257908006 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7438044070 con 0x7f74480d0f80 2024-04-24T06:31:10.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.646+0000 7f743f400640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 0x7f742806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.646+0000 7f743f400640 1 -- 172.21.15.59:0/257908006 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f74380ac8e0 con 0x7f74480d0f80 2024-04-24T06:31:10.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.646+0000 7f7446a00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 0x7f742806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.647+0000 7f745118b740 1 Processor -- start 2024-04-24T06:31:10.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.647+0000 7f7446a00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 0x7f742806ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7434007fd0 tx=0x7f7434009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.647+0000 7f745118b740 1 -- start start 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a64b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56027a5b5b10 con 0x56027a67b360 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56027a55c2b0 con 0x56027a698980 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56027a5b1ad0 con 0x56027a67baa0 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36058/0 (socket says 172.21.15.59:36058) 2024-04-24T06:31:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 -- 172.21.15.59:0/2850682574 learned_addr learned my addr 172.21.15.59:0/2850682574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7446a00640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447400640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a64b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 -- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 msgr2=0x56027a67ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 -- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 msgr2=0x56027a64b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a64b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.648+0000 7f7447e00640 1 -- 172.21.15.59:0/2850682574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027a2c82d0 con 0x56027a67b360 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.649+0000 7f7446a00640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.649+0000 7f7447e00640 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7440008da0 tx=0x7f744000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd023ccb87c52883 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.649+0000 7f7433e00640 1 -- 172.21.15.59:0/2850682574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440015020 con 0x56027a67b360 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.649+0000 7f7433e00640 1 -- 172.21.15.59:0/2850682574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440003990 con 0x56027a67b360 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.649+0000 7f7433e00640 1 -- 172.21.15.59:0/2850682574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440003d60 con 0x56027a67b360 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 -- 172.21.15.59:0/2850682574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 msgr2=0x56027a650170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7440008da0 tx=0x7f744000d6b0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 -- 172.21.15.59:0/2850682574 shutdown_connections 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a67b360 0x56027a650170 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 --2- 172.21.15.59:0/2850682574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a64b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 -- 172.21.15.59:0/2850682574 >> 172.21.15.59:0/2850682574 conn(0x56027a64d580 msgr2=0x56027a695e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 -- 172.21.15.59:0/2850682574 shutdown_connections 2024-04-24T06:31:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.650+0000 7f745118b740 1 -- 172.21.15.59:0/2850682574 wait complete. 2024-04-24T06:31:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.651+0000 7f745118b740 1 -- start start 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.651+0000 7f745118b740 1 Processor -- start 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a676980 0x56027a679d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56027a64f730 con 0x56027a698980 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56027a56f590 con 0x56027a676980 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56027a64eb20 con 0x56027a67baa0 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f745118b740 10 client.0 _ll_register_callbacks cb 0x56027a5523a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f7446a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f7446a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36072/0 (socket says 172.21.15.59:36072) 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.652+0000 7f7446a00640 1 -- 172.21.15.59:0/494458822 learned_addr learned my addr 172.21.15.59:0/494458822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7447e00640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a676980 0x56027a679d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7447400640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 -- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 msgr2=0x56027a67ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 -- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a676980 msgr2=0x56027a679d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a676980 0x56027a679d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 -- 172.21.15.59:0/494458822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56027a3926b0 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7446a00640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f74480741c0 tx=0x7f7434000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4354efdfed8292c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7430c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434021430 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7434020a80 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.653+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7440003990 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7434020c80 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 10 client.5880 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56027a67a790 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440003990 con 0x56027a698980 2024-04-24T06:31:10.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f7434020420 con 0x56027a698980 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 10 client.5880 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 10 client.5880 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 20 client.5880 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a67e660 0x56027a6b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 4 client.5880 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.654+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56027a5c3530 con 0x56027a67e660 2024-04-24T06:31:10.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.655+0000 7f7447400640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a67e660 0x56027a6b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.656+0000 7f7447400640 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a67e660 0x56027a6b4050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c51054a052a30b server_cookie=eff0354146b3b1b0 in_seq=0 out_seq=0 2024-04-24T06:31:10.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.656+0000 7f7430c00640 10 client.5880 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.659+0000 7f7430c00640 1 -- 172.21.15.59:0/494458822 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56027a5c3530 con 0x56027a67e660 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.659+0000 7f7430c00640 10 client.5880 handle_command_reply: tid=0 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.659+0000 7f745118b740 1 client.5880 shutdown 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 msgr2=0x56027a6a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f74480741c0 tx=0x7f7434000fb0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 shutdown_connections 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a676980 0x56027a679d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a698980 0x56027a6a5f10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a67e660 0x56027a6b4050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 --2- 172.21.15.59:0/494458822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a67ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 >> 172.21.15.59:0/494458822 conn(0x56027a67b360 msgr2=0x56027a696f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f7430c00640 0 client.5880 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 shutdown_connections 2024-04-24T06:31:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 1 -- 172.21.15.59:0/494458822 wait complete. 2024-04-24T06:31:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.660+0000 7f745118b740 20 client.5880 trim_cache size 0 max 16384 2024-04-24T06:31:10.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.680+0000 7f745118b740 1 Processor -- start 2024-04-24T06:31:10.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.680+0000 7f745118b740 1 -- start start 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a67baa0 0x56027a64b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a698980 0x56027a6a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74481fffe0 con 0x56027a676980 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74481ff370 con 0x56027a67baa0 2024-04-24T06:31:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.681+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f74481ff990 con 0x56027a698980 2024-04-24T06:31:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36088/0 (socket says 172.21.15.59:36088) 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 -- 172.21.15.59:0/889943215 learned_addr learned my addr 172.21.15.59:0/889943215 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7446a00640 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a67baa0 0x56027a64b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447e00640 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a698980 0x56027a6a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 -- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a698980 msgr2=0x56027a6a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a698980 0x56027a6a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 -- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a67baa0 msgr2=0x56027a64b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a67baa0 0x56027a64b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 -- 172.21.15.59:0/889943215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027a64e6c0 con 0x56027a676980 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.682+0000 7f7447400640 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f743807a620 tx=0x7f7438081040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242110c15d265d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f7433400640 1 -- 172.21.15.59:0/889943215 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743804b040 con 0x56027a676980 2024-04-24T06:31:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f7433400640 1 -- 172.21.15.59:0/889943215 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743807ea40 con 0x56027a676980 2024-04-24T06:31:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f7433400640 1 -- 172.21.15.59:0/889943215 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438083270 con 0x56027a676980 2024-04-24T06:31:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 -- 172.21.15.59:0/889943215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 msgr2=0x56027a679d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f743807a620 tx=0x7f7438081040 comp rx=0 tx=0).stop 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 -- 172.21.15.59:0/889943215 shutdown_connections 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a698980 0x56027a6a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a67baa0 0x56027a64b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 --2- 172.21.15.59:0/889943215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a679d60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.683+0000 7f745118b740 1 -- 172.21.15.59:0/889943215 >> 172.21.15.59:0/889943215 conn(0x56027a67b360 msgr2=0x56027a650b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.684+0000 7f745118b740 1 -- 172.21.15.59:0/889943215 shutdown_connections 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.684+0000 7f745118b740 1 -- 172.21.15.59:0/889943215 wait complete. 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.684+0000 7f745118b740 1 -- start start 2024-04-24T06:31:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.684+0000 7f745118b740 1 Processor -- start 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a6b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a697780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56027a67e4e0 con 0x56027a676980 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56027a67e040 con 0x56027a698980 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56027a67eb00 con 0x56027a67baa0 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.685+0000 7f745118b740 10 client.0 _ll_register_callbacks cb 0x7f743400d9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36092/0 (socket says 172.21.15.59:36092) 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 -- 172.21.15.59:0/2885651709 learned_addr learned my addr 172.21.15.59:0/2885651709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 -- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 msgr2=0x56027a6b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a6b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 -- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 msgr2=0x56027a697780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a697780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 -- 172.21.15.59:0/2885651709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56027a676560 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7447400640 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f743802a1e0 tx=0x7f7434011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6c64c822e9d772 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.686+0000 7f7430c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743804b030 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f743807a8f0 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f743807ea40 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f74380aa050 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 10 client.5886 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56027a67cc00 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743807ea40 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f743807fbd0 con 0x56027a676980 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 10 client.5886 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 10 client.5886 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:31:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 20 client.5886 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a695650 0x56027a695a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 4 client.5886 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:31:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.687+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "status"}) -- 0x56027a5bd3a0 con 0x56027a695650 2024-04-24T06:31:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.688+0000 7f7446a00640 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a695650 0x56027a695a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.689+0000 7f7446a00640 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a695650 0x56027a695a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ac85f4e46d78ca server_cookie=b176528dd8754a45 in_seq=0 out_seq=0 2024-04-24T06:31:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.689+0000 7f7430c00640 10 client.5886 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.689+0000 7f7430c00640 1 -- 172.21.15.59:0/2885651709 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56027a5bd3a0 con 0x56027a695650 2024-04-24T06:31:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.689+0000 7f7430c00640 10 client.5886 handle_command_reply: tid=0 2024-04-24T06:31:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 client.5886 shutdown 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 msgr2=0x56027a64b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f743802a1e0 tx=0x7f7434011000 comp rx=0 tx=0).stop 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 shutdown_connections 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56027a698980 0x56027a697780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56027a695650 0x56027a695a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56027a67baa0 0x56027a6b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 --2- 172.21.15.59:0/2885651709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56027a676980 0x56027a64b2d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 >> 172.21.15.59:0/2885651709 conn(0x56027a67b360 msgr2=0x56027a6a57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f7430c00640 0 client.5886 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 shutdown_connections 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 1 -- 172.21.15.59:0/2885651709 wait complete. 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.690+0000 7f745118b740 20 client.5886 trim_cache size 0 max 16384 2024-04-24T06:31:10.693 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5706, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 11.830271098000001, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 45, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 45, 2024-04-24T06:31:10.694 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 18.247362713000001 2024-04-24T06:31:10.695 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:10.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 msgr2=0x7f742806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 0x7f742806ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7434007fd0 tx=0x7f7434009040 comp rx=0 tx=0).stop 2024-04-24T06:31:10.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 msgr2=0x7f74480b9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7438000c00 tx=0x7f74380046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 shutdown_connections 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f742806c920 0x7f742806ede0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f744815f880 0x7f74480b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f744815ea40 0x7f74480cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 --2- 172.21.15.59:0/257908006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74480d0f80 0x7f74480b9c20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 >> 172.21.15.59:0/257908006 conn(0x7f744808bf90 msgr2=0x7f74481617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 shutdown_connections 2024-04-24T06:31:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.691+0000 7f744ce00640 1 -- 172.21.15.59:0/257908006 wait complete. 2024-04-24T06:31:10.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:31:10.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.902+0000 7fe0c2200640 1 Processor -- start 2024-04-24T06:31:10.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.903+0000 7fe0c2200640 1 -- start start 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0c2200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0bc0cd760 0x7fe0bc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0c2200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0bc160210 con 0x7fe0bc0b4690 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0c2200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0bc1604f0 con 0x7fe0bc15c3d0 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0c2200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe0bc1607d0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0b3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0bc0cd760 0x7fe0bc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0b3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0bc0cd760 0x7fe0bc0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36098/0 (socket says 172.21.15.59:36098) 2024-04-24T06:31:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.904+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1161147837 learned_addr learned my addr 172.21.15.59:0/1161147837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.905+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1161147837 >> v1:172.21.15.66:6789/0 conn(0x7fe0bc15c3d0 legacy=0x7fe0bc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.905+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1161147837 >> v1:172.21.15.59:6789/0 conn(0x7fe0bc0b4690 legacy=0x7fe0bc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.905+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1161147837 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc160ab0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.906+0000 7fe0b3e00640 1 --2- 172.21.15.59:0/1161147837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0b4150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a14e5f29f92b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.906+0000 7fe0bbe00640 1 -- 172.21.15.59:0/1161147837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4037070 con 0x7fe0bc0cd760 2024-04-24T06:31:10.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.906+0000 7fe0bbe00640 1 -- 172.21.15.59:0/1161147837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a4032440 con 0x7fe0bc0cd760 2024-04-24T06:31:10.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.906+0000 7fe0bbe00640 1 -- 172.21.15.59:0/1161147837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4032810 con 0x7fe0bc0cd760 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 -- 172.21.15.59:0/1161147837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 msgr2=0x7fe0bc0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1161147837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0b4150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a402a2a0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 -- 172.21.15.59:0/1161147837 shutdown_connections 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1161147837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0b4150 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 -- 172.21.15.59:0/1161147837 >> 172.21.15.59:0/1161147837 conn(0x7fe0bc0b21a0 msgr2=0x7fe0bc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 -- 172.21.15.59:0/1161147837 shutdown_connections 2024-04-24T06:31:10.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.907+0000 7fe0c2200640 1 -- 172.21.15.59:0/1161147837 wait complete. 2024-04-24T06:31:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.909+0000 7fe0c2200640 1 Processor -- start 2024-04-24T06:31:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.909+0000 7fe0c2200640 1 -- start start 2024-04-24T06:31:10.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0bc0b4690 0x7fe0bc0998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 0x7fe0bc1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc15c7b0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0b4ae0 con 0x7fe0bc0b4690 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0c2200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc15c930 con 0x7fe0bc15c3d0 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36104/0 (socket says 172.21.15.59:36104) 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.910+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1464089028 learned_addr learned my addr 172.21.15.59:0/1464089028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:10.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0c0e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0bc0b4690 0x7fe0bc0998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0c1800640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 0x7fe0bc1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 msgr2=0x7fe0bc1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 0x7fe0bc1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0bc0b4690 msgr2=0x7fe0bc0998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0bc0b4690 0x7fe0bc0998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 -- 172.21.15.59:0/1464089028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc0aefc0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0c1800640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 0x7fe0bc1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:10.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.911+0000 7fe0b3e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.912+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4041020 con 0x7fe0bc0cd760 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.912+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a4030970 con 0x7fe0bc0cd760 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.912+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4037040 con 0x7fe0bc0cd760 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.912+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc0af2a0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.912+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc0af860 con 0x7fe0bc0cd760 2024-04-24T06:31:10.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.913+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc15fa30 con 0x7fe0bc0cd760 2024-04-24T06:31:10.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.914+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0a4044020 con 0x7fe0bc0cd760 2024-04-24T06:31:10.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.915+0000 7fe0b9600640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 0x7fe09806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:10.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.915+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a40ac5e0 con 0x7fe0bc0cd760 2024-04-24T06:31:10.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.915+0000 7fe0c0e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 0x7fe09806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:10.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.916+0000 7fe0c0e00640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 0x7fe09806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006fd0 tx=0x7fe0ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:10.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:10.920+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0bc15fa30 con 0x7fe0bc0cd760 2024-04-24T06:31:11.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.168+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0bc0c6950 con 0x7fe0bc0cd760 2024-04-24T06:31:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.169+0000 7fe0b9600640 1 -- 172.21.15.59:0/1464089028 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7fe0a404b030 con 0x7fe0bc0cd760 2024-04-24T06:31:11.170 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 msgr2=0x7fe09806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 0x7fe09806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006fd0 tx=0x7fe0ac008040 comp rx=0 tx=0).stop 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 msgr2=0x7fe0bc0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a40046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 shutdown_connections 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe09806c8f0 0x7fe09806edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe0bc15c3d0 0x7fe0bc1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:11.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0bc0cd760 0x7fe0bc0cecb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:11.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 --2- 172.21.15.59:0/1464089028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0bc0b4690 0x7fe0bc0998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:11.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 >> 172.21.15.59:0/1464089028 conn(0x7fe0bc0b21a0 msgr2=0x7fe0bc06ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:11.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 shutdown_connections 2024-04-24T06:31:11.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11.172+0000 7fe0c2200640 1 -- 172.21.15.59:0/1464089028 wait complete. 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:31:11.193 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:31:11.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:11.194 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:31:11.247 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.248 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:31:11.249 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:31:11.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:11.250 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:31:11.332 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:31:11.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:11.337 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:31:11.389 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:31:11.389 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:31:11.389 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:31:11.389 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:31:11.389 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:31:11.390 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:31:11.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:31:11.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:31:11.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:11.613 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:31:11.662 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:31:11.662 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:31:11.663 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:31:11.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:11.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:31:11.747 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:31:11.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:31:11.748 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:11.804 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:31:11.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:31:11.804 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:11.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:31:11.950 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:31:12.006 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:31:12.007 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:31:12.008 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:31:12.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:31:12.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.010 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:31:12.062 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:31:12.063 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:31:12.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:31:12.134 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:31:12.178 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:31:12.179 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:31:12.180 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:31:12.181 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:31:12.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.182 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:31:12.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:31:12.237+0000 7fbbd6f63480 -1 init, newargv = 0x5640b761d3d0 newargc=15 2024-04-24T06:31:12.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50814]: starting ceph client 2024-04-24T06:31:12.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50814]: starting fuse 2024-04-24T06:31:12.266 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:31:12.266 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:31:12.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:31:12.267 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:31:12.268 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:31:12.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.50814.asok 2024-04-24T06:31:12.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:31:12.485 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50814.asok 2024-04-24T06:31:12.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50814.asok status 2024-04-24T06:31:12.722 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "50814", 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:31:12.723 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5901, 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 5901 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 162730479 2024-04-24T06:31:12.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 162730479 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.5901 192.168.0.1:0/162730479", 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/162730479", 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 24, 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 45, 2024-04-24T06:31:12.725 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:31:12.726 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:31:12.726 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:31:12.726 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.738 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:31:12.738 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:12.797 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:31:12.797 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:12.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.797 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:12.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.893 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-24T06:31:12.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:12.970 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-24T06:31:13.038 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:31:13.038 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:31:13.038 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:31:13.039 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:31:13.039 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-24T06:31:13.039 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:31:13.195 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:31:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:31:13.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:13.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.464+0000 7ff957e00640 1 Processor -- start 2024-04-24T06:31:13.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.464+0000 7ff957e00640 1 -- start start 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff95815c3d0 0x7ff9580cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9580cd590 0x7ff9580b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff9580b4690 0x7ff95806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff958160210 con 0x7ff95815c3d0 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff9581604f0 con 0x7ff9580b4690 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff957e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff9581607d0 con 0x7ff9580cd590 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff956a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff95815c3d0 0x7ff9580cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:13.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff956a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff95815c3d0 0x7ff9580cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36106/0 (socket says 172.21.15.59:36106) 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.466+0000 7ff956a00640 1 -- 172.21.15.59:0/676702356 learned_addr learned my addr 172.21.15.59:0/676702356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.467+0000 7ff956a00640 1 -- 172.21.15.59:0/676702356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9580cd590 msgr2=0x7ff9580b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.467+0000 7ff956a00640 1 --2- 172.21.15.59:0/676702356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9580cd590 0x7ff9580b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.467+0000 7ff956a00640 1 -- 172.21.15.59:0/676702356 >> v2:172.21.15.66:3300/0 conn(0x7ff9580b4690 msgr2=0x7ff95806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.467+0000 7ff956a00640 1 --2- 172.21.15.59:0/676702356 >> v2:172.21.15.66:3300/0 conn(0x7ff9580b4690 0x7ff95806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.467+0000 7ff956a00640 1 -- 172.21.15.59:0/676702356 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff958160ab0 con 0x7ff95815c3d0 2024-04-24T06:31:13.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.468+0000 7ff956a00640 1 --2- 172.21.15.59:0/676702356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff95815c3d0 0x7ff9580cd050 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff958161170 tx=0x7ff94802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7909696b4dac3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:13.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.468+0000 7ff956000640 1 -- 172.21.15.59:0/676702356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948037070 con 0x7ff95815c3d0 2024-04-24T06:31:13.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.468+0000 7ff956000640 1 -- 172.21.15.59:0/676702356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff948032440 con 0x7ff95815c3d0 2024-04-24T06:31:13.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.468+0000 7ff956000640 1 -- 172.21.15.59:0/676702356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948032810 con 0x7ff95815c3d0 2024-04-24T06:31:13.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 -- 172.21.15.59:0/676702356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff95815c3d0 msgr2=0x7ff9580cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:13.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 --2- 172.21.15.59:0/676702356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff95815c3d0 0x7ff9580cd050 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff958161170 tx=0x7ff94802a1d0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 -- 172.21.15.59:0/676702356 shutdown_connections 2024-04-24T06:31:13.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 --2- 172.21.15.59:0/676702356 >> v2:172.21.15.66:3300/0 conn(0x7ff9580b4690 0x7ff95806b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 --2- 172.21.15.59:0/676702356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9580cd590 0x7ff9580b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 --2- 172.21.15.59:0/676702356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff95815c3d0 0x7ff9580cd050 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 -- 172.21.15.59:0/676702356 >> 172.21.15.59:0/676702356 conn(0x7ff9580b21a0 msgr2=0x7ff9580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:13.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 -- 172.21.15.59:0/676702356 shutdown_connections 2024-04-24T06:31:13.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.469+0000 7ff957e00640 1 -- 172.21.15.59:0/676702356 wait complete. 2024-04-24T06:31:13.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.471+0000 7ff957e00640 1 Processor -- start 2024-04-24T06:31:13.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.471+0000 7ff957e00640 1 -- start start 2024-04-24T06:31:13.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9580b4690 0x7ff9580c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff95815c3d0 0x7ff9580cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff95815c7b0 con 0x7ff9580cd590 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9580b4b00 con 0x7ff9580b4690 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff95815c930 con 0x7ff95815c3d0 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36112/0 (socket says 172.21.15.59:36112) 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 -- 172.21.15.59:0/2104346617 learned_addr learned my addr 172.21.15.59:0/2104346617 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff956a00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9580b4690 0x7ff9580c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff957400640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff95815c3d0 0x7ff9580cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 -- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff95815c3d0 msgr2=0x7ff9580cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff95815c3d0 0x7ff9580cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 -- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9580b4690 msgr2=0x7ff9580c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9580b4690 0x7ff9580c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.472+0000 7ff94fe00640 1 -- 172.21.15.59:0/2104346617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95815e9d0 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff94fe00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff94000b4d0 tx=0x7ff94000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940014070 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff940010870 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940010c60 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9580c6280 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.473+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95815dc10 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.474+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95804fd90 con 0x7ff9580cd590 2024-04-24T06:31:13.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.476+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff940017070 con 0x7ff9580cd590 2024-04-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.477+0000 7ff94ea00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 0x7ff93406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.477+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff94008b940 con 0x7ff9580cd590 2024-04-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.477+0000 7ff956a00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 0x7ff93406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.478+0000 7ff956a00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 0x7ff93406edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff948034000 tx=0x7ff94802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.481+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff940059cd0 con 0x7ff9580cd590 2024-04-24T06:31:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.726+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff95806e400 con 0x7ff9580cd590 2024-04-24T06:31:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.728+0000 7ff94ea00640 1 -- 172.21.15.59:0/2104346617 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7ff9400595d0 con 0x7ff9580cd590 2024-04-24T06:31:13.729 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:13.729 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 msgr2=0x7ff93406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 0x7ff93406edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff948034000 tx=0x7ff94802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 msgr2=0x7ff9580ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff94000b4d0 tx=0x7ff94000b990 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 shutdown_connections 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff93406c8f0 0x7ff93406edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff95815c3d0 0x7ff9580cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9580cd590 0x7ff9580ce780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 --2- 172.21.15.59:0/2104346617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9580b4690 0x7ff9580c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.730+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 >> 172.21.15.59:0/2104346617 conn(0x7ff9580b21a0 msgr2=0x7ff9580b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.731+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 shutdown_connections 2024-04-24T06:31:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:13.731+0000 7ff957e00640 1 -- 172.21.15.59:0/2104346617 wait complete. 2024-04-24T06:31:13.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-24T06:31:14.027 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:31:14.028 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:31:14.028 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-24T06:31:14.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.147+0000 7fdf4ce87f40 1 Processor -- start 2024-04-24T06:31:14.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.147+0000 7fdf4ce87f40 1 -- start start 2024-04-24T06:31:14.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300022a0 0x55b8f6665a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 0x55b8f65ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8f6546cc0 con 0x55b8f6676400 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8f654ad00 con 0x7fdf300022a0 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b8f65ea420 con 0x55b8f6675c10 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36136/0 (socket says 172.21.15.59:36136) 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.148+0000 7fdf4aa00640 1 -- 172.21.15.59:0/1862249634 learned_addr learned my addr 172.21.15.59:0/1862249634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4a000640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300022a0 0x55b8f6665a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf49600640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 0x55b8f65ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 -- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 msgr2=0x55b8f65ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 0x55b8f65ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 -- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300022a0 msgr2=0x55b8f6665a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300022a0 0x55b8f6665a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 -- 172.21.15.59:0/1862249634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f65cf1b0 con 0x55b8f6676400 2024-04-24T06:31:14.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf49600640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 0x55b8f65ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:14.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.149+0000 7fdf4aa00640 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdf440094c0 tx=0x7fdf440095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd801adb74c9793b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf43e00640 1 -- 172.21.15.59:0/1862249634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44019070 con 0x55b8f6676400 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf43e00640 1 -- 172.21.15.59:0/1862249634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf440145f0 con 0x55b8f6676400 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf43e00640 1 -- 172.21.15.59:0/1862249634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf440149a0 con 0x55b8f6676400 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/1862249634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 msgr2=0x55b8f6676880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdf440094c0 tx=0x7fdf440095b0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/1862249634 shutdown_connections 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6676880 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8f6675c10 0x55b8f65ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/1862249634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf300022a0 0x55b8f6665a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/1862249634 >> 172.21.15.59:0/1862249634 conn(0x55b8f65eb290 msgr2=0x55b8f66774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.150+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/1862249634 shutdown_connections 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.151+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/1862249634 wait complete. 2024-04-24T06:31:14.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.151+0000 7fdf4ce87f40 1 Processor -- start 2024-04-24T06:31:14.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.151+0000 7fdf4ce87f40 1 -- start start 2024-04-24T06:31:14.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8f6675c10 0x55b8f6665160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6672760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf300022a0 0x55b8f6672ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8f65c9e60 con 0x55b8f6676400 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8f65a1b70 con 0x55b8f6675c10 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.152+0000 7fdf4ce87f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b8f6678f90 con 0x7fdf300022a0 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6672760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6672760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36140/0 (socket says 172.21.15.59:36140) 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 -- 172.21.15.59:0/418634898 learned_addr learned my addr 172.21.15.59:0/418634898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 -- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf300022a0 msgr2=0x55b8f6672ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf300022a0 0x55b8f6672ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 -- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8f6675c10 msgr2=0x55b8f6665160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8f6675c10 0x55b8f6665160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 -- 172.21.15.59:0/418634898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f6678230 con 0x55b8f6676400 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.153+0000 7fdf49600640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6672760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800b600 tx=0x7fdf3800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.154+0000 7fdf42a00640 1 -- 172.21.15.59:0/418634898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800e880 con 0x55b8f6676400 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.154+0000 7fdf42a00640 1 -- 172.21.15.59:0/418634898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf38012030 con 0x55b8f6676400 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.154+0000 7fdf42a00640 1 -- 172.21.15.59:0/418634898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800ec20 con 0x55b8f6676400 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.154+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8f6678510 con 0x55b8f6676400 2024-04-24T06:31:14.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.154+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f666f920 con 0x55b8f6676400 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.156+0000 7fdf42a00640 1 -- 172.21.15.59:0/418634898 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf38009040 con 0x55b8f6676400 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.157+0000 7fdf42a00640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdf2c06b300 0x7fdf2c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.157+0000 7fdf42a00640 1 -- 172.21.15.59:0/418634898 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdf3808ad40 con 0x55b8f6676400 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.157+0000 7fdf4a000640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdf2c06b300 0x7fdf2c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.157+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55b8f6667c40 0x55b8f666c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.157+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) -- 0x55b8f66656a0 con 0x55b8f6667c40 2024-04-24T06:31:14.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.158+0000 7fdf4a000640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdf2c06b300 0x7fdf2c06d7c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c055000 tx=0x7fdf3c051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.158+0000 7fdf4aa00640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55b8f6667c40 0x55b8f666c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.159+0000 7fdf4aa00640 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55b8f6667c40 0x55b8f666c140 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4aa00640 1 -- 172.21.15.59:0/418634898 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fdf440149a0 con 0x55b8f6667c40 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55b8f6667c40 msgr2=0x55b8f666c140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55b8f6667c40 0x55b8f666c140 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdf2c06b300 msgr2=0x7fdf2c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdf2c06b300 0x7fdf2c06d7c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c055000 tx=0x7fdf3c051000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 msgr2=0x55b8f6672760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 --2- 172.21.15.59:0/418634898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8f6676400 0x55b8f6672760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800b600 tx=0x7fdf3800bac0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4aa00640 1 -- 172.21.15.59:0/418634898 reap_dead start 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 shutdown_connections 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 >> 172.21.15.59:0/418634898 conn(0x55b8f65eb290 msgr2=0x55b8f65a04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 shutdown_connections 2024-04-24T06:31:14.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.160+0000 7fdf4ce87f40 1 -- 172.21.15.59:0/418634898 wait complete. 2024-04-24T06:31:14.166 INFO:teuthology.orchestra.run.smithi059.stdout:Yalphaparent_a 2024-04-24T06:31:14.166 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "alpha", 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:14.325 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "parent_a", 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 7, 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:14.326 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:14.340 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-24T06:31:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.462+0000 7ff397b08f40 1 Processor -- start 2024-04-24T06:31:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.462+0000 7ff397b08f40 1 -- start start 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af9364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af88a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7af8a8e70 con 0x55c7af9195b0 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c7af805cc0 con 0x55c7af8a66a0 2024-04-24T06:31:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.463+0000 7ff397b08f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c7af8a9110 con 0x55c7af8a8450 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36148/0 (socket says 172.21.15.59:36148) 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 -- 172.21.15.59:0/2773400388 learned_addr learned my addr 172.21.15.59:0/2773400388 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff394e00640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af9364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff38c400640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af88a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 -- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 msgr2=0x55c7af88a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af88a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 -- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 msgr2=0x55c7af9364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af9364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 -- 172.21.15.59:0/2773400388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7af85ed30 con 0x55c7af9195b0 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff38c400640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af88a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.464+0000 7ff395800640 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff390050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571b7d103ef2f838 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff38f400640 1 -- 172.21.15.59:0/2773400388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390055030 con 0x55c7af9195b0 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff38f400640 1 -- 172.21.15.59:0/2773400388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff390051bb0 con 0x55c7af9195b0 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff38f400640 1 -- 172.21.15.59:0/2773400388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39005b430 con 0x55c7af9195b0 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 -- 172.21.15.59:0/2773400388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 msgr2=0x55c7af88a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff390050000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 -- 172.21.15.59:0/2773400388 shutdown_connections 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af88a930 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af88a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.465+0000 7ff397b08f40 1 --2- 172.21.15.59:0/2773400388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af9364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.466+0000 7ff397b08f40 1 -- 172.21.15.59:0/2773400388 >> 172.21.15.59:0/2773400388 conn(0x55c7af88be20 msgr2=0x55c7af889950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.466+0000 7ff397b08f40 1 -- 172.21.15.59:0/2773400388 shutdown_connections 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.466+0000 7ff397b08f40 1 -- 172.21.15.59:0/2773400388 wait complete. 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.466+0000 7ff397b08f40 1 Processor -- start 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.467+0000 7ff397b08f40 1 -- start start 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af917920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af8a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af8a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7af8a5820 con 0x55c7af9195b0 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c7af932d70 con 0x55c7af8a66a0 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff397b08f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c7af8a53b0 con 0x55c7af8a8450 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff395800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af8a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff395800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af8a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36160/0 (socket says 172.21.15.59:36160) 2024-04-24T06:31:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff395800640 1 -- 172.21.15.59:0/894406181 learned_addr learned my addr 172.21.15.59:0/894406181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff38c400640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af8a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.468+0000 7ff394e00640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af917920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 -- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 msgr2=0x55c7af8a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c7af8a8450 0x55c7af8a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 -- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 msgr2=0x55c7af917920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c7af8a66a0 0x55c7af917920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 -- 172.21.15.59:0/894406181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7af918290 con 0x55c7af9195b0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.469+0000 7ff395800640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af8a3f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff390003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.470+0000 7ff38e000640 1 -- 172.21.15.59:0/894406181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900519b0 con 0x55c7af9195b0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.470+0000 7ff38e000640 1 -- 172.21.15.59:0/894406181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff390051d60 con 0x55c7af9195b0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.470+0000 7ff38e000640 1 -- 172.21.15.59:0/894406181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390067b10 con 0x55c7af9195b0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.470+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c7af934780 con 0x55c7af9195b0 2024-04-24T06:31:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.470+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7af9c4240 con 0x55c7af9195b0 2024-04-24T06:31:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.472+0000 7ff38e000640 1 -- 172.21.15.59:0/894406181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff39006f030 con 0x55c7af9195b0 2024-04-24T06:31:14.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.473+0000 7ff38e000640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff37006b2d0 0x7ff37006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.473+0000 7ff38e000640 1 -- 172.21.15.59:0/894406181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff3900cf280 con 0x55c7af9195b0 2024-04-24T06:31:14.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.473+0000 7ff394e00640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff37006b2d0 0x7ff37006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.474+0000 7ff397b08f40 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55c7af935db0 0x55c7af9d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.474+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x55c7af9d91f0 con 0x55c7af935db0 2024-04-24T06:31:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.474+0000 7ff394e00640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff37006b2d0 0x7ff37006d790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55c7af918e30 tx=0x7ff384003550 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.474+0000 7ff38c400640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55c7af935db0 0x55c7af9d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.475+0000 7ff38c400640 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55c7af935db0 0x55c7af9d8cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff38c400640 1 -- 172.21.15.59:0/894406181 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v45'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff37c0098f0 con 0x55c7af935db0 2024-04-24T06:31:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55c7af935db0 msgr2=0x55c7af9d8cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55c7af935db0 0x55c7af9d8cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff37006b2d0 msgr2=0x7ff37006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff37006b2d0 0x7ff37006d790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55c7af918e30 tx=0x7ff384003550 comp rx=0 tx=0).stop 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 msgr2=0x55c7af8a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 --2- 172.21.15.59:0/894406181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af9195b0 0x55c7af8a3f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff390003330 comp rx=0 tx=0).stop 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff395800640 1 -- 172.21.15.59:0/894406181 reap_dead start 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 shutdown_connections 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 >> 172.21.15.59:0/894406181 conn(0x55c7af88be20 msgr2=0x55c7af848de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 shutdown_connections 2024-04-24T06:31:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.478+0000 7ff397b08f40 1 -- 172.21.15.59:0/894406181 wait complete. 2024-04-24T06:31:14.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-24T06:31:14.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:14.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-24T06:31:14.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.674+0000 7f2629400640 1 Processor -- start 2024-04-24T06:31:14.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.675+0000 7f2629400640 1 -- start start 2024-04-24T06:31:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.676+0000 7f2629400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.676+0000 7f2629400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26240d10a0 0x7f26240d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.676+0000 7f2629400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26240d0310 con 0x7f26240cf5e0 2024-04-24T06:31:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.676+0000 7f2629400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f26240d1a40 con 0x7f26240d10a0 2024-04-24T06:31:14.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.676+0000 7f2629400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26240d05f0 con 0x7f2624161fd0 2024-04-24T06:31:14.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26240d10a0 0x7f26240d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26240d10a0 0x7f26240d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36168/0 (socket says 172.21.15.59:36168) 2024-04-24T06:31:14.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 -- 172.21.15.59:0/3017548466 learned_addr learned my addr 172.21.15.59:0/3017548466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2622800640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 -- 172.21.15.59:0/3017548466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 msgr2=0x7f26240b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 -- 172.21.15.59:0/3017548466 >> v1:172.21.15.66:6789/0 conn(0x7f26240cf5e0 legacy=0x7f26240cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.677+0000 7f2621e00640 1 -- 172.21.15.59:0/3017548466 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240d2520 con 0x7f26240d10a0 2024-04-24T06:31:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.678+0000 7f2621e00640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240d10a0 0x7f26240d1500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2610009870 tx=0x7f261002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a862621904bf01b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.679+0000 7f2621400640 1 -- 172.21.15.59:0/3017548466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610037070 con 0x7f26240d10a0 2024-04-24T06:31:14.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.679+0000 7f2621400640 1 -- 172.21.15.59:0/3017548466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610032440 con 0x7f26240d10a0 2024-04-24T06:31:14.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.679+0000 7f2621400640 1 -- 172.21.15.59:0/3017548466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610032810 con 0x7f26240d10a0 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.679+0000 7f2629400640 1 -- 172.21.15.59:0/3017548466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240d10a0 msgr2=0x7f26240d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.679+0000 7f2629400640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240d10a0 0x7f26240d1500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2610009870 tx=0x7f261002a210 comp rx=0 tx=0).stop 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 -- 172.21.15.59:0/3017548466 shutdown_connections 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240d10a0 0x7f26240d1500 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 --2- 172.21.15.59:0/3017548466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 -- 172.21.15.59:0/3017548466 >> 172.21.15.59:0/3017548466 conn(0x7f262408bf90 msgr2=0x7f26240d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 -- 172.21.15.59:0/3017548466 shutdown_connections 2024-04-24T06:31:14.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.680+0000 7f2629400640 1 -- 172.21.15.59:0/3017548466 wait complete. 2024-04-24T06:31:14.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.682+0000 7f2629400640 1 Processor -- start 2024-04-24T06:31:14.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.682+0000 7f2629400640 1 -- start start 2024-04-24T06:31:14.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2622800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2622800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36178/0 (socket says 172.21.15.59:36178) 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26240d10a0 0x7f2624152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2622800640 1 -- 172.21.15.59:0/4287752783 learned_addr learned my addr 172.21.15.59:0/4287752783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26240d2d20 con 0x7f26240cf5e0 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26240b4610 con 0x7f26240d10a0 2024-04-24T06:31:14.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.683+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26240d2ea0 con 0x7f2624161fd0 2024-04-24T06:31:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 -- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 msgr2=0x7f26240ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 -- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26240d10a0 msgr2=0x7f2624152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2621e00640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26240d10a0 0x7f2624152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26240d10a0 0x7f2624152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240c9460 con 0x7f26240cf5e0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.684+0000 7f2622800640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f261400d620 tx=0x7f261400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.685+0000 7f261aa00640 1 -- 172.21.15.59:0/4287752783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400b620 con 0x7f26240cf5e0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.685+0000 7f261aa00640 1 -- 172.21.15.59:0/4287752783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614019070 con 0x7f26240cf5e0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.685+0000 7f261aa00640 1 -- 172.21.15.59:0/4287752783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400ba00 con 0x7f26240cf5e0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.685+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26240c9740 con 0x7f26240cf5e0 2024-04-24T06:31:14.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.685+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26240c9fe0 con 0x7f26240cf5e0 2024-04-24T06:31:14.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.687+0000 7f261aa00640 1 -- 172.21.15.59:0/4287752783 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f261400bba0 con 0x7f26240cf5e0 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.688+0000 7f261aa00640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 0x7f260406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.689+0000 7f261aa00640 1 -- 172.21.15.59:0/4287752783 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f261408c070 con 0x7f26240cf5e0 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.689+0000 7f2621e00640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 0x7f260406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.689+0000 7f262c247740 1 Processor -- start 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.689+0000 7f262c247740 1 -- start start 2024-04-24T06:31:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.689+0000 7f2621e00640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 0x7f260406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2610034000 tx=0x7f261002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608391e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 0x5608391e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56083914ee70 con 0x560839215580 2024-04-24T06:31:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5608390f5380 con 0x560839232460 2024-04-24T06:31:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56083914ae30 con 0x560839214e40 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f2621e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f2621e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36186/0 (socket says 172.21.15.59:36186) 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.690+0000 7f2621e00640 1 -- 172.21.15.59:0/1170641786 learned_addr learned my addr 172.21.15.59:0/1170641786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2622800640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608391e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 -- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 msgr2=0x5608391e9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2623200640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 0x5608391e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 0x5608391e9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 -- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 msgr2=0x5608391e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608391e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 -- 172.21.15.59:0/1170641786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560838f297e0 con 0x560839215580 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2622800640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608391e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2623200640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 0x5608391e9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.691+0000 7f2621e00640 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f261002a800 tx=0x7f261002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d27ae2dbbd1655 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.692+0000 7f260f000640 1 -- 172.21.15.59:0/1170641786 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261004b070 con 0x560839215580 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.692+0000 7f260f000640 1 -- 172.21.15.59:0/1170641786 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610032cb0 con 0x560839215580 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.692+0000 7f260f000640 1 -- 172.21.15.59:0/1170641786 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610037040 con 0x560839215580 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.692+0000 7f262c247740 1 -- 172.21.15.59:0/1170641786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 msgr2=0x560839214900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.692+0000 7f262c247740 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f261002a800 tx=0x7f261002c000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 -- 172.21.15.59:0/1170641786 shutdown_connections 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839214e40 0x5608391e9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839214900 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 --2- 172.21.15.59:0/1170641786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608391e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 -- 172.21.15.59:0/1170641786 >> 172.21.15.59:0/1170641786 conn(0x5608391e7060 msgr2=0x56083922f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 -- 172.21.15.59:0/1170641786 shutdown_connections 2024-04-24T06:31:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.693+0000 7f262c247740 1 -- 172.21.15.59:0/1170641786 wait complete. 2024-04-24T06:31:14.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.694+0000 7f262c247740 1 -- start start 2024-04-24T06:31:14.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.694+0000 7f262c247740 1 Processor -- start 2024-04-24T06:31:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x560839230d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x5608392171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5608391e9210 con 0x560839227190 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560839108660 con 0x560839215580 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5608391e8600 con 0x560839232460 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f262c247740 10 client.0 _ll_register_callbacks cb 0x5608390f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f2623200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f2623200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36200/0 (socket says 172.21.15.59:36200) 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.696+0000 7f2623200640 1 -- 172.21.15.59:0/1328611995 learned_addr learned my addr 172.21.15.59:0/1328611995 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2622800640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x560839230d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2621e00640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x5608392171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 -- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 msgr2=0x5608392171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x5608392171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 -- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 msgr2=0x560839230d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x560839230d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 -- 172.21.15.59:0/1328611995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560838f89f70 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2623200640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d480 tx=0x7f261c00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf10e292def1be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2603400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c016070 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f261c012800 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.697+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2610032cb0 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f261c006b50 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 10 client.5934 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560838e21170 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610032cb0 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f261c024080 con 0x560839227190 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 10 client.5934 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 10 client.5934 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 20 client.5934 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839203ae0 0x560839215f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 4 client.5934 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.698+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56083915c890 con 0x560839203ae0 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.699+0000 7f2622800640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839203ae0 0x560839215f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.700+0000 7f2622800640 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839203ae0 0x560839215f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b42341401ec66d server_cookie=44f565a181f25284 in_seq=0 out_seq=0 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.700+0000 7f2603400640 10 client.5934 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f2603400640 1 -- 172.21.15.59:0/1328611995 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56083915c890 con 0x560839203ae0 2024-04-24T06:31:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f2603400640 10 client.5934 handle_command_reply: tid=0 2024-04-24T06:31:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 client.5934 shutdown 2024-04-24T06:31:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7fd9c3000640 1 Processor -- start 2024-04-24T06:31:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 msgr2=0x5608392275d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d480 tx=0x7f261c00d940 comp rx=0 tx=0).stop 2024-04-24T06:31:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7fd9c3000640 1 -- start start 2024-04-24T06:31:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 shutdown_connections 2024-04-24T06:31:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x5608392275d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839203ae0 0x560839215f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x5608392171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 --2- 172.21.15.59:0/1328611995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x560839230d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 >> 172.21.15.59:0/1328611995 conn(0x560839214e40 msgr2=0x5608391e4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.702+0000 7f2603400640 0 client.5934 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.703+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 shutdown_connections 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.703+0000 7f262c247740 1 -- 172.21.15.59:0/1328611995 wait complete. 2024-04-24T06:31:14.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.703+0000 7f262c247740 20 client.5934 trim_cache size 0 max 16384 2024-04-24T06:31:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c3000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd9bc15ed00 0x7fd9bc163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c3000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 0x7fd9bc0d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c3000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc0cfc70 con 0x7fd9bc15e920 2024-04-24T06:31:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c3000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc163550 con 0x7fd9bc15ed00 2024-04-24T06:31:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c3000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0cff50 con 0x7fd9bc0d1960 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9bbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd9bc15e920 0x7fd9bc0d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33582/0 (socket says 172.21.15.59:33582) 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9bbe00640 1 -- 172.21.15.59:0/3071776643 learned_addr learned my addr 172.21.15.59:0/3071776643 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9bb400640 1 --2- 172.21.15.59:0/3071776643 >> v2:172.21.15.59:3300/0 conn(0x7fd9bc15ed00 0x7fd9bc163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9c1000640 1 --2- 172.21.15.59:0/3071776643 >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 0x7fd9bc0d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9baa00640 1 -- 172.21.15.59:0/3071776643 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358602858 0 0) 0x7fd9bc0cfc70 con 0x7fd9bc15e920 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.704+0000 7fd9baa00640 1 -- 172.21.15.59:0/3071776643 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a0003520 con 0x7fd9bc15e920 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.705+0000 7fd9bb400640 1 -- 172.21.15.59:0/3071776643 >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 msgr2=0x7fd9bc0d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.705+0000 7fd9bb400640 1 --2- 172.21.15.59:0/3071776643 >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 0x7fd9bc0d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.705+0000 7fd9bb400640 1 -- 172.21.15.59:0/3071776643 >> v1:172.21.15.59:6789/0 conn(0x7fd9bc15e920 legacy=0x7fd9bc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.705+0000 7fd9bb400640 1 -- 172.21.15.59:0/3071776643 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc0d2f40 con 0x7fd9bc15ed00 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.705+0000 7fd9c1000640 1 --2- 172.21.15.59:0/3071776643 >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 0x7fd9bc0d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.706+0000 7fd9bb400640 1 --2- 172.21.15.59:0/3071776643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc163010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0d0e60 tx=0x7fd9a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a43376f1ce1475 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.706+0000 7fd9baa00640 1 -- 172.21.15.59:0/3071776643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8037070 con 0x7fd9bc15ed00 2024-04-24T06:31:14.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.706+0000 7fd9baa00640 1 -- 172.21.15.59:0/3071776643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a8032440 con 0x7fd9bc15ed00 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.706+0000 7fd9baa00640 1 -- 172.21.15.59:0/3071776643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a802ae60 con 0x7fd9bc15ed00 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 -- 172.21.15.59:0/3071776643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc15ed00 msgr2=0x7fd9bc163010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 --2- 172.21.15.59:0/3071776643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc163010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0d0e60 tx=0x7fd9a802a110 comp rx=0 tx=0).stop 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 -- 172.21.15.59:0/3071776643 shutdown_connections 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 --2- 172.21.15.59:0/3071776643 >> v2:172.21.15.66:3300/0 conn(0x7fd9bc0d1960 0x7fd9bc0d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 --2- 172.21.15.59:0/3071776643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc163010 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 -- 172.21.15.59:0/3071776643 >> 172.21.15.59:0/3071776643 conn(0x7fd9bc067760 msgr2=0x7fd9bc067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 -- 172.21.15.59:0/3071776643 shutdown_connections 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.707+0000 7fd9c3000640 1 -- 172.21.15.59:0/3071776643 wait complete. 2024-04-24T06:31:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.709+0000 7fd9c3000640 1 Processor -- start 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.709+0000 7fd9c3000640 1 -- start start 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc0d1960 0x7fd9bc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9bc15e920 0x7fd9bc0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0cf920 con 0x7fd9bc0d1960 2024-04-24T06:31:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0d3220 con 0x7fd9bc15ed00 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c3000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0cf390 con 0x7fd9bc15e920 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc0d1960 0x7fd9bc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc0d1960 0x7fd9bc153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36224/0 (socket says 172.21.15.59:36224) 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 -- 172.21.15.59:0/2196384577 learned_addr learned my addr 172.21.15.59:0/2196384577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bb400640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9bc15e920 0x7fd9bc0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 -- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9bc15e920 msgr2=0x7fd9bc0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9bc15e920 0x7fd9bc0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 -- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9bc15ed00 msgr2=0x7fd9bc0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9c1000640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9bc15ed00 0x7fd9bc0c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.710+0000 7fd9bbe00640 1 -- 172.21.15.59:0/2196384577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc154300 con 0x7fd9bc0d1960 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9bbe00640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9bc0d1960 0x7fd9bc153dc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00c460 tx=0x7fd9ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac017070 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ac005ce0 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac007290 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9c3000640 1 -- 172.21.15.59:0/2196384577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc0ca340 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.711+0000 7fd9c3000640 1 -- 172.21.15.59:0/2196384577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc0ca900 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.712+0000 7fd9c3000640 1 -- 172.21.15.59:0/2196384577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd9bc06e260 con 0x7fd9bc0d1960 2024-04-24T06:31:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.715+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9ac01a070 con 0x7fd9bc0d1960 2024-04-24T06:31:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.715+0000 7fd9b3e00640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd99c06c8f0 0x7fd99c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.716+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fd9ac08b680 con 0x7fd9bc0d1960 2024-04-24T06:31:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.716+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7fd9ac021070 con 0x7fd9bc0d1960 2024-04-24T06:31:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.716+0000 7fd9bb400640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd99c06c8f0 0x7fd99c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.716+0000 7fd9c3000640 1 -- 172.21.15.59:0/2196384577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fd9bc04fe70 con 0x7fd9bc0d1960 2024-04-24T06:31:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.717+0000 7fd9bb400640 1 --2- 172.21.15.59:0/2196384577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd99c06c8f0 0x7fd99c06edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8034000 tx=0x7fd9a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.725+0000 7f262c247740 1 Processor -- start 2024-04-24T06:31:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.725+0000 7f262c247740 1 -- start start 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x56083920fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x56083922ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2624200e00 con 0x560839227190 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26241ffe60 con 0x560839215580 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26241ff1f0 con 0x560839232460 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f2621e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f2621e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36234/0 (socket says 172.21.15.59:36234) 2024-04-24T06:31:14.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.726+0000 7f2621e00640 1 -- 172.21.15.59:0/617068494 learned_addr learned my addr 172.21.15.59:0/617068494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 -- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 msgr2=0x56083922ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x56083922ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2622800640 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x56083920fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 -- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 msgr2=0x56083920fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x56083920fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 -- 172.21.15.59:0/617068494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560839231c50 con 0x560839227190 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f2621e00640 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2610009630 tx=0x7f261003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587adca0726b4940 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f260e600640 1 -- 172.21.15.59:0/617068494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261004b070 con 0x560839227190 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f260e600640 1 -- 172.21.15.59:0/617068494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100044a0 con 0x560839227190 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.727+0000 7f260e600640 1 -- 172.21.15.59:0/617068494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610037070 con 0x560839227190 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 -- 172.21.15.59:0/617068494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 msgr2=0x56083922e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2610009630 tx=0x7f261003a000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 -- 172.21.15.59:0/617068494 shutdown_connections 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839232460 0x56083922ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839227190 0x56083922e7b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 --2- 172.21.15.59:0/617068494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839215580 0x56083920fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 -- 172.21.15.59:0/617068494 >> 172.21.15.59:0/617068494 conn(0x560839214e40 msgr2=0x560839226560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 -- 172.21.15.59:0/617068494 shutdown_connections 2024-04-24T06:31:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.728+0000 7f262c247740 1 -- 172.21.15.59:0/617068494 wait complete. 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.729+0000 7f262c247740 1 -- start start 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.729+0000 7f262c247740 1 Processor -- start 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839227190 0x560839204400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608392ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5608391e5b30 con 0x560839215580 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26241ff810 con 0x560839232460 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5608391e7e30 con 0x560839227190 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.730+0000 7f262c247740 10 client.0 _ll_register_callbacks cb 0x7f261c003cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36244/0 (socket says 172.21.15.59:36244) 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 -- 172.21.15.59:0/1533439229 learned_addr learned my addr 172.21.15.59:0/1533439229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2623200640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608392ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 -- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839227190 msgr2=0x560839204400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2621e00640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839227190 0x560839204400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839227190 0x560839204400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 -- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 msgr2=0x5608392ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608392ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 -- 172.21.15.59:0/1533439229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083922f1c0 con 0x560839215580 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2622800640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2614059c00 tx=0x7f2614020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93557c3d55775d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2603400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.731+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614083e60 con 0x560839215580 2024-04-24T06:31:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f261408a120 con 0x560839215580 2024-04-24T06:31:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100044a0 con 0x560839215580 2024-04-24T06:31:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26140721c0 con 0x560839215580 2024-04-24T06:31:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 10 client.5949 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560839203240 con 0x560839215580 2024-04-24T06:31:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100044a0 con 0x560839215580 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f2614013070 con 0x560839215580 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 10 client.5949 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 10 client.5949 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 20 client.5949 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839242790 0x5608392add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 4 client.5949 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.732+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x560839156700 con 0x560839242790 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.733+0000 7f2621e00640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839242790 0x5608392add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.734+0000 7f2621e00640 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839242790 0x5608392add90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3766d05a20570424 server_cookie=774e2450792d7364 in_seq=0 out_seq=0 2024-04-24T06:31:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.734+0000 7f2603400640 10 client.5949 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.735+0000 7f2603400640 1 -- 172.21.15.59:0/1533439229 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560839156700 con 0x560839242790 2024-04-24T06:31:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.735+0000 7f2603400640 10 client.5949 handle_command_reply: tid=0 2024-04-24T06:31:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.735+0000 7f262c247740 1 client.5949 shutdown 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 msgr2=0x560839203ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2614059c00 tx=0x7f2614020040 comp rx=0 tx=0).stop 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 shutdown_connections 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560839232460 0x5608392ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x560839242790 0x5608392add90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560839227190 0x560839204400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 --2- 172.21.15.59:0/1533439229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560839215580 0x560839203ec0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f2603400640 0 client.5949 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 >> 172.21.15.59:0/1533439229 conn(0x560839214e40 msgr2=0x560839226560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 shutdown_connections 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 1 -- 172.21.15.59:0/1533439229 wait complete. 2024-04-24T06:31:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.736+0000 7f262c247740 20 client.5949 trim_cache size 0 max 16384 2024-04-24T06:31:14.739 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:14.739 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:31:14.739 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "45f4be23-56fa-414d-bee1-c1f36e244672", 2024-04-24T06:31:14.739 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:31:14.739 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 msgr2=0x7f260406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 0x7f260406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2610034000 tx=0x7f261002e000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 msgr2=0x7f26240b9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f261400d620 tx=0x7f261400dae0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 shutdown_connections 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f260406c8f0 0x7f260406edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2624161fd0 0x7f26240ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26240d10a0 0x7f2624152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 --2- 172.21.15.59:0/4287752783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26240cf5e0 0x7f26240b9c30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 >> 172.21.15.59:0/4287752783 conn(0x7f262408bf90 msgr2=0x7f262406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 shutdown_connections 2024-04-24T06:31:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.737+0000 7f2629400640 1 -- 172.21.15.59:0/4287752783 wait complete. 2024-04-24T06:31:14.741 INFO:tasks.ceph.mds.b.smithi066.stderr:2024-04-24T06:31:14.736+0000 7fb4f8e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5901=0-4194304@1} caps={5901=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5901 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x555a414eec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:31:14.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:14.755 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-24T06:31:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.947+0000 7f126ec00640 1 Processor -- start 2024-04-24T06:31:14.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.947+0000 7f126ec00640 1 -- start start 2024-04-24T06:31:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f126806fb50 0x7f12680b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126ec00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f12680b3ed0 0x7f12680d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126ec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12680728b0 con 0x7f12680724d0 2024-04-24T06:31:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126ec00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1268162a70 con 0x7f12680b3ed0 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1268162d50 con 0x7f126806fb50 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126e200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f12680b3ed0 0x7f12680d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126e200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f12680b3ed0 0x7f12680d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36254/0 (socket says 172.21.15.59:36254) 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.949+0000 7f126e200640 1 -- 172.21.15.59:0/4086251112 learned_addr learned my addr 172.21.15.59:0/4086251112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.950+0000 7f126ce00640 1 --2- 172.21.15.59:0/4086251112 >> v2:172.21.15.66:3300/0 conn(0x7f126806fb50 0x7f12680b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.950+0000 7f126e200640 1 -- 172.21.15.59:0/4086251112 >> v2:172.21.15.66:3300/0 conn(0x7f126806fb50 msgr2=0x7f12680b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.950+0000 7f126e200640 1 --2- 172.21.15.59:0/4086251112 >> v2:172.21.15.66:3300/0 conn(0x7f126806fb50 0x7f12680b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.950+0000 7f126e200640 1 -- 172.21.15.59:0/4086251112 >> v1:172.21.15.66:6789/0 conn(0x7f12680724d0 legacy=0x7f1268162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.950+0000 7f126e200640 1 -- 172.21.15.59:0/4086251112 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1268163030 con 0x7f12680b3ed0 2024-04-24T06:31:14.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.951+0000 7f126e200640 1 --2- 172.21.15.59:0/4086251112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680b3ed0 0x7f12680d05d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1260009e50 tx=0x7f126002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1765d80460fb1874 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.952+0000 7f1267e00640 1 -- 172.21.15.59:0/4086251112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260036070 con 0x7f12680b3ed0 2024-04-24T06:31:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.952+0000 7f1267e00640 1 -- 172.21.15.59:0/4086251112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1260031600 con 0x7f12680b3ed0 2024-04-24T06:31:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.952+0000 7f1267e00640 1 -- 172.21.15.59:0/4086251112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600319d0 con 0x7f12680b3ed0 2024-04-24T06:31:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.952+0000 7f126ec00640 1 -- 172.21.15.59:0/4086251112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680b3ed0 msgr2=0x7f12680d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.952+0000 7f126ec00640 1 --2- 172.21.15.59:0/4086251112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680b3ed0 0x7f12680d05d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1260009e50 tx=0x7f126002a710 comp rx=0 tx=0).stop 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 -- 172.21.15.59:0/4086251112 shutdown_connections 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 --2- 172.21.15.59:0/4086251112 >> v2:172.21.15.66:3300/0 conn(0x7f126806fb50 0x7f12680b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 --2- 172.21.15.59:0/4086251112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680b3ed0 0x7f12680d05d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 -- 172.21.15.59:0/4086251112 >> 172.21.15.59:0/4086251112 conn(0x7f126806bdf0 msgr2=0x7f126806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 -- 172.21.15.59:0/4086251112 shutdown_connections 2024-04-24T06:31:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.953+0000 7f126ec00640 1 -- 172.21.15.59:0/4086251112 wait complete. 2024-04-24T06:31:14.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.955+0000 7f126ec00640 1 Processor -- start 2024-04-24T06:31:14.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.955+0000 7f126ec00640 1 -- start start 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12680724d0 0x7f126814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f126814e230 0x7f12680998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12680d0b10 con 0x7f126806fb50 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12680b3470 con 0x7f12680724d0 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f12680d0c90 con 0x7f126814e230 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36256/0 (socket says 172.21.15.59:36256) 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126d800640 1 -- 172.21.15.59:0/4215269133 learned_addr learned my addr 172.21.15.59:0/4215269133 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.956+0000 7f126ce00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12680724d0 0x7f126814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 -- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f126814e230 msgr2=0x7f12680998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126e200640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f126814e230 0x7f12680998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f126814e230 0x7f12680998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 -- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12680724d0 msgr2=0x7f126814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12680724d0 0x7f126814f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 -- 172.21.15.59:0/4215269133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126814df50 con 0x7f126806fb50 2024-04-24T06:31:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.957+0000 7f126d800640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f125c00c410 tx=0x7f125c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.962+0000 7f1265600640 1 -- 172.21.15.59:0/4215269133 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c017070 con 0x7f126806fb50 2024-04-24T06:31:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.962+0000 7f1265600640 1 -- 172.21.15.59:0/4215269133 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c0035f0 con 0x7f126806fb50 2024-04-24T06:31:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.962+0000 7f1265600640 1 -- 172.21.15.59:0/4215269133 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c003a20 con 0x7f126806fb50 2024-04-24T06:31:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.962+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126814e690 con 0x7f126806fb50 2024-04-24T06:31:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.962+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126814ec10 con 0x7f126806fb50 2024-04-24T06:31:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.965+0000 7f1265600640 1 -- 172.21.15.59:0/4215269133 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f125c01a070 con 0x7f126806fb50 2024-04-24T06:31:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.966+0000 7f1265600640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 0x7f124806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.966+0000 7f1265600640 1 -- 172.21.15.59:0/4215269133 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f125c08afd0 con 0x7f126806fb50 2024-04-24T06:31:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.966+0000 7f126ce00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 0x7f124806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.966+0000 7f1272fe4740 1 Processor -- start 2024-04-24T06:31:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.966+0000 7f1272fe4740 1 -- start start 2024-04-24T06:31:14.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f126ce00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 0x7f124806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12681501a0 tx=0x7f1254008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea17e0 0x556bede765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bedddb1f0 con 0x556bedea1f20 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556bedd81350 con 0x556bedea17e0 2024-04-24T06:31:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.967+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556beddd71b0 con 0x556bedebee00 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36258/0 (socket says 172.21.15.59:36258) 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 -- 172.21.15.59:0/2511236890 learned_addr learned my addr 172.21.15.59:0/2511236890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126e200640 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea17e0 0x556bede765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126d800640 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 -- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 msgr2=0x556bede71790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede71790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 -- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea17e0 msgr2=0x556bede765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea17e0 0x556bede765f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 -- 172.21.15.59:0/2511236890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556beddd5f50 con 0x556bedea1f20 2024-04-24T06:31:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.968+0000 7f126ce00640 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f125400eba0 tx=0x7f1254011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e16a284e7b5bc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.969+0000 7f125a000640 1 -- 172.21.15.59:0/2511236890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125401a050 con 0x556bedea1f20 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.969+0000 7f125a000640 1 -- 172.21.15.59:0/2511236890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1254024070 con 0x556bedea1f20 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.969+0000 7f125a000640 1 -- 172.21.15.59:0/2511236890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125401f440 con 0x556bedea1f20 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.969+0000 7f1272fe4740 1 -- 172.21.15.59:0/2511236890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 msgr2=0x556bedea12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f125400eba0 tx=0x7f1254011000 comp rx=0 tx=0).stop 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 -- 172.21.15.59:0/2511236890 shutdown_connections 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea17e0 0x556bede765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bedea12a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2511236890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede71790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 -- 172.21.15.59:0/2511236890 >> 172.21.15.59:0/2511236890 conn(0x556bede73a00 msgr2=0x556bedebc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 -- 172.21.15.59:0/2511236890 shutdown_connections 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.970+0000 7f1272fe4740 1 -- 172.21.15.59:0/2511236890 wait complete. 2024-04-24T06:31:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.971+0000 7f1272fe4740 1 -- start start 2024-04-24T06:31:14.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.971+0000 7f1272fe4740 1 Processor -- start 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea1f20 0x556bedebdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bedea2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bede75bb0 con 0x556bede8b650 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556bedd94690 con 0x556bedea1f20 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556bede74fa0 con 0x556bedebee00 2024-04-24T06:31:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f1272fe4740 10 client.0 _ll_register_callbacks cb 0x556beddd0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36270/0 (socket says 172.21.15.59:36270) 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 -- 172.21.15.59:0/2525921943 learned_addr learned my addr 172.21.15.59:0/2525921943 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126d800640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea1f20 0x556bedebdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126ce00640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bedea2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 -- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 msgr2=0x556bedea2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bedea2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 -- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea1f20 msgr2=0x556bedebdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.972+0000 7f126e200640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea1f20 0x556bedebdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f126e200640 1 -- 172.21.15.59:0/2525921943 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bedd93920 con 0x556bede8b650 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f126e200640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f126002ad00 tx=0x7f1260004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c28de61add7138 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f124ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260041030 con 0x556bede8b650 2024-04-24T06:31:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1260004580 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1254024070 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f126004a050 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f1272fe4740 10 client.5961 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.973+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x556bedbb77d0 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1254024070 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f126004d020 con 0x556bede8b650 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 10 client.5961 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 10 client.5961 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 20 client.5961 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede8e800 0x556bede8ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 4 client.5961 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:31:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.974+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bedde8c10 con 0x556bede8e800 2024-04-24T06:31:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.975+0000 7f126d800640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede8e800 0x556bede8ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.976+0000 7f126d800640 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede8e800 0x556bede8ec70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5fa040daa5b6ff7 server_cookie=931f5c68935c9229 in_seq=0 out_seq=0 2024-04-24T06:31:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.976+0000 7f124ea00640 10 client.5961 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f124ea00640 1 -- 172.21.15.59:0/2525921943 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bedde8c10 con 0x556bede8e800 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f124ea00640 10 client.5961 handle_command_reply: tid=0 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 client.5961 shutdown 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 msgr2=0x556bede8bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f126002ad00 tx=0x7f1260004ab0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 shutdown_connections 2024-04-24T06:31:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bede8bad0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede8e800 0x556bede8ec70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bedea2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 --2- 172.21.15.59:0/2525921943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedea1f20 0x556bedebdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 >> 172.21.15.59:0/2525921943 conn(0x556bedea17e0 msgr2=0x556bedebc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f124ea00640 0 client.5961 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 shutdown_connections 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 1 -- 172.21.15.59:0/2525921943 wait complete. 2024-04-24T06:31:14.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.978+0000 7f1272fe4740 20 client.5961 trim_cache size 0 max 16384 2024-04-24T06:31:14.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.998+0000 7f1272fe4740 1 Processor -- start 2024-04-24T06:31:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.998+0000 7f1272fe4740 1 -- start start 2024-04-24T06:31:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedea1f20 0x556bedebc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 0x556bedebc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1268200c10 con 0x556bede8b650 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12681ffc60 con 0x556bedebee00 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f126815fe70 con 0x556bedea1f20 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36274/0 (socket says 172.21.15.59:36274) 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126d800640 1 -- 172.21.15.59:0/3219894484 learned_addr learned my addr 172.21.15.59:0/3219894484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126e200640 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 0x556bedebc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126d800640 1 -- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedea1f20 msgr2=0x556bedebc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:14.999+0000 7f126d800640 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedea1f20 0x556bedebc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f126d800640 1 -- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 msgr2=0x556bedebc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f126d800640 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 0x556bedebc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f126d800640 1 -- 172.21.15.59:0/3219894484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bede74b40 con 0x556bede8b650 2024-04-24T06:31:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f126e200640 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 0x556bedebc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f126d800640 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f125c00cdd0 tx=0x7f125c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c933a6e99781d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f1259600640 1 -- 172.21.15.59:0/3219894484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c056e30 con 0x556bede8b650 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f1259600640 1 -- 172.21.15.59:0/3219894484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c057090 con 0x556bede8b650 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.000+0000 7f1259600640 1 -- 172.21.15.59:0/3219894484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c003bc0 con 0x556bede8b650 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 -- 172.21.15.59:0/3219894484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 msgr2=0x556bedebe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f125c00cdd0 tx=0x7f125c01e040 comp rx=0 tx=0).stop 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 -- 172.21.15.59:0/3219894484 shutdown_connections 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bedebee00 0x556bedebc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedea1f20 0x556bedebc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3219894484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bede8b650 0x556bedebe550 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 -- 172.21.15.59:0/3219894484 >> 172.21.15.59:0/3219894484 conn(0x556bedea17e0 msgr2=0x556bedea2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 -- 172.21.15.59:0/3219894484 shutdown_connections 2024-04-24T06:31:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.001+0000 7f1272fe4740 1 -- 172.21.15.59:0/3219894484 wait complete. 2024-04-24T06:31:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.002+0000 7f1272fe4740 1 -- start start 2024-04-24T06:31:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.002+0000 7f1272fe4740 1 Processor -- start 2024-04-24T06:31:15.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bede8b650 0x556bedf35860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede70740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556bede75040 con 0x556bedea1f20 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12681604c0 con 0x556bede8b650 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556bedea2bb0 con 0x556bedebee00 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f1272fe4740 10 client.0 _ll_register_callbacks cb 0x7f1260034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.003+0000 7f126ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36282/0 (socket says 172.21.15.59:36282) 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 -- 172.21.15.59:0/3185144165 learned_addr learned my addr 172.21.15.59:0/3185144165 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126d800640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bede8b650 0x556bedf35860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126e200640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede70740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 -- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 msgr2=0x556bede70740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede70740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 -- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bede8b650 msgr2=0x556bedf35860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bede8b650 0x556bedf35860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 -- 172.21.15.59:0/3185144165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bedaed3f0 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126e200640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede70740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f126ce00640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f125400eb70 tx=0x7f12540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8950800ee2cf09bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.004+0000 7f124ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254006040 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1254033ae0 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c057090 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f125401a020 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 10 client.5967 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x556bedaad0f0 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c057090 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f1254033ce0 con 0x556bedea1f20 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 10 client.5967 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 10 client.5967 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 20 client.5967 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede70c80 0x556bedebbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:15.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 4 client.5967 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:31:15.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.005+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bedde2a80 con 0x556bede70c80 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.006+0000 7f126d800640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede70c80 0x556bedebbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.007+0000 7f126d800640 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede70c80 0x556bedebbdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923a12cd7c6af970 server_cookie=c80c8a593e6ee66a in_seq=0 out_seq=0 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.007+0000 7f124ea00640 10 client.5967 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.007+0000 7f124ea00640 1 -- 172.21.15.59:0/3185144165 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x556bedde2a80 con 0x556bede70c80 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.007+0000 7f124ea00640 10 client.5967 handle_command_reply: tid=0 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.007+0000 7f1272fe4740 1 client.5967 shutdown 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 msgr2=0x556bede9e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:15.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f125400eb70 tx=0x7f12540046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 shutdown_connections 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x556bede70c80 0x556bedebbdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556bedebee00 0x556bede70740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556bedea1f20 0x556bede9e350 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f124ea00640 0 client.5967 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 --2- 172.21.15.59:0/3185144165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556bede8b650 0x556bedf35860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 >> 172.21.15.59:0/3185144165 conn(0x556bedea17e0 msgr2=0x556bedebe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 shutdown_connections 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 1 -- 172.21.15.59:0/3185144165 wait complete. 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.008+0000 7f1272fe4740 20 client.5967 trim_cache size 0 max 16384 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:15.011 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: "45f4be23-56fa-414d-bee1-c1f36e244672": { 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "45f4be23-56fa-414d-bee1-c1f36e244672", 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:31:15.012 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 msgr2=0x7f124806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 0x7f124806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12681501a0 tx=0x7f1254008040 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 msgr2=0x7f126814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f125c00c410 tx=0x7f125c00c8d0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 shutdown_connections 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f124806c8f0 0x7f124806edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f126814e230 0x7f12680998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12680724d0 0x7f126814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 --2- 172.21.15.59:0/4215269133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126806fb50 0x7f126814d6f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:15.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 >> 172.21.15.59:0/4215269133 conn(0x7f126806bdf0 msgr2=0x7f1268074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:15.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 shutdown_connections 2024-04-24T06:31:15.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.009+0000 7f126ec00640 1 -- 172.21.15.59:0/4215269133 wait complete. 2024-04-24T06:31:15.027 INFO:tasks.cephfs.filesystem:scrub status for tag:45f4be23-56fa-414d-bee1-c1f36e244672 - {'path': '/', 'tag': '45f4be23-56fa-414d-bee1-c1f36e244672', 'options': 'recursive,repair'} 2024-04-24T06:31:15.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:15.972+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-24T06:31:14.735223+0000) ==== 1078+0+0 (secure 0 0 0) 0x7fd9ac0573b0 con 0x7fd9bc0d1960 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: id: fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 2m), standbys: x 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:15.974 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 24 objects, 579 KiB 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 224 MiB used, 720 GiB / 720 GiB avail 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.735223+0000 mds.b [INF] scrub queued for path: / 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.735233+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.735248+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-24T06:31:15.975 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.737929+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-24T06:31:15.976 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.737949+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-24T06:31:15.976 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:14.738015+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:31:17.985 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:31:17.447169+0000 mds.b [INF] scrub summary: idle 2024-04-24T06:31:17.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:17.984+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-24T06:31:17.447169+0000) ==== 189+0+0 (secure 0 0 0) 0x7fd9ac057610 con 0x7fd9bc0d1960 2024-04-24T06:31:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:27.028+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 9 ==== log(4 entries from seq 115 at 2024-04-24T06:31:26.034466+0000) ==== 1118+0+0 (secure 0 0 0) 0x7fd9ac089c20 con 0x7fd9bc0d1960 2024-04-24T06:31:28.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:28.031+0000 7fd9b3e00640 1 -- 172.21.15.59:0/2196384577 <== mon.0 v2:172.21.15.59:3300/0 10 ==== log(2 entries from seq 470 at 2024-04-24T06:31:27.012762+0000) ==== 557+0+0 (secure 0 0 0) 0x7fd9ac055980 con 0x7fd9bc0d1960 2024-04-24T06:31:35.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:31:35.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:31:45.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:45.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-24T06:31:45.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.252+0000 7f8a77e00640 1 Processor -- start 2024-04-24T06:31:45.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.252+0000 7f8a77e00640 1 -- start start 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.253+0000 7f8a77e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8a780af750 0x7f8a7806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.253+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a780d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.253+0000 7f8a77e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a780ae6b0 con 0x7f8a7815b220 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.253+0000 7f8a77e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8a780afbe0 con 0x7f8a780af750 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.253+0000 7f8a77e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a780d0f40 con 0x7f8a780d05d0 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a76a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8a7815b220 0x7f8a780adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48876/0 (socket says 172.21.15.59:48876) 2024-04-24T06:31:45.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a76a00640 1 -- 172.21.15.59:0/1263024418 learned_addr learned my addr 172.21.15.59:0/1263024418 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939247248 0 0) 0x7f8a780ae6b0 con 0x7f8a7815b220 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/1263024418 >> v2:172.21.15.66:3300/0 conn(0x7f8a780af750 0x7f8a7806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a5c003520 con 0x7f8a7815b220 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.254+0000 7f8a77400640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a780d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 186049969 0 0) 0x7f8a5c003520 con 0x7f8a7815b220 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 msgr2=0x7f8a780d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a780d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 >> v2:172.21.15.66:3300/0 conn(0x7f8a780af750 msgr2=0x7f8a7806f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 --2- 172.21.15.59:0/1263024418 >> v2:172.21.15.66:3300/0 conn(0x7f8a780af750 0x7f8a7806f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a780ae990 con 0x7f8a7815b220 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8a68001690 con 0x7f8a7815b220 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 >> v1:172.21.15.59:6789/0 conn(0x7f8a7815b220 legacy=0x7f8a780adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a5c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a5c004010 0x7f8a5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5c004d40 0x7f8a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a780d0f40 con 0x7f8a5c004d40 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00dc80 con 0x7f8a5c004010 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c00de80 con 0x7f8a780d05d0 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/1263024418 >> v2:172.21.15.66:3300/0 conn(0x7f8a780af750 0x7f8a7806f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.255+0000 7f8a77400640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5c004d40 0x7f8a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a76a00640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a5c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a5c004010 0x7f8a5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a77400640 1 -- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 msgr2=0x7f8a5c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a77400640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a780d05d0 0x7f8a5c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a77400640 1 -- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a5c004010 msgr2=0x7f8a5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a77400640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a5c004010 0x7f8a5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a77400640 1 -- 172.21.15.59:0/1263024418 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c003520 con 0x7f8a5c004d40 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.256+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a5c004010 0x7f8a5c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.257+0000 7f8a77400640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5c004d40 0x7f8a5c0051c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007360 tx=0x7f8a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78196282b4c1464e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70009be0 con 0x7f8a5c004d40 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a70036600 con 0x7f8a5c004d40 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a76000640 1 -- 172.21.15.59:0/1263024418 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700369d0 con 0x7f8a5c004d40 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 -- 172.21.15.59:0/1263024418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5c004d40 msgr2=0x7f8a5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 --2- 172.21.15.59:0/1263024418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a5c004d40 0x7f8a5c0051c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007360 tx=0x7f8a5c005830 comp rx=0 tx=0).stop 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77400640 1 -- 172.21.15.59:0/1263024418 reap_dead start 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 -- 172.21.15.59:0/1263024418 shutdown_connections 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 -- 172.21.15.59:0/1263024418 >> 172.21.15.59:0/1263024418 conn(0x7f8a7806bdf0 msgr2=0x7f8a7806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 -- 172.21.15.59:0/1263024418 shutdown_connections 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.258+0000 7f8a77e00640 1 -- 172.21.15.59:0/1263024418 wait complete. 2024-04-24T06:31:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.260+0000 7f8a77e00640 1 Processor -- start 2024-04-24T06:31:45.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.261+0000 7f8a77e00640 1 -- start start 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.261+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 0x7f8a7809b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.261+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.261+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a780b38b0 0x7f8a780b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a77e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a780af230 con 0x7f8a781f3fb0 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a77e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a780aef50 con 0x7f8a780b38b0 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a77e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7815b600 con 0x7f8a5c004d40 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50780/0 (socket says 172.21.15.59:50780) 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a6fe00640 1 -- 172.21.15.59:0/3751720726 learned_addr learned my addr 172.21.15.59:0/3751720726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.262+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 0x7f8a7809b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 -- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 msgr2=0x7f8a7809b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 0x7f8a7809b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 -- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a780b38b0 msgr2=0x7f8a780b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a780b38b0 0x7f8a780b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 -- 172.21.15.59:0/3751720726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7815ce30 con 0x7f8a781f3fb0 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 0x7f8a7809b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000f620 tx=0x7f8a6000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6ea00640 1 -- 172.21.15.59:0/3751720726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60014890 con 0x7f8a781f3fb0 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6ea00640 1 -- 172.21.15.59:0/3751720726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a60014a30 con 0x7f8a781f3fb0 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a6ea00640 1 -- 172.21.15.59:0/3751720726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60014de0 con 0x7f8a781f3fb0 2024-04-24T06:31:45.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7815cb50 con 0x7f8a781f3fb0 2024-04-24T06:31:45.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.263+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a780b15e0 con 0x7f8a781f3fb0 2024-04-24T06:31:45.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.266+0000 7f8a6ea00640 1 -- 172.21.15.59:0/3751720726 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a60012070 con 0x7f8a781f3fb0 2024-04-24T06:31:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.267+0000 7f8a6ea00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 0x7f8a5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.267+0000 7f8a6ea00640 1 -- 172.21.15.59:0/3751720726 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8a6008bdf0 con 0x7f8a781f3fb0 2024-04-24T06:31:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.267+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 0x7f8a5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.268+0000 7f8a80418740 1 Processor -- start 2024-04-24T06:31:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.268+0000 7f8a80418740 1 -- start start 2024-04-24T06:31:45.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.268+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 0x7f8a5406ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001eb0 tx=0x7f8a68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea17f20 0x56026ea172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea177e0 0x56026e9ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56026e9511f0 con 0x56026ea34e00 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56026e8f7350 con 0x56026ea177e0 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56026e94d1b0 con 0x56026ea17f20 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50792/0 (socket says 172.21.15.59:50792) 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.269+0000 7f8a76a00640 1 -- 172.21.15.59:0/1971881300 learned_addr learned my addr 172.21.15.59:0/1971881300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 -- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea17f20 msgr2=0x56026ea172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea17f20 0x56026ea172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 -- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea177e0 msgr2=0x56026e9ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea177e0 0x56026e9ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 -- 172.21.15.59:0/1971881300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026e94bf50 con 0x56026ea34e00 2024-04-24T06:31:45.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a76a00640 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8a68010c50 tx=0x7f8a6800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f384da21e00a3fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a67000640 1 -- 172.21.15.59:0/1971881300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68027070 con 0x56026ea34e00 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a67000640 1 -- 172.21.15.59:0/1971881300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a68014e50 con 0x56026ea34e00 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.270+0000 7f8a67000640 1 -- 172.21.15.59:0/1971881300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68013560 con 0x56026ea34e00 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 -- 172.21.15.59:0/1971881300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 msgr2=0x56026e9e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8a68010c50 tx=0x7f8a6800c000 comp rx=0 tx=0).stop 2024-04-24T06:31:45.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 -- 172.21.15.59:0/1971881300 shutdown_connections 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea177e0 0x56026e9ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea17f20 0x56026ea172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 --2- 172.21.15.59:0/1971881300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea34e00 0x56026e9e7790 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 -- 172.21.15.59:0/1971881300 >> 172.21.15.59:0/1971881300 conn(0x56026e9e9a00 msgr2=0x56026ea322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 -- 172.21.15.59:0/1971881300 shutdown_connections 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.271+0000 7f8a80418740 1 -- 172.21.15.59:0/1971881300 wait complete. 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.272+0000 7f8a80418740 1 -- start start 2024-04-24T06:31:45.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.272+0000 7f8a80418740 1 Processor -- start 2024-04-24T06:31:45.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026ea1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026ea45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56026e9ebbb0 con 0x56026ea17f20 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56026e90a690 con 0x56026ea34e00 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56026e9eafa0 con 0x56026ea12e20 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a80418740 10 client.0 _ll_register_callbacks cb 0x56026e946650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50800/0 (socket says 172.21.15.59:50800) 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.273+0000 7f8a76a00640 1 -- 172.21.15.59:0/3758193050 learned_addr learned my addr 172.21.15.59:0/3758193050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a77400640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026ea45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026ea1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 -- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 msgr2=0x56026ea45c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026ea45c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 -- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 msgr2=0x56026ea1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026ea1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 -- 172.21.15.59:0/3758193050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026e909920 con 0x56026ea17f20 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a77400640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026ea45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026ea1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001ce0 tx=0x7f8a780aeb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cf2c05b16bdee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.274+0000 7f8a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68031020 con 0x56026ea17f20 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a68014e50 con 0x56026ea17f20 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8a6802d420 con 0x56026ea17f20 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8a6803b050 con 0x56026ea17f20 2024-04-24T06:31:45.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 10 client.5979 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56026e72d7d0 con 0x56026ea17f20 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a68014e50 con 0x56026ea17f20 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f8a6803e020 con 0x56026ea17f20 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 10 client.5979 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 20 client.5979 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea14040 0x56026ea14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 4 client.5979 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.275+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026e95ec10 con 0x56026ea14040 2024-04-24T06:31:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.276+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea14040 0x56026ea14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.277+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea14040 0x56026ea14440 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe58b7c36bb2000b server_cookie=7ff2e983c9b7d7a5 in_seq=0 out_seq=0 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.277+0000 7f8a5b400640 10 client.5979 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.279+0000 7f8a5b400640 1 -- 172.21.15.59:0/3758193050 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56026e95ec10 con 0x56026ea14040 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.279+0000 7f8a5b400640 10 client.5979 handle_command_reply: tid=0 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.279+0000 7f8a80418740 1 client.5979 shutdown 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 msgr2=0x56026ea43190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001ce0 tx=0x7f8a780aeb60 comp rx=0 tx=0).stop 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 shutdown_connections 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea14040 0x56026ea14440 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026ea45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026ea1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 --2- 172.21.15.59:0/3758193050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026ea43190 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a5b400640 0 client.5979 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 >> 172.21.15.59:0/3758193050 conn(0x56026ea177e0 msgr2=0x56026ea322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 shutdown_connections 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 1 -- 172.21.15.59:0/3758193050 wait complete. 2024-04-24T06:31:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.280+0000 7f8a80418740 20 client.5979 trim_cache size 0 max 16384 2024-04-24T06:31:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 Processor -- start 2024-04-24T06:31:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 -- start start 2024-04-24T06:31:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026e9e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026e9e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78201400 con 0x56026ea17f20 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78200420 con 0x56026ea34e00 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.300+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a781ff7b0 con 0x56026ea12e20 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50810/0 (socket says 172.21.15.59:50810) 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 -- 172.21.15.59:0/2849065406 learned_addr learned my addr 172.21.15.59:0/2849065406 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 -- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 msgr2=0x56026e9e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026e9e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 -- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 msgr2=0x56026e9e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026e9e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 -- 172.21.15.59:0/2849065406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026e6633f0 con 0x56026ea17f20 2024-04-24T06:31:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.301+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d490 tx=0x7f8a6000d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c437c61b42826e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a66600640 1 -- 172.21.15.59:0/2849065406 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60088070 con 0x56026ea17f20 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a66600640 1 -- 172.21.15.59:0/2849065406 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a60057b70 con 0x56026ea17f20 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a66600640 1 -- 172.21.15.59:0/2849065406 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60021070 con 0x56026ea17f20 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 -- 172.21.15.59:0/2849065406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 msgr2=0x56026e9e6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d490 tx=0x7f8a6000d420 comp rx=0 tx=0).stop 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 -- 172.21.15.59:0/2849065406 shutdown_connections 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea34e00 0x56026e9e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea17f20 0x56026e9e6740 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 --2- 172.21.15.59:0/2849065406 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea12e20 0x56026e9e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 -- 172.21.15.59:0/2849065406 >> 172.21.15.59:0/2849065406 conn(0x56026ea177e0 msgr2=0x56026e9ea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.302+0000 7f8a80418740 1 -- 172.21.15.59:0/2849065406 shutdown_connections 2024-04-24T06:31:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.303+0000 7f8a80418740 1 -- 172.21.15.59:0/2849065406 wait complete. 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.303+0000 7f8a80418740 1 -- start start 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.303+0000 7f8a80418740 1 Processor -- start 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 0x56026ea18cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea34e00 0x56026ea27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56026e9e34e0 con 0x56026ea12e20 2024-04-24T06:31:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781ffe00 con 0x56026ea17f20 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56026ea34800 con 0x56026ea34e00 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.304+0000 7f8a80418740 10 client.0 _ll_register_callbacks cb 0x7f8a68025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50822/0 (socket says 172.21.15.59:50822) 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 -- 172.21.15.59:0/3122667254 learned_addr learned my addr 172.21.15.59:0/3122667254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 -- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea34e00 msgr2=0x56026ea27c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a77400640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea34e00 0x56026ea27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 0x56026ea18cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea34e00 0x56026ea27c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 -- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 msgr2=0x56026ea18cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 0x56026ea18cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 -- 172.21.15.59:0/3122667254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026e6230f0 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 0x56026ea18cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a76a00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8a68022800 tx=0x7f8a68001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dc8d1177f321d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.305+0000 7f8a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68044060 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a60057b70 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8a68005030 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8a68031070 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a80418740 10 client.5985 fetch_fsmap learned FSMap version 24 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56026e916810 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a60057b70 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.306+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7f8a6803e040 con 0x56026ea12e20 2024-04-24T06:31:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 10 client.5985 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 20 client.5985 populate_metadata read hostname 'smithi059' 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea124b0 0x56026ea13290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 4 client.5985 mds_command: new command op to 5706 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 --> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56026ea1b4a0 con 0x56026ea124b0 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.307+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea124b0 0x56026ea13290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.308+0000 7f8a6fe00640 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea124b0 0x56026ea13290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4187d66bf3fe4763 server_cookie=e9a19a323aa1ea3c in_seq=0 out_seq=0 2024-04-24T06:31:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.308+0000 7f8a5b400640 10 client.5985 ms_handle_connect on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a5b400640 1 -- 172.21.15.59:0/3122667254 <== mds.0 v2:172.21.15.66:6833/1331520600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56026ea1b4a0 con 0x56026ea124b0 2024-04-24T06:31:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a5b400640 10 client.5985 handle_command_reply: tid=0 2024-04-24T06:31:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 client.5985 shutdown 2024-04-24T06:31:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 msgr2=0x56026e9eb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8a68022800 tx=0x7f8a68001c70 comp rx=0 tx=0).stop 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 shutdown_connections 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:6833/1331520600,v1:172.21.15.66:6835/1331520600] conn(0x56026ea124b0 0x56026ea13290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56026ea34e00 0x56026ea27c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56026ea17f20 0x56026ea18cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a5b400640 0 client.5985 ms_handle_reset on v2:172.21.15.66:6833/1331520600 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 --2- 172.21.15.59:0/3122667254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56026ea12e20 0x56026e9eb8b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 >> 172.21.15.59:0/3122667254 conn(0x56026ea177e0 msgr2=0x56026ea45180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 shutdown_connections 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 1 -- 172.21.15.59:0/3122667254 wait complete. 2024-04-24T06:31:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.309+0000 7f8a80418740 20 client.5985 trim_cache size 0 max 16384 2024-04-24T06:31:45.315 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:45.315 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:31:45.315 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:31:45.315 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 msgr2=0x7f8a5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 0x7f8a5406ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001eb0 tx=0x7f8a68009040 comp rx=0 tx=0).stop 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 msgr2=0x7f8a780b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000f620 tx=0x7f8a6000fae0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 shutdown_connections 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a5406c920 0x7f8a5406ede0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a780b38b0 0x7f8a780b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a781f3fb0 0x7f8a780b1de0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 --2- 172.21.15.59:0/3751720726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8a5c004d40 0x7f8a7809b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 >> 172.21.15.59:0/3751720726 conn(0x7f8a7806bdf0 msgr2=0x7f8a780b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.310+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 shutdown_connections 2024-04-24T06:31:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.311+0000 7f8a77e00640 1 -- 172.21.15.59:0/3751720726 wait complete. 2024-04-24T06:31:45.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:31:45.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:45.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.641+0000 7f264d400640 1 Processor -- start 2024-04-24T06:31:45.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.642+0000 7f264d400640 1 -- start start 2024-04-24T06:31:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.643+0000 7f264d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f264815e7c0 0x7f26480d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.643+0000 7f264d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480d2630 0x7f26480b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.643+0000 7f264d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26480cf030 con 0x7f26480b6530 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.643+0000 7f264d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f26480b7000 con 0x7f264815e7c0 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.643+0000 7f264d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26480cf620 con 0x7f26480d2630 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f264815e7c0 0x7f26480d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f264815e7c0 0x7f26480d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50834/0 (socket says 172.21.15.59:50834) 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f264ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f26480b6530 0x7f26480d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48892/0 (socket says 172.21.15.59:48892) 2024-04-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 -- 172.21.15.59:0/1124473905 learned_addr learned my addr 172.21.15.59:0/1124473905 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647400640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480d2630 0x7f26480b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 -- 172.21.15.59:0/1124473905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480d2630 msgr2=0x7f26480b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480d2630 0x7f26480b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 -- 172.21.15.59:0/1124473905 >> v1:172.21.15.59:6789/0 conn(0x7f26480b6530 legacy=0x7f26480d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.644+0000 7f2647e00640 1 -- 172.21.15.59:0/1124473905 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26480b6a40 con 0x7f264815e7c0 2024-04-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.646+0000 7f2647e00640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f264815e7c0 0x7f26480d20f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2630009870 tx=0x7f263002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfc8fee5eab757c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.646+0000 7f2646a00640 1 -- 172.21.15.59:0/1124473905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630037070 con 0x7f264815e7c0 2024-04-24T06:31:45.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.646+0000 7f2646a00640 1 -- 172.21.15.59:0/1124473905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2630032440 con 0x7f264815e7c0 2024-04-24T06:31:45.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.646+0000 7f2646a00640 1 -- 172.21.15.59:0/1124473905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630032810 con 0x7f264815e7c0 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 -- 172.21.15.59:0/1124473905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f264815e7c0 msgr2=0x7f26480d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f264815e7c0 0x7f26480d20f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2630009870 tx=0x7f263002a250 comp rx=0 tx=0).stop 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 -- 172.21.15.59:0/1124473905 shutdown_connections 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480d2630 0x7f26480b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 --2- 172.21.15.59:0/1124473905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f264815e7c0 0x7f26480d20f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 -- 172.21.15.59:0/1124473905 >> 172.21.15.59:0/1124473905 conn(0x7f264809d920 msgr2=0x7f264808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 -- 172.21.15.59:0/1124473905 shutdown_connections 2024-04-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.647+0000 7f264d400640 1 -- 172.21.15.59:0/1124473905 wait complete. 2024-04-24T06:31:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.649+0000 7f264d400640 1 Processor -- start 2024-04-24T06:31:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.649+0000 7f264d400640 1 -- start start 2024-04-24T06:31:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480b6530 0x7f26481e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f264815e7c0 0x7f264814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26480d1a80 con 0x7f26480d2630 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26480cf2b0 con 0x7f264815e7c0 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f264d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26480d1c00 con 0x7f26480b6530 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50836/0 (socket says 172.21.15.59:50836) 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 -- 172.21.15.59:0/3718108017 learned_addr learned my addr 172.21.15.59:0/3718108017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 -- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480b6530 msgr2=0x7f26481e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480b6530 0x7f26481e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 -- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f264815e7c0 msgr2=0x7f264814ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f264815e7c0 0x7f264814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.650+0000 7f2647400640 1 -- 172.21.15.59:0/3718108017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264814cfb0 con 0x7f26480d2630 2024-04-24T06:31:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f2647400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2638007ec0 tx=0x7f263800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638014620 con 0x7f26480d2630 2024-04-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638019070 con 0x7f26480d2630 2024-04-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26480cc490 con 0x7f26480d2630 2024-04-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638014a00 con 0x7f26480d2630 2024-04-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.651+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26480cca10 con 0x7f26480d2630 2024-04-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.652+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264804fd90 con 0x7f26480d2630 2024-04-24T06:31:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.653+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2638014ba0 con 0x7f26480d2630 2024-04-24T06:31:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.654+0000 7f263fe00640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 0x7f262406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.654+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f263808c4b0 con 0x7f26480d2630 2024-04-24T06:31:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.655+0000 7f2647e00640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 0x7f262406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.655+0000 7f2647e00640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 0x7f262406edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2630034000 tx=0x7f263002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:45.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.660+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f263805a890 con 0x7f26480d2630 2024-04-24T06:31:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.909+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26480b6020 con 0x7f26480d2630 2024-04-24T06:31:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.910+0000 7f263fe00640 1 -- 172.21.15.59:0/3718108017 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f263805a190 con 0x7f26480d2630 2024-04-24T06:31:45.912 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:45.913 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 msgr2=0x7f262406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 0x7f262406edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2630034000 tx=0x7f263002c000 comp rx=0 tx=0).stop 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 msgr2=0x7f264814c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2638007ec0 tx=0x7f263800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 shutdown_connections 2024-04-24T06:31:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262406c8f0 0x7f262406edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f264815e7c0 0x7f264814ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26480d2630 0x7f264814c530 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 --2- 172.21.15.59:0/3718108017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26480b6530 0x7f26481e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 >> 172.21.15.59:0/3718108017 conn(0x7f264809d920 msgr2=0x7f26480cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 shutdown_connections 2024-04-24T06:31:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:45.913+0000 7f264d400640 1 -- 172.21.15.59:0/3718108017 wait complete. 2024-04-24T06:31:45.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-24T06:31:46.186 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:31:46.187 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:31:46.187 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-24T06:31:46.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.297+0000 7fee0de18f40 1 Processor -- start 2024-04-24T06:31:46.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.297+0000 7fee0de18f40 1 -- start start 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46bb4eac0 0x55a46bb6a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46babec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a46ba919c0 con 0x55a46ba8f1d0 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a46ba38cc0 con 0x55a46bb69ad0 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0de18f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a46ba8fb20 con 0x55a46bb4eac0 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50850/0 (socket says 172.21.15.59:50850) 2024-04-24T06:31:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.298+0000 7fee0a400640 1 -- 172.21.15.59:0/2060027354 learned_addr learned my addr 172.21.15.59:0/2060027354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 -- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46bb4eac0 msgr2=0x55a46bb6a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46bb4eac0 0x55a46bb6a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0ae00640 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46babec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 -- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 msgr2=0x55a46babec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46babec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 -- 172.21.15.59:0/2060027354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46bada570 con 0x55a46ba8f1d0 2024-04-24T06:31:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee0a400640 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f80 tx=0x7fedfc050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10b707c08018db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee03e00640 1 -- 172.21.15.59:0/2060027354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc054040 con 0x55a46ba8f1d0 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee03e00640 1 -- 172.21.15.59:0/2060027354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedfc050a20 con 0x55a46ba8f1d0 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.299+0000 7fee03e00640 1 -- 172.21.15.59:0/2060027354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc050e10 con 0x55a46ba8f1d0 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 -- 172.21.15.59:0/2060027354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 msgr2=0x55a46bb65240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000f80 tx=0x7fedfc050000 comp rx=0 tx=0).stop 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 -- 172.21.15.59:0/2060027354 shutdown_connections 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46babec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46bb4eac0 0x55a46bb6a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2060027354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46ba8f1d0 0x55a46bb65240 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 -- 172.21.15.59:0/2060027354 >> 172.21.15.59:0/2060027354 conn(0x55a46badbcc0 msgr2=0x55a46babf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 -- 172.21.15.59:0/2060027354 shutdown_connections 2024-04-24T06:31:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.300+0000 7fee0de18f40 1 -- 172.21.15.59:0/2060027354 wait complete. 2024-04-24T06:31:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.301+0000 7fee0de18f40 1 Processor -- start 2024-04-24T06:31:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.301+0000 7fee0de18f40 1 -- start start 2024-04-24T06:31:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46ba8f1d0 0x55a46bb62080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 0x55a46bb63310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46bb4d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a46babff50 con 0x55a46bb4eac0 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a46bb65fd0 con 0x55a46bb69ad0 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee0de18f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a46bb65a10 con 0x55a46ba8f1d0 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee09a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 0x55a46bb63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee09a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 0x55a46bb63310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50856/0 (socket says 172.21.15.59:50856) 2024-04-24T06:31:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.302+0000 7fee09a00640 1 -- 172.21.15.59:0/2610635618 learned_addr learned my addr 172.21.15.59:0/2610635618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee0a400640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46ba8f1d0 0x55a46bb62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee0ae00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46bb4d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 -- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46ba8f1d0 msgr2=0x55a46bb62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46ba8f1d0 0x55a46bb62080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 -- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 msgr2=0x55a46bb4d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46bb4d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 -- 172.21.15.59:0/2610635618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46bb652e0 con 0x55a46bb4eac0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee0ae00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a46bb69ad0 0x55a46bb4d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee0a400640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a46ba8f1d0 0x55a46bb62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.303+0000 7fee09a00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 0x55a46bb63310 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ae70 tx=0x7fedf8008690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.304+0000 7fee02a00640 1 -- 172.21.15.59:0/2610635618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8008e60 con 0x55a46bb4eac0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.304+0000 7fee02a00640 1 -- 172.21.15.59:0/2610635618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf800c850 con 0x55a46bb4eac0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.304+0000 7fee02a00640 1 -- 172.21.15.59:0/2610635618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf800cc40 con 0x55a46bb4eac0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.304+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a46bb62830 con 0x55a46bb4eac0 2024-04-24T06:31:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.304+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46bb62df0 con 0x55a46bb4eac0 2024-04-24T06:31:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.306+0000 7fee02a00640 1 -- 172.21.15.59:0/2610635618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedf8009040 con 0x55a46bb4eac0 2024-04-24T06:31:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.307+0000 7fee02a00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedec06b300 0x7fedec06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.307+0000 7fee02a00640 1 -- 172.21.15.59:0/2610635618 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fedf808a590 con 0x55a46bb4eac0 2024-04-24T06:31:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.307+0000 7fee0a400640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedec06b300 0x7fedec06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.307+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a46babeef0 0x55a46bac3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.307+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) -- 0x55a46bac3590 con 0x55a46babeef0 2024-04-24T06:31:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.308+0000 7fee0a400640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedec06b300 0x7fedec06d7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fedfc045c50 tx=0x7fedfc051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.308+0000 7fee0ae00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a46babeef0 0x55a46bac3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.309+0000 7fee0ae00640 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a46babeef0 0x55a46bac3050 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0ae00640 1 -- 172.21.15.59:0/2610635618 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fee040075f0 con 0x55a46babeef0 2024-04-24T06:31:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a46babeef0 msgr2=0x55a46bac3050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a46babeef0 0x55a46bac3050 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedec06b300 msgr2=0x7fedec06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedec06b300 0x7fedec06d7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fedfc045c50 tx=0x7fedfc051000 comp rx=0 tx=0).stop 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 msgr2=0x55a46bb63310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 --2- 172.21.15.59:0/2610635618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a46bb4eac0 0x55a46bb63310 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ae70 tx=0x7fedf8008690 comp rx=0 tx=0).stop 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0ae00640 1 -- 172.21.15.59:0/2610635618 reap_dead start 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 shutdown_connections 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.310+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 >> 172.21.15.59:0/2610635618 conn(0x55a46badbcc0 msgr2=0x55a46babf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.311+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 shutdown_connections 2024-04-24T06:31:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.311+0000 7fee0de18f40 1 -- 172.21.15.59:0/2610635618 wait complete. 2024-04-24T06:31:46.317 INFO:teuthology.orchestra.run.smithi059.stdout:Yalphaparent_a 2024-04-24T06:31:46.317 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-24T06:31:46.423 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "alpha", 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "parent_a", 2024-04-24T06:31:46.424 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 7 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 7, 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:31:46.425 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:46.432 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:31:46.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:31:46.540 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:31:46.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:31:46.677 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:31:46.677 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:31:46.677 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:31:46.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.960+0000 7fe905800640 1 Processor -- start 2024-04-24T06:31:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.960+0000 7fe905800640 1 -- start start 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.961+0000 7fe905800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe90015c700 0x7fe9000d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.961+0000 7fe905800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 0x7fe9000b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe905800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9000d0c70 con 0x7fe900160d50 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe905800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe9000b46b0 con 0x7fe90015c700 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe905800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe9000d0f50 con 0x7fe90006c0e0 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ff400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe900160d50 0x7fe90006b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48906/0 (socket says 172.21.15.59:48906) 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ff400640 1 -- 172.21.15.59:0/1329845928 learned_addr learned my addr 172.21.15.59:0/1329845928 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:46.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ffe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe90015c700 0x7fe9000d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8fea00640 1 -- 172.21.15.59:0/1329845928 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124151951 0 0) 0x7fe9000d0c70 con 0x7fe900160d50 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8fea00640 1 -- 172.21.15.59:0/1329845928 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8e4003520 con 0x7fe900160d50 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe904e00640 1 --2- 172.21.15.59:0/1329845928 >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 0x7fe9000b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ffe00640 1 -- 172.21.15.59:0/1329845928 >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 msgr2=0x7fe9000b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/1329845928 >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 0x7fe9000b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ffe00640 1 -- 172.21.15.59:0/1329845928 >> v1:172.21.15.59:6789/0 conn(0x7fe900160d50 legacy=0x7fe90006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.962+0000 7fe8ffe00640 1 -- 172.21.15.59:0/1329845928 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9000b4340 con 0x7fe90015c700 2024-04-24T06:31:46.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.963+0000 7fe904e00640 1 --2- 172.21.15.59:0/1329845928 >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 0x7fe9000b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.964+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/1329845928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90015c700 0x7fe9000d0600 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe900073150 tx=0x7fe8e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa0a5c7f9480527 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.964+0000 7fe8fea00640 1 -- 172.21.15.59:0/1329845928 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8037070 con 0x7fe90015c700 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.964+0000 7fe8fea00640 1 -- 172.21.15.59:0/1329845928 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8032440 con 0x7fe90015c700 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.964+0000 7fe8fea00640 1 -- 172.21.15.59:0/1329845928 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e802ae60 con 0x7fe90015c700 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 -- 172.21.15.59:0/1329845928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90015c700 msgr2=0x7fe9000d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 --2- 172.21.15.59:0/1329845928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90015c700 0x7fe9000d0600 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe900073150 tx=0x7fe8e802a110 comp rx=0 tx=0).stop 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 -- 172.21.15.59:0/1329845928 shutdown_connections 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 --2- 172.21.15.59:0/1329845928 >> v2:172.21.15.66:3300/0 conn(0x7fe90006c0e0 0x7fe9000b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 --2- 172.21.15.59:0/1329845928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90015c700 0x7fe9000d0600 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 -- 172.21.15.59:0/1329845928 >> 172.21.15.59:0/1329845928 conn(0x7fe90009aa00 msgr2=0x7fe9000d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:46.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 -- 172.21.15.59:0/1329845928 shutdown_connections 2024-04-24T06:31:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.965+0000 7fe905800640 1 -- 172.21.15.59:0/1329845928 wait complete. 2024-04-24T06:31:46.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.967+0000 7fe905800640 1 Processor -- start 2024-04-24T06:31:46.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.967+0000 7fe905800640 1 -- start start 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe90015c700 0x7fe90009cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe900160d50 0x7fe90015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9000b5080 con 0x7fe90006c0e0 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001611b0 con 0x7fe90015c700 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe905800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe9000b5200 con 0x7fe900160d50 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50874/0 (socket says 172.21.15.59:50874) 2024-04-24T06:31:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 -- 172.21.15.59:0/2250088427 learned_addr learned my addr 172.21.15.59:0/2250088427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:46.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 -- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe900160d50 msgr2=0x7fe90015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe900160d50 0x7fe90015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 -- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe90015c700 msgr2=0x7fe90009cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.968+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe90015c700 0x7fe90009cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe8ffe00640 1 -- 172.21.15.59:0/2250088427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90009f180 con 0x7fe90006c0e0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe8ffe00640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8037040 con 0x7fe90006c0e0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8042070 con 0x7fe90006c0e0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8030940 con 0x7fe90006c0e0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9001f2260 con 0x7fe90006c0e0 2024-04-24T06:31:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.969+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001f2800 con 0x7fe90006c0e0 2024-04-24T06:31:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.971+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90004fd90 con 0x7fe90006c0e0 2024-04-24T06:31:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.972+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8e8030ae0 con 0x7fe90006c0e0 2024-04-24T06:31:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.973+0000 7fe8f7e00640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.973+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe8e80ac580 con 0x7fe90006c0e0 2024-04-24T06:31:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.973+0000 7fe8ff400640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.974+0000 7fe8ff400640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00012e0 tx=0x7fe8f00015b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:46.978+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8e807a8d0 con 0x7fe90006c0e0 2024-04-24T06:31:47.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.182+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9000b7260 con 0x7fe90006c0e0 2024-04-24T06:31:47.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.184+0000 7fe8f7e00640 1 -- 172.21.15.59:0/2250088427 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16620 (secure 0 0 0) 0x7fe8e807a1d0 con 0x7fe90006c0e0 2024-04-24T06:31:47.187 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:47.187 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:31:28.027248+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:47.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.186+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 msgr2=0x7fe8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.186+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00012e0 tx=0x7fe8f00015b0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.186+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 msgr2=0x7fe90009ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.186+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e80046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 shutdown_connections 2024-04-24T06:31:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe900160d50 0x7fe90015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe90015c700 0x7fe90009cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 --2- 172.21.15.59:0/2250088427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe90006c0e0 0x7fe90009ec40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 >> 172.21.15.59:0/2250088427 conn(0x7fe90009aa00 msgr2=0x7fe900070590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:47.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 shutdown_connections 2024-04-24T06:31:47.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.187+0000 7fe905800640 1 -- 172.21.15.59:0/2250088427 wait complete. 2024-04-24T06:31:47.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:30:56.190519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:30:57.282382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:31:47.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:31:47.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.427+0000 7f1ced400640 1 Processor -- start 2024-04-24T06:31:47.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.427+0000 7f1ced400640 1 -- start start 2024-04-24T06:31:47.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ced400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ce80cd460 0x7f1ce80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ced400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce806bb70 con 0x7f1ce80b0560 2024-04-24T06:31:47.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ced400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce806be50 con 0x7f1ce80d0680 2024-04-24T06:31:47.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ced400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce806c130 con 0x7f1ce80cd460 2024-04-24T06:31:47.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ceca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1ce80b0560 0x7f1ce80ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48910/0 (socket says 172.21.15.59:48910) 2024-04-24T06:31:47.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.429+0000 7f1ceca00640 1 -- 172.21.15.59:0/2148490877 learned_addr learned my addr 172.21.15.59:0/2148490877 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:47.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/2148490877 >> v2:172.21.15.66:3300/0 conn(0x7f1ce80cd460 0x7f1ce80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426076273 0 0) 0x7f1ce806bb70 con 0x7f1ce80b0560 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ccc003520 con 0x7f1ce80b0560 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2185637297 0 0) 0x7f1ccc003520 con 0x7f1ce80b0560 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 >> v2:172.21.15.66:3300/0 conn(0x7f1ce80cd460 msgr2=0x7f1ce80cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 --2- 172.21.15.59:0/2148490877 >> v2:172.21.15.66:3300/0 conn(0x7f1ce80cd460 0x7f1ce80cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 >> v1:172.21.15.66:6789/0 conn(0x7f1ce80d0680 legacy=0x7f1ce80d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce806c410 con 0x7f1ce80b0560 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1ce00019f0 con 0x7f1ce80b0560 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.430+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 >> v1:172.21.15.59:6789/0 conn(0x7f1ce80b0560 legacy=0x7f1ce80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc003a30 0x7f1ccc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 0x7f1ccc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc005230 0x7f1ccc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce806c130 con 0x7f1ccc004800 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc00c190 con 0x7f1ccc005230 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc00c390 con 0x7f1ccc003a30 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 0x7f1ccc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc003a30 0x7f1ccc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 -- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc003a30 msgr2=0x7f1ccc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc003a30 0x7f1ccc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 -- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc005230 msgr2=0x7f1ccc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:47.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ccc005230 0x7f1ccc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1cdfe00640 1 -- 172.21.15.59:0/2148490877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc003520 con 0x7f1ccc004800 2024-04-24T06:31:47.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ceca00640 1 -- 172.21.15.59:0/2148490877 reap_dead start 2024-04-24T06:31:47.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.431+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc003a30 0x7f1ccc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:47.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.432+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 0x7f1ccc004c80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ac0 tx=0x7f1ccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1436fa0bfd3310 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0036070 con 0x7f1ccc004800 2024-04-24T06:31:47.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0031400 con 0x7f1ccc004800 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ce7400640 1 -- 172.21.15.59:0/2148490877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009690 con 0x7f1ccc004800 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ced400640 1 -- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 msgr2=0x7f1ccc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ced400640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 0x7f1ccc004c80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ac0 tx=0x7f1ccc005830 comp rx=0 tx=0).stop 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ced400640 1 -- 172.21.15.59:0/2148490877 shutdown_connections 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ced400640 1 --2- 172.21.15.59:0/2148490877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ccc004800 0x7f1ccc004c80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.433+0000 7f1ced400640 1 -- 172.21.15.59:0/2148490877 >> 172.21.15.59:0/2148490877 conn(0x7f1ce80b2ea0 msgr2=0x7f1ce80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.434+0000 7f1ced400640 1 -- 172.21.15.59:0/2148490877 shutdown_connections 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.434+0000 7f1ced400640 1 -- 172.21.15.59:0/2148490877 wait complete. 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.435+0000 7f1ced400640 1 Processor -- start 2024-04-24T06:31:47.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.435+0000 7f1ced400640 1 -- start start 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc004800 0x7f1ce8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce81617c0 0x7f1ce8161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80b3dc0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80b3c40 con 0x7f1ce81617c0 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.436+0000 7f1ced400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce80cb940 con 0x7f1ccc004800 2024-04-24T06:31:47.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50892/0 (socket says 172.21.15.59:50892) 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 -- 172.21.15.59:0/127640523 learned_addr learned my addr 172.21.15.59:0/127640523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc004800 0x7f1ce8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 -- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc004800 msgr2=0x7f1ce8154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc004800 0x7f1ce8154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 -- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce81617c0 msgr2=0x7f1ce8161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce81617c0 0x7f1ce8161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.437+0000 7f1ceca00640 1 -- 172.21.15.59:0/127640523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce815fa50 con 0x7f1ce80c6d50 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ceca00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80d2300 tx=0x7f1ce000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000a610 con 0x7f1ce80c6d50 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce000a7b0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000ab80 con 0x7f1ce80c6d50 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce815fd30 con 0x7f1ce80c6d50 2024-04-24T06:31:47.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.438+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81602f0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.439+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce804fd90 con 0x7f1ce80c6d50 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.440+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce000ad20 con 0x7f1ce80c6d50 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.442+0000 7f1ce4c00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 0x7f1cc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.442+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f1ce008b3e0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.442+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 0x7f1cc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.443+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 0x7f1cc406ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ce806a7b0 tx=0x7f1cd8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.446+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ce00597d0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.652+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ce80b3400 con 0x7f1ce80c6d50 2024-04-24T06:31:47.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.653+0000 7f1ce4c00640 1 -- 172.21.15.59:0/127640523 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f1ce00590d0 con 0x7f1ce80c6d50 2024-04-24T06:31:47.655 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:31:47.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 msgr2=0x7f1cc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 0x7f1cc406ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ce806a7b0 tx=0x7f1cd8009040 comp rx=0 tx=0).stop 2024-04-24T06:31:47.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 msgr2=0x7f1ce80c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80d2300 tx=0x7f1ce000ccd0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 shutdown_connections 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1cc406c920 0x7f1cc406ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80c6d50 0x7f1ce80c71d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce81617c0 0x7f1ce8161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 --2- 172.21.15.59:0/127640523 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ccc004800 0x7f1ce8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 >> 172.21.15.59:0/127640523 conn(0x7f1ce80b2ea0 msgr2=0x7f1ce80cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 shutdown_connections 2024-04-24T06:31:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.656+0000 7f1ced400640 1 -- 172.21.15.59:0/127640523 wait complete. 2024-04-24T06:31:47.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:31:47.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.894+0000 7f4375800640 1 Processor -- start 2024-04-24T06:31:47.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.894+0000 7f4375800640 1 -- start start 2024-04-24T06:31:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f4375800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f437015e790 0x7f43700aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f4375800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4370071d70 0x7f43700721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f4375800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43700adc80 con 0x7f437006f6f0 2024-04-24T06:31:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f4375800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f43700aef40 con 0x7f437015e790 2024-04-24T06:31:47.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f4375800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f43700afd80 con 0x7f4370071d70 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.896+0000 7f436ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f437015e790 0x7f43700aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f437015e790 0x7f43700aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50898/0 (socket says 172.21.15.59:50898) 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 -- 172.21.15.59:0/3725233316 learned_addr learned my addr 172.21.15.59:0/3725233316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436f400640 1 --2- 172.21.15.59:0/3725233316 >> v2:172.21.15.66:3300/0 conn(0x7f4370071d70 0x7f43700721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 -- 172.21.15.59:0/3725233316 >> v2:172.21.15.66:3300/0 conn(0x7f4370071d70 msgr2=0x7f43700721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 --2- 172.21.15.59:0/3725233316 >> v2:172.21.15.66:3300/0 conn(0x7f4370071d70 0x7f43700721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 -- 172.21.15.59:0/3725233316 >> v1:172.21.15.59:6789/0 conn(0x7f437006f6f0 legacy=0x7f4370162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.897+0000 7f436ea00640 1 -- 172.21.15.59:0/3725233316 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43700adf20 con 0x7f437015e790 2024-04-24T06:31:47.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.898+0000 7f436ea00640 1 --2- 172.21.15.59:0/3725233316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43700aea00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f43700b0050 tx=0x7f436002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5da7afd5e1e699 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.898+0000 7f436e000640 1 -- 172.21.15.59:0/3725233316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360037070 con 0x7f437015e790 2024-04-24T06:31:47.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f436e000640 1 -- 172.21.15.59:0/3725233316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4360032440 con 0x7f437015e790 2024-04-24T06:31:47.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f436e000640 1 -- 172.21.15.59:0/3725233316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360032810 con 0x7f437015e790 2024-04-24T06:31:47.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 -- 172.21.15.59:0/3725233316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 msgr2=0x7f43700aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 --2- 172.21.15.59:0/3725233316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43700aea00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f43700b0050 tx=0x7f436002a280 comp rx=0 tx=0).stop 2024-04-24T06:31:47.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 -- 172.21.15.59:0/3725233316 shutdown_connections 2024-04-24T06:31:47.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 --2- 172.21.15.59:0/3725233316 >> v2:172.21.15.66:3300/0 conn(0x7f4370071d70 0x7f43700721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 --2- 172.21.15.59:0/3725233316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43700aea00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 -- 172.21.15.59:0/3725233316 >> 172.21.15.59:0/3725233316 conn(0x7f4370067d90 msgr2=0x7f437006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:47.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 -- 172.21.15.59:0/3725233316 shutdown_connections 2024-04-24T06:31:47.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.899+0000 7f4375800640 1 -- 172.21.15.59:0/3725233316 wait complete. 2024-04-24T06:31:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.901+0000 7f4375800640 1 Processor -- start 2024-04-24T06:31:47.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.901+0000 7f4375800640 1 -- start start 2024-04-24T06:31:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 0x7f4370145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4370071d70 0x7f43701f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43700ae4e0 con 0x7f437015e790 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43700726e0 con 0x7f437006f6f0 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.902+0000 7f4375800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f437006fbb0 con 0x7f4370071d70 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50906/0 (socket says 172.21.15.59:50906) 2024-04-24T06:31:47.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 -- 172.21.15.59:0/2025097379 learned_addr learned my addr 172.21.15.59:0/2025097379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:47.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436ea00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 0x7f4370145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f4367e00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4370071d70 0x7f43701f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 -- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4370071d70 msgr2=0x7f43701f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4370071d70 0x7f43701f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 -- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 msgr2=0x7f4370145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 0x7f4370145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436f400640 1 -- 172.21.15.59:0/2025097379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43700ce370 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.903+0000 7f436ea00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 0x7f4370145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f436f400640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4368007e40 tx=0x7f436800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368014620 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368019070 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368014a20 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43700ce610 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.904+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43700cebd0 con 0x7f437015e790 2024-04-24T06:31:47.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.905+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370068a80 con 0x7f437015e790 2024-04-24T06:31:47.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.906+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4368014bc0 con 0x7f437015e790 2024-04-24T06:31:47.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.907+0000 7f4366a00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 0x7f434c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:47.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.907+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f436808c4b0 con 0x7f437015e790 2024-04-24T06:31:47.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.907+0000 7f436ea00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 0x7f434c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:47.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.908+0000 7f436ea00640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 0x7f434c06edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4360034000 tx=0x7f436002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:47.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:47.912+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f436805a890 con 0x7f437015e790 2024-04-24T06:31:48.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.124+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4370162190 con 0x7f437015e790 2024-04-24T06:31:48.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.125+0000 7f4366a00640 1 -- 172.21.15.59:0/2025097379 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f436805a190 con 0x7f437015e790 2024-04-24T06:31:48.127 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 msgr2=0x7f434c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 0x7f434c06edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4360034000 tx=0x7f436002e000 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 msgr2=0x7f43701f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4368007e40 tx=0x7f436800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 shutdown_connections 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434c06c8f0 0x7f434c06edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f437015e790 0x7f43701f8340 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4370071d70 0x7f43701f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 --2- 172.21.15.59:0/2025097379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f437006f6f0 0x7f4370145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.127+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 >> 172.21.15.59:0/2025097379 conn(0x7f4370067d90 msgr2=0x7f43700b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.128+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 shutdown_connections 2024-04-24T06:31:48.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.128+0000 7f4375800640 1 -- 172.21.15.59:0/2025097379 wait complete. 2024-04-24T06:31:48.150 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:31:48.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.370+0000 7f88c2800640 1 Processor -- start 2024-04-24T06:31:48.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.370+0000 7f88c2800640 1 -- start start 2024-04-24T06:31:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88c2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88bc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88c2800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88bc0d01c0 con 0x7f88bc163770 2024-04-24T06:31:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88c2800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88bc15f6a0 con 0x7f88bc0d1950 2024-04-24T06:31:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88c2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc15cf00 con 0x7f88bc15cb20 2024-04-24T06:31:48.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88baa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f88bc163770 0x7f88bc0d1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48934/0 (socket says 172.21.15.59:48934) 2024-04-24T06:31:48.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.372+0000 7f88baa00640 1 -- 172.21.15.59:0/3106913519 learned_addr learned my addr 172.21.15.59:0/3106913519 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:48.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88bb400640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88bc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159911674 0 0) 0x7f88bc0d01c0 con 0x7f88bc163770 2024-04-24T06:31:48.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a0003520 con 0x7f88bc163770 2024-04-24T06:31:48.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2919412755 0 0) 0x7f88a0003520 con 0x7f88bc163770 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 msgr2=0x7f88bc0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88bc0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 >> v1:172.21.15.66:6789/0 conn(0x7f88bc0d1950 legacy=0x7f88bc0d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc15f980 con 0x7f88bc163770 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f88a4001670 con 0x7f88bc163770 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 >> v1:172.21.15.59:6789/0 conn(0x7f88bc163770 legacy=0x7f88bc0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.373+0000 7f88ba000640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88ba000640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88a0004d40 0x7f88a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc15cf00 con 0x7f88a0003f90 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88a000bb80 con 0x7f88a0004d40 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f88a000bd80 con 0x7f88bc15cb20 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88bb400640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 -- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 msgr2=0x7f88a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 -- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88a0004d40 msgr2=0x7f88a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88a0004d40 0x7f88a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88baa00640 1 -- 172.21.15.59:0/3106913519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a0003520 con 0x7f88a0003f90 2024-04-24T06:31:48.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.374+0000 7f88bb400640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc15cb20 0x7f88a00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.375+0000 7f88baa00640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88a0004800 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c110 tx=0x7f88a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221f3741d64d23bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4007c40 con 0x7f88a0003f90 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a4005af0 con 0x7f88a0003f90 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88ba000640 1 -- 172.21.15.59:0/3106913519 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4007070 con 0x7f88a0003f90 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88c2800640 1 -- 172.21.15.59:0/3106913519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 msgr2=0x7f88a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88c2800640 1 --2- 172.21.15.59:0/3106913519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88a0004800 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c110 tx=0x7f88a0005830 comp rx=0 tx=0).stop 2024-04-24T06:31:48.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88bbe00640 1 -- 172.21.15.59:0/3106913519 reap_dead start 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88c2800640 1 -- 172.21.15.59:0/3106913519 shutdown_connections 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88c2800640 1 -- 172.21.15.59:0/3106913519 >> 172.21.15.59:0/3106913519 conn(0x7f88bc06cc70 msgr2=0x7f88bc06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.376+0000 7f88c2800640 1 -- 172.21.15.59:0/3106913519 shutdown_connections 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.377+0000 7f88c2800640 1 -- 172.21.15.59:0/3106913519 wait complete. 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.378+0000 7f88c2800640 1 Processor -- start 2024-04-24T06:31:48.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.379+0000 7f88c2800640 1 -- start start 2024-04-24T06:31:48.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88bc1f5090 0x7f88bc159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc1592b0 0x7f88bc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0cf840 con 0x7f88a0003f90 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc160390 con 0x7f88bc1f5090 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88c2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0cf9c0 con 0x7f88bc1592b0 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50924/0 (socket says 172.21.15.59:50924) 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 -- 172.21.15.59:0/2195732365 learned_addr learned my addr 172.21.15.59:0/2195732365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bbe00640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc1592b0 0x7f88bc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 -- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc1592b0 msgr2=0x7f88bc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc1592b0 0x7f88bc0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 -- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88bc1f5090 msgr2=0x7f88bc159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88bc1f5090 0x7f88bc159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.380+0000 7f88bb400640 1 -- 172.21.15.59:0/2195732365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc159740 con 0x7f88a0003f90 2024-04-24T06:31:48.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88bb400640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f350 tx=0x7f88ac00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac003b90 con 0x7f88a0003f90 2024-04-24T06:31:48.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac003d30 con 0x7f88a0003f90 2024-04-24T06:31:48.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac004220 con 0x7f88a0003f90 2024-04-24T06:31:48.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc0c9240 con 0x7f88a0003f90 2024-04-24T06:31:48.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.381+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc0c9800 con 0x7f88a0003f90 2024-04-24T06:31:48.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.382+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc068a80 con 0x7f88a0003f90 2024-04-24T06:31:48.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.383+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88ac01e020 con 0x7f88a0003f90 2024-04-24T06:31:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.384+0000 7f88b3400640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.384+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f88ac08c360 con 0x7f88a0003f90 2024-04-24T06:31:48.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.384+0000 7f88baa00640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.385+0000 7f88baa00640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 0x7f889806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a4032000 tx=0x7f88a4007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.388+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88bc159740 con 0x7f88a0003f90 2024-04-24T06:31:48.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.591+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88bc0d29c0 con 0x7f88a0003f90 2024-04-24T06:31:48.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.592+0000 7f88b3400640 1 -- 172.21.15.59:0/2195732365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f88ac012070 con 0x7f88a0003f90 2024-04-24T06:31:48.594 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 msgr2=0x7f889806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 0x7f889806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a4032000 tx=0x7f88a4007000 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 msgr2=0x7f88bc0ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f350 tx=0x7f88ac00f810 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 shutdown_connections 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88bc1592b0 0x7f88bc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88bc1f5090 0x7f88bc159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.595+0000 7f88c2800640 1 --2- 172.21.15.59:0/2195732365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a0003f90 0x7f88bc0ce680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.596+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 >> 172.21.15.59:0/2195732365 conn(0x7f88bc06cc70 msgr2=0x7f88bc0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.596+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 shutdown_connections 2024-04-24T06:31:48.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.596+0000 7f88c2800640 1 -- 172.21.15.59:0/2195732365 wait complete. 2024-04-24T06:31:48.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:31:48.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.832+0000 7f66aec00640 1 Processor -- start 2024-04-24T06:31:48.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.832+0000 7f66aec00640 1 -- start start 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.833+0000 7f66aec00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66a80d5410 0x7f66a80d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.833+0000 7f66aec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 0x7f66a806d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.833+0000 7f66aec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a815eec0 con 0x7f66a815eae0 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.833+0000 7f66aec00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f66a80d3430 con 0x7f66a80d5410 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.833+0000 7f66aec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f66a80d1ae0 con 0x7f66a80d1670 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66a80d5410 0x7f66a80d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f66a80d5410 0x7f66a80d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50928/0 (socket says 172.21.15.59:50928) 2024-04-24T06:31:48.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 -- 172.21.15.59:0/3880399808 learned_addr learned my addr 172.21.15.59:0/3880399808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66acc00640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 0x7f66a806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 -- 172.21.15.59:0/3880399808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 msgr2=0x7f66a806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 0x7f66a806d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 -- 172.21.15.59:0/3880399808 >> v1:172.21.15.66:6789/0 conn(0x7f66a815eae0 legacy=0x7f66a80d2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66a7400640 1 -- 172.21.15.59:0/3880399808 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a80d3650 con 0x7f66a80d5410 2024-04-24T06:31:48.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.834+0000 7f66acc00640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 0x7f66a806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:48.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.835+0000 7f66a7400640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d5410 0x7f66a80d1130 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6690009ea0 tx=0x7f669002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c434d87fc225623 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.836+0000 7f66a6a00640 1 -- 172.21.15.59:0/3880399808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690036070 con 0x7f66a80d5410 2024-04-24T06:31:48.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.836+0000 7f66a6a00640 1 -- 172.21.15.59:0/3880399808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690031600 con 0x7f66a80d5410 2024-04-24T06:31:48.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.836+0000 7f66a6a00640 1 -- 172.21.15.59:0/3880399808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900319d0 con 0x7f66a80d5410 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 -- 172.21.15.59:0/3880399808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d5410 msgr2=0x7f66a80d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d5410 0x7f66a80d1130 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6690009ea0 tx=0x7f669002a6d0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 -- 172.21.15.59:0/3880399808 shutdown_connections 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d1670 0x7f66a806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 --2- 172.21.15.59:0/3880399808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d5410 0x7f66a80d1130 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 -- 172.21.15.59:0/3880399808 >> 172.21.15.59:0/3880399808 conn(0x7f66a808bf90 msgr2=0x7f66a80d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:48.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 -- 172.21.15.59:0/3880399808 shutdown_connections 2024-04-24T06:31:48.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.837+0000 7f66aec00640 1 -- 172.21.15.59:0/3880399808 wait complete. 2024-04-24T06:31:48.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.839+0000 7f66aec00640 1 Processor -- start 2024-04-24T06:31:48.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.839+0000 7f66aec00640 1 -- start start 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d5410 0x7f66a80c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66a815eae0 0x7f66a80b9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66a806d3a0 con 0x7f66a80d1670 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f66a806cb00 con 0x7f66a815eae0 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66aec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f66a806d030 con 0x7f66a80d5410 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50932/0 (socket says 172.21.15.59:50932) 2024-04-24T06:31:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 -- 172.21.15.59:0/267452978 learned_addr learned my addr 172.21.15.59:0/267452978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 -- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d5410 msgr2=0x7f66a80c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d5410 0x7f66a80c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.841+0000 7f66a7e00640 1 -- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66a815eae0 msgr2=0x7f66a80b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f66a7e00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66a815eae0 0x7f66a80b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f66a7e00640 1 -- 172.21.15.59:0/267452978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a80b7550 con 0x7f66a80d1670 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f66a7e00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f669800b420 tx=0x7f669800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698011830 con 0x7f66a80d1670 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698016070 con 0x7f66a80d1670 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81f8f60 con 0x7f66a80d1670 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698011c10 con 0x7f66a80d1670 2024-04-24T06:31:48.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.842+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a80b7af0 con 0x7f66a80d1670 2024-04-24T06:31:48.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.844+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a8068a80 con 0x7f66a80d1670 2024-04-24T06:31:48.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.844+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f669801b460 con 0x7f66a80d1670 2024-04-24T06:31:48.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.845+0000 7f669fe00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:48.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.845+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f669808b680 con 0x7f66a80d1670 2024-04-24T06:31:48.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.846+0000 7f66a7400640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:48.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.846+0000 7f66a7400640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6690033000 tx=0x7f669002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:48.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:48.850+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6698020020 con 0x7f66a80d1670 2024-04-24T06:31:49.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.066+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66a815e2e0 con 0x7f66a80d1670 2024-04-24T06:31:49.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.067+0000 7f669fe00640 1 -- 172.21.15.59:0/267452978 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f66980599d0 con 0x7f66a80d1670 2024-04-24T06:31:49.069 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.069+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 msgr2=0x7f668406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.069+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6690033000 tx=0x7f669002e000 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.069+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 msgr2=0x7f66a8153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.069+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f669800b420 tx=0x7f669800b8e0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 shutdown_connections 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66a815eae0 0x7f66a80b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f66a80d5410 0x7f66a80c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 --2- 172.21.15.59:0/267452978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66a80d1670 0x7f66a8153550 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 >> 172.21.15.59:0/267452978 conn(0x7f66a808bf90 msgr2=0x7f66a80cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 shutdown_connections 2024-04-24T06:31:49.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.070+0000 7f66aec00640 1 -- 172.21.15.59:0/267452978 wait complete. 2024-04-24T06:31:49.089 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:31:49.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:49.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:31:49.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50814]: fuse finished with error 0 and tester_r 0 2024-04-24T06:31:49.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:31:49.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:49.166 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:31:49.272 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:31:49.272 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:31:49.272 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:31:49.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:31:49.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:31:49.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:49.314 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:31:49.407 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:31:49.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:31:49.408 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:31:49.408 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:31:49.408 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:31:49.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:31:49.460 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:31:49.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:49.461 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:31:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:31:49.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:31:49.557 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:31:49.557 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:31:49.557 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:31:49.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:31:49.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:31:49.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:31:49.619 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:31:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:31:49.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:31:49.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:49.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.893+0000 7fb5fd200640 1 Processor -- start 2024-04-24T06:31:49.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.894+0000 7fb5fd200640 1 -- start start 2024-04-24T06:31:49.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.895+0000 7fb5fd200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5f815e7c0 0x7fb5f80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.895+0000 7fb5fd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb5f80d2630 0x7fb5f80b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.895+0000 7fb5fd200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f80cf030 con 0x7fb5f80b6530 2024-04-24T06:31:49.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.895+0000 7fb5fd200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f80b7000 con 0x7fb5f815e7c0 2024-04-24T06:31:49.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.895+0000 7fb5fd200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f80cf620 con 0x7fb5f80d2630 2024-04-24T06:31:49.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5f815e7c0 0x7fb5f80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:49.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5f815e7c0 0x7fb5f80d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56154/0 (socket says 172.21.15.59:56154) 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 -- 172.21.15.59:0/2422746271 learned_addr learned my addr 172.21.15.59:0/2422746271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f6a00640 1 --2- 172.21.15.59:0/2422746271 >> v2:172.21.15.66:3300/0 conn(0x7fb5f80d2630 0x7fb5f80b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 -- 172.21.15.59:0/2422746271 >> v2:172.21.15.66:3300/0 conn(0x7fb5f80d2630 msgr2=0x7fb5f80b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 --2- 172.21.15.59:0/2422746271 >> v2:172.21.15.66:3300/0 conn(0x7fb5f80d2630 0x7fb5f80b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 -- 172.21.15.59:0/2422746271 >> v1:172.21.15.66:6789/0 conn(0x7fb5f80b6530 legacy=0x7fb5f80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.896+0000 7fb5f7400640 1 -- 172.21.15.59:0/2422746271 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f80b6a40 con 0x7fb5f815e7c0 2024-04-24T06:31:49.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.897+0000 7fb5f7400640 1 --2- 172.21.15.59:0/2422746271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f815e7c0 0x7fb5f80d20f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80d55c0 tx=0x7fb5e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9012ecb154c32a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:49.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.898+0000 7fb5f6000640 1 -- 172.21.15.59:0/2422746271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8037070 con 0x7fb5f815e7c0 2024-04-24T06:31:49.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.898+0000 7fb5f6000640 1 -- 172.21.15.59:0/2422746271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e8032440 con 0x7fb5f815e7c0 2024-04-24T06:31:49.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.898+0000 7fb5f6000640 1 -- 172.21.15.59:0/2422746271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8032810 con 0x7fb5f815e7c0 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.898+0000 7fb5fd200640 1 -- 172.21.15.59:0/2422746271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f815e7c0 msgr2=0x7fb5f80d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.898+0000 7fb5fd200640 1 --2- 172.21.15.59:0/2422746271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f815e7c0 0x7fb5f80d20f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80d55c0 tx=0x7fb5e802a270 comp rx=0 tx=0).stop 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 -- 172.21.15.59:0/2422746271 shutdown_connections 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 --2- 172.21.15.59:0/2422746271 >> v2:172.21.15.66:3300/0 conn(0x7fb5f80d2630 0x7fb5f80b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 --2- 172.21.15.59:0/2422746271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f815e7c0 0x7fb5f80d20f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 -- 172.21.15.59:0/2422746271 >> 172.21.15.59:0/2422746271 conn(0x7fb5f809d920 msgr2=0x7fb5f808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 -- 172.21.15.59:0/2422746271 shutdown_connections 2024-04-24T06:31:49.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.899+0000 7fb5fd200640 1 -- 172.21.15.59:0/2422746271 wait complete. 2024-04-24T06:31:49.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.900+0000 7fb5fd200640 1 Processor -- start 2024-04-24T06:31:49.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.900+0000 7fb5fd200640 1 -- start start 2024-04-24T06:31:49.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5f80d2630 0x7fb5f814f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb5f815e7c0 0x7fb5f80cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80d1a80 con 0x7fb5f80b6530 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80cf2b0 con 0x7fb5f80d2630 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.901+0000 7fb5fd200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80d1c00 con 0x7fb5f815e7c0 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56158/0 (socket says 172.21.15.59:56158) 2024-04-24T06:31:49.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 -- 172.21.15.59:0/251890764 learned_addr learned my addr 172.21.15.59:0/251890764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:49.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f6a00640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5f80d2630 0x7fb5f814f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:49.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 -- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb5f815e7c0 msgr2=0x7fb5f80cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb5f815e7c0 0x7fb5f80cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 -- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5f80d2630 msgr2=0x7fb5f814f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:49.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5f80d2630 0x7fb5f814f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 -- 172.21.15.59:0/251890764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f8159e70 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.902+0000 7fb5f7400640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802a2b0 tx=0x7fb5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.903+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8037040 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.903+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e8042070 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.903+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8030880 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.903+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f80cc200 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.903+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f815a8f0 con 0x7fb5f80b6530 2024-04-24T06:31:49.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.904+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f804fd90 con 0x7fb5f80b6530 2024-04-24T06:31:49.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.905+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5e8030a20 con 0x7fb5f80b6530 2024-04-24T06:31:49.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.906+0000 7fb5ef400640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 0x7fb5d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:49.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.906+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb5e80ac4c0 con 0x7fb5f80b6530 2024-04-24T06:31:49.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.906+0000 7fb5f6a00640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 0x7fb5d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:49.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.907+0000 7fb5f6a00640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 0x7fb5d806edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80b3490 tx=0x7fb5e400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:49.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:49.911+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5f8159e70 con 0x7fb5f80b6530 2024-04-24T06:31:50.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.150+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f808c030 con 0x7fb5f80b6530 2024-04-24T06:31:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.151+0000 7fb5ef400640 1 -- 172.21.15.59:0/251890764 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7fb5e807a810 con 0x7fb5f80b6530 2024-04-24T06:31:50.153 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:50.154 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 msgr2=0x7fb5d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 0x7fb5d806edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80b3490 tx=0x7fb5e400b040 comp rx=0 tx=0).stop 2024-04-24T06:31:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 msgr2=0x7fb5f80c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802a2b0 tx=0x7fb5e80046c0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 shutdown_connections 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb5d806c8f0 0x7fb5d806edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb5f815e7c0 0x7fb5f80cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5f80d2630 0x7fb5f814f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 --2- 172.21.15.59:0/251890764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f80b6530 0x7fb5f80c8030 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 >> 172.21.15.59:0/251890764 conn(0x7fb5f809d920 msgr2=0x7fb5f80b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 shutdown_connections 2024-04-24T06:31:50.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.154+0000 7fb5fd200640 1 -- 172.21.15.59:0/251890764 wait complete. 2024-04-24T06:31:50.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.377+0000 7fb77fa00640 1 Processor -- start 2024-04-24T06:31:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.377+0000 7fb77fa00640 1 -- start start 2024-04-24T06:31:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.378+0000 7fb77fa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb77815c450 0x7fb778067760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.378+0000 7fb77fa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb77806c9e0 0x7fb77806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77fa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77815f2e0 con 0x7fb778162510 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77fa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb778073a80 con 0x7fb77815c450 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77fa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb77815f5c0 con 0x7fb77806c9e0 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb778162510 0x7fb778162980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58434/0 (socket says 172.21.15.59:58434) 2024-04-24T06:31:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77da00640 1 -- 172.21.15.59:0/2152905120 learned_addr learned my addr 172.21.15.59:0/2152905120 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb77d000640 1 --2- 172.21.15.59:0/2152905120 >> v2:172.21.15.66:3300/0 conn(0x7fb77815c450 0x7fb778067760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb77806c9e0 0x7fb77806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476678732 0 0) 0x7fb77815f2e0 con 0x7fb778162510 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.379+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb760003520 con 0x7fb778162510 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 677835679 0 0) 0x7fb760003520 con 0x7fb778162510 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb77806c9e0 msgr2=0x7fb77806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb77806c9e0 0x7fb77806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 >> v2:172.21.15.66:3300/0 conn(0x7fb77815c450 msgr2=0x7fb778067760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 --2- 172.21.15.59:0/2152905120 >> v2:172.21.15.66:3300/0 conn(0x7fb77815c450 0x7fb778067760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77815f8a0 con 0x7fb778162510 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb7700019f0 con 0x7fb778162510 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 >> v1:172.21.15.59:6789/0 conn(0x7fb778162510 legacy=0x7fb778162980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7600039b0 0x7fb760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7600051c0 0x7fb76000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb778073a80 con 0x7fb760004800 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb77815f5c0 con 0x7fb7600039b0 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb76000c030 con 0x7fb7600051c0 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb77d000640 1 --2- 172.21.15.59:0/2152905120 >> v2:172.21.15.66:3300/0 conn(0x7fb77815c450 0x7fb778067760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb77806c9e0 0x7fb77806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.380+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb77d000640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7600039b0 0x7fb760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb777e00640 1 -- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7600051c0 msgr2=0x7fb76000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7600051c0 0x7fb76000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb777e00640 1 -- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7600039b0 msgr2=0x7fb760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7600039b0 0x7fb760003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.381+0000 7fb777e00640 1 -- 172.21.15.59:0/2152905120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760003520 con 0x7fb760004800 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.382+0000 7fb777e00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb760004c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bdda066574195f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.382+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764037070 con 0x7fb760004800 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.382+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76402ab10 con 0x7fb760004800 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.382+0000 7fb777400640 1 -- 172.21.15.59:0/2152905120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76400f040 con 0x7fb760004800 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 -- 172.21.15.59:0/2152905120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 msgr2=0x7fb760004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 --2- 172.21.15.59:0/2152905120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb760004c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb760005830 comp rx=0 tx=0).stop 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77da00640 1 -- 172.21.15.59:0/2152905120 reap_dead start 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 -- 172.21.15.59:0/2152905120 shutdown_connections 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 -- 172.21.15.59:0/2152905120 >> 172.21.15.59:0/2152905120 conn(0x7fb778072650 msgr2=0x7fb778072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 -- 172.21.15.59:0/2152905120 shutdown_connections 2024-04-24T06:31:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.383+0000 7fb77fa00640 1 -- 172.21.15.59:0/2152905120 wait complete. 2024-04-24T06:31:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.385+0000 7fb77fa00640 1 Processor -- start 2024-04-24T06:31:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.385+0000 7fb77fa00640 1 -- start start 2024-04-24T06:31:50.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb778152f20 0x7fb7781ea160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7781ea6a0 0x7fb7780af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb77815c830 con 0x7fb760004800 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb77815eef0 con 0x7fb7781ea6a0 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77fa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7780cf860 con 0x7fb778152f20 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56174/0 (socket says 172.21.15.59:56174) 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.386+0000 7fb77d000640 1 -- 172.21.15.59:0/733851073 learned_addr learned my addr 172.21.15.59:0/733851073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 -- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb778152f20 msgr2=0x7fb7781ea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb778152f20 0x7fb7781ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 -- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7781ea6a0 msgr2=0x7fb7780af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7781ea6a0 0x7fb7780af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 -- 172.21.15.59:0/733851073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7781eab40 con 0x7fb760004800 2024-04-24T06:31:50.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77d000640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00f570 tx=0x7fb76c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c018070 con 0x7fb760004800 2024-04-24T06:31:50.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c014890 con 0x7fb760004800 2024-04-24T06:31:50.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7780ce840 con 0x7fb760004800 2024-04-24T06:31:50.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c014c80 con 0x7fb760004800 2024-04-24T06:31:50.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.387+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7780ce460 con 0x7fb760004800 2024-04-24T06:31:50.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.389+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb778068a80 con 0x7fb760004800 2024-04-24T06:31:50.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.389+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb76c01b070 con 0x7fb760004800 2024-04-24T06:31:50.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.390+0000 7fb774c00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 0x7fb75806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.391+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb76c08c070 con 0x7fb760004800 2024-04-24T06:31:50.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.391+0000 7fb777e00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 0x7fb75806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.392+0000 7fb777e00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 0x7fb75806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb76402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.395+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb76c05a330 con 0x7fb760004800 2024-04-24T06:31:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.599+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7780cef40 con 0x7fb760004800 2024-04-24T06:31:50.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.602+0000 7fb774c00640 1 -- 172.21.15.59:0/733851073 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16620 (secure 0 0 0) 0x7fb76c083e20 con 0x7fb760004800 2024-04-24T06:31:50.605 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:50.605 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:31:28.027248+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 msgr2=0x7fb75806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 0x7fb75806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb76402c000 comp rx=0 tx=0).stop 2024-04-24T06:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 msgr2=0x7fb7780cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00f570 tx=0x7fb76c00fa30 comp rx=0 tx=0).stop 2024-04-24T06:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 shutdown_connections 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb75806c8f0 0x7fb75806edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7781ea6a0 0x7fb7780af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb778152f20 0x7fb7781ea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 --2- 172.21.15.59:0/733851073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb760004800 0x7fb7780cf320 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 >> 172.21.15.59:0/733851073 conn(0x7fb778072650 msgr2=0x7fb7780b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 shutdown_connections 2024-04-24T06:31:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.605+0000 7fb77fa00640 1 -- 172.21.15.59:0/733851073 wait complete. 2024-04-24T06:31:50.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:30:56.190519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:30:57.282382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:31:50.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:31:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.839+0000 7fe481800640 1 Processor -- start 2024-04-24T06:31:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.839+0000 7fe481800640 1 -- start start 2024-04-24T06:31:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.840+0000 7fe481800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.840+0000 7fe481800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 0x7fe47c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.840+0000 7fe481800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47c15cab0 con 0x7fe47c1567e0 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.840+0000 7fe481800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe47c160e40 con 0x7fe47c15d810 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.840+0000 7fe481800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c15cd30 con 0x7fe47c0d0d10 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.841+0000 7fe47b400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.841+0000 7fe480e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 0x7fe47c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.841+0000 7fe480e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 0x7fe47c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:48386/0 (socket says 172.21.15.59:48386) 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.841+0000 7fe480e00640 1 -- 172.21.15.59:0/2960820137 learned_addr learned my addr 172.21.15.59:0/2960820137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.841+0000 7fe47b400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:47194/0 (socket says 172.21.15.59:47194) 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.842+0000 7fe47b400640 1 -- 172.21.15.59:0/2960820137 >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 msgr2=0x7fe47c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.842+0000 7fe47b400640 1 --2- 172.21.15.59:0/2960820137 >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 0x7fe47c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.842+0000 7fe47b400640 1 -- 172.21.15.59:0/2960820137 >> v1:172.21.15.66:6789/0 conn(0x7fe47c1567e0 legacy=0x7fe47c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.842+0000 7fe47b400640 1 -- 172.21.15.59:0/2960820137 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c15d010 con 0x7fe47c0d0d10 2024-04-24T06:31:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.844+0000 7fe47b400640 1 --2- 172.21.15.59:0/2960820137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097a0 tx=0x7fe46802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e755565d8c57ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.844+0000 7fe47aa00640 1 -- 172.21.15.59:0/2960820137 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468037070 con 0x7fe47c0d0d10 2024-04-24T06:31:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe47aa00640 1 -- 172.21.15.59:0/2960820137 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe468032440 con 0x7fe47c0d0d10 2024-04-24T06:31:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe47aa00640 1 -- 172.21.15.59:0/2960820137 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468032810 con 0x7fe47c0d0d10 2024-04-24T06:31:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 -- 172.21.15.59:0/2960820137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 msgr2=0x7fe47c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 --2- 172.21.15.59:0/2960820137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097a0 tx=0x7fe46802a180 comp rx=0 tx=0).stop 2024-04-24T06:31:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 -- 172.21.15.59:0/2960820137 shutdown_connections 2024-04-24T06:31:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 --2- 172.21.15.59:0/2960820137 >> v2:172.21.15.66:3300/0 conn(0x7fe47c15d810 0x7fe47c160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 --2- 172.21.15.59:0/2960820137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c156bc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 -- 172.21.15.59:0/2960820137 >> 172.21.15.59:0/2960820137 conn(0x7fe47c06bdf0 msgr2=0x7fe47c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 -- 172.21.15.59:0/2960820137 shutdown_connections 2024-04-24T06:31:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.845+0000 7fe481800640 1 -- 172.21.15.59:0/2960820137 wait complete. 2024-04-24T06:31:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.847+0000 7fe481800640 1 Processor -- start 2024-04-24T06:31:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.848+0000 7fe481800640 1 -- start start 2024-04-24T06:31:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.848+0000 7fe481800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c1f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe481800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe47c1567e0 0x7fe47c14e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe481800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe481800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c1607d0 con 0x7fe47c15d810 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe481800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c160650 con 0x7fe47c1567e0 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe481800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c160950 con 0x7fe47c0d0d10 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56176/0 (socket says 172.21.15.59:56176) 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 -- 172.21.15.59:0/1873253934 learned_addr learned my addr 172.21.15.59:0/1873253934 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe47b400640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe47c1567e0 0x7fe47c14e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe47be00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c1f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 -- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 msgr2=0x7fe47c1f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c1f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 -- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe47c1567e0 msgr2=0x7fe47c14e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe47c1567e0 0x7fe47c14e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.849+0000 7fe480e00640 1 -- 172.21.15.59:0/1873253934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c1f8540 con 0x7fe47c15d810 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe480e00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe47400e870 tx=0x7fe47400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474017070 con 0x7fe47c15d810 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47400c8e0 con 0x7fe47c15d810 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47400ccd0 con 0x7fe47c15d810 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c1f87b0 con 0x7fe47c15d810 2024-04-24T06:31:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.850+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c1f8d70 con 0x7fe47c15d810 2024-04-24T06:31:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.851+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47c04fd90 con 0x7fe47c15d810 2024-04-24T06:31:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.853+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe47401a070 con 0x7fe47c15d810 2024-04-24T06:31:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.853+0000 7fe473e00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 0x7fe45c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.853+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe47408bb30 con 0x7fe47c15d810 2024-04-24T06:31:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.854+0000 7fe47be00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 0x7fe45c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.854+0000 7fe47be00640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 0x7fe45c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe46c001d20 tx=0x7fe46c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:50.858+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe474059f10 con 0x7fe47c15d810 2024-04-24T06:31:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.067+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe47c0b5150 con 0x7fe47c15d810 2024-04-24T06:31:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.067+0000 7fe473e00640 1 -- 172.21.15.59:0/1873253934 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fe474059810 con 0x7fe47c15d810 2024-04-24T06:31:51.069 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:31:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 msgr2=0x7fe45c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 0x7fe45c06edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe46c001d20 tx=0x7fe46c009040 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 msgr2=0x7fe47c14ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe47400e870 tx=0x7fe47400ed30 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 shutdown_connections 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe45c06c8f0 0x7fe45c06edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe47c15d810 0x7fe47c14ec20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe47c1567e0 0x7fe47c14e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 --2- 172.21.15.59:0/1873253934 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe47c0d0d10 0x7fe47c1f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 >> 172.21.15.59:0/1873253934 conn(0x7fe47c06bdf0 msgr2=0x7fe47c14b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 shutdown_connections 2024-04-24T06:31:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.070+0000 7fe481800640 1 -- 172.21.15.59:0/1873253934 wait complete. 2024-04-24T06:31:51.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:31:51.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.299+0000 7fd618c00640 1 Processor -- start 2024-04-24T06:31:51.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.299+0000 7fd618c00640 1 -- start start 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd618c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd61415ed20 0x7fd6140d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd618c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6140b3600 0x7fd6140d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd618c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6140b5da0 con 0x7fd6140b3f10 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd618c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd61415f100 con 0x7fd61415ed20 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd618c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd6140d0a20 con 0x7fd6140b3600 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd613400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd61415ed20 0x7fd6140d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd613400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd61415ed20 0x7fd6140d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56184/0 (socket says 172.21.15.59:56184) 2024-04-24T06:31:51.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd613400640 1 -- 172.21.15.59:0/581571247 learned_addr learned my addr 172.21.15.59:0/581571247 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:51.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd612a00640 1 --2- 172.21.15.59:0/581571247 >> v2:172.21.15.66:3300/0 conn(0x7fd6140b3600 0x7fd6140d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.301+0000 7fd613400640 1 -- 172.21.15.59:0/581571247 >> v2:172.21.15.66:3300/0 conn(0x7fd6140b3600 msgr2=0x7fd6140d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:51.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.302+0000 7fd613400640 1 --2- 172.21.15.59:0/581571247 >> v2:172.21.15.66:3300/0 conn(0x7fd6140b3600 0x7fd6140d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.302+0000 7fd613400640 1 -- 172.21.15.59:0/581571247 >> v1:172.21.15.66:6789/0 conn(0x7fd6140b3f10 legacy=0x7fd6140b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:51.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.302+0000 7fd613400640 1 -- 172.21.15.59:0/581571247 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6140d2ed0 con 0x7fd61415ed20 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.303+0000 7fd613400640 1 --2- 172.21.15.59:0/581571247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140d0ea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd6140d4520 tx=0x7fd60402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fee13efef2bd96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.303+0000 7fd612000640 1 -- 172.21.15.59:0/581571247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604036070 con 0x7fd61415ed20 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.303+0000 7fd612000640 1 -- 172.21.15.59:0/581571247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604031600 con 0x7fd61415ed20 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.303+0000 7fd612000640 1 -- 172.21.15.59:0/581571247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6040319d0 con 0x7fd61415ed20 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 -- 172.21.15.59:0/581571247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 msgr2=0x7fd6140d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 --2- 172.21.15.59:0/581571247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140d0ea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd6140d4520 tx=0x7fd60402a5d0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 -- 172.21.15.59:0/581571247 shutdown_connections 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 --2- 172.21.15.59:0/581571247 >> v2:172.21.15.66:3300/0 conn(0x7fd6140b3600 0x7fd6140d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 --2- 172.21.15.59:0/581571247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140d0ea0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 -- 172.21.15.59:0/581571247 >> 172.21.15.59:0/581571247 conn(0x7fd61408bf90 msgr2=0x7fd6140d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 -- 172.21.15.59:0/581571247 shutdown_connections 2024-04-24T06:31:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.304+0000 7fd618c00640 1 -- 172.21.15.59:0/581571247 wait complete. 2024-04-24T06:31:51.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.305+0000 7fd618c00640 1 Processor -- start 2024-04-24T06:31:51.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.306+0000 7fd618c00640 1 -- start start 2024-04-24T06:31:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6140b3600 0x7fd614154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6140b3f10 0x7fd6140b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6140d0290 con 0x7fd61415ed20 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd6140d3490 con 0x7fd6140b3600 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd618c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd6140d0410 con 0x7fd6140b3f10 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56190/0 (socket says 172.21.15.59:56190) 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 -- 172.21.15.59:0/1435280042 learned_addr learned my addr 172.21.15.59:0/1435280042 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd612a00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6140b3f10 0x7fd6140b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613400640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6140b3600 0x7fd614154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 -- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6140b3f10 msgr2=0x7fd6140b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6140b3f10 0x7fd6140b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 -- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6140b3600 msgr2=0x7fd614154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.307+0000 7fd613e00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6140b3600 0x7fd614154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd613e00640 1 -- 172.21.15.59:0/1435280042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd614155010 con 0x7fd61415ed20 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd613e00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c390 tx=0x7fd60c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd61415ed20 2024-04-24T06:31:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd60c0035f0 con 0x7fd61415ed20 2024-04-24T06:31:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0039e0 con 0x7fd61415ed20 2024-04-24T06:31:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6140cb7c0 con 0x7fd61415ed20 2024-04-24T06:31:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.308+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6140cbd80 con 0x7fd61415ed20 2024-04-24T06:31:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.309+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61404fd90 con 0x7fd61415ed20 2024-04-24T06:31:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.311+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd60c01a070 con 0x7fd61415ed20 2024-04-24T06:31:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.312+0000 7fd60b400640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 0x7fd5f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.312+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd60c08aff0 con 0x7fd61415ed20 2024-04-24T06:31:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.312+0000 7fd613400640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 0x7fd5f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.313+0000 7fd613400640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 0x7fd5f006ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd604033000 tx=0x7fd604009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.317+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd60c059380 con 0x7fd61415ed20 2024-04-24T06:31:51.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.518+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6140d51e0 con 0x7fd61415ed20 2024-04-24T06:31:51.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.519+0000 7fd60b400640 1 -- 172.21.15.59:0/1435280042 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7fd60c058c80 con 0x7fd61415ed20 2024-04-24T06:31:51.521 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 msgr2=0x7fd5f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 0x7fd5f006ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd604033000 tx=0x7fd604009000 comp rx=0 tx=0).stop 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 msgr2=0x7fd6140b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c390 tx=0x7fd60c00c850 comp rx=0 tx=0).stop 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 shutdown_connections 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd5f006c920 0x7fd5f006ede0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd61415ed20 0x7fd6140b3000 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6140b3f10 0x7fd6140b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 --2- 172.21.15.59:0/1435280042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6140b3600 0x7fd614154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 >> 172.21.15.59:0/1435280042 conn(0x7fd61408bf90 msgr2=0x7fd6140b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 shutdown_connections 2024-04-24T06:31:51.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.522+0000 7fd618c00640 1 -- 172.21.15.59:0/1435280042 wait complete. 2024-04-24T06:31:51.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:31:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.757+0000 7fd807400640 1 Processor -- start 2024-04-24T06:31:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.757+0000 7fd807400640 1 -- start start 2024-04-24T06:31:51.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd807400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd80015e790 0x7fd8000aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd807400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd800071d70 0x7fd8000721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd807400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8000adc80 con 0x7fd80006f6f0 2024-04-24T06:31:51.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd807400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd8000aef40 con 0x7fd800071d70 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd807400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd8000afd80 con 0x7fd80015e790 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd805400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd800071d70 0x7fd8000721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd805400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd800071d70 0x7fd8000721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56200/0 (socket says 172.21.15.59:56200) 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.759+0000 7fd805400640 1 -- 172.21.15.59:0/2327555034 learned_addr learned my addr 172.21.15.59:0/2327555034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.760+0000 7fd804a00640 1 --2- 172.21.15.59:0/2327555034 >> v2:172.21.15.66:3300/0 conn(0x7fd80015e790 0x7fd8000aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.760+0000 7fd805400640 1 -- 172.21.15.59:0/2327555034 >> v2:172.21.15.66:3300/0 conn(0x7fd80015e790 msgr2=0x7fd8000aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.760+0000 7fd805400640 1 --2- 172.21.15.59:0/2327555034 >> v2:172.21.15.66:3300/0 conn(0x7fd80015e790 0x7fd8000aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.760+0000 7fd805400640 1 -- 172.21.15.59:0/2327555034 >> v1:172.21.15.66:6789/0 conn(0x7fd80006f6f0 legacy=0x7fd800162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:51.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.760+0000 7fd805400640 1 -- 172.21.15.59:0/2327555034 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000adf20 con 0x7fd800071d70 2024-04-24T06:31:51.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.761+0000 7fd805400640 1 --2- 172.21.15.59:0/2327555034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd8000721a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b0050 tx=0x7fd7f802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e118ed37529b6fed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.762+0000 7fd7ff400640 1 -- 172.21.15.59:0/2327555034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8036070 con 0x7fd800071d70 2024-04-24T06:31:51.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.762+0000 7fd7ff400640 1 -- 172.21.15.59:0/2327555034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8031600 con 0x7fd800071d70 2024-04-24T06:31:51.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.762+0000 7fd7ff400640 1 -- 172.21.15.59:0/2327555034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80319d0 con 0x7fd800071d70 2024-04-24T06:31:51.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.762+0000 7fd807400640 1 -- 172.21.15.59:0/2327555034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 msgr2=0x7fd8000721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.762+0000 7fd807400640 1 --2- 172.21.15.59:0/2327555034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd8000721a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b0050 tx=0x7fd7f802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 -- 172.21.15.59:0/2327555034 shutdown_connections 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 --2- 172.21.15.59:0/2327555034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd8000721a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 --2- 172.21.15.59:0/2327555034 >> v2:172.21.15.66:3300/0 conn(0x7fd80015e790 0x7fd8000aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 -- 172.21.15.59:0/2327555034 >> 172.21.15.59:0/2327555034 conn(0x7fd800067d90 msgr2=0x7fd80006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 -- 172.21.15.59:0/2327555034 shutdown_connections 2024-04-24T06:31:51.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.763+0000 7fd807400640 1 -- 172.21.15.59:0/2327555034 wait complete. 2024-04-24T06:31:51.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.765+0000 7fd807400640 1 Processor -- start 2024-04-24T06:31:51.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.765+0000 7fd807400640 1 -- start start 2024-04-24T06:31:51.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd80006f6f0 0x7fd80014a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 0x7fd8000cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000ae4e0 con 0x7fd800071d70 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000726e0 con 0x7fd80015e790 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd807400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd80006fbb0 con 0x7fd80006f6f0 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56204/0 (socket says 172.21.15.59:56204) 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd7ffe00640 1 -- 172.21.15.59:0/4039483047 learned_addr learned my addr 172.21.15.59:0/4039483047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd805400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 0x7fd8000cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.766+0000 7fd804a00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd80006f6f0 0x7fd80014a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 -- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd80006f6f0 msgr2=0x7fd80014a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd80006f6f0 0x7fd80014a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 -- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 msgr2=0x7fd8000cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 0x7fd8000cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 -- 172.21.15.59:0/4039483047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80015f960 con 0x7fd800071d70 2024-04-24T06:31:51.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd805400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 0x7fd8000cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:51.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7ffe00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00b420 tx=0x7fd7ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec011830 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec016070 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec011c10 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80015fc00 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.767+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8001601c0 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.769+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd800068a80 con 0x7fd800071d70 2024-04-24T06:31:51.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.770+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7ec01b460 con 0x7fd800071d70 2024-04-24T06:31:51.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.770+0000 7fd7fcc00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 0x7fd7e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:51.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.771+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd7ec08b6f0 con 0x7fd800071d70 2024-04-24T06:31:51.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.771+0000 7fd804a00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 0x7fd7e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:51.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.772+0000 7fd804a00640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 0x7fd7e006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:51.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.775+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7ec020020 con 0x7fd800071d70 2024-04-24T06:31:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.977+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8000af930 con 0x7fd800071d70 2024-04-24T06:31:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.978+0000 7fd7fcc00640 1 -- 172.21.15.59:0/4039483047 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fd7ec059a40 con 0x7fd800071d70 2024-04-24T06:31:51.980 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:31:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 msgr2=0x7fd7e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 0x7fd7e006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-04-24T06:31:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 msgr2=0x7fd80014a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00b420 tx=0x7fd7ec00b8e0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 shutdown_connections 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd7e006c8f0 0x7fd7e006edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd80015e790 0x7fd8000cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd800071d70 0x7fd80014a000 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 --2- 172.21.15.59:0/4039483047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd80006f6f0 0x7fd80014a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 >> 172.21.15.59:0/4039483047 conn(0x7fd800067d90 msgr2=0x7fd8000b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 shutdown_connections 2024-04-24T06:31:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:51.981+0000 7fd807400640 1 -- 172.21.15.59:0/4039483047 wait complete. 2024-04-24T06:31:52.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.213+0000 7f56b3200640 1 Processor -- start 2024-04-24T06:31:52.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.214+0000 7f56b3200640 1 -- start start 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.215+0000 7f56b3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56b40b5ff0 0x7f56b40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.215+0000 7f56b3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56b40d0d90 0x7f56b40d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.215+0000 7f56b3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b40cf030 con 0x7f56b415e7c0 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.215+0000 7f56b3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f56b40b7000 con 0x7f56b40b5ff0 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.215+0000 7f56b3200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f56b40d1720 con 0x7f56b40d0d90 2024-04-24T06:31:52.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56b40b5ff0 0x7f56b40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56b40b5ff0 0x7f56b40b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56210/0 (socket says 172.21.15.59:56210) 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 -- 172.21.15.59:0/3838673151 learned_addr learned my addr 172.21.15.59:0/3838673151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56b2800640 1 --2- 172.21.15.59:0/3838673151 >> v2:172.21.15.66:3300/0 conn(0x7f56b40d0d90 0x7f56b40d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 -- 172.21.15.59:0/3838673151 >> v2:172.21.15.66:3300/0 conn(0x7f56b40d0d90 msgr2=0x7f56b40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 --2- 172.21.15.59:0/3838673151 >> v2:172.21.15.66:3300/0 conn(0x7f56b40d0d90 0x7f56b40d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 -- 172.21.15.59:0/3838673151 >> v1:172.21.15.66:6789/0 conn(0x7f56b415e7c0 legacy=0x7f56b40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.216+0000 7f56abe00640 1 -- 172.21.15.59:0/3838673151 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b40b6a40 con 0x7f56b40b5ff0 2024-04-24T06:31:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.217+0000 7f56abe00640 1 --2- 172.21.15.59:0/3838673151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40b6450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f56b408da50 tx=0x7f569c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4210fe3f4b050593 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.218+0000 7f56b1400640 1 -- 172.21.15.59:0/3838673151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c037070 con 0x7f56b40b5ff0 2024-04-24T06:31:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.218+0000 7f56b1400640 1 -- 172.21.15.59:0/3838673151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c032440 con 0x7f56b40b5ff0 2024-04-24T06:31:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.218+0000 7f56b1400640 1 -- 172.21.15.59:0/3838673151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c032810 con 0x7f56b40b5ff0 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 -- 172.21.15.59:0/3838673151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 msgr2=0x7f56b40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 --2- 172.21.15.59:0/3838673151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40b6450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f56b408da50 tx=0x7f569c02a230 comp rx=0 tx=0).stop 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 -- 172.21.15.59:0/3838673151 shutdown_connections 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 --2- 172.21.15.59:0/3838673151 >> v2:172.21.15.66:3300/0 conn(0x7f56b40d0d90 0x7f56b40d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 --2- 172.21.15.59:0/3838673151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40b6450 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 -- 172.21.15.59:0/3838673151 >> 172.21.15.59:0/3838673151 conn(0x7f56b409d920 msgr2=0x7f56b408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 -- 172.21.15.59:0/3838673151 shutdown_connections 2024-04-24T06:31:52.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.219+0000 7f56b3200640 1 -- 172.21.15.59:0/3838673151 wait complete. 2024-04-24T06:31:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.221+0000 7f56b3200640 1 Processor -- start 2024-04-24T06:31:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.221+0000 7f56b3200640 1 -- start start 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56b40d0d90 0x7f56b41ea120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56b415e7c0 0x7f56b415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56b40cf2b0 con 0x7f56b40b5ff0 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56b40cf620 con 0x7f56b415e7c0 2024-04-24T06:31:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f56b40d2800 con 0x7f56b40d0d90 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56214/0 (socket says 172.21.15.59:56214) 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 -- 172.21.15.59:0/2234149960 learned_addr learned my addr 172.21.15.59:0/2234149960 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 -- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56b40d0d90 msgr2=0x7f56b41ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56b40d0d90 0x7f56b41ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 -- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56b415e7c0 msgr2=0x7f56b415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56b415e7c0 0x7f56b415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.223+0000 7f56b1e00640 1 -- 172.21.15.59:0/2234149960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b40cc830 con 0x7f56b40b5ff0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56b1e00640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b3f0 tx=0x7f56a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4015020 con 0x7f56b40b5ff0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a4011830 con 0x7f56b40b5ff0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4011c20 con 0x7f56b40b5ff0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b40ccb10 con 0x7f56b40b5ff0 2024-04-24T06:31:52.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.224+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b40cc540 con 0x7f56b40b5ff0 2024-04-24T06:31:52.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.225+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40ccce0 con 0x7f56b40b5ff0 2024-04-24T06:31:52.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.226+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56a4018020 con 0x7f56b40b5ff0 2024-04-24T06:31:52.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.227+0000 7f56aa000640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 0x7f569006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.227+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f56a408b890 con 0x7f56b40b5ff0 2024-04-24T06:31:52.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.228+0000 7f56abe00640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 0x7f569006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.229+0000 7f56abe00640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 0x7f569006ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f569c034000 tx=0x7f569c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.234+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56a401f030 con 0x7f56b40b5ff0 2024-04-24T06:31:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.482+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56b404fd90 con 0x7f56b40b5ff0 2024-04-24T06:31:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.484+0000 7f56aa000640 1 -- 172.21.15.59:0/2234149960 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f56a40172c0 con 0x7f56b40b5ff0 2024-04-24T06:31:52.488 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:52.488 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.486+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 msgr2=0x7f569006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.486+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 0x7f569006ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f569c034000 tx=0x7f569c02c000 comp rx=0 tx=0).stop 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 msgr2=0x7f56b40cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b3f0 tx=0x7f56a400b8b0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 shutdown_connections 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f569006c920 0x7f569006ede0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56b415e7c0 0x7f56b415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56b40d0d90 0x7f56b41ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 --2- 172.21.15.59:0/2234149960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56b40b5ff0 0x7f56b40cc000 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 >> 172.21.15.59:0/2234149960 conn(0x7f56b409d920 msgr2=0x7f56b40b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 shutdown_connections 2024-04-24T06:31:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.487+0000 7f56b3200640 1 -- 172.21.15.59:0/2234149960 wait complete. 2024-04-24T06:31:52.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:52.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.717+0000 7fcf12e00640 1 Processor -- start 2024-04-24T06:31:52.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.717+0000 7fcf12e00640 1 -- start start 2024-04-24T06:31:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.718+0000 7fcf12e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf0c09b410 0x7fcf0c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.718+0000 7fcf12e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.718+0000 7fcf12e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c0b35f0 con 0x7fcf0c15c9d0 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.718+0000 7fcf12e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0cd2d0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.718+0000 7fcf12e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0d13b0 con 0x7fcf0c09b410 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56222/0 (socket says 172.21.15.59:56222) 2024-04-24T06:31:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 -- 172.21.15.59:0/2675650314 learned_addr learned my addr 172.21.15.59:0/2675650314 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf11000640 1 --2- 172.21.15.59:0/2675650314 >> v2:172.21.15.66:3300/0 conn(0x7fcf0c09b410 0x7fcf0c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 -- 172.21.15.59:0/2675650314 >> v2:172.21.15.66:3300/0 conn(0x7fcf0c09b410 msgr2=0x7fcf0c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 --2- 172.21.15.59:0/2675650314 >> v2:172.21.15.66:3300/0 conn(0x7fcf0c09b410 0x7fcf0c0af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 -- 172.21.15.59:0/2675650314 >> v1:172.21.15.59:6789/0 conn(0x7fcf0c15c9d0 legacy=0x7fcf0c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.719+0000 7fcf12400640 1 -- 172.21.15.59:0/2675650314 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0b3890 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.720+0000 7fcf12400640 1 --2- 172.21.15.59:0/2675650314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c06c490 tx=0x7fcf0402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a31286f3a030ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf0be00640 1 -- 172.21.15.59:0/2675650314 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04036070 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf0be00640 1 -- 172.21.15.59:0/2675650314 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04031600 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf0be00640 1 -- 172.21.15.59:0/2675650314 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040319d0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 -- 172.21.15.59:0/2675650314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 msgr2=0x7fcf0c0d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2675650314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c06c490 tx=0x7fcf0402a730 comp rx=0 tx=0).stop 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 -- 172.21.15.59:0/2675650314 shutdown_connections 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2675650314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0d0e70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2675650314 >> v2:172.21.15.66:3300/0 conn(0x7fcf0c09b410 0x7fcf0c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 -- 172.21.15.59:0/2675650314 >> 172.21.15.59:0/2675650314 conn(0x7fcf0c0b3ef0 msgr2=0x7fcf0c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 -- 172.21.15.59:0/2675650314 shutdown_connections 2024-04-24T06:31:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.721+0000 7fcf12e00640 1 -- 172.21.15.59:0/2675650314 wait complete. 2024-04-24T06:31:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.723+0000 7fcf12e00640 1 Processor -- start 2024-04-24T06:31:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.723+0000 7fcf12e00640 1 -- start start 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf0c09b410 0x7fcf0c0caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf0c15c9d0 0x7fcf0c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0ccef0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0af5d0 con 0x7fcf0c15c9d0 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf12e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c15cdb0 con 0x7fcf0c09b410 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56228/0 (socket says 172.21.15.59:56228) 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.724+0000 7fcf11000640 1 -- 172.21.15.59:0/2795969109 learned_addr learned my addr 172.21.15.59:0/2795969109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 -- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf0c09b410 msgr2=0x7fcf0c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf0c09b410 0x7fcf0c0caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 -- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf0c15c9d0 msgr2=0x7fcf0c1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf0c15c9d0 0x7fcf0c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 -- 172.21.15.59:0/2795969109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0cb430 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf11000640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d5f0 tx=0x7fcefc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0125f0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc019070 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0129b0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c0c6750 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.725+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c06cdc0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.726+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c04fd90 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.728+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcefc012b50 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.729+0000 7fcf09600640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 0x7fcef006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.729+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fcefc08bfe0 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.729+0000 7fcf11a00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 0x7fcef006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.730+0000 7fcf11a00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 0x7fcef006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f60 tx=0x7fcf000079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:52.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.733+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf0c0cb430 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.931+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0c0cfc50 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.934+0000 7fcf09600640 1 -- 172.21.15.59:0/2795969109 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16620 (secure 0 0 0) 0x7fcefc083e20 con 0x7fcf0c0d09e0 2024-04-24T06:31:52.936 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:52.937 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:31:28.027248+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 msgr2=0x7fcef006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 0x7fcef006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f60 tx=0x7fcf000079d0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 msgr2=0x7fcf0c0c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d5f0 tx=0x7fcefc00dab0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 shutdown_connections 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef006c8f0 0x7fcef006edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf0c15c9d0 0x7fcf0c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf0c0d09e0 0x7fcf0c0c6210 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 --2- 172.21.15.59:0/2795969109 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf0c09b410 0x7fcf0c0caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 >> 172.21.15.59:0/2795969109 conn(0x7fcf0c0b3ef0 msgr2=0x7fcf0c0713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 shutdown_connections 2024-04-24T06:31:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:52.937+0000 7fcf12e00640 1 -- 172.21.15.59:0/2795969109 wait complete. 2024-04-24T06:31:52.957 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:31:52.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.165+0000 7f10d5c00640 1 Processor -- start 2024-04-24T06:31:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.165+0000 7f10d5c00640 1 -- start start 2024-04-24T06:31:53.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.166+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 0x7f10d009b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.166+0000 7f10d5c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d00b35f0 con 0x7f10d00b2e20 2024-04-24T06:31:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.166+0000 7f10d5c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d00b3890 con 0x7f10d00d09e0 2024-04-24T06:31:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.166+0000 7f10d5c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10d00cd2d0 con 0x7f10d015c9d0 2024-04-24T06:31:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.167+0000 7f10cf400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f10d00b2e20 0x7f10d00af090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58454/0 (socket says 172.21.15.59:58454) 2024-04-24T06:31:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.167+0000 7f10cf400640 1 -- 172.21.15.59:0/3932222211 learned_addr learned my addr 172.21.15.59:0/3932222211 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.167+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 0x7f10d009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.167+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875119388 0 0) 0x7f10d00b35f0 con 0x7f10d00b2e20 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.167+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b0003520 con 0x7f10d00b2e20 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1924913438 0 0) 0x7f10b0003520 con 0x7f10d00b2e20 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 msgr2=0x7f10d009b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 0x7f10d009b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 >> v1:172.21.15.66:6789/0 conn(0x7f10d00d09e0 legacy=0x7f10d00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d00b3b70 con 0x7f10d00b2e20 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f10b8001a10 con 0x7f10d00b2e20 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 >> v1:172.21.15.59:6789/0 conn(0x7f10d00b2e20 legacy=0x7f10d00af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10b0004800 0x7f10b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 0x7f10b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00cd2d0 con 0x7f10b00039e0 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10b000feb0 con 0x7f10b0004800 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10b00100b0 con 0x7f10d015c9d0 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.168+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cfe00640 1 -- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 msgr2=0x7f10b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cf400640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10b0004800 0x7f10b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015c9d0 0x7f10b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cfe00640 1 -- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10b0004800 msgr2=0x7f10b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10b0004800 0x7f10b0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.169+0000 7f10cfe00640 1 -- 172.21.15.59:0/3932222211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b0003520 con 0x7f10b00039e0 2024-04-24T06:31:53.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.170+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10b0003e00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009a90 tx=0x7f10b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317189c864542379 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.170+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0036070 con 0x7f10b00039e0 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.170+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0031400 con 0x7f10b00039e0 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.170+0000 7f10cea00640 1 -- 172.21.15.59:0/3932222211 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0009690 con 0x7f10b00039e0 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.170+0000 7f10d5c00640 1 -- 172.21.15.59:0/3932222211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 msgr2=0x7f10b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3932222211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10b0003e00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009a90 tx=0x7f10b0005830 comp rx=0 tx=0).stop 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5200640 1 -- 172.21.15.59:0/3932222211 reap_dead start 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5c00640 1 -- 172.21.15.59:0/3932222211 shutdown_connections 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5c00640 1 -- 172.21.15.59:0/3932222211 >> 172.21.15.59:0/3932222211 conn(0x7f10d00b3ef0 msgr2=0x7f10d00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5c00640 1 -- 172.21.15.59:0/3932222211 shutdown_connections 2024-04-24T06:31:53.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.171+0000 7f10d5c00640 1 -- 172.21.15.59:0/3932222211 wait complete. 2024-04-24T06:31:53.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.172+0000 7f10d5c00640 1 Processor -- start 2024-04-24T06:31:53.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.173+0000 7f10d5c00640 1 -- start start 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 0x7f10d00c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 0x7f10d015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10d015cdb0 con 0x7f10b00039e0 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00b3280 con 0x7f10d00c8be0 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10d015cf30 con 0x7f10d015f6f0 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56260/0 (socket says 172.21.15.59:56260) 2024-04-24T06:31:53.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 -- 172.21.15.59:0/3426172246 learned_addr learned my addr 172.21.15.59:0/3426172246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:53.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cf400640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 0x7f10d00c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5200640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 0x7f10d015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 -- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 msgr2=0x7f10d00c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 0x7f10d00c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 -- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 msgr2=0x7f10d015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 0x7f10d015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cfe00640 1 -- 172.21.15.59:0/3426172246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d00c9700 con 0x7f10b00039e0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10d5200640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 0x7f10d015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.174+0000 7f10cf400640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 0x7f10d00c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10cfe00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f10c0034040 tx=0x7f10c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0040020 con 0x7f10b00039e0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0003bc0 con 0x7f10b00039e0 2024-04-24T06:31:53.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0036040 con 0x7f10b00039e0 2024-04-24T06:31:53.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d00c99e0 con 0x7f10b00039e0 2024-04-24T06:31:53.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.175+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d00c9fa0 con 0x7f10b00039e0 2024-04-24T06:31:53.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.176+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d004fd90 con 0x7f10b00039e0 2024-04-24T06:31:53.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.178+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10c0043020 con 0x7f10b00039e0 2024-04-24T06:31:53.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.178+0000 7f10c7e00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 0x7f10ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.179+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f10c00abc50 con 0x7f10b00039e0 2024-04-24T06:31:53.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.179+0000 7f10cf400640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 0x7f10ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.179+0000 7f10cf400640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 0x7f10ac06ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10b8002230 tx=0x7f10b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.183+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10c004a070 con 0x7f10b00039e0 2024-04-24T06:31:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.430+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10d015a9d0 con 0x7f10b00039e0 2024-04-24T06:31:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.431+0000 7f10c7e00640 1 -- 172.21.15.59:0/3426172246 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f10c00044e0 con 0x7f10b00039e0 2024-04-24T06:31:53.433 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:53.434 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 msgr2=0x7f10ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 0x7f10ac06ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10b8002230 tx=0x7f10b8009040 comp rx=0 tx=0).stop 2024-04-24T06:31:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 msgr2=0x7f10d015d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f10c0034040 tx=0x7f10c0004730 comp rx=0 tx=0).stop 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 shutdown_connections 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10ac06c920 0x7f10ac06ede0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d00c8be0 0x7f10d015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d015f6f0 0x7f10d00c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 --2- 172.21.15.59:0/3426172246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b00039e0 0x7f10d015d490 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 >> 172.21.15.59:0/3426172246 conn(0x7f10d00b3ef0 msgr2=0x7f10d006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 shutdown_connections 2024-04-24T06:31:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.434+0000 7f10d5c00640 1 -- 172.21.15.59:0/3426172246 wait complete. 2024-04-24T06:31:53.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:31:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.665+0000 7f69c2a00640 1 Processor -- start 2024-04-24T06:31:53.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.666+0000 7f69c2a00640 1 -- start start 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c2a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f69bc15c3d0 0x7f69bc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69bc0cd590 0x7f69bc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c2a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69bc160210 con 0x7f69bc0b4690 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c2a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f69bc1604f0 con 0x7f69bc0cd590 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c2a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f69bc1607d0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.667+0000 7f69c0c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69bc0cd590 0x7f69bc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69bc0cd590 0x7f69bc0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56276/0 (socket says 172.21.15.59:56276) 2024-04-24T06:31:53.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 -- 172.21.15.59:0/1792890296 learned_addr learned my addr 172.21.15.59:0/1792890296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:53.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c1600640 1 --2- 172.21.15.59:0/1792890296 >> v2:172.21.15.66:3300/0 conn(0x7f69bc15c3d0 0x7f69bc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 -- 172.21.15.59:0/1792890296 >> v2:172.21.15.66:3300/0 conn(0x7f69bc15c3d0 msgr2=0x7f69bc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:53.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 --2- 172.21.15.59:0/1792890296 >> v2:172.21.15.66:3300/0 conn(0x7f69bc15c3d0 0x7f69bc0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 -- 172.21.15.59:0/1792890296 >> v1:172.21.15.66:6789/0 conn(0x7f69bc0b4690 legacy=0x7f69bc06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:53.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.668+0000 7f69c0c00640 1 -- 172.21.15.59:0/1792890296 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc160ab0 con 0x7f69bc0cd590 2024-04-24T06:31:53.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.669+0000 7f69c0c00640 1 --2- 172.21.15.59:0/1792890296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc0cd590 0x7f69bc0b4150 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f69bc06d9b0 tx=0x7f69a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9c794d814f50d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69bbe00640 1 -- 172.21.15.59:0/1792890296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8037070 con 0x7f69bc0cd590 2024-04-24T06:31:53.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69bbe00640 1 -- 172.21.15.59:0/1792890296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a8032440 con 0x7f69bc0cd590 2024-04-24T06:31:53.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69bbe00640 1 -- 172.21.15.59:0/1792890296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8032810 con 0x7f69bc0cd590 2024-04-24T06:31:53.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 -- 172.21.15.59:0/1792890296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc0cd590 msgr2=0x7f69bc0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 --2- 172.21.15.59:0/1792890296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc0cd590 0x7f69bc0b4150 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f69bc06d9b0 tx=0x7f69a802a230 comp rx=0 tx=0).stop 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 -- 172.21.15.59:0/1792890296 shutdown_connections 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 --2- 172.21.15.59:0/1792890296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc0cd590 0x7f69bc0b4150 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 --2- 172.21.15.59:0/1792890296 >> v2:172.21.15.66:3300/0 conn(0x7f69bc15c3d0 0x7f69bc0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 -- 172.21.15.59:0/1792890296 >> 172.21.15.59:0/1792890296 conn(0x7f69bc0b21a0 msgr2=0x7f69bc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 -- 172.21.15.59:0/1792890296 shutdown_connections 2024-04-24T06:31:53.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.670+0000 7f69c2a00640 1 -- 172.21.15.59:0/1792890296 wait complete. 2024-04-24T06:31:53.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.672+0000 7f69c2a00640 1 Processor -- start 2024-04-24T06:31:53.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.672+0000 7f69c2a00640 1 -- start start 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69bc0b4690 0x7f69bc0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69bc0cd590 0x7f69bc0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc15c7b0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc0b4ac0 con 0x7f69bc0b4690 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.673+0000 7f69c2a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f69bc15c930 con 0x7f69bc0cd590 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56286/0 (socket says 172.21.15.59:56286) 2024-04-24T06:31:53.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 -- 172.21.15.59:0/2129019343 learned_addr learned my addr 172.21.15.59:0/2129019343 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c0c00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69bc0cd590 0x7f69bc0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c1600640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69bc0b4690 0x7f69bc0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 -- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69bc0cd590 msgr2=0x7f69bc0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69bc0cd590 0x7f69bc0d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 -- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69bc0b4690 msgr2=0x7f69bc0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69bc0b4690 0x7f69bc0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 -- 172.21.15.59:0/2129019343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc0adae0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.674+0000 7f69c2000640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f69b400c780 tx=0x7f69b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.675+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4017070 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.675+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b4003da0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.675+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40052b0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.675+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc0addc0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.675+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc0ae380 con 0x7f69bc15c3d0 2024-04-24T06:31:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.676+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc04fd90 con 0x7f69bc15c3d0 2024-04-24T06:31:53.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.677+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69b401a070 con 0x7f69bc15c3d0 2024-04-24T06:31:53.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.678+0000 7f69b9600640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 0x7f699c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:53.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.678+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f69b408b210 con 0x7f69bc15c3d0 2024-04-24T06:31:53.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.679+0000 7f69c1600640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 0x7f699c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.679+0000 7f69c1600640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 0x7f699c06edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006fd0 tx=0x7f69b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.683+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69b40595f0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.937+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69bc06e400 con 0x7f69bc15c3d0 2024-04-24T06:31:53.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.938+0000 7f69b9600640 1 -- 172.21.15.59:0/2129019343 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f69b4058ef0 con 0x7f69bc15c3d0 2024-04-24T06:31:53.940 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:53.940 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:30:58.292001+0000","modified":"2024-04-24T06:31:01.372687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5706},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5706":{"gid":5706,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1331520600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1331520600},{"type":"v1","addr":"172.21.15.66:6835","nonce":1331520600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5706,"qdb_cluster":[5706]},"id":3}]} 2024-04-24T06:31:53.940 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 msgr2=0x7f699c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 0x7f699c06edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006fd0 tx=0x7f69b0008040 comp rx=0 tx=0).stop 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 msgr2=0x7f69bc0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f69b400c780 tx=0x7f69b400cc40 comp rx=0 tx=0).stop 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 shutdown_connections 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f699c06c8f0 0x7f699c06edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69bc15c3d0 0x7f69bc0d0660 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f69bc0cd590 0x7f69bc0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 --2- 172.21.15.59:0/2129019343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69bc0b4690 0x7f69bc0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 >> 172.21.15.59:0/2129019343 conn(0x7f69bc0b21a0 msgr2=0x7f69bc0afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 shutdown_connections 2024-04-24T06:31:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:53.941+0000 7f69c2a00640 1 -- 172.21.15.59:0/2129019343 wait complete. 2024-04-24T06:31:53.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:54.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.168+0000 7f0c75800640 1 Processor -- start 2024-04-24T06:31:54.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.168+0000 7f0c75800640 1 -- start start 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c75800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c7015e9d0 0x7f0c700d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c75800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c700d02e0 0x7f0c700d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c75800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c700cf520 con 0x7f0c700d0f70 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c75800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0c700cf800 con 0x7f0c700d02e0 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c75800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c700cfae0 con 0x7f0c7015e9d0 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c67e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c700d02e0 0x7f0c700d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c67e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c700d02e0 0x7f0c700d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56292/0 (socket says 172.21.15.59:56292) 2024-04-24T06:31:54.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.170+0000 7f0c67e00640 1 -- 172.21.15.59:0/3710419595 learned_addr learned my addr 172.21.15.59:0/3710419595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:54.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.171+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c7015e9d0 0x7f0c700d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.171+0000 7f0c67e00640 1 -- 172.21.15.59:0/3710419595 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c7015e9d0 msgr2=0x7f0c700d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.171+0000 7f0c67e00640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c7015e9d0 0x7f0c700d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.171+0000 7f0c67e00640 1 -- 172.21.15.59:0/3710419595 >> v1:172.21.15.66:6789/0 conn(0x7f0c700d0f70 legacy=0x7f0c700d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.171+0000 7f0c67e00640 1 -- 172.21.15.59:0/3710419595 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c700cfdc0 con 0x7f0c700d02e0 2024-04-24T06:31:54.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.172+0000 7f0c67e00640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c700d3090 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009870 tx=0x7f0c5802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0463d046589b2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.172+0000 7f0c6f400640 1 -- 172.21.15.59:0/3710419595 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58037070 con 0x7f0c700d02e0 2024-04-24T06:31:54.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.172+0000 7f0c6f400640 1 -- 172.21.15.59:0/3710419595 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c58032440 con 0x7f0c700d02e0 2024-04-24T06:31:54.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.172+0000 7f0c6f400640 1 -- 172.21.15.59:0/3710419595 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58032810 con 0x7f0c700d02e0 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 -- 172.21.15.59:0/3710419595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 msgr2=0x7f0c700d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c700d3090 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009870 tx=0x7f0c5802a210 comp rx=0 tx=0).stop 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 -- 172.21.15.59:0/3710419595 shutdown_connections 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c700d3090 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 --2- 172.21.15.59:0/3710419595 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c7015e9d0 0x7f0c700d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 -- 172.21.15.59:0/3710419595 >> 172.21.15.59:0/3710419595 conn(0x7f0c7008bf90 msgr2=0x7f0c700d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 -- 172.21.15.59:0/3710419595 shutdown_connections 2024-04-24T06:31:54.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.173+0000 7f0c75800640 1 -- 172.21.15.59:0/3710419595 wait complete. 2024-04-24T06:31:54.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.175+0000 7f0c75800640 1 Processor -- start 2024-04-24T06:31:54.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.175+0000 7f0c75800640 1 -- start start 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 0x7f0c700b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c701549c0 0x7f0c7015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7006cc70 con 0x7f0c700d02e0 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7006caf0 con 0x7f0c700d0f70 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c75800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7006cdf0 con 0x7f0c701549c0 2024-04-24T06:31:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56306/0 (socket says 172.21.15.59:56306) 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c6fe00640 1 -- 172.21.15.59:0/1103585677 learned_addr learned my addr 172.21.15.59:0/1103585677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c67e00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 0x7f0c700b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c74e00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c701549c0 0x7f0c7015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c6fe00640 1 -- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c701549c0 msgr2=0x7f0c7015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.176+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c701549c0 0x7f0c7015d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6fe00640 1 -- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 msgr2=0x7f0c700b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 0x7f0c700b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6fe00640 1 -- 172.21.15.59:0/1103585677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c701f46d0 con 0x7f0c700d02e0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c67e00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 0x7f0c700b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6fe00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c7f0 tx=0x7f0c6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60015070 con 0x7f0c700d02e0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c60004220 con 0x7f0c700d02e0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60007620 con 0x7f0c700d02e0 2024-04-24T06:31:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c701f4940 con 0x7f0c700d02e0 2024-04-24T06:31:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.177+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c701f4f00 con 0x7f0c700d02e0 2024-04-24T06:31:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.178+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7004fd90 con 0x7f0c700d02e0 2024-04-24T06:31:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.179+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c600079d0 con 0x7f0c700d02e0 2024-04-24T06:31:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.180+0000 7f0c6cc00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 0x7f0c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.180+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0c6008bbe0 con 0x7f0c700d02e0 2024-04-24T06:31:54.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.181+0000 7f0c67e00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 0x7f0c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.181+0000 7f0c67e00640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 0x7f0c4c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0c58034000 tx=0x7f0c5802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.185+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c60059f30 con 0x7f0c700d02e0 2024-04-24T06:31:54.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.391+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c70161380 con 0x7f0c700d02e0 2024-04-24T06:31:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.394+0000 7f0c6cc00640 1 -- 172.21.15.59:0/1103585677 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16620 (secure 0 0 0) 0x7f0c60059830 con 0x7f0c700d02e0 2024-04-24T06:31:54.397 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:54.397 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:31:28.027248+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:30:56.190519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-24T06:30:57.282382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 msgr2=0x7f0c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 0x7f0c4c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0c58034000 tx=0x7f0c5802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 msgr2=0x7f0c701543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c7f0 tx=0x7f0c6000ccb0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 shutdown_connections 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c4c06c8f0 0x7f0c4c06edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c701549c0 0x7f0c7015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c700d0f70 0x7f0c700b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 --2- 172.21.15.59:0/1103585677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c700d02e0 0x7f0c701543e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 >> 172.21.15.59:0/1103585677 conn(0x7f0c7008bf90 msgr2=0x7f0c70162b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 shutdown_connections 2024-04-24T06:31:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.397+0000 7f0c75800640 1 -- 172.21.15.59:0/1103585677 wait complete. 2024-04-24T06:31:54.420 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:31:54.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.616+0000 7f24fde00640 1 Processor -- start 2024-04-24T06:31:54.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.616+0000 7f24fde00640 1 -- start start 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24fde00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24f815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24fde00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f24f806e250 0x7f24f80cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24fde00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f815dda0 con 0x7f24f806ddf0 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24fde00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f24f80cd880 con 0x7f24f806e250 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24fde00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f24f8071ec0 con 0x7f24f80cf410 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24f6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f24f806ddf0 0x7f24f8072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58470/0 (socket says 172.21.15.59:58470) 2024-04-24T06:31:54.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.618+0000 7f24f6000640 1 -- 172.21.15.59:0/1422037798 learned_addr learned my addr 172.21.15.59:0/1422037798 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f6a00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24f815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f7400640 1 --2- 172.21.15.59:0/1422037798 >> v2:172.21.15.66:3300/0 conn(0x7f24f806e250 0x7f24f80cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341626608 0 0) 0x7f24e0001490 con 0x7f24f806ddf0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24d8003520 con 0x7f24f806ddf0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3969001056 0 0) 0x7f24d8003520 con 0x7f24f806ddf0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 msgr2=0x7f24f815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24f815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 >> v2:172.21.15.66:3300/0 conn(0x7f24f806e250 msgr2=0x7f24f80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 --2- 172.21.15.59:0/1422037798 >> v2:172.21.15.66:3300/0 conn(0x7f24f806e250 0x7f24f80cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f80cdb20 con 0x7f24f806ddf0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f24e0001bc0 con 0x7f24f806ddf0 2024-04-24T06:31:54.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 >> v1:172.21.15.59:6789/0 conn(0x7f24f806ddf0 legacy=0x7f24f8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.619+0000 7f24f5600640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d80039b0 0x7f24d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f5600640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f5600640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d8004d40 0x7f24d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24f80cd880 con 0x7f24d80039b0 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8071ec0 con 0x7f24d8004d40 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f24d800b9c0 con 0x7f24f80cf410 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f6a00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d80039b0 0x7f24d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f6000640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f7400640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d8004d40 0x7f24d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.620+0000 7f24f6a00640 1 -- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 msgr2=0x7f24d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.621+0000 7f24f6a00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80cf410 0x7f24d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.621+0000 7f24f6a00640 1 -- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d8004d40 msgr2=0x7f24d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.621+0000 7f24f6a00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d8004d40 0x7f24d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.621+0000 7f24f6a00640 1 -- 172.21.15.59:0/1422037798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d8003520 con 0x7f24d80039b0 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.622+0000 7f24f6a00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d80039b0 0x7f24d8003dd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ac0 tx=0x7f24d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b89d3dedd597bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.622+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8036070 con 0x7f24d80039b0 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.622+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8031400 con 0x7f24d80039b0 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.622+0000 7f24f5600640 1 -- 172.21.15.59:0/1422037798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8009690 con 0x7f24d80039b0 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 -- 172.21.15.59:0/1422037798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d80039b0 msgr2=0x7f24d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 --2- 172.21.15.59:0/1422037798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d80039b0 0x7f24d8003dd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ac0 tx=0x7f24d8005830 comp rx=0 tx=0).stop 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24f7400640 1 -- 172.21.15.59:0/1422037798 reap_dead start 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 -- 172.21.15.59:0/1422037798 shutdown_connections 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 -- 172.21.15.59:0/1422037798 >> 172.21.15.59:0/1422037798 conn(0x7f24f80b2130 msgr2=0x7f24f80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 -- 172.21.15.59:0/1422037798 shutdown_connections 2024-04-24T06:31:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.623+0000 7f24fde00640 1 -- 172.21.15.59:0/1422037798 wait complete. 2024-04-24T06:31:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.625+0000 7f24fde00640 1 Processor -- start 2024-04-24T06:31:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.625+0000 7f24fde00640 1 -- start start 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24f8145d50 0x7f24f81e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80b4800 0x7f24f8146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24f815a6a0 con 0x7f24f8146130 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24f80cde00 con 0x7f24f8145d50 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24fde00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f24f815a820 con 0x7f24f80b4800 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24f6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24f6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56318/0 (socket says 172.21.15.59:56318) 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.626+0000 7f24f6000640 1 -- 172.21.15.59:0/2715470770 learned_addr learned my addr 172.21.15.59:0/2715470770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 -- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80b4800 msgr2=0x7f24f8146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80b4800 0x7f24f8146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 -- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24f8145d50 msgr2=0x7f24f81e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24f8145d50 0x7f24f81e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 -- 172.21.15.59:0/2715470770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f8159c30 con 0x7f24f8146130 2024-04-24T06:31:54.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24f6000640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f24e0001750 tx=0x7f24e000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0017020 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e000ee00 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0013660 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.627+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f80b3d60 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.628+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f80b4000 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.629+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f8068a80 con 0x7f24f8146130 2024-04-24T06:31:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.630+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e001a020 con 0x7f24f8146130 2024-04-24T06:31:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.631+0000 7f24ef400640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 0x7f24d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.631+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f24e008c4f0 con 0x7f24f8146130 2024-04-24T06:31:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.631+0000 7f24f6a00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 0x7f24d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.632+0000 7f24f6a00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 0x7f24d406ede0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24e8033000 tx=0x7f24e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.635+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24e0021020 con 0x7f24f8146130 2024-04-24T06:31:54.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.888+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f24f80700b0 con 0x7f24f8146130 2024-04-24T06:31:54.890 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:31:54.889+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:31:54.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.893+0000 7f24ef400640 1 -- 172.21.15.59:0/2715470770 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f24f815dda0 con 0x7f24f8146130 2024-04-24T06:31:54.895 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:31:54.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 msgr2=0x7f24d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 0x7f24d406ede0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24e8033000 tx=0x7f24e802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 msgr2=0x7f24f81466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f24e0001750 tx=0x7f24e000e4d0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 shutdown_connections 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24d406c920 0x7f24d406ede0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24f80b4800 0x7f24f8146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f8146130 0x7f24f81466f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 --2- 172.21.15.59:0/2715470770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24f8145d50 0x7f24f81e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 >> 172.21.15.59:0/2715470770 conn(0x7f24f80b2130 msgr2=0x7f24f815df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.896+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 shutdown_connections 2024-04-24T06:31:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:54.897+0000 7f24fde00640 1 -- 172.21.15.59:0/2715470770 wait complete. 2024-04-24T06:31:54.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:31:54.932 INFO:tasks.ceph.mds.b.smithi066.stderr: -5192> 2024-04-24T06:31:03.242+0000 7fb4fde00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:31:54.944 INFO:tasks.ceph.mds.b.smithi066.stderr: -3251> 2024-04-24T06:31:14.736+0000 7fb4f8e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5901=0-4194304@1} caps={5901=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5901 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x555a414eec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:31:55.006 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:31:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.123+0000 7f52e7e00640 1 Processor -- start 2024-04-24T06:31:55.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.123+0000 7f52e7e00640 1 -- start start 2024-04-24T06:31:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.125+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e80af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.125+0000 7f52e7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f52e80d05d0 0x7f52e80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.125+0000 7f52e7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e806fd30 con 0x7f52e806f8d0 2024-04-24T06:31:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.125+0000 7f52e7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f52e80d0f60 con 0x7f52e80d05d0 2024-04-24T06:31:55.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.125+0000 7f52e7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80ae6b0 con 0x7f52e815b220 2024-04-24T06:31:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.126+0000 7f52e7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f52e80d05d0 0x7f52e80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.126+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.126+0000 7f52e7400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f52e80d05d0 0x7f52e80d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:48536/0 (socket says 172.21.15.59:48536) 2024-04-24T06:31:55.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.126+0000 7f52e7400640 1 -- 172.21.15.59:0/4010554891 learned_addr learned my addr 172.21.15.59:0/4010554891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.127+0000 7f52e7400640 1 -- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 msgr2=0x7f52e80af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.127+0000 7f52e7400640 1 --2- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e80af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.127+0000 7f52e7400640 1 -- 172.21.15.59:0/4010554891 >> v1:172.21.15.66:6789/0 conn(0x7f52e806f8d0 legacy=0x7f52e80adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.127+0000 7f52e7400640 1 -- 172.21.15.59:0/4010554891 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80ae990 con 0x7f52e80d05d0 2024-04-24T06:31:55.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.128+0000 7f52e7400640 1 --2- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e80d0a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009e50 tx=0x7f52e002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c015158450438a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:55.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.129+0000 7f52e5600640 1 -- 172.21.15.59:0/4010554891 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0036070 con 0x7f52e80d05d0 2024-04-24T06:31:55.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.129+0000 7f52e5600640 1 -- 172.21.15.59:0/4010554891 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e0031600 con 0x7f52e80d05d0 2024-04-24T06:31:55.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.129+0000 7f52e5600640 1 -- 172.21.15.59:0/4010554891 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e00319d0 con 0x7f52e80d05d0 2024-04-24T06:31:55.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 -- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 msgr2=0x7f52e80d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 --2- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e80d0a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009e50 tx=0x7f52e002a710 comp rx=0 tx=0).stop 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 -- 172.21.15.59:0/4010554891 shutdown_connections 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 --2- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e80d0a20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 --2- 172.21.15.59:0/4010554891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e80af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 -- 172.21.15.59:0/4010554891 >> 172.21.15.59:0/4010554891 conn(0x7f52e806bdf0 msgr2=0x7f52e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 -- 172.21.15.59:0/4010554891 shutdown_connections 2024-04-24T06:31:55.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.130+0000 7f52e7e00640 1 -- 172.21.15.59:0/4010554891 wait complete. 2024-04-24T06:31:55.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.133+0000 7f52e7e00640 1 Processor -- start 2024-04-24T06:31:55.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.133+0000 7f52e7e00640 1 -- start start 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e8143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e8167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80af230 con 0x7f52e806f8d0 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80aef50 con 0x7f52e80d05d0 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.134+0000 7f52e7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f52e815b600 con 0x7f52e815b220 2024-04-24T06:31:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56328/0 (socket says 172.21.15.59:56328) 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 -- 172.21.15.59:0/2852084635 learned_addr learned my addr 172.21.15.59:0/2852084635 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6000640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e8143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e7400640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e8167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 -- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 msgr2=0x7f52e8167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e8167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 -- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 msgr2=0x7f52e8143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e8143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.135+0000 7f52e6a00640 1 -- 172.21.15.59:0/2852084635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e815ea10 con 0x7f52e806f8d0 2024-04-24T06:31:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52e6a00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f52d800e870 tx=0x7f52d800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8017070 con 0x7f52e806f8d0 2024-04-24T06:31:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d800c940 con 0x7f52e806f8d0 2024-04-24T06:31:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800cd30 con 0x7f52e806f8d0 2024-04-24T06:31:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e8099de0 con 0x7f52e806f8d0 2024-04-24T06:31:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.136+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e8166fc0 con 0x7f52e806f8d0 2024-04-24T06:31:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.138+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e804fd90 con 0x7f52e806f8d0 2024-04-24T06:31:55.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.139+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52d801a070 con 0x7f52e806f8d0 2024-04-24T06:31:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.140+0000 7f52dea00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 0x7f52c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.141+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f52d808bb50 con 0x7f52e806f8d0 2024-04-24T06:31:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.141+0000 7f52e6000640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 0x7f52c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.141+0000 7f52e6000640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 0x7f52c806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52e815d990 tx=0x7f52d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.146+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52e815ea10 con 0x7f52e806f8d0 2024-04-24T06:31:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.393+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f52e806c370 con 0x7f52e806f8d0 2024-04-24T06:31:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.896+0000 7f52dea00640 1 -- 172.21.15.59:0/2852084635 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f52d8059e50 con 0x7f52e806f8d0 2024-04-24T06:31:55.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 msgr2=0x7f52c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 0x7f52c806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52e815d990 tx=0x7f52d4009040 comp rx=0 tx=0).stop 2024-04-24T06:31:55.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 msgr2=0x7f52e8099400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f52d800e870 tx=0x7f52d800ed30 comp rx=0 tx=0).stop 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 shutdown_connections 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52c806c8f0 0x7f52c806edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52e815b220 0x7f52e8167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e80d05d0 0x7f52e8143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 --2- 172.21.15.59:0/2852084635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e806f8d0 0x7f52e8099400 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 >> 172.21.15.59:0/2852084635 conn(0x7f52e806bdf0 msgr2=0x7f52e8158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.900+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 shutdown_connections 2024-04-24T06:31:55.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:55.901+0000 7f52e7e00640 1 -- 172.21.15.59:0/2852084635 wait complete. 2024-04-24T06:31:55.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:31:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.140+0000 7f4de7200640 1 Processor -- start 2024-04-24T06:31:56.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.140+0000 7f4de7200640 1 -- start start 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4de7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4de00cd7e0 0x7f4de006e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4de7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de00cf340 con 0x7f4de01431b0 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4de7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de006ef10 con 0x7f4de006e840 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4de7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4de006ecd0 con 0x7f4de00cd7e0 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4dd7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4de00cd7e0 0x7f4de006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4dd7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4de00cd7e0 0x7f4de006e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56340/0 (socket says 172.21.15.59:56340) 2024-04-24T06:31:56.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4dd7400640 1 -- 172.21.15.59:0/69663390 learned_addr learned my addr 172.21.15.59:0/69663390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.142+0000 7f4ddfe00640 1 --1- 172.21.15.59:0/69663390 >> v1:172.21.15.59:6789/0 conn(0x7f4de01431b0 0x7f4de00cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58476/0 (socket says 172.21.15.59:58476) 2024-04-24T06:31:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.143+0000 7f4dd7400640 1 -- 172.21.15.59:0/69663390 >> v1:172.21.15.66:6789/0 conn(0x7f4de006e840 legacy=0x7f4de00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.143+0000 7f4dd7400640 1 -- 172.21.15.59:0/69663390 >> v1:172.21.15.59:6789/0 conn(0x7f4de01431b0 legacy=0x7f4de00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.143+0000 7f4dd7400640 1 -- 172.21.15.59:0/69663390 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de006f1f0 con 0x7f4de00cd7e0 2024-04-24T06:31:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.144+0000 7f4dd7400640 1 --2- 172.21.15.59:0/69663390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de00cd7e0 0x7f4de006e300 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009870 tx=0x7f4dc802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648add531f67225e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.144+0000 7f4ddf400640 1 -- 172.21.15.59:0/69663390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8037070 con 0x7f4de00cd7e0 2024-04-24T06:31:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.144+0000 7f4ddf400640 1 -- 172.21.15.59:0/69663390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc8032640 con 0x7f4de00cd7e0 2024-04-24T06:31:56.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.144+0000 7f4ddf400640 1 -- 172.21.15.59:0/69663390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8032a10 con 0x7f4de00cd7e0 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 -- 172.21.15.59:0/69663390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de00cd7e0 msgr2=0x7f4de006e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 --2- 172.21.15.59:0/69663390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de00cd7e0 0x7f4de006e300 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009870 tx=0x7f4dc802a330 comp rx=0 tx=0).stop 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 -- 172.21.15.59:0/69663390 shutdown_connections 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 --2- 172.21.15.59:0/69663390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de00cd7e0 0x7f4de006e300 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 -- 172.21.15.59:0/69663390 >> 172.21.15.59:0/69663390 conn(0x7f4de006b9d0 msgr2=0x7f4de006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 -- 172.21.15.59:0/69663390 shutdown_connections 2024-04-24T06:31:56.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.145+0000 7f4de7200640 1 -- 172.21.15.59:0/69663390 wait complete. 2024-04-24T06:31:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.147+0000 7f4de7200640 1 Processor -- start 2024-04-24T06:31:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.147+0000 7f4de7200640 1 -- start start 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de00cd7e0 0x7f4de013e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4de01431b0 0x7f4de0140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4de00d0210 con 0x7f4de006e840 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4de00cf670 con 0x7f4de00cd7e0 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4de7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4de00d0390 con 0x7f4de01431b0 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4ddfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4ddfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56356/0 (socket says 172.21.15.59:56356) 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.148+0000 7f4ddfe00640 1 -- 172.21.15.59:0/1578266504 learned_addr learned my addr 172.21.15.59:0/1578266504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 -- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4de01431b0 msgr2=0x7f4de0140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4de01431b0 0x7f4de0140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4dd7400640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de00cd7e0 0x7f4de013e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:56.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 -- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de00cd7e0 msgr2=0x7f4de013e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de00cd7e0 0x7f4de013e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 -- 172.21.15.59:0/1578266504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de00c9e40 con 0x7f4de006e840 2024-04-24T06:31:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddfe00640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c730 tx=0x7f4dd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0017070 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd000ce40 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de00ca120 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.149+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000a860 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.150+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de00ca6e0 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.151+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de0068a80 con 0x7f4de006e840 2024-04-24T06:31:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.152+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dd001a070 con 0x7f4de006e840 2024-04-24T06:31:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.153+0000 7f4ddcc00640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 0x7f4dbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.153+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f4dd008b260 con 0x7f4de006e840 2024-04-24T06:31:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.153+0000 7f4dd7400640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 0x7f4dbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.154+0000 7f4dd7400640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 0x7f4dbc06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8034000 tx=0x7f4dc802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.158+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dd0059560 con 0x7f4de006e840 2024-04-24T06:31:56.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.370+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4de00b1a80 con 0x7f4de006e840 2024-04-24T06:31:56.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.908+0000 7f4ddcc00640 1 -- 172.21.15.59:0/1578266504 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f4dd0058e60 con 0x7f4de006e840 2024-04-24T06:31:56.910 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:31:56.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 msgr2=0x7f4dbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 0x7f4dbc06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8034000 tx=0x7f4dc802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 msgr2=0x7f4de00c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c730 tx=0x7f4dd000cbf0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 shutdown_connections 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4dbc06c8f0 0x7f4dbc06edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4de01431b0 0x7f4de0140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4de00cd7e0 0x7f4de013e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 --2- 172.21.15.59:0/1578266504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de006e840 0x7f4de00c97b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 >> 172.21.15.59:0/1578266504 conn(0x7f4de006b9d0 msgr2=0x7f4de015f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 shutdown_connections 2024-04-24T06:31:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:56.912+0000 7f4de7200640 1 -- 172.21.15.59:0/1578266504 wait complete. 2024-04-24T06:31:56.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:31:57.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.146+0000 7fe8dd400640 1 Processor -- start 2024-04-24T06:31:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.146+0000 7fe8dd400640 1 -- start start 2024-04-24T06:31:57.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.148+0000 7fe8dd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:57.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.148+0000 7fe8dd400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d80cf070 con 0x7fe8d80d4210 2024-04-24T06:31:57.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.148+0000 7fe8dd400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d80cf2f0 con 0x7fe8d806cbb0 2024-04-24T06:31:57.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.148+0000 7fe8dd400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d80d1c60 con 0x7fe8d815e8e0 2024-04-24T06:31:57.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:57.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56358/0 (socket says 172.21.15.59:56358) 2024-04-24T06:31:57.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 -- 172.21.15.59:0/3669767019 learned_addr learned my addr 172.21.15.59:0/3669767019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:57.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 -- 172.21.15.59:0/3669767019 >> v1:172.21.15.66:6789/0 conn(0x7fe8d806cbb0 legacy=0x7fe8d806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 -- 172.21.15.59:0/3669767019 >> v1:172.21.15.59:6789/0 conn(0x7fe8d80d4210 legacy=0x7fe8d80d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d7e00640 1 -- 172.21.15.59:0/3669767019 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80cf5b0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.149+0000 7fe8d6a00640 1 -- 172.21.15.59:0/3669767019 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 713996328 0 0) 0x7fe8c4001180 con 0x7fe8d80d4210 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.150+0000 7fe8d7e00640 1 --2- 172.21.15.59:0/3669767019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009870 tx=0x7fe8c802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2677f83e8333cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8d6a00640 1 -- 172.21.15.59:0/3669767019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8037070 con 0x7fe8d815e8e0 2024-04-24T06:31:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8d6a00640 1 -- 172.21.15.59:0/3669767019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c8032440 con 0x7fe8d815e8e0 2024-04-24T06:31:57.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8d6a00640 1 -- 172.21.15.59:0/3669767019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8032810 con 0x7fe8d815e8e0 2024-04-24T06:31:57.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 -- 172.21.15.59:0/3669767019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 msgr2=0x7fe8d80cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 --2- 172.21.15.59:0/3669767019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009870 tx=0x7fe8c802a2a0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 -- 172.21.15.59:0/3669767019 shutdown_connections 2024-04-24T06:31:57.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 --2- 172.21.15.59:0/3669767019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d80cfdb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 -- 172.21.15.59:0/3669767019 >> 172.21.15.59:0/3669767019 conn(0x7fe8d808e060 msgr2=0x7fe8d809d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:57.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.151+0000 7fe8dd400640 1 -- 172.21.15.59:0/3669767019 shutdown_connections 2024-04-24T06:31:57.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.152+0000 7fe8dd400640 1 -- 172.21.15.59:0/3669767019 wait complete. 2024-04-24T06:31:57.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.153+0000 7fe8dd400640 1 Processor -- start 2024-04-24T06:31:57.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.153+0000 7fe8dd400640 1 -- start start 2024-04-24T06:31:57.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe8d806cbb0 0x7fe8d81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:57.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8d80d4210 0x7fe8d8161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:57.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80d02f0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d815ecc0 con 0x7fe8d80d4210 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.154+0000 7fe8dd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d80d0470 con 0x7fe8d806cbb0 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56366/0 (socket says 172.21.15.59:56366) 2024-04-24T06:31:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 -- 172.21.15.59:0/1513842306 learned_addr learned my addr 172.21.15.59:0/1513842306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8d7400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8d80d4210 0x7fe8d8161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8d7e00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe8d806cbb0 0x7fe8d81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 -- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe8d806cbb0 msgr2=0x7fe8d81505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe8d806cbb0 0x7fe8d81505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 -- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8d80d4210 msgr2=0x7fe8d8161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8d80d4210 0x7fe8d8161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.155+0000 7fe8dca00640 1 -- 172.21.15.59:0/1513842306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d81f49b0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8dca00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c780 tx=0x7fe8d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004b60 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0004d00 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d815f640 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000aa30 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.156+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d815fc00 con 0x7fe8d815e8e0 2024-04-24T06:31:57.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.157+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d804fd90 con 0x7fe8d815e8e0 2024-04-24T06:31:57.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.158+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8d000abd0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.160+0000 7fe8cfe00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 0x7fe8b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:57.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.160+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7fe8d008bff0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.160+0000 7fe8d7e00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 0x7fe8b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:57.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.161+0000 7fe8d7e00640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 0x7fe8b806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8034000 tx=0x7fe8c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:57.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.164+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8d005a430 con 0x7fe8d815e8e0 2024-04-24T06:31:57.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.373+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe8d81621b0 con 0x7fe8d815e8e0 2024-04-24T06:31:57.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.912+0000 7fe8cfe00640 1 -- 172.21.15.59:0/1513842306 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7fe8d0059d30 con 0x7fe8d815e8e0 2024-04-24T06:31:57.914 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:31:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.915+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 msgr2=0x7fe8b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.915+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 0x7fe8b806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8034000 tx=0x7fe8c802e000 comp rx=0 tx=0).stop 2024-04-24T06:31:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.915+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 msgr2=0x7fe8d815eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.915+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c780 tx=0x7fe8d000cc40 comp rx=0 tx=0).stop 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 shutdown_connections 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe8b806c8f0 0x7fe8b806edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d815e8e0 0x7fe8d815eed0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8d80d4210 0x7fe8d8161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 --2- 172.21.15.59:0/1513842306 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe8d806cbb0 0x7fe8d81505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.917+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 >> 172.21.15.59:0/1513842306 conn(0x7fe8d808e060 msgr2=0x7fe8d80d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.918+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 shutdown_connections 2024-04-24T06:31:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:57.918+0000 7fe8dd400640 1 -- 172.21.15.59:0/1513842306 wait complete. 2024-04-24T06:31:57.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:31:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.144+0000 7efc68a00640 1 Processor -- start 2024-04-24T06:31:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.144+0000 7efc68a00640 1 -- start start 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc68a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efc640724d0 0x7efc64162240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc68a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640728b0 con 0x7efc6406fb50 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc68a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640b40a0 con 0x7efc640d05d0 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc68a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efc64162a70 con 0x7efc640724d0 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc62a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efc6406fb50 0x7efc640b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58484/0 (socket says 172.21.15.59:58484) 2024-04-24T06:31:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.145+0000 7efc62a00640 1 -- 172.21.15.59:0/1210092330 learned_addr learned my addr 172.21.15.59:0/1210092330 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc63400640 1 --2- 172.21.15.59:0/1210092330 >> v2:172.21.15.66:3300/0 conn(0x7efc640724d0 0x7efc64162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2966239034 0 0) 0x7efc640728b0 con 0x7efc6406fb50 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc48003520 con 0x7efc6406fb50 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4060422767 0 0) 0x7efc48003520 con 0x7efc6406fb50 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 >> v2:172.21.15.66:3300/0 conn(0x7efc640724d0 msgr2=0x7efc64162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 --2- 172.21.15.59:0/1210092330 >> v2:172.21.15.66:3300/0 conn(0x7efc640724d0 0x7efc64162240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 >> v1:172.21.15.66:6789/0 conn(0x7efc640d05d0 legacy=0x7efc6415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.146+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64162d50 con 0x7efc6406fb50 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7efc50001670 con 0x7efc6406fb50 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 >> v1:172.21.15.59:6789/0 conn(0x7efc6406fb50 legacy=0x7efc640b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 0x7efc48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc480051c0 0x7efc4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc64162a70 con 0x7efc48003a00 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc4800c0c0 con 0x7efc48004800 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc4800c2c0 con 0x7efc480051c0 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 0x7efc48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc62a00640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63e00640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc480051c0 0x7efc4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 -- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc480051c0 msgr2=0x7efc4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc480051c0 0x7efc4800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 -- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48004800 msgr2=0x7efc48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.147+0000 7efc63400640 1 -- 172.21.15.59:0/1210092330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc48003520 con 0x7efc48003a00 2024-04-24T06:31:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.148+0000 7efc62a00640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48004800 0x7efc48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.148+0000 7efc63e00640 1 -- 172.21.15.59:0/1210092330 reap_dead start 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc63400640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 0x7efc48003e20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ac0 tx=0x7efc48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d8e38621215b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54036070 con 0x7efc48003a00 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc54031400 con 0x7efc48003a00 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc62000640 1 -- 172.21.15.59:0/1210092330 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54009690 con 0x7efc48003a00 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc68a00640 1 -- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 msgr2=0x7efc48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.149+0000 7efc68a00640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 0x7efc48003e20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ac0 tx=0x7efc48005830 comp rx=0 tx=0).stop 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.150+0000 7efc68a00640 1 -- 172.21.15.59:0/1210092330 shutdown_connections 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.150+0000 7efc68a00640 1 --2- 172.21.15.59:0/1210092330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc48003a00 0x7efc48003e20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.150+0000 7efc68a00640 1 -- 172.21.15.59:0/1210092330 >> 172.21.15.59:0/1210092330 conn(0x7efc6406bdf0 msgr2=0x7efc6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.150+0000 7efc68a00640 1 -- 172.21.15.59:0/1210092330 shutdown_connections 2024-04-24T06:31:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.150+0000 7efc68a00640 1 -- 172.21.15.59:0/1210092330 wait complete. 2024-04-24T06:31:58.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.151+0000 7efc68a00640 1 Processor -- start 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.152+0000 7efc68a00640 1 -- start start 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.152+0000 7efc68a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48003a00 0x7efc641f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.152+0000 7efc68a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc68a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc6409a280 0x7efc6409a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc68a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc640b3470 con 0x7efc640ae690 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc68a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc640b3190 con 0x7efc48003a00 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc68a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc640d0a20 con 0x7efc6409a280 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56392/0 (socket says 172.21.15.59:56392) 2024-04-24T06:31:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 -- 172.21.15.59:0/2303217626 learned_addr learned my addr 172.21.15.59:0/2303217626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc63400640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48003a00 0x7efc641f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc63e00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc6409a280 0x7efc6409a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 -- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc6409a280 msgr2=0x7efc6409a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc6409a280 0x7efc6409a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 -- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48003a00 msgr2=0x7efc641f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48003a00 0x7efc641f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.153+0000 7efc62a00640 1 -- 172.21.15.59:0/2303217626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6409ac40 con 0x7efc640ae690 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc62a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d8e0 tx=0x7efc5000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50019070 con 0x7efc640ae690 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc500074e0 con 0x7efc640ae690 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50004a00 con 0x7efc640ae690 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6409af00 con 0x7efc640ae690 2024-04-24T06:31:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.154+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc64145940 con 0x7efc640ae690 2024-04-24T06:31:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.155+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6404fd90 con 0x7efc640ae690 2024-04-24T06:31:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.156+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc5001c070 con 0x7efc640ae690 2024-04-24T06:31:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.157+0000 7efc5b400640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 0x7efc4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.157+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7efc5008b310 con 0x7efc640ae690 2024-04-24T06:31:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.157+0000 7efc63400640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 0x7efc4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.158+0000 7efc63400640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 0x7efc4406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc54033000 tx=0x7efc54008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.161+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc50059ad0 con 0x7efc640ae690 2024-04-24T06:31:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.364+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efc641627b0 con 0x7efc640ae690 2024-04-24T06:31:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.371+0000 7efc5b400640 1 -- 172.21.15.59:0/2303217626 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7efc500593d0 con 0x7efc640ae690 2024-04-24T06:31:58.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 msgr2=0x7efc4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 0x7efc4406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc54033000 tx=0x7efc54008030 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 msgr2=0x7efc640aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d8e0 tx=0x7efc5000dda0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 shutdown_connections 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc4406c8f0 0x7efc4406edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc6409a280 0x7efc6409a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc640ae690 0x7efc640aeb10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 --2- 172.21.15.59:0/2303217626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc48003a00 0x7efc641f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 >> 172.21.15.59:0/2303217626 conn(0x7efc6406bdf0 msgr2=0x7efc6415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.374+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 shutdown_connections 2024-04-24T06:31:58.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.375+0000 7efc68a00640 1 -- 172.21.15.59:0/2303217626 wait complete. 2024-04-24T06:31:58.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-24T06:31:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.606+0000 7fbcf6000640 1 Processor -- start 2024-04-24T06:31:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.606+0000 7fbcf6000640 1 -- start start 2024-04-24T06:31:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.607+0000 7fbcf6000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.607+0000 7fbcf6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.607+0000 7fbcf6000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf00cd520 con 0x7fbcf00b3e40 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.607+0000 7fbcf6000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf00cddc0 con 0x7fbcf00cd020 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.607+0000 7fbcf6000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf00cd800 con 0x7fbcf01454e0 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56404/0 (socket says 172.21.15.59:56404) 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 -- 172.21.15.59:0/1704287911 learned_addr learned my addr 172.21.15.59:0/1704287911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbcefe00640 1 --1- 172.21.15.59:0/1704287911 >> v1:172.21.15.59:6789/0 conn(0x7fbcf00b3e40 0x7fbcf0160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58492/0 (socket says 172.21.15.59:58492) 2024-04-24T06:31:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbcef400640 1 --2- 172.21.15.59:0/1704287911 >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 -- 172.21.15.59:0/1704287911 >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 msgr2=0x7fbcf0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 --2- 172.21.15.59:0/1704287911 >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 -- 172.21.15.59:0/1704287911 >> v1:172.21.15.59:6789/0 conn(0x7fbcf00b3e40 legacy=0x7fbcf0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbce7e00640 1 -- 172.21.15.59:0/1704287911 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf00cdae0 con 0x7fbcf00cd020 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbceea00640 1 -- 172.21.15.59:0/1704287911 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962338465 0 0) 0x7fbcf00cd520 con 0x7fbcf00b3e40 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.608+0000 7fbcef400640 1 --2- 172.21.15.59:0/1704287911 >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.609+0000 7fbce7e00640 1 --2- 172.21.15.59:0/1704287911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0073280 tx=0x7fbcd802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f0e4e9eac7a5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbceea00640 1 -- 172.21.15.59:0/1704287911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8037070 con 0x7fbcf00cd020 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbceea00640 1 -- 172.21.15.59:0/1704287911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8032440 con 0x7fbcf00cd020 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbceea00640 1 -- 172.21.15.59:0/1704287911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8032810 con 0x7fbcf00cd020 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 -- 172.21.15.59:0/1704287911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00cd020 msgr2=0x7fbcf00b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 --2- 172.21.15.59:0/1704287911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0073280 tx=0x7fbcd802a200 comp rx=0 tx=0).stop 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 -- 172.21.15.59:0/1704287911 shutdown_connections 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 --2- 172.21.15.59:0/1704287911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 --2- 172.21.15.59:0/1704287911 >> v2:172.21.15.66:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 -- 172.21.15.59:0/1704287911 >> 172.21.15.59:0/1704287911 conn(0x7fbcf0067ea0 msgr2=0x7fbcf006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 -- 172.21.15.59:0/1704287911 shutdown_connections 2024-04-24T06:31:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.610+0000 7fbcf6000640 1 -- 172.21.15.59:0/1704287911 wait complete. 2024-04-24T06:31:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.612+0000 7fbcf6000640 1 Processor -- start 2024-04-24T06:31:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.612+0000 7fbcf6000640 1 -- start start 2024-04-24T06:31:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00cd020 0x7fbcf015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf01454e0 0x7fbcf013ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0146270 con 0x7fbcf00b3e40 2024-04-24T06:31:58.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01460f0 con 0x7fbcf01454e0 2024-04-24T06:31:58.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcf6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf0161020 con 0x7fbcf00cd020 2024-04-24T06:31:58.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcef400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcef400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56420/0 (socket says 172.21.15.59:56420) 2024-04-24T06:31:58.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.613+0000 7fbcef400640 1 -- 172.21.15.59:0/3421036949 learned_addr learned my addr 172.21.15.59:0/3421036949 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 -- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00cd020 msgr2=0x7fbcf015f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00cd020 0x7fbcf015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 -- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf01454e0 msgr2=0x7fbcf013ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf01454e0 0x7fbcf013ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 -- 172.21.15.59:0/3421036949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf013f9d0 con 0x7fbcf00b3e40 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcef400640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c7f0 tx=0x7fbce000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015070 con 0x7fbcf00b3e40 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce0007820 con 0x7fbcf00b3e40 2024-04-24T06:31:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0007bd0 con 0x7fbcf00b3e40 2024-04-24T06:31:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.614+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf006ef40 con 0x7fbcf00b3e40 2024-04-24T06:31:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.615+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf006f500 con 0x7fbcf00b3e40 2024-04-24T06:31:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.616+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf0068a80 con 0x7fbcf00b3e40 2024-04-24T06:31:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.617+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbce0018070 con 0x7fbcf00b3e40 2024-04-24T06:31:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.617+0000 7fbce7400640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.618+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fbce008ba60 con 0x7fbcf00b3e40 2024-04-24T06:31:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.618+0000 7fbce7e00640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.618+0000 7fbce7e00640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8034000 tx=0x7fbcd802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.622+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbce005a0f0 con 0x7fbcf00b3e40 2024-04-24T06:31:58.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.901+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fbcf0160030 con 0x7fbcf00b3e40 2024-04-24T06:31:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.928+0000 7fbce7400640 1 -- 172.21.15.59:0/3421036949 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v188) ==== 149+0+0 (secure 0 0 0) 0x7fbce00599f0 con 0x7fbcf00b3e40 2024-04-24T06:31:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 msgr2=0x7fbccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8034000 tx=0x7fbcd802c000 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 msgr2=0x7fbcf006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c7f0 tx=0x7fbce000ccb0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 shutdown_connections 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbcf01454e0 0x7fbcf013ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbcf00cd020 0x7fbcf015f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 --2- 172.21.15.59:0/3421036949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf006e7c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 >> 172.21.15.59:0/3421036949 conn(0x7fbcf0067ea0 msgr2=0x7fbcf00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.931+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 shutdown_connections 2024-04-24T06:31:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:58.932+0000 7fbcf6000640 1 -- 172.21.15.59:0/3421036949 wait complete. 2024-04-24T06:31:58.952 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-24T06:31:58.952 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-24T06:31:58.952 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:31:58.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:31:59.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.161+0000 7f5843e00640 1 Processor -- start 2024-04-24T06:31:59.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.161+0000 7f5843e00640 1 -- start start 2024-04-24T06:31:59.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.162+0000 7f5843e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f584406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5843e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58440b3080 0x7f58440cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5843e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584415f160 con 0x7f584415ece0 2024-04-24T06:31:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5843e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f58440ced20 con 0x7f58440b3080 2024-04-24T06:31:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5843e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f584415fcd0 con 0x7f58440cd050 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f583be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f584415ece0 0x7f58440cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36330/0 (socket says 172.21.15.59:36330) 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f583be00640 1 -- 172.21.15.59:0/1175515953 learned_addr learned my addr 172.21.15.59:0/1175515953 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f584406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.163+0000 7f5843400640 1 --2- 172.21.15.59:0/1175515953 >> v2:172.21.15.66:3300/0 conn(0x7f58440b3080 0x7f58440cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099439826 0 0) 0x7f584415f160 con 0x7f584415ece0 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5828003520 con 0x7f584415ece0 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 585468383 0 0) 0x7f5828003520 con 0x7f584415ece0 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 msgr2=0x7f584406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f584406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 >> v2:172.21.15.66:3300/0 conn(0x7f58440b3080 msgr2=0x7f58440cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 --2- 172.21.15.59:0/1175515953 >> v2:172.21.15.66:3300/0 conn(0x7f58440b3080 0x7f58440cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584415f430 con 0x7f584415ece0 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f582c001670 con 0x7f584415ece0 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f584406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 >> v1:172.21.15.59:6789/0 conn(0x7f584415ece0 legacy=0x7f58440cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f5828003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f5828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5828004d40 0x7f58280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f584415fcd0 con 0x7f58280039b0 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f582800b9c0 con 0x7f5828004d40 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f582800bb90 con 0x7f58440cd050 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.164+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f5828003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5843400640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5828004d40 0x7f58280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5842a00640 1 -- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 msgr2=0x7f5828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440cd050 0x7f5828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5842a00640 1 -- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5828004d40 msgr2=0x7f58280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5828004d40 0x7f58280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.165+0000 7f5842a00640 1 -- 172.21.15.59:0/1175515953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5828003520 con 0x7f58280039b0 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.166+0000 7f5842a00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f5828003dd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58340098a0 tx=0x7f5828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d73b309182b1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583400f040 con 0x7f58280039b0 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583402ad00 con 0x7f58280039b0 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5842000640 1 -- 172.21.15.59:0/1175515953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834004250 con 0x7f58280039b0 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 -- 172.21.15.59:0/1175515953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 msgr2=0x7f5828003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 --2- 172.21.15.59:0/1175515953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f5828003dd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58340098a0 tx=0x7f5828005830 comp rx=0 tx=0).stop 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843400640 1 -- 172.21.15.59:0/1175515953 reap_dead start 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 -- 172.21.15.59:0/1175515953 shutdown_connections 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 -- 172.21.15.59:0/1175515953 >> 172.21.15.59:0/1175515953 conn(0x7f58440b21a0 msgr2=0x7f58440b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 -- 172.21.15.59:0/1175515953 shutdown_connections 2024-04-24T06:31:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.167+0000 7f5843e00640 1 -- 172.21.15.59:0/1175515953 wait complete. 2024-04-24T06:31:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.169+0000 7f5843e00640 1 Processor -- start 2024-04-24T06:31:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.169+0000 7f5843e00640 1 -- start start 2024-04-24T06:31:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 0x7f58441f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441563b0 0x7f58441f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58440cef50 con 0x7f58280039b0 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f584415f9f0 con 0x7f58441563b0 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.170+0000 7f5843e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5844159110 con 0x7f58440c9bb0 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55684/0 (socket says 172.21.15.59:55684) 2024-04-24T06:31:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 -- 172.21.15.59:0/4033692850 learned_addr learned my addr 172.21.15.59:0/4033692850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5843400640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441563b0 0x7f58441f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f583be00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 0x7f58441f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 -- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 msgr2=0x7f58441f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 0x7f58441f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 -- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441563b0 msgr2=0x7f58441f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441563b0 0x7f58441f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f5842a00640 1 -- 172.21.15.59:0/4033692850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440cb520 con 0x7f58280039b0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.171+0000 7f583be00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 0x7f58441f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f5842a00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f58340099c0 tx=0x7f5834004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834041070 con 0x7f58280039b0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583400f000 con 0x7f58280039b0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58440cb800 con 0x7f58280039b0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834003bd0 con 0x7f58280039b0 2024-04-24T06:31:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.172+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58440cbdc0 con 0x7f58280039b0 2024-04-24T06:31:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.173+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584404fd90 con 0x7f58280039b0 2024-04-24T06:31:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.175+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5834044070 con 0x7f58280039b0 2024-04-24T06:31:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.175+0000 7f583b400640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 0x7f582006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.176+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f58340abf50 con 0x7f58280039b0 2024-04-24T06:31:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.176+0000 7f583be00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 0x7f582006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.177+0000 7f583be00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 0x7f582006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f582c001860 tx=0x7f582c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.180+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f583407a5f0 con 0x7f58280039b0 2024-04-24T06:31:59.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.389+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58440cb1f0 con 0x7f58280039b0 2024-04-24T06:31:59.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.391+0000 7f583b400640 1 -- 172.21.15.59:0/4033692850 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+12936 (secure 0 0 0) 0x7f5834079ef0 con 0x7f58280039b0 2024-04-24T06:31:59.393 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:31:59.394 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":50,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:31:57.909328+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1331520600":"2024-04-25T06:31:54.890069+0000","172.21.15.66:6833/1331520600":"2024-04-25T06:31:54.890069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 msgr2=0x7f582006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 0x7f582006ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f582c001860 tx=0x7f582c009040 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 msgr2=0x7f58440b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f58340099c0 tx=0x7f5834004730 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 shutdown_connections 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f582006c920 0x7f582006ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.395+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58441563b0 0x7f58441f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.396+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58440c9bb0 0x7f58441f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.396+0000 7f5843e00640 1 --2- 172.21.15.59:0/4033692850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58280039b0 0x7f58440b3e60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.396+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 >> 172.21.15.59:0/4033692850 conn(0x7f58440b21a0 msgr2=0x7f58440b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:59.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.396+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 shutdown_connections 2024-04-24T06:31:59.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.396+0000 7f5843e00640 1 -- 172.21.15.59:0/4033692850 wait complete. 2024-04-24T06:31:59.418 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:31:59.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:31:59.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.636+0000 7f7094400640 1 Processor -- start 2024-04-24T06:31:59.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.636+0000 7f7094400640 1 -- start start 2024-04-24T06:31:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.638+0000 7f7094400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f708c06d170 0x7f708c0726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.638+0000 7f7094400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708c06f490 con 0x7f708c072c00 2024-04-24T06:31:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.638+0000 7f7094400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708c15ee60 con 0x7f708c15ea80 2024-04-24T06:31:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.638+0000 7f7094400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f708c070320 con 0x7f708c06d170 2024-04-24T06:31:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f7092400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f708c072c00 0x7f708c0cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36338/0 (socket says 172.21.15.59:36338) 2024-04-24T06:31:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f7092400640 1 -- 172.21.15.59:0/4048441049 learned_addr learned my addr 172.21.15.59:0/4048441049 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:31:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f7091000640 1 --2- 172.21.15.59:0/4048441049 >> v2:172.21.15.66:3300/0 conn(0x7f708c06d170 0x7f708c0726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316475132 0 0) 0x7f7084001110 con 0x7f708c072c00 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7074003520 con 0x7f708c072c00 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.639+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 748558079 0 0) 0x7f7074003520 con 0x7f708c072c00 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 >> v2:172.21.15.66:3300/0 conn(0x7f708c06d170 msgr2=0x7f708c0726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 --2- 172.21.15.59:0/4048441049 >> v2:172.21.15.66:3300/0 conn(0x7f708c06d170 0x7f708c0726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 >> v1:172.21.15.66:6789/0 conn(0x7f708c15ea80 legacy=0x7f708c162f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c06f710 con 0x7f708c072c00 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7084001840 con 0x7f708c072c00 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 >> v1:172.21.15.59:6789/0 conn(0x7f708c072c00 legacy=0x7f708c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70740039b0 0x7f7074003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7074004800 0x7f7074004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 0x7f707400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f708c070320 con 0x7f70740051c0 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f707400c160 con 0x7f70740039b0 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f707400c360 con 0x7f7074004800 2024-04-24T06:31:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 0x7f707400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7091a00640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70740039b0 0x7f7074003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7091000640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7074004800 0x7f7074004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 -- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7074004800 msgr2=0x7f7074004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7074004800 0x7f7074004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 -- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70740039b0 msgr2=0x7f7074003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70740039b0 0x7f7074003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.640+0000 7f7092400640 1 -- 172.21.15.59:0/4048441049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7074003520 con 0x7f70740051c0 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.641+0000 7f7092400640 1 -- 172.21.15.59:0/4048441049 reap_dead start 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f7092400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 0x7f707400b9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f708400c4c0 tx=0x7f7074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aea63db78e21de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708400c0e0 con 0x7f70740051c0 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70840058d0 con 0x7f70740051c0 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f708be00640 1 -- 172.21.15.59:0/4048441049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084005ce0 con 0x7f70740051c0 2024-04-24T06:31:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f7094400640 1 -- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 msgr2=0x7f707400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.642+0000 7f7094400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 0x7f707400b9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f708400c4c0 tx=0x7f7074005830 comp rx=0 tx=0).stop 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.643+0000 7f7094400640 1 -- 172.21.15.59:0/4048441049 shutdown_connections 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.643+0000 7f7094400640 1 --2- 172.21.15.59:0/4048441049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70740051c0 0x7f707400b9c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.643+0000 7f7094400640 1 -- 172.21.15.59:0/4048441049 >> 172.21.15.59:0/4048441049 conn(0x7f708c0d0d00 msgr2=0x7f708c0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.643+0000 7f7094400640 1 -- 172.21.15.59:0/4048441049 shutdown_connections 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.643+0000 7f7094400640 1 -- 172.21.15.59:0/4048441049 wait complete. 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.645+0000 7f7094400640 1 Processor -- start 2024-04-24T06:31:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.645+0000 7f7094400640 1 -- start start 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70740051c0 0x7f708c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f708c0b2040 0x7f708c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f708c06ffb0 con 0x7f708c0afcc0 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f708c06fcd0 con 0x7f708c0b2040 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7094400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f708c0726c0 con 0x7f70740051c0 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55706/0 (socket says 172.21.15.59:55706) 2024-04-24T06:31:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 -- 172.21.15.59:0/273929045 learned_addr learned my addr 172.21.15.59:0/273929045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091a00640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70740051c0 0x7f708c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 -- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70740051c0 msgr2=0x7f708c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70740051c0 0x7f708c1528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 -- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f708c0b2040 msgr2=0x7f708c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7092400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f708c0b2040 0x7f708c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f708c0b2040 0x7f708c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.646+0000 7f7091000640 1 -- 172.21.15.59:0/273929045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c1f4f80 con 0x7f708c0afcc0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7091000640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70780108c0 tx=0x7f7078010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800f040 con 0x7f708c0afcc0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707800eb60 con 0x7f708c0afcc0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c1f5260 con 0x7f708c0afcc0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707801c400 con 0x7f708c0afcc0 2024-04-24T06:31:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.647+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1f5820 con 0x7f708c0afcc0 2024-04-24T06:31:59.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.649+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c069190 con 0x7f708c0afcc0 2024-04-24T06:31:59.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.650+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f707801c5a0 con 0x7f708c0afcc0 2024-04-24T06:31:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.650+0000 7f7089600640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 0x7f706c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:31:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.651+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f707808bf70 con 0x7f708c0afcc0 2024-04-24T06:31:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.651+0000 7f7091a00640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 0x7f706c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:31:59.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.651+0000 7f7091a00640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 0x7f706c06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7080000f80 tx=0x7f7080009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:31:59.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.655+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7078018030 con 0x7f708c0afcc0 2024-04-24T06:31:59.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.864+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f708c067c50 con 0x7f708c0afcc0 2024-04-24T06:31:59.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.864+0000 7f7089600640 1 -- 172.21.15.59:0/273929045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f707805a690 con 0x7f708c0afcc0 2024-04-24T06:31:59.866 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:31:59.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.867+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 msgr2=0x7f706c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.867+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 0x7f706c06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7080000f80 tx=0x7f7080009040 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.867+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 msgr2=0x7f708c0b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.867+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70780108c0 tx=0x7f7078010d80 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 shutdown_connections 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f706c06c8f0 0x7f706c06edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f708c0b2040 0x7f708c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0afcc0 0x7f708c0b1b00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 --2- 172.21.15.59:0/273929045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70740051c0 0x7f708c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:31:59.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 >> 172.21.15.59:0/273929045 conn(0x7f708c0d0d00 msgr2=0x7f708c0b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:31:59.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 shutdown_connections 2024-04-24T06:31:59.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:31:59.868+0000 7f7094400640 1 -- 172.21.15.59:0/273929045 wait complete. 2024-04-24T06:31:59.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-24T06:32:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.102+0000 7f14dae00640 1 Processor -- start 2024-04-24T06:32:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.102+0000 7f14dae00640 1 -- start start 2024-04-24T06:32:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14dae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f14d41644b0 0x7f14d4163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14dae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d4160d40 con 0x7f14d4160960 2024-04-24T06:32:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14dae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d4166de0 con 0x7f14d40b4550 2024-04-24T06:32:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14dae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f14d4164270 con 0x7f14d41644b0 2024-04-24T06:32:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14d3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f14d4160960 0x7f14d40b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36348/0 (socket says 172.21.15.59:36348) 2024-04-24T06:32:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.104+0000 7f14d3e00640 1 -- 172.21.15.59:0/2241120525 learned_addr learned my addr 172.21.15.59:0/2241120525 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d3400640 1 --2- 172.21.15.59:0/2241120525 >> v2:172.21.15.66:3300/0 conn(0x7f14d41644b0 0x7f14d4163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535890448 0 0) 0x7f14d4160d40 con 0x7f14d4160960 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14b8003520 con 0x7f14d4160960 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 622847251 0 0) 0x7f14b8003520 con 0x7f14d4160960 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 >> v2:172.21.15.66:3300/0 conn(0x7f14d41644b0 msgr2=0x7f14d4163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 --2- 172.21.15.59:0/2241120525 >> v2:172.21.15.66:3300/0 conn(0x7f14d41644b0 0x7f14d4163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 >> v1:172.21.15.66:6789/0 conn(0x7f14d40b4550 legacy=0x7f14d406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41670c0 con 0x7f14d4160960 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f14c4001690 con 0x7f14d4160960 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 >> v1:172.21.15.59:6789/0 conn(0x7f14d4160960 legacy=0x7f14d40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14b8003a00 0x7f14b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14b80051c0 0x7f14b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4164270 con 0x7f14b8004800 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14b800c090 con 0x7f14b80051c0 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.105+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f14b800c290 con 0x7f14b8003a00 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 -- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14b8003a00 msgr2=0x7f14b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3e00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14b8003a00 0x7f14b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d8e00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14b80051c0 0x7f14b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14b8003a00 0x7f14b8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 -- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14b80051c0 msgr2=0x7f14b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14b80051c0 0x7f14b800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d3400640 1 -- 172.21.15.59:0/2241120525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b8003520 con 0x7f14b8004800 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.106+0000 7f14d8e00640 1 -- 172.21.15.59:0/2241120525 reap_dead start 2024-04-24T06:32:00.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.107+0000 7f14d3400640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14b8004c80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ac0 tx=0x7f14b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0275a256e83af29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.107+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0036070 con 0x7f14b8004800 2024-04-24T06:32:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.107+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c0031400 con 0x7f14b8004800 2024-04-24T06:32:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.107+0000 7f14d2a00640 1 -- 172.21.15.59:0/2241120525 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0009690 con 0x7f14b8004800 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 -- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 msgr2=0x7f14b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14b8004c80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ac0 tx=0x7f14b8005830 comp rx=0 tx=0).stop 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 -- 172.21.15.59:0/2241120525 shutdown_connections 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 --2- 172.21.15.59:0/2241120525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14b8004c80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 -- 172.21.15.59:0/2241120525 >> 172.21.15.59:0/2241120525 conn(0x7f14d4067ea0 msgr2=0x7f14d406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 -- 172.21.15.59:0/2241120525 shutdown_connections 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.108+0000 7f14dae00640 1 -- 172.21.15.59:0/2241120525 wait complete. 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.110+0000 7f14dae00640 1 Processor -- start 2024-04-24T06:32:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.110+0000 7f14dae00640 1 -- start start 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14d414d5a0 0x7f14d415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14d415f990 0x7f14d41fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41668e0 con 0x7f14b8004800 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4167680 con 0x7f14d415f990 2024-04-24T06:32:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14dae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f14d4166a60 con 0x7f14d414d5a0 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55730/0 (socket says 172.21.15.59:55730) 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.111+0000 7f14d3e00640 1 -- 172.21.15.59:0/1797218520 learned_addr learned my addr 172.21.15.59:0/1797218520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 -- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14d414d5a0 msgr2=0x7f14d415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14d414d5a0 0x7f14d415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 -- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14d415f990 msgr2=0x7f14d41fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14d415f990 0x7f14d41fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 -- 172.21.15.59:0/1797218520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d4158d00 con 0x7f14b8004800 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14d3e00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f14c400d7e0 tx=0x7f14c400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4019070 con 0x7f14b8004800 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c4007560 con 0x7f14b8004800 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4004a00 con 0x7f14b8004800 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.112+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d40cefe0 con 0x7f14b8004800 2024-04-24T06:32:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.113+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d40cf5a0 con 0x7f14b8004800 2024-04-24T06:32:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.114+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d415fdb0 con 0x7f14b8004800 2024-04-24T06:32:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.115+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14c401c070 con 0x7f14b8004800 2024-04-24T06:32:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.116+0000 7f14cbe00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 0x7f14b406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.116+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f14c408b1d0 con 0x7f14b8004800 2024-04-24T06:32:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.116+0000 7f14d3400640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 0x7f14b406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.117+0000 7f14d3400640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 0x7f14b406ee30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14c0033000 tx=0x7f14c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:00.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.121+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14c4059320 con 0x7f14b8004800 2024-04-24T06:32:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.397+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f14d4068a80 con 0x7f14b8004800 2024-04-24T06:32:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.943+0000 7f14cbe00640 1 -- 172.21.15.59:0/1797218520 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v190) ==== 172+0+0 (secure 0 0 0) 0x7f14c405d620 con 0x7f14b8004800 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 msgr2=0x7f14b406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 0x7f14b406ee30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14c0033000 tx=0x7f14c002e000 comp rx=0 tx=0).stop 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 msgr2=0x7f14d41587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f14c400d7e0 tx=0x7f14c400dca0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 shutdown_connections 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f14b406c970 0x7f14b406ee30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14d415f990 0x7f14d41fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f14d414d5a0 0x7f14d415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 --2- 172.21.15.59:0/1797218520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b8004800 0x7f14d41587c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:00.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 >> 172.21.15.59:0/1797218520 conn(0x7f14d4067ea0 msgr2=0x7f14d415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:00.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 shutdown_connections 2024-04-24T06:32:00.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:00.947+0000 7f14dae00640 1 -- 172.21.15.59:0/1797218520 wait complete. 2024-04-24T06:32:00.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:32:01.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.182+0000 7f2a54a00640 1 Processor -- start 2024-04-24T06:32:01.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.183+0000 7f2a54a00640 1 -- start start 2024-04-24T06:32:01.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.184+0000 7f2a54a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a5006bc40 0x7f2a500cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:01.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.184+0000 7f2a54a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a500cf0f0 con 0x7f2a5015d5e0 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.184+0000 7f2a54a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a500d0ea0 con 0x7f2a5006c020 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.184+0000 7f2a54a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2a50160a40 con 0x7f2a5006bc40 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a5006bc40 0x7f2a500cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a5006bc40 0x7f2a500cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55732/0 (socket says 172.21.15.59:55732) 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 -- 172.21.15.59:0/2586675435 learned_addr learned my addr 172.21.15.59:0/2586675435 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2a5015d5e0 0x7f2a500d0790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36362/0 (socket says 172.21.15.59:36362) 2024-04-24T06:32:01.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 -- 172.21.15.59:0/2586675435 >> v1:172.21.15.66:6789/0 conn(0x7f2a5006c020 legacy=0x7f2a50160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 -- 172.21.15.59:0/2586675435 >> v1:172.21.15.59:6789/0 conn(0x7f2a5015d5e0 legacy=0x7f2a500d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4de00640 1 -- 172.21.15.59:0/2586675435 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a50160d20 con 0x7f2a5006bc40 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.185+0000 7f2a4d400640 1 -- 172.21.15.59:0/2586675435 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797710096 0 0) 0x7f2a500cf0f0 con 0x7f2a5015d5e0 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.186+0000 7f2a4de00640 1 --2- 172.21.15.59:0/2586675435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006bc40 0x7f2a500cdac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009870 tx=0x7f2a4002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f40e37e9d2cd5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a4d400640 1 -- 172.21.15.59:0/2586675435 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40037070 con 0x7f2a5006bc40 2024-04-24T06:32:01.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a4d400640 1 -- 172.21.15.59:0/2586675435 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a40032440 con 0x7f2a5006bc40 2024-04-24T06:32:01.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a4d400640 1 -- 172.21.15.59:0/2586675435 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40032810 con 0x7f2a5006bc40 2024-04-24T06:32:01.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 -- 172.21.15.59:0/2586675435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006bc40 msgr2=0x7f2a500cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 --2- 172.21.15.59:0/2586675435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006bc40 0x7f2a500cdac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009870 tx=0x7f2a4002a2a0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 -- 172.21.15.59:0/2586675435 shutdown_connections 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 --2- 172.21.15.59:0/2586675435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006bc40 0x7f2a500cdac0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 -- 172.21.15.59:0/2586675435 >> 172.21.15.59:0/2586675435 conn(0x7f2a50073430 msgr2=0x7f2a5009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 -- 172.21.15.59:0/2586675435 shutdown_connections 2024-04-24T06:32:01.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.187+0000 7f2a54a00640 1 -- 172.21.15.59:0/2586675435 wait complete. 2024-04-24T06:32:01.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.189+0000 7f2a54a00640 1 Processor -- start 2024-04-24T06:32:01.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.189+0000 7f2a54a00640 1 -- start start 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5006bc40 0x7f2a500cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a5015d5e0 0x7f2a50151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5015da90 con 0x7f2a5006c020 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500cf390 con 0x7f2a5006bc40 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a54a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2a500cf8b0 con 0x7f2a5015d5e0 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55736/0 (socket says 172.21.15.59:55736) 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.190+0000 7f2a47e00640 1 -- 172.21.15.59:0/4173679139 learned_addr learned my addr 172.21.15.59:0/4173679139 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a4de00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5006bc40 0x7f2a500cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 -- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a5015d5e0 msgr2=0x7f2a50151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a5015d5e0 0x7f2a50151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 -- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5006bc40 msgr2=0x7f2a500cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5006bc40 0x7f2a500cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 -- 172.21.15.59:0/4173679139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a500b48b0 con 0x7f2a5006c020 2024-04-24T06:32:01.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a47e00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c400 tx=0x7f2a3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:01.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38011790 con 0x7f2a5006c020 2024-04-24T06:32:01.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a38017070 con 0x7f2a5006c020 2024-04-24T06:32:01.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38011bb0 con 0x7f2a5006c020 2024-04-24T06:32:01.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.191+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a500c7bb0 con 0x7f2a5006c020 2024-04-24T06:32:01.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.192+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a500b7110 con 0x7f2a5006c020 2024-04-24T06:32:01.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.193+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50069190 con 0x7f2a5006c020 2024-04-24T06:32:01.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.194+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a3801c460 con 0x7f2a5006c020 2024-04-24T06:32:01.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.195+0000 7f2a46000640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:01.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.195+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f2a3808b710 con 0x7f2a5006c020 2024-04-24T06:32:01.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.195+0000 7f2a4de00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:01.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.196+0000 7f2a4de00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2a40034000 tx=0x7f2a4002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:01.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.199+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a38059da0 con 0x7f2a5006c020 2024-04-24T06:32:01.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.404+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2a5015dff0 con 0x7f2a5006c020 2024-04-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.411+0000 7f2a46000640 1 -- 172.21.15.59:0/4173679139 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2a380596a0 con 0x7f2a5006c020 2024-04-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 msgr2=0x7f2a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2a40034000 tx=0x7f2a4002e000 comp rx=0 tx=0).stop 2024-04-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 msgr2=0x7f2a500b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c400 tx=0x7f2a3800c8c0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 shutdown_connections 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2a5015d5e0 0x7f2a50151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5006c020 0x7f2a500b4370 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 --2- 172.21.15.59:0/4173679139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5006bc40 0x7f2a500cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.414+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 >> 172.21.15.59:0/4173679139 conn(0x7f2a50073430 msgr2=0x7f2a5009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.415+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 shutdown_connections 2024-04-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:01.415+0000 7f2a54a00640 1 -- 172.21.15.59:0/4173679139 wait complete. 2024-04-24T06:32:01.436 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:32:01.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:32:01.547 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:32:01.676 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:32:01.676 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:32:01.676 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:32:01.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:32:01.737 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:32:01.737 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:32:01.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:32:01.798 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:32:01.798 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:32:01.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:32:01.860 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:32:01.860 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:32:01.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:32:01.922 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:32:01.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:32:02.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.138+0000 7fc561c00640 1 Processor -- start 2024-04-24T06:32:02.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.139+0000 7fc561c00640 1 -- start start 2024-04-24T06:32:02.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 0x7fc55c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc55c0d2630 0x7fc55c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c0b6530 0x7fc55c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc55c0b7000 con 0x7fc55c0d2630 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc55c0cf620 con 0x7fc55c15e7c0 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.140+0000 7fc561c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc55c0cf030 con 0x7fc55c0b6530 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc55c0d2630 0x7fc55c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc55c0d2630 0x7fc55c0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55744/0 (socket says 172.21.15.59:55744) 2024-04-24T06:32:02.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 -- 172.21.15.59:0/516247166 learned_addr learned my addr 172.21.15.59:0/516247166 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:02.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55be00640 1 --2- 172.21.15.59:0/516247166 >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 0x7fc55c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 -- 172.21.15.59:0/516247166 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c0b6530 msgr2=0x7fc55c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:02.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc561200640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c0b6530 0x7fc55c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c0b6530 0x7fc55c0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 -- 172.21.15.59:0/516247166 >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 msgr2=0x7fc55c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 --2- 172.21.15.59:0/516247166 >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 0x7fc55c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55b400640 1 -- 172.21.15.59:0/516247166 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c0b6a40 con 0x7fc55c0d2630 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.141+0000 7fc55be00640 1 --2- 172.21.15.59:0/516247166 >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 0x7fc55c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.142+0000 7fc55b400640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0b5ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc55c08c6d0 tx=0x7fc54802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de78c9041dcadd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.142+0000 7fc55aa00640 1 -- 172.21.15.59:0/516247166 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548037070 con 0x7fc55c0d2630 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc55aa00640 1 -- 172.21.15.59:0/516247166 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc548032440 con 0x7fc55c0d2630 2024-04-24T06:32:02.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc55aa00640 1 -- 172.21.15.59:0/516247166 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548032810 con 0x7fc55c0d2630 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 -- 172.21.15.59:0/516247166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 msgr2=0x7fc55c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0b5ff0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc55c08c6d0 tx=0x7fc54802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 -- 172.21.15.59:0/516247166 shutdown_connections 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c0b6530 0x7fc55c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 --2- 172.21.15.59:0/516247166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0b5ff0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 --2- 172.21.15.59:0/516247166 >> v2:172.21.15.66:3300/0 conn(0x7fc55c15e7c0 0x7fc55c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.143+0000 7fc561c00640 1 -- 172.21.15.59:0/516247166 >> 172.21.15.59:0/516247166 conn(0x7fc55c09d920 msgr2=0x7fc55c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.144+0000 7fc561c00640 1 -- 172.21.15.59:0/516247166 shutdown_connections 2024-04-24T06:32:02.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.144+0000 7fc561c00640 1 -- 172.21.15.59:0/516247166 wait complete. 2024-04-24T06:32:02.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.146+0000 7fc561c00640 1 Processor -- start 2024-04-24T06:32:02.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.146+0000 7fc561c00640 1 -- start start 2024-04-24T06:32:02.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 0x7fc55c148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c15e7c0 0x7fc55c161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c0d1a80 con 0x7fc55c0d2630 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c0cf2b0 con 0x7fc55c0b6530 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.147+0000 7fc561c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc55c0d1c00 con 0x7fc55c15e7c0 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55760/0 (socket says 172.21.15.59:55760) 2024-04-24T06:32:02.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 -- 172.21.15.59:0/4279806515 learned_addr learned my addr 172.21.15.59:0/4279806515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:02.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 -- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c15e7c0 msgr2=0x7fc55c161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc561200640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c15e7c0 0x7fc55c161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55be00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 0x7fc55c148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c15e7c0 0x7fc55c161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 -- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 msgr2=0x7fc55c148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 0x7fc55c148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55b400640 1 -- 172.21.15.59:0/4279806515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c14b6f0 con 0x7fc55c0d2630 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.148+0000 7fc55be00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 0x7fc55c148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc55b400640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc548000c00 tx=0x7fc5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548037040 con 0x7fc55c0d2630 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc548042070 con 0x7fc55c0d2630 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55c14b9c0 con 0x7fc55c0d2630 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548030bb0 con 0x7fc55c0d2630 2024-04-24T06:32:02.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.149+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55c14bf80 con 0x7fc55c0d2630 2024-04-24T06:32:02.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.150+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55c04fd90 con 0x7fc55c0d2630 2024-04-24T06:32:02.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.152+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc548047460 con 0x7fc55c0d2630 2024-04-24T06:32:02.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.153+0000 7fc553e00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 0x7fc53c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.153+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fc5480ac560 con 0x7fc55c0d2630 2024-04-24T06:32:02.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.153+0000 7fc55be00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 0x7fc53c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.154+0000 7fc55be00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 0x7fc53c06ede0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc54c006fd0 tx=0x7fc54c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.157+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc548045360 con 0x7fc55c0d2630 2024-04-24T06:32:02.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.400+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc55c0b3bd0 con 0x7fc55c0d2630 2024-04-24T06:32:02.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.402+0000 7fc553e00640 1 -- 172.21.15.59:0/4279806515 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3866 (secure 0 0 0) 0x7fc54807ab60 con 0x7fc55c0d2630 2024-04-24T06:32:02.404 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:02.404 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2732856146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2732856146},{"type":"v1","addr":"172.21.15.59:6836","nonce":2732856146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2978342056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2978342056},{"type":"v1","addr":"172.21.15.66:6834","nonce":2978342056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/479419283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":479419283},{"type":"v1","addr":"172.21.15.66:6835","nonce":479419283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1633972622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1633972622},{"type":"v1","addr":"172.21.15.59:6837","nonce":1633972622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-24T06:32:02.404 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 26 2024-04-24T06:32:02.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 msgr2=0x7fc53c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 0x7fc53c06ede0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc54c006fd0 tx=0x7fc54c008040 comp rx=0 tx=0).stop 2024-04-24T06:32:02.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 msgr2=0x7fc55c0c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc548000c00 tx=0x7fc5480046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 shutdown_connections 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc53c06c920 0x7fc53c06ede0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc55c15e7c0 0x7fc55c161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc55c0d2630 0x7fc55c0c8360 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.404+0000 7fc561c00640 1 --2- 172.21.15.59:0/4279806515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc55c0b6530 0x7fc55c148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.405+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 >> 172.21.15.59:0/4279806515 conn(0x7fc55c09d920 msgr2=0x7fc55c0d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.405+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 shutdown_connections 2024-04-24T06:32:02.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.405+0000 7fc561c00640 1 -- 172.21.15.59:0/4279806515 wait complete. 2024-04-24T06:32:02.427 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:32:02.428 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:32:02.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:32:02.428 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:32:02.428 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:32:02.428 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:32:02.428 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:32:02.429 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:32:02.429 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:32:02.429 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:32:02.430 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:32:02.430 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:32:02.430 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:32:02.430 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:32:02.430 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:32:02.430 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:32:02.447 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:32:02.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:32:02.448 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:32:02.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:32:02.454 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:32:02.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:32:02.457 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:32:02.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:32:02.496 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:32:02.496 DEBUG:teuthology.parallel:result is None 2024-04-24T06:32:02.502 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:32:02.503 DEBUG:teuthology.parallel:result is None 2024-04-24T06:32:02.505 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:32:02.505 DEBUG:teuthology.parallel:result is None 2024-04-24T06:32:02.511 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:32:02.511 DEBUG:teuthology.parallel:result is None 2024-04-24T06:32:02.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:32:02.648 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:32:02.649 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:32:02.671 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:32:02.681 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:32:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.691+0000 7f080ce00640 1 Processor -- start 2024-04-24T06:32:02.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.692+0000 7f080ce00640 1 -- start start 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f080ce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 0x7f08080cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f080ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f080ce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080815fd30 con 0x7f080815ea40 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f080ce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f08080cf930 con 0x7f08080d0f80 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f080ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f08080cfc10 con 0x7f080815f880 2024-04-24T06:32:02.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f0807e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f0807e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-04-24T06:32:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f0807e00640 1 -- 172.21.15.59:0/1844534818 learned_addr learned my addr 172.21.15.59:0/1844534818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:02.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.693+0000 7f0807400640 1 --2- 172.21.15.59:0/1844534818 >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 0x7f08080cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.694+0000 7f0807e00640 1 -- 172.21.15.59:0/1844534818 >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 msgr2=0x7f08080cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.694+0000 7f0807e00640 1 --2- 172.21.15.59:0/1844534818 >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 0x7f08080cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.694+0000 7f0807e00640 1 -- 172.21.15.59:0/1844534818 >> v1:172.21.15.66:6789/0 conn(0x7f080815ea40 legacy=0x7f08080d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.694+0000 7f0807e00640 1 -- 172.21.15.59:0/1844534818 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08080cfef0 con 0x7f080815f880 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.694+0000 7f0807400640 1 --2- 172.21.15.59:0/1844534818 >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 0x7f08080cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.695+0000 7f0807e00640 1 --2- 172.21.15.59:0/1844534818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0800009e50 tx=0x7f080002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cebe9d864db8c0f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.695+0000 7f0806a00640 1 -- 172.21.15.59:0/1844534818 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800036070 con 0x7f080815f880 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.695+0000 7f0806a00640 1 -- 172.21.15.59:0/1844534818 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800031600 con 0x7f080815f880 2024-04-24T06:32:02.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.695+0000 7f0806a00640 1 -- 172.21.15.59:0/1844534818 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000319d0 con 0x7f080815f880 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 -- 172.21.15.59:0/1844534818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 msgr2=0x7f08080d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 --2- 172.21.15.59:0/1844534818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0800009e50 tx=0x7f080002a680 comp rx=0 tx=0).stop 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 -- 172.21.15.59:0/1844534818 shutdown_connections 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 --2- 172.21.15.59:0/1844534818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f080815f880 0x7f08080d2310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 --2- 172.21.15.59:0/1844534818 >> v2:172.21.15.66:3300/0 conn(0x7f08080d0f80 0x7f08080cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 -- 172.21.15.59:0/1844534818 >> 172.21.15.59:0/1844534818 conn(0x7f080808bf90 msgr2=0x7f08080d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 -- 172.21.15.59:0/1844534818 shutdown_connections 2024-04-24T06:32:02.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.696+0000 7f080ce00640 1 -- 172.21.15.59:0/1844534818 wait complete. 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.698+0000 7f080ce00640 1 Processor -- start 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.698+0000 7f080ce00640 1 -- start start 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08080d0f80 0x7f08080ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f080815f880 0x7f08080ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08080d2850 con 0x7f080815ea40 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08080d04b0 con 0x7f080815f880 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f080ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f08080d29d0 con 0x7f08080d0f80 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55796/0 (socket says 172.21.15.59:55796) 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 -- 172.21.15.59:0/2026443590 learned_addr learned my addr 172.21.15.59:0/2026443590 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f0807400640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08080d0f80 0x7f08080ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f0807e00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f080815f880 0x7f08080ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 -- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08080d0f80 msgr2=0x7f08080ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08080d0f80 0x7f08080ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 -- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f080815f880 msgr2=0x7f08080ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f080815f880 0x7f08080ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.699+0000 7f07ffe00640 1 -- 172.21.15.59:0/2026443590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08080b8d10 con 0x7f080815ea40 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f07ffe00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c3a0 tx=0x7f07f000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f00052d0 con 0x7f080815ea40 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f0005490 con 0x7f080815ea40 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08080b8ff0 con 0x7f080815ea40 2024-04-24T06:32:02.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0007260 con 0x7f080815ea40 2024-04-24T06:32:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.700+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08080b95b0 con 0x7f080815ea40 2024-04-24T06:32:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.702+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080804fd90 con 0x7f080815ea40 2024-04-24T06:32:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.702+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07f0007400 con 0x7f080815ea40 2024-04-24T06:32:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.704+0000 7f07ff400640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.704+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f07f008abf0 con 0x7f080815ea40 2024-04-24T06:32:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.704+0000 7f0807400640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.705+0000 7f0807400640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08081f8050 tx=0x7f07f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:02.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.708+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07f0058d60 con 0x7f080815ea40 2024-04-24T06:32:02.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.926+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f08080d1ec0 con 0x7f080815ea40 2024-04-24T06:32:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.979+0000 7f07ff400640 1 -- 172.21.15.59:0/2026443590 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) ==== 97+0+0 (secure 0 0 0) 0x7f07f005d060 con 0x7f080815ea40 2024-04-24T06:32:02.982 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:32:02.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 msgr2=0x7f07e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08081f8050 tx=0x7f07f8008040 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 msgr2=0x7f08080b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c3a0 tx=0x7f07f000c860 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 shutdown_connections 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f080815f880 0x7f08080ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f080815ea40 0x7f08080b4760 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 --2- 172.21.15.59:0/2026443590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08080d0f80 0x7f08080ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 >> 172.21.15.59:0/2026443590 conn(0x7f080808bf90 msgr2=0x7f080815ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.983+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 shutdown_connections 2024-04-24T06:32:02.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:02.984+0000 7f080ce00640 1 -- 172.21.15.59:0/2026443590 wait complete. 2024-04-24T06:32:03.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:32:03.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.208+0000 7f5037c00640 1 Processor -- start 2024-04-24T06:32:03.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.208+0000 7f5037c00640 1 -- start start 2024-04-24T06:32:03.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.209+0000 7f5037c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 0x7f503015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.209+0000 7f5037c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f503015edb0 con 0x7f50300d42c0 2024-04-24T06:32:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.209+0000 7f5037c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50300ba630 con 0x7f503015e9d0 2024-04-24T06:32:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.209+0000 7f5037c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50300d3e30 con 0x7f50300d1670 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.210+0000 7f502fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50300d42c0 0x7f50300d2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36382/0 (socket says 172.21.15.59:36382) 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.210+0000 7f502fe00640 1 -- 172.21.15.59:0/421460787 learned_addr learned my addr 172.21.15.59:0/421460787 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.210+0000 7f5035c00640 1 --2- 172.21.15.59:0/421460787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 0x7f503015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769454483 0 0) 0x7f503015edb0 con 0x7f50300d42c0 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5014003520 con 0x7f50300d42c0 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f5035c00640 1 -- 172.21.15.59:0/421460787 >> v1:172.21.15.66:6789/0 conn(0x7f503015e9d0 legacy=0x7f50300d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f5035c00640 1 -- 172.21.15.59:0/421460787 >> v1:172.21.15.59:6789/0 conn(0x7f50300d42c0 legacy=0x7f50300d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1927955620 0 0) 0x7f5014003520 con 0x7f50300d42c0 2024-04-24T06:32:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.211+0000 7f5035c00640 1 -- 172.21.15.59:0/421460787 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50300cf7b0 con 0x7f50300d1670 2024-04-24T06:32:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.212+0000 7f5035c00640 1 --2- 172.21.15.59:0/421460787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 0x7f503015f9e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50300d3fb0 tx=0x7f502802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4a43894e095e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.213+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028037070 con 0x7f50300d1670 2024-04-24T06:32:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.213+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502802ac10 con 0x7f50300d1670 2024-04-24T06:32:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.213+0000 7f502f400640 1 -- 172.21.15.59:0/421460787 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018001670 con 0x7f50300d1670 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.213+0000 7f5037c00640 1 -- 172.21.15.59:0/421460787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 msgr2=0x7f503015f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.213+0000 7f5037c00640 1 --2- 172.21.15.59:0/421460787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 0x7f503015f9e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50300d3fb0 tx=0x7f502802a180 comp rx=0 tx=0).stop 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.214+0000 7f5037c00640 1 -- 172.21.15.59:0/421460787 shutdown_connections 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.214+0000 7f5037c00640 1 --2- 172.21.15.59:0/421460787 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d1670 0x7f503015f9e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.214+0000 7f5037c00640 1 -- 172.21.15.59:0/421460787 >> 172.21.15.59:0/421460787 conn(0x7f503006cbd0 msgr2=0x7f503006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.214+0000 7f5037c00640 1 -- 172.21.15.59:0/421460787 shutdown_connections 2024-04-24T06:32:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.214+0000 7f5037c00640 1 -- 172.21.15.59:0/421460787 wait complete. 2024-04-24T06:32:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.216+0000 7f5037c00640 1 Processor -- start 2024-04-24T06:32:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.216+0000 7f5037c00640 1 -- start start 2024-04-24T06:32:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50300d1670 0x7f5030153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d42c0 0x7f50301538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50300cfd70 con 0x7f503015e9d0 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f503015ff20 con 0x7f50300d1670 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5037c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50300ba0e0 con 0x7f50300d42c0 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.217+0000 7f5035c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55800/0 (socket says 172.21.15.59:55800) 2024-04-24T06:32:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 -- 172.21.15.59:0/4194592566 learned_addr learned my addr 172.21.15.59:0/4194592566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f502fe00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d42c0 0x7f50301538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035200640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50300d1670 0x7f5030153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 -- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d42c0 msgr2=0x7f50301538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d42c0 0x7f50301538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 -- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50300d1670 msgr2=0x7f5030153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50300d1670 0x7f5030153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 -- 172.21.15.59:0/4194592566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50300cdd80 con 0x7f503015e9d0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.218+0000 7f5035c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f502802a1c0 tx=0x7f50280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.219+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028041070 con 0x7f503015e9d0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.219+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502803c660 con 0x7f503015e9d0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.219+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028037040 con 0x7f503015e9d0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.219+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50300ce060 con 0x7f503015e9d0 2024-04-24T06:32:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.219+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50300ce620 con 0x7f503015e9d0 2024-04-24T06:32:03.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.220+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50301607b0 con 0x7f503015e9d0 2024-04-24T06:32:03.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.221+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5028044070 con 0x7f503015e9d0 2024-04-24T06:32:03.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.222+0000 7f502cc00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.223+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f50280ac1c0 con 0x7f503015e9d0 2024-04-24T06:32:03.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.223+0000 7f5035200640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.223+0000 7f5035200640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5020007fd0 tx=0x7f5020009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.227+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50301607b0 con 0x7f503015e9d0 2024-04-24T06:32:03.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.504+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5030069190 con 0x7f503015e9d0 2024-04-24T06:32:03.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.505+0000 7f502cc00640 1 -- 172.21.15.59:0/4194592566 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f502807a9c0 con 0x7f503015e9d0 2024-04-24T06:32:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:03.507 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:32:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:32:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:32:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:32:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:32:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:32:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:32:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:32:03.523 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:03.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 msgr2=0x7f500c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 0x7f500c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5020007fd0 tx=0x7f5020009040 comp rx=0 tx=0).stop 2024-04-24T06:32:03.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 msgr2=0x7f50300cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f502802a1c0 tx=0x7f50280046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 shutdown_connections 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f500c06c8f0 0x7f500c06edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f503015e9d0 0x7f50300cb890 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300d42c0 0x7f50301538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 --2- 172.21.15.59:0/4194592566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50300d1670 0x7f5030153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.507+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 >> 172.21.15.59:0/4194592566 conn(0x7f503006cbd0 msgr2=0x7f50301600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.508+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 shutdown_connections 2024-04-24T06:32:03.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.508+0000 7f5037c00640 1 -- 172.21.15.59:0/4194592566 wait complete. 2024-04-24T06:32:03.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:32:03.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.731+0000 7f986a200640 1 Processor -- start 2024-04-24T06:32:03.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.731+0000 7f986a200640 1 -- start start 2024-04-24T06:32:03.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.732+0000 7f986a200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f986415e7c0 0x7f98640d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.732+0000 7f986a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.732+0000 7f986a200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98640cf030 con 0x7f98640b6530 2024-04-24T06:32:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.732+0000 7f986a200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f98640b7000 con 0x7f986415e7c0 2024-04-24T06:32:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.732+0000 7f986a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f98640cf620 con 0x7f98640d2630 2024-04-24T06:32:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.733+0000 7f9863e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.733+0000 7f9863e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:46612/0 (socket says 172.21.15.59:46612) 2024-04-24T06:32:03.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.733+0000 7f9863e00640 1 -- 172.21.15.59:0/2739826903 learned_addr learned my addr 172.21.15.59:0/2739826903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.733+0000 7f9868e00640 1 --2- 172.21.15.59:0/2739826903 >> v2:172.21.15.66:3300/0 conn(0x7f986415e7c0 0x7f98640d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.733+0000 7f9869800640 1 --1- 172.21.15.59:0/2739826903 >> v1:172.21.15.66:6789/0 conn(0x7f98640b6530 0x7f98640d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:45804/0 (socket says 172.21.15.59:45804) 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.734+0000 7f9863e00640 1 -- 172.21.15.59:0/2739826903 >> v2:172.21.15.66:3300/0 conn(0x7f986415e7c0 msgr2=0x7f98640d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.734+0000 7f9863e00640 1 --2- 172.21.15.59:0/2739826903 >> v2:172.21.15.66:3300/0 conn(0x7f986415e7c0 0x7f98640d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.734+0000 7f9863e00640 1 -- 172.21.15.59:0/2739826903 >> v1:172.21.15.66:6789/0 conn(0x7f98640b6530 legacy=0x7f98640d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.734+0000 7f9863e00640 1 -- 172.21.15.59:0/2739826903 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98640b6a40 con 0x7f98640d2630 2024-04-24T06:32:03.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.735+0000 7f9863e00640 1 --2- 172.21.15.59:0/2739826903 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9850009870 tx=0x7f985002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ca673427542c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.736+0000 7f9863400640 1 -- 172.21.15.59:0/2739826903 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850037070 con 0x7f98640d2630 2024-04-24T06:32:03.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.736+0000 7f9863400640 1 -- 172.21.15.59:0/2739826903 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9850032440 con 0x7f98640d2630 2024-04-24T06:32:03.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.736+0000 7f9863400640 1 -- 172.21.15.59:0/2739826903 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850032810 con 0x7f98640d2630 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 -- 172.21.15.59:0/2739826903 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 msgr2=0x7f98640b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 --2- 172.21.15.59:0/2739826903 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9850009870 tx=0x7f985002a250 comp rx=0 tx=0).stop 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 -- 172.21.15.59:0/2739826903 shutdown_connections 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 --2- 172.21.15.59:0/2739826903 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640d2630 0x7f98640b5ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 --2- 172.21.15.59:0/2739826903 >> v2:172.21.15.66:3300/0 conn(0x7f986415e7c0 0x7f98640d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 -- 172.21.15.59:0/2739826903 >> 172.21.15.59:0/2739826903 conn(0x7f986409d920 msgr2=0x7f986408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 -- 172.21.15.59:0/2739826903 shutdown_connections 2024-04-24T06:32:03.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.737+0000 7f986a200640 1 -- 172.21.15.59:0/2739826903 wait complete. 2024-04-24T06:32:03.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.739+0000 7f986a200640 1 Processor -- start 2024-04-24T06:32:03.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.739+0000 7f986a200640 1 -- start start 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640b6530 0x7f986414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986415e7c0 0x7f98640cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98640d1a80 con 0x7f98640d2630 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98640cf2b0 con 0x7f986415e7c0 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f986a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f98640d1c00 con 0x7f98640b6530 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55816/0 (socket says 172.21.15.59:55816) 2024-04-24T06:32:03.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 -- 172.21.15.59:0/2899308653 learned_addr learned my addr 172.21.15.59:0/2899308653 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 -- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640b6530 msgr2=0x7f986414f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640b6530 0x7f986414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 -- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986415e7c0 msgr2=0x7f98640cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986415e7c0 0x7f98640cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.740+0000 7f9863e00640 1 -- 172.21.15.59:0/2899308653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98640b2490 con 0x7f98640d2630 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f9863e00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f98500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850037040 con 0x7f98640d2630 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9850042070 con 0x7f98640d2630 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98640b2770 con 0x7f98640d2630 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850030880 con 0x7f98640d2630 2024-04-24T06:32:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.741+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986415a7f0 con 0x7f98640d2630 2024-04-24T06:32:03.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.742+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986404fd90 con 0x7f98640d2630 2024-04-24T06:32:03.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.744+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9850030a20 con 0x7f98640d2630 2024-04-24T06:32:03.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.744+0000 7f9860c00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 0x7f984406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:03.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.745+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f98500ac0a0 con 0x7f98640d2630 2024-04-24T06:32:03.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.745+0000 7f9868e00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 0x7f984406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:03.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.746+0000 7f9868e00640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 0x7f984406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9854006a90 tx=0x7f9854006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:03.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.749+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9850045360 con 0x7f98640d2630 2024-04-24T06:32:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.957+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986415d530 con 0x7f98640d2630 2024-04-24T06:32:03.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.958+0000 7f9860c00640 1 -- 172.21.15.59:0/2899308653 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+12807 (secure 0 0 0) 0x7f985007a810 con 0x7f98640d2630 2024-04-24T06:32:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:03.961 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:02.997333+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 msgr2=0x7f984406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 0x7f984406edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9854006a90 tx=0x7f9854006a20 comp rx=0 tx=0).stop 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 msgr2=0x7f98640cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f98500046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 shutdown_connections 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f984406c8f0 0x7f984406edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f986415e7c0 0x7f98640cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98640d2630 0x7f98640cd7a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 --2- 172.21.15.59:0/2899308653 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98640b6530 0x7f986414f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 >> 172.21.15.59:0/2899308653 conn(0x7f986409d920 msgr2=0x7f98640b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 shutdown_connections 2024-04-24T06:32:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:03.961+0000 7f986a200640 1 -- 172.21.15.59:0/2899308653 wait complete. 2024-04-24T06:32:03.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:32:03.983 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:32:04.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.194+0000 7fc50f200640 1 Processor -- start 2024-04-24T06:32:04.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.194+0000 7fc50f200640 1 -- start start 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc508145e10 0x7fc50806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 0x7fc5080cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 0x7fc508072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5080cfc20 con 0x7fc508145e10 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc5080cff00 con 0x7fc5080d0cc0 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc50f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc5080d01e0 con 0x7fc508071d70 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc507e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc508145e10 0x7fc50806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc507e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc508145e10 0x7fc50806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55832/0 (socket says 172.21.15.59:55832) 2024-04-24T06:32:04.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.196+0000 7fc507e00640 1 -- 172.21.15.59:0/3940179017 learned_addr learned my addr 172.21.15.59:0/3940179017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:04.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507e00640 1 -- 172.21.15.59:0/3940179017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 msgr2=0x7fc508072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:04.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507400640 1 --2- 172.21.15.59:0/3940179017 >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 0x7fc5080cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc50d200640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 0x7fc508072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507e00640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 0x7fc508072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507e00640 1 -- 172.21.15.59:0/3940179017 >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 msgr2=0x7fc5080cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507e00640 1 --2- 172.21.15.59:0/3940179017 >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 0x7fc5080cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507e00640 1 -- 172.21.15.59:0/3940179017 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080d04c0 con 0x7fc508145e10 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc507400640 1 --2- 172.21.15.59:0/3940179017 >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 0x7fc5080cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.197+0000 7fc50d200640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 0x7fc508072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:04.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.198+0000 7fc507e00640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50806f130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b5740 tx=0x7fc4f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96c64d190ef2472 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.199+0000 7fc506a00640 1 -- 172.21.15.59:0/3940179017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8037070 con 0x7fc508145e10 2024-04-24T06:32:04.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.199+0000 7fc506a00640 1 -- 172.21.15.59:0/3940179017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8032440 con 0x7fc508145e10 2024-04-24T06:32:04.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.199+0000 7fc506a00640 1 -- 172.21.15.59:0/3940179017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8032810 con 0x7fc508145e10 2024-04-24T06:32:04.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.199+0000 7fc50f200640 1 -- 172.21.15.59:0/3940179017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 msgr2=0x7fc50806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.199+0000 7fc50f200640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50806f130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b5740 tx=0x7fc4f802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 -- 172.21.15.59:0/3940179017 shutdown_connections 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc508071d70 0x7fc508072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 --2- 172.21.15.59:0/3940179017 >> v2:172.21.15.66:3300/0 conn(0x7fc5080d0cc0 0x7fc5080cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 --2- 172.21.15.59:0/3940179017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50806f130 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 -- 172.21.15.59:0/3940179017 >> 172.21.15.59:0/3940179017 conn(0x7fc508067d90 msgr2=0x7fc50806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 -- 172.21.15.59:0/3940179017 shutdown_connections 2024-04-24T06:32:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.200+0000 7fc50f200640 1 -- 172.21.15.59:0/3940179017 wait complete. 2024-04-24T06:32:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.201+0000 7fc50f200640 1 Processor -- start 2024-04-24T06:32:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.202+0000 7fc50f200640 1 -- start start 2024-04-24T06:32:04.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc508071d70 0x7fc5081e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5080d0cc0 0x7fc5080b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080cfa00 con 0x7fc508145e10 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080cf880 con 0x7fc508071d70 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc5081461f0 con 0x7fc5080d0cc0 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55848/0 (socket says 172.21.15.59:55848) 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50d200640 1 -- 172.21.15.59:0/1291822509 learned_addr learned my addr 172.21.15.59:0/1291822509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc507e00640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc508071d70 0x7fc5081e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc507400640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5080d0cc0 0x7fc5080b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50d200640 1 -- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5080d0cc0 msgr2=0x7fc5080b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.203+0000 7fc50d200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5080d0cc0 0x7fc5080b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50d200640 1 -- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc508071d70 msgr2=0x7fc5081e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50d200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc508071d70 0x7fc5081e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50d200640 1 -- 172.21.15.59:0/1291822509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080af2d0 con 0x7fc508145e10 2024-04-24T06:32:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50d200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc50000cd10 tx=0x7fc50000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50000f680 con 0x7fc508145e10 2024-04-24T06:32:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50000f840 con 0x7fc508145e10 2024-04-24T06:32:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50000fc30 con 0x7fc508145e10 2024-04-24T06:32:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5080af5b0 con 0x7fc508145e10 2024-04-24T06:32:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.204+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5080afb70 con 0x7fc508145e10 2024-04-24T06:32:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.205+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc508068a80 con 0x7fc508145e10 2024-04-24T06:32:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.207+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc50000e3e0 con 0x7fc508145e10 2024-04-24T06:32:04.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.208+0000 7fc4ffe00640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.208+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fc50008c4f0 con 0x7fc508145e10 2024-04-24T06:32:04.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.208+0000 7fc507e00640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.209+0000 7fc507e00640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 0x7fc4e406edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034000 tx=0x7fc4f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.213+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc500014030 con 0x7fc508145e10 2024-04-24T06:32:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.422+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5080d1190 con 0x7fc508145e10 2024-04-24T06:32:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.423+0000 7fc4ffe00640 1 -- 172.21.15.59:0/1291822509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fc50005ad00 con 0x7fc508145e10 2024-04-24T06:32:04.425 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 msgr2=0x7fc4e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 0x7fc4e406edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034000 tx=0x7fc4f802e000 comp rx=0 tx=0).stop 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 msgr2=0x7fc50815bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc50000cd10 tx=0x7fc50000a650 comp rx=0 tx=0).stop 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 shutdown_connections 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc508145e10 0x7fc50815bd00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5080d0cc0 0x7fc5080b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 --2- 172.21.15.59:0/1291822509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc508071d70 0x7fc5081e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 >> 172.21.15.59:0/1291822509 conn(0x7fc508067d90 msgr2=0x7fc5081440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 shutdown_connections 2024-04-24T06:32:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.426+0000 7fc50f200640 1 -- 172.21.15.59:0/1291822509 wait complete. 2024-04-24T06:32:04.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:32:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.655+0000 7fb1c9c00640 1 Processor -- start 2024-04-24T06:32:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.656+0000 7fb1c9c00640 1 -- start start 2024-04-24T06:32:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.657+0000 7fb1c9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1c40d08b0 0x7fb1c4169860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.657+0000 7fb1c9c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c415f0f0 con 0x7fb1c40afaf0 2024-04-24T06:32:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.657+0000 7fb1c9c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c415f3d0 con 0x7fb1c415c5e0 2024-04-24T06:32:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.657+0000 7fb1c9c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c415fa20 con 0x7fb1c40d08b0 2024-04-24T06:32:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.658+0000 7fb1c2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb1c40afaf0 0x7fb1c40d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36394/0 (socket says 172.21.15.59:36394) 2024-04-24T06:32:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.658+0000 7fb1c2000640 1 -- 172.21.15.59:0/472254150 learned_addr learned my addr 172.21.15.59:0/472254150 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.658+0000 7fb1c3400640 1 --2- 172.21.15.59:0/472254150 >> v2:172.21.15.66:3300/0 conn(0x7fb1c40d08b0 0x7fb1c4169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.658+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246834121 0 0) 0x7fb1c415f0f0 con 0x7fb1c40afaf0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.658+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1a8003520 con 0x7fb1c40afaf0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 888882469 0 0) 0x7fb1a8003520 con 0x7fb1c40afaf0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 >> v2:172.21.15.66:3300/0 conn(0x7fb1c40d08b0 msgr2=0x7fb1c4169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 --2- 172.21.15.59:0/472254150 >> v2:172.21.15.66:3300/0 conn(0x7fb1c40d08b0 0x7fb1c4169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 >> v1:172.21.15.66:6789/0 conn(0x7fb1c415c5e0 legacy=0x7fb1c40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c415f6b0 con 0x7fb1c40afaf0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb1ac001670 con 0x7fb1c40afaf0 2024-04-24T06:32:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 >> v1:172.21.15.59:6789/0 conn(0x7fb1c40afaf0 legacy=0x7fb1c40d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 0x7fb1a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a8004800 0x7fb1a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a80051c0 0x7fb1a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c415fa20 con 0x7fb1a8003a00 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a800c0c0 con 0x7fb1a80051c0 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a800c2c0 con 0x7fb1a8004800 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 0x7fb1a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.659+0000 7fb1c2000640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a8004800 0x7fb1a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c2a00640 1 -- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a8004800 msgr2=0x7fb1a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a8004800 0x7fb1a8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c3400640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a80051c0 0x7fb1a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c2a00640 1 -- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a80051c0 msgr2=0x7fb1a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a80051c0 0x7fb1a800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c2a00640 1 -- 172.21.15.59:0/472254150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a8003520 con 0x7fb1a8003a00 2024-04-24T06:32:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.660+0000 7fb1c3400640 1 -- 172.21.15.59:0/472254150 reap_dead start 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.661+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 0x7fb1a8003e20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4001e50 tx=0x7fb1a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef7e3a52d28a17b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.661+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4001a70 con 0x7fb1a8003a00 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.661+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b402add0 con 0x7fb1a8003a00 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.661+0000 7fb1c1600640 1 -- 172.21.15.59:0/472254150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4003bb0 con 0x7fb1a8003a00 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 -- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 msgr2=0x7fb1a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 0x7fb1a8003e20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4001e50 tx=0x7fb1a8005830 comp rx=0 tx=0).stop 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 -- 172.21.15.59:0/472254150 shutdown_connections 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/472254150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8003a00 0x7fb1a8003e20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 -- 172.21.15.59:0/472254150 >> 172.21.15.59:0/472254150 conn(0x7fb1c4067ea0 msgr2=0x7fb1c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 -- 172.21.15.59:0/472254150 shutdown_connections 2024-04-24T06:32:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.662+0000 7fb1c9c00640 1 -- 172.21.15.59:0/472254150 wait complete. 2024-04-24T06:32:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.664+0000 7fb1c9c00640 1 Processor -- start 2024-04-24T06:32:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.664+0000 7fb1c9c00640 1 -- start start 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a8003a00 0x7fb1c415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1c415e2d0 0x7fb1c4158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c40d0320 con 0x7fb1c4158b70 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c40d01a0 con 0x7fb1a8003a00 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c9c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c40d04a0 con 0x7fb1c415e2d0 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55880/0 (socket says 172.21.15.59:55880) 2024-04-24T06:32:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 -- 172.21.15.59:0/2938280868 learned_addr learned my addr 172.21.15.59:0/2938280868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c2000640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1c415e2d0 0x7fb1c4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 -- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1c415e2d0 msgr2=0x7fb1c4158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1c415e2d0 0x7fb1c4158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 -- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a8003a00 msgr2=0x7fb1c415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.665+0000 7fb1c3400640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a8003a00 0x7fb1c415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1c3400640 1 -- 172.21.15.59:0/2938280868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c415ccf0 con 0x7fb1c4158b70 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1c3400640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40af1a0 tx=0x7fb1bc010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc00f040 con 0x7fb1c4158b70 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1bc01b070 con 0x7fb1c4158b70 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc004490 con 0x7fb1c4158b70 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c415cfd0 con 0x7fb1c4158b70 2024-04-24T06:32:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.666+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c415d590 con 0x7fb1c4158b70 2024-04-24T06:32:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.667+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c4068a80 con 0x7fb1c4158b70 2024-04-24T06:32:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.668+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1bc003610 con 0x7fb1c4158b70 2024-04-24T06:32:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.669+0000 7fb1baa00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 0x7fb1a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.669+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fb1bc08b6a0 con 0x7fb1c4158b70 2024-04-24T06:32:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.670+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 0x7fb1a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.670+0000 7fb1c2a00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 0x7fb1a006ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4032000 tx=0x7fb1b402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:04.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.674+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1bc059d80 con 0x7fb1c4158b70 2024-04-24T06:32:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.880+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1c406e260 con 0x7fb1c4158b70 2024-04-24T06:32:04.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.881+0000 7fb1baa00640 1 -- 172.21.15.59:0/2938280868 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fb1bc059680 con 0x7fb1c4158b70 2024-04-24T06:32:04.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:04.884 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:32:04.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 msgr2=0x7fb1a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 0x7fb1a006ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4032000 tx=0x7fb1b402c000 comp rx=0 tx=0).stop 2024-04-24T06:32:04.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 msgr2=0x7fb1c4148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40af1a0 tx=0x7fb1bc010c20 comp rx=0 tx=0).stop 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 shutdown_connections 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1a006c920 0x7fb1a006ede0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1c4158b70 0x7fb1c4148070 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1c415e2d0 0x7fb1c4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 --2- 172.21.15.59:0/2938280868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a8003a00 0x7fb1c415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.884+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 >> 172.21.15.59:0/2938280868 conn(0x7fb1c4067ea0 msgr2=0x7fb1c406f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.885+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 shutdown_connections 2024-04-24T06:32:04.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:04.885+0000 7fb1c9c00640 1 -- 172.21.15.59:0/2938280868 wait complete. 2024-04-24T06:32:04.910 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:32:04.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:32:05.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.133+0000 7fc8d2800640 1 Processor -- start 2024-04-24T06:32:05.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.133+0000 7fc8d2800640 1 -- start start 2024-04-24T06:32:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.135+0000 7fc8d2800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8cc161fd0 0x7fc8cc0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.135+0000 7fc8d2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.135+0000 7fc8d2800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8cc0d0310 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.135+0000 7fc8d2800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc8cc0d1a40 con 0x7fc8cc161fd0 2024-04-24T06:32:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.135+0000 7fc8d2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc0d05f0 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.136+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.136+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:46672/0 (socket says 172.21.15.59:46672) 2024-04-24T06:32:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.136+0000 7fc8cbe00640 1 -- 172.21.15.59:0/4207101383 learned_addr learned my addr 172.21.15.59:0/4207101383 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:05.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.136+0000 7fc8cb400640 1 --2- 172.21.15.59:0/4207101383 >> v2:172.21.15.66:3300/0 conn(0x7fc8cc161fd0 0x7fc8cc0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.137+0000 7fc8cbe00640 1 -- 172.21.15.59:0/4207101383 >> v2:172.21.15.66:3300/0 conn(0x7fc8cc161fd0 msgr2=0x7fc8cc0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.137+0000 7fc8cbe00640 1 --2- 172.21.15.59:0/4207101383 >> v2:172.21.15.66:3300/0 conn(0x7fc8cc161fd0 0x7fc8cc0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.137+0000 7fc8cbe00640 1 -- 172.21.15.59:0/4207101383 >> v1:172.21.15.66:6789/0 conn(0x7fc8cc0d10a0 legacy=0x7fc8cc0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.137+0000 7fc8ca000640 1 -- 172.21.15.59:0/4207101383 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257884733 0 0) 0x7fc8cc0d0310 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.137+0000 7fc8cbe00640 1 -- 172.21.15.59:0/4207101383 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc0d2520 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.138+0000 7fc8cbe00640 1 --2- 172.21.15.59:0/4207101383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009870 tx=0x7fc8c402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1450bbc5af6cefba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:05.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.139+0000 7fc8ca000640 1 -- 172.21.15.59:0/4207101383 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4037070 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.139+0000 7fc8ca000640 1 -- 172.21.15.59:0/4207101383 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c4032440 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.139+0000 7fc8ca000640 1 -- 172.21.15.59:0/4207101383 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4032810 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 -- 172.21.15.59:0/4207101383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 msgr2=0x7fc8cc0d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 --2- 172.21.15.59:0/4207101383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009870 tx=0x7fc8c402a270 comp rx=0 tx=0).stop 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 -- 172.21.15.59:0/4207101383 shutdown_connections 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 --2- 172.21.15.59:0/4207101383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc0d1500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 --2- 172.21.15.59:0/4207101383 >> v2:172.21.15.66:3300/0 conn(0x7fc8cc161fd0 0x7fc8cc0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 -- 172.21.15.59:0/4207101383 >> 172.21.15.59:0/4207101383 conn(0x7fc8cc08bf90 msgr2=0x7fc8cc0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 -- 172.21.15.59:0/4207101383 shutdown_connections 2024-04-24T06:32:05.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.140+0000 7fc8d2800640 1 -- 172.21.15.59:0/4207101383 wait complete. 2024-04-24T06:32:05.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.143+0000 7fc8d2800640 1 Processor -- start 2024-04-24T06:32:05.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.143+0000 7fc8d2800640 1 -- start start 2024-04-24T06:32:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8cc161fd0 0x7fc8cc0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0d2d20 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0b4610 con 0x7fc8cc161fd0 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.144+0000 7fc8d2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc0d2ea0 con 0x7fc8cc0cfcf0 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55890/0 (socket says 172.21.15.59:55890) 2024-04-24T06:32:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 -- 172.21.15.59:0/1497282174 learned_addr learned my addr 172.21.15.59:0/1497282174 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8cb400640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8cbe00640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8cc161fd0 0x7fc8cc0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 -- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 msgr2=0x7fc8cc15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 -- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8cc161fd0 msgr2=0x7fc8cc0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8cc161fd0 0x7fc8cc0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.145+0000 7fc8caa00640 1 -- 172.21.15.59:0/1497282174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc15eb90 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8caa00640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c490 tx=0x7fc8b400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4005480 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b40035f0 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40039c0 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc0cbb90 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.146+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc0cc150 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.147+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc068a80 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.149+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b4003b60 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.150+0000 7fc8c3400640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 0x7fc8a806efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.150+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fc8b408b3e0 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.150+0000 7fc8cb400640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 0x7fc8a806efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.151+0000 7fc8cb400640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 0x7fc8a806efa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc1f9210 tx=0x7fc8bc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:05.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.155+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8b40182b0 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.358+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc8cc0d1ea0 con 0x7fc8cc0d10a0 2024-04-24T06:32:05.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:32:05.852 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:32:06.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:05.998+0000 7fc8c3400640 1 -- 172.21.15.59:0/1497282174 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7fc8b4059bd0 con 0x7fc8cc0d10a0 2024-04-24T06:32:06.001 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:32:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 msgr2=0x7fc8a806efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 0x7fc8a806efa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc1f9210 tx=0x7fc8bc00b040 comp rx=0 tx=0).stop 2024-04-24T06:32:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 msgr2=0x7fc8cc0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c490 tx=0x7fc8b400c950 comp rx=0 tx=0).stop 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 shutdown_connections 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8a806cae0 0x7fc8a806efa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8cc161fd0 0x7fc8cc0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8cc0d10a0 0x7fc8cc0ccc30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 --2- 172.21.15.59:0/1497282174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8cc0cfcf0 0x7fc8cc15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 >> 172.21.15.59:0/1497282174 conn(0x7fc8cc08bf90 msgr2=0x7fc8cc0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 shutdown_connections 2024-04-24T06:32:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.002+0000 7fc8d2800640 1 -- 172.21.15.59:0/1497282174 wait complete. 2024-04-24T06:32:06.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.297+0000 7f6506400640 1 Processor -- start 2024-04-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.297+0000 7f6506400640 1 -- start start 2024-04-24T06:32:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.298+0000 7f6506400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f650015c3e0 0x7f650015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.298+0000 7f6506400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65000d1ea0 con 0x7f65000b2930 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6506400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65000d0d90 con 0x7f65000d0810 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6506400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f65000d1070 con 0x7f650015c3e0 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6505000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f650015c3e0 0x7f650015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6505a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65000b2930 0x7f65000b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36402/0 (socket says 172.21.15.59:36402) 2024-04-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6505a00640 1 -- 172.21.15.59:0/2245061766 learned_addr learned my addr 172.21.15.59:0/2245061766 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6505000640 1 -- 172.21.15.59:0/2245061766 >> v1:172.21.15.66:6789/0 conn(0x7f65000d0810 legacy=0x7f65000b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.299+0000 7f6505000640 1 -- 172.21.15.59:0/2245061766 >> v1:172.21.15.59:6789/0 conn(0x7f65000b2930 legacy=0x7f65000b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.300+0000 7f6505000640 1 -- 172.21.15.59:0/2245061766 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650015f040 con 0x7f650015c3e0 2024-04-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.300+0000 7f64ff400640 1 -- 172.21.15.59:0/2245061766 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910667731 0 0) 0x7f65000d1ea0 con 0x7f65000b2930 2024-04-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.301+0000 7f6505000640 1 --2- 172.21.15.59:0/2245061766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650015c3e0 0x7f650015fb20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d90 tx=0x7f64f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3380336f7d8d7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.301+0000 7f64ff400640 1 -- 172.21.15.59:0/2245061766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007740 con 0x7f650015c3e0 2024-04-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.301+0000 7f64ff400640 1 -- 172.21.15.59:0/2245061766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f00078e0 con 0x7f650015c3e0 2024-04-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.301+0000 7f64ff400640 1 -- 172.21.15.59:0/2245061766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007cb0 con 0x7f650015c3e0 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 -- 172.21.15.59:0/2245061766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650015c3e0 msgr2=0x7f650015fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 --2- 172.21.15.59:0/2245061766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650015c3e0 0x7f650015fb20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d90 tx=0x7f64f002a6a0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 -- 172.21.15.59:0/2245061766 shutdown_connections 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 --2- 172.21.15.59:0/2245061766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650015c3e0 0x7f650015fb20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 -- 172.21.15.59:0/2245061766 >> 172.21.15.59:0/2245061766 conn(0x7f650009aa00 msgr2=0x7f65000d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 -- 172.21.15.59:0/2245061766 shutdown_connections 2024-04-24T06:32:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.302+0000 7f6506400640 1 -- 172.21.15.59:0/2245061766 wait complete. 2024-04-24T06:32:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.304+0000 7f6506400640 1 Processor -- start 2024-04-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.304+0000 7f6506400640 1 -- start start 2024-04-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65000b2930 0x7f65000ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 0x7f65000c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f650015c7c0 con 0x7f65000d0810 2024-04-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f650015f600 con 0x7f650015c3e0 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6506400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f650015c940 con 0x7f65000b2930 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55910/0 (socket says 172.21.15.59:55910) 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f64ffe00640 1 -- 172.21.15.59:0/3724857103 learned_addr learned my addr 172.21.15.59:0/3724857103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6505a00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 0x7f65000c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.305+0000 7f6505000640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65000b2930 0x7f65000ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 -- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65000b2930 msgr2=0x7f65000ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65000b2930 0x7f65000ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 -- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 msgr2=0x7f65000c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 0x7f65000c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 -- 172.21.15.59:0/3724857103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650015b170 con 0x7f65000d0810 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f6505a00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 0x7f65000c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c430 tx=0x7f64ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011790 con 0x7f65000d0810 2024-04-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec017070 con 0x7f65000d0810 2024-04-24T06:32:06.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011b70 con 0x7f65000d0810 2024-04-24T06:32:06.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65000b52e0 con 0x7f65000d0810 2024-04-24T06:32:06.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.306+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65000b58a0 con 0x7f65000d0810 2024-04-24T06:32:06.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.308+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6500162ec0 con 0x7f65000d0810 2024-04-24T06:32:06.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.309+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64ec011d10 con 0x7f65000d0810 2024-04-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.310+0000 7f64fcc00640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 0x7f64e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.310+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f64ec08b840 con 0x7f65000d0810 2024-04-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.310+0000 7f6505000640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 0x7f64e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.311+0000 7f6505000640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 0x7f64e006ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64f0034000 tx=0x7f64f002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:06.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.315+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6500162ec0 con 0x7f65000d0810 2024-04-24T06:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:06.518+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f650004fd90 con 0x7f65000d0810 2024-04-24T06:32:07.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.005+0000 7f64fcc00640 1 -- 172.21.15.59:0/3724857103 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f64ec059db0 con 0x7f65000d0810 2024-04-24T06:32:07.007 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.008+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 msgr2=0x7f64e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.008+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 0x7f64e006ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64f0034000 tx=0x7f64f002f000 comp rx=0 tx=0).stop 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.008+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 msgr2=0x7f650015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.008+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c430 tx=0x7f64ec00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.008+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 shutdown_connections 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f64e006c920 0x7f64e006ede0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f650015c3e0 0x7f65000c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65000d0810 0x7f650015ac30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 --2- 172.21.15.59:0/3724857103 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65000b2930 0x7f65000ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 >> 172.21.15.59:0/3724857103 conn(0x7f650009aa00 msgr2=0x7f6500161b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:07.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 shutdown_connections 2024-04-24T06:32:07.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.009+0000 7f6506400640 1 -- 172.21.15.59:0/3724857103 wait complete. 2024-04-24T06:32:07.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:32:07.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.227+0000 7ff08de00640 1 Processor -- start 2024-04-24T06:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.228+0000 7ff08de00640 1 -- start start 2024-04-24T06:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff0880d15b0 0x7ff0880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880b4f20 0x7ff0880cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08de00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0880b5da0 con 0x7ff08815ed20 2024-04-24T06:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08de00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff08815f100 con 0x7ff0880d15b0 2024-04-24T06:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880d0a20 con 0x7ff0880b4f20 2024-04-24T06:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff08815ed20 0x7ff0880d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36418/0 (socket says 172.21.15.59:36418) 2024-04-24T06:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.229+0000 7ff08ca00640 1 -- 172.21.15.59:0/1016904302 learned_addr learned my addr 172.21.15.59:0/1016904302 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087e00640 1 --2- 172.21.15.59:0/1016904302 >> v2:172.21.15.59:3300/0 conn(0x7ff0880d15b0 0x7ff0880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff08d400640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880b4f20 0x7ff0880cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087400640 1 -- 172.21.15.59:0/1016904302 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513860660 0 0) 0x7ff0880b5da0 con 0x7ff08815ed20 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087400640 1 -- 172.21.15.59:0/1016904302 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff06c003520 con 0x7ff08815ed20 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087e00640 1 -- 172.21.15.59:0/1016904302 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880b4f20 msgr2=0x7ff0880cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087e00640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880b4f20 0x7ff0880cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087e00640 1 -- 172.21.15.59:0/1016904302 >> v1:172.21.15.59:6789/0 conn(0x7ff08815ed20 legacy=0x7ff0880d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.230+0000 7ff087e00640 1 -- 172.21.15.59:0/1016904302 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0880d2ed0 con 0x7ff0880d15b0 2024-04-24T06:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.231+0000 7ff087e00640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880d15b0 0x7ff0880cf280 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097a0 tx=0x7ff07002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c95244e54b2723a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff087400640 1 -- 172.21.15.59:0/1016904302 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070037070 con 0x7ff0880d15b0 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff087400640 1 -- 172.21.15.59:0/1016904302 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07002adb0 con 0x7ff0880d15b0 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff087400640 1 -- 172.21.15.59:0/1016904302 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070032640 con 0x7ff0880d15b0 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 -- 172.21.15.59:0/1016904302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880d15b0 msgr2=0x7ff0880cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880d15b0 0x7ff0880cf280 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff0700097a0 tx=0x7ff07002a060 comp rx=0 tx=0).stop 2024-04-24T06:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 -- 172.21.15.59:0/1016904302 shutdown_connections 2024-04-24T06:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880b4f20 0x7ff0880cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 --2- 172.21.15.59:0/1016904302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880d15b0 0x7ff0880cf280 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 -- 172.21.15.59:0/1016904302 >> 172.21.15.59:0/1016904302 conn(0x7ff08808bf90 msgr2=0x7ff0880d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 -- 172.21.15.59:0/1016904302 shutdown_connections 2024-04-24T06:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.232+0000 7ff08de00640 1 -- 172.21.15.59:0/1016904302 wait complete. 2024-04-24T06:32:07.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.234+0000 7ff08de00640 1 Processor -- start 2024-04-24T06:32:07.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.234+0000 7ff08de00640 1 -- start start 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880d15b0 0x7ff0880ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff08815ed20 0x7ff0880c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880b3600 con 0x7ff0880b4f20 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880d3490 con 0x7ff08815ed20 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.235+0000 7ff08de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880b3780 con 0x7ff0880d15b0 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55938/0 (socket says 172.21.15.59:55938) 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 -- 172.21.15.59:0/521870168 learned_addr learned my addr 172.21.15.59:0/521870168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff087e00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880d15b0 0x7ff0880ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 -- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880d15b0 msgr2=0x7ff0880ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880d15b0 0x7ff0880ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08d400640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff08815ed20 0x7ff0880c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 -- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff08815ed20 msgr2=0x7ff0880c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff08815ed20 0x7ff0880c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 -- 172.21.15.59:0/521870168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0881f8ff0 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.236+0000 7ff08ca00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff07800cb40 tx=0x7ff07800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.237+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078017070 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.237+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07800aca0 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.237+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078012620 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.237+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08815b600 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.237+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08815bbc0 con 0x7ff0880b4f20 2024-04-24T06:32:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.238+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0880d1260 con 0x7ff0880b4f20 2024-04-24T06:32:07.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.240+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff07801a070 con 0x7ff0880b4f20 2024-04-24T06:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.241+0000 7ff084c00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 0x7ff06406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.241+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7ff07808b9c0 con 0x7ff0880b4f20 2024-04-24T06:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.242+0000 7ff087e00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 0x7ff06406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:07.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.242+0000 7ff087e00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 0x7ff06406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff070034000 tx=0x7ff07002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:07.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.246+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0880d1260 con 0x7ff0880b4f20 2024-04-24T06:32:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:07.492+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff08804fd90 con 0x7ff0880b4f20 2024-04-24T06:32:08.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.050+0000 7ff084c00640 1 -- 172.21.15.59:0/521870168 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7ff078059db0 con 0x7ff0880b4f20 2024-04-24T06:32:08.053 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 8 and data pool 9 2024-04-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.059+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 msgr2=0x7ff06406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.059+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 0x7ff06406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff070034000 tx=0x7ff07002e000 comp rx=0 tx=0).stop 2024-04-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.059+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 msgr2=0x7ff0881f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.059+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff07800cb40 tx=0x7ff07800a060 comp rx=0 tx=0).stop 2024-04-24T06:32:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 shutdown_connections 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff06406c920 0x7ff06406ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff08815ed20 0x7ff0880c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff0880d15b0 0x7ff0880ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 --2- 172.21.15.59:0/521870168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0880b4f20 0x7ff0881f8ab0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 >> 172.21.15.59:0/521870168 conn(0x7ff08808bf90 msgr2=0x7ff08815dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.061+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 shutdown_connections 2024-04-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.062+0000 7ff08de00640 1 -- 172.21.15.59:0/521870168 wait complete. 2024-04-24T06:32:08.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:32:08.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.275+0000 7f224fa00640 1 Processor -- start 2024-04-24T06:32:08.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.275+0000 7f224fa00640 1 -- start start 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.276+0000 7f224fa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f224806f7c0 0x7f22480b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224fa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224806ec60 con 0x7f224806df00 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224fa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22480b46e0 con 0x7f224815b840 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224fa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f22480b49c0 con 0x7f224806f7c0 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f224806f7c0 0x7f22480b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f224806f7c0 0x7f22480b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55942/0 (socket says 172.21.15.59:55942) 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 -- 172.21.15.59:0/1310339108 learned_addr learned my addr 172.21.15.59:0/1310339108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:08.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224da00640 1 --1- 172.21.15.59:0/1310339108 >> v1:172.21.15.59:6789/0 conn(0x7f224806df00 0x7f224806e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36426/0 (socket says 172.21.15.59:36426) 2024-04-24T06:32:08.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 -- 172.21.15.59:0/1310339108 >> v1:172.21.15.66:6789/0 conn(0x7f224815b840 legacy=0x7f224806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:08.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 -- 172.21.15.59:0/1310339108 >> v1:172.21.15.59:6789/0 conn(0x7f224806df00 legacy=0x7f224806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.277+0000 7f224d000640 1 -- 172.21.15.59:0/1310339108 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22480b4ca0 con 0x7f224806f7c0 2024-04-24T06:32:08.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f224d000640 1 --2- 172.21.15.59:0/1310339108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806f7c0 0x7f22480b54a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f223c009870 tx=0x7f223c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265d1263b750e5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f2247400640 1 -- 172.21.15.59:0/1310339108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c037070 con 0x7f224806f7c0 2024-04-24T06:32:08.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f2247400640 1 -- 172.21.15.59:0/1310339108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223c032440 con 0x7f224806f7c0 2024-04-24T06:32:08.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f2247400640 1 -- 172.21.15.59:0/1310339108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c032810 con 0x7f224806f7c0 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f224fa00640 1 -- 172.21.15.59:0/1310339108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806f7c0 msgr2=0x7f22480b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.279+0000 7f224fa00640 1 --2- 172.21.15.59:0/1310339108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806f7c0 0x7f22480b54a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f223c009870 tx=0x7f223c02a2a0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.280+0000 7f224fa00640 1 -- 172.21.15.59:0/1310339108 shutdown_connections 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.280+0000 7f224fa00640 1 --2- 172.21.15.59:0/1310339108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806f7c0 0x7f22480b54a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.280+0000 7f224fa00640 1 -- 172.21.15.59:0/1310339108 >> 172.21.15.59:0/1310339108 conn(0x7f2248067ea0 msgr2=0x7f224806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.280+0000 7f224fa00640 1 -- 172.21.15.59:0/1310339108 shutdown_connections 2024-04-24T06:32:08.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.280+0000 7f224fa00640 1 -- 172.21.15.59:0/1310339108 wait complete. 2024-04-24T06:32:08.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.281+0000 7f224fa00640 1 Processor -- start 2024-04-24T06:32:08.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.282+0000 7f224fa00640 1 -- start start 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 0x7f224815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f224815c170 0x7f22481420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22480ce340 con 0x7f224806df00 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22480ce1c0 con 0x7f224806f7c0 2024-04-24T06:32:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224fa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f22480ce4c0 con 0x7f224815c170 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55956/0 (socket says 172.21.15.59:55956) 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 -- 172.21.15.59:0/229922128 learned_addr learned my addr 172.21.15.59:0/229922128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f2247e00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 0x7f224815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 -- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f224815c170 msgr2=0x7f22481420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224da00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f224815c170 0x7f22481420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f224815c170 0x7f22481420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 -- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 msgr2=0x7f224815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 0x7f224815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f224d000640 1 -- 172.21.15.59:0/229922128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22481e6760 con 0x7f224806df00 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.283+0000 7f2247e00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 0x7f224815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f224d000640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f223c009870 tx=0x7f223c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c041020 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223c030970 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c037040 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22481e6a40 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.284+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481e7000 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.286+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224815c580 con 0x7f224806df00 2024-04-24T06:32:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.287+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f223c044020 con 0x7f224806df00 2024-04-24T06:32:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.288+0000 7f2244c00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.288+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f223c0ac550 con 0x7f224806df00 2024-04-24T06:32:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.288+0000 7f2247e00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.289+0000 7f2247e00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 0x7f222806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22481e7d40 tx=0x7f2234008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.293+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f224815c580 con 0x7f224806df00 2024-04-24T06:32:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.501+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2248068a80 con 0x7f224806df00 2024-04-24T06:32:08.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.504+0000 7f2244c00640 1 -- 172.21.15.59:0/229922128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16594 (secure 0 0 0) 0x7f223c04b030 con 0x7f224806df00 2024-04-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":57,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:07.494643+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 msgr2=0x7f222806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 0x7f222806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22481e7d40 tx=0x7f2234008040 comp rx=0 tx=0).stop 2024-04-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 msgr2=0x7f22481e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f223c009870 tx=0x7f223c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 shutdown_connections 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f222806c8f0 0x7f222806edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f224815c170 0x7f22481420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f224806f7c0 0x7f224815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 --2- 172.21.15.59:0/229922128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f224806df00 0x7f22481e6000 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.506+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 >> 172.21.15.59:0/229922128 conn(0x7f2248067ea0 msgr2=0x7f22480b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.507+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 shutdown_connections 2024-04-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.507+0000 7f224fa00640 1 -- 172.21.15.59:0/229922128 wait complete. 2024-04-24T06:32:08.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:32:08.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.715+0000 7f4484200640 1 Processor -- start 2024-04-24T06:32:08.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.715+0000 7f4484200640 1 -- start start 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4484200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f447c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4484200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0cd520 con 0x7f447c1454e0 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4484200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0cd800 con 0x7f447c0b3e40 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4484200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f447c0cddc0 con 0x7f447c0cd020 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4481800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f447c1454e0 0x7f447c145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36438/0 (socket says 172.21.15.59:36438) 2024-04-24T06:32:08.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.717+0000 7f4481800640 1 -- 172.21.15.59:0/2226643921 learned_addr learned my addr 172.21.15.59:0/2226643921 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787081875 0 0) 0x7f447c0cd520 con 0x7f447c1454e0 2024-04-24T06:32:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4464003520 con 0x7f447c1454e0 2024-04-24T06:32:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f4480e00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f447c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3732420661 0 0) 0x7f4464003520 con 0x7f447c1454e0 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 msgr2=0x7f447c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f447c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 >> v1:172.21.15.66:6789/0 conn(0x7f447c0b3e40 legacy=0x7f447c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0cdae0 con 0x7f447c1454e0 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f446c001670 con 0x7f447c1454e0 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 >> v1:172.21.15.59:6789/0 conn(0x7f447c1454e0 legacy=0x7f447c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f4464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f446400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f447c0cddc0 con 0x7f4464004800 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44640054f0 con 0x7f44640039e0 2024-04-24T06:32:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f446400bf00 con 0x7f447c0cd020 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.718+0000 7f4480e00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f4464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4481800640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4482200640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f446400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4480e00640 1 -- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 msgr2=0x7f446400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4480e00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f446400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4480e00640 1 -- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44640039e0 msgr2=0x7f4464003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4480e00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4480e00640 1 -- 172.21.15.59:0/2226643921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4464003520 con 0x7f4464004800 2024-04-24T06:32:08.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.719+0000 7f4482200640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0cd020 0x7f446400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.720+0000 7f4480e00640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f4464004c80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f447000ba30 tx=0x7f4464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1d4483ba87d63f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.720+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470030070 con 0x7f4464004800 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.720+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447002ade0 con 0x7f4464004800 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.720+0000 7f447be00640 1 -- 172.21.15.59:0/2226643921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447000b600 con 0x7f4464004800 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 -- 172.21.15.59:0/2226643921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 msgr2=0x7f4464004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 --2- 172.21.15.59:0/2226643921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f4464004c80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f447000ba30 tx=0x7f4464005830 comp rx=0 tx=0).stop 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4482200640 1 -- 172.21.15.59:0/2226643921 reap_dead start 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 -- 172.21.15.59:0/2226643921 shutdown_connections 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 -- 172.21.15.59:0/2226643921 >> 172.21.15.59:0/2226643921 conn(0x7f447c067ea0 msgr2=0x7f447c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 -- 172.21.15.59:0/2226643921 shutdown_connections 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.721+0000 7f4484200640 1 -- 172.21.15.59:0/2226643921 wait complete. 2024-04-24T06:32:08.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.723+0000 7f4484200640 1 Processor -- start 2024-04-24T06:32:08.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.723+0000 7f4484200640 1 -- start start 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0b27b0 0x7f447c0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f447c0b2c30 0x7f447c0ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f447c161020 con 0x7f4464004800 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1462c0 con 0x7f447c0b2c30 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4484200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f447c1611a0 con 0x7f447c0b27b0 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4481800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4481800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55974/0 (socket says 172.21.15.59:55974) 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.724+0000 7f4481800640 1 -- 172.21.15.59:0/2258250551 learned_addr learned my addr 172.21.15.59:0/2258250551 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4480e00640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0b27b0 0x7f447c0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 -- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0b27b0 msgr2=0x7f447c0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4482200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f447c0b2c30 0x7f447c0ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0b27b0 0x7f447c0ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 -- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f447c0b2c30 msgr2=0x7f447c0ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f447c0b2c30 0x7f447c0ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 -- 172.21.15.59:0/2258250551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0af4d0 con 0x7f4464004800 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4481800640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f446c001050 tx=0x7f446c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c00c870 con 0x7f4464004800 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c018070 con 0x7f4464004800 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c00cc70 con 0x7f4464004800 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c0af7b0 con 0x7f4464004800 2024-04-24T06:32:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.725+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c0ae330 con 0x7f4464004800 2024-04-24T06:32:08.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.726+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c068a80 con 0x7f4464004800 2024-04-24T06:32:08.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.728+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f446c01d460 con 0x7f4464004800 2024-04-24T06:32:08.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.729+0000 7f4479600640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 0x7f446006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:08.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.729+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f446c08b9c0 con 0x7f4464004800 2024-04-24T06:32:08.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.729+0000 7f4480e00640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 0x7f446006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:08.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.730+0000 7f4480e00640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 0x7f446006ee30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f447000ba00 tx=0x7f4470030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:08.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.733+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f446c059ed0 con 0x7f4464004800 2024-04-24T06:32:08.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.937+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f447c160030 con 0x7f4464004800 2024-04-24T06:32:08.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.941+0000 7f4479600640 1 -- 172.21.15.59:0/2258250551 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16594 (secure 0 0 0) 0x7f446c0597d0 con 0x7f4464004800 2024-04-24T06:32:08.943 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:08.943 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":57,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:07.494643+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 msgr2=0x7f446006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 0x7f446006ee30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f447000ba00 tx=0x7f4470030000 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 msgr2=0x7f447c140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f446c001050 tx=0x7f446c00ec80 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 shutdown_connections 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f446006c970 0x7f446006ee30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f447c0b2c30 0x7f447c0ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f447c0b27b0 0x7f447c0ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 --2- 172.21.15.59:0/2258250551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4464004800 0x7f447c140b00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 >> 172.21.15.59:0/2258250551 conn(0x7f447c067ea0 msgr2=0x7f447c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:08.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 shutdown_connections 2024-04-24T06:32:08.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:08.944+0000 7f4484200640 1 -- 172.21.15.59:0/2258250551 wait complete. 2024-04-24T06:32:08.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:32:09.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.178+0000 7fb696200640 1 Processor -- start 2024-04-24T06:32:09.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.178+0000 7fb696200640 1 -- start start 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb696200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6900d0d10 0x7fb690156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb696200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb69015d810 0x7fb690160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb696200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69015cab0 con 0x7fb6901567e0 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb696200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb690160e40 con 0x7fb6900d0d10 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb696200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb69015cd30 con 0x7fb69015d810 2024-04-24T06:32:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb68fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6900d0d10 0x7fb690156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb68fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6900d0d10 0x7fb690156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60020/0 (socket says 172.21.15.59:60020) 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.180+0000 7fb68fe00640 1 -- 172.21.15.59:0/3516153296 learned_addr learned my addr 172.21.15.59:0/3516153296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.181+0000 7fb68fe00640 1 -- 172.21.15.59:0/3516153296 >> v2:172.21.15.66:3300/0 conn(0x7fb69015d810 msgr2=0x7fb690160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.181+0000 7fb695800640 1 --2- 172.21.15.59:0/3516153296 >> v2:172.21.15.66:3300/0 conn(0x7fb69015d810 0x7fb690160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.181+0000 7fb68fe00640 1 --2- 172.21.15.59:0/3516153296 >> v2:172.21.15.66:3300/0 conn(0x7fb69015d810 0x7fb690160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:09.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.181+0000 7fb68fe00640 1 -- 172.21.15.59:0/3516153296 >> v1:172.21.15.66:6789/0 conn(0x7fb6901567e0 legacy=0x7fb6900d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.181+0000 7fb68fe00640 1 -- 172.21.15.59:0/3516153296 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69015d010 con 0x7fb6900d0d10 2024-04-24T06:32:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.182+0000 7fb68fe00640 1 --2- 172.21.15.59:0/3516153296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6900d0d10 0x7fb690156bc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b4cb0 tx=0x7fb67c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f02da20aca704d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.182+0000 7fb68f400640 1 -- 172.21.15.59:0/3516153296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c037070 con 0x7fb6900d0d10 2024-04-24T06:32:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.182+0000 7fb68f400640 1 -- 172.21.15.59:0/3516153296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c032440 con 0x7fb6900d0d10 2024-04-24T06:32:09.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.182+0000 7fb68f400640 1 -- 172.21.15.59:0/3516153296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c032810 con 0x7fb6900d0d10 2024-04-24T06:32:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 -- 172.21.15.59:0/3516153296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6900d0d10 msgr2=0x7fb690156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 --2- 172.21.15.59:0/3516153296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6900d0d10 0x7fb690156bc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b4cb0 tx=0x7fb67c02a230 comp rx=0 tx=0).stop 2024-04-24T06:32:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 -- 172.21.15.59:0/3516153296 shutdown_connections 2024-04-24T06:32:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 --2- 172.21.15.59:0/3516153296 >> v2:172.21.15.66:3300/0 conn(0x7fb69015d810 0x7fb690160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 --2- 172.21.15.59:0/3516153296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6900d0d10 0x7fb690156bc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 -- 172.21.15.59:0/3516153296 >> 172.21.15.59:0/3516153296 conn(0x7fb69006bdf0 msgr2=0x7fb69006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 -- 172.21.15.59:0/3516153296 shutdown_connections 2024-04-24T06:32:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.183+0000 7fb696200640 1 -- 172.21.15.59:0/3516153296 wait complete. 2024-04-24T06:32:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.185+0000 7fb696200640 1 Processor -- start 2024-04-24T06:32:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.185+0000 7fb696200640 1 -- start start 2024-04-24T06:32:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6900d0d10 0x7fb690151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6901567e0 0x7fb6900986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901607d0 con 0x7fb69015d810 2024-04-24T06:32:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb690160650 con 0x7fb6900d0d10 2024-04-24T06:32:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb696200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb690160950 con 0x7fb6901567e0 2024-04-24T06:32:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb695800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.186+0000 7fb695800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60028/0 (socket says 172.21.15.59:60028) 2024-04-24T06:32:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 -- 172.21.15.59:0/142518747 learned_addr learned my addr 172.21.15.59:0/142518747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb694e00640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6900d0d10 0x7fb690151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb68fe00640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6901567e0 0x7fb6900986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 -- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6901567e0 msgr2=0x7fb6900986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6901567e0 0x7fb6900986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 -- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6900d0d10 msgr2=0x7fb690151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6900d0d10 0x7fb690151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 -- 172.21.15.59:0/142518747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6901f41a0 con 0x7fb69015d810 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.187+0000 7fb695800640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb68800cd40 tx=0x7fb68800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.188+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688003b90 con 0x7fb69015d810 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.188+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688003d30 con 0x7fb69015d810 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.188+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688004240 con 0x7fb69015d810 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.188+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6901f4480 con 0x7fb69015d810 2024-04-24T06:32:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.188+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6901f4a40 con 0x7fb69015d810 2024-04-24T06:32:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.189+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6901f4c40 con 0x7fb69015d810 2024-04-24T06:32:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.190+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6880043e0 con 0x7fb69015d810 2024-04-24T06:32:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.191+0000 7fb68cc00640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.191+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb68808c650 con 0x7fb69015d810 2024-04-24T06:32:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.191+0000 7fb694e00640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:09.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.192+0000 7fb694e00640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6901601b0 tx=0x7fb684009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:09.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.195+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6901f4c40 con 0x7fb69015d810 2024-04-24T06:32:09.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:09.443+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb69004fd90 con 0x7fb69015d810 2024-04-24T06:32:10.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.072+0000 7fb68cc00640 1 -- 172.21.15.59:0/142518747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7fb68805aa80 con 0x7fb69015d810 2024-04-24T06:32:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.076+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 msgr2=0x7fb67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.076+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6901601b0 tx=0x7fb684009040 comp rx=0 tx=0).stop 2024-04-24T06:32:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.076+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 msgr2=0x7fb690098bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.076+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb68800cd40 tx=0x7fb68800a660 comp rx=0 tx=0).stop 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 shutdown_connections 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb69015d810 0x7fb690098bf0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6901567e0 0x7fb6900986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 --2- 172.21.15.59:0/142518747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6900d0d10 0x7fb690151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.077+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 >> 172.21.15.59:0/142518747 conn(0x7fb69006bdf0 msgr2=0x7fb690073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.078+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 shutdown_connections 2024-04-24T06:32:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.078+0000 7fb696200640 1 -- 172.21.15.59:0/142518747 wait complete. 2024-04-24T06:32:10.106 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:32:10.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:32:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.314+0000 7f2327200640 1 Processor -- start 2024-04-24T06:32:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.314+0000 7f2327200640 1 -- start start 2024-04-24T06:32:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.315+0000 7f2327200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f232006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.315+0000 7f2327200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23200ce830 con 0x7f232006d910 2024-04-24T06:32:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.315+0000 7f2327200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232009bc60 con 0x7f232015e9e0 2024-04-24T06:32:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.315+0000 7f2327200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23200cd390 con 0x7f232006ccf0 2024-04-24T06:32:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.316+0000 7f2326800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f232006d910 0x7f23200b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51752/0 (socket says 172.21.15.59:51752) 2024-04-24T06:32:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.316+0000 7f2326800640 1 -- 172.21.15.59:0/2393673131 learned_addr learned my addr 172.21.15.59:0/2393673131 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.316+0000 7f2325400640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f232006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.316+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383748564 0 0) 0x7f23200ce830 con 0x7f232006d910 2024-04-24T06:32:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.316+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2308003520 con 0x7f232006d910 2024-04-24T06:32:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2997097998 0 0) 0x7f2308003520 con 0x7f232006d910 2024-04-24T06:32:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 msgr2=0x7f232006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f232006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 >> v1:172.21.15.66:6789/0 conn(0x7f232015e9e0 legacy=0x7f232009b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200cca60 con 0x7f232006d910 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f231c0019f0 con 0x7f232006d910 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 >> v1:172.21.15.59:6789/0 conn(0x7f232006d910 legacy=0x7f23200b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f2308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f23080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23200cd390 con 0x7f2308004800 2024-04-24T06:32:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f230800bb30 con 0x7f23080039e0 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f230800bd30 con 0x7f232006ccf0 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.317+0000 7f2325400640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f2308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2326800640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f23080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325400640 1 -- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 msgr2=0x7f23080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325e00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325400640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f232006ccf0 0x7f23080051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325400640 1 -- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23080039e0 msgr2=0x7f2308003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325400640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325400640 1 -- 172.21.15.59:0/2393673131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2308003520 con 0x7f2308004800 2024-04-24T06:32:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.318+0000 7f2325e00640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23080039e0 0x7f2308003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.319+0000 7f2325400640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f2308004c80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2310009a90 tx=0x7f2308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff565508fc987ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.319+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310036070 con 0x7f2308004800 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.319+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310031400 con 0x7f2308004800 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.319+0000 7f2324a00640 1 -- 172.21.15.59:0/2393673131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310009690 con 0x7f2308004800 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 -- 172.21.15.59:0/2393673131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 msgr2=0x7f2308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 --2- 172.21.15.59:0/2393673131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f2308004c80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2310009a90 tx=0x7f2308005830 comp rx=0 tx=0).stop 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2326800640 1 -- 172.21.15.59:0/2393673131 reap_dead start 2024-04-24T06:32:10.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 -- 172.21.15.59:0/2393673131 shutdown_connections 2024-04-24T06:32:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 -- 172.21.15.59:0/2393673131 >> 172.21.15.59:0/2393673131 conn(0x7f23200b3f00 msgr2=0x7f23200b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 -- 172.21.15.59:0/2393673131 shutdown_connections 2024-04-24T06:32:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.320+0000 7f2327200640 1 -- 172.21.15.59:0/2393673131 wait complete. 2024-04-24T06:32:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.322+0000 7f2327200640 1 Processor -- start 2024-04-24T06:32:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.322+0000 7f2327200640 1 -- start start 2024-04-24T06:32:10.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 0x7f23200c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 0x7f2320151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f232015edc0 con 0x7f2308004800 2024-04-24T06:32:10.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23200cd020 con 0x7f23200c7d60 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2327200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23200b5260 con 0x7f2320161c90 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2325e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2325e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60052/0 (socket says 172.21.15.59:60052) 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.323+0000 7f2325e00640 1 -- 172.21.15.59:0/1815089784 learned_addr learned my addr 172.21.15.59:0/1815089784 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325400640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 0x7f23200c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2326800640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 0x7f2320151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 -- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 msgr2=0x7f23200c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 0x7f23200c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 -- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 msgr2=0x7f2320151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 0x7f2320151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 -- 172.21.15.59:0/1815089784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232015ef40 con 0x7f2308004800 2024-04-24T06:32:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325400640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 0x7f23200c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2326800640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 0x7f2320151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2325e00640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f231400d7a0 tx=0x7f231400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314005070 con 0x7f2308004800 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2314005210 con 0x7f2308004800 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314007030 con 0x7f2308004800 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23200c7210 con 0x7f2308004800 2024-04-24T06:32:10.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.324+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232015f7a0 con 0x7f2308004800 2024-04-24T06:32:10.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.326+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232004fd90 con 0x7f2308004800 2024-04-24T06:32:10.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.327+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f231401a030 con 0x7f2308004800 2024-04-24T06:32:10.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.328+0000 7f231a000640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 0x7f230406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:10.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.328+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f231408b320 con 0x7f2308004800 2024-04-24T06:32:10.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.328+0000 7f2325400640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 0x7f230406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:10.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.329+0000 7f2325400640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 0x7f230406ede0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:10.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.333+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2314021020 con 0x7f2308004800 2024-04-24T06:32:10.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:10.581+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f23200d10a0 con 0x7f2308004800 2024-04-24T06:32:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.075+0000 7f231a000640 1 -- 172.21.15.59:0/1815089784 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f2314059740 con 0x7f2308004800 2024-04-24T06:32:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 msgr2=0x7f230406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 0x7f230406ede0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310008030 comp rx=0 tx=0).stop 2024-04-24T06:32:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 msgr2=0x7f23200c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f231400d7a0 tx=0x7f231400dc60 comp rx=0 tx=0).stop 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 shutdown_connections 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f230406c920 0x7f230406ede0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23200c7d60 0x7f2320151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2320161c90 0x7f23200c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 --2- 172.21.15.59:0/1815089784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2308004800 0x7f23200c9ff0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 >> 172.21.15.59:0/1815089784 conn(0x7f23200b3f00 msgr2=0x7f2320071020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 shutdown_connections 2024-04-24T06:32:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.079+0000 7f2327200640 1 -- 172.21.15.59:0/1815089784 wait complete. 2024-04-24T06:32:11.102 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:32:11.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:32:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.301+0000 7f4acce00640 1 Processor -- start 2024-04-24T06:32:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.301+0000 7f4acce00640 1 -- start start 2024-04-24T06:32:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.302+0000 7f4acce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 0x7f4ac80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.302+0000 7f4acce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ac8163770 0x7f4ac80d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.302+0000 7f4acce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac80d01c0 con 0x7f4ac8160180 2024-04-24T06:32:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.302+0000 7f4acce00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac815cf00 con 0x7f4ac8163770 2024-04-24T06:32:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.302+0000 7f4acce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac815f6a0 con 0x7f4ac815cb20 2024-04-24T06:32:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ac8163770 0x7f4ac80d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ac8163770 0x7f4ac80d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60068/0 (socket says 172.21.15.59:60068) 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 -- 172.21.15.59:0/2374042468 learned_addr learned my addr 172.21.15.59:0/2374042468 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 -- 172.21.15.59:0/2374042468 >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 msgr2=0x7f4ac80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5e00640 1 --2- 172.21.15.59:0/2374042468 >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 0x7f4ac80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 --2- 172.21.15.59:0/2374042468 >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 0x7f4ac80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 -- 172.21.15.59:0/2374042468 >> v1:172.21.15.59:6789/0 conn(0x7f4ac8160180 legacy=0x7f4ac80b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5400640 1 -- 172.21.15.59:0/2374042468 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac815f980 con 0x7f4ac8163770 2024-04-24T06:32:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.303+0000 7f4ac5e00640 1 --2- 172.21.15.59:0/2374042468 >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 0x7f4ac80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.304+0000 7f4ac5400640 1 --2- 172.21.15.59:0/2374042468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac8163770 0x7f4ac80d1240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4ac808cb40 tx=0x7f4ab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18dc3dde635318f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4ac4a00640 1 -- 172.21.15.59:0/2374042468 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037070 con 0x7f4ac8163770 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4ac4a00640 1 -- 172.21.15.59:0/2374042468 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4032440 con 0x7f4ac8163770 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4ac4a00640 1 -- 172.21.15.59:0/2374042468 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4032810 con 0x7f4ac8163770 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4acce00640 1 -- 172.21.15.59:0/2374042468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac8163770 msgr2=0x7f4ac80d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4acce00640 1 --2- 172.21.15.59:0/2374042468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac8163770 0x7f4ac80d1240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4ac808cb40 tx=0x7f4ab402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4acce00640 1 -- 172.21.15.59:0/2374042468 shutdown_connections 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4acce00640 1 --2- 172.21.15.59:0/2374042468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac8163770 0x7f4ac80d1240 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.305+0000 7f4acce00640 1 --2- 172.21.15.59:0/2374042468 >> v2:172.21.15.66:3300/0 conn(0x7f4ac815cb20 0x7f4ac80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.306+0000 7f4acce00640 1 -- 172.21.15.59:0/2374042468 >> 172.21.15.59:0/2374042468 conn(0x7f4ac806cc70 msgr2=0x7f4ac806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.306+0000 7f4acce00640 1 -- 172.21.15.59:0/2374042468 shutdown_connections 2024-04-24T06:32:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.306+0000 7f4acce00640 1 -- 172.21.15.59:0/2374042468 wait complete. 2024-04-24T06:32:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.307+0000 7f4acce00640 1 Processor -- start 2024-04-24T06:32:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 -- start start 2024-04-24T06:32:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ac8160180 0x7f4ac80cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 0x7f4ac80c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac80d1900 con 0x7f4ac815cb20 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac80d1780 con 0x7f4ac8163770 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.308+0000 7f4acce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac80d1a80 con 0x7f4ac8160180 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60078/0 (socket says 172.21.15.59:60078) 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 -- 172.21.15.59:0/1296547614 learned_addr learned my addr 172.21.15.59:0/1296547614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac6800640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 0x7f4ac80c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 -- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ac8160180 msgr2=0x7f4ac80cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5400640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ac8160180 0x7f4ac80cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ac8160180 0x7f4ac80cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 -- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 msgr2=0x7f4ac80c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 0x7f4ac80c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac5e00640 1 -- 172.21.15.59:0/1296547614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac80c94c0 con 0x7f4ac815cb20 2024-04-24T06:32:11.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.309+0000 7f4ac6800640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 0x7f4ac80c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4ac5e00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c820 tx=0x7f4ab800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8015070 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab8007a30 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8007de0 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac8155b00 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.310+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81560c0 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.311+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac8068a80 con 0x7f4ac815cb20 2024-04-24T06:32:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.312+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ab8018070 con 0x7f4ac815cb20 2024-04-24T06:32:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.313+0000 7f4abe000640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 0x7f4aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.313+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4ab808bae0 con 0x7f4ac815cb20 2024-04-24T06:32:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.314+0000 7f4ac5400640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 0x7f4aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.315+0000 7f4ac5400640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 0x7f4aa806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4034000 tx=0x7f4ab402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.318+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ab8059e70 con 0x7f4ac815cb20 2024-04-24T06:32:11.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.561+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ac80d3b40 con 0x7f4ac815cb20 2024-04-24T06:32:11.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.562+0000 7f4abe000640 1 -- 172.21.15.59:0/1296547614 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5178 (secure 0 0 0) 0x7f4ab8059770 con 0x7f4ac815cb20 2024-04-24T06:32:11.564 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:11.565 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:10.584021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:32:11.565 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 msgr2=0x7f4aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 0x7f4aa806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4034000 tx=0x7f4ab402e000 comp rx=0 tx=0).stop 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 msgr2=0x7f4ac8150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c820 tx=0x7f4ab800cce0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 shutdown_connections 2024-04-24T06:32:11.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4aa806c8f0 0x7f4aa806edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac8163770 0x7f4ac80c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ac8160180 0x7f4ac80cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 --2- 172.21.15.59:0/1296547614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ac815cb20 0x7f4ac8150d20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 >> 172.21.15.59:0/1296547614 conn(0x7f4ac806cc70 msgr2=0x7f4ac8160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 shutdown_connections 2024-04-24T06:32:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.565+0000 7f4acce00640 1 -- 172.21.15.59:0/1296547614 wait complete. 2024-04-24T06:32:11.586 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:32:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.785+0000 7f8b37e00640 1 Processor -- start 2024-04-24T06:32:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.786+0000 7f8b37e00640 1 -- start start 2024-04-24T06:32:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.787+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.787+0000 7f8b37e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 0x7f8b380cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.787+0000 7f8b37e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b380b3870 con 0x7f8b380b3080 2024-04-24T06:32:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.787+0000 7f8b37e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380b41a0 con 0x7f8b3806f3f0 2024-04-24T06:32:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.787+0000 7f8b37e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380b3b50 con 0x7f8b3815b450 2024-04-24T06:32:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.788+0000 7f8b36a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.788+0000 7f8b36a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:34626/0 (socket says 172.21.15.59:34626) 2024-04-24T06:32:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.788+0000 7f8b36a00640 1 -- 172.21.15.59:0/3714609037 learned_addr learned my addr 172.21.15.59:0/3714609037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b2fe00640 1 --2- 172.21.15.59:0/3714609037 >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 0x7f8b380cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b36a00640 1 -- 172.21.15.59:0/3714609037 >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 msgr2=0x7f8b380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b36a00640 1 --2- 172.21.15.59:0/3714609037 >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 0x7f8b380cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b36a00640 1 -- 172.21.15.59:0/3714609037 >> v1:172.21.15.66:6789/0 conn(0x7f8b380b3080 legacy=0x7f8b380ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b36a00640 1 -- 172.21.15.59:0/3714609037 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380b3e30 con 0x7f8b3815b450 2024-04-24T06:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.789+0000 7f8b2fe00640 1 --2- 172.21.15.59:0/3714609037 >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 0x7f8b380cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:11.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.791+0000 7f8b36a00640 1 --2- 172.21.15.59:0/3714609037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009870 tx=0x7f8b2802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d6f4b24d5b853c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.791+0000 7f8b36000640 1 -- 172.21.15.59:0/3714609037 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28037070 con 0x7f8b3815b450 2024-04-24T06:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.791+0000 7f8b36000640 1 -- 172.21.15.59:0/3714609037 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28032440 con 0x7f8b3815b450 2024-04-24T06:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.791+0000 7f8b36000640 1 -- 172.21.15.59:0/3714609037 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28032810 con 0x7f8b3815b450 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 -- 172.21.15.59:0/3714609037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 msgr2=0x7f8b380b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 --2- 172.21.15.59:0/3714609037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009870 tx=0x7f8b2802a250 comp rx=0 tx=0).stop 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 -- 172.21.15.59:0/3714609037 shutdown_connections 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 --2- 172.21.15.59:0/3714609037 >> v2:172.21.15.66:3300/0 conn(0x7f8b3806f3f0 0x7f8b380cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 --2- 172.21.15.59:0/3714609037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3815b450 0x7f8b380b4630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 -- 172.21.15.59:0/3714609037 >> 172.21.15.59:0/3714609037 conn(0x7f8b380b21a0 msgr2=0x7f8b380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:11.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 -- 172.21.15.59:0/3714609037 shutdown_connections 2024-04-24T06:32:11.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.792+0000 7f8b37e00640 1 -- 172.21.15.59:0/3714609037 wait complete. 2024-04-24T06:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.795+0000 7f8b37e00640 1 Processor -- start 2024-04-24T06:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.795+0000 7f8b37e00640 1 -- start start 2024-04-24T06:32:11.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3806f3f0 0x7f8b3815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 0x7f8b381e8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380b4ff0 con 0x7f8b380b3080 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3815b830 con 0x7f8b3815b450 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b37e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380cd590 con 0x7f8b3806f3f0 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.796+0000 7f8b2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60094/0 (socket says 172.21.15.59:60094) 2024-04-24T06:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 -- 172.21.15.59:0/1974200102 learned_addr learned my addr 172.21.15.59:0/1974200102 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b36a00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3806f3f0 0x7f8b3815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 -- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3806f3f0 msgr2=0x7f8b3815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3806f3f0 0x7f8b3815c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b37400640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 0x7f8b381e8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 -- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 msgr2=0x7f8b381e8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 0x7f8b381e8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 -- 172.21.15.59:0/1974200102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380b1100 con 0x7f8b380b3080 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b37400640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 0x7f8b381e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.797+0000 7f8b2fe00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000c820 tx=0x7f8b2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.798+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20015070 con 0x7f8b380b3080 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.798+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b20007a30 con 0x7f8b380b3080 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.798+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b380b19f0 con 0x7f8b380b3080 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.798+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20007e00 con 0x7f8b380b3080 2024-04-24T06:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.798+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b380b13a0 con 0x7f8b380b3080 2024-04-24T06:32:11.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.799+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3804fd90 con 0x7f8b380b3080 2024-04-24T06:32:11.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.800+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b20018070 con 0x7f8b380b3080 2024-04-24T06:32:11.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.801+0000 7f8b2f400640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 0x7f8b1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.802+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2008bb30 con 0x7f8b380b3080 2024-04-24T06:32:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.802+0000 7f8b36a00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 0x7f8b1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.802+0000 7f8b36a00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 0x7f8b1406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8b28034000 tx=0x7f8b2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:11.806+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b20059f50 con 0x7f8b380b3080 2024-04-24T06:32:12.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.013+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b3806e020 con 0x7f8b380b3080 2024-04-24T06:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.016+0000 7f8b2f400640 1 -- 172.21.15.59:0/1974200102 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16581 (secure 0 0 0) 0x7f8b20059850 con 0x7f8b380b3080 2024-04-24T06:32:12.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:12.019 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:09.051581+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 msgr2=0x7f8b1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 0x7f8b1406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8b28034000 tx=0x7f8b2802e000 comp rx=0 tx=0).stop 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 msgr2=0x7f8b3809d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000c820 tx=0x7f8b2000cc20 comp rx=0 tx=0).stop 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 shutdown_connections 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b1406c8f0 0x7f8b1406edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b3815b450 0x7f8b381e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b380b3080 0x7f8b3809d860 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 --2- 172.21.15.59:0/1974200102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b3806f3f0 0x7f8b3815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 >> 172.21.15.59:0/1974200102 conn(0x7f8b380b21a0 msgr2=0x7f8b3806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:12.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 shutdown_connections 2024-04-24T06:32:12.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.019+0000 7f8b37e00640 1 -- 172.21.15.59:0/1974200102 wait complete. 2024-04-24T06:32:12.042 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:32:12.042 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:32:12.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.240+0000 7f861d000640 1 Processor -- start 2024-04-24T06:32:12.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.240+0000 7f861d000640 1 -- start start 2024-04-24T06:32:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f861d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8618160910 0x7f8618160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f861d000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86180cd520 con 0x7f86181454e0 2024-04-24T06:32:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f861d000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86180cd800 con 0x7f86180cd020 2024-04-24T06:32:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f861d000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f86180cddc0 con 0x7f8618160910 2024-04-24T06:32:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f8615e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f86181454e0 0x7f8618145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51758/0 (socket says 172.21.15.59:51758) 2024-04-24T06:32:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.242+0000 7f8615e00640 1 -- 172.21.15.59:0/928665414 learned_addr learned my addr 172.21.15.59:0/928665414 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432592795 0 0) 0x7f86180cd520 con 0x7f86181454e0 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85fc003520 con 0x7f86181454e0 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8616800640 1 --2- 172.21.15.59:0/928665414 >> v2:172.21.15.66:3300/0 conn(0x7f8618160910 0x7f8618160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 26425869 0 0) 0x7f85fc003520 con 0x7f86181454e0 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 >> v2:172.21.15.66:3300/0 conn(0x7f8618160910 msgr2=0x7f8618160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 --2- 172.21.15.59:0/928665414 >> v2:172.21.15.66:3300/0 conn(0x7f8618160910 0x7f8618160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 >> v1:172.21.15.66:6789/0 conn(0x7f86180cd020 legacy=0x7f86180b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86180cdae0 con 0x7f86181454e0 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f86080019f0 con 0x7f86181454e0 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 >> v1:172.21.15.59:6789/0 conn(0x7f86181454e0 legacy=0x7f8618145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85fc003a00 0x7f85fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 0x7f85fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc0052e0 0x7f85fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f86180cddc0 con 0x7f85fc004800 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc00c200 con 0x7f85fc003a00 2024-04-24T06:32:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.243+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc00c400 con 0x7f85fc0052e0 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 0x7f85fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f8615e00640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85fc003a00 0x7f85fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f8616800640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc0052e0 0x7f85fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 -- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc0052e0 msgr2=0x7f85fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc0052e0 0x7f85fc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 -- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85fc003a00 msgr2=0x7f85fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85fc003a00 0x7f85fc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f860d400640 1 -- 172.21.15.59:0/928665414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc003520 con 0x7f85fc004800 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f8615e00640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85fc003a00 0x7f85fc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.244+0000 7f8616800640 1 -- 172.21.15.59:0/928665414 reap_dead start 2024-04-24T06:32:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.245+0000 7f860d400640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 0x7f85fc004c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8600001750 tx=0x7f85fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9b839e0551b8de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.245+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600001370 con 0x7f85fc004800 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86000052d0 con 0x7f85fc004800 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f8615400640 1 -- 172.21.15.59:0/928665414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860002abc0 con 0x7f85fc004800 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 -- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 msgr2=0x7f85fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 0x7f85fc004c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8600001750 tx=0x7f85fc005830 comp rx=0 tx=0).stop 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 -- 172.21.15.59:0/928665414 shutdown_connections 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 --2- 172.21.15.59:0/928665414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85fc004800 0x7f85fc004c80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 -- 172.21.15.59:0/928665414 >> 172.21.15.59:0/928665414 conn(0x7f8618067ea0 msgr2=0x7f861806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 -- 172.21.15.59:0/928665414 shutdown_connections 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.246+0000 7f861d000640 1 -- 172.21.15.59:0/928665414 wait complete. 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.248+0000 7f861d000640 1 Processor -- start 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.248+0000 7f861d000640 1 -- start start 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 0x7f861815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 0x7f861813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8618160f00 con 0x7f861806f590 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8618160d80 con 0x7f86181fcf30 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f861d000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8618161080 con 0x7f85fc004800 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f860d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.249+0000 7f860d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60122/0 (socket says 172.21.15.59:60122) 2024-04-24T06:32:12.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 -- 172.21.15.59:0/1719706412 learned_addr learned my addr 172.21.15.59:0/1719706412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f8616800640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 0x7f861813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 -- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 msgr2=0x7f861815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f8615e00640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 0x7f861815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 0x7f861815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 -- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 msgr2=0x7f861813fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 0x7f861813fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 -- 172.21.15.59:0/1719706412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86181fd3b0 con 0x7f861806f590 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f8616800640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 0x7f861813fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f8615e00640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 0x7f861815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.250+0000 7f860d400640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f8600001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.251+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860003f030 con 0x7f861806f590 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.251+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8600005480 con 0x7f861806f590 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.251+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600037040 con 0x7f861806f590 2024-04-24T06:32:12.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.251+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86181fd690 con 0x7f861806f590 2024-04-24T06:32:12.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.251+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861815f090 con 0x7f861806f590 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.252+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8618068a80 con 0x7f861806f590 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.253+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8600042070 con 0x7f861806f590 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.254+0000 7f860ea00640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.254+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f86000ab4c0 con 0x7f861806f590 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.254+0000 7f8615e00640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.255+0000 7f8615e00640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 0x7f85f406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8608007c40 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.258+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86000433b0 con 0x7f861806f590 2024-04-24T06:32:12.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.460+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86180cce00 con 0x7f85f406c8f0 2024-04-24T06:32:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.481+0000 7f860ea00640 1 -- 172.21.15.59:0/1719706412 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7f86180cce00 con 0x7f85f406c8f0 2024-04-24T06:32:12.485 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:32:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 msgr2=0x7f85f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 0x7f85f406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8608007c40 tx=0x7f8608009040 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 msgr2=0x7f86181fc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f8600001120 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 shutdown_connections 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f85f406c8f0 0x7f85f406edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86181fcf30 0x7f861813fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f861806f590 0x7f86181fc9f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 --2- 172.21.15.59:0/1719706412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f85fc004800 0x7f861815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.488+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 >> 172.21.15.59:0/1719706412 conn(0x7f8618067ea0 msgr2=0x7f86180b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.489+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 shutdown_connections 2024-04-24T06:32:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.489+0000 7f861d000640 1 -- 172.21.15.59:0/1719706412 wait complete. 2024-04-24T06:32:12.510 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-24T06:32:11.254341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235120,"kb_used_data":6460,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739600,"statfs":{"total":773094113280,"available":772853350400,"internally_reserved":0,"allocated":6615040,"data_stored":3420889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252635"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067015+0000","last_change":"2024-04-24T06:32:08.064363+0000","last_active":"2024-04-24T06:32:08.067015+0000","last_peered":"2024-04-24T06:32:08.067015+0000","last_clean":"2024-04-24T06:32:08.067015+0000","last_became_active":"2024-04-24T06:32:08.064035+0000","last_became_peered":"2024-04-24T06:32:08.064035+0000","last_unstale":"2024-04-24T06:32:08.067015+0000","last_undegraded":"2024-04-24T06:32:08.067015+0000","last_fullsized":"2024-04-24T06:32:08.067015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:31:20.628402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075463+0000","last_change":"2024-04-24T06:32:08.071119+0000","last_active":"2024-04-24T06:32:08.075463+0000","last_peered":"2024-04-24T06:32:08.075463+0000","last_clean":"2024-04-24T06:32:08.075463+0000","last_became_active":"2024-04-24T06:32:08.069986+0000","last_became_peered":"2024-04-24T06:32:08.069986+0000","last_unstale":"2024-04-24T06:32:08.075463+0000","last_undegraded":"2024-04-24T06:32:08.075463+0000","last_fullsized":"2024-04-24T06:32:08.075463+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:55.360567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079541+0000","last_change":"2024-04-24T06:32:08.076131+0000","last_active":"2024-04-24T06:32:08.079541+0000","last_peered":"2024-04-24T06:32:08.079541+0000","last_clean":"2024-04-24T06:32:08.079541+0000","last_became_active":"2024-04-24T06:32:08.075814+0000","last_became_peered":"2024-04-24T06:32:08.075814+0000","last_unstale":"2024-04-24T06:32:08.079541+0000","last_undegraded":"2024-04-24T06:32:08.079541+0000","last_fullsized":"2024-04-24T06:32:08.079541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:45:21.497171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073287+0000","last_change":"2024-04-24T06:32:08.069461+0000","last_active":"2024-04-24T06:32:08.073287+0000","last_peered":"2024-04-24T06:32:08.073287+0000","last_clean":"2024-04-24T06:32:08.073287+0000","last_became_active":"2024-04-24T06:32:08.069145+0000","last_became_peered":"2024-04-24T06:32:08.069145+0000","last_unstale":"2024-04-24T06:32:08.073287+0000","last_undegraded":"2024-04-24T06:32:08.073287+0000","last_fullsized":"2024-04-24T06:32:08.073287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082572+0000","last_change":"2024-04-24T06:32:08.075454+0000","last_active":"2024-04-24T06:32:08.082572+0000","last_peered":"2024-04-24T06:32:08.082572+0000","last_clean":"2024-04-24T06:32:08.082572+0000","last_became_active":"2024-04-24T06:32:08.071601+0000","last_became_peered":"2024-04-24T06:32:08.071601+0000","last_unstale":"2024-04-24T06:32:08.082572+0000","last_undegraded":"2024-04-24T06:32:08.082572+0000","last_fullsized":"2024-04-24T06:32:08.082572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:21.131075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062777+0000","last_change":"2024-04-24T06:32:08.060886+0000","last_active":"2024-04-24T06:32:08.062777+0000","last_peered":"2024-04-24T06:32:08.062777+0000","last_clean":"2024-04-24T06:32:08.062777+0000","last_became_active":"2024-04-24T06:32:08.060585+0000","last_became_peered":"2024-04-24T06:32:08.060585+0000","last_unstale":"2024-04-24T06:32:08.062777+0000","last_undegraded":"2024-04-24T06:32:08.062777+0000","last_fullsized":"2024-04-24T06:32:08.062777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:33.445994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075046+0000","last_change":"2024-04-24T06:32:08.066735+0000","last_active":"2024-04-24T06:32:08.075046+0000","last_peered":"2024-04-24T06:32:08.075046+0000","last_clean":"2024-04-24T06:32:08.075046+0000","last_became_active":"2024-04-24T06:32:08.065965+0000","last_became_peered":"2024-04-24T06:32:08.065965+0000","last_unstale":"2024-04-24T06:32:08.075046+0000","last_undegraded":"2024-04-24T06:32:08.075046+0000","last_fullsized":"2024-04-24T06:32:08.075046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:49.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082742+0000","last_change":"2024-04-24T06:32:08.075677+0000","last_active":"2024-04-24T06:32:08.082742+0000","last_peered":"2024-04-24T06:32:08.082742+0000","last_clean":"2024-04-24T06:32:08.082742+0000","last_became_active":"2024-04-24T06:32:08.073073+0000","last_became_peered":"2024-04-24T06:32:08.073073+0000","last_unstale":"2024-04-24T06:32:08.082742+0000","last_undegraded":"2024-04-24T06:32:08.082742+0000","last_fullsized":"2024-04-24T06:32:08.082742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:56.922190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073562+0000","last_change":"2024-04-24T06:32:08.063622+0000","last_active":"2024-04-24T06:32:08.073562+0000","last_peered":"2024-04-24T06:32:08.073562+0000","last_clean":"2024-04-24T06:32:08.073562+0000","last_became_active":"2024-04-24T06:32:08.061505+0000","last_became_peered":"2024-04-24T06:32:08.061505+0000","last_unstale":"2024-04-24T06:32:08.073562+0000","last_undegraded":"2024-04-24T06:32:08.073562+0000","last_fullsized":"2024-04-24T06:32:08.073562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:43.299504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073387+0000","last_change":"2024-04-24T06:32:08.065248+0000","last_active":"2024-04-24T06:32:08.073387+0000","last_peered":"2024-04-24T06:32:08.073387+0000","last_clean":"2024-04-24T06:32:08.073387+0000","last_became_active":"2024-04-24T06:32:08.062492+0000","last_became_peered":"2024-04-24T06:32:08.062492+0000","last_unstale":"2024-04-24T06:32:08.073387+0000","last_undegraded":"2024-04-24T06:32:08.073387+0000","last_fullsized":"2024-04-24T06:32:08.073387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:26:06.928050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082319+0000","last_change":"2024-04-24T06:32:08.075699+0000","last_active":"2024-04-24T06:32:08.082319+0000","last_peered":"2024-04-24T06:32:08.082319+0000","last_clean":"2024-04-24T06:32:08.082319+0000","last_became_active":"2024-04-24T06:32:08.074679+0000","last_became_peered":"2024-04-24T06:32:08.074679+0000","last_unstale":"2024-04-24T06:32:08.082319+0000","last_undegraded":"2024-04-24T06:32:08.082319+0000","last_fullsized":"2024-04-24T06:32:08.082319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:07.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075561+0000","last_change":"2024-04-24T06:32:08.070611+0000","last_active":"2024-04-24T06:32:08.075561+0000","last_peered":"2024-04-24T06:32:08.075561+0000","last_clean":"2024-04-24T06:32:08.075561+0000","last_became_active":"2024-04-24T06:32:08.067527+0000","last_became_peered":"2024-04-24T06:32:08.067527+0000","last_unstale":"2024-04-24T06:32:08.075561+0000","last_undegraded":"2024-04-24T06:32:08.075561+0000","last_fullsized":"2024-04-24T06:32:08.075561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081501+0000","last_change":"2024-04-24T06:32:08.078255+0000","last_active":"2024-04-24T06:32:08.081501+0000","last_peered":"2024-04-24T06:32:08.081501+0000","last_clean":"2024-04-24T06:32:08.081501+0000","last_became_active":"2024-04-24T06:32:08.077953+0000","last_became_peered":"2024-04-24T06:32:08.077953+0000","last_unstale":"2024-04-24T06:32:08.081501+0000","last_undegraded":"2024-04-24T06:32:08.081501+0000","last_fullsized":"2024-04-24T06:32:08.081501+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:45.782063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.070944+0000","last_change":"2024-04-24T06:32:08.056139+0000","last_active":"2024-04-24T06:32:08.070944+0000","last_peered":"2024-04-24T06:32:08.070944+0000","last_clean":"2024-04-24T06:32:08.070944+0000","last_became_active":"2024-04-24T06:32:08.055835+0000","last_became_peered":"2024-04-24T06:32:08.055835+0000","last_unstale":"2024-04-24T06:32:08.070944+0000","last_undegraded":"2024-04-24T06:32:08.070944+0000","last_fullsized":"2024-04-24T06:32:08.070944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:58:54.435910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073592+0000","last_change":"2024-04-24T06:32:08.063986+0000","last_active":"2024-04-24T06:32:08.073592+0000","last_peered":"2024-04-24T06:32:08.073592+0000","last_clean":"2024-04-24T06:32:08.073592+0000","last_became_active":"2024-04-24T06:32:08.063658+0000","last_became_peered":"2024-04-24T06:32:08.063658+0000","last_unstale":"2024-04-24T06:32:08.073592+0000","last_undegraded":"2024-04-24T06:32:08.073592+0000","last_fullsized":"2024-04-24T06:32:08.073592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:08.567405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075050+0000","last_change":"2024-04-24T06:32:08.065832+0000","last_active":"2024-04-24T06:32:08.075050+0000","last_peered":"2024-04-24T06:32:08.075050+0000","last_clean":"2024-04-24T06:32:08.075050+0000","last_became_active":"2024-04-24T06:32:08.065546+0000","last_became_peered":"2024-04-24T06:32:08.065546+0000","last_unstale":"2024-04-24T06:32:08.075050+0000","last_undegraded":"2024-04-24T06:32:08.075050+0000","last_fullsized":"2024-04-24T06:32:08.075050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082409+0000","last_change":"2024-04-24T06:32:08.077202+0000","last_active":"2024-04-24T06:32:08.082409+0000","last_peered":"2024-04-24T06:32:08.082409+0000","last_clean":"2024-04-24T06:32:08.082409+0000","last_became_active":"2024-04-24T06:32:08.076913+0000","last_became_peered":"2024-04-24T06:32:08.076913+0000","last_unstale":"2024-04-24T06:32:08.082409+0000","last_undegraded":"2024-04-24T06:32:08.082409+0000","last_fullsized":"2024-04-24T06:32:08.082409+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:24.208051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.080823+0000","last_change":"2024-04-24T06:32:08.079778+0000","last_active":"2024-04-24T06:32:08.080823+0000","last_peered":"2024-04-24T06:32:08.080823+0000","last_clean":"2024-04-24T06:32:08.080823+0000","last_became_active":"2024-04-24T06:32:08.079029+0000","last_became_peered":"2024-04-24T06:32:08.079029+0000","last_unstale":"2024-04-24T06:32:08.080823+0000","last_undegraded":"2024-04-24T06:32:08.080823+0000","last_fullsized":"2024-04-24T06:32:08.080823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:49.830107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067046+0000","last_change":"2024-04-24T06:32:08.064002+0000","last_active":"2024-04-24T06:32:08.067046+0000","last_peered":"2024-04-24T06:32:08.067046+0000","last_clean":"2024-04-24T06:32:08.067046+0000","last_became_active":"2024-04-24T06:32:08.063708+0000","last_became_peered":"2024-04-24T06:32:08.063708+0000","last_unstale":"2024-04-24T06:32:08.067046+0000","last_undegraded":"2024-04-24T06:32:08.067046+0000","last_fullsized":"2024-04-24T06:32:08.067046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:21.565304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074767+0000","last_change":"2024-04-24T06:32:08.068037+0000","last_active":"2024-04-24T06:32:08.074767+0000","last_peered":"2024-04-24T06:32:08.074767+0000","last_clean":"2024-04-24T06:32:08.074767+0000","last_became_active":"2024-04-24T06:32:08.066334+0000","last_became_peered":"2024-04-24T06:32:08.066334+0000","last_unstale":"2024-04-24T06:32:08.074767+0000","last_undegraded":"2024-04-24T06:32:08.074767+0000","last_fullsized":"2024-04-24T06:32:08.074767+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:58:23.381313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077669+0000","last_change":"2024-04-24T06:32:08.072318+0000","last_active":"2024-04-24T06:32:08.077669+0000","last_peered":"2024-04-24T06:32:08.077669+0000","last_clean":"2024-04-24T06:32:08.077669+0000","last_became_active":"2024-04-24T06:32:08.071998+0000","last_became_peered":"2024-04-24T06:32:08.071998+0000","last_unstale":"2024-04-24T06:32:08.077669+0000","last_undegraded":"2024-04-24T06:32:08.077669+0000","last_fullsized":"2024-04-24T06:32:08.077669+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072943+0000","last_change":"2024-04-24T06:32:08.067399+0000","last_active":"2024-04-24T06:32:08.072943+0000","last_peered":"2024-04-24T06:32:08.072943+0000","last_clean":"2024-04-24T06:32:08.072943+0000","last_became_active":"2024-04-24T06:32:08.066564+0000","last_became_peered":"2024-04-24T06:32:08.066564+0000","last_unstale":"2024-04-24T06:32:08.072943+0000","last_undegraded":"2024-04-24T06:32:08.072943+0000","last_fullsized":"2024-04-24T06:32:08.072943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:11:19.060664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074602+0000","last_change":"2024-04-24T06:32:08.069298+0000","last_active":"2024-04-24T06:32:08.074602+0000","last_peered":"2024-04-24T06:32:08.074602+0000","last_clean":"2024-04-24T06:32:08.074602+0000","last_became_active":"2024-04-24T06:32:08.068567+0000","last_became_peered":"2024-04-24T06:32:08.068567+0000","last_unstale":"2024-04-24T06:32:08.074602+0000","last_undegraded":"2024-04-24T06:32:08.074602+0000","last_fullsized":"2024-04-24T06:32:08.074602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081567+0000","last_change":"2024-04-24T06:32:08.078328+0000","last_active":"2024-04-24T06:32:08.081567+0000","last_peered":"2024-04-24T06:32:08.081567+0000","last_clean":"2024-04-24T06:32:08.081567+0000","last_became_active":"2024-04-24T06:32:08.077912+0000","last_became_peered":"2024-04-24T06:32:08.077912+0000","last_unstale":"2024-04-24T06:32:08.081567+0000","last_undegraded":"2024-04-24T06:32:08.081567+0000","last_fullsized":"2024-04-24T06:32:08.081567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:37.463043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.064229+0000","last_change":"2024-04-24T06:32:08.056278+0000","last_active":"2024-04-24T06:32:08.064229+0000","last_peered":"2024-04-24T06:32:08.064229+0000","last_clean":"2024-04-24T06:32:08.064229+0000","last_became_active":"2024-04-24T06:32:08.055988+0000","last_became_peered":"2024-04-24T06:32:08.055988+0000","last_unstale":"2024-04-24T06:32:08.064229+0000","last_undegraded":"2024-04-24T06:32:08.064229+0000","last_fullsized":"2024-04-24T06:32:08.064229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:10.369782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.069989+0000","last_change":"2024-04-24T06:32:08.066710+0000","last_active":"2024-04-24T06:32:08.069989+0000","last_peered":"2024-04-24T06:32:08.069989+0000","last_clean":"2024-04-24T06:32:08.069989+0000","last_became_active":"2024-04-24T06:32:08.065429+0000","last_became_peered":"2024-04-24T06:32:08.065429+0000","last_unstale":"2024-04-24T06:32:08.069989+0000","last_undegraded":"2024-04-24T06:32:08.069989+0000","last_fullsized":"2024-04-24T06:32:08.069989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:49.769247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078497+0000","last_change":"2024-04-24T06:32:08.071344+0000","last_active":"2024-04-24T06:32:08.078497+0000","last_peered":"2024-04-24T06:32:08.078497+0000","last_clean":"2024-04-24T06:32:08.078497+0000","last_became_active":"2024-04-24T06:32:08.071034+0000","last_became_peered":"2024-04-24T06:32:08.071034+0000","last_unstale":"2024-04-24T06:32:08.078497+0000","last_undegraded":"2024-04-24T06:32:08.078497+0000","last_fullsized":"2024-04-24T06:32:08.078497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:21.568721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062194+0000","last_change":"2024-04-24T06:32:08.059701+0000","last_active":"2024-04-24T06:32:08.062194+0000","last_peered":"2024-04-24T06:32:08.062194+0000","last_clean":"2024-04-24T06:32:08.062194+0000","last_became_active":"2024-04-24T06:32:08.059407+0000","last_became_peered":"2024-04-24T06:32:08.059407+0000","last_unstale":"2024-04-24T06:32:08.062194+0000","last_undegraded":"2024-04-24T06:32:08.062194+0000","last_fullsized":"2024-04-24T06:32:08.062194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:54:25.098465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081432+0000","last_change":"2024-04-24T06:32:08.072926+0000","last_active":"2024-04-24T06:32:08.081432+0000","last_peered":"2024-04-24T06:32:08.081432+0000","last_clean":"2024-04-24T06:32:08.081432+0000","last_became_active":"2024-04-24T06:32:08.072622+0000","last_became_peered":"2024-04-24T06:32:08.072622+0000","last_unstale":"2024-04-24T06:32:08.081432+0000","last_undegraded":"2024-04-24T06:32:08.081432+0000","last_fullsized":"2024-04-24T06:32:08.081432+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:40:45.744599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078438+0000","last_change":"2024-04-24T06:32:08.074611+0000","last_active":"2024-04-24T06:32:08.078438+0000","last_peered":"2024-04-24T06:32:08.078438+0000","last_clean":"2024-04-24T06:32:08.078438+0000","last_became_active":"2024-04-24T06:32:08.074296+0000","last_became_peered":"2024-04-24T06:32:08.074296+0000","last_unstale":"2024-04-24T06:32:08.078438+0000","last_undegraded":"2024-04-24T06:32:08.078438+0000","last_fullsized":"2024-04-24T06:32:08.078438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:21.568721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082663+0000","last_change":"2024-04-24T06:32:08.077143+0000","last_active":"2024-04-24T06:32:08.082663+0000","last_peered":"2024-04-24T06:32:08.082663+0000","last_clean":"2024-04-24T06:32:08.082663+0000","last_became_active":"2024-04-24T06:32:08.072152+0000","last_became_peered":"2024-04-24T06:32:08.072152+0000","last_unstale":"2024-04-24T06:32:08.082663+0000","last_undegraded":"2024-04-24T06:32:08.082663+0000","last_fullsized":"2024-04-24T06:32:08.082663+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:29.128899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.084149+0000","last_change":"2024-04-24T06:32:08.080965+0000","last_active":"2024-04-24T06:32:08.084149+0000","last_peered":"2024-04-24T06:32:08.084149+0000","last_clean":"2024-04-24T06:32:08.084149+0000","last_became_active":"2024-04-24T06:32:08.080404+0000","last_became_peered":"2024-04-24T06:32:08.080404+0000","last_unstale":"2024-04-24T06:32:08.084149+0000","last_undegraded":"2024-04-24T06:32:08.084149+0000","last_fullsized":"2024-04-24T06:32:08.084149+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:36.894775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081251+0000","last_change":"2024-04-24T06:32:08.068034+0000","last_active":"2024-04-24T06:32:08.081251+0000","last_peered":"2024-04-24T06:32:08.081251+0000","last_clean":"2024-04-24T06:32:08.081251+0000","last_became_active":"2024-04-24T06:32:08.067742+0000","last_became_peered":"2024-04-24T06:32:08.067742+0000","last_unstale":"2024-04-24T06:32:08.081251+0000","last_undegraded":"2024-04-24T06:32:08.081251+0000","last_fullsized":"2024-04-24T06:32:08.081251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095763+0000","last_change":"2024-04-24T06:32:07.050936+0000","last_active":"2024-04-24T06:32:08.095763+0000","last_peered":"2024-04-24T06:32:08.095763+0000","last_clean":"2024-04-24T06:32:08.095763+0000","last_became_active":"2024-04-24T06:32:07.050670+0000","last_became_peered":"2024-04-24T06:32:07.050670+0000","last_unstale":"2024-04-24T06:32:08.095763+0000","last_undegraded":"2024-04-24T06:32:08.095763+0000","last_fullsized":"2024-04-24T06:32:08.095763+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:44.448627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.070145+0000","last_change":"2024-04-24T06:32:08.063948+0000","last_active":"2024-04-24T06:32:08.070145+0000","last_peered":"2024-04-24T06:32:08.070145+0000","last_clean":"2024-04-24T06:32:08.070145+0000","last_became_active":"2024-04-24T06:32:08.063673+0000","last_became_peered":"2024-04-24T06:32:08.063673+0000","last_unstale":"2024-04-24T06:32:08.070145+0000","last_undegraded":"2024-04-24T06:32:08.070145+0000","last_fullsized":"2024-04-24T06:32:08.070145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:38:27.189569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.094887+0000","last_change":"2024-04-24T06:32:07.044432+0000","last_active":"2024-04-24T06:32:08.094887+0000","last_peered":"2024-04-24T06:32:08.094887+0000","last_clean":"2024-04-24T06:32:08.094887+0000","last_became_active":"2024-04-24T06:32:07.044014+0000","last_became_peered":"2024-04-24T06:32:07.044014+0000","last_unstale":"2024-04-24T06:32:08.094887+0000","last_undegraded":"2024-04-24T06:32:08.094887+0000","last_fullsized":"2024-04-24T06:32:08.094887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.074198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075678+0000","last_change":"2024-04-24T06:32:08.066000+0000","last_active":"2024-04-24T06:32:08.075678+0000","last_peered":"2024-04-24T06:32:08.075678+0000","last_clean":"2024-04-24T06:32:08.075678+0000","last_became_active":"2024-04-24T06:32:08.064744+0000","last_became_peered":"2024-04-24T06:32:08.064744+0000","last_unstale":"2024-04-24T06:32:08.075678+0000","last_undegraded":"2024-04-24T06:32:08.075678+0000","last_fullsized":"2024-04-24T06:32:08.075678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095052+0000","last_change":"2024-04-24T06:32:07.046255+0000","last_active":"2024-04-24T06:32:08.095052+0000","last_peered":"2024-04-24T06:32:08.095052+0000","last_clean":"2024-04-24T06:32:08.095052+0000","last_became_active":"2024-04-24T06:32:07.045977+0000","last_became_peered":"2024-04-24T06:32:07.045977+0000","last_unstale":"2024-04-24T06:32:08.095052+0000","last_undegraded":"2024-04-24T06:32:08.095052+0000","last_fullsized":"2024-04-24T06:32:08.095052+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:28.348442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078459+0000","last_change":"2024-04-24T06:32:08.067616+0000","last_active":"2024-04-24T06:32:08.078459+0000","last_peered":"2024-04-24T06:32:08.078459+0000","last_clean":"2024-04-24T06:32:08.078459+0000","last_became_active":"2024-04-24T06:32:08.067338+0000","last_became_peered":"2024-04-24T06:32:08.067338+0000","last_unstale":"2024-04-24T06:32:08.078459+0000","last_undegraded":"2024-04-24T06:32:08.078459+0000","last_fullsized":"2024-04-24T06:32:08.078459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092638+0000","last_change":"2024-04-24T06:32:07.044352+0000","last_active":"2024-04-24T06:32:08.092638+0000","last_peered":"2024-04-24T06:32:08.092638+0000","last_clean":"2024-04-24T06:32:08.092638+0000","last_became_active":"2024-04-24T06:32:07.043680+0000","last_became_peered":"2024-04-24T06:32:07.043680+0000","last_unstale":"2024-04-24T06:32:08.092638+0000","last_undegraded":"2024-04-24T06:32:08.092638+0000","last_fullsized":"2024-04-24T06:32:08.092638+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:44.448627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072918+0000","last_change":"2024-04-24T06:32:08.067822+0000","last_active":"2024-04-24T06:32:08.072918+0000","last_peered":"2024-04-24T06:32:08.072918+0000","last_clean":"2024-04-24T06:32:08.072918+0000","last_became_active":"2024-04-24T06:32:08.067251+0000","last_became_peered":"2024-04-24T06:32:08.067251+0000","last_unstale":"2024-04-24T06:32:08.072918+0000","last_undegraded":"2024-04-24T06:32:08.072918+0000","last_fullsized":"2024-04-24T06:32:08.072918+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:29.108481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.089442+0000","last_change":"2024-04-24T06:32:07.026561+0000","last_active":"2024-04-24T06:32:08.089442+0000","last_peered":"2024-04-24T06:32:08.089442+0000","last_clean":"2024-04-24T06:32:08.089442+0000","last_became_active":"2024-04-24T06:32:07.023320+0000","last_became_peered":"2024-04-24T06:32:07.023320+0000","last_unstale":"2024-04-24T06:32:08.089442+0000","last_undegraded":"2024-04-24T06:32:08.089442+0000","last_fullsized":"2024-04-24T06:32:08.089442+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:35.322600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082909+0000","last_change":"2024-04-24T06:32:08.078652+0000","last_active":"2024-04-24T06:32:08.082909+0000","last_peered":"2024-04-24T06:32:08.082909+0000","last_clean":"2024-04-24T06:32:08.082909+0000","last_became_active":"2024-04-24T06:32:08.078126+0000","last_became_peered":"2024-04-24T06:32:08.078126+0000","last_unstale":"2024-04-24T06:32:08.082909+0000","last_undegraded":"2024-04-24T06:32:08.082909+0000","last_fullsized":"2024-04-24T06:32:08.082909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:54:39.691836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.090395+0000","last_change":"2024-04-24T06:32:07.049980+0000","last_active":"2024-04-24T06:32:08.090395+0000","last_peered":"2024-04-24T06:32:08.090395+0000","last_clean":"2024-04-24T06:32:08.090395+0000","last_became_active":"2024-04-24T06:32:07.049563+0000","last_became_peered":"2024-04-24T06:32:07.049563+0000","last_unstale":"2024-04-24T06:32:08.090395+0000","last_undegraded":"2024-04-24T06:32:08.090395+0000","last_fullsized":"2024-04-24T06:32:08.090395+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:07.552980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072528+0000","last_change":"2024-04-24T06:32:08.065091+0000","last_active":"2024-04-24T06:32:08.072528+0000","last_peered":"2024-04-24T06:32:08.072528+0000","last_clean":"2024-04-24T06:32:08.072528+0000","last_became_active":"2024-04-24T06:32:08.064747+0000","last_became_peered":"2024-04-24T06:32:08.064747+0000","last_unstale":"2024-04-24T06:32:08.072528+0000","last_undegraded":"2024-04-24T06:32:08.072528+0000","last_fullsized":"2024-04-24T06:32:08.072528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:11.318107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.050288+0000","last_change":"2024-04-24T06:32:07.048665+0000","last_active":"2024-04-24T06:32:08.050288+0000","last_peered":"2024-04-24T06:32:08.050288+0000","last_clean":"2024-04-24T06:32:08.050288+0000","last_became_active":"2024-04-24T06:32:07.047973+0000","last_became_peered":"2024-04-24T06:32:07.047973+0000","last_unstale":"2024-04-24T06:32:08.050288+0000","last_undegraded":"2024-04-24T06:32:08.050288+0000","last_fullsized":"2024-04-24T06:32:08.050288+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:07.552980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.076577+0000","last_change":"2024-04-24T06:32:08.072036+0000","last_active":"2024-04-24T06:32:08.076577+0000","last_peered":"2024-04-24T06:32:08.076577+0000","last_clean":"2024-04-24T06:32:08.076577+0000","last_became_active":"2024-04-24T06:32:08.071660+0000","last_became_peered":"2024-04-24T06:32:08.071660+0000","last_unstale":"2024-04-24T06:32:08.076577+0000","last_undegraded":"2024-04-24T06:32:08.076577+0000","last_fullsized":"2024-04-24T06:32:08.076577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:30.676306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.041363+0000","last_change":"2024-04-24T06:32:07.041803+0000","last_active":"2024-04-24T06:32:08.041363+0000","last_peered":"2024-04-24T06:32:08.041363+0000","last_clean":"2024-04-24T06:32:08.041363+0000","last_became_active":"2024-04-24T06:32:07.041441+0000","last_became_peered":"2024-04-24T06:32:07.041441+0000","last_unstale":"2024-04-24T06:32:08.041363+0000","last_undegraded":"2024-04-24T06:32:08.041363+0000","last_fullsized":"2024-04-24T06:32:08.041363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:25.286075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.071870+0000","last_change":"2024-04-24T06:32:08.060153+0000","last_active":"2024-04-24T06:32:08.071870+0000","last_peered":"2024-04-24T06:32:08.071870+0000","last_clean":"2024-04-24T06:32:08.071870+0000","last_became_active":"2024-04-24T06:32:08.059375+0000","last_became_peered":"2024-04-24T06:32:08.059375+0000","last_unstale":"2024-04-24T06:32:08.071870+0000","last_undegraded":"2024-04-24T06:32:08.071870+0000","last_fullsized":"2024-04-24T06:32:08.071870+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:07.233569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.053839+0000","last_change":"2024-04-24T06:32:07.043903+0000","last_active":"2024-04-24T06:32:08.053839+0000","last_peered":"2024-04-24T06:32:08.053839+0000","last_clean":"2024-04-24T06:32:08.053839+0000","last_became_active":"2024-04-24T06:32:07.043569+0000","last_became_peered":"2024-04-24T06:32:07.043569+0000","last_unstale":"2024-04-24T06:32:08.053839+0000","last_undegraded":"2024-04-24T06:32:08.053839+0000","last_fullsized":"2024-04-24T06:32:08.053839+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:48.754822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075130+0000","last_change":"2024-04-24T06:32:08.070807+0000","last_active":"2024-04-24T06:32:08.075130+0000","last_peered":"2024-04-24T06:32:08.075130+0000","last_clean":"2024-04-24T06:32:08.075130+0000","last_became_active":"2024-04-24T06:32:08.069637+0000","last_became_peered":"2024-04-24T06:32:08.069637+0000","last_unstale":"2024-04-24T06:32:08.075130+0000","last_undegraded":"2024-04-24T06:32:08.075130+0000","last_fullsized":"2024-04-24T06:32:08.075130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.064614+0000","last_change":"2024-04-24T06:32:07.043619+0000","last_active":"2024-04-24T06:32:08.064614+0000","last_peered":"2024-04-24T06:32:08.064614+0000","last_clean":"2024-04-24T06:32:08.064614+0000","last_became_active":"2024-04-24T06:32:07.043332+0000","last_became_peered":"2024-04-24T06:32:07.043332+0000","last_unstale":"2024-04-24T06:32:08.064614+0000","last_undegraded":"2024-04-24T06:32:08.064614+0000","last_fullsized":"2024-04-24T06:32:08.064614+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:11:40.657747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067726+0000","last_change":"2024-04-24T06:32:08.057500+0000","last_active":"2024-04-24T06:32:08.067726+0000","last_peered":"2024-04-24T06:32:08.067726+0000","last_clean":"2024-04-24T06:32:08.067726+0000","last_became_active":"2024-04-24T06:32:08.057153+0000","last_became_peered":"2024-04-24T06:32:08.057153+0000","last_unstale":"2024-04-24T06:32:08.067726+0000","last_undegraded":"2024-04-24T06:32:08.067726+0000","last_fullsized":"2024-04-24T06:32:08.067726+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:37.463043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092950+0000","last_change":"2024-04-24T06:32:07.046720+0000","last_active":"2024-04-24T06:32:08.092950+0000","last_peered":"2024-04-24T06:32:08.092950+0000","last_clean":"2024-04-24T06:32:08.092950+0000","last_became_active":"2024-04-24T06:32:07.046325+0000","last_became_peered":"2024-04-24T06:32:07.046325+0000","last_unstale":"2024-04-24T06:32:08.092950+0000","last_undegraded":"2024-04-24T06:32:08.092950+0000","last_fullsized":"2024-04-24T06:32:08.092950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:52.305363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079508+0000","last_change":"2024-04-24T06:32:08.075794+0000","last_active":"2024-04-24T06:32:08.079508+0000","last_peered":"2024-04-24T06:32:08.079508+0000","last_clean":"2024-04-24T06:32:08.079508+0000","last_became_active":"2024-04-24T06:32:08.072340+0000","last_became_peered":"2024-04-24T06:32:08.072340+0000","last_unstale":"2024-04-24T06:32:08.079508+0000","last_undegraded":"2024-04-24T06:32:08.079508+0000","last_fullsized":"2024-04-24T06:32:08.079508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:45:20.717382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.097394+0000","last_change":"2024-04-24T06:32:07.041461+0000","last_active":"2024-04-24T06:32:08.097394+0000","last_peered":"2024-04-24T06:32:08.097394+0000","last_clean":"2024-04-24T06:32:08.097394+0000","last_became_active":"2024-04-24T06:32:07.040792+0000","last_became_peered":"2024-04-24T06:32:07.040792+0000","last_unstale":"2024-04-24T06:32:08.097394+0000","last_undegraded":"2024-04-24T06:32:08.097394+0000","last_fullsized":"2024-04-24T06:32:08.097394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:28.656256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077082+0000","last_change":"2024-04-24T06:32:08.073385+0000","last_active":"2024-04-24T06:32:08.077082+0000","last_peered":"2024-04-24T06:32:08.077082+0000","last_clean":"2024-04-24T06:32:08.077082+0000","last_became_active":"2024-04-24T06:32:08.073058+0000","last_became_peered":"2024-04-24T06:32:08.073058+0000","last_unstale":"2024-04-24T06:32:08.077082+0000","last_undegraded":"2024-04-24T06:32:08.077082+0000","last_fullsized":"2024-04-24T06:32:08.077082+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.089711+0000","last_change":"2024-04-24T06:32:07.046467+0000","last_active":"2024-04-24T06:32:08.089711+0000","last_peered":"2024-04-24T06:32:08.089711+0000","last_clean":"2024-04-24T06:32:08.089711+0000","last_became_active":"2024-04-24T06:32:07.046172+0000","last_became_peered":"2024-04-24T06:32:07.046172+0000","last_unstale":"2024-04-24T06:32:08.089711+0000","last_undegraded":"2024-04-24T06:32:08.089711+0000","last_fullsized":"2024-04-24T06:32:08.089711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:11.320302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077693+0000","last_change":"2024-04-24T06:32:08.074013+0000","last_active":"2024-04-24T06:32:08.077693+0000","last_peered":"2024-04-24T06:32:08.077693+0000","last_clean":"2024-04-24T06:32:08.077693+0000","last_became_active":"2024-04-24T06:32:08.071567+0000","last_became_peered":"2024-04-24T06:32:08.071567+0000","last_unstale":"2024-04-24T06:32:08.077693+0000","last_undegraded":"2024-04-24T06:32:08.077693+0000","last_fullsized":"2024-04-24T06:32:08.077693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.059847+0000","last_change":"2024-04-24T06:32:07.047479+0000","last_active":"2024-04-24T06:32:08.059847+0000","last_peered":"2024-04-24T06:32:08.059847+0000","last_clean":"2024-04-24T06:32:08.059847+0000","last_became_active":"2024-04-24T06:32:07.046228+0000","last_became_peered":"2024-04-24T06:32:07.046228+0000","last_unstale":"2024-04-24T06:32:08.059847+0000","last_undegraded":"2024-04-24T06:32:08.059847+0000","last_fullsized":"2024-04-24T06:32:08.059847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:40.500121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075302+0000","last_change":"2024-04-24T06:32:08.066452+0000","last_active":"2024-04-24T06:32:08.075302+0000","last_peered":"2024-04-24T06:32:08.075302+0000","last_clean":"2024-04-24T06:32:08.075302+0000","last_became_active":"2024-04-24T06:32:08.062268+0000","last_became_peered":"2024-04-24T06:32:08.062268+0000","last_unstale":"2024-04-24T06:32:08.075302+0000","last_undegraded":"2024-04-24T06:32:08.075302+0000","last_fullsized":"2024-04-24T06:32:08.075302+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:29.362867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.050766+0000","last_change":"2024-04-24T06:32:07.043629+0000","last_active":"2024-04-24T06:32:08.050766+0000","last_peered":"2024-04-24T06:32:08.050766+0000","last_clean":"2024-04-24T06:32:08.050766+0000","last_became_active":"2024-04-24T06:32:07.043326+0000","last_became_peered":"2024-04-24T06:32:07.043326+0000","last_unstale":"2024-04-24T06:32:08.050766+0000","last_undegraded":"2024-04-24T06:32:08.050766+0000","last_fullsized":"2024-04-24T06:32:08.050766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:28.348442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074635+0000","last_change":"2024-04-24T06:32:08.069595+0000","last_active":"2024-04-24T06:32:08.074635+0000","last_peered":"2024-04-24T06:32:08.074635+0000","last_clean":"2024-04-24T06:32:08.074635+0000","last_became_active":"2024-04-24T06:32:08.068971+0000","last_became_peered":"2024-04-24T06:32:08.068971+0000","last_unstale":"2024-04-24T06:32:08.074635+0000","last_undegraded":"2024-04-24T06:32:08.074635+0000","last_fullsized":"2024-04-24T06:32:08.074635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.093313+0000","last_change":"2024-04-24T06:32:07.051220+0000","last_active":"2024-04-24T06:32:08.093313+0000","last_peered":"2024-04-24T06:32:08.093313+0000","last_clean":"2024-04-24T06:32:08.093313+0000","last_became_active":"2024-04-24T06:32:07.050841+0000","last_became_peered":"2024-04-24T06:32:07.050841+0000","last_unstale":"2024-04-24T06:32:08.093313+0000","last_undegraded":"2024-04-24T06:32:08.093313+0000","last_fullsized":"2024-04-24T06:32:08.093313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:32.431569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079692+0000","last_change":"2024-04-24T06:32:08.077705+0000","last_active":"2024-04-24T06:32:08.079692+0000","last_peered":"2024-04-24T06:32:08.079692+0000","last_clean":"2024-04-24T06:32:08.079692+0000","last_became_active":"2024-04-24T06:32:08.077336+0000","last_became_peered":"2024-04-24T06:32:08.077336+0000","last_unstale":"2024-04-24T06:32:08.079692+0000","last_undegraded":"2024-04-24T06:32:08.079692+0000","last_fullsized":"2024-04-24T06:32:08.079692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:21.543323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092158+0000","last_change":"2024-04-24T06:32:07.047276+0000","last_active":"2024-04-24T06:32:08.092158+0000","last_peered":"2024-04-24T06:32:08.092158+0000","last_clean":"2024-04-24T06:32:08.092158+0000","last_became_active":"2024-04-24T06:32:07.046802+0000","last_became_peered":"2024-04-24T06:32:07.046802+0000","last_unstale":"2024-04-24T06:32:08.092158+0000","last_undegraded":"2024-04-24T06:32:08.092158+0000","last_fullsized":"2024-04-24T06:32:08.092158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:50:55.375117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062397+0000","last_change":"2024-04-24T06:32:08.059281+0000","last_active":"2024-04-24T06:32:08.062397+0000","last_peered":"2024-04-24T06:32:08.062397+0000","last_clean":"2024-04-24T06:32:08.062397+0000","last_became_active":"2024-04-24T06:32:08.058948+0000","last_became_peered":"2024-04-24T06:32:08.058948+0000","last_unstale":"2024-04-24T06:32:08.062397+0000","last_undegraded":"2024-04-24T06:32:08.062397+0000","last_fullsized":"2024-04-24T06:32:08.062397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:49.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095269+0000","last_change":"2024-04-24T06:32:07.039487+0000","last_active":"2024-04-24T06:32:08.095269+0000","last_peered":"2024-04-24T06:32:08.095269+0000","last_clean":"2024-04-24T06:32:08.095269+0000","last_became_active":"2024-04-24T06:32:07.039162+0000","last_became_peered":"2024-04-24T06:32:07.039162+0000","last_unstale":"2024-04-24T06:32:08.095269+0000","last_undegraded":"2024-04-24T06:32:08.095269+0000","last_fullsized":"2024-04-24T06:32:08.095269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:48.961434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081951+0000","last_change":"2024-04-24T06:32:08.074884+0000","last_active":"2024-04-24T06:32:08.081951+0000","last_peered":"2024-04-24T06:32:08.081951+0000","last_clean":"2024-04-24T06:32:08.081951+0000","last_became_active":"2024-04-24T06:32:08.074584+0000","last_became_peered":"2024-04-24T06:32:08.074584+0000","last_unstale":"2024-04-24T06:32:08.081951+0000","last_undegraded":"2024-04-24T06:32:08.081951+0000","last_fullsized":"2024-04-24T06:32:08.081951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:38.336846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092969+0000","last_change":"2024-04-24T06:32:07.044126+0000","last_active":"2024-04-24T06:32:08.092969+0000","last_peered":"2024-04-24T06:32:08.092969+0000","last_clean":"2024-04-24T06:32:08.092969+0000","last_became_active":"2024-04-24T06:32:07.043243+0000","last_became_peered":"2024-04-24T06:32:07.043243+0000","last_unstale":"2024-04-24T06:32:08.092969+0000","last_undegraded":"2024-04-24T06:32:08.092969+0000","last_fullsized":"2024-04-24T06:32:08.092969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:16:32.231040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078566+0000","last_change":"2024-04-24T06:32:08.075286+0000","last_active":"2024-04-24T06:32:08.078566+0000","last_peered":"2024-04-24T06:32:08.078566+0000","last_clean":"2024-04-24T06:32:08.078566+0000","last_became_active":"2024-04-24T06:32:08.072461+0000","last_became_peered":"2024-04-24T06:32:08.072461+0000","last_unstale":"2024-04-24T06:32:08.078566+0000","last_undegraded":"2024-04-24T06:32:08.078566+0000","last_fullsized":"2024-04-24T06:32:08.078566+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:36.894775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.044988+0000","last_change":"2024-04-24T06:32:07.045408+0000","last_active":"2024-04-24T06:32:08.044988+0000","last_peered":"2024-04-24T06:32:08.044988+0000","last_clean":"2024-04-24T06:32:08.044988+0000","last_became_active":"2024-04-24T06:32:07.043456+0000","last_became_peered":"2024-04-24T06:32:07.043456+0000","last_unstale":"2024-04-24T06:32:08.044988+0000","last_undegraded":"2024-04-24T06:32:08.044988+0000","last_fullsized":"2024-04-24T06:32:08.044988+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:42.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077060+0000","last_change":"2024-04-24T06:32:08.073233+0000","last_active":"2024-04-24T06:32:08.077060+0000","last_peered":"2024-04-24T06:32:08.077060+0000","last_clean":"2024-04-24T06:32:08.077060+0000","last_became_active":"2024-04-24T06:32:08.072922+0000","last_became_peered":"2024-04-24T06:32:08.072922+0000","last_unstale":"2024-04-24T06:32:08.077060+0000","last_undegraded":"2024-04-24T06:32:08.077060+0000","last_fullsized":"2024-04-24T06:32:08.077060+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.093509+0000","last_change":"2024-04-24T06:32:07.042679+0000","last_active":"2024-04-24T06:32:08.093509+0000","last_peered":"2024-04-24T06:32:08.093509+0000","last_clean":"2024-04-24T06:32:08.093509+0000","last_became_active":"2024-04-24T06:32:07.042050+0000","last_became_peered":"2024-04-24T06:32:07.042050+0000","last_unstale":"2024-04-24T06:32:08.093509+0000","last_undegraded":"2024-04-24T06:32:08.093509+0000","last_fullsized":"2024-04-24T06:32:08.093509+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:54:22.132919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077168+0000","last_change":"2024-04-24T06:32:08.068917+0000","last_active":"2024-04-24T06:32:08.077168+0000","last_peered":"2024-04-24T06:32:08.077168+0000","last_clean":"2024-04-24T06:32:08.077168+0000","last_became_active":"2024-04-24T06:32:08.068648+0000","last_became_peered":"2024-04-24T06:32:08.068648+0000","last_unstale":"2024-04-24T06:32:08.077168+0000","last_undegraded":"2024-04-24T06:32:08.077168+0000","last_fullsized":"2024-04-24T06:32:08.077168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.034337+0000","last_change":"2024-04-24T06:32:07.036954+0000","last_active":"2024-04-24T06:32:08.034337+0000","last_peered":"2024-04-24T06:32:08.034337+0000","last_clean":"2024-04-24T06:32:08.034337+0000","last_became_active":"2024-04-24T06:32:07.036666+0000","last_became_peered":"2024-04-24T06:32:07.036666+0000","last_unstale":"2024-04-24T06:32:08.034337+0000","last_undegraded":"2024-04-24T06:32:08.034337+0000","last_fullsized":"2024-04-24T06:32:08.034337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:21:55.451625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075037+0000","last_change":"2024-04-24T06:32:08.063385+0000","last_active":"2024-04-24T06:32:08.075037+0000","last_peered":"2024-04-24T06:32:08.075037+0000","last_clean":"2024-04-24T06:32:08.075037+0000","last_became_active":"2024-04-24T06:32:08.063058+0000","last_became_peered":"2024-04-24T06:32:08.063058+0000","last_unstale":"2024-04-24T06:32:08.075037+0000","last_undegraded":"2024-04-24T06:32:08.075037+0000","last_fullsized":"2024-04-24T06:32:08.075037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:49.769247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.049472+0000","last_change":"2024-04-24T06:32:07.047707+0000","last_active":"2024-04-24T06:32:08.049472+0000","last_peered":"2024-04-24T06:32:08.049472+0000","last_clean":"2024-04-24T06:32:08.049472+0000","last_became_active":"2024-04-24T06:32:07.047392+0000","last_became_peered":"2024-04-24T06:32:07.047392+0000","last_unstale":"2024-04-24T06:32:08.049472+0000","last_undegraded":"2024-04-24T06:32:08.049472+0000","last_fullsized":"2024-04-24T06:32:08.049472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:42.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072108+0000","last_change":"2024-04-24T06:32:08.068433+0000","last_active":"2024-04-24T06:32:08.072108+0000","last_peered":"2024-04-24T06:32:08.072108+0000","last_clean":"2024-04-24T06:32:08.072108+0000","last_became_active":"2024-04-24T06:32:08.067941+0000","last_became_peered":"2024-04-24T06:32:08.067941+0000","last_unstale":"2024-04-24T06:32:08.072108+0000","last_undegraded":"2024-04-24T06:32:08.072108+0000","last_fullsized":"2024-04-24T06:32:08.072108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:53.319788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.091989+0000","last_change":"2024-04-24T06:32:07.044900+0000","last_active":"2024-04-24T06:32:08.091989+0000","last_peered":"2024-04-24T06:32:08.091989+0000","last_clean":"2024-04-24T06:32:08.091989+0000","last_became_active":"2024-04-24T06:32:07.043759+0000","last_became_peered":"2024-04-24T06:32:07.043759+0000","last_unstale":"2024-04-24T06:32:08.091989+0000","last_undegraded":"2024-04-24T06:32:08.091989+0000","last_fullsized":"2024-04-24T06:32:08.091989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:10.303682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.076963+0000","last_change":"2024-04-24T06:32:08.073201+0000","last_active":"2024-04-24T06:32:08.076963+0000","last_peered":"2024-04-24T06:32:08.076963+0000","last_clean":"2024-04-24T06:32:08.076963+0000","last_became_active":"2024-04-24T06:32:08.072794+0000","last_became_peered":"2024-04-24T06:32:08.072794+0000","last_unstale":"2024-04-24T06:32:08.076963+0000","last_undegraded":"2024-04-24T06:32:08.076963+0000","last_fullsized":"2024-04-24T06:32:08.076963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.049303+0000","last_change":"2024-04-24T06:32:07.049288+0000","last_active":"2024-04-24T06:32:08.049303+0000","last_peered":"2024-04-24T06:32:08.049303+0000","last_clean":"2024-04-24T06:32:08.049303+0000","last_became_active":"2024-04-24T06:32:07.049006+0000","last_became_peered":"2024-04-24T06:32:07.049006+0000","last_unstale":"2024-04-24T06:32:08.049303+0000","last_undegraded":"2024-04-24T06:32:08.049303+0000","last_fullsized":"2024-04-24T06:32:08.049303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:16:32.231040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067702+0000","last_change":"2024-04-24T06:32:08.056596+0000","last_active":"2024-04-24T06:32:08.067702+0000","last_peered":"2024-04-24T06:32:08.067702+0000","last_clean":"2024-04-24T06:32:08.067702+0000","last_became_active":"2024-04-24T06:32:08.056294+0000","last_became_peered":"2024-04-24T06:32:08.056294+0000","last_unstale":"2024-04-24T06:32:08.067702+0000","last_undegraded":"2024-04-24T06:32:08.067702+0000","last_fullsized":"2024-04-24T06:32:08.067702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:45.782063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.059202+0000","last_change":"2024-04-24T06:32:07.049043+0000","last_active":"2024-04-24T06:32:08.059202+0000","last_peered":"2024-04-24T06:32:08.059202+0000","last_clean":"2024-04-24T06:32:08.059202+0000","last_became_active":"2024-04-24T06:32:07.048730+0000","last_became_peered":"2024-04-24T06:32:07.048730+0000","last_unstale":"2024-04-24T06:32:08.059202+0000","last_undegraded":"2024-04-24T06:32:08.059202+0000","last_fullsized":"2024-04-24T06:32:08.059202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:21:55.451625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075170+0000","last_change":"2024-04-24T06:32:08.064445+0000","last_active":"2024-04-24T06:32:08.075170+0000","last_peered":"2024-04-24T06:32:08.075170+0000","last_clean":"2024-04-24T06:32:08.075170+0000","last_became_active":"2024-04-24T06:32:08.064162+0000","last_became_peered":"2024-04-24T06:32:08.064162+0000","last_unstale":"2024-04-24T06:32:08.075170+0000","last_undegraded":"2024-04-24T06:32:08.075170+0000","last_fullsized":"2024-04-24T06:32:08.075170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:53.319788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062842+0000","last_change":"2024-04-24T06:32:07.048053+0000","last_active":"2024-04-24T06:32:08.062842+0000","last_peered":"2024-04-24T06:32:08.062842+0000","last_clean":"2024-04-24T06:32:08.062842+0000","last_became_active":"2024-04-24T06:32:07.047709+0000","last_became_peered":"2024-04-24T06:32:07.047709+0000","last_unstale":"2024-04-24T06:32:08.062842+0000","last_undegraded":"2024-04-24T06:32:08.062842+0000","last_fullsized":"2024-04-24T06:32:08.062842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:32.431569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.084945+0000","last_change":"2024-04-24T06:32:08.081923+0000","last_active":"2024-04-24T06:32:08.084945+0000","last_peered":"2024-04-24T06:32:08.084945+0000","last_clean":"2024-04-24T06:32:08.084945+0000","last_became_active":"2024-04-24T06:32:08.080515+0000","last_became_peered":"2024-04-24T06:32:08.080515+0000","last_unstale":"2024-04-24T06:32:08.084945+0000","last_undegraded":"2024-04-24T06:32:08.084945+0000","last_fullsized":"2024-04-24T06:32:08.084945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:10.528279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.035039+0000","last_change":"2024-04-24T06:32:07.045434+0000","last_active":"2024-04-24T06:32:08.035039+0000","last_peered":"2024-04-24T06:32:08.035039+0000","last_clean":"2024-04-24T06:32:08.035039+0000","last_became_active":"2024-04-24T06:32:07.044426+0000","last_became_peered":"2024-04-24T06:32:07.044426+0000","last_unstale":"2024-04-24T06:32:08.035039+0000","last_undegraded":"2024-04-24T06:32:08.035039+0000","last_fullsized":"2024-04-24T06:32:08.035039+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:40.500121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079577+0000","last_change":"2024-04-24T06:32:08.077512+0000","last_active":"2024-04-24T06:32:08.079577+0000","last_peered":"2024-04-24T06:32:08.079577+0000","last_clean":"2024-04-24T06:32:08.079577+0000","last_became_active":"2024-04-24T06:32:08.075340+0000","last_became_peered":"2024-04-24T06:32:08.075340+0000","last_unstale":"2024-04-24T06:32:08.079577+0000","last_undegraded":"2024-04-24T06:32:08.079577+0000","last_fullsized":"2024-04-24T06:32:08.079577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:31:57.020680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.042140+0000","last_change":"2024-04-24T06:32:07.037051+0000","last_active":"2024-04-24T06:32:08.042140+0000","last_peered":"2024-04-24T06:32:08.042140+0000","last_clean":"2024-04-24T06:32:08.042140+0000","last_became_active":"2024-04-24T06:32:07.036646+0000","last_became_peered":"2024-04-24T06:32:07.036646+0000","last_unstale":"2024-04-24T06:32:08.042140+0000","last_undegraded":"2024-04-24T06:32:08.042140+0000","last_fullsized":"2024-04-24T06:32:08.042140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:51.653420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077362+0000","last_change":"2024-04-24T06:32:08.065635+0000","last_active":"2024-04-24T06:32:08.077362+0000","last_peered":"2024-04-24T06:32:08.077362+0000","last_clean":"2024-04-24T06:32:08.077362+0000","last_became_active":"2024-04-24T06:32:08.065315+0000","last_became_peered":"2024-04-24T06:32:08.065315+0000","last_unstale":"2024-04-24T06:32:08.077362+0000","last_undegraded":"2024-04-24T06:32:08.077362+0000","last_fullsized":"2024-04-24T06:32:08.077362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:42:27.125485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.034305+0000","last_change":"2024-04-24T06:32:07.041456+0000","last_active":"2024-04-24T06:32:08.034305+0000","last_peered":"2024-04-24T06:32:08.034305+0000","last_clean":"2024-04-24T06:32:08.034305+0000","last_became_active":"2024-04-24T06:32:07.040553+0000","last_became_peered":"2024-04-24T06:32:07.040553+0000","last_unstale":"2024-04-24T06:32:08.034305+0000","last_undegraded":"2024-04-24T06:32:08.034305+0000","last_fullsized":"2024-04-24T06:32:08.034305+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:12.424776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082822+0000","last_change":"2024-04-24T06:32:08.078440+0000","last_active":"2024-04-24T06:32:08.082822+0000","last_peered":"2024-04-24T06:32:08.082822+0000","last_clean":"2024-04-24T06:32:08.082822+0000","last_became_active":"2024-04-24T06:32:08.078098+0000","last_became_peered":"2024-04-24T06:32:08.078098+0000","last_unstale":"2024-04-24T06:32:08.082822+0000","last_undegraded":"2024-04-24T06:32:08.082822+0000","last_fullsized":"2024-04-24T06:32:08.082822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:04.783260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.060093+0000","last_change":"2024-04-24T06:32:07.040317+0000","last_active":"2024-04-24T06:32:08.060093+0000","last_peered":"2024-04-24T06:32:08.060093+0000","last_clean":"2024-04-24T06:32:08.060093+0000","last_became_active":"2024-04-24T06:32:07.039786+0000","last_became_peered":"2024-04-24T06:32:07.039786+0000","last_unstale":"2024-04-24T06:32:08.060093+0000","last_undegraded":"2024-04-24T06:32:08.060093+0000","last_fullsized":"2024-04-24T06:32:08.060093+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:44:01.621242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075819+0000","last_change":"2024-04-24T06:32:08.062899+0000","last_active":"2024-04-24T06:32:08.075819+0000","last_peered":"2024-04-24T06:32:08.075819+0000","last_clean":"2024-04-24T06:32:08.075819+0000","last_became_active":"2024-04-24T06:32:08.062555+0000","last_became_peered":"2024-04-24T06:32:08.062555+0000","last_unstale":"2024-04-24T06:32:08.075819+0000","last_undegraded":"2024-04-24T06:32:08.075819+0000","last_fullsized":"2024-04-24T06:32:08.075819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:52:37.479080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.058297+0000","last_change":"2024-04-24T06:32:07.051249+0000","last_active":"2024-04-24T06:32:08.058297+0000","last_peered":"2024-04-24T06:32:08.058297+0000","last_clean":"2024-04-24T06:32:08.058297+0000","last_became_active":"2024-04-24T06:32:07.050004+0000","last_became_peered":"2024-04-24T06:32:07.050004+0000","last_unstale":"2024-04-24T06:32:08.058297+0000","last_undegraded":"2024-04-24T06:32:08.058297+0000","last_fullsized":"2024-04-24T06:32:08.058297+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:12.424776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.052128+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:32:08.052128+0000","last_peered":"2024-04-24T06:32:08.052128+0000","last_clean":"2024-04-24T06:32:08.052128+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:32:08.052128+0000","last_undegraded":"2024-04-24T06:32:08.052128+0000","last_fullsized":"2024-04-24T06:32:08.052128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":867758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":284165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":284817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":284221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":867758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:32:12.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:32:12.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.772+0000 7fd1c1c00640 1 Processor -- start 2024-04-24T06:32:12.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.772+0000 7fd1c1c00640 1 -- start start 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1c1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1bc06f7c0 0x7fd1bc0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1c1c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1bc06df00 0x7fd1bc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1c1c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc06ec60 con 0x7fd1bc15b840 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1c1c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd1bc0b46e0 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1c1c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd1bc0b49c0 con 0x7fd1bc06df00 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1b2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd1bc15b840 0x7fd1bc06fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51760/0 (socket says 172.21.15.59:51760) 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1baa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1bc06f7c0 0x7fd1bc0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1b2000640 1 -- 172.21.15.59:0/1757701206 learned_addr learned my addr 172.21.15.59:0/1757701206 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:12.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.774+0000 7fd1bb400640 1 --2- 172.21.15.59:0/1757701206 >> v2:172.21.15.66:3300/0 conn(0x7fd1bc06df00 0x7fd1bc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.775+0000 7fd1baa00640 1 -- 172.21.15.59:0/1757701206 >> v2:172.21.15.66:3300/0 conn(0x7fd1bc06df00 msgr2=0x7fd1bc06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:12.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.775+0000 7fd1baa00640 1 --2- 172.21.15.59:0/1757701206 >> v2:172.21.15.66:3300/0 conn(0x7fd1bc06df00 0x7fd1bc06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.775+0000 7fd1baa00640 1 -- 172.21.15.59:0/1757701206 >> v1:172.21.15.59:6789/0 conn(0x7fd1bc15b840 legacy=0x7fd1bc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.775+0000 7fd1ba000640 1 -- 172.21.15.59:0/1757701206 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851511885 0 0) 0x7fd1bc06ec60 con 0x7fd1bc15b840 2024-04-24T06:32:12.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.775+0000 7fd1baa00640 1 -- 172.21.15.59:0/1757701206 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0b4ca0 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.776+0000 7fd1baa00640 1 --2- 172.21.15.59:0/1757701206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc0b54a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0731b0 tx=0x7fd1ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205e80a1b72477af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1ba000640 1 -- 172.21.15.59:0/1757701206 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac007740 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1ba000640 1 -- 172.21.15.59:0/1757701206 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac0078e0 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1ba000640 1 -- 172.21.15.59:0/1757701206 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac007cb0 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 -- 172.21.15.59:0/1757701206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06f7c0 msgr2=0x7fd1bc0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/1757701206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc0b54a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0731b0 tx=0x7fd1ac02a680 comp rx=0 tx=0).stop 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 -- 172.21.15.59:0/1757701206 shutdown_connections 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/1757701206 >> v2:172.21.15.66:3300/0 conn(0x7fd1bc06df00 0x7fd1bc06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/1757701206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc0b54a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 -- 172.21.15.59:0/1757701206 >> 172.21.15.59:0/1757701206 conn(0x7fd1bc067ea0 msgr2=0x7fd1bc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.777+0000 7fd1c1c00640 1 -- 172.21.15.59:0/1757701206 shutdown_connections 2024-04-24T06:32:12.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.778+0000 7fd1c1c00640 1 -- 172.21.15.59:0/1757701206 wait complete. 2024-04-24T06:32:12.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.779+0000 7fd1c1c00640 1 Processor -- start 2024-04-24T06:32:12.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.779+0000 7fd1c1c00640 1 -- start start 2024-04-24T06:32:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.780+0000 7fd1c1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1c1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1c1c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1bc142a40 0x7fd1bc152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1c1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ce1c0 con 0x7fd1bc06df00 2024-04-24T06:32:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1c1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc06e8d0 con 0x7fd1bc06f7c0 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1c1c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc0ce340 con 0x7fd1bc142a40 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60148/0 (socket says 172.21.15.59:60148) 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 -- 172.21.15.59:0/3373735102 learned_addr learned my addr 172.21.15.59:0/3373735102 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1b2000640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1bb400640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1bc142a40 0x7fd1bc152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 -- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1bc142a40 msgr2=0x7fd1bc152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1bc142a40 0x7fd1bc152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 -- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1bc06f7c0 msgr2=0x7fd1bc151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.781+0000 7fd1baa00640 1 -- 172.21.15.59:0/3373735102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc156610 con 0x7fd1bc06df00 2024-04-24T06:32:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1baa00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac000f80 tx=0x7fd1ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac040070 con 0x7fd1bc06df00 2024-04-24T06:32:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac031c40 con 0x7fd1bc06df00 2024-04-24T06:32:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac009450 con 0x7fd1bc06df00 2024-04-24T06:32:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc1568b0 con 0x7fd1bc06df00 2024-04-24T06:32:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.782+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc156b90 con 0x7fd1bc06df00 2024-04-24T06:32:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.783+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc068a80 con 0x7fd1bc06df00 2024-04-24T06:32:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.785+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1ac039040 con 0x7fd1bc06df00 2024-04-24T06:32:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.786+0000 7fd1b3400640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 0x7fd19806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.786+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd1ac0acf00 con 0x7fd1bc06df00 2024-04-24T06:32:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.786+0000 7fd1b2000640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 0x7fd19806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.787+0000 7fd1b2000640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 0x7fd19806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc15e830 tx=0x7fd1a4007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.790+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1ac07b290 con 0x7fd1bc06df00 2024-04-24T06:32:12.977 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:32:12.975+0000 7f544b800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:32:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:12.992+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1bc14fe30 con 0x7fd19806c8f0 2024-04-24T06:32:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.003+0000 7fd1b3400640 1 -- 172.21.15.59:0/3373735102 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7fd1a4007270 con 0x7fd19806c8f0 2024-04-24T06:32:13.009 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:32:13.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 msgr2=0x7fd19806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 0x7fd19806edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc15e830 tx=0x7fd1a4007c00 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 msgr2=0x7fd1bc153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac000f80 tx=0x7fd1ac004a40 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 shutdown_connections 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd19806c8f0 0x7fd19806edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1bc142a40 0x7fd1bc152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1bc06f7c0 0x7fd1bc151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 --2- 172.21.15.59:0/3373735102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1bc06df00 0x7fd1bc153900 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.010+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 >> 172.21.15.59:0/3373735102 conn(0x7fd1bc067ea0 msgr2=0x7fd1bc0af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.011+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 shutdown_connections 2024-04-24T06:32:13.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.011+0000 7fd1c1c00640 1 -- 172.21.15.59:0/3373735102 wait complete. 2024-04-24T06:32:13.033 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-24T06:32:11.254341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235120,"kb_used_data":6460,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739600,"statfs":{"total":773094113280,"available":772853350400,"internally_reserved":0,"allocated":6615040,"data_stored":3420889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252635"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067015+0000","last_change":"2024-04-24T06:32:08.064363+0000","last_active":"2024-04-24T06:32:08.067015+0000","last_peered":"2024-04-24T06:32:08.067015+0000","last_clean":"2024-04-24T06:32:08.067015+0000","last_became_active":"2024-04-24T06:32:08.064035+0000","last_became_peered":"2024-04-24T06:32:08.064035+0000","last_unstale":"2024-04-24T06:32:08.067015+0000","last_undegraded":"2024-04-24T06:32:08.067015+0000","last_fullsized":"2024-04-24T06:32:08.067015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:31:20.628402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075463+0000","last_change":"2024-04-24T06:32:08.071119+0000","last_active":"2024-04-24T06:32:08.075463+0000","last_peered":"2024-04-24T06:32:08.075463+0000","last_clean":"2024-04-24T06:32:08.075463+0000","last_became_active":"2024-04-24T06:32:08.069986+0000","last_became_peered":"2024-04-24T06:32:08.069986+0000","last_unstale":"2024-04-24T06:32:08.075463+0000","last_undegraded":"2024-04-24T06:32:08.075463+0000","last_fullsized":"2024-04-24T06:32:08.075463+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:55.360567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079541+0000","last_change":"2024-04-24T06:32:08.076131+0000","last_active":"2024-04-24T06:32:08.079541+0000","last_peered":"2024-04-24T06:32:08.079541+0000","last_clean":"2024-04-24T06:32:08.079541+0000","last_became_active":"2024-04-24T06:32:08.075814+0000","last_became_peered":"2024-04-24T06:32:08.075814+0000","last_unstale":"2024-04-24T06:32:08.079541+0000","last_undegraded":"2024-04-24T06:32:08.079541+0000","last_fullsized":"2024-04-24T06:32:08.079541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:45:21.497171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073287+0000","last_change":"2024-04-24T06:32:08.069461+0000","last_active":"2024-04-24T06:32:08.073287+0000","last_peered":"2024-04-24T06:32:08.073287+0000","last_clean":"2024-04-24T06:32:08.073287+0000","last_became_active":"2024-04-24T06:32:08.069145+0000","last_became_peered":"2024-04-24T06:32:08.069145+0000","last_unstale":"2024-04-24T06:32:08.073287+0000","last_undegraded":"2024-04-24T06:32:08.073287+0000","last_fullsized":"2024-04-24T06:32:08.073287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082572+0000","last_change":"2024-04-24T06:32:08.075454+0000","last_active":"2024-04-24T06:32:08.082572+0000","last_peered":"2024-04-24T06:32:08.082572+0000","last_clean":"2024-04-24T06:32:08.082572+0000","last_became_active":"2024-04-24T06:32:08.071601+0000","last_became_peered":"2024-04-24T06:32:08.071601+0000","last_unstale":"2024-04-24T06:32:08.082572+0000","last_undegraded":"2024-04-24T06:32:08.082572+0000","last_fullsized":"2024-04-24T06:32:08.082572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:21.131075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062777+0000","last_change":"2024-04-24T06:32:08.060886+0000","last_active":"2024-04-24T06:32:08.062777+0000","last_peered":"2024-04-24T06:32:08.062777+0000","last_clean":"2024-04-24T06:32:08.062777+0000","last_became_active":"2024-04-24T06:32:08.060585+0000","last_became_peered":"2024-04-24T06:32:08.060585+0000","last_unstale":"2024-04-24T06:32:08.062777+0000","last_undegraded":"2024-04-24T06:32:08.062777+0000","last_fullsized":"2024-04-24T06:32:08.062777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:33.445994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075046+0000","last_change":"2024-04-24T06:32:08.066735+0000","last_active":"2024-04-24T06:32:08.075046+0000","last_peered":"2024-04-24T06:32:08.075046+0000","last_clean":"2024-04-24T06:32:08.075046+0000","last_became_active":"2024-04-24T06:32:08.065965+0000","last_became_peered":"2024-04-24T06:32:08.065965+0000","last_unstale":"2024-04-24T06:32:08.075046+0000","last_undegraded":"2024-04-24T06:32:08.075046+0000","last_fullsized":"2024-04-24T06:32:08.075046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:49.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082742+0000","last_change":"2024-04-24T06:32:08.075677+0000","last_active":"2024-04-24T06:32:08.082742+0000","last_peered":"2024-04-24T06:32:08.082742+0000","last_clean":"2024-04-24T06:32:08.082742+0000","last_became_active":"2024-04-24T06:32:08.073073+0000","last_became_peered":"2024-04-24T06:32:08.073073+0000","last_unstale":"2024-04-24T06:32:08.082742+0000","last_undegraded":"2024-04-24T06:32:08.082742+0000","last_fullsized":"2024-04-24T06:32:08.082742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:56.922190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073562+0000","last_change":"2024-04-24T06:32:08.063622+0000","last_active":"2024-04-24T06:32:08.073562+0000","last_peered":"2024-04-24T06:32:08.073562+0000","last_clean":"2024-04-24T06:32:08.073562+0000","last_became_active":"2024-04-24T06:32:08.061505+0000","last_became_peered":"2024-04-24T06:32:08.061505+0000","last_unstale":"2024-04-24T06:32:08.073562+0000","last_undegraded":"2024-04-24T06:32:08.073562+0000","last_fullsized":"2024-04-24T06:32:08.073562+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:43.299504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073387+0000","last_change":"2024-04-24T06:32:08.065248+0000","last_active":"2024-04-24T06:32:08.073387+0000","last_peered":"2024-04-24T06:32:08.073387+0000","last_clean":"2024-04-24T06:32:08.073387+0000","last_became_active":"2024-04-24T06:32:08.062492+0000","last_became_peered":"2024-04-24T06:32:08.062492+0000","last_unstale":"2024-04-24T06:32:08.073387+0000","last_undegraded":"2024-04-24T06:32:08.073387+0000","last_fullsized":"2024-04-24T06:32:08.073387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:26:06.928050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082319+0000","last_change":"2024-04-24T06:32:08.075699+0000","last_active":"2024-04-24T06:32:08.082319+0000","last_peered":"2024-04-24T06:32:08.082319+0000","last_clean":"2024-04-24T06:32:08.082319+0000","last_became_active":"2024-04-24T06:32:08.074679+0000","last_became_peered":"2024-04-24T06:32:08.074679+0000","last_unstale":"2024-04-24T06:32:08.082319+0000","last_undegraded":"2024-04-24T06:32:08.082319+0000","last_fullsized":"2024-04-24T06:32:08.082319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:07.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075561+0000","last_change":"2024-04-24T06:32:08.070611+0000","last_active":"2024-04-24T06:32:08.075561+0000","last_peered":"2024-04-24T06:32:08.075561+0000","last_clean":"2024-04-24T06:32:08.075561+0000","last_became_active":"2024-04-24T06:32:08.067527+0000","last_became_peered":"2024-04-24T06:32:08.067527+0000","last_unstale":"2024-04-24T06:32:08.075561+0000","last_undegraded":"2024-04-24T06:32:08.075561+0000","last_fullsized":"2024-04-24T06:32:08.075561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081501+0000","last_change":"2024-04-24T06:32:08.078255+0000","last_active":"2024-04-24T06:32:08.081501+0000","last_peered":"2024-04-24T06:32:08.081501+0000","last_clean":"2024-04-24T06:32:08.081501+0000","last_became_active":"2024-04-24T06:32:08.077953+0000","last_became_peered":"2024-04-24T06:32:08.077953+0000","last_unstale":"2024-04-24T06:32:08.081501+0000","last_undegraded":"2024-04-24T06:32:08.081501+0000","last_fullsized":"2024-04-24T06:32:08.081501+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:45.782063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.070944+0000","last_change":"2024-04-24T06:32:08.056139+0000","last_active":"2024-04-24T06:32:08.070944+0000","last_peered":"2024-04-24T06:32:08.070944+0000","last_clean":"2024-04-24T06:32:08.070944+0000","last_became_active":"2024-04-24T06:32:08.055835+0000","last_became_peered":"2024-04-24T06:32:08.055835+0000","last_unstale":"2024-04-24T06:32:08.070944+0000","last_undegraded":"2024-04-24T06:32:08.070944+0000","last_fullsized":"2024-04-24T06:32:08.070944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:58:54.435910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.073592+0000","last_change":"2024-04-24T06:32:08.063986+0000","last_active":"2024-04-24T06:32:08.073592+0000","last_peered":"2024-04-24T06:32:08.073592+0000","last_clean":"2024-04-24T06:32:08.073592+0000","last_became_active":"2024-04-24T06:32:08.063658+0000","last_became_peered":"2024-04-24T06:32:08.063658+0000","last_unstale":"2024-04-24T06:32:08.073592+0000","last_undegraded":"2024-04-24T06:32:08.073592+0000","last_fullsized":"2024-04-24T06:32:08.073592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:08.567405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075050+0000","last_change":"2024-04-24T06:32:08.065832+0000","last_active":"2024-04-24T06:32:08.075050+0000","last_peered":"2024-04-24T06:32:08.075050+0000","last_clean":"2024-04-24T06:32:08.075050+0000","last_became_active":"2024-04-24T06:32:08.065546+0000","last_became_peered":"2024-04-24T06:32:08.065546+0000","last_unstale":"2024-04-24T06:32:08.075050+0000","last_undegraded":"2024-04-24T06:32:08.075050+0000","last_fullsized":"2024-04-24T06:32:08.075050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082409+0000","last_change":"2024-04-24T06:32:08.077202+0000","last_active":"2024-04-24T06:32:08.082409+0000","last_peered":"2024-04-24T06:32:08.082409+0000","last_clean":"2024-04-24T06:32:08.082409+0000","last_became_active":"2024-04-24T06:32:08.076913+0000","last_became_peered":"2024-04-24T06:32:08.076913+0000","last_unstale":"2024-04-24T06:32:08.082409+0000","last_undegraded":"2024-04-24T06:32:08.082409+0000","last_fullsized":"2024-04-24T06:32:08.082409+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:24.208051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.080823+0000","last_change":"2024-04-24T06:32:08.079778+0000","last_active":"2024-04-24T06:32:08.080823+0000","last_peered":"2024-04-24T06:32:08.080823+0000","last_clean":"2024-04-24T06:32:08.080823+0000","last_became_active":"2024-04-24T06:32:08.079029+0000","last_became_peered":"2024-04-24T06:32:08.079029+0000","last_unstale":"2024-04-24T06:32:08.080823+0000","last_undegraded":"2024-04-24T06:32:08.080823+0000","last_fullsized":"2024-04-24T06:32:08.080823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:49.830107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067046+0000","last_change":"2024-04-24T06:32:08.064002+0000","last_active":"2024-04-24T06:32:08.067046+0000","last_peered":"2024-04-24T06:32:08.067046+0000","last_clean":"2024-04-24T06:32:08.067046+0000","last_became_active":"2024-04-24T06:32:08.063708+0000","last_became_peered":"2024-04-24T06:32:08.063708+0000","last_unstale":"2024-04-24T06:32:08.067046+0000","last_undegraded":"2024-04-24T06:32:08.067046+0000","last_fullsized":"2024-04-24T06:32:08.067046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:21.565304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074767+0000","last_change":"2024-04-24T06:32:08.068037+0000","last_active":"2024-04-24T06:32:08.074767+0000","last_peered":"2024-04-24T06:32:08.074767+0000","last_clean":"2024-04-24T06:32:08.074767+0000","last_became_active":"2024-04-24T06:32:08.066334+0000","last_became_peered":"2024-04-24T06:32:08.066334+0000","last_unstale":"2024-04-24T06:32:08.074767+0000","last_undegraded":"2024-04-24T06:32:08.074767+0000","last_fullsized":"2024-04-24T06:32:08.074767+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:58:23.381313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077669+0000","last_change":"2024-04-24T06:32:08.072318+0000","last_active":"2024-04-24T06:32:08.077669+0000","last_peered":"2024-04-24T06:32:08.077669+0000","last_clean":"2024-04-24T06:32:08.077669+0000","last_became_active":"2024-04-24T06:32:08.071998+0000","last_became_peered":"2024-04-24T06:32:08.071998+0000","last_unstale":"2024-04-24T06:32:08.077669+0000","last_undegraded":"2024-04-24T06:32:08.077669+0000","last_fullsized":"2024-04-24T06:32:08.077669+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072943+0000","last_change":"2024-04-24T06:32:08.067399+0000","last_active":"2024-04-24T06:32:08.072943+0000","last_peered":"2024-04-24T06:32:08.072943+0000","last_clean":"2024-04-24T06:32:08.072943+0000","last_became_active":"2024-04-24T06:32:08.066564+0000","last_became_peered":"2024-04-24T06:32:08.066564+0000","last_unstale":"2024-04-24T06:32:08.072943+0000","last_undegraded":"2024-04-24T06:32:08.072943+0000","last_fullsized":"2024-04-24T06:32:08.072943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:11:19.060664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074602+0000","last_change":"2024-04-24T06:32:08.069298+0000","last_active":"2024-04-24T06:32:08.074602+0000","last_peered":"2024-04-24T06:32:08.074602+0000","last_clean":"2024-04-24T06:32:08.074602+0000","last_became_active":"2024-04-24T06:32:08.068567+0000","last_became_peered":"2024-04-24T06:32:08.068567+0000","last_unstale":"2024-04-24T06:32:08.074602+0000","last_undegraded":"2024-04-24T06:32:08.074602+0000","last_fullsized":"2024-04-24T06:32:08.074602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:31.916367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081567+0000","last_change":"2024-04-24T06:32:08.078328+0000","last_active":"2024-04-24T06:32:08.081567+0000","last_peered":"2024-04-24T06:32:08.081567+0000","last_clean":"2024-04-24T06:32:08.081567+0000","last_became_active":"2024-04-24T06:32:08.077912+0000","last_became_peered":"2024-04-24T06:32:08.077912+0000","last_unstale":"2024-04-24T06:32:08.081567+0000","last_undegraded":"2024-04-24T06:32:08.081567+0000","last_fullsized":"2024-04-24T06:32:08.081567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:37.463043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.064229+0000","last_change":"2024-04-24T06:32:08.056278+0000","last_active":"2024-04-24T06:32:08.064229+0000","last_peered":"2024-04-24T06:32:08.064229+0000","last_clean":"2024-04-24T06:32:08.064229+0000","last_became_active":"2024-04-24T06:32:08.055988+0000","last_became_peered":"2024-04-24T06:32:08.055988+0000","last_unstale":"2024-04-24T06:32:08.064229+0000","last_undegraded":"2024-04-24T06:32:08.064229+0000","last_fullsized":"2024-04-24T06:32:08.064229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:10.369782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.069989+0000","last_change":"2024-04-24T06:32:08.066710+0000","last_active":"2024-04-24T06:32:08.069989+0000","last_peered":"2024-04-24T06:32:08.069989+0000","last_clean":"2024-04-24T06:32:08.069989+0000","last_became_active":"2024-04-24T06:32:08.065429+0000","last_became_peered":"2024-04-24T06:32:08.065429+0000","last_unstale":"2024-04-24T06:32:08.069989+0000","last_undegraded":"2024-04-24T06:32:08.069989+0000","last_fullsized":"2024-04-24T06:32:08.069989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:49.769247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078497+0000","last_change":"2024-04-24T06:32:08.071344+0000","last_active":"2024-04-24T06:32:08.078497+0000","last_peered":"2024-04-24T06:32:08.078497+0000","last_clean":"2024-04-24T06:32:08.078497+0000","last_became_active":"2024-04-24T06:32:08.071034+0000","last_became_peered":"2024-04-24T06:32:08.071034+0000","last_unstale":"2024-04-24T06:32:08.078497+0000","last_undegraded":"2024-04-24T06:32:08.078497+0000","last_fullsized":"2024-04-24T06:32:08.078497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:21.568721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062194+0000","last_change":"2024-04-24T06:32:08.059701+0000","last_active":"2024-04-24T06:32:08.062194+0000","last_peered":"2024-04-24T06:32:08.062194+0000","last_clean":"2024-04-24T06:32:08.062194+0000","last_became_active":"2024-04-24T06:32:08.059407+0000","last_became_peered":"2024-04-24T06:32:08.059407+0000","last_unstale":"2024-04-24T06:32:08.062194+0000","last_undegraded":"2024-04-24T06:32:08.062194+0000","last_fullsized":"2024-04-24T06:32:08.062194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:54:25.098465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081432+0000","last_change":"2024-04-24T06:32:08.072926+0000","last_active":"2024-04-24T06:32:08.081432+0000","last_peered":"2024-04-24T06:32:08.081432+0000","last_clean":"2024-04-24T06:32:08.081432+0000","last_became_active":"2024-04-24T06:32:08.072622+0000","last_became_peered":"2024-04-24T06:32:08.072622+0000","last_unstale":"2024-04-24T06:32:08.081432+0000","last_undegraded":"2024-04-24T06:32:08.081432+0000","last_fullsized":"2024-04-24T06:32:08.081432+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:40:45.744599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078438+0000","last_change":"2024-04-24T06:32:08.074611+0000","last_active":"2024-04-24T06:32:08.078438+0000","last_peered":"2024-04-24T06:32:08.078438+0000","last_clean":"2024-04-24T06:32:08.078438+0000","last_became_active":"2024-04-24T06:32:08.074296+0000","last_became_peered":"2024-04-24T06:32:08.074296+0000","last_unstale":"2024-04-24T06:32:08.078438+0000","last_undegraded":"2024-04-24T06:32:08.078438+0000","last_fullsized":"2024-04-24T06:32:08.078438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:51:21.568721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082663+0000","last_change":"2024-04-24T06:32:08.077143+0000","last_active":"2024-04-24T06:32:08.082663+0000","last_peered":"2024-04-24T06:32:08.082663+0000","last_clean":"2024-04-24T06:32:08.082663+0000","last_became_active":"2024-04-24T06:32:08.072152+0000","last_became_peered":"2024-04-24T06:32:08.072152+0000","last_unstale":"2024-04-24T06:32:08.082663+0000","last_undegraded":"2024-04-24T06:32:08.082663+0000","last_fullsized":"2024-04-24T06:32:08.082663+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:10:29.128899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.084149+0000","last_change":"2024-04-24T06:32:08.080965+0000","last_active":"2024-04-24T06:32:08.084149+0000","last_peered":"2024-04-24T06:32:08.084149+0000","last_clean":"2024-04-24T06:32:08.084149+0000","last_became_active":"2024-04-24T06:32:08.080404+0000","last_became_peered":"2024-04-24T06:32:08.080404+0000","last_unstale":"2024-04-24T06:32:08.084149+0000","last_undegraded":"2024-04-24T06:32:08.084149+0000","last_fullsized":"2024-04-24T06:32:08.084149+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:36.894775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081251+0000","last_change":"2024-04-24T06:32:08.068034+0000","last_active":"2024-04-24T06:32:08.081251+0000","last_peered":"2024-04-24T06:32:08.081251+0000","last_clean":"2024-04-24T06:32:08.081251+0000","last_became_active":"2024-04-24T06:32:08.067742+0000","last_became_peered":"2024-04-24T06:32:08.067742+0000","last_unstale":"2024-04-24T06:32:08.081251+0000","last_undegraded":"2024-04-24T06:32:08.081251+0000","last_fullsized":"2024-04-24T06:32:08.081251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095763+0000","last_change":"2024-04-24T06:32:07.050936+0000","last_active":"2024-04-24T06:32:08.095763+0000","last_peered":"2024-04-24T06:32:08.095763+0000","last_clean":"2024-04-24T06:32:08.095763+0000","last_became_active":"2024-04-24T06:32:07.050670+0000","last_became_peered":"2024-04-24T06:32:07.050670+0000","last_unstale":"2024-04-24T06:32:08.095763+0000","last_undegraded":"2024-04-24T06:32:08.095763+0000","last_fullsized":"2024-04-24T06:32:08.095763+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:44.448627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.070145+0000","last_change":"2024-04-24T06:32:08.063948+0000","last_active":"2024-04-24T06:32:08.070145+0000","last_peered":"2024-04-24T06:32:08.070145+0000","last_clean":"2024-04-24T06:32:08.070145+0000","last_became_active":"2024-04-24T06:32:08.063673+0000","last_became_peered":"2024-04-24T06:32:08.063673+0000","last_unstale":"2024-04-24T06:32:08.070145+0000","last_undegraded":"2024-04-24T06:32:08.070145+0000","last_fullsized":"2024-04-24T06:32:08.070145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:38:27.189569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.094887+0000","last_change":"2024-04-24T06:32:07.044432+0000","last_active":"2024-04-24T06:32:08.094887+0000","last_peered":"2024-04-24T06:32:08.094887+0000","last_clean":"2024-04-24T06:32:08.094887+0000","last_became_active":"2024-04-24T06:32:07.044014+0000","last_became_peered":"2024-04-24T06:32:07.044014+0000","last_unstale":"2024-04-24T06:32:08.094887+0000","last_undegraded":"2024-04-24T06:32:08.094887+0000","last_fullsized":"2024-04-24T06:32:08.094887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.074198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075678+0000","last_change":"2024-04-24T06:32:08.066000+0000","last_active":"2024-04-24T06:32:08.075678+0000","last_peered":"2024-04-24T06:32:08.075678+0000","last_clean":"2024-04-24T06:32:08.075678+0000","last_became_active":"2024-04-24T06:32:08.064744+0000","last_became_peered":"2024-04-24T06:32:08.064744+0000","last_unstale":"2024-04-24T06:32:08.075678+0000","last_undegraded":"2024-04-24T06:32:08.075678+0000","last_fullsized":"2024-04-24T06:32:08.075678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095052+0000","last_change":"2024-04-24T06:32:07.046255+0000","last_active":"2024-04-24T06:32:08.095052+0000","last_peered":"2024-04-24T06:32:08.095052+0000","last_clean":"2024-04-24T06:32:08.095052+0000","last_became_active":"2024-04-24T06:32:07.045977+0000","last_became_peered":"2024-04-24T06:32:07.045977+0000","last_unstale":"2024-04-24T06:32:08.095052+0000","last_undegraded":"2024-04-24T06:32:08.095052+0000","last_fullsized":"2024-04-24T06:32:08.095052+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:28.348442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078459+0000","last_change":"2024-04-24T06:32:08.067616+0000","last_active":"2024-04-24T06:32:08.078459+0000","last_peered":"2024-04-24T06:32:08.078459+0000","last_clean":"2024-04-24T06:32:08.078459+0000","last_became_active":"2024-04-24T06:32:08.067338+0000","last_became_peered":"2024-04-24T06:32:08.067338+0000","last_unstale":"2024-04-24T06:32:08.078459+0000","last_undegraded":"2024-04-24T06:32:08.078459+0000","last_fullsized":"2024-04-24T06:32:08.078459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092638+0000","last_change":"2024-04-24T06:32:07.044352+0000","last_active":"2024-04-24T06:32:08.092638+0000","last_peered":"2024-04-24T06:32:08.092638+0000","last_clean":"2024-04-24T06:32:08.092638+0000","last_became_active":"2024-04-24T06:32:07.043680+0000","last_became_peered":"2024-04-24T06:32:07.043680+0000","last_unstale":"2024-04-24T06:32:08.092638+0000","last_undegraded":"2024-04-24T06:32:08.092638+0000","last_fullsized":"2024-04-24T06:32:08.092638+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:44.448627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072918+0000","last_change":"2024-04-24T06:32:08.067822+0000","last_active":"2024-04-24T06:32:08.072918+0000","last_peered":"2024-04-24T06:32:08.072918+0000","last_clean":"2024-04-24T06:32:08.072918+0000","last_became_active":"2024-04-24T06:32:08.067251+0000","last_became_peered":"2024-04-24T06:32:08.067251+0000","last_unstale":"2024-04-24T06:32:08.072918+0000","last_undegraded":"2024-04-24T06:32:08.072918+0000","last_fullsized":"2024-04-24T06:32:08.072918+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:29.108481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.089442+0000","last_change":"2024-04-24T06:32:07.026561+0000","last_active":"2024-04-24T06:32:08.089442+0000","last_peered":"2024-04-24T06:32:08.089442+0000","last_clean":"2024-04-24T06:32:08.089442+0000","last_became_active":"2024-04-24T06:32:07.023320+0000","last_became_peered":"2024-04-24T06:32:07.023320+0000","last_unstale":"2024-04-24T06:32:08.089442+0000","last_undegraded":"2024-04-24T06:32:08.089442+0000","last_fullsized":"2024-04-24T06:32:08.089442+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:35.322600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082909+0000","last_change":"2024-04-24T06:32:08.078652+0000","last_active":"2024-04-24T06:32:08.082909+0000","last_peered":"2024-04-24T06:32:08.082909+0000","last_clean":"2024-04-24T06:32:08.082909+0000","last_became_active":"2024-04-24T06:32:08.078126+0000","last_became_peered":"2024-04-24T06:32:08.078126+0000","last_unstale":"2024-04-24T06:32:08.082909+0000","last_undegraded":"2024-04-24T06:32:08.082909+0000","last_fullsized":"2024-04-24T06:32:08.082909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:54:39.691836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.090395+0000","last_change":"2024-04-24T06:32:07.049980+0000","last_active":"2024-04-24T06:32:08.090395+0000","last_peered":"2024-04-24T06:32:08.090395+0000","last_clean":"2024-04-24T06:32:08.090395+0000","last_became_active":"2024-04-24T06:32:07.049563+0000","last_became_peered":"2024-04-24T06:32:07.049563+0000","last_unstale":"2024-04-24T06:32:08.090395+0000","last_undegraded":"2024-04-24T06:32:08.090395+0000","last_fullsized":"2024-04-24T06:32:08.090395+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:07.552980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072528+0000","last_change":"2024-04-24T06:32:08.065091+0000","last_active":"2024-04-24T06:32:08.072528+0000","last_peered":"2024-04-24T06:32:08.072528+0000","last_clean":"2024-04-24T06:32:08.072528+0000","last_became_active":"2024-04-24T06:32:08.064747+0000","last_became_peered":"2024-04-24T06:32:08.064747+0000","last_unstale":"2024-04-24T06:32:08.072528+0000","last_undegraded":"2024-04-24T06:32:08.072528+0000","last_fullsized":"2024-04-24T06:32:08.072528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:11.318107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.050288+0000","last_change":"2024-04-24T06:32:07.048665+0000","last_active":"2024-04-24T06:32:08.050288+0000","last_peered":"2024-04-24T06:32:08.050288+0000","last_clean":"2024-04-24T06:32:08.050288+0000","last_became_active":"2024-04-24T06:32:07.047973+0000","last_became_peered":"2024-04-24T06:32:07.047973+0000","last_unstale":"2024-04-24T06:32:08.050288+0000","last_undegraded":"2024-04-24T06:32:08.050288+0000","last_fullsized":"2024-04-24T06:32:08.050288+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:07.552980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.076577+0000","last_change":"2024-04-24T06:32:08.072036+0000","last_active":"2024-04-24T06:32:08.076577+0000","last_peered":"2024-04-24T06:32:08.076577+0000","last_clean":"2024-04-24T06:32:08.076577+0000","last_became_active":"2024-04-24T06:32:08.071660+0000","last_became_peered":"2024-04-24T06:32:08.071660+0000","last_unstale":"2024-04-24T06:32:08.076577+0000","last_undegraded":"2024-04-24T06:32:08.076577+0000","last_fullsized":"2024-04-24T06:32:08.076577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:30.676306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.041363+0000","last_change":"2024-04-24T06:32:07.041803+0000","last_active":"2024-04-24T06:32:08.041363+0000","last_peered":"2024-04-24T06:32:08.041363+0000","last_clean":"2024-04-24T06:32:08.041363+0000","last_became_active":"2024-04-24T06:32:07.041441+0000","last_became_peered":"2024-04-24T06:32:07.041441+0000","last_unstale":"2024-04-24T06:32:08.041363+0000","last_undegraded":"2024-04-24T06:32:08.041363+0000","last_fullsized":"2024-04-24T06:32:08.041363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:25.286075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.071870+0000","last_change":"2024-04-24T06:32:08.060153+0000","last_active":"2024-04-24T06:32:08.071870+0000","last_peered":"2024-04-24T06:32:08.071870+0000","last_clean":"2024-04-24T06:32:08.071870+0000","last_became_active":"2024-04-24T06:32:08.059375+0000","last_became_peered":"2024-04-24T06:32:08.059375+0000","last_unstale":"2024-04-24T06:32:08.071870+0000","last_undegraded":"2024-04-24T06:32:08.071870+0000","last_fullsized":"2024-04-24T06:32:08.071870+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:07.233569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.053839+0000","last_change":"2024-04-24T06:32:07.043903+0000","last_active":"2024-04-24T06:32:08.053839+0000","last_peered":"2024-04-24T06:32:08.053839+0000","last_clean":"2024-04-24T06:32:08.053839+0000","last_became_active":"2024-04-24T06:32:07.043569+0000","last_became_peered":"2024-04-24T06:32:07.043569+0000","last_unstale":"2024-04-24T06:32:08.053839+0000","last_undegraded":"2024-04-24T06:32:08.053839+0000","last_fullsized":"2024-04-24T06:32:08.053839+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:48.754822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075130+0000","last_change":"2024-04-24T06:32:08.070807+0000","last_active":"2024-04-24T06:32:08.075130+0000","last_peered":"2024-04-24T06:32:08.075130+0000","last_clean":"2024-04-24T06:32:08.075130+0000","last_became_active":"2024-04-24T06:32:08.069637+0000","last_became_peered":"2024-04-24T06:32:08.069637+0000","last_unstale":"2024-04-24T06:32:08.075130+0000","last_undegraded":"2024-04-24T06:32:08.075130+0000","last_fullsized":"2024-04-24T06:32:08.075130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.064614+0000","last_change":"2024-04-24T06:32:07.043619+0000","last_active":"2024-04-24T06:32:08.064614+0000","last_peered":"2024-04-24T06:32:08.064614+0000","last_clean":"2024-04-24T06:32:08.064614+0000","last_became_active":"2024-04-24T06:32:07.043332+0000","last_became_peered":"2024-04-24T06:32:07.043332+0000","last_unstale":"2024-04-24T06:32:08.064614+0000","last_undegraded":"2024-04-24T06:32:08.064614+0000","last_fullsized":"2024-04-24T06:32:08.064614+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:11:40.657747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067726+0000","last_change":"2024-04-24T06:32:08.057500+0000","last_active":"2024-04-24T06:32:08.067726+0000","last_peered":"2024-04-24T06:32:08.067726+0000","last_clean":"2024-04-24T06:32:08.067726+0000","last_became_active":"2024-04-24T06:32:08.057153+0000","last_became_peered":"2024-04-24T06:32:08.057153+0000","last_unstale":"2024-04-24T06:32:08.067726+0000","last_undegraded":"2024-04-24T06:32:08.067726+0000","last_fullsized":"2024-04-24T06:32:08.067726+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:37.463043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092950+0000","last_change":"2024-04-24T06:32:07.046720+0000","last_active":"2024-04-24T06:32:08.092950+0000","last_peered":"2024-04-24T06:32:08.092950+0000","last_clean":"2024-04-24T06:32:08.092950+0000","last_became_active":"2024-04-24T06:32:07.046325+0000","last_became_peered":"2024-04-24T06:32:07.046325+0000","last_unstale":"2024-04-24T06:32:08.092950+0000","last_undegraded":"2024-04-24T06:32:08.092950+0000","last_fullsized":"2024-04-24T06:32:08.092950+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:52.305363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079508+0000","last_change":"2024-04-24T06:32:08.075794+0000","last_active":"2024-04-24T06:32:08.079508+0000","last_peered":"2024-04-24T06:32:08.079508+0000","last_clean":"2024-04-24T06:32:08.079508+0000","last_became_active":"2024-04-24T06:32:08.072340+0000","last_became_peered":"2024-04-24T06:32:08.072340+0000","last_unstale":"2024-04-24T06:32:08.079508+0000","last_undegraded":"2024-04-24T06:32:08.079508+0000","last_fullsized":"2024-04-24T06:32:08.079508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:45:20.717382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.097394+0000","last_change":"2024-04-24T06:32:07.041461+0000","last_active":"2024-04-24T06:32:08.097394+0000","last_peered":"2024-04-24T06:32:08.097394+0000","last_clean":"2024-04-24T06:32:08.097394+0000","last_became_active":"2024-04-24T06:32:07.040792+0000","last_became_peered":"2024-04-24T06:32:07.040792+0000","last_unstale":"2024-04-24T06:32:08.097394+0000","last_undegraded":"2024-04-24T06:32:08.097394+0000","last_fullsized":"2024-04-24T06:32:08.097394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:52:28.656256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077082+0000","last_change":"2024-04-24T06:32:08.073385+0000","last_active":"2024-04-24T06:32:08.077082+0000","last_peered":"2024-04-24T06:32:08.077082+0000","last_clean":"2024-04-24T06:32:08.077082+0000","last_became_active":"2024-04-24T06:32:08.073058+0000","last_became_peered":"2024-04-24T06:32:08.073058+0000","last_unstale":"2024-04-24T06:32:08.077082+0000","last_undegraded":"2024-04-24T06:32:08.077082+0000","last_fullsized":"2024-04-24T06:32:08.077082+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.089711+0000","last_change":"2024-04-24T06:32:07.046467+0000","last_active":"2024-04-24T06:32:08.089711+0000","last_peered":"2024-04-24T06:32:08.089711+0000","last_clean":"2024-04-24T06:32:08.089711+0000","last_became_active":"2024-04-24T06:32:07.046172+0000","last_became_peered":"2024-04-24T06:32:07.046172+0000","last_unstale":"2024-04-24T06:32:08.089711+0000","last_undegraded":"2024-04-24T06:32:08.089711+0000","last_fullsized":"2024-04-24T06:32:08.089711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:07:11.320302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077693+0000","last_change":"2024-04-24T06:32:08.074013+0000","last_active":"2024-04-24T06:32:08.077693+0000","last_peered":"2024-04-24T06:32:08.077693+0000","last_clean":"2024-04-24T06:32:08.077693+0000","last_became_active":"2024-04-24T06:32:08.071567+0000","last_became_peered":"2024-04-24T06:32:08.071567+0000","last_unstale":"2024-04-24T06:32:08.077693+0000","last_undegraded":"2024-04-24T06:32:08.077693+0000","last_fullsized":"2024-04-24T06:32:08.077693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.059847+0000","last_change":"2024-04-24T06:32:07.047479+0000","last_active":"2024-04-24T06:32:08.059847+0000","last_peered":"2024-04-24T06:32:08.059847+0000","last_clean":"2024-04-24T06:32:08.059847+0000","last_became_active":"2024-04-24T06:32:07.046228+0000","last_became_peered":"2024-04-24T06:32:07.046228+0000","last_unstale":"2024-04-24T06:32:08.059847+0000","last_undegraded":"2024-04-24T06:32:08.059847+0000","last_fullsized":"2024-04-24T06:32:08.059847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:40.500121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075302+0000","last_change":"2024-04-24T06:32:08.066452+0000","last_active":"2024-04-24T06:32:08.075302+0000","last_peered":"2024-04-24T06:32:08.075302+0000","last_clean":"2024-04-24T06:32:08.075302+0000","last_became_active":"2024-04-24T06:32:08.062268+0000","last_became_peered":"2024-04-24T06:32:08.062268+0000","last_unstale":"2024-04-24T06:32:08.075302+0000","last_undegraded":"2024-04-24T06:32:08.075302+0000","last_fullsized":"2024-04-24T06:32:08.075302+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:29.362867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.050766+0000","last_change":"2024-04-24T06:32:07.043629+0000","last_active":"2024-04-24T06:32:08.050766+0000","last_peered":"2024-04-24T06:32:08.050766+0000","last_clean":"2024-04-24T06:32:08.050766+0000","last_became_active":"2024-04-24T06:32:07.043326+0000","last_became_peered":"2024-04-24T06:32:07.043326+0000","last_unstale":"2024-04-24T06:32:08.050766+0000","last_undegraded":"2024-04-24T06:32:08.050766+0000","last_fullsized":"2024-04-24T06:32:08.050766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:14:28.348442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.074635+0000","last_change":"2024-04-24T06:32:08.069595+0000","last_active":"2024-04-24T06:32:08.074635+0000","last_peered":"2024-04-24T06:32:08.074635+0000","last_clean":"2024-04-24T06:32:08.074635+0000","last_became_active":"2024-04-24T06:32:08.068971+0000","last_became_peered":"2024-04-24T06:32:08.068971+0000","last_unstale":"2024-04-24T06:32:08.074635+0000","last_undegraded":"2024-04-24T06:32:08.074635+0000","last_fullsized":"2024-04-24T06:32:08.074635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:56.208867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.093313+0000","last_change":"2024-04-24T06:32:07.051220+0000","last_active":"2024-04-24T06:32:08.093313+0000","last_peered":"2024-04-24T06:32:08.093313+0000","last_clean":"2024-04-24T06:32:08.093313+0000","last_became_active":"2024-04-24T06:32:07.050841+0000","last_became_peered":"2024-04-24T06:32:07.050841+0000","last_unstale":"2024-04-24T06:32:08.093313+0000","last_undegraded":"2024-04-24T06:32:08.093313+0000","last_fullsized":"2024-04-24T06:32:08.093313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:32.431569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079692+0000","last_change":"2024-04-24T06:32:08.077705+0000","last_active":"2024-04-24T06:32:08.079692+0000","last_peered":"2024-04-24T06:32:08.079692+0000","last_clean":"2024-04-24T06:32:08.079692+0000","last_became_active":"2024-04-24T06:32:08.077336+0000","last_became_peered":"2024-04-24T06:32:08.077336+0000","last_unstale":"2024-04-24T06:32:08.079692+0000","last_undegraded":"2024-04-24T06:32:08.079692+0000","last_fullsized":"2024-04-24T06:32:08.079692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:21.543323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092158+0000","last_change":"2024-04-24T06:32:07.047276+0000","last_active":"2024-04-24T06:32:08.092158+0000","last_peered":"2024-04-24T06:32:08.092158+0000","last_clean":"2024-04-24T06:32:08.092158+0000","last_became_active":"2024-04-24T06:32:07.046802+0000","last_became_peered":"2024-04-24T06:32:07.046802+0000","last_unstale":"2024-04-24T06:32:08.092158+0000","last_undegraded":"2024-04-24T06:32:08.092158+0000","last_fullsized":"2024-04-24T06:32:08.092158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:50:55.375117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062397+0000","last_change":"2024-04-24T06:32:08.059281+0000","last_active":"2024-04-24T06:32:08.062397+0000","last_peered":"2024-04-24T06:32:08.062397+0000","last_clean":"2024-04-24T06:32:08.062397+0000","last_became_active":"2024-04-24T06:32:08.058948+0000","last_became_peered":"2024-04-24T06:32:08.058948+0000","last_unstale":"2024-04-24T06:32:08.062397+0000","last_undegraded":"2024-04-24T06:32:08.062397+0000","last_fullsized":"2024-04-24T06:32:08.062397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:49.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.095269+0000","last_change":"2024-04-24T06:32:07.039487+0000","last_active":"2024-04-24T06:32:08.095269+0000","last_peered":"2024-04-24T06:32:08.095269+0000","last_clean":"2024-04-24T06:32:08.095269+0000","last_became_active":"2024-04-24T06:32:07.039162+0000","last_became_peered":"2024-04-24T06:32:07.039162+0000","last_unstale":"2024-04-24T06:32:08.095269+0000","last_undegraded":"2024-04-24T06:32:08.095269+0000","last_fullsized":"2024-04-24T06:32:08.095269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:48.961434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.081951+0000","last_change":"2024-04-24T06:32:08.074884+0000","last_active":"2024-04-24T06:32:08.081951+0000","last_peered":"2024-04-24T06:32:08.081951+0000","last_clean":"2024-04-24T06:32:08.081951+0000","last_became_active":"2024-04-24T06:32:08.074584+0000","last_became_peered":"2024-04-24T06:32:08.074584+0000","last_unstale":"2024-04-24T06:32:08.081951+0000","last_undegraded":"2024-04-24T06:32:08.081951+0000","last_fullsized":"2024-04-24T06:32:08.081951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:38.336846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.092969+0000","last_change":"2024-04-24T06:32:07.044126+0000","last_active":"2024-04-24T06:32:08.092969+0000","last_peered":"2024-04-24T06:32:08.092969+0000","last_clean":"2024-04-24T06:32:08.092969+0000","last_became_active":"2024-04-24T06:32:07.043243+0000","last_became_peered":"2024-04-24T06:32:07.043243+0000","last_unstale":"2024-04-24T06:32:08.092969+0000","last_undegraded":"2024-04-24T06:32:08.092969+0000","last_fullsized":"2024-04-24T06:32:08.092969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:16:32.231040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.078566+0000","last_change":"2024-04-24T06:32:08.075286+0000","last_active":"2024-04-24T06:32:08.078566+0000","last_peered":"2024-04-24T06:32:08.078566+0000","last_clean":"2024-04-24T06:32:08.078566+0000","last_became_active":"2024-04-24T06:32:08.072461+0000","last_became_peered":"2024-04-24T06:32:08.072461+0000","last_unstale":"2024-04-24T06:32:08.078566+0000","last_undegraded":"2024-04-24T06:32:08.078566+0000","last_fullsized":"2024-04-24T06:32:08.078566+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:21:36.894775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.044988+0000","last_change":"2024-04-24T06:32:07.045408+0000","last_active":"2024-04-24T06:32:08.044988+0000","last_peered":"2024-04-24T06:32:08.044988+0000","last_clean":"2024-04-24T06:32:08.044988+0000","last_became_active":"2024-04-24T06:32:07.043456+0000","last_became_peered":"2024-04-24T06:32:07.043456+0000","last_unstale":"2024-04-24T06:32:08.044988+0000","last_undegraded":"2024-04-24T06:32:08.044988+0000","last_fullsized":"2024-04-24T06:32:08.044988+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:42.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077060+0000","last_change":"2024-04-24T06:32:08.073233+0000","last_active":"2024-04-24T06:32:08.077060+0000","last_peered":"2024-04-24T06:32:08.077060+0000","last_clean":"2024-04-24T06:32:08.077060+0000","last_became_active":"2024-04-24T06:32:08.072922+0000","last_became_peered":"2024-04-24T06:32:08.072922+0000","last_unstale":"2024-04-24T06:32:08.077060+0000","last_undegraded":"2024-04-24T06:32:08.077060+0000","last_fullsized":"2024-04-24T06:32:08.077060+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:15:06.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.093509+0000","last_change":"2024-04-24T06:32:07.042679+0000","last_active":"2024-04-24T06:32:08.093509+0000","last_peered":"2024-04-24T06:32:08.093509+0000","last_clean":"2024-04-24T06:32:08.093509+0000","last_became_active":"2024-04-24T06:32:07.042050+0000","last_became_peered":"2024-04-24T06:32:07.042050+0000","last_unstale":"2024-04-24T06:32:08.093509+0000","last_undegraded":"2024-04-24T06:32:08.093509+0000","last_fullsized":"2024-04-24T06:32:08.093509+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:54:22.132919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077168+0000","last_change":"2024-04-24T06:32:08.068917+0000","last_active":"2024-04-24T06:32:08.077168+0000","last_peered":"2024-04-24T06:32:08.077168+0000","last_clean":"2024-04-24T06:32:08.077168+0000","last_became_active":"2024-04-24T06:32:08.068648+0000","last_became_peered":"2024-04-24T06:32:08.068648+0000","last_unstale":"2024-04-24T06:32:08.077168+0000","last_undegraded":"2024-04-24T06:32:08.077168+0000","last_fullsized":"2024-04-24T06:32:08.077168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:39:00.527373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.034337+0000","last_change":"2024-04-24T06:32:07.036954+0000","last_active":"2024-04-24T06:32:08.034337+0000","last_peered":"2024-04-24T06:32:08.034337+0000","last_clean":"2024-04-24T06:32:08.034337+0000","last_became_active":"2024-04-24T06:32:07.036666+0000","last_became_peered":"2024-04-24T06:32:07.036666+0000","last_unstale":"2024-04-24T06:32:08.034337+0000","last_undegraded":"2024-04-24T06:32:08.034337+0000","last_fullsized":"2024-04-24T06:32:08.034337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:21:55.451625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075037+0000","last_change":"2024-04-24T06:32:08.063385+0000","last_active":"2024-04-24T06:32:08.075037+0000","last_peered":"2024-04-24T06:32:08.075037+0000","last_clean":"2024-04-24T06:32:08.075037+0000","last_became_active":"2024-04-24T06:32:08.063058+0000","last_became_peered":"2024-04-24T06:32:08.063058+0000","last_unstale":"2024-04-24T06:32:08.075037+0000","last_undegraded":"2024-04-24T06:32:08.075037+0000","last_fullsized":"2024-04-24T06:32:08.075037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:17:49.769247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.049472+0000","last_change":"2024-04-24T06:32:07.047707+0000","last_active":"2024-04-24T06:32:08.049472+0000","last_peered":"2024-04-24T06:32:08.049472+0000","last_clean":"2024-04-24T06:32:08.049472+0000","last_became_active":"2024-04-24T06:32:07.047392+0000","last_became_peered":"2024-04-24T06:32:07.047392+0000","last_unstale":"2024-04-24T06:32:08.049472+0000","last_undegraded":"2024-04-24T06:32:08.049472+0000","last_fullsized":"2024-04-24T06:32:08.049472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:42.285079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.072108+0000","last_change":"2024-04-24T06:32:08.068433+0000","last_active":"2024-04-24T06:32:08.072108+0000","last_peered":"2024-04-24T06:32:08.072108+0000","last_clean":"2024-04-24T06:32:08.072108+0000","last_became_active":"2024-04-24T06:32:08.067941+0000","last_became_peered":"2024-04-24T06:32:08.067941+0000","last_unstale":"2024-04-24T06:32:08.072108+0000","last_undegraded":"2024-04-24T06:32:08.072108+0000","last_fullsized":"2024-04-24T06:32:08.072108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:53.319788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.091989+0000","last_change":"2024-04-24T06:32:07.044900+0000","last_active":"2024-04-24T06:32:08.091989+0000","last_peered":"2024-04-24T06:32:08.091989+0000","last_clean":"2024-04-24T06:32:08.091989+0000","last_became_active":"2024-04-24T06:32:07.043759+0000","last_became_peered":"2024-04-24T06:32:07.043759+0000","last_unstale":"2024-04-24T06:32:08.091989+0000","last_undegraded":"2024-04-24T06:32:08.091989+0000","last_fullsized":"2024-04-24T06:32:08.091989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:10.303682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.076963+0000","last_change":"2024-04-24T06:32:08.073201+0000","last_active":"2024-04-24T06:32:08.076963+0000","last_peered":"2024-04-24T06:32:08.076963+0000","last_clean":"2024-04-24T06:32:08.076963+0000","last_became_active":"2024-04-24T06:32:08.072794+0000","last_became_peered":"2024-04-24T06:32:08.072794+0000","last_unstale":"2024-04-24T06:32:08.076963+0000","last_undegraded":"2024-04-24T06:32:08.076963+0000","last_fullsized":"2024-04-24T06:32:08.076963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:36:42.786258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.049303+0000","last_change":"2024-04-24T06:32:07.049288+0000","last_active":"2024-04-24T06:32:08.049303+0000","last_peered":"2024-04-24T06:32:08.049303+0000","last_clean":"2024-04-24T06:32:08.049303+0000","last_became_active":"2024-04-24T06:32:07.049006+0000","last_became_peered":"2024-04-24T06:32:07.049006+0000","last_unstale":"2024-04-24T06:32:08.049303+0000","last_undegraded":"2024-04-24T06:32:08.049303+0000","last_fullsized":"2024-04-24T06:32:08.049303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:16:32.231040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.067702+0000","last_change":"2024-04-24T06:32:08.056596+0000","last_active":"2024-04-24T06:32:08.067702+0000","last_peered":"2024-04-24T06:32:08.067702+0000","last_clean":"2024-04-24T06:32:08.067702+0000","last_became_active":"2024-04-24T06:32:08.056294+0000","last_became_peered":"2024-04-24T06:32:08.056294+0000","last_unstale":"2024-04-24T06:32:08.067702+0000","last_undegraded":"2024-04-24T06:32:08.067702+0000","last_fullsized":"2024-04-24T06:32:08.067702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:45.782063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.059202+0000","last_change":"2024-04-24T06:32:07.049043+0000","last_active":"2024-04-24T06:32:08.059202+0000","last_peered":"2024-04-24T06:32:08.059202+0000","last_clean":"2024-04-24T06:32:08.059202+0000","last_became_active":"2024-04-24T06:32:07.048730+0000","last_became_peered":"2024-04-24T06:32:07.048730+0000","last_unstale":"2024-04-24T06:32:08.059202+0000","last_undegraded":"2024-04-24T06:32:08.059202+0000","last_fullsized":"2024-04-24T06:32:08.059202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:21:55.451625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075170+0000","last_change":"2024-04-24T06:32:08.064445+0000","last_active":"2024-04-24T06:32:08.075170+0000","last_peered":"2024-04-24T06:32:08.075170+0000","last_clean":"2024-04-24T06:32:08.075170+0000","last_became_active":"2024-04-24T06:32:08.064162+0000","last_became_peered":"2024-04-24T06:32:08.064162+0000","last_unstale":"2024-04-24T06:32:08.075170+0000","last_undegraded":"2024-04-24T06:32:08.075170+0000","last_fullsized":"2024-04-24T06:32:08.075170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:53.319788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.062842+0000","last_change":"2024-04-24T06:32:07.048053+0000","last_active":"2024-04-24T06:32:08.062842+0000","last_peered":"2024-04-24T06:32:08.062842+0000","last_clean":"2024-04-24T06:32:08.062842+0000","last_became_active":"2024-04-24T06:32:07.047709+0000","last_became_peered":"2024-04-24T06:32:07.047709+0000","last_unstale":"2024-04-24T06:32:08.062842+0000","last_undegraded":"2024-04-24T06:32:08.062842+0000","last_fullsized":"2024-04-24T06:32:08.062842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:32.431569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.084945+0000","last_change":"2024-04-24T06:32:08.081923+0000","last_active":"2024-04-24T06:32:08.084945+0000","last_peered":"2024-04-24T06:32:08.084945+0000","last_clean":"2024-04-24T06:32:08.084945+0000","last_became_active":"2024-04-24T06:32:08.080515+0000","last_became_peered":"2024-04-24T06:32:08.080515+0000","last_unstale":"2024-04-24T06:32:08.084945+0000","last_undegraded":"2024-04-24T06:32:08.084945+0000","last_fullsized":"2024-04-24T06:32:08.084945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:10.528279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.035039+0000","last_change":"2024-04-24T06:32:07.045434+0000","last_active":"2024-04-24T06:32:08.035039+0000","last_peered":"2024-04-24T06:32:08.035039+0000","last_clean":"2024-04-24T06:32:08.035039+0000","last_became_active":"2024-04-24T06:32:07.044426+0000","last_became_peered":"2024-04-24T06:32:07.044426+0000","last_unstale":"2024-04-24T06:32:08.035039+0000","last_undegraded":"2024-04-24T06:32:08.035039+0000","last_fullsized":"2024-04-24T06:32:08.035039+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:40.500121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.079577+0000","last_change":"2024-04-24T06:32:08.077512+0000","last_active":"2024-04-24T06:32:08.079577+0000","last_peered":"2024-04-24T06:32:08.079577+0000","last_clean":"2024-04-24T06:32:08.079577+0000","last_became_active":"2024-04-24T06:32:08.075340+0000","last_became_peered":"2024-04-24T06:32:08.075340+0000","last_unstale":"2024-04-24T06:32:08.079577+0000","last_undegraded":"2024-04-24T06:32:08.079577+0000","last_fullsized":"2024-04-24T06:32:08.079577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:31:57.020680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.042140+0000","last_change":"2024-04-24T06:32:07.037051+0000","last_active":"2024-04-24T06:32:08.042140+0000","last_peered":"2024-04-24T06:32:08.042140+0000","last_clean":"2024-04-24T06:32:08.042140+0000","last_became_active":"2024-04-24T06:32:07.036646+0000","last_became_peered":"2024-04-24T06:32:07.036646+0000","last_unstale":"2024-04-24T06:32:08.042140+0000","last_undegraded":"2024-04-24T06:32:08.042140+0000","last_fullsized":"2024-04-24T06:32:08.042140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:33:51.653420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.077362+0000","last_change":"2024-04-24T06:32:08.065635+0000","last_active":"2024-04-24T06:32:08.077362+0000","last_peered":"2024-04-24T06:32:08.077362+0000","last_clean":"2024-04-24T06:32:08.077362+0000","last_became_active":"2024-04-24T06:32:08.065315+0000","last_became_peered":"2024-04-24T06:32:08.065315+0000","last_unstale":"2024-04-24T06:32:08.077362+0000","last_undegraded":"2024-04-24T06:32:08.077362+0000","last_fullsized":"2024-04-24T06:32:08.077362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:42:27.125485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.034305+0000","last_change":"2024-04-24T06:32:07.041456+0000","last_active":"2024-04-24T06:32:08.034305+0000","last_peered":"2024-04-24T06:32:08.034305+0000","last_clean":"2024-04-24T06:32:08.034305+0000","last_became_active":"2024-04-24T06:32:07.040553+0000","last_became_peered":"2024-04-24T06:32:07.040553+0000","last_unstale":"2024-04-24T06:32:08.034305+0000","last_undegraded":"2024-04-24T06:32:08.034305+0000","last_fullsized":"2024-04-24T06:32:08.034305+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:12.424776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.082822+0000","last_change":"2024-04-24T06:32:08.078440+0000","last_active":"2024-04-24T06:32:08.082822+0000","last_peered":"2024-04-24T06:32:08.082822+0000","last_clean":"2024-04-24T06:32:08.082822+0000","last_became_active":"2024-04-24T06:32:08.078098+0000","last_became_peered":"2024-04-24T06:32:08.078098+0000","last_unstale":"2024-04-24T06:32:08.082822+0000","last_undegraded":"2024-04-24T06:32:08.082822+0000","last_fullsized":"2024-04-24T06:32:08.082822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:04.783260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.060093+0000","last_change":"2024-04-24T06:32:07.040317+0000","last_active":"2024-04-24T06:32:08.060093+0000","last_peered":"2024-04-24T06:32:08.060093+0000","last_clean":"2024-04-24T06:32:08.060093+0000","last_became_active":"2024-04-24T06:32:07.039786+0000","last_became_peered":"2024-04-24T06:32:07.039786+0000","last_unstale":"2024-04-24T06:32:08.060093+0000","last_undegraded":"2024-04-24T06:32:08.060093+0000","last_fullsized":"2024-04-24T06:32:08.060093+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:44:01.621242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.075819+0000","last_change":"2024-04-24T06:32:08.062899+0000","last_active":"2024-04-24T06:32:08.075819+0000","last_peered":"2024-04-24T06:32:08.075819+0000","last_clean":"2024-04-24T06:32:08.075819+0000","last_became_active":"2024-04-24T06:32:08.062555+0000","last_became_peered":"2024-04-24T06:32:08.062555+0000","last_unstale":"2024-04-24T06:32:08.075819+0000","last_undegraded":"2024-04-24T06:32:08.075819+0000","last_fullsized":"2024-04-24T06:32:08.075819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:06.999580+0000","last_clean_scrub_stamp":"2024-04-24T06:32:06.999580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:52:37.479080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.058297+0000","last_change":"2024-04-24T06:32:07.051249+0000","last_active":"2024-04-24T06:32:08.058297+0000","last_peered":"2024-04-24T06:32:08.058297+0000","last_clean":"2024-04-24T06:32:08.058297+0000","last_became_active":"2024-04-24T06:32:07.050004+0000","last_became_peered":"2024-04-24T06:32:07.050004+0000","last_unstale":"2024-04-24T06:32:08.058297+0000","last_undegraded":"2024-04-24T06:32:08.058297+0000","last_fullsized":"2024-04-24T06:32:08.058297+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:32:05.985155+0000","last_clean_scrub_stamp":"2024-04-24T06:32:05.985155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:12.424776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-24T06:32:08.052128+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:32:08.052128+0000","last_peered":"2024-04-24T06:32:08.052128+0000","last_clean":"2024-04-24T06:32:08.052128+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:32:08.052128+0000","last_undegraded":"2024-04-24T06:32:08.052128+0000","last_fullsized":"2024-04-24T06:32:08.052128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":867758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":284165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":284817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":284221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":277390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":867758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:32:13.037 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:32:13.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:32:13.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.298+0000 7f60b2400640 1 Processor -- start 2024-04-24T06:32:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.299+0000 7f60b2400640 1 -- start start 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.300+0000 7f60b2400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60ac169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.300+0000 7f60b2400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac0cff50 con 0x7f60ac15cb20 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.300+0000 7f60b2400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac0d01c0 con 0x7f60ac163770 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.300+0000 7f60b2400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac15cf00 con 0x7f60ac0d1950 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.301+0000 7f60ab400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f60ac15cb20 0x7f60ac0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51764/0 (socket says 172.21.15.59:51764) 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.301+0000 7f60ab400640 1 -- 172.21.15.59:0/2978958323 learned_addr learned my addr 172.21.15.59:0/2978958323 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.301+0000 7f60abe00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60ac169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.301+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120422508 0 0) 0x7f60ac0cff50 con 0x7f60ac15cb20 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.301+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6090003520 con 0x7f60ac15cb20 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 134754734 0 0) 0x7f6090003520 con 0x7f60ac15cb20 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 msgr2=0x7f60ac169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60ac169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 >> v1:172.21.15.66:6789/0 conn(0x7f60ac163770 legacy=0x7f60ac0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac15f6a0 con 0x7f60ac15cb20 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f609c001690 con 0x7f60ac15cb20 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 >> v1:172.21.15.59:6789/0 conn(0x7f60ac15cb20 legacy=0x7f60ac0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090003fb0 0x7f6090004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6090004d40 0x7f60900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac15cf00 con 0x7f6090004d40 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f609000bbb0 con 0x7f6090003fb0 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f609000bdb0 con 0x7f60ac0d1950 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60abe00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6090004d40 0x7f60900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.302+0000 7f60ab400640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60aaa00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090003fb0 0x7f6090004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60abe00640 1 -- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 msgr2=0x7f60900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60abe00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0d1950 0x7f60900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60abe00640 1 -- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090003fb0 msgr2=0x7f6090004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60abe00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090003fb0 0x7f6090004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.303+0000 7f60abe00640 1 -- 172.21.15.59:0/2978958323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6090003520 con 0x7f6090004d40 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.304+0000 7f60abe00640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6090004d40 0x7f60900051c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009a90 tx=0x7f6090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79828a8eeca4ff38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.304+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4036070 con 0x7f6090004d40 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.304+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4031400 con 0x7f6090004d40 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.304+0000 7f60aa000640 1 -- 172.21.15.59:0/2978958323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4009690 con 0x7f6090004d40 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 -- 172.21.15.59:0/2978958323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6090004d40 msgr2=0x7f60900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 --2- 172.21.15.59:0/2978958323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6090004d40 0x7f60900051c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009a90 tx=0x7f6090005830 comp rx=0 tx=0).stop 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60abe00640 1 -- 172.21.15.59:0/2978958323 reap_dead start 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 -- 172.21.15.59:0/2978958323 shutdown_connections 2024-04-24T06:32:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 -- 172.21.15.59:0/2978958323 >> 172.21.15.59:0/2978958323 conn(0x7f60ac06cc70 msgr2=0x7f60ac06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 -- 172.21.15.59:0/2978958323 shutdown_connections 2024-04-24T06:32:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.305+0000 7f60b2400640 1 -- 172.21.15.59:0/2978958323 wait complete. 2024-04-24T06:32:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.307+0000 7f60b2400640 1 Processor -- start 2024-04-24T06:32:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.307+0000 7f60b2400640 1 -- start start 2024-04-24T06:32:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090004d40 0x7f60ac1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0c8d70 0x7f60ac1f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac160300 con 0x7f60ac161a30 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac160180 con 0x7f6090004d40 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60b2400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac160480 con 0x7f60ac0c8d70 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60156/0 (socket says 172.21.15.59:60156) 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.308+0000 7f60abe00640 1 -- 172.21.15.59:0/406249995 learned_addr learned my addr 172.21.15.59:0/406249995 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 -- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0c8d70 msgr2=0x7f60ac1f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0c8d70 0x7f60ac1f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 -- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090004d40 msgr2=0x7f60ac1f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090004d40 0x7f60ac1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 -- 172.21.15.59:0/406249995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac155af0 con 0x7f60ac161a30 2024-04-24T06:32:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60abe00640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009b80 tx=0x7f60a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4036070 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4041070 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4003d20 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1f8550 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.309+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1f8b10 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.311+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac068a80 con 0x7f60ac161a30 2024-04-24T06:32:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.311+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60a4045410 con 0x7f60ac161a30 2024-04-24T06:32:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.312+0000 7f60a3400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 0x7f608c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.313+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f60a40ab5d0 con 0x7f60ac161a30 2024-04-24T06:32:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.313+0000 7f60ab400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 0x7f608c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.313+0000 7f60ab400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 0x7f608c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f609c001eb0 tx=0x7f609c001dc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.317+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60a404a020 con 0x7f60ac161a30 2024-04-24T06:32:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.596+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f60ac0d3180 con 0x7f60ac161a30 2024-04-24T06:32:13.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.601+0000 7f60a3400640 1 -- 172.21.15.59:0/406249995 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f60a4079960 con 0x7f60ac161a30 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:32:13.603 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:32:13.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.603+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 msgr2=0x7f608c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.603+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 0x7f608c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f609c001eb0 tx=0x7f609c001dc0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.603+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 msgr2=0x7f60ac161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.603+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009b80 tx=0x7f60a4004730 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 shutdown_connections 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f608c06c8f0 0x7f608c06edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60ac161a30 0x7f60ac161eb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60ac0c8d70 0x7f60ac1f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 --2- 172.21.15.59:0/406249995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6090004d40 0x7f60ac1f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 >> 172.21.15.59:0/406249995 conn(0x7f60ac06cc70 msgr2=0x7f60ac06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 shutdown_connections 2024-04-24T06:32:13.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.604+0000 7f60b2400640 1 -- 172.21.15.59:0/406249995 wait complete. 2024-04-24T06:32:13.626 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:32:13.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.833+0000 7f9b32200640 1 Processor -- start 2024-04-24T06:32:13.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.833+0000 7f9b32200640 1 -- start start 2024-04-24T06:32:13.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.834+0000 7f9b32200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.834+0000 7f9b32200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b2c0d2630 0x7f9b2c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.834+0000 7f9b32200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2c0cf030 con 0x7f9b2c0b6530 2024-04-24T06:32:13.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b32200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2c0b7000 con 0x7f9b2c0d2630 2024-04-24T06:32:13.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b32200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0cf620 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b30e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b30e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:34708/0 (socket says 172.21.15.59:34708) 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b30e00640 1 -- 172.21.15.59:0/2175608453 learned_addr learned my addr 172.21.15.59:0/2175608453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.835+0000 7f9b23e00640 1 --2- 172.21.15.59:0/2175608453 >> v2:172.21.15.66:3300/0 conn(0x7f9b2c0d2630 0x7f9b2c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.836+0000 7f9b30e00640 1 -- 172.21.15.59:0/2175608453 >> v2:172.21.15.66:3300/0 conn(0x7f9b2c0d2630 msgr2=0x7f9b2c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.836+0000 7f9b30e00640 1 --2- 172.21.15.59:0/2175608453 >> v2:172.21.15.66:3300/0 conn(0x7f9b2c0d2630 0x7f9b2c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.836+0000 7f9b30e00640 1 -- 172.21.15.59:0/2175608453 >> v1:172.21.15.66:6789/0 conn(0x7f9b2c0b6530 legacy=0x7f9b2c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.836+0000 7f9b30e00640 1 -- 172.21.15.59:0/2175608453 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c0b6a40 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.837+0000 7f9b30e00640 1 --2- 172.21.15.59:0/2175608453 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fb4018947080aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.838+0000 7f9b2be00640 1 -- 172.21.15.59:0/2175608453 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c037070 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.838+0000 7f9b2be00640 1 -- 172.21.15.59:0/2175608453 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c032440 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.838+0000 7f9b2be00640 1 -- 172.21.15.59:0/2175608453 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c032810 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 -- 172.21.15.59:0/2175608453 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 msgr2=0x7f9b2c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 --2- 172.21.15.59:0/2175608453 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c02a270 comp rx=0 tx=0).stop 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 -- 172.21.15.59:0/2175608453 shutdown_connections 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 --2- 172.21.15.59:0/2175608453 >> v2:172.21.15.66:3300/0 conn(0x7f9b2c0d2630 0x7f9b2c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 --2- 172.21.15.59:0/2175608453 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c15e7c0 0x7f9b2c0d20f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 -- 172.21.15.59:0/2175608453 >> 172.21.15.59:0/2175608453 conn(0x7f9b2c09d920 msgr2=0x7f9b2c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 -- 172.21.15.59:0/2175608453 shutdown_connections 2024-04-24T06:32:13.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.839+0000 7f9b32200640 1 -- 172.21.15.59:0/2175608453 wait complete. 2024-04-24T06:32:13.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.841+0000 7f9b32200640 1 Processor -- start 2024-04-24T06:32:13.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.842+0000 7f9b32200640 1 -- start start 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c0b6530 0x7f9b2c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b2c0d2630 0x7f9b2c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d1a80 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0cf2b0 con 0x7f9b2c0d2630 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b32200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d1c00 con 0x7f9b2c0b6530 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.843+0000 7f9b31800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60160/0 (socket says 172.21.15.59:60160) 2024-04-24T06:32:13.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 -- 172.21.15.59:0/3429684917 learned_addr learned my addr 172.21.15.59:0/3429684917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b30e00640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c0b6530 0x7f9b2c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b23e00640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b2c0d2630 0x7f9b2c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 -- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c0b6530 msgr2=0x7f9b2c0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c0b6530 0x7f9b2c0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 -- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b2c0d2630 msgr2=0x7f9b2c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b2c0d2630 0x7f9b2c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 -- 172.21.15.59:0/3429684917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c0ce3b0 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.844+0000 7f9b31800640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c730 tx=0x7f9b2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.845+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24003b70 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.845+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24003d10 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.845+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400aa90 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.845+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c0b2020 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.845+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c0b25e0 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.846+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c04fd90 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.847+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b2400ac30 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.848+0000 7f9b29600640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 0x7f9b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:13.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.848+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9b2408ba30 con 0x7f9b2c15e7c0 2024-04-24T06:32:13.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.849+0000 7f9b30e00640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 0x7f9b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:13.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.849+0000 7f9b30e00640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 0x7f9b0806edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c034000 tx=0x7f9b1c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:13.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:13.853+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b2c0cf030 con 0x7f9b2c15e7c0 2024-04-24T06:32:14.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.099+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b2c0b3bd0 con 0x7f9b2c15e7c0 2024-04-24T06:32:14.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.100+0000 7f9b29600640 1 -- 172.21.15.59:0/3429684917 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5178 (secure 0 0 0) 0x7f9b24059e50 con 0x7f9b2c15e7c0 2024-04-24T06:32:14.102 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:14.102 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:10.584021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:32:14.102 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 msgr2=0x7f9b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 0x7f9b0806edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c034000 tx=0x7f9b1c02e000 comp rx=0 tx=0).stop 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 msgr2=0x7f9b2c0cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c730 tx=0x7f9b2400cbf0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 shutdown_connections 2024-04-24T06:32:14.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b0806c8f0 0x7f9b0806edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b2c15e7c0 0x7f9b2c0cde70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b2c0d2630 0x7f9b2c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 --2- 172.21.15.59:0/3429684917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b2c0b6530 0x7f9b2c0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.103+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 >> 172.21.15.59:0/3429684917 conn(0x7f9b2c09d920 msgr2=0x7f9b2c15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.104+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 shutdown_connections 2024-04-24T06:32:14.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.104+0000 7f9b32200640 1 -- 172.21.15.59:0/3429684917 wait complete. 2024-04-24T06:32:14.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:32:07.494567+0000', 'modified': '2024-04-24T06:32:10.584021+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6192}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6192': {'gid': 6192, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6837/2525582962', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2525582962}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2525582962}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6192, 'qdb_cluster': [6192]} 2024-04-24T06:32:14.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:32:14.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T06:32:14.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.751+0000 7fc9c6800640 1 Processor -- start 2024-04-24T06:32:14.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.751+0000 7fc9c6800640 1 -- start start 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.752+0000 7fc9c6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc9c006e250 0x7fc9c00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.752+0000 7fc9c6800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c00cd880 con 0x7fc9c006ddf0 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.752+0000 7fc9c6800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c0071ec0 con 0x7fc9c00cf410 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.752+0000 7fc9c6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c00cdb20 con 0x7fc9c006e250 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc9c006ddf0 0x7fc9c0072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51778/0 (socket says 172.21.15.59:51778) 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bea00640 1 -- 172.21.15.59:0/2700164522 learned_addr learned my addr 172.21.15.59:0/2700164522 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2700164522 >> v2:172.21.15.59:3300/0 conn(0x7fc9c006e250 0x7fc9c00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9be000640 1 -- 172.21.15.59:0/2700164522 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266593849 0 0) 0x7fc9c00cd880 con 0x7fc9c006ddf0 2024-04-24T06:32:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9be000640 1 -- 172.21.15.59:0/2700164522 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a4003520 con 0x7fc9c006ddf0 2024-04-24T06:32:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2700164522 >> v1:172.21.15.66:6789/0 conn(0x7fc9c00cf410 legacy=0x7fc9c015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2700164522 >> v1:172.21.15.59:6789/0 conn(0x7fc9c006ddf0 legacy=0x7fc9c0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.753+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2700164522 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0072160 con 0x7fc9c006e250 2024-04-24T06:32:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2700164522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c006e250 0x7fc9c00cf7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80097a0 tx=0x7fc9b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30864e3ec1048ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9be000640 1 -- 172.21.15.59:0/2700164522 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8037070 con 0x7fc9c006e250 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9be000640 1 -- 172.21.15.59:0/2700164522 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b8032440 con 0x7fc9c006e250 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9be000640 1 -- 172.21.15.59:0/2700164522 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b80327f0 con 0x7fc9c006e250 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9c6800640 1 -- 172.21.15.59:0/2700164522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c006e250 msgr2=0x7fc9c00cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.755+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2700164522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c006e250 0x7fc9c00cf7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80097a0 tx=0x7fc9b802a1a0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.756+0000 7fc9c6800640 1 -- 172.21.15.59:0/2700164522 shutdown_connections 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.756+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2700164522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c006e250 0x7fc9c00cf7f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.756+0000 7fc9c6800640 1 -- 172.21.15.59:0/2700164522 >> 172.21.15.59:0/2700164522 conn(0x7fc9c00b2130 msgr2=0x7fc9c00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.756+0000 7fc9c6800640 1 -- 172.21.15.59:0/2700164522 shutdown_connections 2024-04-24T06:32:14.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.756+0000 7fc9c6800640 1 -- 172.21.15.59:0/2700164522 wait complete. 2024-04-24T06:32:14.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.757+0000 7fc9c6800640 1 Processor -- start 2024-04-24T06:32:14.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.757+0000 7fc9c6800640 1 -- start start 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 0x7fc9c0142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 0x7fc9c01f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c015a820 con 0x7fc9c00cf410 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c015a6a0 con 0x7fc9c006e250 2024-04-24T06:32:14.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.758+0000 7fc9c6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c0072d90 con 0x7fc9c006ddf0 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60174/0 (socket says 172.21.15.59:60174) 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2296421657 learned_addr learned my addr 172.21.15.59:0/2296421657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bea00640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 0x7fc9c01f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 0x7fc9c0142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 msgr2=0x7fc9c0142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 0x7fc9c0142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 msgr2=0x7fc9c01f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 0x7fc9c01f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2296421657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0156690 con 0x7fc9c00cf410 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bea00640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 0x7fc9c01f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.759+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 0x7fc9c0142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80097a0 tx=0x7fc9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9b7400640 1 -- 172.21.15.59:0/2296421657 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8037050 con 0x7fc9c00cf410 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9b7400640 1 -- 172.21.15.59:0/2296421657 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b8043070 con 0x7fc9c00cf410 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9b7400640 1 -- 172.21.15.59:0/2296421657 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8006e10 con 0x7fc9c00cf410 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c0156970 con 0x7fc9c00cf410 2024-04-24T06:32:14.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.760+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c0159510 con 0x7fc9c00cf410 2024-04-24T06:32:14.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.762+0000 7fc9b7400640 1 -- 172.21.15.59:0/2296421657 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9b80327f0 con 0x7fc9c00cf410 2024-04-24T06:32:14.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.763+0000 7fc9b7400640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 0x7fc9a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.763+0000 7fc9b7400640 1 -- 172.21.15.59:0/2296421657 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b80ac5a0 con 0x7fc9c00cf410 2024-04-24T06:32:14.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.763+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 0x7fc9a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.764+0000 7fc9c9630740 1 Processor -- start 2024-04-24T06:32:14.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.764+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 0x7fc9a006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001680 tx=0x7fc9b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.764+0000 7fc9c9630740 1 -- start start 2024-04-24T06:32:14.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e817140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e847190 0x561d8e81bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d8e781b10 con 0x561d8e8478d0 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d8e7282b0 con 0x561d8e8647b0 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.765+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561d8e77dad0 con 0x561d8e847190 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60178/0 (socket says 172.21.15.59:60178) 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 -- 172.21.15.59:0/1924224707 learned_addr learned my addr 172.21.15.59:0/1924224707 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bf400640 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e817140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e847190 0x561d8e81bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 -- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e847190 msgr2=0x561d8e81bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e847190 0x561d8e81bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 -- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 msgr2=0x561d8e817140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e817140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.766+0000 7fc9bea00640 1 -- 172.21.15.59:0/1924224707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8e4942d0 con 0x561d8e8478d0 2024-04-24T06:32:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9bea00640 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0021f0 tx=0x7fc9ac00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90786fd1643eb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9abe00640 1 -- 172.21.15.59:0/1924224707 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac018070 con 0x561d8e8478d0 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9abe00640 1 -- 172.21.15.59:0/1924224707 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac013670 con 0x561d8e8478d0 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9abe00640 1 -- 172.21.15.59:0/1924224707 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac013a20 con 0x561d8e8478d0 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9c9630740 1 -- 172.21.15.59:0/1924224707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 msgr2=0x561d8e846c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.767+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0021f0 tx=0x7fc9ac00d5e0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 -- 172.21.15.59:0/1924224707 shutdown_connections 2024-04-24T06:32:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e847190 0x561d8e81bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e846c50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1924224707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e817140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 -- 172.21.15.59:0/1924224707 >> 172.21.15.59:0/1924224707 conn(0x561d8e8193b0 msgr2=0x561d8e861c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 -- 172.21.15.59:0/1924224707 shutdown_connections 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 -- 172.21.15.59:0/1924224707 wait complete. 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.768+0000 7fc9c9630740 1 -- start start 2024-04-24T06:32:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.769+0000 7fc9c9630740 1 Processor -- start 2024-04-24T06:32:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e835fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e83bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d8e81b560 con 0x561d8e863640 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d8e73b590 con 0x561d8e8647b0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561d8e81a950 con 0x561d8e8478d0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9c9630740 10 client.0 _ll_register_callbacks cb 0x561d8e71e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60192/0 (socket says 172.21.15.59:60192) 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2828498383 learned_addr learned my addr 172.21.15.59:0/2828498383 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e835fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.770+0000 7fc9bea00640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e83bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 msgr2=0x561d8e835fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e835fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 msgr2=0x561d8e83bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e83bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 -- 172.21.15.59:0/2828498383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8e55e6b0 con 0x561d8e863640 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e835fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bea00640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e83bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8076720 tx=0x7fc9b80093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3136cb0343ba2f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b807a720 con 0x561d8e863640 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b8082a10 con 0x561d8e863640 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac013670 con 0x561d8e863640 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc9b807a2c0 con 0x561d8e863640 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9c9630740 10 client.6330 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.771+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x561d8e84aa50 con 0x561d8e863640 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac013670 con 0x561d8e863640 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7fc9b807e170 con 0x561d8e863640 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 10 client.6330 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 10 client.6330 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 20 client.6330 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e838ab0 0x561d8e85d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 4 client.6330 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8e78f530 con 0x561d8e838ab0 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.772+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e838ab0 0x561d8e85d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.773+0000 7fc9bf400640 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e838ab0 0x561d8e85d6f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b422240b8a5313c9 server_cookie=885d5930afa1f0f0 in_seq=0 out_seq=0 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.773+0000 7fc9a8c00640 10 client.6330 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:14.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9a8c00640 1 -- 172.21.15.59:0/2828498383 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d8e78f530 con 0x561d8e838ab0 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9a8c00640 10 client.6330 handle_command_reply: tid=0 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 client.6330 shutdown 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 msgr2=0x561d8e835370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8076720 tx=0x7fc9b80093c0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 shutdown_connections 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e838ab0 0x561d8e85d6f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e863640 0x561d8e835370 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e8647b0 0x561d8e83bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 --2- 172.21.15.59:0/2828498383 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e835fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 >> 172.21.15.59:0/2828498383 conn(0x561d8e847190 msgr2=0x561d8e81ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9a8c00640 0 client.6330 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:14.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 shutdown_connections 2024-04-24T06:32:14.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.774+0000 7fc9c9630740 1 -- 172.21.15.59:0/2828498383 wait complete. 2024-04-24T06:32:14.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.775+0000 7fc9c9630740 20 client.6330 trim_cache size 0 max 16384 2024-04-24T06:32:14.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.795+0000 7fc9c9630740 1 Processor -- start 2024-04-24T06:32:14.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.795+0000 7fc9c9630740 1 -- start start 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.795+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.795+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e861af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8647b0 0x561d8e817460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9c9630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c00b5580 con 0x561d8e8478d0 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c00b48e0 con 0x561d8e863640 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c00b4f30 con 0x561d8e8647b0 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60196/0 (socket says 172.21.15.59:60196) 2024-04-24T06:32:14.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 -- 172.21.15.59:0/1558231828 learned_addr learned my addr 172.21.15.59:0/1558231828 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:14.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 -- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8647b0 msgr2=0x561d8e817460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8647b0 0x561d8e817460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 -- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 msgr2=0x561d8e861af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e861af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.796+0000 7fc9bf400640 1 -- 172.21.15.59:0/1558231828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d8e81dc10 con 0x561d8e8478d0 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.797+0000 7fc9bf400640 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x561d8e85d260 tx=0x7fc9b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2f7c7b106df250 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.797+0000 7fc9ab400640 1 -- 172.21.15.59:0/1558231828 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b001d040 con 0x561d8e8478d0 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.797+0000 7fc9ab400640 1 -- 172.21.15.59:0/1558231828 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b002c070 con 0x561d8e8478d0 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.797+0000 7fc9ab400640 1 -- 172.21.15.59:0/1558231828 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0014c70 con 0x561d8e8478d0 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 -- 172.21.15.59:0/1558231828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 msgr2=0x561d8e834920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x561d8e85d260 tx=0x7fc9b0013000 comp rx=0 tx=0).stop 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 -- 172.21.15.59:0/1558231828 shutdown_connections 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8647b0 0x561d8e817460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e861af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 --2- 172.21.15.59:0/1558231828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8478d0 0x561d8e834920 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 -- 172.21.15.59:0/1558231828 >> 172.21.15.59:0/1558231828 conn(0x561d8e847190 msgr2=0x561d8e85e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 -- 172.21.15.59:0/1558231828 shutdown_connections 2024-04-24T06:32:14.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.798+0000 7fc9c9630740 1 -- 172.21.15.59:0/1558231828 wait complete. 2024-04-24T06:32:14.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.799+0000 7fc9c9630740 1 -- start start 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.799+0000 7fc9c9630740 1 Processor -- start 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e834880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e847fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d8e838930 con 0x561d8e8647b0 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d8e838430 con 0x561d8e863640 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561d8e838f80 con 0x561d8e8478d0 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9c9630740 10 client.0 _ll_register_callbacks cb 0x7fc9b8003e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60206/0 (socket says 172.21.15.59:60206) 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 -- 172.21.15.59:0/4164582737 learned_addr learned my addr 172.21.15.59:0/4164582737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 -- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 msgr2=0x561d8e834880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e834880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 -- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 msgr2=0x561d8e847fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.800+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e847fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9bfe00640 1 -- 172.21.15.59:0/4164582737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8e888e70 con 0x561d8e8647b0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9bfe00640 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc9b807a250 tx=0x7fc9b8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66920f99e435c053 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b807a520 con 0x561d8e8647b0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b807a6c0 con 0x561d8e8647b0 2024-04-24T06:32:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b002c070 con 0x561d8e8647b0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc9b8099be0 con 0x561d8e8647b0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9c9630740 10 client.6336 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.801+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x561d8e889710 con 0x561d8e8647b0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b002c070 con 0x561d8e8647b0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7fc9b80ac0b0 con 0x561d8e8647b0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 10 client.6336 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 10 client.6336 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 20 client.6336 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e843ee0 0x561d8e8e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 4 client.6336 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "status"}) -- 0x561d8e7893a0 con 0x561d8e843ee0 2024-04-24T06:32:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.802+0000 7fc9bf400640 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e843ee0 0x561d8e8e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9a8c00640 10 client.6336 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9bf400640 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e843ee0 0x561d8e8e1070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a1f4188cbb7b8f server_cookie=fa453d8e4687898b in_seq=0 out_seq=0 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9a8c00640 1 -- 172.21.15.59:0/4164582737 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x561d8e7893a0 con 0x561d8e843ee0 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9a8c00640 10 client.6336 handle_command_reply: tid=0 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 client.6336 shutdown 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 msgr2=0x561d8e848500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc9b807a250 tx=0x7fc9b8009190 comp rx=0 tx=0).stop 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 shutdown_connections 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561d8e843ee0 0x561d8e8e1070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d8e8647b0 0x561d8e848500 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d8e863640 0x561d8e847fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 --2- 172.21.15.59:0/4164582737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561d8e8478d0 0x561d8e834880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 >> 172.21.15.59:0/4164582737 conn(0x561d8e847190 msgr2=0x561d8e817460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9a8c00640 0 client.6336 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 shutdown_connections 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.803+0000 7fc9c9630740 1 -- 172.21.15.59:0/4164582737 wait complete. 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c9630740 20 client.6336 trim_cache size 0 max 16384 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:14.810 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6192, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7490226260000004, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 33, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 58, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 58, 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.15153235 2024-04-24T06:32:14.811 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 msgr2=0x7fc9a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 0x7fc9a006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001680 tx=0x7fc9b0009040 comp rx=0 tx=0).stop 2024-04-24T06:32:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 msgr2=0x7fc9c0145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80097a0 tx=0x7fc9b80046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 shutdown_connections 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc9a006c8f0 0x7fc9a006edb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9c00cf410 0x7fc9c0145800 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9c006e250 0x7fc9c01f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 --2- 172.21.15.59:0/2296421657 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9c006ddf0 0x7fc9c0142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 >> 172.21.15.59:0/2296421657 conn(0x7fc9c00b2130 msgr2=0x7fc9c015aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 shutdown_connections 2024-04-24T06:32:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:14.804+0000 7fc9c6800640 1 -- 172.21.15.59:0/2296421657 wait complete. 2024-04-24T06:32:14.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:32:15.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.033+0000 7f04bb200640 1 Processor -- start 2024-04-24T06:32:15.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.034+0000 7f04bb200640 1 -- start start 2024-04-24T06:32:15.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04bb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc15e7c0 0x7f04bc0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04bb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04bc0d2630 0x7f04bc0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04bb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0cf030 con 0x7f04bc0b6530 2024-04-24T06:32:15.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04bb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc0b7000 con 0x7f04bc0d2630 2024-04-24T06:32:15.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04bb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0cf620 con 0x7f04bc15e7c0 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04b9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04bc0d2630 0x7f04bc0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04b9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04bc0d2630 0x7f04bc0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60214/0 (socket says 172.21.15.59:60214) 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.035+0000 7f04b9400640 1 -- 172.21.15.59:0/3762786568 learned_addr learned my addr 172.21.15.59:0/3762786568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b9e00640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc15e7c0 0x7f04bc0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b9400640 1 -- 172.21.15.59:0/3762786568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc15e7c0 msgr2=0x7f04bc0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b9400640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc15e7c0 0x7f04bc0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b9400640 1 -- 172.21.15.59:0/3762786568 >> v1:172.21.15.59:6789/0 conn(0x7f04bc0b6530 legacy=0x7f04bc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b9400640 1 -- 172.21.15.59:0/3762786568 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0b6a40 con 0x7f04bc0d2630 2024-04-24T06:32:15.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.036+0000 7f04b8a00640 1 -- 172.21.15.59:0/3762786568 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170522103 0 0) 0x7f04b4001110 con 0x7f04bc0b6530 2024-04-24T06:32:15.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.037+0000 7f04b9400640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0d2630 0x7f04bc0b5ff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009e50 tx=0x7f04a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8253a76f7adec5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:15.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.037+0000 7f04b8a00640 1 -- 172.21.15.59:0/3762786568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4036070 con 0x7f04bc0d2630 2024-04-24T06:32:15.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.037+0000 7f04b8a00640 1 -- 172.21.15.59:0/3762786568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a4031600 con 0x7f04bc0d2630 2024-04-24T06:32:15.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.037+0000 7f04b8a00640 1 -- 172.21.15.59:0/3762786568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40319d0 con 0x7f04bc0d2630 2024-04-24T06:32:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 -- 172.21.15.59:0/3762786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0d2630 msgr2=0x7f04bc0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0d2630 0x7f04bc0b5ff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009e50 tx=0x7f04a402a710 comp rx=0 tx=0).stop 2024-04-24T06:32:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 -- 172.21.15.59:0/3762786568 shutdown_connections 2024-04-24T06:32:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0d2630 0x7f04bc0b5ff0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 --2- 172.21.15.59:0/3762786568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc15e7c0 0x7f04bc0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 -- 172.21.15.59:0/3762786568 >> 172.21.15.59:0/3762786568 conn(0x7f04bc09d920 msgr2=0x7f04bc08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:15.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 -- 172.21.15.59:0/3762786568 shutdown_connections 2024-04-24T06:32:15.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.038+0000 7f04bb200640 1 -- 172.21.15.59:0/3762786568 wait complete. 2024-04-24T06:32:15.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.040+0000 7f04bb200640 1 Processor -- start 2024-04-24T06:32:15.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.040+0000 7f04bb200640 1 -- start start 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc0d2630 0x7f04bc15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e7c0 0x7f04bc0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0d1a80 con 0x7f04bc0b6530 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0cf2b0 con 0x7f04bc15e7c0 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04bb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0d1c00 con 0x7f04bc0d2630 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.041+0000 7f04b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60226/0 (socket says 172.21.15.59:60226) 2024-04-24T06:32:15.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 -- 172.21.15.59:0/2750285806 learned_addr learned my addr 172.21.15.59:0/2750285806 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:15.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04ba800640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e7c0 0x7f04bc0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9400640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc0d2630 0x7f04bc15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 -- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc0d2630 msgr2=0x7f04bc15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc0d2630 0x7f04bc15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 -- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e7c0 msgr2=0x7f04bc0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e7c0 0x7f04bc0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.042+0000 7f04b9e00640 1 -- 172.21.15.59:0/2750285806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0b1820 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04b9e00640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00d5f0 tx=0x7f04ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0125f0 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac019070 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0129d0 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc0b1b00 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.043+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc0b20c0 con 0x7f04bc0b6530 2024-04-24T06:32:15.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.044+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc04fd90 con 0x7f04bc0b6530 2024-04-24T06:32:15.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.046+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04ac012b70 con 0x7f04bc0b6530 2024-04-24T06:32:15.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.046+0000 7f04b2000640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:15.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.047+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f04ac08bf10 con 0x7f04bc0b6530 2024-04-24T06:32:15.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.047+0000 7f04b9400640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:15.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.048+0000 7f04b9400640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04a4033000 tx=0x7f04a4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:15.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.052+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04ac05a340 con 0x7f04bc0b6530 2024-04-24T06:32:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.303+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04bc0b3bd0 con 0x7f04bc0b6530 2024-04-24T06:32:15.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.304+0000 7f04b2000640 1 -- 172.21.15.59:0/2750285806 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f04ac059c40 con 0x7f04bc0b6530 2024-04-24T06:32:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:32:15.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 msgr2=0x7f049806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 0x7f049806edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04a4033000 tx=0x7f04a4009000 comp rx=0 tx=0).stop 2024-04-24T06:32:15.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 msgr2=0x7f04bc14f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:15.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00d5f0 tx=0x7f04ac00dab0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 shutdown_connections 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049806c8f0 0x7f049806edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04bc15e7c0 0x7f04bc0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04bc0d2630 0x7f04bc15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 --2- 172.21.15.59:0/2750285806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc0b6530 0x7f04bc14f7d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 >> 172.21.15.59:0/2750285806 conn(0x7f04bc09d920 msgr2=0x7f04bc15c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.307+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 shutdown_connections 2024-04-24T06:32:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15.308+0000 7f04bb200640 1 -- 172.21.15.59:0/2750285806 wait complete. 2024-04-24T06:32:15.329 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:32:15.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:32:15.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:32:15.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:32:15.330 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:15.330 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:32:15.330 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:32:15.330 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:32:15.330 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:32:15.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:15.330 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.384 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42409sec preferred_lft 42409sec 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:32:15.385 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:32:15.386 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:32:15.387 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:32:15.387 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:32:15.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:15.388 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:32:15.473 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:32:15.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:15.478 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:32:15.532 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:32:15.532 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:32:15.532 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:32:15.532 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:32:15.532 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:32:15.533 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:32:15.533 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:32:15.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:32:15.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:32:15.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:32:15.534 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:32:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:32:15.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:32:15.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:15.754 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:32:15.802 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:32:15.802 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:32:15.803 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:32:15.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:15.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:32:15.890 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:32:15.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:32:15.890 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:15.945 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:32:15.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:32:15.945 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:15.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:32:16.084 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:32:16.141 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:32:16.141 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:32:16.141 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:32:16.141 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:32:16.141 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:32:16.142 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:32:16.143 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:32:16.144 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:32:16.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:16.144 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:32:16.196 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:32:16.196 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:32:16.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:32:16.270 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:32:16.314 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:32:16.315 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:32:16.316 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:32:16.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:32:16.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:16.318 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:32:16.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:32:16.368+0000 7fc2bf91f480 -1 init, newargv = 0x5600cb352930 newargc=15 2024-04-24T06:32:16.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[53951]: starting ceph client 2024-04-24T06:32:16.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[53951]: starting fuse 2024-04-24T06:32:16.401 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:32:16.401 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:32:16.402 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:32:16.403 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:32:16.607 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.53951.asok 2024-04-24T06:32:16.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:32:16.625 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53951.asok 2024-04-24T06:32:16.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:16.625 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53951.asok status 2024-04-24T06:32:16.864 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:16.864 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:32:16.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:32:16.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "53951", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6351, 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 6351 2024-04-24T06:32:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1382438545 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1382438545 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:16.866 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.6351 192.168.0.1:0/1382438545", 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1382438545", 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 33, 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 58, 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:32:16.867 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:16.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:16.880 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:32:16.880 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:16.935 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:32:16.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:16.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:16.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:32:17.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:17.026 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-24T06:32:17.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:17.097 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-24T06:32:17.160 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:32:17.160 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:32:17.160 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:32:17.160 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:32:17.161 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-24T06:32:17.161 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:32:17.306 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:32:17.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:32:17.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:32:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.580+0000 7fc741e00640 1 Processor -- start 2024-04-24T06:32:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.581+0000 7fc741e00640 1 -- start start 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc741e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc73c0cd760 0x7fc73c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc741e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73c160210 con 0x7fc73c0b4690 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc741e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73c1604f0 con 0x7fc73c15c3d0 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc741e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc73c1607d0 con 0x7fc73c0cd760 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc73be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc73c0cd760 0x7fc73c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc741400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc73c0b4690 0x7fc73c0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51802/0 (socket says 172.21.15.59:51802) 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.582+0000 7fc73be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc73c0cd760 0x7fc73c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60232/0 (socket says 172.21.15.59:60232) 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.583+0000 7fc741400640 1 -- 172.21.15.59:0/3959264730 learned_addr learned my addr 172.21.15.59:0/3959264730 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.583+0000 7fc73be00640 1 -- 172.21.15.59:0/3959264730 >> v1:172.21.15.66:6789/0 conn(0x7fc73c15c3d0 legacy=0x7fc73c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.583+0000 7fc73be00640 1 -- 172.21.15.59:0/3959264730 >> v1:172.21.15.59:6789/0 conn(0x7fc73c0b4690 legacy=0x7fc73c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.583+0000 7fc73be00640 1 -- 172.21.15.59:0/3959264730 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c160ab0 con 0x7fc73c0cd760 2024-04-24T06:32:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.583+0000 7fc73b400640 1 -- 172.21.15.59:0/3959264730 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099072536 0 0) 0x7fc73c160210 con 0x7fc73c0b4690 2024-04-24T06:32:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.584+0000 7fc73be00640 1 --2- 172.21.15.59:0/3959264730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0cd760 0x7fc73c0b4150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097a0 tx=0x7fc72802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf386849c3af6870 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc73b400640 1 -- 172.21.15.59:0/3959264730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728037070 con 0x7fc73c0cd760 2024-04-24T06:32:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc73b400640 1 -- 172.21.15.59:0/3959264730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc728032440 con 0x7fc73c0cd760 2024-04-24T06:32:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc73b400640 1 -- 172.21.15.59:0/3959264730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728032810 con 0x7fc73c0cd760 2024-04-24T06:32:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 -- 172.21.15.59:0/3959264730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0cd760 msgr2=0x7fc73c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 --2- 172.21.15.59:0/3959264730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0cd760 0x7fc73c0b4150 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097a0 tx=0x7fc72802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 -- 172.21.15.59:0/3959264730 shutdown_connections 2024-04-24T06:32:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 --2- 172.21.15.59:0/3959264730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0cd760 0x7fc73c0b4150 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 -- 172.21.15.59:0/3959264730 >> 172.21.15.59:0/3959264730 conn(0x7fc73c0b21a0 msgr2=0x7fc73c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 -- 172.21.15.59:0/3959264730 shutdown_connections 2024-04-24T06:32:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.585+0000 7fc741e00640 1 -- 172.21.15.59:0/3959264730 wait complete. 2024-04-24T06:32:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.587+0000 7fc741e00640 1 Processor -- start 2024-04-24T06:32:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.587+0000 7fc741e00640 1 -- start start 2024-04-24T06:32:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 0x7fc73c0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c15c3d0 0x7fc73c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c15c7b0 con 0x7fc73c0b4690 2024-04-24T06:32:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c0b4aa0 con 0x7fc73c0cd760 2024-04-24T06:32:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc741e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc73c15c930 con 0x7fc73c15c3d0 2024-04-24T06:32:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc740a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc740a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60246/0 (socket says 172.21.15.59:60246) 2024-04-24T06:32:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.588+0000 7fc740a00640 1 -- 172.21.15.59:0/4127976099 learned_addr learned my addr 172.21.15.59:0/4127976099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc73be00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 0x7fc73c0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 -- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c15c3d0 msgr2=0x7fc73c0c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc741400640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c15c3d0 0x7fc73c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c15c3d0 0x7fc73c0c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 -- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 msgr2=0x7fc73c0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 0x7fc73c0c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 -- 172.21.15.59:0/4127976099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c0c3fd0 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc73be00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 0x7fc73c0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc740a00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b3f0 tx=0x7fc72c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.589+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c015020 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.590+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c011830 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.590+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c011c20 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.590+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c0c42b0 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.590+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c0c9e30 con 0x7fc73c0b4690 2024-04-24T06:32:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.591+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73c0ca0c0 con 0x7fc73c0b4690 2024-04-24T06:32:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.592+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc72c018020 con 0x7fc73c0b4690 2024-04-24T06:32:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.593+0000 7fc738c00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.593+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc72c08b810 con 0x7fc73c0b4690 2024-04-24T06:32:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.593+0000 7fc73be00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.594+0000 7fc73be00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc728034000 tx=0x7fc72802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.598+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc72c01f030 con 0x7fc73c0b4690 2024-04-24T06:32:17.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.842+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc73c04fd90 con 0x7fc73c0b4690 2024-04-24T06:32:17.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.843+0000 7fc738c00640 1 -- 172.21.15.59:0/4127976099 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5178 (secure 0 0 0) 0x7fc72c0172c0 con 0x7fc73c0b4690 2024-04-24T06:32:17.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:17.845 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:10.584021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:32:17.845 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.845+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 msgr2=0x7fc71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.845+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc728034000 tx=0x7fc72802e000 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.845+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 msgr2=0x7fc73c0c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.845+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b3f0 tx=0x7fc72c00b8b0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 shutdown_connections 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c15c3d0 0x7fc73c0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c0cd760 0x7fc73c0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 --2- 172.21.15.59:0/4127976099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0b4690 0x7fc73c0c98f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:17.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 >> 172.21.15.59:0/4127976099 conn(0x7fc73c0b21a0 msgr2=0x7fc73c0afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:17.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 shutdown_connections 2024-04-24T06:32:17.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:17.846+0000 7fc741e00640 1 -- 172.21.15.59:0/4127976099 wait complete. 2024-04-24T06:32:17.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-24T06:32:18.120 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-24T06:32:18.120 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:32:18.120 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-24T06:32:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.229+0000 7f646091df40 1 Processor -- start 2024-04-24T06:32:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.229+0000 7f646091df40 1 -- start start 2024-04-24T06:32:18.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6444002460 0x55b848eae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b848f4a790 0x55b848eaa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b848e26cc0 con 0x55b848eaad90 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b848e2ad00 con 0x55b848f4a790 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f646091df40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b848f55820 con 0x7f6444002460 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.230+0000 7f645e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60266/0 (socket says 172.21.15.59:60266) 2024-04-24T06:32:18.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 -- 172.21.15.59:0/737787343 learned_addr learned my addr 172.21.15.59:0/737787343 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 -- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6444002460 msgr2=0x55b848eae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6444002460 0x55b848eae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 -- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b848f4a790 msgr2=0x55b848eaa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b848f4a790 0x55b848eaa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 -- 172.21.15.59:0/737787343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b848e81ea0 con 0x55b848eaad90 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.231+0000 7f645e600640 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6458000fb0 tx=0x7f645800d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b345fcb8a2b2da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f6457400640 1 -- 172.21.15.59:0/737787343 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645800dd10 con 0x55b848eaad90 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f6457400640 1 -- 172.21.15.59:0/737787343 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458003110 con 0x55b848eaad90 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f6457400640 1 -- 172.21.15.59:0/737787343 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458003300 con 0x55b848eaad90 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 -- 172.21.15.59:0/737787343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 msgr2=0x55b848e81170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6458000fb0 tx=0x7f645800d630 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 -- 172.21.15.59:0/737787343 shutdown_connections 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848e81170 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b848f4a790 0x55b848eaa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 --2- 172.21.15.59:0/737787343 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6444002460 0x55b848eae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 -- 172.21.15.59:0/737787343 >> 172.21.15.59:0/737787343 conn(0x55b848ecb290 msgr2=0x55b848eaeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 -- 172.21.15.59:0/737787343 shutdown_connections 2024-04-24T06:32:18.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.232+0000 7f646091df40 1 -- 172.21.15.59:0/737787343 wait complete. 2024-04-24T06:32:18.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.233+0000 7f646091df40 1 Processor -- start 2024-04-24T06:32:18.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.233+0000 7f646091df40 1 -- start start 2024-04-24T06:32:18.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848f45400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b848f4a790 0x55b848f43720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6444002460 0x55b848f43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b848f56b70 con 0x55b848eaad90 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b848f56920 con 0x7f6444002460 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.234+0000 7f646091df40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b848f56e50 con 0x55b848f4a790 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848f45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848f45400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60282/0 (socket says 172.21.15.59:60282) 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 -- 172.21.15.59:0/3141509318 learned_addr learned my addr 172.21.15.59:0/3141509318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645d200640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b848f4a790 0x55b848f43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 -- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b848f4a790 msgr2=0x55b848f43720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b848f4a790 0x55b848f43720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 -- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6444002460 msgr2=0x55b848f43ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6444002460 0x55b848f43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645dc00640 1 -- 172.21.15.59:0/3141509318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b848f44710 con 0x55b848eaad90 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.235+0000 7f645d200640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b848f4a790 0x55b848f43720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f645dc00640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848f45400 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6450025e40 tx=0x7f6450005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f6456000640 1 -- 172.21.15.59:0/3141509318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450064020 con 0x55b848eaad90 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f6456000640 1 -- 172.21.15.59:0/3141509318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645004d910 con 0x55b848eaad90 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f6456000640 1 -- 172.21.15.59:0/3141509318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000a3b0 con 0x55b848eaad90 2024-04-24T06:32:18.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b848f449f0 con 0x55b848eaad90 2024-04-24T06:32:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.236+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b848f44fb0 con 0x55b848eaad90 2024-04-24T06:32:18.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.238+0000 7f6456000640 1 -- 172.21.15.59:0/3141509318 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6450067020 con 0x55b848eaad90 2024-04-24T06:32:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.239+0000 7f6456000640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643c06b300 0x7f643c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.239+0000 7f6456000640 1 -- 172.21.15.59:0/3141509318 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f64500ce340 con 0x55b848eaad90 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.240+0000 7f645d200640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643c06b300 0x7f643c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.240+0000 7f646091df40 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55b848e7e6c0 0x55b848e7eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.240+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58) -- 0x55b848e7f050 con 0x55b848e7e6c0 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.240+0000 7f645d200640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643c06b300 0x7f643c06d7c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6448001ed0 tx=0x7f64480014b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.241+0000 7f645e600640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55b848e7e6c0 0x55b848e7eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.242+0000 7f645e600640 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55b848e7e6c0 0x55b848e7eb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f645e600640 1 -- 172.21.15.59:0/3141509318 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f645800e040 con 0x55b848e7e6c0 2024-04-24T06:32:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55b848e7e6c0 msgr2=0x55b848e7eb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55b848e7e6c0 0x55b848e7eb10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643c06b300 msgr2=0x7f643c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643c06b300 0x7f643c06d7c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6448001ed0 tx=0x7f64480014b0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 msgr2=0x55b848f45400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 --2- 172.21.15.59:0/3141509318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b848eaad90 0x55b848f45400 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6450025e40 tx=0x7f6450005520 comp rx=0 tx=0).stop 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f645e600640 1 -- 172.21.15.59:0/3141509318 reap_dead start 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 shutdown_connections 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 >> 172.21.15.59:0/3141509318 conn(0x55b848ecb290 msgr2=0x55b848f55ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 shutdown_connections 2024-04-24T06:32:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.243+0000 7f646091df40 1 -- 172.21.15.59:0/3141509318 wait complete. 2024-04-24T06:32:18.250 INFO:teuthology.orchestra.run.smithi059.stdout:Xfiledir_test 2024-04-24T06:32:18.251 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-24T06:32:18.354 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "file", 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "dir_test", 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-24T06:32:18.355 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 9, 2024-04-24T06:32:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-24T06:32:18.356 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:18.360 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.473+0000 7f5a8481ef40 1 Processor -- start 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.473+0000 7f5a8481ef40 1 -- start start 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 0x56235b7b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b729450 0x56235b799850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56235b729e70 con 0x56235b79a5b0 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56235b686cc0 con 0x56235b729450 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56235b72a110 con 0x56235b7276a0 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a82400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a82400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60306/0 (socket says 172.21.15.59:60306) 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a82400640 1 -- 172.21.15.59:0/1999042881 learned_addr learned my addr 172.21.15.59:0/1999042881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a81a00640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 0x56235b7b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.474+0000 7f5a82400640 1 -- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 msgr2=0x56235b7b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a81000640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b729450 0x56235b799850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a82400640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 0x56235b7b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a82400640 1 -- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b729450 msgr2=0x56235b799850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a82400640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b729450 0x56235b799850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a82400640 1 -- 172.21.15.59:0/1999042881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235b799d90 con 0x56235b79a5b0 2024-04-24T06:32:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a81a00640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 0x56235b7b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a82400640 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00a4b0 tx=0x7f5a7c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11c5d6a36030180 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a7b400640 1 -- 172.21.15.59:0/1999042881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c055030 con 0x56235b79a5b0 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a7b400640 1 -- 172.21.15.59:0/1999042881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7c051c30 con 0x56235b79a5b0 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.475+0000 7f5a7b400640 1 -- 172.21.15.59:0/1999042881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c05b430 con 0x56235b79a5b0 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1999042881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 msgr2=0x56235b79a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00a4b0 tx=0x7f5a7c050000 comp rx=0 tx=0).stop 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1999042881 shutdown_connections 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b79a5b0 0x56235b79a990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b729450 0x56235b799850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1999042881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b7276a0 0x56235b7b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1999042881 >> 172.21.15.59:0/1999042881 conn(0x56235b70ce20 msgr2=0x56235b7b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1999042881 shutdown_connections 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.476+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1999042881 wait complete. 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.477+0000 7f5a8481ef40 1 Processor -- start 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.477+0000 7f5a8481ef40 1 -- start start 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b7276a0 0x56235b792520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 0x56235b796230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b79a5b0 0x56235b7b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56235b7b10c0 con 0x56235b729450 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56235b7b6520 con 0x56235b7276a0 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a8481ef40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56235b7b6260 con 0x56235b79a5b0 2024-04-24T06:32:18.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a81000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 0x56235b796230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a81000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 0x56235b796230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60308/0 (socket says 172.21.15.59:60308) 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.478+0000 7f5a81000640 1 -- 172.21.15.59:0/1815939316 learned_addr learned my addr 172.21.15.59:0/1815939316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a82400640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b79a5b0 0x56235b7b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 -- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b79a5b0 msgr2=0x56235b7b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56235b79a5b0 0x56235b7b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81a00640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b7276a0 0x56235b792520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 -- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b7276a0 msgr2=0x56235b792520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56235b7276a0 0x56235b792520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 -- 172.21.15.59:0/1815939316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235b792db0 con 0x56235b729450 2024-04-24T06:32:18.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.479+0000 7f5a81000640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 0x56235b796230 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00be00 tx=0x7f5a6c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.480+0000 7f5a7a000640 1 -- 172.21.15.59:0/1815939316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c019070 con 0x56235b729450 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.480+0000 7f5a7a000640 1 -- 172.21.15.59:0/1815939316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6c007af0 con 0x56235b729450 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.480+0000 7f5a7a000640 1 -- 172.21.15.59:0/1815939316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c014410 con 0x56235b729450 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.480+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56235b793090 con 0x56235b729450 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.480+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235b795d10 con 0x56235b729450 2024-04-24T06:32:18.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.482+0000 7f5a7a000640 1 -- 172.21.15.59:0/1815939316 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a6c01c070 con 0x56235b729450 2024-04-24T06:32:18.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.483+0000 7f5a7a000640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6006b2d0 0x7f5a6006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.483+0000 7f5a7a000640 1 -- 172.21.15.59:0/1815939316 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5a6c089720 con 0x56235b729450 2024-04-24T06:32:18.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.483+0000 7f5a81a00640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6006b2d0 0x7f5a6006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.484+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x56235b796770 0x56235b797240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.484+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x56235b7af5f0 con 0x56235b796770 2024-04-24T06:32:18.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.484+0000 7f5a81a00640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6006b2d0 0x7f5a6006d790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56235b7b4400 tx=0x7f5a74002410 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.484+0000 7f5a82400640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x56235b796770 0x56235b797240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.486+0000 7f5a82400640 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x56235b796770 0x56235b797240 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.489+0000 7f5a82400640 1 -- 172.21.15.59:0/1815939316 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v58'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5a7c065430 con 0x56235b796770 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.489+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x56235b796770 msgr2=0x56235b797240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.489+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x56235b796770 0x56235b797240 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6006b2d0 msgr2=0x7f5a6006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6006b2d0 0x7f5a6006d790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56235b7b4400 tx=0x7f5a74002410 comp rx=0 tx=0).stop 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 msgr2=0x56235b796230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 --2- 172.21.15.59:0/1815939316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56235b729450 0x56235b796230 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00be00 tx=0x7f5a6c00bef0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a82400640 1 -- 172.21.15.59:0/1815939316 reap_dead start 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 shutdown_connections 2024-04-24T06:32:18.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 >> 172.21.15.59:0/1815939316 conn(0x56235b70ce20 msgr2=0x56235b7b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 shutdown_connections 2024-04-24T06:32:18.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.490+0000 7f5a8481ef40 1 -- 172.21.15.59:0/1815939316 wait complete. 2024-04-24T06:32:18.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:18.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-24T06:32:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.691+0000 7f93ed200640 1 Processor -- start 2024-04-24T06:32:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.691+0000 7f93ed200640 1 -- start start 2024-04-24T06:32:18.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.693+0000 7f93ed200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.693+0000 7f93ed200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e80cfcf0 0x7f93e80d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.693+0000 7f93ed200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e80d0310 con 0x7f93e80d10a0 2024-04-24T06:32:18.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.693+0000 7f93ed200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f93e80d1a40 con 0x7f93e80cfcf0 2024-04-24T06:32:18.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.693+0000 7f93ed200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93e80d05f0 con 0x7f93e8161fd0 2024-04-24T06:32:18.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e80cfcf0 0x7f93e80d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e80cfcf0 0x7f93e80d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60322/0 (socket says 172.21.15.59:60322) 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 -- 172.21.15.59:0/4000982947 learned_addr learned my addr 172.21.15.59:0/4000982947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e5e00640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 -- 172.21.15.59:0/4000982947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 msgr2=0x7f93e80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 -- 172.21.15.59:0/4000982947 >> v1:172.21.15.66:6789/0 conn(0x7f93e80d10a0 legacy=0x7f93e80cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.694+0000 7f93e6800640 1 -- 172.21.15.59:0/4000982947 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e80d2520 con 0x7f93e80cfcf0 2024-04-24T06:32:18.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.695+0000 7f93e6800640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e80d1500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e50 tx=0x7f93e002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb4f16bd9dbe18b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.696+0000 7f93e5400640 1 -- 172.21.15.59:0/4000982947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0036070 con 0x7f93e80cfcf0 2024-04-24T06:32:18.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.696+0000 7f93e5400640 1 -- 172.21.15.59:0/4000982947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0031600 con 0x7f93e80cfcf0 2024-04-24T06:32:18.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.696+0000 7f93e5400640 1 -- 172.21.15.59:0/4000982947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e00319d0 con 0x7f93e80cfcf0 2024-04-24T06:32:18.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 -- 172.21.15.59:0/4000982947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 msgr2=0x7f93e80d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e80d1500 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e50 tx=0x7f93e002a6c0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 -- 172.21.15.59:0/4000982947 shutdown_connections 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e80d1500 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 --2- 172.21.15.59:0/4000982947 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 -- 172.21.15.59:0/4000982947 >> 172.21.15.59:0/4000982947 conn(0x7f93e808bf90 msgr2=0x7f93e80d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 -- 172.21.15.59:0/4000982947 shutdown_connections 2024-04-24T06:32:18.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.697+0000 7f93ed200640 1 -- 172.21.15.59:0/4000982947 wait complete. 2024-04-24T06:32:18.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.699+0000 7f93ed200640 1 Processor -- start 2024-04-24T06:32:18.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.699+0000 7f93ed200640 1 -- start start 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e80d10a0 0x7f93e80b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80d2d20 con 0x7f93e80cfcf0 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80b4610 con 0x7f93e80d10a0 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93ed200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93e80d2ea0 con 0x7f93e8161fd0 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60334/0 (socket says 172.21.15.59:60334) 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 -- 172.21.15.59:0/3693512910 learned_addr learned my addr 172.21.15.59:0/3693512910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e6800640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 -- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 msgr2=0x7f93e80b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93dfe00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e80d10a0 0x7f93e80b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.700+0000 7f93e5e00640 1 -- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e80d10a0 msgr2=0x7f93e80b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93e5e00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e80d10a0 0x7f93e80b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93e5e00640 1 -- 172.21.15.59:0/3693512910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e80ca8e0 con 0x7f93e80cfcf0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93e6800640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93e5e00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d620 tx=0x7f93d800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93de000640 1 -- 172.21.15.59:0/3693512910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800b620 con 0x7f93e80cfcf0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93de000640 1 -- 172.21.15.59:0/3693512910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d8019070 con 0x7f93e80cfcf0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93de000640 1 -- 172.21.15.59:0/3693512910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800ba00 con 0x7f93e80cfcf0 2024-04-24T06:32:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e80b7500 con 0x7f93e80cfcf0 2024-04-24T06:32:18.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.701+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e80b77a0 con 0x7f93e80cfcf0 2024-04-24T06:32:18.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.704+0000 7f93de000640 1 -- 172.21.15.59:0/3693512910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93d800bba0 con 0x7f93e80cfcf0 2024-04-24T06:32:18.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.705+0000 7f93de000640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.705+0000 7f93de000640 1 -- 172.21.15.59:0/3693512910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f93d808bf20 con 0x7f93e80cfcf0 2024-04-24T06:32:18.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.705+0000 7f93dfe00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.705+0000 7f93eff6f740 1 Processor -- start 2024-04-24T06:32:18.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 -- start start 2024-04-24T06:32:18.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93dfe00640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 0x7f93c406ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3060 tx=0x7f93d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c51c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f6df0 0x55f3c51cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f3c512ff50 con 0x55f3c51f7530 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3c50d62c0 con 0x55f3c5214410 2024-04-24T06:32:18.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.706+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55f3c512bf10 con 0x55f3c51f6df0 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60336/0 (socket says 172.21.15.59:60336) 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 -- 172.21.15.59:0/327948243 learned_addr learned my addr 172.21.15.59:0/327948243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93e6800640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f6df0 0x55f3c51cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93e5e00640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c51c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 -- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f6df0 msgr2=0x55f3c51cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f6df0 0x55f3c51cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 -- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 msgr2=0x55f3c51c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c51c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93dfe00640 1 -- 172.21.15.59:0/327948243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c4f0e7b0 con 0x55f3c51f7530 2024-04-24T06:32:18.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.707+0000 7f93e5e00640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c51c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.708+0000 7f93dfe00640 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005460 tx=0x7f93d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa25aefa177296a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.708+0000 7f93d6a00640 1 -- 172.21.15.59:0/327948243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d000d6a0 con 0x55f3c51f7530 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.708+0000 7f93d6a00640 1 -- 172.21.15.59:0/327948243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d000d840 con 0x55f3c51f7530 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.708+0000 7f93d6a00640 1 -- 172.21.15.59:0/327948243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d000dc30 con 0x55f3c51f7530 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 -- 172.21.15.59:0/327948243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 msgr2=0x55f3c51f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005460 tx=0x7f93d0010000 comp rx=0 tx=0).stop 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 -- 172.21.15.59:0/327948243 shutdown_connections 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f6df0 0x55f3c51cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f7530 0x55f3c51f68b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 --2- 172.21.15.59:0/327948243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c51c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 -- 172.21.15.59:0/327948243 >> 172.21.15.59:0/327948243 conn(0x55f3c51c9010 msgr2=0x55f3c52118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 -- 172.21.15.59:0/327948243 shutdown_connections 2024-04-24T06:32:18.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.709+0000 7f93eff6f740 1 -- 172.21.15.59:0/327948243 wait complete. 2024-04-24T06:32:18.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.710+0000 7f93eff6f740 1 -- start start 2024-04-24T06:32:18.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.710+0000 7f93eff6f740 1 Processor -- start 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c51fa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c520fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f3c51cb1c0 con 0x55f3c51f1950 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f3c50e9680 con 0x55f3c5214410 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55f3c51ca5b0 con 0x55f3c51f7530 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93eff6f740 10 client.0 _ll_register_callbacks cb 0x55f3c51264f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60352/0 (socket says 172.21.15.59:60352) 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.711+0000 7f93e6800640 1 -- 172.21.15.59:0/4089920248 learned_addr learned my addr 172.21.15.59:0/4089920248 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e5e00640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c51fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93dfe00640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c520fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 -- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 msgr2=0x55f3c51fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c51fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 -- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 msgr2=0x55f3c520fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c520fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 -- 172.21.15.59:0/4089920248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c4f6ef40 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93e6800640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009f70 tx=0x7f93e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bd572fc259cf56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93cb400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0041030 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.712+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f93e0031dd0 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d000d840 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93e0036040 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 10 client.6390 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f3c4e020a0 con 0x55f3c51f1950 2024-04-24T06:32:18.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d000d840 con 0x55f3c51f1950 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f93e004f080 con 0x55f3c51f1950 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 10 client.6390 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 10 client.6390 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 20 client.6390 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f8870 0x55f3c51f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 4 client.6390 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.713+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c513d970 con 0x55f3c51f8870 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.714+0000 7f93e5e00640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f8870 0x55f3c51f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.714+0000 7f93e5e00640 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f8870 0x55f3c51f8c50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809f04d80d743dea server_cookie=2d91c691c4c08c99 in_seq=0 out_seq=0 2024-04-24T06:32:18.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.714+0000 7f93cb400640 10 client.6390 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93cb400640 1 -- 172.21.15.59:0/4089920248 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3c513d970 con 0x55f3c51f8870 2024-04-24T06:32:18.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93cb400640 10 client.6390 handle_command_reply: tid=0 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 client.6390 shutdown 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 msgr2=0x55f3c51f1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009f70 tx=0x7f93e0004ab0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 shutdown_connections 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f8870 0x55f3c51f8c50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c51f1950 0x55f3c51f1dd0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c5214410 0x55f3c520fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93cb400640 0 client.6390 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 --2- 172.21.15.59:0/4089920248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c51fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.715+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 >> 172.21.15.59:0/4089920248 conn(0x55f3c51f6df0 msgr2=0x55f3c52118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.716+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 shutdown_connections 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.716+0000 7f93eff6f740 1 -- 172.21.15.59:0/4089920248 wait complete. 2024-04-24T06:32:18.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.716+0000 7f93eff6f740 20 client.6390 trim_cache size 0 max 16384 2024-04-24T06:32:18.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.736+0000 7f93eff6f740 1 Processor -- start 2024-04-24T06:32:18.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.736+0000 7f93eff6f740 1 -- start start 2024-04-24T06:32:18.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f1950 0x55f3c5221610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c52116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8200f20 con 0x55f3c5214410 2024-04-24T06:32:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80b91b0 con 0x55f3c51f1950 2024-04-24T06:32:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f93e81f8b80 con 0x55f3c51f7530 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60360/0 (socket says 172.21.15.59:60360) 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 -- 172.21.15.59:0/72334460 learned_addr learned my addr 172.21.15.59:0/72334460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93dfe00640 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c52116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 -- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 msgr2=0x55f3c52116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c52116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e6800640 1 -- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f1950 msgr2=0x55f3c5221610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.737+0000 7f93e5e00640 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f1950 0x55f3c5221610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93e6800640 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f1950 0x55f3c5221610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93e6800640 1 -- 172.21.15.59:0/72334460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c5211bf0 con 0x55f3c5214410 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93e6800640 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f93e002ae20 tx=0x7f93e003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadef748fd009290 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93d6000640 1 -- 172.21.15.59:0/72334460 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0041070 con 0x55f3c5214410 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93d6000640 1 -- 172.21.15.59:0/72334460 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0052020 con 0x55f3c5214410 2024-04-24T06:32:18.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.738+0000 7f93d6000640 1 -- 172.21.15.59:0/72334460 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003b90 con 0x55f3c5214410 2024-04-24T06:32:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 -- 172.21.15.59:0/72334460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 msgr2=0x55f3c5286410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f93e002ae20 tx=0x7f93e003d000 comp rx=0 tx=0).stop 2024-04-24T06:32:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 -- 172.21.15.59:0/72334460 shutdown_connections 2024-04-24T06:32:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c5286410 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f7530 0x55f3c52116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 --2- 172.21.15.59:0/72334460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f1950 0x55f3c5221610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 -- 172.21.15.59:0/72334460 >> 172.21.15.59:0/72334460 conn(0x55f3c51f6df0 msgr2=0x55f3c5208460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 -- 172.21.15.59:0/72334460 shutdown_connections 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.739+0000 7f93eff6f740 1 -- 172.21.15.59:0/72334460 wait complete. 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.740+0000 7f93eff6f740 1 -- start start 2024-04-24T06:32:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.740+0000 7f93eff6f740 1 Processor -- start 2024-04-24T06:32:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f1950 0x55f3c5283310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f7530 0x55f3c51fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f3c5209110 con 0x55f3c5214410 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80b8b20 con 0x55f3c51f7530 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55f3c51f9be0 con 0x55f3c51f1950 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93eff6f740 10 client.0 _ll_register_callbacks cb 0x7f93e0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60368/0 (socket says 172.21.15.59:60368) 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.741+0000 7f93e6800640 1 -- 172.21.15.59:0/3050439995 learned_addr learned my addr 172.21.15.59:0/3050439995 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 -- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f1950 msgr2=0x55f3c5283310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f1950 0x55f3c5283310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 -- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f7530 msgr2=0x55f3c51fa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f7530 0x55f3c51fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 -- 172.21.15.59:0/3050439995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c51f9360 con 0x55f3c5214410 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93e6800640 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f93e002acb0 tx=0x7f93e0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a00b2c19fadb0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93cb400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0041050 con 0x55f3c5214410 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f93e0004030 con 0x55f3c5214410 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.742+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e0052020 con 0x55f3c5214410 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93e0065050 con 0x55f3c5214410 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 10 client.6396 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:18.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f3c51f3de0 con 0x55f3c5214410 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0052020 con 0x55f3c5214410 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f93e0068020 con 0x55f3c5214410 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 10 client.6396 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 10 client.6396 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 20 client.6396 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f0970 0x55f3c51ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 4 client.6396 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55f3c51377e0 con 0x55f3c51f0970 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.743+0000 7f93e5e00640 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f0970 0x55f3c51ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.744+0000 7f93e5e00640 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f0970 0x55f3c51ccb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3464284a687d80 server_cookie=9723f90b18d904c1 in_seq=0 out_seq=0 2024-04-24T06:32:18.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.744+0000 7f93cb400640 10 client.6396 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93cb400640 1 -- 172.21.15.59:0/3050439995 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3c51377e0 con 0x55f3c51f0970 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93cb400640 10 client.6396 handle_command_reply: tid=0 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 client.6396 shutdown 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 msgr2=0x55f3c51cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f93e002acb0 tx=0x7f93e0009810 comp rx=0 tx=0).stop 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 shutdown_connections 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55f3c51f0970 0x55f3c51ccb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f3c5214410 0x55f3c51cc5c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f3c51f7530 0x55f3c51fa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 --2- 172.21.15.59:0/3050439995 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f3c51f1950 0x55f3c5283310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 >> 172.21.15.59:0/3050439995 conn(0x55f3c51f6df0 msgr2=0x55f3c5208460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93cb400640 0 client.6396 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 shutdown_connections 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 1 -- 172.21.15.59:0/3050439995 wait complete. 2024-04-24T06:32:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.745+0000 7f93eff6f740 20 client.6396 trim_cache size 0 max 16384 2024-04-24T06:32:18.750 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f5446800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6351=0-4194304@1} caps={6351=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6351 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55d8a99c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:32:18.750 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:32:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "0d892c50-e384-4f0d-82f3-2b7f28bb3fec", 2024-04-24T06:32:18.750 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:32:18.750 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:18.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 msgr2=0x7f93c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 0x7f93c406ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3060 tx=0x7f93d0009040 comp rx=0 tx=0).stop 2024-04-24T06:32:18.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 msgr2=0x7f93e814fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f93d800d620 tx=0x7f93d800dae0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 shutdown_connections 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f93e8161fd0 0x7f93e80b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93e80d10a0 0x7f93e80b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 --2- 172.21.15.59:0/3693512910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e80cfcf0 0x7f93e814fb70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.746+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 >> 172.21.15.59:0/3693512910 conn(0x7f93e808bf90 msgr2=0x7f93e806cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.747+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 shutdown_connections 2024-04-24T06:32:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.747+0000 7f93ed200640 1 -- 172.21.15.59:0/3693512910 wait complete. 2024-04-24T06:32:18.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:18.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T06:32:18.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.967+0000 7ff303000640 1 Processor -- start 2024-04-24T06:32:18.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.968+0000 7ff303000640 1 -- start start 2024-04-24T06:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.969+0000 7ff303000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 0x7ff2fc0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.969+0000 7ff303000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2fc0d0b10 0x7ff2fc0af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.969+0000 7ff303000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff303000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2fc0d0f70 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff303000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff2fc160120 con 0x7ff2fc15c410 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff303000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc160400 con 0x7ff2fc0af5c0 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff301200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2fc0d0b10 0x7ff2fc0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff301200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2fc0d0b10 0x7ff2fc0af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39916/0 (socket says 172.21.15.59:39916) 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff301200640 1 -- 172.21.15.59:0/563033555 learned_addr learned my addr 172.21.15.59:0/563033555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff302600640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.970+0000 7ff301c00640 1 --2- 172.21.15.59:0/563033555 >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 0x7ff2fc0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301200640 1 -- 172.21.15.59:0/563033555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 msgr2=0x7ff2fc0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301200640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301200640 1 -- 172.21.15.59:0/563033555 >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 msgr2=0x7ff2fc0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301200640 1 --2- 172.21.15.59:0/563033555 >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 0x7ff2fc0d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301200640 1 -- 172.21.15.59:0/563033555 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc1606e0 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.971+0000 7ff301c00640 1 --2- 172.21.15.59:0/563033555 >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 0x7ff2fc0d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.972+0000 7ff301200640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0af080 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0741a0 tx=0x7ff2e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1d0b82674ce0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.972+0000 7ff2fbe00640 1 -- 172.21.15.59:0/563033555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8037070 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.972+0000 7ff2fbe00640 1 -- 172.21.15.59:0/563033555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e8032440 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.972+0000 7ff2fbe00640 1 -- 172.21.15.59:0/563033555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8032810 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 -- 172.21.15.59:0/563033555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 msgr2=0x7ff2fc0af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0af080 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0741a0 tx=0x7ff2e802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 -- 172.21.15.59:0/563033555 shutdown_connections 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 --2- 172.21.15.59:0/563033555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0af080 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 --2- 172.21.15.59:0/563033555 >> v2:172.21.15.66:3300/0 conn(0x7ff2fc15c410 0x7ff2fc0d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 -- 172.21.15.59:0/563033555 >> 172.21.15.59:0/563033555 conn(0x7ff2fc06bdf0 msgr2=0x7ff2fc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 -- 172.21.15.59:0/563033555 shutdown_connections 2024-04-24T06:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.973+0000 7ff303000640 1 -- 172.21.15.59:0/563033555 wait complete. 2024-04-24T06:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.975+0000 7ff303000640 1 Processor -- start 2024-04-24T06:32:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.975+0000 7ff303000640 1 -- start start 2024-04-24T06:32:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2fc15c410 0x7ff2fc15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc15c7f0 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0afa30 con 0x7ff2fc15c410 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff303000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc1f6ba0 con 0x7ff2fc0af5c0 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39928/0 (socket says 172.21.15.59:39928) 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 -- 172.21.15.59:0/378680273 learned_addr learned my addr 172.21.15.59:0/378680273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 -- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 msgr2=0x7ff2fc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 -- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2fc15c410 msgr2=0x7ff2fc15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff302600640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2fc15c410 0x7ff2fc15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.976+0000 7ff301200640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2fc15c410 0x7ff2fc15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff301200640 1 -- 172.21.15.59:0/378680273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc1f6e80 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff301200640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802a220 tx=0x7ff2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff2f9600640 1 -- 172.21.15.59:0/378680273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8041020 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff2f9600640 1 -- 172.21.15.59:0/378680273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e80306c0 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff2f9600640 1 -- 172.21.15.59:0/378680273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8037040 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.977+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc1f7160 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.978+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1f7720 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.980+0000 7ff2f9600640 1 -- 172.21.15.59:0/378680273 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2e8044020 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.981+0000 7ff2f9600640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 0x7ff2dc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.981+0000 7ff2f9600640 1 -- 172.21.15.59:0/378680273 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e80ac360 con 0x7ff2fc0d0b10 2024-04-24T06:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.981+0000 7ff301c00640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 0x7ff2dc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.982+0000 7ff30742b740 1 Processor -- start 2024-04-24T06:32:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.982+0000 7ff30742b740 1 -- start start 2024-04-24T06:32:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.982+0000 7ff301c00640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 0x7ff2dc06ee00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0008ea0 tx=0x7ff2f00083d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e069e00 0x55d04e01c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04c7e0 0x55d04e0215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d04df861f0 con 0x55d04e04cf20 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d04df2c350 con 0x55d04e04c7e0 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55d04df821b0 con 0x55d04e069e00 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39934/0 (socket says 172.21.15.59:39934) 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.983+0000 7ff301200640 1 -- 172.21.15.59:0/1418279218 learned_addr learned my addr 172.21.15.59:0/1418279218 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 -- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e069e00 msgr2=0x55d04e01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301c00640 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e069e00 0x55d04e01c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e069e00 0x55d04e01c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 -- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04c7e0 msgr2=0x55d04e0215f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04c7e0 0x55d04e0215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 -- 172.21.15.59:0/1418279218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04df80f50 con 0x55d04e04cf20 2024-04-24T06:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff301200640 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8079e00 tx=0x7ff2e807a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268b692bfb1e958e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff2ee000640 1 -- 172.21.15.59:0/1418279218 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e807e4b0 con 0x55d04e04cf20 2024-04-24T06:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff2ee000640 1 -- 172.21.15.59:0/1418279218 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e8082ce0 con 0x55d04e04cf20 2024-04-24T06:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.984+0000 7ff2ee000640 1 -- 172.21.15.59:0/1418279218 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80ac0b0 con 0x55d04e04cf20 2024-04-24T06:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 -- 172.21.15.59:0/1418279218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 msgr2=0x55d04e04c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8079e00 tx=0x7ff2e807a800 comp rx=0 tx=0).stop 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 -- 172.21.15.59:0/1418279218 shutdown_connections 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04c7e0 0x55d04e0215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e04c2a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 --2- 172.21.15.59:0/1418279218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e069e00 0x55d04e01c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 -- 172.21.15.59:0/1418279218 >> 172.21.15.59:0/1418279218 conn(0x55d04e01ea00 msgr2=0x55d04e0672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 -- 172.21.15.59:0/1418279218 shutdown_connections 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.985+0000 7ff30742b740 1 -- 172.21.15.59:0/1418279218 wait complete. 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.986+0000 7ff30742b740 1 -- start start 2024-04-24T06:32:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.986+0000 7ff30742b740 1 Processor -- start 2024-04-24T06:32:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04cf20 0x55d04e0692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e04bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d04df861f0 con 0x55d04e069e00 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d04df3f690 con 0x55d04e04cf20 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55d04e020bb0 con 0x55d04e07a1a0 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.987+0000 7ff30742b740 10 client.0 _ll_register_callbacks cb 0x55d04df7b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39942/0 (socket says 172.21.15.59:39942) 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 -- 172.21.15.59:0/2604025376 learned_addr learned my addr 172.21.15.59:0/2604025376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff302600640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e04bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301c00640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04cf20 0x55d04e0692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 -- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 msgr2=0x55d04e04bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e04bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 -- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04cf20 msgr2=0x55d04e0692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04cf20 0x55d04e0692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 -- 172.21.15.59:0/2604025376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04df3e920 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff302600640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e04bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff301200640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2e807a9b0 tx=0x7ff2e803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff90c8eef6da071 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.988+0000 7ff2e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e807e170 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e8082850 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e8082ce0 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff2e80a0050 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 10 client.6408 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55d04dd627d0 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e8082ce0 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7ff2e80a3020 con 0x55d04e069e00 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 10 client.6408 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 10 client.6408 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 20 client.6408 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0786b0 0x55d04e078a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 4 client.6408 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.989+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04df93c10 con 0x55d04e0786b0 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.990+0000 7ff301c00640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0786b0 0x55d04e078a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.990+0000 7ff301c00640 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0786b0 0x55d04e078a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0c6f5cac963f5d server_cookie=b465379770b75e86 in_seq=0 out_seq=0 2024-04-24T06:32:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.990+0000 7ff2e2a00640 10 client.6408 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff2e2a00640 1 -- 172.21.15.59:0/2604025376 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d04df93c10 con 0x55d04e0786b0 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff2e2a00640 10 client.6408 handle_command_reply: tid=0 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 client.6408 shutdown 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 msgr2=0x55d04e04b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2e807a9b0 tx=0x7ff2e803e000 comp rx=0 tx=0).stop 2024-04-24T06:32:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 shutdown_connections 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0786b0 0x55d04e078a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e04bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e069e00 0x55d04e04b580 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff2e2a00640 0 client.6408 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 --2- 172.21.15.59:0/2604025376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e04cf20 0x55d04e0692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.991+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 >> 172.21.15.59:0/2604025376 conn(0x55d04e04c7e0 msgr2=0x55d04e0201b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.992+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 shutdown_connections 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.992+0000 7ff30742b740 1 -- 172.21.15.59:0/2604025376 wait complete. 2024-04-24T06:32:18.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:18.992+0000 7ff30742b740 20 client.6408 trim_cache size 0 max 16384 2024-04-24T06:32:19.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.011+0000 7ff30742b740 1 Processor -- start 2024-04-24T06:32:19.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.011+0000 7ff30742b740 1 -- start start 2024-04-24T06:32:19.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 0x55d04e01ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e0204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc200b20 con 0x55d04e07a1a0 2024-04-24T06:32:19.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1f5460 con 0x55d04e069e00 2024-04-24T06:32:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc15dd10 con 0x55d04e04cf20 2024-04-24T06:32:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.012+0000 7ff302600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39956/0 (socket says 172.21.15.59:39956) 2024-04-24T06:32:19.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 -- 172.21.15.59:0/1937980976 learned_addr learned my addr 172.21.15.59:0/1937980976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:19.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff301c00640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 0x55d04e01ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff301200640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e0204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 -- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 msgr2=0x55d04e01ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 0x55d04e01ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 -- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 msgr2=0x55d04e0204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e0204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 -- 172.21.15.59:0/1937980976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04dc983f0 con 0x55d04e07a1a0 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff301c00640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 0x55d04e01ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.013+0000 7ff302600640 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d520 tx=0x7ff2f400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c120954b825c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff2ed600640 1 -- 172.21.15.59:0/1937980976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4018070 con 0x55d04e07a1a0 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff2ed600640 1 -- 172.21.15.59:0/1937980976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4004220 con 0x55d04e07a1a0 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff2ed600640 1 -- 172.21.15.59:0/1937980976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003610 con 0x55d04e07a1a0 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 -- 172.21.15.59:0/1937980976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 msgr2=0x55d04e06a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d520 tx=0x7ff2f400d9e0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 -- 172.21.15.59:0/1937980976 shutdown_connections 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e07a1a0 0x55d04e06a6c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e0204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 --2- 172.21.15.59:0/1937980976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e04cf20 0x55d04e01ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 -- 172.21.15.59:0/1937980976 >> 172.21.15.59:0/1937980976 conn(0x55d04e04c7e0 msgr2=0x55d04e01b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 -- 172.21.15.59:0/1937980976 shutdown_connections 2024-04-24T06:32:19.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.014+0000 7ff30742b740 1 -- 172.21.15.59:0/1937980976 wait complete. 2024-04-24T06:32:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.015+0000 7ff30742b740 1 -- start start 2024-04-24T06:32:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.015+0000 7ff30742b740 1 Processor -- start 2024-04-24T06:32:19.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e089980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e089ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1f4e10 con 0x55d04e04cf20 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d04e079fb0 con 0x55d04e069e00 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55d04e01d6d0 con 0x55d04e07a1a0 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.016+0000 7ff30742b740 10 client.0 _ll_register_callbacks cb 0x7ff2e8091070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39964/0 (socket says 172.21.15.59:39964) 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 -- 172.21.15.59:0/477324920 learned_addr learned my addr 172.21.15.59:0/477324920 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff302600640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e089ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 -- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 msgr2=0x55d04e089ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e089ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 -- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 msgr2=0x55d04e089980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:19.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e089980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 -- 172.21.15.59:0/477324920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04dc580f0 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff301c00640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e5f0 tx=0x7ff2f0005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e085403517f852 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.017+0000 7ff2e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0023e40 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff2f00232e0 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4004220 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff2f0031050 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 10 client.6414 fetch_fsmap learned FSMap version 33 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55d04df4b810 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4004220 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7ff2f0034020 con 0x55d04e04cf20 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 10 client.6414 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T06:32:19.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:19.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 20 client.6414 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:19.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0ce5f0 0x55d04e06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:19.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 4 client.6414 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:32:19.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.018+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d04df8da80 con 0x55d04e0ce5f0 2024-04-24T06:32:19.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff301200640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0ce5f0 0x55d04e06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff301200640 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0ce5f0 0x55d04e06a6c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb1af4ef815ef2c server_cookie=439fe2d25d4a52aa in_seq=0 out_seq=0 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff2e2a00640 10 client.6414 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff2e2a00640 1 -- 172.21.15.59:0/477324920 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x55d04df8da80 con 0x55d04e0ce5f0 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff2e2a00640 10 client.6414 handle_command_reply: tid=0 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.019+0000 7ff30742b740 1 client.6414 shutdown 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 msgr2=0x55d04e021b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:19.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000e5f0 tx=0x7ff2f0005d10 comp rx=0 tx=0).stop 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 shutdown_connections 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55d04e0ce5f0 0x55d04e06a6c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55d04e07a1a0 0x55d04e089ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04e069e00 0x55d04e089980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 --2- 172.21.15.59:0/477324920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d04e04cf20 0x55d04e021b30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff2e2a00640 0 client.6414 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 >> 172.21.15.59:0/477324920 conn(0x55d04e04c7e0 msgr2=0x55d04e01ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 shutdown_connections 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 1 -- 172.21.15.59:0/477324920 wait complete. 2024-04-24T06:32:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.020+0000 7ff30742b740 20 client.6414 trim_cache size 0 max 16384 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "0d892c50-e384-4f0d-82f3-2b7f28bb3fec": { 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "0d892c50-e384-4f0d-82f3-2b7f28bb3fec", 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:19.026 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 msgr2=0x7ff2dc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 0x7ff2dc06ee00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0008ea0 tx=0x7ff2f00083d0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 msgr2=0x7ff2fc0b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802a220 tx=0x7ff2e80046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 shutdown_connections 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff2dc06c940 0x7ff2dc06ee00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2fc15c410 0x7ff2fc15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2fc0d0b10 0x7ff2fc0b0ff0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 --2- 172.21.15.59:0/378680273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2fc0af5c0 0x7ff2fc0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 >> 172.21.15.59:0/378680273 conn(0x7ff2fc06bdf0 msgr2=0x7ff2fc15a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 shutdown_connections 2024-04-24T06:32:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:19.021+0000 7ff303000640 1 -- 172.21.15.59:0/378680273 wait complete. 2024-04-24T06:32:19.041 INFO:tasks.cephfs.filesystem:scrub status for tag:0d892c50-e384-4f0d-82f3-2b7f28bb3fec - {'path': '/', 'tag': '0d892c50-e384-4f0d-82f3-2b7f28bb3fec', 'options': 'recursive'} 2024-04-24T06:32:24.331 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:32:24.330+0000 7f6490400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T06:32:35.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:32:35.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:32:49.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:49.042 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T06:32:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.264+0000 7f92d4a00640 1 Processor -- start 2024-04-24T06:32:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.265+0000 7f92d4a00640 1 -- start start 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.266+0000 7f92d4a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92d015c410 0x7f92d00d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.266+0000 7f92d4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f92d00af790 0x7f92d0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.266+0000 7f92d4a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d0160120 con 0x7f92d00d0b10 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.266+0000 7f92d4a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f92d00afc00 con 0x7f92d00af790 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.266+0000 7f92d4a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f92d00d0f70 con 0x7f92d015c410 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f92d00af790 0x7f92d0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f92d00af790 0x7f92d0071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45016/0 (socket says 172.21.15.59:45016) 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 -- 172.21.15.59:0/1164460097 learned_addr learned my addr 172.21.15.59:0/1164460097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92ce800640 1 --2- 172.21.15.59:0/1164460097 >> v2:172.21.15.66:3300/0 conn(0x7f92d015c410 0x7f92d00d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 -- 172.21.15.59:0/1164460097 >> v2:172.21.15.66:3300/0 conn(0x7f92d015c410 msgr2=0x7f92d00d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 --2- 172.21.15.59:0/1164460097 >> v2:172.21.15.66:3300/0 conn(0x7f92d015c410 0x7f92d00d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 -- 172.21.15.59:0/1164460097 >> v1:172.21.15.59:6789/0 conn(0x7f92d00d0b10 legacy=0x7f92d00af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.267+0000 7f92cf200640 1 -- 172.21.15.59:0/1164460097 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d0160400 con 0x7f92d00af790 2024-04-24T06:32:49.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.268+0000 7f92cf200640 1 --2- 172.21.15.59:0/1164460097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0071c70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f92d00741a0 tx=0x7f92c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7b90e502dec6aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92cd400640 1 -- 172.21.15.59:0/1164460097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8037070 con 0x7f92d00af790 2024-04-24T06:32:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92cd400640 1 -- 172.21.15.59:0/1164460097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c8032440 con 0x7f92d00af790 2024-04-24T06:32:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92cd400640 1 -- 172.21.15.59:0/1164460097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8032810 con 0x7f92d00af790 2024-04-24T06:32:49.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92d4a00640 1 -- 172.21.15.59:0/1164460097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 msgr2=0x7f92d0071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92d4a00640 1 --2- 172.21.15.59:0/1164460097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0071c70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f92d00741a0 tx=0x7f92c802a270 comp rx=0 tx=0).stop 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92d4a00640 1 -- 172.21.15.59:0/1164460097 shutdown_connections 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92d4a00640 1 --2- 172.21.15.59:0/1164460097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0071c70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.269+0000 7f92d4a00640 1 --2- 172.21.15.59:0/1164460097 >> v2:172.21.15.66:3300/0 conn(0x7f92d015c410 0x7f92d00d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.270+0000 7f92d4a00640 1 -- 172.21.15.59:0/1164460097 >> 172.21.15.59:0/1164460097 conn(0x7f92d006bdf0 msgr2=0x7f92d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.270+0000 7f92d4a00640 1 -- 172.21.15.59:0/1164460097 shutdown_connections 2024-04-24T06:32:49.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.270+0000 7f92d4a00640 1 -- 172.21.15.59:0/1164460097 wait complete. 2024-04-24T06:32:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.271+0000 7f92d4a00640 1 Processor -- start 2024-04-24T06:32:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.271+0000 7f92d4a00640 1 -- start start 2024-04-24T06:32:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92d00d0b10 0x7f92d01537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d015c410 0x7f92d0153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92d015c7f0 con 0x7f92d00af790 2024-04-24T06:32:49.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01609c0 con 0x7f92d015c410 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.272+0000 7f92d4a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92d00ced80 con 0x7f92d00d0b10 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45030/0 (socket says 172.21.15.59:45030) 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 -- 172.21.15.59:0/2048861446 learned_addr learned my addr 172.21.15.59:0/2048861446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92cf200640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d015c410 0x7f92d0153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92cde00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92d00d0b10 0x7f92d01537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 -- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92d00d0b10 msgr2=0x7f92d01537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92d00d0b10 0x7f92d01537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 -- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d015c410 msgr2=0x7f92d0153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d015c410 0x7f92d0153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.273+0000 7f92ce800640 1 -- 172.21.15.59:0/2048861446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d00cf2b0 con 0x7f92d00af790 2024-04-24T06:32:49.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92ce800640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d5f0 tx=0x7f92c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92c6a00640 1 -- 172.21.15.59:0/2048861446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00125f0 con 0x7f92d00af790 2024-04-24T06:32:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92c6a00640 1 -- 172.21.15.59:0/2048861446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c0019070 con 0x7f92d00af790 2024-04-24T06:32:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92c6a00640 1 -- 172.21.15.59:0/2048861446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00129d0 con 0x7f92d00af790 2024-04-24T06:32:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d0155410 con 0x7f92d00af790 2024-04-24T06:32:49.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.274+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d00aff30 con 0x7f92d00af790 2024-04-24T06:32:49.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.276+0000 7f92c6a00640 1 -- 172.21.15.59:0/2048861446 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92c0012b70 con 0x7f92d00af790 2024-04-24T06:32:49.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.277+0000 7f92c6a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 0x7f92ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.277+0000 7f92c6a00640 1 -- 172.21.15.59:0/2048861446 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f92c008bf40 con 0x7f92d00af790 2024-04-24T06:32:49.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.278+0000 7f92cde00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 0x7f92ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.278+0000 7f92d816a740 1 Processor -- start 2024-04-24T06:32:49.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.278+0000 7f92d816a740 1 -- start start 2024-04-24T06:32:49.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.278+0000 7f92cde00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 0x7f92ac06ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f92b8007fd0 tx=0x7f92b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x56230848d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 0x5623084925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5623083f71f0 con 0x5623084bdf20 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56230839d350 con 0x5623084bd7e0 2024-04-24T06:32:49.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5623083f31b0 con 0x5623084dae00 2024-04-24T06:32:49.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92cde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92cde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45038/0 (socket says 172.21.15.59:45038) 2024-04-24T06:32:49.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.279+0000 7f92cde00640 1 -- 172.21.15.59:0/440677645 learned_addr learned my addr 172.21.15.59:0/440677645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92ce800640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x56230848d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cf200640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 0x5623084925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 -- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 msgr2=0x56230848d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x56230848d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 -- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 msgr2=0x5623084925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 0x5623084925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 -- 172.21.15.59:0/440677645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623083f1f50 con 0x5623084bdf20 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cf200640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 0x5623084925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92cde00640 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f92b800dbc0 tx=0x7f92b8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2a1f259020e863 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.280+0000 7f92bf000640 1 -- 172.21.15.59:0/440677645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8024070 con 0x5623084bdf20 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.281+0000 7f92bf000640 1 -- 172.21.15.59:0/440677645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b801f400 con 0x5623084bdf20 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.281+0000 7f92bf000640 1 -- 172.21.15.59:0/440677645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b801f7b0 con 0x5623084bdf20 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.281+0000 7f92d816a740 1 -- 172.21.15.59:0/440677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 msgr2=0x5623084bd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.281+0000 7f92d816a740 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f92b800dbc0 tx=0x7f92b8010000 comp rx=0 tx=0).stop 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 -- 172.21.15.59:0/440677645 shutdown_connections 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bd7e0 0x5623084925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084bdf20 0x5623084bd2a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 --2- 172.21.15.59:0/440677645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x56230848d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 -- 172.21.15.59:0/440677645 >> 172.21.15.59:0/440677645 conn(0x56230848fa00 msgr2=0x5623084d82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 -- 172.21.15.59:0/440677645 shutdown_connections 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 -- 172.21.15.59:0/440677645 wait complete. 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.282+0000 7f92d816a740 1 -- start start 2024-04-24T06:32:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.283+0000 7f92d816a740 1 Processor -- start 2024-04-24T06:32:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623084bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x562308493490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562308491bb0 con 0x5623084ec540 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5623083b0690 con 0x5623084bdf20 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562308490fa0 con 0x5623084dae00 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92d816a740 10 client.0 _ll_register_callbacks cb 0x5623083ec650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45054/0 (socket says 172.21.15.59:45054) 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92cf200640 1 -- 172.21.15.59:0/2531385342 learned_addr learned my addr 172.21.15.59:0/2531385342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92ce800640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623084bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.284+0000 7f92cde00640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x562308493490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 -- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 msgr2=0x562308493490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x562308493490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 -- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 msgr2=0x5623084bfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623084bfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 -- 172.21.15.59:0/2531385342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623083af920 con 0x5623084ec540 2024-04-24T06:32:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92cf200640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f92c802a2b0 tx=0x7f92c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ac1354efa4e030 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92b7400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:49.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.285+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8043070 con 0x5623084ec540 2024-04-24T06:32:49.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92b801f400 con 0x5623084ec540 2024-04-24T06:32:49.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f92c803c440 con 0x5623084ec540 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f92c804e050 con 0x5623084ec540 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92d816a740 10 client.6429 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5623081d37d0 con 0x5623084ec540 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b801f400 con 0x5623084ec540 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f92c8051020 con 0x5623084ec540 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92d816a740 10 client.6429 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.286+0000 7f92d816a740 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.287+0000 7f92d816a740 20 client.6429 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.287+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084be8e0 0x5623084bece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.287+0000 7f92d816a740 4 client.6429 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.287+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562308404c10 con 0x5623084be8e0 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.287+0000 7f92ce800640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084be8e0 0x5623084bece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.288+0000 7f92ce800640 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084be8e0 0x5623084bece0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36910087793038f0 server_cookie=1792a9ba5d3fd93e in_seq=0 out_seq=0 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.288+0000 7f92b7400640 10 client.6429 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.288+0000 7f92b7400640 1 -- 172.21.15.59:0/2531385342 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562308404c10 con 0x5623084be8e0 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.288+0000 7f92b7400640 10 client.6429 handle_command_reply: tid=0 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.288+0000 7f92d816a740 1 client.6429 shutdown 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 msgr2=0x5623084eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f92c802a2b0 tx=0x7f92c8004730 comp rx=0 tx=0).stop 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 shutdown_connections 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084be8e0 0x5623084bece0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084eac80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x562308493490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 --2- 172.21.15.59:0/2531385342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623084bfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92b7400640 0 client.6429 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 >> 172.21.15.59:0/2531385342 conn(0x5623084bd7e0 msgr2=0x5623084d93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 shutdown_connections 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 1 -- 172.21.15.59:0/2531385342 wait complete. 2024-04-24T06:32:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.289+0000 7f92d816a740 20 client.6429 trim_cache size 0 max 16384 2024-04-24T06:32:49.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.309+0000 7f92d816a740 1 Processor -- start 2024-04-24T06:32:49.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.309+0000 7f92d816a740 1 -- start start 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084bdf20 0x5623084fb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 0x5623084db6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01fc880 con 0x5623084ec540 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01fba20 con 0x5623084dae00 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92d01f44a0 con 0x5623084bdf20 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45056/0 (socket says 172.21.15.59:45056) 2024-04-24T06:32:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 -- 172.21.15.59:0/2927492380 learned_addr learned my addr 172.21.15.59:0/2927492380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cde00640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 0x5623084db6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92ce800640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084bdf20 0x5623084fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 -- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084bdf20 msgr2=0x5623084fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084bdf20 0x5623084fb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 -- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 msgr2=0x5623084db6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 0x5623084db6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cf200640 1 -- 172.21.15.59:0/2927492380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623081093f0 con 0x5623084ec540 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.310+0000 7f92cde00640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 0x5623084db6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.311+0000 7f92cf200640 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f92c8001eb0 tx=0x7f92c803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276e6f78e8f7cfb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.311+0000 7f92be600640 1 -- 172.21.15.59:0/2927492380 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8043040 con 0x5623084ec540 2024-04-24T06:32:49.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.311+0000 7f92be600640 1 -- 172.21.15.59:0/2927492380 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c803c440 con 0x5623084ec540 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.311+0000 7f92be600640 1 -- 172.21.15.59:0/2927492380 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8037040 con 0x5623084ec540 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 -- 172.21.15.59:0/2927492380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 msgr2=0x5623084dbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f92c8001eb0 tx=0x7f92c803c000 comp rx=0 tx=0).stop 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 -- 172.21.15.59:0/2927492380 shutdown_connections 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084dbc00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084dae00 0x5623084db6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 --2- 172.21.15.59:0/2927492380 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084bdf20 0x5623084fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 -- 172.21.15.59:0/2927492380 >> 172.21.15.59:0/2927492380 conn(0x5623084bd7e0 msgr2=0x5623084b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 -- 172.21.15.59:0/2927492380 shutdown_connections 2024-04-24T06:32:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.312+0000 7f92d816a740 1 -- 172.21.15.59:0/2927492380 wait complete. 2024-04-24T06:32:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.313+0000 7f92d816a740 1 -- start start 2024-04-24T06:32:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.313+0000 7f92d816a740 1 Processor -- start 2024-04-24T06:32:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623085522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x5623084ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5623084896b0 con 0x5623084ec540 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00b1190 con 0x5623084bdf20 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5623084c12f0 con 0x5623084dae00 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92d816a740 10 client.0 _ll_register_callbacks cb 0x7f92c8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45062/0 (socket says 172.21.15.59:45062) 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.314+0000 7f92cf200640 1 -- 172.21.15.59:0/1506988288 learned_addr learned my addr 172.21.15.59:0/1506988288 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 -- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 msgr2=0x5623084ba4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x5623084ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92ce800640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623085522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 -- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 msgr2=0x5623085522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623085522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 -- 172.21.15.59:0/1506988288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623080c90f0 con 0x5623084ec540 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92cf200640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f92c804b040 tx=0x7f92c8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5064a6973e58c5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92b7400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8037040 con 0x5623084ec540 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f92c803ca00 con 0x5623084ec540 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.315+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c803c440 con 0x5623084ec540 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f92c8061050 con 0x5623084ec540 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 10 client.6435 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5623083bc810 con 0x5623084ec540 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c803c440 con 0x5623084ec540 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f92c8064020 con 0x5623084ec540 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 10 client.6435 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 10 client.6435 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 20 client.6435 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084da4e0 0x5623084d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 4 client.6435 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5623083fea80 con 0x5623084da4e0 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.316+0000 7f92ce800640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084da4e0 0x5623084d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92ce800640 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084da4e0 0x5623084d5630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963692d1d8ea4c54 server_cookie=bd85652a0b57926 in_seq=0 out_seq=0 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92b7400640 10 client.6435 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92b7400640 1 -- 172.21.15.59:0/1506988288 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5623083fea80 con 0x5623084da4e0 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92b7400640 10 client.6435 handle_command_reply: tid=0 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 client.6435 shutdown 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 msgr2=0x5623084d50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f92c804b040 tx=0x7f92c8009240 comp rx=0 tx=0).stop 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 shutdown_connections 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5623084da4e0 0x5623084d5630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5623084ec540 0x5623084d50f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5623084dae00 0x5623084ba4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92d816a740 1 --2- 172.21.15.59:0/1506988288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623084bdf20 0x5623085522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.317+0000 7f92b7400640 0 client.6435 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.318+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 >> 172.21.15.59:0/1506988288 conn(0x5623084bd7e0 msgr2=0x5623084db520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.318+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 shutdown_connections 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.318+0000 7f92d816a740 1 -- 172.21.15.59:0/1506988288 wait complete. 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.318+0000 7f92d816a740 20 client.6435 trim_cache size 0 max 16384 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:32:49.322 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 msgr2=0x7f92ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 0x7f92ac06ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f92b8007fd0 tx=0x7f92b8009040 comp rx=0 tx=0).stop 2024-04-24T06:32:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 msgr2=0x7f92d0153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d5f0 tx=0x7f92c000dab0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 shutdown_connections 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f92ac06c920 0x7f92ac06ede0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d015c410 0x7f92d0153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92d00d0b10 0x7f92d01537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 --2- 172.21.15.59:0/2048861446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d00af790 0x7f92d0153090 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 >> 172.21.15.59:0/2048861446 conn(0x7f92d006bdf0 msgr2=0x7f92d0072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 shutdown_connections 2024-04-24T06:32:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.319+0000 7f92d4a00640 1 -- 172.21.15.59:0/2048861446 wait complete. 2024-04-24T06:32:49.337 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:32:49.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:32:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.533+0000 7f37f9800640 1 Processor -- start 2024-04-24T06:32:49.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.534+0000 7f37f9800640 1 -- start start 2024-04-24T06:32:49.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.535+0000 7f37f9800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37f40afaf0 0x7f37f40d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.535+0000 7f37f9800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f37f40d06e0 0x7f37f41624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.535+0000 7f37f9800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f415f0f0 con 0x7f37f415c5e0 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.535+0000 7f37f9800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f37f415fa20 con 0x7f37f40afaf0 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.535+0000 7f37f9800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f37f415f3d0 con 0x7f37f40d06e0 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37f40afaf0 0x7f37f40d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37f40afaf0 0x7f37f40d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45066/0 (socket says 172.21.15.59:45066) 2024-04-24T06:32:49.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 -- 172.21.15.59:0/720032307 learned_addr learned my addr 172.21.15.59:0/720032307 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 -- 172.21.15.59:0/720032307 >> v2:172.21.15.66:3300/0 conn(0x7f37f40d06e0 msgr2=0x7f37f41624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 --2- 172.21.15.59:0/720032307 >> v2:172.21.15.66:3300/0 conn(0x7f37f40d06e0 0x7f37f41624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 -- 172.21.15.59:0/720032307 >> v1:172.21.15.66:6789/0 conn(0x7f37f415c5e0 legacy=0x7f37f40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.536+0000 7f37ea000640 1 -- 172.21.15.59:0/720032307 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f415f6b0 con 0x7f37f40afaf0 2024-04-24T06:32:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.537+0000 7f37ea000640 1 --2- 172.21.15.59:0/720032307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f40afaf0 0x7f37f40d01a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f37f406c7c0 tx=0x7f37dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb372451d590b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f2000640 1 -- 172.21.15.59:0/720032307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc037070 con 0x7f37f40afaf0 2024-04-24T06:32:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f2000640 1 -- 172.21.15.59:0/720032307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc032440 con 0x7f37f40afaf0 2024-04-24T06:32:49.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f2000640 1 -- 172.21.15.59:0/720032307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc032810 con 0x7f37f40afaf0 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 -- 172.21.15.59:0/720032307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f40afaf0 msgr2=0x7f37f40d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 --2- 172.21.15.59:0/720032307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f40afaf0 0x7f37f40d01a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f37f406c7c0 tx=0x7f37dc02a230 comp rx=0 tx=0).stop 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 -- 172.21.15.59:0/720032307 shutdown_connections 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 --2- 172.21.15.59:0/720032307 >> v2:172.21.15.66:3300/0 conn(0x7f37f40d06e0 0x7f37f41624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 --2- 172.21.15.59:0/720032307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f40afaf0 0x7f37f40d01a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 -- 172.21.15.59:0/720032307 >> 172.21.15.59:0/720032307 conn(0x7f37f4067ea0 msgr2=0x7f37f406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.538+0000 7f37f9800640 1 -- 172.21.15.59:0/720032307 shutdown_connections 2024-04-24T06:32:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.539+0000 7f37f9800640 1 -- 172.21.15.59:0/720032307 wait complete. 2024-04-24T06:32:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.540+0000 7f37f9800640 1 Processor -- start 2024-04-24T06:32:49.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.541+0000 7f37f9800640 1 -- start start 2024-04-24T06:32:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.541+0000 7f37f9800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37f40afaf0 0x7f37f4140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f40d06e0 0x7f37f41e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37f406e600 con 0x7f37f415c5e0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f9800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37f406e480 con 0x7f37f40d06e0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f9800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f37f4162a10 con 0x7f37f40afaf0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45080/0 (socket says 172.21.15.59:45080) 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 -- 172.21.15.59:0/3266580721 learned_addr learned my addr 172.21.15.59:0/3266580721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f2a00640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37f40afaf0 0x7f37f4140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37ea000640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f40d06e0 0x7f37f41e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 -- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37f40afaf0 msgr2=0x7f37f4140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37f40afaf0 0x7f37f4140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 -- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f40d06e0 msgr2=0x7f37f41e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f40d06e0 0x7f37f41e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.542+0000 7f37f3400640 1 -- 172.21.15.59:0/3266580721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f415ca60 con 0x7f37f415c5e0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37f3400640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b970 tx=0x7f37ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec019070 con 0x7f37f415c5e0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec00cb60 con 0x7f37f415c5e0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec014440 con 0x7f37f415c5e0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f4141040 con 0x7f37f415c5e0 2024-04-24T06:32:49.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.543+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f415cfe0 con 0x7f37f415c5e0 2024-04-24T06:32:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.544+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f4075060 con 0x7f37f415c5e0 2024-04-24T06:32:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.545+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37ec01c070 con 0x7f37f415c5e0 2024-04-24T06:32:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.546+0000 7f37eb400640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.546+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f37ec08b8a0 con 0x7f37f415c5e0 2024-04-24T06:32:49.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.547+0000 7f37f2a00640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:49.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.547+0000 7f37f2a00640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006fd0 tx=0x7f37e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:49.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.551+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37f4075060 con 0x7f37f415c5e0 2024-04-24T06:32:49.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.793+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f4161210 con 0x7f37f415c5e0 2024-04-24T06:32:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.794+0000 7f37eb400640 1 -- 172.21.15.59:0/3266580721 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5178 (secure 0 0 0) 0x7f37ec059cc0 con 0x7f37f415c5e0 2024-04-24T06:32:49.796 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:49.796 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:24.099749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:32:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 34 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.797+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 msgr2=0x7f37d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.797+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006fd0 tx=0x7f37e4008040 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.797+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 msgr2=0x7f37f415dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.797+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b970 tx=0x7f37ec00be30 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 shutdown_connections 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37f415c5e0 0x7f37f415dac0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37f40d06e0 0x7f37f41e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 --2- 172.21.15.59:0/3266580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37f40afaf0 0x7f37f4140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 >> 172.21.15.59:0/3266580721 conn(0x7f37f4067ea0 msgr2=0x7f37f4161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 shutdown_connections 2024-04-24T06:32:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:49.798+0000 7f37f9800640 1 -- 172.21.15.59:0/3266580721 wait complete. 2024-04-24T06:32:49.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-24T06:32:50.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.023+0000 7f10dd800640 1 Processor -- start 2024-04-24T06:32:50.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.024+0000 7f10dd800640 1 -- start start 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10dd800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10d80b81b0 0x7f10d80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10dd800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 0x7f10d80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10dd800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80cce30 con 0x7f10d80d1960 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10dd800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80b8590 con 0x7f10d80b81b0 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10dd800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80cd110 con 0x7f10d809b110 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10d6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10d80b81b0 0x7f10d80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10d6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f10d80d1960 0x7f10d809aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42768/0 (socket says 172.21.15.59:42768) 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10d6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10d80b81b0 0x7f10d80d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45096/0 (socket says 172.21.15.59:45096) 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.025+0000 7f10d6000640 1 -- 172.21.15.59:0/3865316901 learned_addr learned my addr 172.21.15.59:0/3865316901 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d7400640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 0x7f10d80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d6a00640 1 -- 172.21.15.59:0/3865316901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 msgr2=0x7f10d80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d6a00640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 0x7f10d80cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d6a00640 1 -- 172.21.15.59:0/3865316901 >> v1:172.21.15.59:6789/0 conn(0x7f10d80d1960 legacy=0x7f10d809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d6a00640 1 -- 172.21.15.59:0/3865316901 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80cd3f0 con 0x7f10d80b81b0 2024-04-24T06:32:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d7400640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 0x7f10d80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.026+0000 7f10d5600640 1 -- 172.21.15.59:0/3865316901 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925027274 0 0) 0x7f10d80cce30 con 0x7f10d80d1960 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.027+0000 7f10d6a00640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d80b81b0 0x7f10d80d1420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10c80097a0 tx=0x7f10c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b59036876f235 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.027+0000 7f10d5600640 1 -- 172.21.15.59:0/3865316901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8037070 con 0x7f10d80b81b0 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.027+0000 7f10d5600640 1 -- 172.21.15.59:0/3865316901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8032440 con 0x7f10d80b81b0 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.027+0000 7f10d5600640 1 -- 172.21.15.59:0/3865316901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8032810 con 0x7f10d80b81b0 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 -- 172.21.15.59:0/3865316901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d80b81b0 msgr2=0x7f10d80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d80b81b0 0x7f10d80d1420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10c80097a0 tx=0x7f10c802a180 comp rx=0 tx=0).stop 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 -- 172.21.15.59:0/3865316901 shutdown_connections 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d809b110 0x7f10d80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 --2- 172.21.15.59:0/3865316901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d80b81b0 0x7f10d80d1420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 -- 172.21.15.59:0/3865316901 >> 172.21.15.59:0/3865316901 conn(0x7f10d80b4130 msgr2=0x7f10d80b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 -- 172.21.15.59:0/3865316901 shutdown_connections 2024-04-24T06:32:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.028+0000 7f10dd800640 1 -- 172.21.15.59:0/3865316901 wait complete. 2024-04-24T06:32:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.030+0000 7f10dd800640 1 Processor -- start 2024-04-24T06:32:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.030+0000 7f10dd800640 1 -- start start 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80b81b0 0x7f10d815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 0x7f10d815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b5490 con 0x7f10d809b110 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b4f10 con 0x7f10d80b81b0 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10dd800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80b5610 con 0x7f10d80d1960 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45100/0 (socket says 172.21.15.59:45100) 2024-04-24T06:32:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10d6a00640 1 -- 172.21.15.59:0/2264207561 learned_addr learned my addr 172.21.15.59:0/2264207561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.031+0000 7f10d7400640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 0x7f10d815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 -- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 msgr2=0x7f10d815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 0x7f10d815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6000640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80b81b0 0x7f10d815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 -- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80b81b0 msgr2=0x7f10d815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80b81b0 0x7f10d815d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 -- 172.21.15.59:0/2264207561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d81f4670 con 0x7f10d809b110 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d7400640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 0x7f10d815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10cea00640 1 -- 172.21.15.59:0/2264207561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8037040 con 0x7f10d809b110 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10cea00640 1 -- 172.21.15.59:0/2264207561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8042070 con 0x7f10d809b110 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10cea00640 1 -- 172.21.15.59:0/2264207561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8030a70 con 0x7f10d809b110 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.032+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d81f4930 con 0x7f10d809b110 2024-04-24T06:32:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.033+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81f4ef0 con 0x7f10d809b110 2024-04-24T06:32:50.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.035+0000 7f10cea00640 1 -- 172.21.15.59:0/2264207561 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10c8030c10 con 0x7f10d809b110 2024-04-24T06:32:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.036+0000 7f10cea00640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 0x7f10b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.036+0000 7f10cea00640 1 -- 172.21.15.59:0/2264207561 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10c80ac590 con 0x7f10d809b110 2024-04-24T06:32:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.036+0000 7f10d6000640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 0x7f10b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.037+0000 7f10e0776740 1 Processor -- start 2024-04-24T06:32:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.037+0000 7f10e0776740 1 -- start start 2024-04-24T06:32:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.037+0000 7f10d6000640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 0x7f10b406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 0x56468d9e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468da03de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56468d920190 con 0x56468d9bbcd0 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56468d8c6030 con 0x56468da03890 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56468d91c150 con 0x56468d9e5ff0 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45104/0 (socket says 172.21.15.59:45104) 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 -- 172.21.15.59:0/1414259588 learned_addr learned my addr 172.21.15.59:0/1414259588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d6a00640 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 0x56468d9e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 -- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 msgr2=0x56468d9e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 0x56468d9e63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 -- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 msgr2=0x56468da03de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468da03de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d7400640 1 -- 172.21.15.59:0/1414259588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d8aeb60 con 0x56468d9bbcd0 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.038+0000 7f10d6a00640 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 0x56468d9e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.039+0000 7f10d7400640 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f10d000b9f0 tx=0x7f10d000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c5152542ce1ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.039+0000 7f10c7400640 1 -- 172.21.15.59:0/1414259588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0016070 con 0x56468d9bbcd0 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.039+0000 7f10c7400640 1 -- 172.21.15.59:0/1414259588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d000dcd0 con 0x56468d9bbcd0 2024-04-24T06:32:50.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.039+0000 7f10c7400640 1 -- 172.21.15.59:0/1414259588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0012660 con 0x56468d9bbcd0 2024-04-24T06:32:50.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 -- 172.21.15.59:0/1414259588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 msgr2=0x56468da01f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f10d000b9f0 tx=0x7f10d000beb0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 -- 172.21.15.59:0/1414259588 shutdown_connections 2024-04-24T06:32:50.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9bbcd0 0x56468da01f50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468da03de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 --2- 172.21.15.59:0/1414259588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e5ff0 0x56468d9e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 -- 172.21.15.59:0/1414259588 >> 172.21.15.59:0/1414259588 conn(0x56468d9b8600 msgr2=0x56468da00e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 -- 172.21.15.59:0/1414259588 shutdown_connections 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.040+0000 7f10e0776740 1 -- 172.21.15.59:0/1414259588 wait complete. 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.041+0000 7f10e0776740 1 -- start start 2024-04-24T06:32:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.041+0000 7f10e0776740 1 Processor -- start 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468d9e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468d9e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56468d9bca70 con 0x56468d9e5ff0 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56468d8d9500 con 0x56468da03890 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56468d9b6530 con 0x56468d9e9840 2024-04-24T06:32:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10e0776740 10 client.0 _ll_register_callbacks cb 0x56468d916270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.042+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45106/0 (socket says 172.21.15.59:45106) 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 -- 172.21.15.59:0/2366745447 learned_addr learned my addr 172.21.15.59:0/2366745447 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d7400640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468d9e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6000640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468d9e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 -- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 msgr2=0x56468d9e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468d9e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 -- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 msgr2=0x56468d9e9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468d9e9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 -- 172.21.15.59:0/2366745447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d6fe7c0 con 0x56468d9e5ff0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6000640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468d9e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10d6a00640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f10c802a770 tx=0x7f10c8079cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef33839a814b2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.043+0000 7f10bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8045560 con 0x56468d9e5ff0 2024-04-24T06:32:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10c807a510 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d000dcd0 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10c807e660 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 10 client.6456 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x56468d75ef50 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d000dcd0 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f10c803a030 con 0x56468d9e5ff0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 10 client.6456 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 10 client.6456 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 20 client.6456 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e81d0 0x56468da11b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 4 client.6456 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.044+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468d92dbb0 con 0x56468d9e81d0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.045+0000 7f10d6000640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e81d0 0x56468da11b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.045+0000 7f10d6000640 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e81d0 0x56468da11b50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e083309487cbc0db server_cookie=2a9306358aae9ff in_seq=0 out_seq=0 2024-04-24T06:32:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.045+0000 7f10bfe00640 10 client.6456 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.046+0000 7f10bfe00640 1 -- 172.21.15.59:0/2366745447 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56468d92dbb0 con 0x56468d9e81d0 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.046+0000 7f10bfe00640 10 client.6456 handle_command_reply: tid=0 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.046+0000 7f10e0776740 1 client.6456 shutdown 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.046+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 msgr2=0x56468d9e27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.046+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f10c802a770 tx=0x7f10c8079cd0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 shutdown_connections 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e81d0 0x56468da11b50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468d9e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468da03890 0x56468d9e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 --2- 172.21.15.59:0/2366745447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468d9e27b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 >> 172.21.15.59:0/2366745447 conn(0x56468d9bbcd0 msgr2=0x56468da00690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10bfe00640 0 client.6456 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 shutdown_connections 2024-04-24T06:32:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 1 -- 172.21.15.59:0/2366745447 wait complete. 2024-04-24T06:32:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.047+0000 7f10e0776740 20 client.6456 trim_cache size 0 max 16384 2024-04-24T06:32:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.062+0000 7f10e0776740 1 Processor -- start 2024-04-24T06:32:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.062+0000 7f10e0776740 1 -- start start 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 0x56468d9e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468da03890 0x56468d9ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81ffe30 con 0x56468d9e5ff0 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81600c0 con 0x56468d9e9840 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.063+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80ca310 con 0x56468da03890 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45116/0 (socket says 172.21.15.59:45116) 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 -- 172.21.15.59:0/3125852349 learned_addr learned my addr 172.21.15.59:0/3125852349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 -- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468da03890 msgr2=0x56468d9ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468da03890 0x56468d9ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 -- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 msgr2=0x56468d9e90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6000640 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 0x56468d9e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 0x56468d9e90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 -- 172.21.15.59:0/3125852349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468d5f20b0 con 0x56468d9e5ff0 2024-04-24T06:32:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6000640 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 0x56468d9e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.064+0000 7f10d6a00640 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10c802a1c0 tx=0x7f10c8008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f10bf5f0cc79ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10c6a00640 1 -- 172.21.15.59:0/3125852349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80aae40 con 0x56468d9e5ff0 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10c6a00640 1 -- 172.21.15.59:0/3125852349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8045360 con 0x56468d9e5ff0 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10c6a00640 1 -- 172.21.15.59:0/3125852349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8082a00 con 0x56468d9e5ff0 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 -- 172.21.15.59:0/3125852349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 msgr2=0x56468da02790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10c802a1c0 tx=0x7f10c8008000 comp rx=0 tx=0).stop 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 -- 172.21.15.59:0/3125852349 shutdown_connections 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468da03890 0x56468d9ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e9840 0x56468d9e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 --2- 172.21.15.59:0/3125852349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468d9e5ff0 0x56468da02790 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 -- 172.21.15.59:0/3125852349 >> 172.21.15.59:0/3125852349 conn(0x56468d9bbcd0 msgr2=0x56468d9f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 -- 172.21.15.59:0/3125852349 shutdown_connections 2024-04-24T06:32:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.065+0000 7f10e0776740 1 -- 172.21.15.59:0/3125852349 wait complete. 2024-04-24T06:32:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.066+0000 7f10e0776740 1 -- start start 2024-04-24T06:32:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.066+0000 7f10e0776740 1 Processor -- start 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.067+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e5ff0 0x56468d9f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.067+0000 7f10e0776740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468da19730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.067+0000 7f10e0776740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.067+0000 7f10e0776740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56468da12630 con 0x56468da03890 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.067+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56468d9bca70 con 0x56468d9e5ff0 2024-04-24T06:32:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10e0776740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56468d9e85d0 con 0x56468d9e9840 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10e0776740 10 client.0 _ll_register_callbacks cb 0x7f10c8009160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 -- 172.21.15.59:0/548475647 learned_addr learned my addr 172.21.15.59:0/548475647 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d6a00640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e5ff0 0x56468d9f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d6000640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468da19730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 -- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 msgr2=0x56468da19730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468da19730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 -- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e5ff0 msgr2=0x56468d9f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e5ff0 0x56468d9f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.068+0000 7f10d7400640 1 -- 172.21.15.59:0/548475647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d9f9130 con 0x56468da03890 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10d7400640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10d0015000 tx=0x7f10d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba211e14c02af67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0029060 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10d000b2c0 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c8045360 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10d0016040 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10e0776740 10 client.6462 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x56468d9e2a40 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.069+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8045360 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f10d0033080 con 0x56468da03890 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 10 client.6462 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 10 client.6462 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 20 client.6462 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e6cb0 0x56468d9e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 4 client.6462 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56468d927a20 con 0x56468d9e6cb0 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10d6a00640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e6cb0 0x56468d9e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10d6a00640 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e6cb0 0x56468d9e7090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbe8f325c5cfebb server_cookie=1710141c64f0bf0c in_seq=0 out_seq=0 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.070+0000 7f10bfe00640 10 client.6462 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10bfe00640 1 -- 172.21.15.59:0/548475647 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x56468d927a20 con 0x56468d9e6cb0 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10bfe00640 10 client.6462 handle_command_reply: tid=0 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 client.6462 shutdown 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 msgr2=0x56468d9f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10d0015000 tx=0x7f10d00046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 shutdown_connections 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56468d9e6cb0 0x56468d9e7090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56468da03890 0x56468d9f7280 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56468d9e9840 0x56468da19730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 --2- 172.21.15.59:0/548475647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56468d9e5ff0 0x56468d9f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10bfe00640 0 client.6462 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 >> 172.21.15.59:0/548475647 conn(0x56468d9bbcd0 msgr2=0x56468d9b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 shutdown_connections 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 1 -- 172.21.15.59:0/548475647 wait complete. 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.071+0000 7f10e0776740 20 client.6462 trim_cache size 0 max 16384 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:32:50.078 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 1002410782, 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir_test/file" 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:32:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 msgr2=0x7f10b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 0x7f10b406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 msgr2=0x7f10d8154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c80046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 shutdown_connections 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10b406c8f0 0x7f10b406edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10d80d1960 0x7f10d815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10d80b81b0 0x7f10d815d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 --2- 172.21.15.59:0/2264207561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d809b110 0x7f10d8154210 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 >> 172.21.15.59:0/2264207561 conn(0x7f10d80b4130 msgr2=0x7f10d8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 shutdown_connections 2024-04-24T06:32:50.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.072+0000 7f10dd800640 1 -- 172.21.15.59:0/2264207561 wait complete. 2024-04-24T06:32:50.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:32:50.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.293+0000 7f3d72400640 1 Processor -- start 2024-04-24T06:32:50.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.293+0000 7f3d72400640 1 -- start start 2024-04-24T06:32:50.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d72400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c0cd020 0x7f3d6c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d72400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0cd520 con 0x7f3d6c0b3e40 2024-04-24T06:32:50.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d72400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0cd800 con 0x7f3d6c1454e0 2024-04-24T06:32:50.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d72400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0cddc0 con 0x7f3d6c0cd020 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d6aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c0cd020 0x7f3d6c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d6aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c0cd020 0x7f3d6c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45144/0 (socket says 172.21.15.59:45144) 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d6aa00640 1 -- 172.21.15.59:0/203503349 learned_addr learned my addr 172.21.15.59:0/203503349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.295+0000 7f3d6be00640 1 --1- 172.21.15.59:0/203503349 >> v1:172.21.15.59:6789/0 conn(0x7f3d6c0b3e40 0x7f3d6c160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42772/0 (socket says 172.21.15.59:42772) 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.296+0000 7f3d6aa00640 1 -- 172.21.15.59:0/203503349 >> v1:172.21.15.66:6789/0 conn(0x7f3d6c1454e0 legacy=0x7f3d6c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.296+0000 7f3d6aa00640 1 -- 172.21.15.59:0/203503349 >> v1:172.21.15.59:6789/0 conn(0x7f3d6c0b3e40 legacy=0x7f3d6c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.296+0000 7f3d6aa00640 1 -- 172.21.15.59:0/203503349 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0cdae0 con 0x7f3d6c0cd020 2024-04-24T06:32:50.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.297+0000 7f3d6aa00640 1 --2- 172.21.15.59:0/203503349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c0b3900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009870 tx=0x7f3d5802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a5ad42f1906bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.297+0000 7f3d6a000640 1 -- 172.21.15.59:0/203503349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58037070 con 0x7f3d6c0cd020 2024-04-24T06:32:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.297+0000 7f3d6a000640 1 -- 172.21.15.59:0/203503349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58032440 con 0x7f3d6c0cd020 2024-04-24T06:32:50.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.297+0000 7f3d6a000640 1 -- 172.21.15.59:0/203503349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58032810 con 0x7f3d6c0cd020 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 -- 172.21.15.59:0/203503349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 msgr2=0x7f3d6c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 --2- 172.21.15.59:0/203503349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c0b3900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009870 tx=0x7f3d5802a210 comp rx=0 tx=0).stop 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 -- 172.21.15.59:0/203503349 shutdown_connections 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 --2- 172.21.15.59:0/203503349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c0b3900 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 -- 172.21.15.59:0/203503349 >> 172.21.15.59:0/203503349 conn(0x7f3d6c067ea0 msgr2=0x7f3d6c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 -- 172.21.15.59:0/203503349 shutdown_connections 2024-04-24T06:32:50.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.298+0000 7f3d72400640 1 -- 172.21.15.59:0/203503349 wait complete. 2024-04-24T06:32:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.300+0000 7f3d72400640 1 Processor -- start 2024-04-24T06:32:50.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.300+0000 7f3d72400640 1 -- start start 2024-04-24T06:32:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b3e40 0x7f3d6c13ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c1454e0 0x7f3d6c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c161020 con 0x7f3d6c0cd020 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1462c0 con 0x7f3d6c1454e0 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d72400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c1611a0 con 0x7f3d6c0b3e40 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45150/0 (socket says 172.21.15.59:45150) 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 -- 172.21.15.59:0/2218050114 learned_addr learned my addr 172.21.15.59:0/2218050114 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 -- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b3e40 msgr2=0x7f3d6c13ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b3e40 0x7f3d6c13ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 -- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c1454e0 msgr2=0x7f3d6c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c1454e0 0x7f3d6c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.301+0000 7f3d6aa00640 1 -- 172.21.15.59:0/2218050114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c06dde0 con 0x7f3d6c0cd020 2024-04-24T06:32:50.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d6aa00640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009870 tx=0x7f3d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58041020 con 0x7f3d6c0cd020 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58030800 con 0x7f3d6c0cd020 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58037040 con 0x7f3d6c0cd020 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c15f610 con 0x7f3d6c0cd020 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.302+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c06e080 con 0x7f3d6c0cd020 2024-04-24T06:32:50.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.303+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c068a80 con 0x7f3d6c0cd020 2024-04-24T06:32:50.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.304+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d58044020 con 0x7f3d6c0cd020 2024-04-24T06:32:50.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.305+0000 7f3d63400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.305+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3d580ac100 con 0x7f3d6c0cd020 2024-04-24T06:32:50.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.306+0000 7f3d6b400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.306+0000 7f3d6b400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.310+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d5804b070 con 0x7f3d6c0cd020 2024-04-24T06:32:50.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.550+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d6c0aff90 con 0x7f3d6c0cd020 2024-04-24T06:32:50.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.551+0000 7f3d63400640 1 -- 172.21.15.59:0/2218050114 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f3d5807a520 con 0x7f3d6c0cd020 2024-04-24T06:32:50.552 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:32:50.552 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 msgr2=0x7f3d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c006bc0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 msgr2=0x7f3d6c15eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009870 tx=0x7f3d580046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 shutdown_connections 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c1454e0 0x7f3d6c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c0cd020 0x7f3d6c15eb50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 --2- 172.21.15.59:0/2218050114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b3e40 0x7f3d6c13ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 >> 172.21.15.59:0/2218050114 conn(0x7f3d6c067ea0 msgr2=0x7f3d6c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 shutdown_connections 2024-04-24T06:32:50.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.554+0000 7f3d72400640 1 -- 172.21.15.59:0/2218050114 wait complete. 2024-04-24T06:32:50.574 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:32:50.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:50.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-24T06:32:50.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.783+0000 7f17c9a00640 1 Processor -- start 2024-04-24T06:32:50.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.783+0000 7f17c9a00640 1 -- start start 2024-04-24T06:32:50.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c406c500 0x7f17c4141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17c4158ff0 0x7f17c40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c415f1d0 con 0x7f17c4158c10 2024-04-24T06:32:50.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f17c415fd50 con 0x7f17c4158ff0 2024-04-24T06:32:50.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f17c4142350 con 0x7f17c406c500 2024-04-24T06:32:50.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.785+0000 7f17c9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17c4158ff0 0x7f17c40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17c4158ff0 0x7f17c40b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45156/0 (socket says 172.21.15.59:45156) 2024-04-24T06:32:50.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 -- 172.21.15.59:0/1382427481 learned_addr learned my addr 172.21.15.59:0/1382427481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17bb400640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c406c500 0x7f17c4141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 -- 172.21.15.59:0/1382427481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c406c500 msgr2=0x7f17c4141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c406c500 0x7f17c4141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 -- 172.21.15.59:0/1382427481 >> v1:172.21.15.66:6789/0 conn(0x7f17c4158c10 legacy=0x7f17c406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.786+0000 7f17c9000640 1 -- 172.21.15.59:0/1382427481 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c415f4b0 con 0x7f17c4158ff0 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.787+0000 7f17c9000640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158ff0 0x7f17c40b3ac0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009870 tx=0x7f17bc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bd62b750453a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.787+0000 7f17c3400640 1 -- 172.21.15.59:0/1382427481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037070 con 0x7f17c4158ff0 2024-04-24T06:32:50.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.787+0000 7f17c3400640 1 -- 172.21.15.59:0/1382427481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc032440 con 0x7f17c4158ff0 2024-04-24T06:32:50.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.787+0000 7f17c3400640 1 -- 172.21.15.59:0/1382427481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc032810 con 0x7f17c4158ff0 2024-04-24T06:32:50.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 -- 172.21.15.59:0/1382427481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158ff0 msgr2=0x7f17c40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158ff0 0x7f17c40b3ac0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009870 tx=0x7f17bc02a1c0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 -- 172.21.15.59:0/1382427481 shutdown_connections 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c406c500 0x7f17c4141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 --2- 172.21.15.59:0/1382427481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158ff0 0x7f17c40b3ac0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 -- 172.21.15.59:0/1382427481 >> 172.21.15.59:0/1382427481 conn(0x7f17c40b2200 msgr2=0x7f17c40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 -- 172.21.15.59:0/1382427481 shutdown_connections 2024-04-24T06:32:50.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.788+0000 7f17c9a00640 1 -- 172.21.15.59:0/1382427481 wait complete. 2024-04-24T06:32:50.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.790+0000 7f17c9a00640 1 Processor -- start 2024-04-24T06:32:50.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.790+0000 7f17c9a00640 1 -- start start 2024-04-24T06:32:50.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 0x7f17c41f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c4158ff0 0x7f17c40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17c415fa70 con 0x7f17c4158c10 2024-04-24T06:32:50.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17c40b4000 con 0x7f17c406c500 2024-04-24T06:32:50.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17c9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f17c4071e10 con 0x7f17c4158ff0 2024-04-24T06:32:50.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45168/0 (socket says 172.21.15.59:45168) 2024-04-24T06:32:50.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.791+0000 7f17bb400640 1 -- 172.21.15.59:0/889587737 learned_addr learned my addr 172.21.15.59:0/889587737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17c9000640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c4158ff0 0x7f17c40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17c3e00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 0x7f17c41f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 -- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c4158ff0 msgr2=0x7f17c40b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c4158ff0 0x7f17c40b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 -- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 msgr2=0x7f17c41f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 0x7f17c41f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 -- 172.21.15.59:0/889587737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c409ad80 con 0x7f17c4158c10 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17c3e00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 0x7f17c41f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17bb400640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d800 tx=0x7f17ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.792+0000 7f17c0c00640 1 -- 172.21.15.59:0/889587737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00b6c0 con 0x7f17c4158c10 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.793+0000 7f17c0c00640 1 -- 172.21.15.59:0/889587737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac019070 con 0x7f17c4158c10 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.793+0000 7f17c0c00640 1 -- 172.21.15.59:0/889587737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00ba80 con 0x7f17c4158c10 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.793+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c409b060 con 0x7f17c4158c10 2024-04-24T06:32:50.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.793+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41480d0 con 0x7f17c4158c10 2024-04-24T06:32:50.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.795+0000 7f17c0c00640 1 -- 172.21.15.59:0/889587737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17ac00bc20 con 0x7f17c4158c10 2024-04-24T06:32:50.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.796+0000 7f17c0c00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.796+0000 7f17c0c00640 1 -- 172.21.15.59:0/889587737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f17ac08c0e0 con 0x7f17c4158c10 2024-04-24T06:32:50.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.797+0000 7f17c3e00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.797+0000 7f17cdf88740 1 Processor -- start 2024-04-24T06:32:50.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.797+0000 7f17cdf88740 1 -- start start 2024-04-24T06:32:50.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.797+0000 7f17c3e00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 0x7f17a006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f17b4001d70 tx=0x7f17b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed5cdf0 0x56371ed84c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371ed5d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56371ecc1ff0 con 0x56371eda5a60 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56371ec67f40 con 0x56371ed5cdf0 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56371ecbdfb0 con 0x56371ed85170 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.798+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45180/0 (socket says 172.21.15.59:45180) 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 -- 172.21.15.59:0/3027017953 learned_addr learned my addr 172.21.15.59:0/3027017953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c3e00640 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed5cdf0 0x56371ed84c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17bb400640 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371ed5d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 -- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 msgr2=0x56371ed5d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371ed5d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 -- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed5cdf0 msgr2=0x56371ed84c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed5cdf0 0x56371ed84c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 -- 172.21.15.59:0/3027017953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371eaa0940 con 0x56371eda5a60 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.799+0000 7f17c9000640 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02a200 tx=0x7f17bc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e11fb83c4eebe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.800+0000 7f17b8c00640 1 -- 172.21.15.59:0/3027017953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037040 con 0x56371eda5a60 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.800+0000 7f17b8c00640 1 -- 172.21.15.59:0/3027017953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc0326c0 con 0x56371eda5a60 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.800+0000 7f17b8c00640 1 -- 172.21.15.59:0/3027017953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc032a70 con 0x56371eda5a60 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.800+0000 7f17cdf88740 1 -- 172.21.15.59:0/3027017953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 msgr2=0x56371ed5b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.800+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02a200 tx=0x7f17bc02c000 comp rx=0 tx=0).stop 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 -- 172.21.15.59:0/3027017953 shutdown_connections 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ed5b5b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371ed5d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3027017953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed5cdf0 0x56371ed84c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 -- 172.21.15.59:0/3027017953 >> 172.21.15.59:0/3027017953 conn(0x56371ed5a010 msgr2=0x56371eda29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 -- 172.21.15.59:0/3027017953 shutdown_connections 2024-04-24T06:32:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 -- 172.21.15.59:0/3027017953 wait complete. 2024-04-24T06:32:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.801+0000 7f17cdf88740 1 -- start start 2024-04-24T06:32:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.802+0000 7f17cdf88740 1 Processor -- start 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 0x56371ed89c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed8a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56371ed847a0 con 0x56371ed80560 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56371ec7b2c0 con 0x56371ed85170 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56371ed84190 con 0x56371eda5a60 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17cdf88740 10 client.0 _ll_register_callbacks cb 0x56371ec49c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:50.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45188/0 (socket says 172.21.15.59:45188) 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 -- 172.21.15.59:0/2481426603 learned_addr learned my addr 172.21.15.59:0/2481426603 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c3e00640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 0x56371ed89c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17bb400640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed8a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 -- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 msgr2=0x56371ed8a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed8a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 -- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 msgr2=0x56371ed89c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 0x56371ed89c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c9000640 1 -- 172.21.15.59:0/2481426603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371eb010d0 con 0x56371ed80560 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.803+0000 7f17c3e00640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 0x56371ed89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17c9000640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004590 tx=0x7f17bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccafb5d714f083 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17b1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037040 con 0x56371ed80560 2024-04-24T06:32:50.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f17bc032c80 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17bc0326c0 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17bc051050 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17cdf88740 10 client.6483 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x56371e994230 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.804+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc0326c0 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f17bc054020 con 0x56371ed80560 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 10 client.6483 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 10 client.6483 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 20 client.6483 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ed88800 0x56371ed88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 4 client.6483 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371eccfa10 con 0x56371ed88800 2024-04-24T06:32:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17c3e00640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ed88800 0x56371ed88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17c3e00640 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ed88800 0x56371ed88c30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cb24744a60e2ae server_cookie=8e61719d05508bf9 in_seq=0 out_seq=0 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.805+0000 7f17b1600640 10 client.6483 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.806+0000 7f17b1600640 1 -- 172.21.15.59:0/2481426603 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56371eccfa10 con 0x56371ed88800 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.806+0000 7f17b1600640 10 client.6483 handle_command_reply: tid=0 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.806+0000 7f17cdf88740 1 client.6483 shutdown 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.806+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 msgr2=0x56371ed83430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.806+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004590 tx=0x7f17bc004730 comp rx=0 tx=0).stop 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 shutdown_connections 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ed88800 0x56371ed88c30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed80560 0x56371ed83430 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed8a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 --2- 172.21.15.59:0/2481426603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed85170 0x56371ed89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17b1600640 0 client.6483 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 >> 172.21.15.59:0/2481426603 conn(0x56371ed5cdf0 msgr2=0x56371eda27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 shutdown_connections 2024-04-24T06:32:50.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 1 -- 172.21.15.59:0/2481426603 wait complete. 2024-04-24T06:32:50.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.807+0000 7f17cdf88740 20 client.6483 trim_cache size 0 max 16384 2024-04-24T06:32:50.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.826+0000 7f17cdf88740 1 Processor -- start 2024-04-24T06:32:50.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.826+0000 7f17cdf88740 1 -- start start 2024-04-24T06:32:50.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edc6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed5b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4200cc0 con 0x56371ed85170 2024-04-24T06:32:50.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ffce0 con 0x56371ed80560 2024-04-24T06:32:50.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f17c41ff0d0 con 0x56371eda5a60 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45190/0 (socket says 172.21.15.59:45190) 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 -- 172.21.15.59:0/3331784933 learned_addr learned my addr 172.21.15.59:0/3331784933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 -- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 msgr2=0x56371ed5b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed5b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 -- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 msgr2=0x56371edc6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edc6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.827+0000 7f17bb400640 1 -- 172.21.15.59:0/3331784933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371eda22b0 con 0x56371ed85170 2024-04-24T06:32:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17bb400640 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f17ac08a510 tx=0x7f17ac0599e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e825d3627795b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17b3e00640 1 -- 172.21.15.59:0/3331784933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac05a470 con 0x56371ed85170 2024-04-24T06:32:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17b3e00640 1 -- 172.21.15.59:0/3331784933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac059d70 con 0x56371ed85170 2024-04-24T06:32:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17b3e00640 1 -- 172.21.15.59:0/3331784933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac05e120 con 0x56371ed85170 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17cdf88740 1 -- 172.21.15.59:0/3331784933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 msgr2=0x56371ed5dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.828+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f17ac08a510 tx=0x7f17ac0599e0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 -- 172.21.15.59:0/3331784933 shutdown_connections 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371eda5a60 0x56371ed5b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371ed85170 0x56371ed5dae0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 --2- 172.21.15.59:0/3331784933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edc6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 -- 172.21.15.59:0/3331784933 >> 172.21.15.59:0/3331784933 conn(0x56371ed5cdf0 msgr2=0x56371ed99650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 -- 172.21.15.59:0/3331784933 shutdown_connections 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 -- 172.21.15.59:0/3331784933 wait complete. 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 -- start start 2024-04-24T06:32:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.829+0000 7f17cdf88740 1 Processor -- start 2024-04-24T06:32:50.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edb4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371edb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56371ed84ad0 con 0x56371eda5a60 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41ff690 con 0x56371ed80560 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56371ee1e110 con 0x56371ed85170 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.830+0000 7f17cdf88740 10 client.0 _ll_register_callbacks cb 0x7f17bc004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45194/0 (socket says 172.21.15.59:45194) 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 -- 172.21.15.59:0/1734302669 learned_addr learned my addr 172.21.15.59:0/1734302669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17bb400640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371edb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 -- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 msgr2=0x56371edb3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c3e00640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edb4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371edb3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 -- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 msgr2=0x56371edb4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edb4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.831+0000 7f17c9000640 1 -- 172.21.15.59:0/1734302669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371edb3b40 con 0x56371eda5a60 2024-04-24T06:32:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17c9000640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02a9b0 tx=0x7f17bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba60f19d7be0ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc037070 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f17bc003ca0 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac059d70 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17bc032680 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 10 client.6489 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x56371ee1de50 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac059d70 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f17bc054040 con 0x56371eda5a60 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 10 client.6489 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 10 client.6489 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 20 client.6489 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ee1fbb0 0x56371ed98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 4 client.6489 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-24T06:32:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.832+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x56371ecc9880 con 0x56371ee1fbb0 2024-04-24T06:32:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.833+0000 7f17c3e00640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ee1fbb0 0x56371ed98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.833+0000 7f17c3e00640 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ee1fbb0 0x56371ed98380 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e735dab964fc7a server_cookie=a34900c70e82c334 in_seq=0 out_seq=0 2024-04-24T06:32:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.833+0000 7f17b1600640 10 client.6489 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17b1600640 1 -- 172.21.15.59:0/1734302669 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56371ecc9880 con 0x56371ee1fbb0 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17b1600640 10 client.6489 handle_command_reply: tid=0 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 client.6489 shutdown 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 msgr2=0x56371ee1f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02a9b0 tx=0x7f17bc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 shutdown_connections 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x56371ee1fbb0 0x56371ed98380 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56371eda5a60 0x56371ee1f670 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56371ed85170 0x56371edb3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 --2- 172.21.15.59:0/1734302669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56371ed80560 0x56371edb4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 >> 172.21.15.59:0/1734302669 conn(0x56371ed5cdf0 msgr2=0x56371eda27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.834+0000 7f17b1600640 0 client.6489 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 shutdown_connections 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17cdf88740 1 -- 172.21.15.59:0/1734302669 wait complete. 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17cdf88740 20 client.6489 trim_cache size 0 max 16384 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:32:50.839 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "ec652ae2-b00d-41fa-9912-91c8122566be", 2024-04-24T06:32:50.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:32:50.840 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:50.840 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f5446800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-24T06:32:17.158789+0000 1=1+0) caps={6351=pAsLsXsFs/-@2},l=6351(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55d8a99c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 msgr2=0x7f17a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 0x7f17a006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f17b4001d70 tx=0x7f17b4009040 comp rx=0 tx=0).stop 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 msgr2=0x7f17c409a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.835+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d800 tx=0x7f17ac00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 shutdown_connections 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17a006c8f0 0x7f17a006edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f17c4158ff0 0x7f17c40b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17c4158c10 0x7f17c409a280 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 --2- 172.21.15.59:0/889587737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17c406c500 0x7f17c41f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 >> 172.21.15.59:0/889587737 conn(0x7f17c40b2200 msgr2=0x7f17c406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 shutdown_connections 2024-04-24T06:32:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:50.836+0000 7f17c9a00640 1 -- 172.21.15.59:0/889587737 wait complete. 2024-04-24T06:32:50.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:32:50.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T06:32:51.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.051+0000 7faa5c800640 1 Processor -- start 2024-04-24T06:32:51.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.051+0000 7faa5c800640 1 -- start start 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5c800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa540d0cc0 0x7faa5406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5c800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 0x7faa54072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5c800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa540cfc20 con 0x7faa54145e10 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5c800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faa540cff00 con 0x7faa540d0cc0 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5c800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa540d01e0 con 0x7faa54071d70 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa59400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa540d0cc0 0x7faa5406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa59400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa540d0cc0 0x7faa5406f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45208/0 (socket says 172.21.15.59:45208) 2024-04-24T06:32:51.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa59400640 1 -- 172.21.15.59:0/3841580721 learned_addr learned my addr 172.21.15.59:0/3841580721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.053+0000 7faa5a800640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 0x7faa54072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.054+0000 7faa59400640 1 -- 172.21.15.59:0/3841580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 msgr2=0x7faa54072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.054+0000 7faa59400640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 0x7faa54072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.054+0000 7faa59400640 1 -- 172.21.15.59:0/3841580721 >> v1:172.21.15.66:6789/0 conn(0x7faa54145e10 legacy=0x7faa540cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.054+0000 7faa59400640 1 -- 172.21.15.59:0/3841580721 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa540d04c0 con 0x7faa540d0cc0 2024-04-24T06:32:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.054+0000 7faa5a800640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 0x7faa54072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:32:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.055+0000 7faa59400640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa540d0cc0 0x7faa5406f130 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faa44009870 tx=0x7faa4402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb16929dc1905ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.055+0000 7faa58a00640 1 -- 172.21.15.59:0/3841580721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44037070 con 0x7faa540d0cc0 2024-04-24T06:32:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.055+0000 7faa58a00640 1 -- 172.21.15.59:0/3841580721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44032440 con 0x7faa540d0cc0 2024-04-24T06:32:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.055+0000 7faa58a00640 1 -- 172.21.15.59:0/3841580721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44032810 con 0x7faa540d0cc0 2024-04-24T06:32:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 -- 172.21.15.59:0/3841580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa540d0cc0 msgr2=0x7faa5406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa540d0cc0 0x7faa5406f130 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faa44009870 tx=0x7faa4402a210 comp rx=0 tx=0).stop 2024-04-24T06:32:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 -- 172.21.15.59:0/3841580721 shutdown_connections 2024-04-24T06:32:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54071d70 0x7faa54072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 --2- 172.21.15.59:0/3841580721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa540d0cc0 0x7faa5406f130 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 -- 172.21.15.59:0/3841580721 >> 172.21.15.59:0/3841580721 conn(0x7faa54067d90 msgr2=0x7faa5406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 -- 172.21.15.59:0/3841580721 shutdown_connections 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.056+0000 7faa5c800640 1 -- 172.21.15.59:0/3841580721 wait complete. 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.058+0000 7faa5c800640 1 Processor -- start 2024-04-24T06:32:51.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.058+0000 7faa5c800640 1 -- start start 2024-04-24T06:32:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa540d0cc0 0x7faa5415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54145e10 0x7faa5415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa541461f0 con 0x7faa54071d70 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa5415cdd0 con 0x7faa540d0cc0 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa5c800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa54142720 con 0x7faa54145e10 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45216/0 (socket says 172.21.15.59:45216) 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 -- 172.21.15.59:0/2496701180 learned_addr learned my addr 172.21.15.59:0/2496701180 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 -- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54145e10 msgr2=0x7faa5415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54145e10 0x7faa5415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 -- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa540d0cc0 msgr2=0x7faa5415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa540d0cc0 0x7faa5415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.059+0000 7faa59e00640 1 -- 172.21.15.59:0/2496701180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5415bfb0 con 0x7faa54071d70 2024-04-24T06:32:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa59e00640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faa4800b3f0 tx=0x7faa4800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa4e000640 1 -- 172.21.15.59:0/2496701180 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48015020 con 0x7faa54071d70 2024-04-24T06:32:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa4e000640 1 -- 172.21.15.59:0/2496701180 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa48011830 con 0x7faa54071d70 2024-04-24T06:32:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5415c290 con 0x7faa54071d70 2024-04-24T06:32:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa4e000640 1 -- 172.21.15.59:0/2496701180 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48011c20 con 0x7faa54071d70 2024-04-24T06:32:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.060+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa54142c50 con 0x7faa54071d70 2024-04-24T06:32:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.062+0000 7faa4e000640 1 -- 172.21.15.59:0/2496701180 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa48018020 con 0x7faa54071d70 2024-04-24T06:32:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.063+0000 7faa4e000640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.063+0000 7faa4e000640 1 -- 172.21.15.59:0/2496701180 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa4808b760 con 0x7faa54071d70 2024-04-24T06:32:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.064+0000 7faa59400640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.064+0000 7faa5f62c740 1 Processor -- start 2024-04-24T06:32:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.064+0000 7faa5f62c740 1 -- start start 2024-04-24T06:32:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa59400640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 0x7faa3806edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa44034000 tx=0x7faa4402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b4c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7c610 0x561117b51420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561117ab61f0 con 0x561117b7cd50 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561117a5c350 con 0x561117b7c610 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561117ab21b0 con 0x561117b99c30 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45228/0 (socket says 172.21.15.59:45228) 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.065+0000 7faa59400640 1 -- 172.21.15.59:0/1240163661 learned_addr learned my addr 172.21.15.59:0/1240163661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 -- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 msgr2=0x561117b4c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b4c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 -- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7c610 msgr2=0x561117b51420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7c610 0x561117b51420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 -- 172.21.15.59:0/1240163661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561117ab0f50 con 0x561117b7cd50 2024-04-24T06:32:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa59400640 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faa44009960 tx=0x7faa44008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7c03019197836 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa42600640 1 -- 172.21.15.59:0/1240163661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4404b070 con 0x561117b7cd50 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa42600640 1 -- 172.21.15.59:0/1240163661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44032cb0 con 0x561117b7cd50 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.066+0000 7faa42600640 1 -- 172.21.15.59:0/1240163661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44037040 con 0x561117b7cd50 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 -- 172.21.15.59:0/1240163661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 msgr2=0x561117b7c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faa44009960 tx=0x7faa44008030 comp rx=0 tx=0).stop 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 -- 172.21.15.59:0/1240163661 shutdown_connections 2024-04-24T06:32:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7c610 0x561117b51420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7c0d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 --2- 172.21.15.59:0/1240163661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b4c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 -- 172.21.15.59:0/1240163661 >> 172.21.15.59:0/1240163661 conn(0x561117b4e830 msgr2=0x561117b970d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 -- 172.21.15.59:0/1240163661 shutdown_connections 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.067+0000 7faa5f62c740 1 -- 172.21.15.59:0/1240163661 wait complete. 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.068+0000 7faa5f62c740 1 -- start start 2024-04-24T06:32:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.068+0000 7faa5f62c740 1 Processor -- start 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7d780 0x561117ba76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561117b509e0 con 0x561117b7cd50 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561117a6f690 con 0x561117b7d780 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561117b4fdd0 con 0x561117b99c30 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa5f62c740 10 client.0 _ll_register_callbacks cb 0x561117aab650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa59e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa59e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45238/0 (socket says 172.21.15.59:45238) 2024-04-24T06:32:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.069+0000 7faa59e00640 1 -- 172.21.15.59:0/141739196 learned_addr learned my addr 172.21.15.59:0/141739196 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 -- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 msgr2=0x561117b7d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 -- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7d780 msgr2=0x561117ba76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7d780 0x561117ba76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 -- 172.21.15.59:0/141739196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561117a6e920 con 0x561117b7cd50 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa59e00640 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faa480558e0 tx=0x7faa4805d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a9fbc448fa1644 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa36a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48071020 con 0x561117b7cd50 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa48055af0 con 0x561117b7cd50 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa44032cb0 con 0x561117b7cd50 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faa480588d0 con 0x561117b7cd50 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa5f62c740 10 client.6501 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.070+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5611178927d0 con 0x561117b7cd50 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44032cb0 con 0x561117b7cd50 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7faa48058a70 con 0x561117b7cd50 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 10 client.6501 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 10 client.6501 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 20 client.6501 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b98730 0x561117b7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 4 client.6501 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561117ac3c10 con 0x561117b98730 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.071+0000 7faa59400640 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b98730 0x561117b7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.072+0000 7faa59400640 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b98730 0x561117b7a090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d08b861dae0d85b server_cookie=e1e81f83153cdeba in_seq=0 out_seq=0 2024-04-24T06:32:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.072+0000 7faa36a00640 10 client.6501 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.072+0000 7faa36a00640 1 -- 172.21.15.59:0/141739196 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561117ac3c10 con 0x561117b98730 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.072+0000 7faa36a00640 10 client.6501 handle_command_reply: tid=0 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 client.6501 shutdown 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 msgr2=0x561117b7fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faa480558e0 tx=0x7faa4805d830 comp rx=0 tx=0).stop 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 shutdown_connections 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b98730 0x561117b7a090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7d780 0x561117ba76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b7d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa36a00640 0 client.6501 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 --2- 172.21.15.59:0/141739196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b7cd50 0x561117b7fde0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 >> 172.21.15.59:0/141739196 conn(0x561117b7c610 msgr2=0x561117b970d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 shutdown_connections 2024-04-24T06:32:51.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 1 -- 172.21.15.59:0/141739196 wait complete. 2024-04-24T06:32:51.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.073+0000 7faa5f62c740 20 client.6501 trim_cache size 0 max 16384 2024-04-24T06:32:51.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.092+0000 7faa5f62c740 1 Processor -- start 2024-04-24T06:32:51.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.092+0000 7faa5f62c740 1 -- start start 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b7cd50 0x561117bbb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b97770 0x561117bab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa54200dc0 con 0x561117b99c30 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa54144430 con 0x561117b97770 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.093+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa54141d60 con 0x561117b7cd50 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45252/0 (socket says 172.21.15.59:45252) 2024-04-24T06:32:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 -- 172.21.15.59:0/195500146 learned_addr learned my addr 172.21.15.59:0/195500146 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59e00640 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b7cd50 0x561117bbb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 -- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b7cd50 msgr2=0x561117bbb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b7cd50 0x561117bbb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 -- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b97770 msgr2=0x561117bab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b97770 0x561117bab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 -- 172.21.15.59:0/195500146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611177c83f0 con 0x561117b99c30 2024-04-24T06:32:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.094+0000 7faa59400640 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faa440090b0 tx=0x7faa4402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25becbf78b87460f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa41c00640 1 -- 172.21.15.59:0/195500146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4404b040 con 0x561117b99c30 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa41c00640 1 -- 172.21.15.59:0/195500146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44056030 con 0x561117b99c30 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa41c00640 1 -- 172.21.15.59:0/195500146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44037040 con 0x561117b99c30 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 -- 172.21.15.59:0/195500146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 msgr2=0x561117b50640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faa440090b0 tx=0x7faa4402c070 comp rx=0 tx=0).stop 2024-04-24T06:32:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 -- 172.21.15.59:0/195500146 shutdown_connections 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b97770 0x561117bab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b99c30 0x561117b50640 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 --2- 172.21.15.59:0/195500146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b7cd50 0x561117bbb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 -- 172.21.15.59:0/195500146 >> 172.21.15.59:0/195500146 conn(0x561117b7c610 msgr2=0x561117b7ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 -- 172.21.15.59:0/195500146 shutdown_connections 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.095+0000 7faa5f62c740 1 -- 172.21.15.59:0/195500146 wait complete. 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.096+0000 7faa5f62c740 1 -- start start 2024-04-24T06:32:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.096+0000 7faa5f62c740 1 Processor -- start 2024-04-24T06:32:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7cd50 0x561117b7a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561117b80060 con 0x561117b97770 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa54143de0 con 0x561117b7cd50 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561117b96e60 con 0x561117b99c30 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa5f62c740 10 client.0 _ll_register_callbacks cb 0x7faa480592d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa59400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45266/0 (socket says 172.21.15.59:45266) 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.097+0000 7faa59400640 1 -- 172.21.15.59:0/2258939598 learned_addr learned my addr 172.21.15.59:0/2258939598 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 -- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 msgr2=0x561117b78c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 -- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7cd50 msgr2=0x561117b7a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7cd50 0x561117b7a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 -- 172.21.15.59:0/2258939598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611177880f0 con 0x561117b97770 2024-04-24T06:32:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa59400640 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faa440090b0 tx=0x7faa44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d7b930aa2326cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa36a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44060060 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa440525e0 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa44056030 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faa44037070 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa5f62c740 10 client.6507 fetch_fsmap learned FSMap version 34 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.098+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x561117a7b810 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44056030 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7faa44045070 con 0x561117b97770 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 10 client.6507 fetch_fsmap finished waiting for FSMap version 34 2024-04-24T06:32:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 10 client.6507 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 20 client.6507 populate_metadata read hostname 'smithi059' 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b99000 0x561117b79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 4 client.6507 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561117b7c020 con 0x561117b99000 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.099+0000 7faa59e00640 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b99000 0x561117b79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa59e00640 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b99000 0x561117b79160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02ae3c27ef571ae server_cookie=18383347427cc3aa in_seq=0 out_seq=0 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa36a00640 10 client.6507 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa36a00640 1 -- 172.21.15.59:0/2258939598 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x561117b7c020 con 0x561117b99000 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa36a00640 10 client.6507 handle_command_reply: tid=0 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 client.6507 shutdown 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 msgr2=0x561117b786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faa440090b0 tx=0x7faa44004730 comp rx=0 tx=0).stop 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 shutdown_connections 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x561117b99000 0x561117b79160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561117b99c30 0x561117b78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561117b97770 0x561117b786e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 --2- 172.21.15.59:0/2258939598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561117b7cd50 0x561117b7a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa36a00640 0 client.6507 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 >> 172.21.15.59:0/2258939598 conn(0x561117b7c610 msgr2=0x561117bab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 shutdown_connections 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 1 -- 172.21.15.59:0/2258939598 wait complete. 2024-04-24T06:32:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.100+0000 7faa5f62c740 20 client.6507 trim_cache size 0 max 16384 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "ec652ae2-b00d-41fa-9912-91c8122566be": { 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "ec652ae2-b00d-41fa-9912-91c8122566be", 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair,force" 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:51.107 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:32:51.108 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:32:51.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 msgr2=0x7faa3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 0x7faa3806edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa44034000 tx=0x7faa4402c000 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 msgr2=0x7faa5415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faa4800b3f0 tx=0x7faa4800b8b0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 shutdown_connections 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa3806c8f0 0x7faa3806edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa54145e10 0x7faa5415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa540d0cc0 0x7faa5415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 --2- 172.21.15.59:0/2496701180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa54071d70 0x7faa5415a930 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 >> 172.21.15.59:0/2496701180 conn(0x7faa54067d90 msgr2=0x7faa540cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 shutdown_connections 2024-04-24T06:32:51.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:32:51.101+0000 7faa5c800640 1 -- 172.21.15.59:0/2496701180 wait complete. 2024-04-24T06:32:51.119 INFO:tasks.cephfs.filesystem:scrub status for tag:ec652ae2-b00d-41fa-9912-91c8122566be - {'path': '/', 'tag': 'ec652ae2-b00d-41fa-9912-91c8122566be', 'options': 'recursive,repair,force'} 2024-04-24T06:33:06.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:33:06.128 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:33:21.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:21.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-24T06:33:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.344+0000 7faa98e00640 1 Processor -- start 2024-04-24T06:33:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.344+0000 7faa98e00640 1 -- start start 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.345+0000 7faa98e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa94161c50 0x7faa940d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.345+0000 7faa98e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa940d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.345+0000 7faa98e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa940d22e0 con 0x7faa940d0de0 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.345+0000 7faa98e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faa940d2ba0 con 0x7faa94161c50 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.345+0000 7faa98e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa940d2550 con 0x7faa940cfc50 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa94161c50 0x7faa940d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa94161c50 0x7faa940d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45682/0 (socket says 172.21.15.59:45682) 2024-04-24T06:33:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 -- 172.21.15.59:0/139509652 learned_addr learned my addr 172.21.15.59:0/139509652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa92a00640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa940d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 -- 172.21.15.59:0/139509652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 msgr2=0x7faa940d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa940d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 -- 172.21.15.59:0/139509652 >> v1:172.21.15.59:6789/0 conn(0x7faa940d0de0 legacy=0x7faa9406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.346+0000 7faa93400640 1 -- 172.21.15.59:0/139509652 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa940d2830 con 0x7faa94161c50 2024-04-24T06:33:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa93400640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940d5240 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faa84009870 tx=0x7faa8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6384f2ba4ca50617 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa92000640 1 -- 172.21.15.59:0/139509652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84037070 con 0x7faa94161c50 2024-04-24T06:33:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa92000640 1 -- 172.21.15.59:0/139509652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa84032440 con 0x7faa94161c50 2024-04-24T06:33:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa92000640 1 -- 172.21.15.59:0/139509652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84032810 con 0x7faa94161c50 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa98e00640 1 -- 172.21.15.59:0/139509652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 msgr2=0x7faa940d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.348+0000 7faa98e00640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940d5240 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faa84009870 tx=0x7faa8402a270 comp rx=0 tx=0).stop 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 -- 172.21.15.59:0/139509652 shutdown_connections 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa940d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 --2- 172.21.15.59:0/139509652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940d5240 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 -- 172.21.15.59:0/139509652 >> 172.21.15.59:0/139509652 conn(0x7faa9408bf90 msgr2=0x7faa940d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 -- 172.21.15.59:0/139509652 shutdown_connections 2024-04-24T06:33:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.349+0000 7faa98e00640 1 -- 172.21.15.59:0/139509652 wait complete. 2024-04-24T06:33:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.350+0000 7faa98e00640 1 Processor -- start 2024-04-24T06:33:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.351+0000 7faa98e00640 1 -- start start 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.351+0000 7faa98e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa94155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.351+0000 7faa98e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa940d0de0 0x7faa940ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.351+0000 7faa98e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa98e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa940d3570 con 0x7faa94161c50 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa98e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa940d0460 con 0x7faa940d0de0 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa98e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa940d0090 con 0x7faa940cfc50 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45694/0 (socket says 172.21.15.59:45694) 2024-04-24T06:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 -- 172.21.15.59:0/32523710 learned_addr learned my addr 172.21.15.59:0/32523710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93400640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa94155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa92a00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa940d0de0 0x7faa940ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 -- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 msgr2=0x7faa94155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa94155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 -- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa940d0de0 msgr2=0x7faa940ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa940d0de0 0x7faa940ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.352+0000 7faa93e00640 1 -- 172.21.15.59:0/32523710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa940b7790 con 0x7faa94161c50 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa93e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c710 tx=0x7faa8c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa8b400640 1 -- 172.21.15.59:0/32523710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c016020 con 0x7faa94161c50 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa8b400640 1 -- 172.21.15.59:0/32523710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c006d50 con 0x7faa94161c50 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa8b400640 1 -- 172.21.15.59:0/32523710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c003bb0 con 0x7faa94161c50 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa940b7a70 con 0x7faa94161c50 2024-04-24T06:33:21.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.353+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa940b8030 con 0x7faa94161c50 2024-04-24T06:33:21.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.356+0000 7faa8b400640 1 -- 172.21.15.59:0/32523710 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa8c00a850 con 0x7faa94161c50 2024-04-24T06:33:21.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.356+0000 7faa8b400640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.357+0000 7faa8b400640 1 -- 172.21.15.59:0/32523710 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa8c08b830 con 0x7faa94161c50 2024-04-24T06:33:21.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.357+0000 7faa93400640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.357+0000 7faa9d377740 1 Processor -- start 2024-04-24T06:33:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.357+0000 7faa93400640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 0x7faa7406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faa84034000 tx=0x7faa8402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 -- start start 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade70e00 0x5638ade23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638add8d1f0 con 0x5638ade537e0 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5638add33350 con 0x5638ade70e00 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.358+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5638add891b0 con 0x5638ade53f20 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45696/0 (socket says 172.21.15.59:45696) 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 -- 172.21.15.59:0/3022761259 learned_addr learned my addr 172.21.15.59:0/3022761259 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 -- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 msgr2=0x5638ade532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 -- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade70e00 msgr2=0x5638ade23790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade70e00 0x5638ade23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 -- 172.21.15.59:0/3022761259 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638add87f50 con 0x5638ade537e0 2024-04-24T06:33:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.359+0000 7faa93e00640 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faa8c089cc0 tx=0x7faa8c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3e1bbfe6289c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa7fe00640 1 -- 172.21.15.59:0/3022761259 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c062130 con 0x5638ade537e0 2024-04-24T06:33:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa7fe00640 1 -- 172.21.15.59:0/3022761259 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c057850 con 0x5638ade537e0 2024-04-24T06:33:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa7fe00640 1 -- 172.21.15.59:0/3022761259 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00ab40 con 0x5638ade537e0 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 -- 172.21.15.59:0/3022761259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 msgr2=0x5638ade285f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faa8c089cc0 tx=0x7faa8c011000 comp rx=0 tx=0).stop 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 -- 172.21.15.59:0/3022761259 shutdown_connections 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade537e0 0x5638ade285f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 --2- 172.21.15.59:0/3022761259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade70e00 0x5638ade23790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.360+0000 7faa9d377740 1 -- 172.21.15.59:0/3022761259 >> 172.21.15.59:0/3022761259 conn(0x5638ade25a00 msgr2=0x5638ade6e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.361+0000 7faa9d377740 1 -- 172.21.15.59:0/3022761259 shutdown_connections 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.361+0000 7faa9d377740 1 -- 172.21.15.59:0/3022761259 wait complete. 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.361+0000 7faa9d377740 1 -- start start 2024-04-24T06:33:21.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.361+0000 7faa9d377740 1 Processor -- start 2024-04-24T06:33:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5638ade27bb0 con 0x5638ade70e00 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5638add46690 con 0x5638ade56910 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.362+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5638ade26fa0 con 0x5638ade53f20 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa9d377740 10 client.0 _ll_register_callbacks cb 0x5638add82650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45698/0 (socket says 172.21.15.59:45698) 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 -- 172.21.15.59:0/3776443023 learned_addr learned my addr 172.21.15.59:0/3776443023 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 -- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 msgr2=0x5638ade82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa93400640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 -- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 msgr2=0x5638ade56d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.363+0000 7faa92a00640 1 -- 172.21.15.59:0/3776443023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638add45920 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa92a00640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa8000be40 tx=0x7faa8000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a5a5f05bd08ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80018030 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa80004020 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa8c057850 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa80021050 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa9d377740 10 client.6522 fetch_fsmap learned FSMap version 35 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5638adb697d0 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.364+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c057850 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7faa80024020 con 0x5638ade70e00 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 10 client.6522 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 10 client.6522 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 20 client.6522 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade51730 0x5638ade7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 4 client.6522 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:33:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638add9ac10 con 0x5638ade51730 2024-04-24T06:33:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.365+0000 7faa93400640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade51730 0x5638ade7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.366+0000 7faa93400640 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade51730 0x5638ade7e6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eba6c49674ea392 server_cookie=c4dcbf85ddd40782 in_seq=0 out_seq=0 2024-04-24T06:33:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.366+0000 7faa7cc00640 10 client.6522 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.366+0000 7faa7cc00640 1 -- 172.21.15.59:0/3776443023 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638add9ac10 con 0x5638ade51730 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.366+0000 7faa7cc00640 10 client.6522 handle_command_reply: tid=0 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.366+0000 7faa9d377740 1 client.6522 shutdown 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 msgr2=0x5638ade4eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa8000be40 tx=0x7faa8000f5b0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 shutdown_connections 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade51730 0x5638ade7e6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade4eb80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 --2- 172.21.15.59:0/3776443023 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638ade82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa7cc00640 0 client.6522 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 >> 172.21.15.59:0/3776443023 conn(0x5638ade537e0 msgr2=0x5638ade4f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 shutdown_connections 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 1 -- 172.21.15.59:0/3776443023 wait complete. 2024-04-24T06:33:21.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.367+0000 7faa9d377740 20 client.6522 trim_cache size 0 max 16384 2024-04-24T06:33:21.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.387+0000 7faa9d377740 1 Processor -- start 2024-04-24T06:33:21.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.387+0000 7faa9d377740 1 -- start start 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade53f20 0x5638ade6e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade56910 0x5638ade6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa94200d60 con 0x5638ade70e00 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa941ffd80 con 0x5638ade53f20 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa941ff0e0 con 0x5638ade56910 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45706/0 (socket says 172.21.15.59:45706) 2024-04-24T06:33:21.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 -- 172.21.15.59:0/2314636070 learned_addr learned my addr 172.21.15.59:0/2314636070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 -- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade56910 msgr2=0x5638ade6e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade56910 0x5638ade6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 -- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade53f20 msgr2=0x5638ade6e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade53f20 0x5638ade6e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.388+0000 7faa93e00640 1 -- 172.21.15.59:0/2314636070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ada9f3f0 con 0x5638ade70e00 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa93e00640 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7faa8c089d50 tx=0x7faa8c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d6aa4255300462 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa7f400640 1 -- 172.21.15.59:0/2314636070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c019310 con 0x5638ade70e00 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa7f400640 1 -- 172.21.15.59:0/2314636070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c062130 con 0x5638ade70e00 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa7f400640 1 -- 172.21.15.59:0/2314636070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c055bc0 con 0x5638ade70e00 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa9d377740 1 -- 172.21.15.59:0/2314636070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 msgr2=0x5638ade7c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.389+0000 7faa9d377740 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7faa8c089d50 tx=0x7faa8c05c040 comp rx=0 tx=0).stop 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 -- 172.21.15.59:0/2314636070 shutdown_connections 2024-04-24T06:33:21.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade7c9c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade56910 0x5638ade6e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 --2- 172.21.15.59:0/2314636070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade53f20 0x5638ade6e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 -- 172.21.15.59:0/2314636070 >> 172.21.15.59:0/2314636070 conn(0x5638ade537e0 msgr2=0x5638ade52ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 -- 172.21.15.59:0/2314636070 shutdown_connections 2024-04-24T06:33:21.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 -- 172.21.15.59:0/2314636070 wait complete. 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.390+0000 7faa9d377740 1 -- start start 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.391+0000 7faa9d377740 1 Processor -- start 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638adee7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade6e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa941ff730 con 0x5638ade70e00 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5638ade26900 con 0x5638ade56910 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5638ade4fb50 con 0x5638ade53f20 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa9d377740 10 client.0 _ll_register_callbacks cb 0x7faa80006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45716/0 (socket says 172.21.15.59:45716) 2024-04-24T06:33:21.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 -- 172.21.15.59:0/4086583111 learned_addr learned my addr 172.21.15.59:0/4086583111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93400640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638adee7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa92a00640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade6e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 -- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 msgr2=0x5638adee7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638adee7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 -- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 msgr2=0x5638ade6e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade6e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.392+0000 7faa93e00640 1 -- 172.21.15.59:0/4086583111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ada5f0f0 con 0x5638ade70e00 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa93e00640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0594e0 tx=0x7faa8c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751ff2ee348e04d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c019310 con 0x5638ade70e00 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7faa8c00ab40 con 0x5638ade70e00 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa8c062130 con 0x5638ade70e00 2024-04-24T06:33:21.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa8c00ad40 con 0x5638ade70e00 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa9d377740 10 client.6528 fetch_fsmap learned FSMap version 35 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.393+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5638add52810 con 0x5638ade70e00 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c062130 con 0x5638ade70e00 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7faa8c0589a0 con 0x5638ade70e00 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 10 client.6528 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 10 client.6528 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 20 client.6528 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade7c9c0 0x5638ade63dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 4 client.6528 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638add94a80 con 0x5638ade7c9c0 2024-04-24T06:33:21.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.394+0000 7faa93400640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade7c9c0 0x5638ade63dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa93400640 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade7c9c0 0x5638ade63dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac791afdd3e9097 server_cookie=616982512630ca94 in_seq=0 out_seq=0 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa7cc00640 10 client.6528 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa7cc00640 1 -- 172.21.15.59:0/4086583111 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5638add94a80 con 0x5638ade7c9c0 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa7cc00640 10 client.6528 handle_command_reply: tid=0 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 client.6528 shutdown 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 msgr2=0x5638ade6e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0594e0 tx=0x7faa8c000f40 comp rx=0 tx=0).stop 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 shutdown_connections 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x5638ade7c9c0 0x5638ade63dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5638ade70e00 0x5638ade6e7d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5638ade56910 0x5638ade6e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 --2- 172.21.15.59:0/4086583111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5638ade53f20 0x5638adee7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 >> 172.21.15.59:0/4086583111 conn(0x5638ade537e0 msgr2=0x5638ade56170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa7cc00640 0 client.6528 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 shutdown_connections 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.395+0000 7faa9d377740 1 -- 172.21.15.59:0/4086583111 wait complete. 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa9d377740 20 client.6528 trim_cache size 0 max 16384 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:33:21.401 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 msgr2=0x7faa7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 0x7faa7406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faa84034000 tx=0x7faa8402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 msgr2=0x7faa940b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c710 tx=0x7faa8c00cbd0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 shutdown_connections 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa94161c50 0x7faa940b2530 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa940d0de0 0x7faa940ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 --2- 172.21.15.59:0/32523710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa940cfc50 0x7faa94155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 >> 172.21.15.59:0/32523710 conn(0x7faa9408bf90 msgr2=0x7faa940d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 shutdown_connections 2024-04-24T06:33:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.396+0000 7faa98e00640 1 -- 172.21.15.59:0/32523710 wait complete. 2024-04-24T06:33:21.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:33:21.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:21.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.616+0000 7f0686e00640 1 Processor -- start 2024-04-24T06:33:21.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.616+0000 7f0686e00640 1 -- start start 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f0686e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f068015c5e0 0x7f06800af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f0686e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06801624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f0686e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f068015f0f0 con 0x7f06800af920 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f0686e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f06800afd80 con 0x7f068015c5e0 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f0686e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f068015fa20 con 0x7f06800d08b0 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f068015c5e0 0x7f06800af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f06800af920 0x7f06800d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39340/0 (socket says 172.21.15.59:39340) 2024-04-24T06:33:21.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067f400640 1 -- 172.21.15.59:0/2328633471 learned_addr learned my addr 172.21.15.59:0/2328633471 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067fe00640 1 -- 172.21.15.59:0/2328633471 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 msgr2=0x7f06801624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067fe00640 1 --2- 172.21.15.59:0/2328633471 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06801624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067fe00640 1 -- 172.21.15.59:0/2328633471 >> v1:172.21.15.59:6789/0 conn(0x7f06800af920 legacy=0x7f06800d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.619+0000 7f067fe00640 1 -- 172.21.15.59:0/2328633471 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068015f3d0 con 0x7f068015c5e0 2024-04-24T06:33:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.618+0000 7f067ea00640 1 -- 172.21.15.59:0/2328633471 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002554485 0 0) 0x7f068015f0f0 con 0x7f06800af920 2024-04-24T06:33:21.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f067fe00640 1 --2- 172.21.15.59:0/2328633471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800af3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f06700097a0 tx=0x7f067002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ca92004945322c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f067ea00640 1 -- 172.21.15.59:0/2328633471 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670037070 con 0x7f068015c5e0 2024-04-24T06:33:21.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f067ea00640 1 -- 172.21.15.59:0/2328633471 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0670032440 con 0x7f068015c5e0 2024-04-24T06:33:21.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f067ea00640 1 -- 172.21.15.59:0/2328633471 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670032810 con 0x7f068015c5e0 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f0686e00640 1 -- 172.21.15.59:0/2328633471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 msgr2=0x7f06800af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.620+0000 7f0686e00640 1 --2- 172.21.15.59:0/2328633471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800af3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f06700097a0 tx=0x7f067002a180 comp rx=0 tx=0).stop 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 -- 172.21.15.59:0/2328633471 shutdown_connections 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 --2- 172.21.15.59:0/2328633471 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06801624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 --2- 172.21.15.59:0/2328633471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800af3e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 -- 172.21.15.59:0/2328633471 >> 172.21.15.59:0/2328633471 conn(0x7f0680067ea0 msgr2=0x7f068006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 -- 172.21.15.59:0/2328633471 shutdown_connections 2024-04-24T06:33:21.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.621+0000 7f0686e00640 1 -- 172.21.15.59:0/2328633471 wait complete. 2024-04-24T06:33:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.622+0000 7f0686e00640 1 Processor -- start 2024-04-24T06:33:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 -- start start 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06800af920 0x7f0680157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06800ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f068006e600 con 0x7f068015c5e0 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f068006e480 con 0x7f06800af920 2024-04-24T06:33:21.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.623+0000 7f0686e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0680162a10 con 0x7f06800d08b0 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45742/0 (socket says 172.21.15.59:45742) 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 -- 172.21.15.59:0/3378078410 learned_addr learned my addr 172.21.15.59:0/3378078410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f067fe00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06800af920 0x7f0680157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 -- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 msgr2=0x7f06800ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06800ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 -- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06800af920 msgr2=0x7f0680157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06800af920 0x7f0680157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 -- 172.21.15.59:0/3378078410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06801e8e30 con 0x7f068015c5e0 2024-04-24T06:33:21.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.624+0000 7f0684e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0678009fd0 tx=0x7f067800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.625+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678011830 con 0x7f068015c5e0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.625+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0678016070 con 0x7f068015c5e0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.625+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678011c10 con 0x7f068015c5e0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.625+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06801e7ab0 con 0x7f068015c5e0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.625+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801e8070 con 0x7f068015c5e0 2024-04-24T06:33:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.626+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0680068a80 con 0x7f068015c5e0 2024-04-24T06:33:21.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.627+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f067801b460 con 0x7f068015c5e0 2024-04-24T06:33:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.628+0000 7f0677e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 0x7f065c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.628+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f067808b830 con 0x7f068015c5e0 2024-04-24T06:33:21.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.628+0000 7f067fe00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 0x7f065c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:21.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.629+0000 7f067fe00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 0x7f065c06edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0670034000 tx=0x7f067002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:21.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.633+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0678059c50 con 0x7f068015c5e0 2024-04-24T06:33:21.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.877+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06800ced10 con 0x7f068015c5e0 2024-04-24T06:33:21.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.879+0000 7f0677e00640 1 -- 172.21.15.59:0/3378078410 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7f0678059550 con 0x7f068015c5e0 2024-04-24T06:33:21.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:21.881 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:56.100001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:33:21.881 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-24T06:33:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 msgr2=0x7f065c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 0x7f065c06edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0670034000 tx=0x7f067002e000 comp rx=0 tx=0).stop 2024-04-24T06:33:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 msgr2=0x7f06800adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:21.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0678009fd0 tx=0x7f067800b8e0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 shutdown_connections 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f065c06c8f0 0x7f065c06edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f068015c5e0 0x7f06800adae0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06800d08b0 0x7f06800ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 --2- 172.21.15.59:0/3378078410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06800af920 0x7f0680157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.881+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 >> 172.21.15.59:0/3378078410 conn(0x7f0680067ea0 msgr2=0x7f068006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.882+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 shutdown_connections 2024-04-24T06:33:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:21.882+0000 7f0686e00640 1 -- 172.21.15.59:0/3378078410 wait complete. 2024-04-24T06:33:21.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-24T06:33:22.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.111+0000 7fb68d400640 1 Processor -- start 2024-04-24T06:33:22.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.111+0000 7fb68d400640 1 -- start start 2024-04-24T06:33:22.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.112+0000 7fb68d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6880d0ce0 0x7fb6880af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6880af5c0 0x7fb688071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb688160120 con 0x7fb68815c410 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb688160400 con 0x7fb6880af5c0 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb6881606e0 con 0x7fb6880d0ce0 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6880af5c0 0x7fb688071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6880af5c0 0x7fb688071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45746/0 (socket says 172.21.15.59:45746) 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 -- 172.21.15.59:0/3567689947 learned_addr learned my addr 172.21.15.59:0/3567689947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 -- 172.21.15.59:0/3567689947 >> v2:172.21.15.66:3300/0 conn(0x7fb6880d0ce0 msgr2=0x7fb6880af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 --2- 172.21.15.59:0/3567689947 >> v2:172.21.15.66:3300/0 conn(0x7fb6880d0ce0 0x7fb6880af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 -- 172.21.15.59:0/3567689947 >> v1:172.21.15.66:6789/0 conn(0x7fb68815c410 legacy=0x7fb6880d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.113+0000 7fb68ca00640 1 -- 172.21.15.59:0/3567689947 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6881609c0 con 0x7fb6880af5c0 2024-04-24T06:33:22.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.114+0000 7fb68ca00640 1 --2- 172.21.15.59:0/3567689947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb688071c70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb6880b4560 tx=0x7fb68002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f35cacb4c81ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.115+0000 7fb686a00640 1 -- 172.21.15.59:0/3567689947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680036070 con 0x7fb6880af5c0 2024-04-24T06:33:22.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.115+0000 7fb686a00640 1 -- 172.21.15.59:0/3567689947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb680031600 con 0x7fb6880af5c0 2024-04-24T06:33:22.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.115+0000 7fb686a00640 1 -- 172.21.15.59:0/3567689947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800319d0 con 0x7fb6880af5c0 2024-04-24T06:33:22.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.115+0000 7fb68d400640 1 -- 172.21.15.59:0/3567689947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 msgr2=0x7fb688071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.115+0000 7fb68d400640 1 --2- 172.21.15.59:0/3567689947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb688071c70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb6880b4560 tx=0x7fb68002a660 comp rx=0 tx=0).stop 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 -- 172.21.15.59:0/3567689947 shutdown_connections 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 --2- 172.21.15.59:0/3567689947 >> v2:172.21.15.66:3300/0 conn(0x7fb6880d0ce0 0x7fb6880af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 --2- 172.21.15.59:0/3567689947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb688071c70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 -- 172.21.15.59:0/3567689947 >> 172.21.15.59:0/3567689947 conn(0x7fb68806bdf0 msgr2=0x7fb68806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 -- 172.21.15.59:0/3567689947 shutdown_connections 2024-04-24T06:33:22.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.116+0000 7fb68d400640 1 -- 172.21.15.59:0/3567689947 wait complete. 2024-04-24T06:33:22.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.117+0000 7fb68d400640 1 Processor -- start 2024-04-24T06:33:22.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.117+0000 7fb68d400640 1 -- start start 2024-04-24T06:33:22.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 0x7fb6881450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb68815c410 0x7fb6880c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880afb80 con 0x7fb6880af5c0 2024-04-24T06:33:22.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880afa00 con 0x7fb6880d0ce0 2024-04-24T06:33:22.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.118+0000 7fb68d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880721b0 con 0x7fb68815c410 2024-04-24T06:33:22.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45750/0 (socket says 172.21.15.59:45750) 2024-04-24T06:33:22.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 -- 172.21.15.59:0/3496318244 learned_addr learned my addr 172.21.15.59:0/3496318244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 -- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb68815c410 msgr2=0x7fb6880c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb68815c410 0x7fb6880c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 -- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 msgr2=0x7fb6881450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 0x7fb6881450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 0x7fb6881450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 -- 172.21.15.59:0/3496318244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68806dec0 con 0x7fb6880af5c0 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 0x7fb6881450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.119+0000 7fb687e00640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3e0 tx=0x7fb67800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.120+0000 7fb67fe00640 1 -- 172.21.15.59:0/3496318244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678015020 con 0x7fb6880af5c0 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.120+0000 7fb67fe00640 1 -- 172.21.15.59:0/3496318244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb678011850 con 0x7fb6880af5c0 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.120+0000 7fb67fe00640 1 -- 172.21.15.59:0/3496318244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678011c40 con 0x7fb6880af5c0 2024-04-24T06:33:22.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.120+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68806e1a0 con 0x7fb6880af5c0 2024-04-24T06:33:22.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.120+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68806e760 con 0x7fb6880af5c0 2024-04-24T06:33:22.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.122+0000 7fb67fe00640 1 -- 172.21.15.59:0/3496318244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb678018020 con 0x7fb6880af5c0 2024-04-24T06:33:22.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.123+0000 7fb67fe00640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 0x7fb66806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.123+0000 7fb67fe00640 1 -- 172.21.15.59:0/3496318244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb67808bbb0 con 0x7fb6880af5c0 2024-04-24T06:33:22.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.123+0000 7fb687400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 0x7fb66806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.123+0000 7fb6919ce740 1 Processor -- start 2024-04-24T06:33:22.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 -- start start 2024-04-24T06:33:22.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb687400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 0x7fb66806edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb674006fd0 tx=0x7fb674008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b0395f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 0x55b03960c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b03952a190 con 0x55b03960dac0 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b0394d0030 con 0x55b0395c5f00 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.124+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b039526150 con 0x55b0395f0220 2024-04-24T06:33:22.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45760/0 (socket says 172.21.15.59:45760) 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 -- 172.21.15.59:0/3627964618 learned_addr learned my addr 172.21.15.59:0/3627964618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687e00640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b0395f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb68ca00640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 0x55b03960c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 -- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 msgr2=0x55b0395f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b0395f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 -- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 msgr2=0x55b03960c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 0x55b03960c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687400640 1 -- 172.21.15.59:0/3627964618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0394b8b60 con 0x55b03960dac0 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb687e00640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b0395f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.125+0000 7fb68ca00640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 0x55b03960c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:22.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb687400640 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb67400ca30 tx=0x7fb67400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0ce639938a9c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb673e00640 1 -- 172.21.15.59:0/3627964618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674023070 con 0x55b03960dac0 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb673e00640 1 -- 172.21.15.59:0/3627964618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674013db0 con 0x55b03960dac0 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb673e00640 1 -- 172.21.15.59:0/3627964618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67401e620 con 0x55b03960dac0 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 -- 172.21.15.59:0/3627964618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 msgr2=0x55b03960e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb67400ca30 tx=0x7fb67400f000 comp rx=0 tx=0).stop 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 -- 172.21.15.59:0/3627964618 shutdown_connections 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395c5f00 0x55b03960c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b03960dac0 0x55b03960e010 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3627964618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b0395f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.126+0000 7fb6919ce740 1 -- 172.21.15.59:0/3627964618 >> 172.21.15.59:0/3627964618 conn(0x55b0395c2830 msgr2=0x55b03960b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.127+0000 7fb6919ce740 1 -- 172.21.15.59:0/3627964618 shutdown_connections 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.127+0000 7fb6919ce740 1 -- 172.21.15.59:0/3627964618 wait complete. 2024-04-24T06:33:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.127+0000 7fb6919ce740 1 -- start start 2024-04-24T06:33:22.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.127+0000 7fb6919ce740 1 Processor -- start 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b0396020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395e3ab0 0x55b0395f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0395c6ca0 con 0x55b0395f0220 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b0394e3500 con 0x55b03960dac0 2024-04-24T06:33:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.128+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b0395c0760 con 0x55b0395e3ab0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb6919ce740 10 client.0 _ll_register_callbacks cb 0x55b039520270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45766/0 (socket says 172.21.15.59:45766) 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 -- 172.21.15.59:0/1159678588 learned_addr learned my addr 172.21.15.59:0/1159678588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687400640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b0396020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 -- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395e3ab0 msgr2=0x55b0395f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395e3ab0 0x55b0395f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 -- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 msgr2=0x55b0396020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b0396020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 -- 172.21.15.59:0/1159678588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0393087c0 con 0x55b0395f0220 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb687e00640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6780046f0 tx=0x7fb6780592b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf29a21e303568b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.129+0000 7fb670c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67800fdc0 con 0x55b0395f0220 2024-04-24T06:33:22.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb678055eb0 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb674013db0 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb67805ab70 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 10 client.6549 fetch_fsmap learned FSMap version 35 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55b039368f50 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674013db0 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb678070030 con 0x55b0395f0220 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 10 client.6549 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 10 client.6549 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 20 client.6549 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b039604430 0x55b039604810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 4 client.6549 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.130+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b039537bb0 con 0x55b039604430 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.131+0000 7fb687400640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b039604430 0x55b039604810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.131+0000 7fb687400640 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b039604430 0x55b039604810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e630a0b36ccbb5 server_cookie=794afce66f18d2ca in_seq=0 out_seq=0 2024-04-24T06:33:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.131+0000 7fb670c00640 10 client.6549 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb670c00640 1 -- 172.21.15.59:0/1159678588 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b039537bb0 con 0x55b039604430 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb670c00640 10 client.6549 handle_command_reply: tid=0 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 client.6549 shutdown 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 msgr2=0x55b03961b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6780046f0 tx=0x7fb6780592b0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 shutdown_connections 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b039604430 0x55b039604810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395e3ab0 0x55b0395f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b0396020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 --2- 172.21.15.59:0/1159678588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03961b120 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb670c00640 0 client.6549 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 >> 172.21.15.59:0/1159678588 conn(0x55b0395c5f00 msgr2=0x55b03960e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 shutdown_connections 2024-04-24T06:33:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.132+0000 7fb6919ce740 1 -- 172.21.15.59:0/1159678588 wait complete. 2024-04-24T06:33:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.133+0000 7fb6919ce740 20 client.6549 trim_cache size 0 max 16384 2024-04-24T06:33:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.147+0000 7fb6919ce740 1 Processor -- start 2024-04-24T06:33:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.147+0000 7fb6919ce740 1 -- start start 2024-04-24T06:33:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 0x55b03960b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b03960dac0 0x55b0395eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881ffdf0 con 0x55b0395f0220 2024-04-24T06:33:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881ff300 con 0x55b0395e3ab0 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6881ff7a0 con 0x55b03960dac0 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb687400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb687400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45776/0 (socket says 172.21.15.59:45776) 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.148+0000 7fb687400640 1 -- 172.21.15.59:0/3045565080 learned_addr learned my addr 172.21.15.59:0/3045565080 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 -- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b03960dac0 msgr2=0x55b0395eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b03960dac0 0x55b0395eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687e00640 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 0x55b03960b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 -- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 msgr2=0x55b03960b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 0x55b03960b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 -- 172.21.15.59:0/3045565080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0391fc0b0 con 0x55b0395f0220 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687e00640 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 0x55b03960b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb687400640 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb67400a1a0 tx=0x7fb674011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c444276c9be344ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb673400640 1 -- 172.21.15.59:0/3045565080 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402fe60 con 0x55b0395f0220 2024-04-24T06:33:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb673400640 1 -- 172.21.15.59:0/3045565080 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67402f3c0 con 0x55b0395f0220 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.149+0000 7fb673400640 1 -- 172.21.15.59:0/3045565080 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674023040 con 0x55b0395f0220 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 -- 172.21.15.59:0/3045565080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 msgr2=0x55b03960b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb67400a1a0 tx=0x7fb674011000 comp rx=0 tx=0).stop 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 -- 172.21.15.59:0/3045565080 shutdown_connections 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b03960dac0 0x55b0395eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395f0220 0x55b03960b860 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 --2- 172.21.15.59:0/3045565080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b0395e3ab0 0x55b03960b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 -- 172.21.15.59:0/3045565080 >> 172.21.15.59:0/3045565080 conn(0x55b0395c5f00 msgr2=0x55b03961af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 -- 172.21.15.59:0/3045565080 shutdown_connections 2024-04-24T06:33:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.150+0000 7fb6919ce740 1 -- 172.21.15.59:0/3045565080 wait complete. 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.151+0000 7fb6919ce740 1 -- start start 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.151+0000 7fb6919ce740 1 Processor -- start 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b039605730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b03962e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b039605070 con 0x55b0395e3ab0 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b0396037f0 con 0x55b03960dac0 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b03961f300 con 0x55b0395f0220 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb6919ce740 10 client.0 _ll_register_callbacks cb 0x7fb678059610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb687e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45778/0 (socket says 172.21.15.59:45778) 2024-04-24T06:33:22.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.152+0000 7fb687e00640 1 -- 172.21.15.59:0/488946822 learned_addr learned my addr 172.21.15.59:0/488946822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 -- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 msgr2=0x55b039605730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b039605730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 -- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 msgr2=0x55b03962e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b03962e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 -- 172.21.15.59:0/488946822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b039606220 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb687e00640 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6780597d0 tx=0x7fb678089f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297afdd5ed73131 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb670c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67800fdc0 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb678055790 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb67402f3c0 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb678082050 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb6919ce740 10 client.6555 fetch_fsmap learned FSMap version 35 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.153+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55b03961db20 con 0x55b0395e3ab0 2024-04-24T06:33:22.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67402f3c0 con 0x55b0395e3ab0 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb678070040 con 0x55b0395e3ab0 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 10 client.6555 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 10 client.6555 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 20 client.6555 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b03962ec70 0x55b03960af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 4 client.6555 mds_command: new command op to 6192 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 --> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b0396035a0 con 0x55b03962ec70 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.154+0000 7fb687400640 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b03962ec70 0x55b03960af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb687400640 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b03962ec70 0x55b03960af70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb4f13eb7c3ca05 server_cookie=9bd06ab7b0dd6c8f in_seq=0 out_seq=0 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb670c00640 10 client.6555 ms_handle_connect on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb670c00640 1 -- 172.21.15.59:0/488946822 <== mds.0 v2:172.21.15.59:6835/2525582962 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0396035a0 con 0x55b03962ec70 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb670c00640 10 client.6555 handle_command_reply: tid=0 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 client.6555 shutdown 2024-04-24T06:33:22.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 msgr2=0x55b0396051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6780597d0 tx=0x7fb678089f10 comp rx=0 tx=0).stop 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 shutdown_connections 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:6835/2525582962,v1:172.21.15.59:6837/2525582962] conn(0x55b03962ec70 0x55b03960af70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b03960dac0 0x55b03962e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b0395f0220 0x55b039605730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb670c00640 0 client.6555 ms_handle_reset on v2:172.21.15.59:6835/2525582962 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 --2- 172.21.15.59:0/488946822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0395e3ab0 0x55b0396051f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 >> 172.21.15.59:0/488946822 conn(0x55b0395c5f00 msgr2=0x55b03962f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 shutdown_connections 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.155+0000 7fb6919ce740 1 -- 172.21.15.59:0/488946822 wait complete. 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb6919ce740 20 client.6555 trim_cache size 0 max 16384 2024-04-24T06:33:22.161 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:22.162 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-24T06:33:22.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 msgr2=0x7fb66806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 0x7fb66806edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb674006fd0 tx=0x7fb674008040 comp rx=0 tx=0).stop 2024-04-24T06:33:22.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 msgr2=0x7fb6880caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3e0 tx=0x7fb67800b8a0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 shutdown_connections 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb66806c8f0 0x7fb66806edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb68815c410 0x7fb6880c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6880d0ce0 0x7fb6881450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 --2- 172.21.15.59:0/3496318244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6880af5c0 0x7fb6880caf30 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 >> 172.21.15.59:0/3496318244 conn(0x7fb68806bdf0 msgr2=0x7fb6880b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 shutdown_connections 2024-04-24T06:33:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.156+0000 7fb68d400640 1 -- 172.21.15.59:0/3496318244 wait complete. 2024-04-24T06:33:22.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:33:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.363+0000 7f536fe00640 1 Processor -- start 2024-04-24T06:33:22.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.363+0000 7f536fe00640 1 -- start start 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.364+0000 7f536fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53700cd760 0x7f53700b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.364+0000 7f536fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5370160210 con 0x7f53700b4690 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.364+0000 7f536fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53701604f0 con 0x7f537015c3d0 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.364+0000 7f536fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f53701607d0 con 0x7f53700cd760 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53700cd760 0x7f53700b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53700cd760 0x7f53700b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45788/0 (socket says 172.21.15.59:45788) 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 -- 172.21.15.59:0/3077974385 learned_addr learned my addr 172.21.15.59:0/3077974385 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f536f400640 1 --1- 172.21.15.59:0/3077974385 >> v1:172.21.15.59:6789/0 conn(0x7f53700b4690 0x7f53700cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39346/0 (socket says 172.21.15.59:39346) 2024-04-24T06:33:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 -- 172.21.15.59:0/3077974385 >> v1:172.21.15.66:6789/0 conn(0x7f537015c3d0 legacy=0x7f53700cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 -- 172.21.15.59:0/3077974385 >> v1:172.21.15.59:6789/0 conn(0x7f53700b4690 legacy=0x7f53700cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.365+0000 7f5367e00640 1 -- 172.21.15.59:0/3077974385 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5370160ab0 con 0x7f53700cd760 2024-04-24T06:33:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.366+0000 7f5367e00640 1 --2- 172.21.15.59:0/3077974385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700cd760 0x7f53700b4150 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5358009870 tx=0x7f535802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd0b00ae86bf832 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.367+0000 7f536e000640 1 -- 172.21.15.59:0/3077974385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358037070 con 0x7f53700cd760 2024-04-24T06:33:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.367+0000 7f536e000640 1 -- 172.21.15.59:0/3077974385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535802ae50 con 0x7f53700cd760 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.367+0000 7f536e000640 1 -- 172.21.15.59:0/3077974385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358032860 con 0x7f53700cd760 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.367+0000 7f536fe00640 1 -- 172.21.15.59:0/3077974385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700cd760 msgr2=0x7f53700b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.367+0000 7f536fe00640 1 --2- 172.21.15.59:0/3077974385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700cd760 0x7f53700b4150 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5358009870 tx=0x7f535802a300 comp rx=0 tx=0).stop 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.368+0000 7f536fe00640 1 -- 172.21.15.59:0/3077974385 shutdown_connections 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.368+0000 7f536fe00640 1 --2- 172.21.15.59:0/3077974385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700cd760 0x7f53700b4150 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.368+0000 7f536fe00640 1 -- 172.21.15.59:0/3077974385 >> 172.21.15.59:0/3077974385 conn(0x7f53700b21a0 msgr2=0x7f53700b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.368+0000 7f536fe00640 1 -- 172.21.15.59:0/3077974385 shutdown_connections 2024-04-24T06:33:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.368+0000 7f536fe00640 1 -- 172.21.15.59:0/3077974385 wait complete. 2024-04-24T06:33:22.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.369+0000 7f536fe00640 1 Processor -- start 2024-04-24T06:33:22.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.370+0000 7f536fe00640 1 -- start start 2024-04-24T06:33:22.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 0x7f537015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f537015c3d0 0x7f53700c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f537015c7b0 con 0x7f53700b4690 2024-04-24T06:33:22.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53700b4ae0 con 0x7f53700cd760 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f537015c930 con 0x7f537015c3d0 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45804/0 (socket says 172.21.15.59:45804) 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 -- 172.21.15.59:0/1281504417 learned_addr learned my addr 172.21.15.59:0/1281504417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f5367e00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 0x7f537015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 -- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f537015c3d0 msgr2=0x7f53700c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f537015c3d0 0x7f53700c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 -- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 msgr2=0x7f537015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 0x7f537015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f536ea00640 1 -- 172.21.15.59:0/1281504417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700c7a60 con 0x7f53700b4690 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.371+0000 7f5367e00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 0x7f537015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f536ea00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f536000c430 tx=0x7f536000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360017070 con 0x7f53700b4690 2024-04-24T06:33:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53600052d0 con 0x7f53700b4690 2024-04-24T06:33:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360003830 con 0x7f53700b4690 2024-04-24T06:33:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53700c7d20 con 0x7f53700b4690 2024-04-24T06:33:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.372+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53700c82e0 con 0x7f53700b4690 2024-04-24T06:33:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.373+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537004fd90 con 0x7f53700b4690 2024-04-24T06:33:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.374+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f536001a070 con 0x7f53700b4690 2024-04-24T06:33:22.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.375+0000 7f5366a00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 0x7f534c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:22.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.375+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f536008aef0 con 0x7f53700b4690 2024-04-24T06:33:22.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.375+0000 7f5367e00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 0x7f534c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:22.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.376+0000 7f5367e00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 0x7f534c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5358034000 tx=0x7f535802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:22.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.380+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5360059280 con 0x7f53700b4690 2024-04-24T06:33:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.626+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f537006f240 con 0x7f53700b4690 2024-04-24T06:33:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.627+0000 7f5366a00640 1 -- 172.21.15.59:0/1281504417 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5360058b80 con 0x7f53700b4690 2024-04-24T06:33:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:22.629 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 msgr2=0x7f534c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 0x7f534c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5358034000 tx=0x7f535802e000 comp rx=0 tx=0).stop 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 msgr2=0x7f537015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f536000c430 tx=0x7f536000c8f0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 shutdown_connections 2024-04-24T06:33:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f534c06c8f0 0x7f534c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f537015c3d0 0x7f53700c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53700cd760 0x7f537015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 --2- 172.21.15.59:0/1281504417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53700b4690 0x7f537015cab0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 >> 172.21.15.59:0/1281504417 conn(0x7f53700b21a0 msgr2=0x7f53700b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 shutdown_connections 2024-04-24T06:33:22.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:22.630+0000 7f536fe00640 1 -- 172.21.15.59:0/1281504417 wait complete. 2024-04-24T06:33:22.650 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:33:22.650 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:33:22.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:33:22.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:33:22.881 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:33:22.881 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:33:22.881 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:22.881 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:22.941 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:23.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.158+0000 7ff8f0a00640 1 Processor -- start 2024-04-24T06:33:23.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.158+0000 7ff8f0a00640 1 -- start start 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8f0a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8ec06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8f0a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec06ed40 con 0x7ff8ec0b4130 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8f0a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec0b3220 con 0x7ff8ec162df0 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8f0a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec06eaf0 con 0x7ff8ec0c38b0 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8eaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff8ec0b4130 0x7ff8ec0c31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39352/0 (socket says 172.21.15.59:39352) 2024-04-24T06:33:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.160+0000 7ff8eaa00640 1 -- 172.21.15.59:0/1849483695 learned_addr learned my addr 172.21.15.59:0/1849483695 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:23.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ebe00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8ec06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760520222 0 0) 0x7ff8ec06ed40 con 0x7ff8ec0b4130 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8d0003520 con 0x7ff8ec0b4130 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2470651425 0 0) 0x7ff8d0003520 con 0x7ff8ec0b4130 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 msgr2=0x7ff8ec06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8ec06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 >> v1:172.21.15.66:6789/0 conn(0x7ff8ec162df0 legacy=0x7ff8ec0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec0af4c0 con 0x7ff8ec0b4130 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff8d8001670 con 0x7ff8ec0b4130 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 >> v1:172.21.15.59:6789/0 conn(0x7ff8ec0b4130 legacy=0x7ff8ec0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d0003f60 0x7ff8d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec06eaf0 con 0x7ff8d0004d40 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d000baa0 con 0x7ff8d0003f60 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.161+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d000bca0 con 0x7ff8ec0c38b0 2024-04-24T06:33:23.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8eaa00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d0003f60 0x7ff8d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8eb400640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8d00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 -- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 msgr2=0x7ff8d00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec0c38b0 0x7ff8d00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 -- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d0003f60 msgr2=0x7ff8d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d0003f60 0x7ff8d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8ebe00640 1 -- 172.21.15.59:0/1849483695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d0003520 con 0x7ff8d0004d40 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.162+0000 7ff8eaa00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8d0003f60 0x7ff8d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:33:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.164+0000 7ff8ebe00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8d00051c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec06f2e0 tx=0x7ff8d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245a6818c567492b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.164+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8d0004d40 2024-04-24T06:33:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.164+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e402e050 con 0x7ff8d0004d40 2024-04-24T06:33:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.164+0000 7ff8ea000640 1 -- 172.21.15.59:0/1849483695 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4009c10 con 0x7ff8d0004d40 2024-04-24T06:33:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 -- 172.21.15.59:0/1849483695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 msgr2=0x7ff8d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/1849483695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8d00051c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec06f2e0 tx=0x7ff8d0005830 comp rx=0 tx=0).stop 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8ebe00640 1 -- 172.21.15.59:0/1849483695 reap_dead start 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 -- 172.21.15.59:0/1849483695 shutdown_connections 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 -- 172.21.15.59:0/1849483695 >> 172.21.15.59:0/1849483695 conn(0x7ff8ec0b21a0 msgr2=0x7ff8ec0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 -- 172.21.15.59:0/1849483695 shutdown_connections 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.165+0000 7ff8f0a00640 1 -- 172.21.15.59:0/1849483695 wait complete. 2024-04-24T06:33:23.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.167+0000 7ff8f0a00640 1 Processor -- start 2024-04-24T06:33:23.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.167+0000 7ff8f0a00640 1 -- start start 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ec1631d0 0x7ff8ec0ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec161760 0x7ff8ec157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0b3590 con 0x7ff8d0004d40 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0cfde0 con 0x7ff8ec1631d0 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8f0a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec06f680 con 0x7ff8ec161760 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45810/0 (socket says 172.21.15.59:45810) 2024-04-24T06:33:23.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 -- 172.21.15.59:0/2632169118 learned_addr learned my addr 172.21.15.59:0/2632169118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:23.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 -- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec161760 msgr2=0x7ff8ec157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec161760 0x7ff8ec157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eaa00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ec1631d0 0x7ff8ec0ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.168+0000 7ff8eb400640 1 -- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ec1631d0 msgr2=0x7ff8ec0ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8eb400640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ec1631d0 0x7ff8ec0ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8eb400640 1 -- 172.21.15.59:0/2632169118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec0c84e0 con 0x7ff8d0004d40 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8eb400640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e4d0 tx=0x7ff8dc00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0052e0 con 0x7ff8d0004d40 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc018070 con 0x7ff8d0004d40 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc003810 con 0x7ff8d0004d40 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec0c87c0 con 0x7ff8d0004d40 2024-04-24T06:33:23.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.169+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec0c68a0 con 0x7ff8d0004d40 2024-04-24T06:33:23.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.171+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec0afac0 con 0x7ff8d0004d40 2024-04-24T06:33:23.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.171+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8dc0039b0 con 0x7ff8d0004d40 2024-04-24T06:33:23.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.172+0000 7ff8e3400640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 0x7ff8cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.173+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff8dc08b810 con 0x7ff8d0004d40 2024-04-24T06:33:23.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.173+0000 7ff8eaa00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 0x7ff8cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.173+0000 7ff8eaa00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 0x7ff8cc06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8001eb0 tx=0x7ff8d8001e00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.177+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8ec0afac0 con 0x7ff8d0004d40 2024-04-24T06:33:23.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.376+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8ec075150 con 0x7ff8d0004d40 2024-04-24T06:33:23.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.378+0000 7ff8e3400640 1 -- 172.21.15.59:0/2632169118 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16581 (secure 0 0 0) 0x7ff8dc059ba0 con 0x7ff8d0004d40 2024-04-24T06:33:23.380 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:23.380 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:09.051581+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 msgr2=0x7ff8cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 0x7ff8cc06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8001eb0 tx=0x7ff8d8001e00 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 msgr2=0x7ff8ec0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e4d0 tx=0x7ff8dc00e990 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 shutdown_connections 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff8cc06c920 0x7ff8cc06ede0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff8ec161760 0x7ff8ec157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8ec1631d0 0x7ff8ec0ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 --2- 172.21.15.59:0/2632169118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8d0004d40 0x7ff8ec0c6360 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 >> 172.21.15.59:0/2632169118 conn(0x7ff8ec0b21a0 msgr2=0x7ff8ec1621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 shutdown_connections 2024-04-24T06:33:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.381+0000 7ff8f0a00640 1 -- 172.21.15.59:0/2632169118 wait complete. 2024-04-24T06:33:23.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:32:05.373199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:32:06.536666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:23.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:23.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.595+0000 7f1a3c400640 1 Processor -- start 2024-04-24T06:33:23.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.595+0000 7f1a3c400640 1 -- start start 2024-04-24T06:33:23.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.596+0000 7f1a3c400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a3415c890 0x7f1a340ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.596+0000 7f1a3c400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1a3406c0e0 0x7f1a340cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.596+0000 7f1a3c400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3415cc70 con 0x7f1a3415d9d0 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.596+0000 7f1a3c400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1a340cdda0 con 0x7f1a3415c890 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.596+0000 7f1a3c400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3415d540 con 0x7f1a3406c0e0 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a3415c890 0x7f1a340ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a3415c890 0x7f1a340ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45818/0 (socket says 172.21.15.59:45818) 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 -- 172.21.15.59:0/3976383458 learned_addr learned my addr 172.21.15.59:0/3976383458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:23.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 -- 172.21.15.59:0/3976383458 >> v2:172.21.15.66:3300/0 conn(0x7f1a3406c0e0 msgr2=0x7f1a340cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 --2- 172.21.15.59:0/3976383458 >> v2:172.21.15.66:3300/0 conn(0x7f1a3406c0e0 0x7f1a340cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 -- 172.21.15.59:0/3976383458 >> v1:172.21.15.66:6789/0 conn(0x7f1a3415d9d0 legacy=0x7f1a3406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.597+0000 7f1a39a00640 1 -- 172.21.15.59:0/3976383458 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340cdac0 con 0x7f1a3415c890 2024-04-24T06:33:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.598+0000 7f1a39a00640 1 --2- 172.21.15.59:0/3976383458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3415c890 0x7f1a340ccf20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a340711e0 tx=0x7f1a2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afa3fe99b0bf770 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.599+0000 7f1a33e00640 1 -- 172.21.15.59:0/3976383458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28037070 con 0x7f1a3415c890 2024-04-24T06:33:23.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.599+0000 7f1a33e00640 1 -- 172.21.15.59:0/3976383458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a28032440 con 0x7f1a3415c890 2024-04-24T06:33:23.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.599+0000 7f1a33e00640 1 -- 172.21.15.59:0/3976383458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28032810 con 0x7f1a3415c890 2024-04-24T06:33:23.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.599+0000 7f1a3c400640 1 -- 172.21.15.59:0/3976383458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3415c890 msgr2=0x7f1a340ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.599+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3976383458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3415c890 0x7f1a340ccf20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a340711e0 tx=0x7f1a2802a210 comp rx=0 tx=0).stop 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 -- 172.21.15.59:0/3976383458 shutdown_connections 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3976383458 >> v2:172.21.15.66:3300/0 conn(0x7f1a3406c0e0 0x7f1a340cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3976383458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3415c890 0x7f1a340ccf20 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 -- 172.21.15.59:0/3976383458 >> 172.21.15.59:0/3976383458 conn(0x7f1a34073430 msgr2=0x7f1a3409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 -- 172.21.15.59:0/3976383458 shutdown_connections 2024-04-24T06:33:23.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.600+0000 7f1a3c400640 1 -- 172.21.15.59:0/3976383458 wait complete. 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.601+0000 7f1a3c400640 1 Processor -- start 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.602+0000 7f1a3c400640 1 -- start start 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.602+0000 7f1a3c400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.602+0000 7f1a3c400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a3415c890 0x7f1a340af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.602+0000 7f1a3c400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 0x7f1a340af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a3c400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340ceb90 con 0x7f1a3406c0e0 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a3c400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3415d1d0 con 0x7f1a3415d9d0 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a3c400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1a340ced10 con 0x7f1a3415c890 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45828/0 (socket says 172.21.15.59:45828) 2024-04-24T06:33:23.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 -- 172.21.15.59:0/3524948021 learned_addr learned my addr 172.21.15.59:0/3524948021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39000640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a3415c890 0x7f1a340af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a3a400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 0x7f1a340af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 -- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a3415c890 msgr2=0x7f1a340af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a3415c890 0x7f1a340af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 -- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 msgr2=0x7f1a340af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 0x7f1a340af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a39a00640 1 -- 172.21.15.59:0/3524948021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3415bde0 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.603+0000 7f1a3a400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 0x7f1a340af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a39a00640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28041070 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2803c890 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a341f80c0 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28037040 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.604+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a340c7420 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.605+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34069190 con 0x7f1a3406c0e0 2024-04-24T06:33:23.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.606+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a28044070 con 0x7f1a3406c0e0 2024-04-24T06:33:23.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.607+0000 7f1a31600640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 0x7f1a1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.607+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1a280ac7b0 con 0x7f1a3406c0e0 2024-04-24T06:33:23.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.608+0000 7f1a39000640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 0x7f1a1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.608+0000 7f1a39000640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 0x7f1a1406ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a20006fd0 tx=0x7f1a20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.612+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a3415cc70 con 0x7f1a3406c0e0 2024-04-24T06:33:23.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.812+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a340c8390 con 0x7f1a3406c0e0 2024-04-24T06:33:23.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.813+0000 7f1a31600640 1 -- 172.21.15.59:0/3524948021 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f1a2807acf0 con 0x7f1a3406c0e0 2024-04-24T06:33:23.815 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.815+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 msgr2=0x7f1a1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.815+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 0x7f1a1406ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a20006fd0 tx=0x7f1a20008040 comp rx=0 tx=0).stop 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 msgr2=0x7f1a340c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a280046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 shutdown_connections 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a1406c920 0x7f1a1406ede0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a3415d9d0 0x7f1a340af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a3415c890 0x7f1a340af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 --2- 172.21.15.59:0/3524948021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a3406c0e0 0x7f1a340c6ee0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 >> 172.21.15.59:0/3524948021 conn(0x7f1a34073430 msgr2=0x7f1a3415f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 shutdown_connections 2024-04-24T06:33:23.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:23.816+0000 7f1a3c400640 1 -- 172.21.15.59:0/3524948021 wait complete. 2024-04-24T06:33:23.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:33:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.045+0000 7f6efae00640 1 Processor -- start 2024-04-24T06:33:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.045+0000 7f6efae00640 1 -- start start 2024-04-24T06:33:24.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6efae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ef406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6efae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef415bf60 con 0x7f6ef406f340 2024-04-24T06:33:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6efae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef415c240 con 0x7f6ef40d0a60 2024-04-24T06:33:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6efae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef415c520 con 0x7f6ef40b56f0 2024-04-24T06:33:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6ef3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6ef406f340 0x7f6ef40d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39356/0 (socket says 172.21.15.59:39356) 2024-04-24T06:33:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6ef3400640 1 -- 172.21.15.59:0/3178229920 learned_addr learned my addr 172.21.15.59:0/3178229920 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ef406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679783198 0 0) 0x7f6ef415bf60 con 0x7f6ef406f340 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.047+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ed8003520 con 0x7f6ef406f340 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800053558 0 0) 0x7f6ed8003520 con 0x7f6ef406f340 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 msgr2=0x7f6ef406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ef406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 >> v1:172.21.15.66:6789/0 conn(0x7f6ef40d0a60 legacy=0x7f6ef40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef415c800 con 0x7f6ef406f340 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6ee0001670 con 0x7f6ef406f340 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 >> v1:172.21.15.59:6789/0 conn(0x7f6ef406f340 legacy=0x7f6ef40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ed80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004060 0x7f6ed8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed8004d40 0x7f6ed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef415c520 con 0x7f6ed8004d40 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed800de10 con 0x7f6ed8004060 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed800e010 con 0x7f6ef40b56f0 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.048+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed8004d40 0x7f6ed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ed80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef3400640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004060 0x7f6ed8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 msgr2=0x7f6ed80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ed80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004060 msgr2=0x7f6ed8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004060 0x7f6ed8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3178229920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed8003520 con 0x7f6ed8004d40 2024-04-24T06:33:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef40b56f0 0x7f6ed80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:24.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.049+0000 7f6ef3400640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004060 0x7f6ed8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.050+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed8004d40 0x7f6ed80051c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6eec001e20 tx=0x7f6ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94e2bbff0fea05b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.050+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec001a70 con 0x7f6ed8004d40 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.050+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec02ae40 con 0x7f6ed8004d40 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.050+0000 7f6ef2a00640 1 -- 172.21.15.59:0/3178229920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec003bb0 con 0x7f6ed8004d40 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 -- 172.21.15.59:0/3178229920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed8004d40 msgr2=0x7f6ed80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 --2- 172.21.15.59:0/3178229920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed8004d40 0x7f6ed80051c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6eec001e20 tx=0x7f6ed8005830 comp rx=0 tx=0).stop 2024-04-24T06:33:24.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3178229920 reap_dead start 2024-04-24T06:33:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 -- 172.21.15.59:0/3178229920 shutdown_connections 2024-04-24T06:33:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 -- 172.21.15.59:0/3178229920 >> 172.21.15.59:0/3178229920 conn(0x7f6ef4067d90 msgr2=0x7f6ef406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 -- 172.21.15.59:0/3178229920 shutdown_connections 2024-04-24T06:33:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.051+0000 7f6efae00640 1 -- 172.21.15.59:0/3178229920 wait complete. 2024-04-24T06:33:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.053+0000 7f6efae00640 1 Processor -- start 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.053+0000 7f6efae00640 1 -- start start 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 0x7f6ef40c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef4154a30 0x7f6ef4147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef406ddd0 con 0x7f6ef4153490 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef415bc10 con 0x7f6ed8004d40 2024-04-24T06:33:24.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6efae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef406df50 con 0x7f6ef4154a30 2024-04-24T06:33:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6ef8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6ef8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45844/0 (socket says 172.21.15.59:45844) 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3102423919 learned_addr learned my addr 172.21.15.59:0/3102423919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.054+0000 7f6ef3400640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef4154a30 0x7f6ef4147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 0x7f6ef40c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef4154a30 msgr2=0x7f6ef4147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef4154a30 0x7f6ef4147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 msgr2=0x7f6ef40c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 0x7f6ef40c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 -- 172.21.15.59:0/3102423919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef40c58e0 con 0x7f6ef4153490 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 0x7f6ef40c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6ef8e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6eec001f40 tx=0x7f6eec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec001a70 con 0x7f6ef4153490 2024-04-24T06:33:24.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec02ac40 con 0x7f6ef4153490 2024-04-24T06:33:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007db0 con 0x7f6ef4153490 2024-04-24T06:33:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.055+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef4153990 con 0x7f6ef4153490 2024-04-24T06:33:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.056+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef4153f50 con 0x7f6ef4153490 2024-04-24T06:33:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.057+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef4068a80 con 0x7f6ef4153490 2024-04-24T06:33:24.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.058+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6eec043020 con 0x7f6ef4153490 2024-04-24T06:33:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.059+0000 7f6eebe00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.059+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6eec0aaeb0 con 0x7f6ef4153490 2024-04-24T06:33:24.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.060+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.060+0000 7f6ef3e00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 0x7f6ed406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006fd0 tx=0x7f6ee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.064+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6eec0793c0 con 0x7f6ef4153490 2024-04-24T06:33:24.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.266+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ef40b2a40 con 0x7f6ef4153490 2024-04-24T06:33:24.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.266+0000 7f6eebe00640 1 -- 172.21.15.59:0/3102423919 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f6eec078cc0 con 0x7f6ef4153490 2024-04-24T06:33:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:33:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 msgr2=0x7f6ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 0x7f6ed406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006fd0 tx=0x7f6ee4008040 comp rx=0 tx=0).stop 2024-04-24T06:33:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 msgr2=0x7f6ef4155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6eec001f40 tx=0x7f6eec0014a0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 shutdown_connections 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6ed406c8f0 0x7f6ed406edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4153490 0x7f6ef4155f40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ef4154a30 0x7f6ef4147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 --2- 172.21.15.59:0/3102423919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ed8004d40 0x7f6ef40c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 >> 172.21.15.59:0/3102423919 conn(0x7f6ef4067d90 msgr2=0x7f6ef40cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 shutdown_connections 2024-04-24T06:33:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.269+0000 7f6efae00640 1 -- 172.21.15.59:0/3102423919 wait complete. 2024-04-24T06:33:24.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:33:24.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.501+0000 7fb2dcc00640 1 Processor -- start 2024-04-24T06:33:24.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.502+0000 7fb2dcc00640 1 -- start start 2024-04-24T06:33:24.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.503+0000 7fb2dcc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb2d8145e10 0x7fb2d806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.503+0000 7fb2dcc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.503+0000 7fb2dcc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d80cfc20 con 0x7fb2d8071d70 2024-04-24T06:33:24.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.503+0000 7fb2dcc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d80cff00 con 0x7fb2d8145e10 2024-04-24T06:33:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.503+0000 7fb2dcc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d80d01e0 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.504+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.504+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:40586/0 (socket says 172.21.15.59:40586) 2024-04-24T06:33:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.504+0000 7fb2cd400640 1 -- 172.21.15.59:0/3628977331 learned_addr learned my addr 172.21.15.59:0/3628977331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.504+0000 7fb2d5e00640 1 --2- 172.21.15.59:0/3628977331 >> v2:172.21.15.66:3300/0 conn(0x7fb2d8145e10 0x7fb2d806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.505+0000 7fb2cd400640 1 -- 172.21.15.59:0/3628977331 >> v2:172.21.15.66:3300/0 conn(0x7fb2d8145e10 msgr2=0x7fb2d806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.505+0000 7fb2cd400640 1 --2- 172.21.15.59:0/3628977331 >> v2:172.21.15.66:3300/0 conn(0x7fb2d8145e10 0x7fb2d806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.505+0000 7fb2cd400640 1 -- 172.21.15.59:0/3628977331 >> v1:172.21.15.66:6789/0 conn(0x7fb2d8071d70 legacy=0x7fb2d815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.505+0000 7fb2cd400640 1 -- 172.21.15.59:0/3628977331 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d80d04c0 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.506+0000 7fb2cd400640 1 --2- 172.21.15.59:0/3628977331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00098a0 tx=0x7fb2c002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc051ba416a24b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.507+0000 7fb2d5400640 1 -- 172.21.15.59:0/3628977331 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0037070 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.507+0000 7fb2d5400640 1 -- 172.21.15.59:0/3628977331 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c0032440 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.507+0000 7fb2d5400640 1 -- 172.21.15.59:0/3628977331 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0032810 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 -- 172.21.15.59:0/3628977331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 msgr2=0x7fb2d80cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/3628977331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00098a0 tx=0x7fb2c002a290 comp rx=0 tx=0).stop 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 -- 172.21.15.59:0/3628977331 shutdown_connections 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/3628977331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d80cf340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/3628977331 >> v2:172.21.15.66:3300/0 conn(0x7fb2d8145e10 0x7fb2d806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 -- 172.21.15.59:0/3628977331 >> 172.21.15.59:0/3628977331 conn(0x7fb2d8067d90 msgr2=0x7fb2d806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 -- 172.21.15.59:0/3628977331 shutdown_connections 2024-04-24T06:33:24.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.508+0000 7fb2dcc00640 1 -- 172.21.15.59:0/3628977331 wait complete. 2024-04-24T06:33:24.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.510+0000 7fb2dcc00640 1 Processor -- start 2024-04-24T06:33:24.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.510+0000 7fb2dcc00640 1 -- start start 2024-04-24T06:33:24.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d8071d70 0x7fb2d80b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d81e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80cfa00 con 0x7fb2d8145e10 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80cf880 con 0x7fb2d8071d70 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.511+0000 7fb2dcc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d81461f0 con 0x7fb2d80d0cc0 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45860/0 (socket says 172.21.15.59:45860) 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 -- 172.21.15.59:0/294609545 learned_addr learned my addr 172.21.15.59:0/294609545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2cd400640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d81e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d5e00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d8071d70 0x7fb2d80b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 -- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 msgr2=0x7fb2d81e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d81e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 -- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d8071d70 msgr2=0x7fb2d80b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d8071d70 0x7fb2d80b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 -- 172.21.15.59:0/294609545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d80b1e10 con 0x7fb2d8145e10 2024-04-24T06:33:24.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.512+0000 7fb2d6800640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c7d0 tx=0x7fb2d000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.513+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003b70 con 0x7fb2d8145e10 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.513+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0003d30 con 0x7fb2d8145e10 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.513+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000aa30 con 0x7fb2d8145e10 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.513+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d80b0d70 con 0x7fb2d8145e10 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.513+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d80b1330 con 0x7fb2d8145e10 2024-04-24T06:33:24.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.514+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d8068a80 con 0x7fb2d8145e10 2024-04-24T06:33:24.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.515+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2d000abd0 con 0x7fb2d8145e10 2024-04-24T06:33:24.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.516+0000 7fb2cea00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.516+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb2d008ba40 con 0x7fb2d8145e10 2024-04-24T06:33:24.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.516+0000 7fb2d5e00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.517+0000 7fb2d5e00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 0x7fb2b406edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009a30 tx=0x7fb2c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.520+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2d80b1e10 con 0x7fb2d8145e10 2024-04-24T06:33:24.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.724+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2d80d1190 con 0x7fb2d8145e10 2024-04-24T06:33:24.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.725+0000 7fb2cea00640 1 -- 172.21.15.59:0/294609545 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7fb2d0059e60 con 0x7fb2d8145e10 2024-04-24T06:33:24.727 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:33:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 msgr2=0x7fb2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 0x7fb2b406edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009a30 tx=0x7fb2c8008040 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 msgr2=0x7fb2d81e8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c7d0 tx=0x7fb2d000cc90 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 shutdown_connections 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d8145e10 0x7fb2d81e8f10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb2d80d0cc0 0x7fb2d81e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 --2- 172.21.15.59:0/294609545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2d8071d70 0x7fb2d80b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 >> 172.21.15.59:0/294609545 conn(0x7fb2d8067d90 msgr2=0x7fb2d815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 shutdown_connections 2024-04-24T06:33:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.728+0000 7fb2dcc00640 1 -- 172.21.15.59:0/294609545 wait complete. 2024-04-24T06:33:24.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:33:24.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.955+0000 7feab3000640 1 Processor -- start 2024-04-24T06:33:24.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.955+0000 7feab3000640 1 -- start start 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feab3000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feaac141290 0x7feaac0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feab3000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15e900 0x7feaac15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feab3000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac15f710 con 0x7feaac15f2a0 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feab3000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7feaac0b51f0 con 0x7feaac141290 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feab3000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7feaac140a00 con 0x7feaac15e900 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feaabe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feaac141290 0x7feaac0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feaabe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feaac141290 0x7feaac0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45872/0 (socket says 172.21.15.59:45872) 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.957+0000 7feaabe00640 1 -- 172.21.15.59:0/748760329 learned_addr learned my addr 172.21.15.59:0/748760329 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.958+0000 7feaa3400640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15e900 0x7feaac15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.958+0000 7feaabe00640 1 -- 172.21.15.59:0/748760329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15e900 msgr2=0x7feaac15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.958+0000 7feaabe00640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15e900 0x7feaac15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.958+0000 7feaabe00640 1 -- 172.21.15.59:0/748760329 >> v1:172.21.15.66:6789/0 conn(0x7feaac15f2a0 legacy=0x7feaac071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:24.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.958+0000 7feaabe00640 1 -- 172.21.15.59:0/748760329 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0b4670 con 0x7feaac141290 2024-04-24T06:33:24.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.959+0000 7feaabe00640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac141290 0x7feaac0b3e40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009870 tx=0x7fea9c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea1cb9305f475ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.959+0000 7feaab400640 1 -- 172.21.15.59:0/748760329 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c037070 con 0x7feaac141290 2024-04-24T06:33:24.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.959+0000 7feaab400640 1 -- 172.21.15.59:0/748760329 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9c032440 con 0x7feaac141290 2024-04-24T06:33:24.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.959+0000 7feaab400640 1 -- 172.21.15.59:0/748760329 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c032810 con 0x7feaac141290 2024-04-24T06:33:24.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 -- 172.21.15.59:0/748760329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac141290 msgr2=0x7feaac0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac141290 0x7feaac0b3e40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009870 tx=0x7fea9c02a210 comp rx=0 tx=0).stop 2024-04-24T06:33:24.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 -- 172.21.15.59:0/748760329 shutdown_connections 2024-04-24T06:33:24.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15e900 0x7feaac15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 --2- 172.21.15.59:0/748760329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac141290 0x7feaac0b3e40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 -- 172.21.15.59:0/748760329 >> 172.21.15.59:0/748760329 conn(0x7feaac0b2130 msgr2=0x7feaac0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 -- 172.21.15.59:0/748760329 shutdown_connections 2024-04-24T06:33:24.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.960+0000 7feab3000640 1 -- 172.21.15.59:0/748760329 wait complete. 2024-04-24T06:33:24.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.962+0000 7feab3000640 1 Processor -- start 2024-04-24T06:33:24.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.962+0000 7feab3000640 1 -- start start 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaac141290 0x7feaac0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 0x7feaac1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0b4f10 con 0x7feaac15e900 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0b4c30 con 0x7feaac141290 2024-04-24T06:33:24.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feab3000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7feaac0b4380 con 0x7feaac15f2a0 2024-04-24T06:33:24.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feaa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feaa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45886/0 (socket says 172.21.15.59:45886) 2024-04-24T06:33:24.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.963+0000 7feaa3400640 1 -- 172.21.15.59:0/84144538 learned_addr learned my addr 172.21.15.59:0/84144538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feab1000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 0x7feaac1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaabe00640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaac141290 0x7feaac0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 -- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 msgr2=0x7feaac1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 0x7feaac1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 -- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaac141290 msgr2=0x7feaac0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaac141290 0x7feaac0b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 -- 172.21.15.59:0/84144538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0b03a0 con 0x7feaac15e900 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feab1000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 0x7feaac1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:24.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa3400640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea9400d830 tx=0x7fea9400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.964+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400b6c0 con 0x7feaac15e900 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.965+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea94019070 con 0x7feaac15e900 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.965+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400ba80 con 0x7feaac15e900 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.965+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac0b0680 con 0x7feaac15e900 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.965+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac0b0c40 con 0x7feaac15e900 2024-04-24T06:33:24.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.966+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac068a80 con 0x7feaac15e900 2024-04-24T06:33:24.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.967+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea9401e460 con 0x7feaac15e900 2024-04-24T06:33:24.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.968+0000 7feaa8c00640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.968+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fea9408c230 con 0x7feaac15e900 2024-04-24T06:33:24.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.968+0000 7feaabe00640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.969+0000 7feaabe00640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 0x7fea8806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fea9c034000 tx=0x7fea9c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:24.973+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea9405a660 con 0x7feaac15e900 2024-04-24T06:33:25.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.191+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feaac0692b0 con 0x7feaac15e900 2024-04-24T06:33:25.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.192+0000 7feaa8c00640 1 -- 172.21.15.59:0/84144538 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fea94059f60 con 0x7feaac15e900 2024-04-24T06:33:25.194 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:33:25.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 msgr2=0x7fea8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:25.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 0x7fea8806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fea9c034000 tx=0x7fea9c02e000 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 msgr2=0x7feaac1540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea9400d830 tx=0x7fea9400dcf0 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 shutdown_connections 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feaac15f2a0 0x7feaac1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaac15e900 0x7feaac1540b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 --2- 172.21.15.59:0/84144538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaac141290 0x7feaac0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.195+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 >> 172.21.15.59:0/84144538 conn(0x7feaac0b2130 msgr2=0x7feaac15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.196+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 shutdown_connections 2024-04-24T06:33:25.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:25.196+0000 7feab3000640 1 -- 172.21.15.59:0/84144538 wait complete. 2024-04-24T06:33:25.217 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:33:25.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:25.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:25.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[53951]: fuse finished with error 0 and tester_r 0 2024-04-24T06:33:25.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:33:25.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:25.312 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:33:25.457 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:33:25.457 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:33:25.457 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:25.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:33:25.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:33:25.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:25.505 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:33:25.600 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:33:25.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:33:25.601 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:33:25.602 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:25.602 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:25.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:33:25.660 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:33:25.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:25.660 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:33:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:33:25.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:33:25.715 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:33:25.716 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:33:25.716 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:25.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:33:25.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:33:25.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:25.777 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:33:25.835 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:33:25.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:33:25.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:26.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.059+0000 7fac63e00640 1 Processor -- start 2024-04-24T06:33:26.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.059+0000 7fac63e00640 1 -- start start 2024-04-24T06:33:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.060+0000 7fac63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 0x7fac5c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.060+0000 7fac63e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fac5c06b9d0 0x7fac5c06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.060+0000 7fac63e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5c0cdf80 con 0x7fac5c0cd340 2024-04-24T06:33:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.060+0000 7fac63e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fac5c06c360 con 0x7fac5c06b9d0 2024-04-24T06:33:26.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.060+0000 7fac63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fac5c0b2ad0 con 0x7fac5c0b2510 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac61400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fac5c0cd340 0x7fac5c15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39368/0 (socket says 172.21.15.59:39368) 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac61400640 1 -- 172.21.15.59:0/1548568549 learned_addr learned my addr 172.21.15.59:0/1548568549 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac60a00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 0x7fac5c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac61e00640 1 --2- 172.21.15.59:0/1548568549 >> v2:172.21.15.66:3300/0 conn(0x7fac5c06b9d0 0x7fac5c06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832305336 0 0) 0x7fac5c0cdf80 con 0x7fac5c0cd340 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.061+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac44003520 con 0x7fac5c0cd340 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1870698791 0 0) 0x7fac44003520 con 0x7fac5c0cd340 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 msgr2=0x7fac5c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 0x7fac5c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 >> v2:172.21.15.66:3300/0 conn(0x7fac5c06b9d0 msgr2=0x7fac5c06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 --2- 172.21.15.59:0/1548568549 >> v2:172.21.15.66:3300/0 conn(0x7fac5c06b9d0 0x7fac5c06be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c0ce220 con 0x7fac5c0cd340 2024-04-24T06:33:26.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fac50001a10 con 0x7fac5c0cd340 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac61e00640 1 --2- 172.21.15.59:0/1548568549 >> v2:172.21.15.66:3300/0 conn(0x7fac5c06b9d0 0x7fac5c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 >> v1:172.21.15.59:6789/0 conn(0x7fac5c0cd340 legacy=0x7fac5c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac440039b0 0x7fac44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac44004800 0x7fac44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 0x7fac440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c06c360 con 0x7fac440039b0 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c0b2ad0 con 0x7fac44004800 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fac4400ba10 con 0x7fac5c0b2510 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.062+0000 7fac61400640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac440039b0 0x7fac44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.063+0000 7fac61400640 1 -- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 msgr2=0x7fac440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:26.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.063+0000 7fac61400640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac5c0b2510 0x7fac440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.063+0000 7fac61400640 1 -- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac44004800 msgr2=0x7fac44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.063+0000 7fac61400640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac44004800 0x7fac44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.063+0000 7fac61400640 1 -- 172.21.15.59:0/1548568549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac44003520 con 0x7fac440039b0 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac61400640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac440039b0 0x7fac44003dd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0ce150 tx=0x7fac44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e275a261281c4d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5000a410 con 0x7fac440039b0 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5002f070 con 0x7fac440039b0 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac5be00640 1 -- 172.21.15.59:0/1548568549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5002bbd0 con 0x7fac440039b0 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac63e00640 1 -- 172.21.15.59:0/1548568549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac440039b0 msgr2=0x7fac44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac63e00640 1 --2- 172.21.15.59:0/1548568549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac440039b0 0x7fac44003dd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0ce150 tx=0x7fac44005830 comp rx=0 tx=0).stop 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.064+0000 7fac61e00640 1 -- 172.21.15.59:0/1548568549 reap_dead start 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.065+0000 7fac63e00640 1 -- 172.21.15.59:0/1548568549 shutdown_connections 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.065+0000 7fac63e00640 1 -- 172.21.15.59:0/1548568549 >> 172.21.15.59:0/1548568549 conn(0x7fac5c073450 msgr2=0x7fac5c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.065+0000 7fac63e00640 1 -- 172.21.15.59:0/1548568549 shutdown_connections 2024-04-24T06:33:26.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.065+0000 7fac63e00640 1 -- 172.21.15.59:0/1548568549 wait complete. 2024-04-24T06:33:26.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.066+0000 7fac63e00640 1 Processor -- start 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 -- start start 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac440039b0 0x7fac5c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac5c0ca940 0x7fac5c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c0cd720 con 0x7fac5c0c9000 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c0b2db0 con 0x7fac5c0ca940 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.067+0000 7fac63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fac5c0cd8a0 con 0x7fac440039b0 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45900/0 (socket says 172.21.15.59:45900) 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 -- 172.21.15.59:0/4025690390 learned_addr learned my addr 172.21.15.59:0/4025690390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac61400640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac440039b0 0x7fac5c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac61e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac5c0ca940 0x7fac5c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 -- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac440039b0 msgr2=0x7fac5c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac440039b0 0x7fac5c0b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 -- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac5c0ca940 msgr2=0x7fac5c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac5c0ca940 0x7fac5c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.068+0000 7fac60a00640 1 -- 172.21.15.59:0/4025690390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c0c9570 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac60a00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e7d0 tx=0x7fac4800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48017070 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4800c940 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4800cd10 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c0c9850 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.069+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c0c9e10 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.070+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5c069190 con 0x7fac5c0c9000 2024-04-24T06:33:26.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.071+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac4801a070 con 0x7fac5c0c9000 2024-04-24T06:33:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.072+0000 7fac59600640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 0x7fac3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.072+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fac4808beb0 con 0x7fac5c0c9000 2024-04-24T06:33:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.073+0000 7fac61400640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 0x7fac3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.073+0000 7fac61400640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 0x7fac3c06ede0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac50033000 tx=0x7fac50007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.077+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac4805a1b0 con 0x7fac5c0c9000 2024-04-24T06:33:26.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.323+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac5c06c8a0 con 0x7fac5c0c9000 2024-04-24T06:33:26.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.324+0000 7fac59600640 1 -- 172.21.15.59:0/4025690390 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7fac48059ab0 con 0x7fac5c0c9000 2024-04-24T06:33:26.326 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:26.326 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:56.100001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:33:26.326 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-24T06:33:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.326+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 msgr2=0x7fac3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.326+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 0x7fac3c06ede0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac50033000 tx=0x7fac50007000 comp rx=0 tx=0).stop 2024-04-24T06:33:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.326+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 msgr2=0x7fac5c1527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.326+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac4800e7d0 tx=0x7fac4800ec90 comp rx=0 tx=0).stop 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 shutdown_connections 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fac3c06c920 0x7fac3c06ede0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac5c0ca940 0x7fac5c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac5c0c9000 0x7fac5c1527a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 --2- 172.21.15.59:0/4025690390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fac440039b0 0x7fac5c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 >> 172.21.15.59:0/4025690390 conn(0x7fac5c073450 msgr2=0x7fac5c0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 shutdown_connections 2024-04-24T06:33:26.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.327+0000 7fac63e00640 1 -- 172.21.15.59:0/4025690390 wait complete. 2024-04-24T06:33:26.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:26.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.556+0000 7f8f80c00640 1 Processor -- start 2024-04-24T06:33:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.557+0000 7f8f80c00640 1 -- start start 2024-04-24T06:33:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.558+0000 7f8f80c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 0x7f8f7c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.558+0000 7f8f80c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f7c0af920 0x7f8f7c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.558+0000 7f8f80c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c15f0f0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.558+0000 7f8f80c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c0afd80 con 0x7f8f7c0af920 2024-04-24T06:33:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.558+0000 7f8f80c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c15fa20 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.559+0000 7f8f7a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f7c0d06e0 0x7f8f7c071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39376/0 (socket says 172.21.15.59:39376) 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.559+0000 7f8f7a800640 1 -- 172.21.15.59:0/2513933854 learned_addr learned my addr 172.21.15.59:0/2513933854 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.559+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2513933854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 0x7f8f7c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.559+0000 7f8f79400640 1 --2- 172.21.15.59:0/2513933854 >> v2:172.21.15.66:3300/0 conn(0x7f8f7c0af920 0x7f8f7c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.559+0000 7f8f78a00640 1 -- 172.21.15.59:0/2513933854 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219567943 0 0) 0x7f8f7c15f0f0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.560+0000 7f8f78a00640 1 -- 172.21.15.59:0/2513933854 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f60003520 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.560+0000 7f8f79e00640 1 -- 172.21.15.59:0/2513933854 >> v2:172.21.15.66:3300/0 conn(0x7f8f7c0af920 msgr2=0x7f8f7c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.560+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2513933854 >> v2:172.21.15.66:3300/0 conn(0x7f8f7c0af920 0x7f8f7c0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.560+0000 7f8f79e00640 1 -- 172.21.15.59:0/2513933854 >> v1:172.21.15.59:6789/0 conn(0x7f8f7c0d06e0 legacy=0x7f8f7c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.560+0000 7f8f79e00640 1 -- 172.21.15.59:0/2513933854 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c15f3d0 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.561+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2513933854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 0x7f8f7c0af3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0097a0 tx=0x7f8f6c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd89beb5c66c18d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.561+0000 7f8f78a00640 1 -- 172.21.15.59:0/2513933854 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c037070 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f78a00640 1 -- 172.21.15.59:0/2513933854 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c032440 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f78a00640 1 -- 172.21.15.59:0/2513933854 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c032810 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 -- 172.21.15.59:0/2513933854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 msgr2=0x7f8f7c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2513933854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 0x7f8f7c0af3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0097a0 tx=0x7f8f6c02a180 comp rx=0 tx=0).stop 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 -- 172.21.15.59:0/2513933854 shutdown_connections 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2513933854 >> v2:172.21.15.66:3300/0 conn(0x7f8f7c0af920 0x7f8f7c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2513933854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c15c5e0 0x7f8f7c0af3e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 -- 172.21.15.59:0/2513933854 >> 172.21.15.59:0/2513933854 conn(0x7f8f7c067ea0 msgr2=0x7f8f7c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 -- 172.21.15.59:0/2513933854 shutdown_connections 2024-04-24T06:33:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.562+0000 7f8f80c00640 1 -- 172.21.15.59:0/2513933854 wait complete. 2024-04-24T06:33:26.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.564+0000 7f8f80c00640 1 Processor -- start 2024-04-24T06:33:26.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.565+0000 7f8f80c00640 1 -- start start 2024-04-24T06:33:26.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c0af920 0x7f8f7c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 0x7f8f7c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d0cf0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d0b70 con 0x7f8f7c15c5e0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f80c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c06e480 con 0x7f8f7c0af920 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f79400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f79400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45902/0 (socket says 172.21.15.59:45902) 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f79400640 1 -- 172.21.15.59:0/2566328201 learned_addr learned my addr 172.21.15.59:0/2566328201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c0af920 0x7f8f7c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.566+0000 7f8f7a800640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 0x7f8f7c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 -- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c0af920 msgr2=0x7f8f7c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c0af920 0x7f8f7c159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 -- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 msgr2=0x7f8f7c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 0x7f8f7c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 -- 172.21.15.59:0/2566328201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1585c0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f7a800640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 0x7f8f7c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f79400640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800efc0 tx=0x7f8f6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68018020 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68014480 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68014850 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1588a0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.567+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c158e60 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.568+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c068a80 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.570+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f68005d10 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.571+0000 7f8f72000640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.571+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8f6808ca10 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.571+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.572+0000 7f8f79e00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c034000 tx=0x7f8f6c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.575+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f68022030 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.781+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f7c15f1b0 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.784+0000 7f8f72000640 1 -- 172.21.15.59:0/2566328201 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16581 (secure 0 0 0) 0x7f8f68010040 con 0x7f8f7c0d06e0 2024-04-24T06:33:26.787 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:26.787 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:09.051581+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 msgr2=0x7f8f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c034000 tx=0x7f8f6c02e000 comp rx=0 tx=0).stop 2024-04-24T06:33:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 msgr2=0x7f8f7c1558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800efc0 tx=0x7f8f6800c490 comp rx=0 tx=0).stop 2024-04-24T06:33:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 shutdown_connections 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f7c15c5e0 0x7f8f7c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f7c0d06e0 0x7f8f7c1558f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 --2- 172.21.15.59:0/2566328201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f7c0af920 0x7f8f7c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 >> 172.21.15.59:0/2566328201 conn(0x7f8f7c067ea0 msgr2=0x7f8f7c157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 shutdown_connections 2024-04-24T06:33:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:26.787+0000 7f8f80c00640 1 -- 172.21.15.59:0/2566328201 wait complete. 2024-04-24T06:33:26.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:32:05.373199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:32:06.536666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:26.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:27.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.024+0000 7f9e46a00640 1 Processor -- start 2024-04-24T06:33:27.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.024+0000 7f9e46a00640 1 -- start start 2024-04-24T06:33:27.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.025+0000 7f9e46a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e400d0600 0x7f9e400ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.025+0000 7f9e46a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e4006dcb0 0x7f9e40162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.025+0000 7f9e46a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e40163240 con 0x7f9e400d09e0 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.025+0000 7f9e46a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9e40072a20 con 0x7f9e4006dcb0 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.025+0000 7f9e46a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e40071e10 con 0x7f9e400d0600 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e4006dcb0 0x7f9e40162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e4006dcb0 0x7f9e40162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45904/0 (socket says 172.21.15.59:45904) 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 -- 172.21.15.59:0/3296987667 learned_addr learned my addr 172.21.15.59:0/3296987667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e45600640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e400d0600 0x7f9e400ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 -- 172.21.15.59:0/3296987667 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e400d0600 msgr2=0x7f9e400ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e400d0600 0x7f9e400ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 -- 172.21.15.59:0/3296987667 >> v1:172.21.15.66:6789/0 conn(0x7f9e400d09e0 legacy=0x7f9e4006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.026+0000 7f9e44c00640 1 -- 172.21.15.59:0/3296987667 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e400720f0 con 0x7f9e4006dcb0 2024-04-24T06:33:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.027+0000 7f9e44c00640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006dcb0 0x7f9e40162d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f9b24d346c93be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.028+0000 7f9e3fe00640 1 -- 172.21.15.59:0/3296987667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c037070 con 0x7f9e4006dcb0 2024-04-24T06:33:27.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.028+0000 7f9e3fe00640 1 -- 172.21.15.59:0/3296987667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c032440 con 0x7f9e4006dcb0 2024-04-24T06:33:27.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.028+0000 7f9e3fe00640 1 -- 172.21.15.59:0/3296987667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c032810 con 0x7f9e4006dcb0 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.028+0000 7f9e46a00640 1 -- 172.21.15.59:0/3296987667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006dcb0 msgr2=0x7f9e40162d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.028+0000 7f9e46a00640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006dcb0 0x7f9e40162d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02a210 comp rx=0 tx=0).stop 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 -- 172.21.15.59:0/3296987667 shutdown_connections 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006dcb0 0x7f9e40162d00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 --2- 172.21.15.59:0/3296987667 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e400d0600 0x7f9e400ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 -- 172.21.15.59:0/3296987667 >> 172.21.15.59:0/3296987667 conn(0x7f9e4006bdf0 msgr2=0x7f9e4006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 -- 172.21.15.59:0/3296987667 shutdown_connections 2024-04-24T06:33:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.029+0000 7f9e46a00640 1 -- 172.21.15.59:0/3296987667 wait complete. 2024-04-24T06:33:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.030+0000 7f9e46a00640 1 Processor -- start 2024-04-24T06:33:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.031+0000 7f9e46a00640 1 -- start start 2024-04-24T06:33:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.031+0000 7f9e46a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e4006dcb0 0x7f9e40161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 0x7f9e401f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4015eaf0 con 0x7f9e400d09e0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e400726b0 con 0x7f9e400d0600 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e400d0e70 con 0x7f9e4006dcb0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45920/0 (socket says 172.21.15.59:45920) 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 -- 172.21.15.59:0/134230811 learned_addr learned my addr 172.21.15.59:0/134230811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e45600640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e4006dcb0 0x7f9e40161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e44c00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 0x7f9e401f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 -- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e4006dcb0 msgr2=0x7f9e40161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e4006dcb0 0x7f9e40161750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 -- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 msgr2=0x7f9e401f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 0x7f9e401f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e46000640 1 -- 172.21.15.59:0/134230811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e40150d50 con 0x7f9e400d09e0 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.032+0000 7f9e44c00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 0x7f9e401f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e46000640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b9a0 tx=0x7f9e3800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38019070 con 0x7f9e400d09e0 2024-04-24T06:33:27.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3800cba0 con 0x7f9e400d09e0 2024-04-24T06:33:27.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38014440 con 0x7f9e400d09e0 2024-04-24T06:33:27.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e401f82c0 con 0x7f9e400d09e0 2024-04-24T06:33:27.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.033+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e401f8880 con 0x7f9e400d09e0 2024-04-24T06:33:27.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.034+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e401f8ac0 con 0x7f9e400d09e0 2024-04-24T06:33:27.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.035+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e3801c070 con 0x7f9e400d09e0 2024-04-24T06:33:27.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.036+0000 7f9e3d600640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.036+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9e3808b7b0 con 0x7f9e400d09e0 2024-04-24T06:33:27.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.037+0000 7f9e45600640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.037+0000 7f9e45600640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 0x7f9e2006edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.041+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e401f8ac0 con 0x7f9e400d09e0 2024-04-24T06:33:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.248+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e4004fd90 con 0x7f9e400d09e0 2024-04-24T06:33:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.249+0000 7f9e3d600640 1 -- 172.21.15.59:0/134230811 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f9e38059b40 con 0x7f9e400d09e0 2024-04-24T06:33:27.251 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 msgr2=0x7f9e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 0x7f9e2006edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 msgr2=0x7f9e40150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b9a0 tx=0x7f9e3800be60 comp rx=0 tx=0).stop 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 shutdown_connections 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400d09e0 0x7f9e40150810 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e400d0600 0x7f9e401f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 --2- 172.21.15.59:0/134230811 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e4006dcb0 0x7f9e40161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 >> 172.21.15.59:0/134230811 conn(0x7f9e4006bdf0 msgr2=0x7f9e400ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 shutdown_connections 2024-04-24T06:33:27.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.252+0000 7f9e46a00640 1 -- 172.21.15.59:0/134230811 wait complete. 2024-04-24T06:33:27.273 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:33:27.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.487+0000 7f0285a00640 1 Processor -- start 2024-04-24T06:33:27.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.487+0000 7f0285a00640 1 -- start start 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.488+0000 7f0285a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0280161c50 0x7f02800d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.488+0000 7f0285a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f02800cfc50 0x7f02800d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.488+0000 7f0285a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02800d22e0 con 0x7f02800d0de0 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.488+0000 7f0285a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02800d2ba0 con 0x7f0280161c50 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.488+0000 7f0285a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f02800d2550 con 0x7f02800cfc50 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0280161c50 0x7f02800d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0280161c50 0x7f02800d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45936/0 (socket says 172.21.15.59:45936) 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 -- 172.21.15.59:0/1280824554 learned_addr learned my addr 172.21.15.59:0/1280824554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:27.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f0285000640 1 --1- 172.21.15.59:0/1280824554 >> v1:172.21.15.59:6789/0 conn(0x7f02800d0de0 0x7f028006cb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39382/0 (socket says 172.21.15.59:39382) 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027f400640 1 --2- 172.21.15.59:0/1280824554 >> v2:172.21.15.66:3300/0 conn(0x7f02800cfc50 0x7f02800d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 -- 172.21.15.59:0/1280824554 >> v2:172.21.15.66:3300/0 conn(0x7f02800cfc50 msgr2=0x7f02800d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 --2- 172.21.15.59:0/1280824554 >> v2:172.21.15.66:3300/0 conn(0x7f02800cfc50 0x7f02800d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 -- 172.21.15.59:0/1280824554 >> v1:172.21.15.59:6789/0 conn(0x7f02800d0de0 legacy=0x7f028006cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027ea00640 1 -- 172.21.15.59:0/1280824554 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447754664 0 0) 0x7f02800d22e0 con 0x7f02800d0de0 2024-04-24T06:33:27.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.489+0000 7f027fe00640 1 -- 172.21.15.59:0/1280824554 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800d2830 con 0x7f0280161c50 2024-04-24T06:33:27.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f027fe00640 1 --2- 172.21.15.59:0/1280824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0280161c50 0x7f02800d5240 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f02800d2d20 tx=0x7f027002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e6d5d6b6273a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f027ea00640 1 -- 172.21.15.59:0/1280824554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270036070 con 0x7f0280161c50 2024-04-24T06:33:27.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f027ea00640 1 -- 172.21.15.59:0/1280824554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0270031600 con 0x7f0280161c50 2024-04-24T06:33:27.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f027ea00640 1 -- 172.21.15.59:0/1280824554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700319d0 con 0x7f0280161c50 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f0285a00640 1 -- 172.21.15.59:0/1280824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0280161c50 msgr2=0x7f02800d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.491+0000 7f0285a00640 1 --2- 172.21.15.59:0/1280824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0280161c50 0x7f02800d5240 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f02800d2d20 tx=0x7f027002a730 comp rx=0 tx=0).stop 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 -- 172.21.15.59:0/1280824554 shutdown_connections 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 --2- 172.21.15.59:0/1280824554 >> v2:172.21.15.66:3300/0 conn(0x7f02800cfc50 0x7f02800d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 --2- 172.21.15.59:0/1280824554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0280161c50 0x7f02800d5240 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 -- 172.21.15.59:0/1280824554 >> 172.21.15.59:0/1280824554 conn(0x7f028008bf90 msgr2=0x7f02800d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 -- 172.21.15.59:0/1280824554 shutdown_connections 2024-04-24T06:33:27.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.492+0000 7f0285a00640 1 -- 172.21.15.59:0/1280824554 wait complete. 2024-04-24T06:33:27.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.493+0000 7f0285a00640 1 Processor -- start 2024-04-24T06:33:27.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 -- start start 2024-04-24T06:33:27.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 0x7f0280156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0280161c50 0x7f02801f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02800d3570 con 0x7f02800d0de0 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02800d0460 con 0x7f02800cfc50 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.494+0000 7f0285a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f02800d0090 con 0x7f0280161c50 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45950/0 (socket says 172.21.15.59:45950) 2024-04-24T06:33:27.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 -- 172.21.15.59:0/573051295 learned_addr learned my addr 172.21.15.59:0/573051295 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:27.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027fe00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 0x7f0280156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 -- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0280161c50 msgr2=0x7f02801f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0280161c50 0x7f02801f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 -- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 msgr2=0x7f0280156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 0x7f0280156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027f400640 1 -- 172.21.15.59:0/573051295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800c8f20 con 0x7f02800d0de0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.495+0000 7f027fe00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 0x7f0280156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f027f400640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d830 tx=0x7f026c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c00b6c0 con 0x7f02800d0de0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026c019070 con 0x7f02800d0de0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c00ba80 con 0x7f02800d0de0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02800c9200 con 0x7f02800d0de0 2024-04-24T06:33:27.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.496+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02800c97c0 con 0x7f02800d0de0 2024-04-24T06:33:27.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.497+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028004fd90 con 0x7f02800d0de0 2024-04-24T06:33:27.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.500+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f026c01e460 con 0x7f02800d0de0 2024-04-24T06:33:27.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.501+0000 7f0277e00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 0x7f026006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.501+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f026c08c1b0 con 0x7f02800d0de0 2024-04-24T06:33:27.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.501+0000 7f027fe00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 0x7f026006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.502+0000 7f027fe00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 0x7f026006ee00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0270033000 tx=0x7f0270009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.506+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f026c05a650 con 0x7f02800d0de0 2024-04-24T06:33:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.712+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02800d1cf0 con 0x7f02800d0de0 2024-04-24T06:33:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.713+0000 7f0277e00640 1 -- 172.21.15.59:0/573051295 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f026c059f50 con 0x7f02800d0de0 2024-04-24T06:33:27.716 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:33:27.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 msgr2=0x7f026006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 0x7f026006ee00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0270033000 tx=0x7f0270009000 comp rx=0 tx=0).stop 2024-04-24T06:33:27.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 msgr2=0x7f02801f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d830 tx=0x7f026c00dcf0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 shutdown_connections 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026006c940 0x7f026006ee00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0280161c50 0x7f02801f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02800d0de0 0x7f02801f7a20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 --2- 172.21.15.59:0/573051295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02800cfc50 0x7f0280156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.716+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 >> 172.21.15.59:0/573051295 conn(0x7f028008bf90 msgr2=0x7f028006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.717+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 shutdown_connections 2024-04-24T06:33:27.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.717+0000 7f0285a00640 1 -- 172.21.15.59:0/573051295 wait complete. 2024-04-24T06:33:27.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:33:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.954+0000 7fcdc7800640 1 Processor -- start 2024-04-24T06:33:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.954+0000 7fcdc7800640 1 -- start start 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc7800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcdc00b56f0 0x7fcdc006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc7800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc015bf60 con 0x7fcdc00d0a60 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc7800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc015c240 con 0x7fcdc006f340 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc7800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc015c520 con 0x7fcdc00b56f0 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcdc00b56f0 0x7fcdc006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc5800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcdc00d0a60 0x7fcdc00b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39398/0 (socket says 172.21.15.59:39398) 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc5800640 1 -- 172.21.15.59:0/2236985790 learned_addr learned my addr 172.21.15.59:0/2236985790 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc4e00640 1 --2- 172.21.15.59:0/2236985790 >> v2:172.21.15.59:3300/0 conn(0x7fcdc00b56f0 0x7fcdc006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45952/0 (socket says 172.21.15.59:45952) 2024-04-24T06:33:27.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.956+0000 7fcdc4e00640 1 -- 172.21.15.59:0/2236985790 >> v1:172.21.15.66:6789/0 conn(0x7fcdc006f340 legacy=0x7fcdc00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:27.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.957+0000 7fcdc4e00640 1 -- 172.21.15.59:0/2236985790 >> v1:172.21.15.59:6789/0 conn(0x7fcdc00d0a60 legacy=0x7fcdc00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.957+0000 7fcdc4e00640 1 -- 172.21.15.59:0/2236985790 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc015c800 con 0x7fcdc00b56f0 2024-04-24T06:33:27.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.958+0000 7fcdc4e00640 1 --2- 172.21.15.59:0/2236985790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc006ddd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40097a0 tx=0x7fcdb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e6b352a0a82bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.958+0000 7fcdbf400640 1 -- 172.21.15.59:0/2236985790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4037070 con 0x7fcdc00b56f0 2024-04-24T06:33:27.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.958+0000 7fcdbf400640 1 -- 172.21.15.59:0/2236985790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb4032440 con 0x7fcdc00b56f0 2024-04-24T06:33:27.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.958+0000 7fcdbf400640 1 -- 172.21.15.59:0/2236985790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4032810 con 0x7fcdc00b56f0 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 -- 172.21.15.59:0/2236985790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 msgr2=0x7fcdc006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2236985790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc006ddd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40097a0 tx=0x7fcdb402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 -- 172.21.15.59:0/2236985790 shutdown_connections 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2236985790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc006ddd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 -- 172.21.15.59:0/2236985790 >> 172.21.15.59:0/2236985790 conn(0x7fcdc0067d90 msgr2=0x7fcdc006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 -- 172.21.15.59:0/2236985790 shutdown_connections 2024-04-24T06:33:27.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.959+0000 7fcdc7800640 1 -- 172.21.15.59:0/2236985790 wait complete. 2024-04-24T06:33:27.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.961+0000 7fcdc7800640 1 Processor -- start 2024-04-24T06:33:27.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.961+0000 7fcdc7800640 1 -- start start 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdc006f340 0x7fcdc0151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdc00d0a60 0x7fcdc015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc006f7c0 con 0x7fcdc00b56f0 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc015bc10 con 0x7fcdc00d0a60 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.962+0000 7fcdc7800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc006f940 con 0x7fcdc006f340 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45968/0 (socket says 172.21.15.59:45968) 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 -- 172.21.15.59:0/2715667245 learned_addr learned my addr 172.21.15.59:0/2715667245 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdc4e00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdc006f340 0x7fcdc0151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdc5800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdc00d0a60 0x7fcdc015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 -- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdc006f340 msgr2=0x7fcdc0151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdc006f340 0x7fcdc0151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 -- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdc00d0a60 msgr2=0x7fcdc015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdc00d0a60 0x7fcdc015e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.963+0000 7fcdbfe00640 1 -- 172.21.15.59:0/2715667245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc014f690 con 0x7fcdc00b56f0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdbfe00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c410 tx=0x7fcdac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac017070 con 0x7fcdc00b56f0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac0035f0 con 0x7fcdc00b56f0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0039e0 con 0x7fcdc00b56f0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc015eb90 con 0x7fcdc00b56f0 2024-04-24T06:33:27.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.964+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc0151770 con 0x7fcdc00b56f0 2024-04-24T06:33:27.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.965+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc0068a80 con 0x7fcdc00b56f0 2024-04-24T06:33:27.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.966+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdac01a070 con 0x7fcdc00b56f0 2024-04-24T06:33:27.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.967+0000 7fcdbcc00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 0x7fcda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:27.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.967+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcdac08b0d0 con 0x7fcdc00b56f0 2024-04-24T06:33:27.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.967+0000 7fcdc4e00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 0x7fcda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:27.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.968+0000 7fcdc4e00640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 0x7fcda006edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4034000 tx=0x7fcdb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:27.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:27.972+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdac059460 con 0x7fcdc00b56f0 2024-04-24T06:33:28.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.184+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcdc006ef20 con 0x7fcdc00b56f0 2024-04-24T06:33:28.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.185+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2715667245 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7fcdac0010c0 con 0x7fcdc00b56f0 2024-04-24T06:33:28.186 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.187+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 msgr2=0x7fcda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.187+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 0x7fcda006edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4034000 tx=0x7fcdb402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.187+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 msgr2=0x7fcdc014f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.187+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c410 tx=0x7fcdac00c8d0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 shutdown_connections 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcda006c8f0 0x7fcda006edb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcdc00d0a60 0x7fcdc015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdc00b56f0 0x7fcdc014f150 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 --2- 172.21.15.59:0/2715667245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcdc006f340 0x7fcdc0151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 >> 172.21.15.59:0/2715667245 conn(0x7fcdc0067d90 msgr2=0x7fcdc00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 shutdown_connections 2024-04-24T06:33:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.188+0000 7fcdc7800640 1 -- 172.21.15.59:0/2715667245 wait complete. 2024-04-24T06:33:28.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:28.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.411+0000 7f2d74e00640 1 Processor -- start 2024-04-24T06:33:28.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.411+0000 7f2d74e00640 1 -- start start 2024-04-24T06:33:28.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.412+0000 7f2d74e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2d700d3260 0x7f2d700d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.412+0000 7f2d74e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700cf520 con 0x7f2d7015e9d0 2024-04-24T06:33:28.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d74e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700cf800 con 0x7f2d700d02e0 2024-04-24T06:33:28.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d74e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2d700cfae0 con 0x7f2d700d3260 2024-04-24T06:33:28.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d6f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2d7015e9d0 0x7f2d700d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39412/0 (socket says 172.21.15.59:39412) 2024-04-24T06:33:28.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d6f400640 1 -- 172.21.15.59:0/3964066864 learned_addr learned my addr 172.21.15.59:0/3964066864 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:28.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d67e00640 1 --2- 172.21.15.59:0/3964066864 >> v2:172.21.15.66:3300/0 conn(0x7f2d700d3260 0x7f2d700d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619196277 0 0) 0x7f2d700cf520 con 0x7f2d7015e9d0 2024-04-24T06:33:28.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.413+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d54003520 con 0x7f2d7015e9d0 2024-04-24T06:33:28.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2356070903 0 0) 0x7f2d54003520 con 0x7f2d7015e9d0 2024-04-24T06:33:28.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 >> v2:172.21.15.66:3300/0 conn(0x7f2d700d3260 msgr2=0x7f2d700d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 --2- 172.21.15.59:0/3964066864 >> v2:172.21.15.66:3300/0 conn(0x7f2d700d3260 0x7f2d700d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 >> v1:172.21.15.66:6789/0 conn(0x7f2d700d02e0 legacy=0x7f2d700d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700cfdc0 con 0x7f2d7015e9d0 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2d60001670 con 0x7f2d7015e9d0 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 >> v1:172.21.15.59:6789/0 conn(0x7f2d7015e9d0 legacy=0x7f2d700d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d54004800 0x7f2d54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d540051c0 0x7f2d5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700cfae0 con 0x7f2d540039b0 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5400c060 con 0x7f2d54004800 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5400c260 con 0x7f2d540051c0 2024-04-24T06:33:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.414+0000 7f2d6f400640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6f400640 1 -- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d540051c0 msgr2=0x7f2d5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d540051c0 0x7f2d5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d67e00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d54004800 0x7f2d54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6f400640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d540051c0 0x7f2d5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6f400640 1 -- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d54004800 msgr2=0x7f2d54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6f400640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d54004800 0x7f2d54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6f400640 1 -- 172.21.15.59:0/3964066864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d54003520 con 0x7f2d540039b0 2024-04-24T06:33:28.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d540051c0 0x7f2d5400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d67e00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d54004800 0x7f2d54004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.415+0000 7f2d6fe00640 1 -- 172.21.15.59:0/3964066864 reap_dead start 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.416+0000 7f2d6f400640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d54003dd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c240 tx=0x7f2d54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da32d9cf982bd66f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.416+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60007cf0 con 0x7f2d540039b0 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.416+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d600058d0 con 0x7f2d540039b0 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.416+0000 7f2d6ea00640 1 -- 172.21.15.59:0/3964066864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60005d20 con 0x7f2d540039b0 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 -- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 msgr2=0x7f2d54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d54003dd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c240 tx=0x7f2d54005830 comp rx=0 tx=0).stop 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 -- 172.21.15.59:0/3964066864 shutdown_connections 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 --2- 172.21.15.59:0/3964066864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d54003dd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 -- 172.21.15.59:0/3964066864 >> 172.21.15.59:0/3964066864 conn(0x7f2d7008bf90 msgr2=0x7f2d700d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 -- 172.21.15.59:0/3964066864 shutdown_connections 2024-04-24T06:33:28.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.417+0000 7f2d74e00640 1 -- 172.21.15.59:0/3964066864 wait complete. 2024-04-24T06:33:28.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.419+0000 7f2d74e00640 1 Processor -- start 2024-04-24T06:33:28.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.419+0000 7f2d74e00640 1 -- start start 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d700cd3a0 0x7f2d700b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d7015fb60 0x7f2d7015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700d2cd0 con 0x7f2d540039b0 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700d2b50 con 0x7f2d700cd3a0 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d74e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700d2e50 con 0x7f2d7015fb60 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45982/0 (socket says 172.21.15.59:45982) 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.420+0000 7f2d6f400640 1 -- 172.21.15.59:0/1950153732 learned_addr learned my addr 172.21.15.59:0/1950153732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d7015fb60 0x7f2d7015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d67e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d700cd3a0 0x7f2d700b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 -- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d7015fb60 msgr2=0x7f2d7015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d7015fb60 0x7f2d7015ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 -- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d700cd3a0 msgr2=0x7f2d700b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d700cd3a0 0x7f2d700b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 -- 172.21.15.59:0/1950153732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d70158040 con 0x7f2d540039b0 2024-04-24T06:33:28.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.421+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d60001970 tx=0x7f2d700d1ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.422+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60001c30 con 0x7f2d540039b0 2024-04-24T06:33:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.422+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d60000ed0 con 0x7f2d540039b0 2024-04-24T06:33:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.422+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600049f0 con 0x7f2d540039b0 2024-04-24T06:33:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.422+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d700b88f0 con 0x7f2d540039b0 2024-04-24T06:33:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.422+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d700b7d40 con 0x7f2d540039b0 2024-04-24T06:33:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.423+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7004fd90 con 0x7f2d540039b0 2024-04-24T06:33:28.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.425+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d6003b070 con 0x7f2d540039b0 2024-04-24T06:33:28.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.425+0000 7f2d67400640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.426+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2d600aba90 con 0x7f2d540039b0 2024-04-24T06:33:28.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.426+0000 7f2d67e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.426+0000 7f2d67e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d70158240 tx=0x7f2d58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.430+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d70158040 con 0x7f2d540039b0 2024-04-24T06:33:28.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.670+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d700b3e00 con 0x7f2d540039b0 2024-04-24T06:33:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.671+0000 7f2d67400640 1 -- 172.21.15.59:0/1950153732 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7f2d60079f30 con 0x7f2d540039b0 2024-04-24T06:33:28.673 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:28.673 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:56.100001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:33:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 msgr2=0x7f2d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2d70158240 tx=0x7f2d58008040 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 msgr2=0x7f2d70157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d60001970 tx=0x7f2d700d1ce0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 shutdown_connections 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d7015fb60 0x7f2d7015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d700cd3a0 0x7f2d700b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 --2- 172.21.15.59:0/1950153732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d540039b0 0x7f2d70157b00 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 >> 172.21.15.59:0/1950153732 conn(0x7f2d7008bf90 msgr2=0x7f2d7015f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 shutdown_connections 2024-04-24T06:33:28.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.674+0000 7f2d74e00640 1 -- 172.21.15.59:0/1950153732 wait complete. 2024-04-24T06:33:28.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.892+0000 7f5269e00640 1 Processor -- start 2024-04-24T06:33:28.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.892+0000 7f5269e00640 1 -- start start 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5269e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f526415c3d0 0x7f52640cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5269e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5264160210 con 0x7f52640cd590 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5269e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52641604f0 con 0x7f52640b4860 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5269e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f52641607d0 con 0x7f526415c3d0 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5268a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f526415c3d0 0x7f52640cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5263e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f52640cd590 0x7f52640b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51942/0 (socket says 172.21.15.59:51942) 2024-04-24T06:33:28.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.894+0000 7f5263e00640 1 -- 172.21.15.59:0/2117750683 learned_addr learned my addr 172.21.15.59:0/2117750683 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:28.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.895+0000 7f5268a00640 1 -- 172.21.15.59:0/2117750683 >> v1:172.21.15.66:6789/0 conn(0x7f52640b4860 legacy=0x7f52640cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:28.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.895+0000 7f5268a00640 1 -- 172.21.15.59:0/2117750683 >> v1:172.21.15.59:6789/0 conn(0x7f52640cd590 legacy=0x7f52640b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.895+0000 7f5268a00640 1 -- 172.21.15.59:0/2117750683 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5264160ab0 con 0x7f526415c3d0 2024-04-24T06:33:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.896+0000 7f5268a00640 1 --2- 172.21.15.59:0/2117750683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526415c3d0 0x7f52640cd050 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5254009d90 tx=0x7f525402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0684be23d15206 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.896+0000 7f5263400640 1 -- 172.21.15.59:0/2117750683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254007740 con 0x7f526415c3d0 2024-04-24T06:33:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.896+0000 7f5263400640 1 -- 172.21.15.59:0/2117750683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52540078e0 con 0x7f526415c3d0 2024-04-24T06:33:28.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.896+0000 7f5263400640 1 -- 172.21.15.59:0/2117750683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254007cb0 con 0x7f526415c3d0 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 -- 172.21.15.59:0/2117750683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526415c3d0 msgr2=0x7f52640cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 --2- 172.21.15.59:0/2117750683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526415c3d0 0x7f52640cd050 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5254009d90 tx=0x7f525402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 -- 172.21.15.59:0/2117750683 shutdown_connections 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 --2- 172.21.15.59:0/2117750683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526415c3d0 0x7f52640cd050 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 -- 172.21.15.59:0/2117750683 >> 172.21.15.59:0/2117750683 conn(0x7f52640b21a0 msgr2=0x7f52640b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 -- 172.21.15.59:0/2117750683 shutdown_connections 2024-04-24T06:33:28.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.897+0000 7f5269e00640 1 -- 172.21.15.59:0/2117750683 wait complete. 2024-04-24T06:33:28.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.899+0000 7f5269e00640 1 Processor -- start 2024-04-24T06:33:28.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.899+0000 7f5269e00640 1 -- start start 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52640b4860 0x7f52641f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 0x7f526415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f526415c930 con 0x7f52640cd590 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f526415c7b0 con 0x7f526415c3d0 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5269e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f52640cbee0 con 0x7f52640b4860 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5263e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5263e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54126/0 (socket says 172.21.15.59:54126) 2024-04-24T06:33:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.900+0000 7f5263e00640 1 -- 172.21.15.59:0/1863434744 learned_addr learned my addr 172.21.15.59:0/1863434744 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5268a00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52640b4860 0x7f52641f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 -- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52640b4860 msgr2=0x7f52641f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52640b4860 0x7f52641f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 -- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 msgr2=0x7f526415cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5269400640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 0x7f526415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 0x7f526415cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 -- 172.21.15.59:0/1863434744 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52640cdf20 con 0x7f52640cd590 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5269400640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 0x7f526415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:28.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.901+0000 7f5263e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c400 tx=0x7f524c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.902+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c017070 con 0x7f52640cd590 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.902+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c005ce0 con 0x7f52640cd590 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.902+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c0072f0 con 0x7f52640cd590 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.902+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52640ce1c0 con 0x7f52640cd590 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.902+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52641f5960 con 0x7f52640cd590 2024-04-24T06:33:28.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.903+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526404fd90 con 0x7f52640cd590 2024-04-24T06:33:28.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.904+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f524c01a070 con 0x7f52640cd590 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.905+0000 7f5260c00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.906+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f524c08af70 con 0x7f52640cd590 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.906+0000 7f5268a00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.906+0000 7f5268a00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 0x7f524006edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5254034000 tx=0x7f5254033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:28.910+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f524c059390 con 0x7f52640cd590 2024-04-24T06:33:29.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.122+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f526406e400 con 0x7f52640cd590 2024-04-24T06:33:29.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.124+0000 7f5260c00640 1 -- 172.21.15.59:0/1863434744 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16581 (secure 0 0 0) 0x7f524c058c90 con 0x7f52640cd590 2024-04-24T06:33:29.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:29.127 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:09.051581+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 msgr2=0x7f524006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 0x7f524006edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5254034000 tx=0x7f5254033040 comp rx=0 tx=0).stop 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 msgr2=0x7f52641f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c400 tx=0x7f524c00c8c0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 shutdown_connections 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526415c3d0 0x7f526415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52640cd590 0x7f52641f5df0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 --2- 172.21.15.59:0/1863434744 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52640b4860 0x7f52641f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 >> 172.21.15.59:0/1863434744 conn(0x7f52640b21a0 msgr2=0x7f52640cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 shutdown_connections 2024-04-24T06:33:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.128+0000 7f5269e00640 1 -- 172.21.15.59:0/1863434744 wait complete. 2024-04-24T06:33:29.149 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:33:29.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:29.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.359+0000 7f72a7e00640 1 Processor -- start 2024-04-24T06:33:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.359+0000 7f72a7e00640 1 -- start start 2024-04-24T06:33:29.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72a815e7c0 0x7f72a80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f72a80d0d90 0x7f72a80d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a80cf030 con 0x7f72a80d2630 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80b7000 con 0x7f72a815e7c0 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80cf620 con 0x7f72a80d0d90 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72a815e7c0 0x7f72a80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.360+0000 7f72a6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f72a80d2630 0x7f72a80b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51944/0 (socket says 172.21.15.59:51944) 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6000640 1 -- 172.21.15.59:0/4036637528 learned_addr learned my addr 172.21.15.59:0/4036637528 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6a00640 1 --2- 172.21.15.59:0/4036637528 >> v2:172.21.15.59:3300/0 conn(0x7f72a815e7c0 0x7f72a80d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54142/0 (socket says 172.21.15.59:54142) 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a7400640 1 --2- 172.21.15.59:0/4036637528 >> v2:172.21.15.66:3300/0 conn(0x7f72a80d0d90 0x7f72a80d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6a00640 1 -- 172.21.15.59:0/4036637528 >> v2:172.21.15.66:3300/0 conn(0x7f72a80d0d90 msgr2=0x7f72a80d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6a00640 1 --2- 172.21.15.59:0/4036637528 >> v2:172.21.15.66:3300/0 conn(0x7f72a80d0d90 0x7f72a80d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6a00640 1 -- 172.21.15.59:0/4036637528 >> v1:172.21.15.59:6789/0 conn(0x7f72a80d2630 legacy=0x7f72a80b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.361+0000 7f72a6a00640 1 -- 172.21.15.59:0/4036637528 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a80b6a40 con 0x7f72a815e7c0 2024-04-24T06:33:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.362+0000 7f72a6a00640 1 --2- 172.21.15.59:0/4036637528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a815e7c0 0x7f72a80d20f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f72a80cb580 tx=0x7f729802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d5cc18453ddb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.363+0000 7f72a5600640 1 -- 172.21.15.59:0/4036637528 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298007740 con 0x7f72a815e7c0 2024-04-24T06:33:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.363+0000 7f72a5600640 1 -- 172.21.15.59:0/4036637528 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72980078e0 con 0x7f72a815e7c0 2024-04-24T06:33:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.363+0000 7f72a5600640 1 -- 172.21.15.59:0/4036637528 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298007cb0 con 0x7f72a815e7c0 2024-04-24T06:33:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.363+0000 7f72a7e00640 1 -- 172.21.15.59:0/4036637528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a815e7c0 msgr2=0x7f72a80d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.363+0000 7f72a7e00640 1 --2- 172.21.15.59:0/4036637528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a815e7c0 0x7f72a80d20f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f72a80cb580 tx=0x7f729802a680 comp rx=0 tx=0).stop 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 -- 172.21.15.59:0/4036637528 shutdown_connections 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 --2- 172.21.15.59:0/4036637528 >> v2:172.21.15.66:3300/0 conn(0x7f72a80d0d90 0x7f72a80d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 --2- 172.21.15.59:0/4036637528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a815e7c0 0x7f72a80d20f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 -- 172.21.15.59:0/4036637528 >> 172.21.15.59:0/4036637528 conn(0x7f72a809d920 msgr2=0x7f72a808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 -- 172.21.15.59:0/4036637528 shutdown_connections 2024-04-24T06:33:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.364+0000 7f72a7e00640 1 -- 172.21.15.59:0/4036637528 wait complete. 2024-04-24T06:33:29.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.365+0000 7f72a7e00640 1 Processor -- start 2024-04-24T06:33:29.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 -- start start 2024-04-24T06:33:29.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72a80d0d90 0x7f72a80cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72a815e7c0 0x7f72a80b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80cf2b0 con 0x7f72a80d2630 2024-04-24T06:33:29.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80d1700 con 0x7f72a80d0d90 2024-04-24T06:33:29.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.366+0000 7f72a7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f72a80b6700 con 0x7f72a815e7c0 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54148/0 (socket says 172.21.15.59:54148) 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6000640 1 -- 172.21.15.59:0/3937714131 learned_addr learned my addr 172.21.15.59:0/3937714131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6a00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72a80d0d90 0x7f72a80cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a7400640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72a815e7c0 0x7f72a80b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6000640 1 -- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72a815e7c0 msgr2=0x7f72a80b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.367+0000 7f72a6000640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72a815e7c0 0x7f72a80b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a6000640 1 -- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72a80d0d90 msgr2=0x7f72a80cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a6000640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72a80d0d90 0x7f72a80cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a6000640 1 -- 172.21.15.59:0/3937714131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a80c90b0 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a6000640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f729400c440 tx=0x7f7294001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294017070 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294005ce0 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940072f0 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a80cd9a0 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.368+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a80c8810 con 0x7f72a80d2630 2024-04-24T06:33:29.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.369+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a804fd90 con 0x7f72a80d2630 2024-04-24T06:33:29.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.371+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f729401a070 con 0x7f72a80d2630 2024-04-24T06:33:29.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.372+0000 7f729ea00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 0x7f728806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.372+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f729408ae50 con 0x7f72a80d2630 2024-04-24T06:33:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.372+0000 7f72a6a00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 0x7f728806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.373+0000 7f72a6a00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 0x7f728806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7298034000 tx=0x7f729802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.376+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72a80cf030 con 0x7f72a80d2630 2024-04-24T06:33:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.619+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a80b3bd0 con 0x7f72a80d2630 2024-04-24T06:33:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.623+0000 7f729ea00640 1 -- 172.21.15.59:0/3937714131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7f7294058c20 con 0x7f72a80d2630 2024-04-24T06:33:29.625 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:29.625 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:56.100001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:33:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 msgr2=0x7f728806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 0x7f728806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7298034000 tx=0x7f729802f000 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 msgr2=0x7f72a80c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f729400c440 tx=0x7f7294001e70 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 shutdown_connections 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f728806c8f0 0x7f728806edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72a815e7c0 0x7f72a80b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72a80d2630 0x7f72a80c9ff0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 --2- 172.21.15.59:0/3937714131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72a80d0d90 0x7f72a80cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.625+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 >> 172.21.15.59:0/3937714131 conn(0x7f72a809d920 msgr2=0x7f72a815eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.626+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 shutdown_connections 2024-04-24T06:33:29.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.626+0000 7f72a7e00640 1 -- 172.21.15.59:0/3937714131 wait complete. 2024-04-24T06:33:29.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:29.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.853+0000 7ff040e00640 1 Processor -- start 2024-04-24T06:33:29.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.854+0000 7ff040e00640 1 -- start start 2024-04-24T06:33:29.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.855+0000 7ff040e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff03c0cce70 0x7ff03c162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.855+0000 7ff040e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff03c0d07c0 0x7ff03c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff040e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff03c1634e0 con 0x7ff03c0b1f60 2024-04-24T06:33:29.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff040e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff03c0ce190 con 0x7ff03c0d07c0 2024-04-24T06:33:29.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff040e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff03c0ceb60 con 0x7ff03c0cce70 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff033e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff03c0d07c0 0x7ff03c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff033e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff03c0d07c0 0x7ff03c0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54162/0 (socket says 172.21.15.59:54162) 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff033e00640 1 -- 172.21.15.59:0/2704440151 learned_addr learned my addr 172.21.15.59:0/2704440151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.856+0000 7ff03b400640 1 --2- 172.21.15.59:0/2704440151 >> v2:172.21.15.66:3300/0 conn(0x7ff03c0cce70 0x7ff03c162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.857+0000 7ff033e00640 1 -- 172.21.15.59:0/2704440151 >> v2:172.21.15.66:3300/0 conn(0x7ff03c0cce70 msgr2=0x7ff03c162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.857+0000 7ff033e00640 1 --2- 172.21.15.59:0/2704440151 >> v2:172.21.15.66:3300/0 conn(0x7ff03c0cce70 0x7ff03c162fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.857+0000 7ff033e00640 1 -- 172.21.15.59:0/2704440151 >> v1:172.21.15.66:6789/0 conn(0x7ff03c0b1f60 legacy=0x7ff03c0b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:29.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.857+0000 7ff033e00640 1 -- 172.21.15.59:0/2704440151 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0d0ca0 con 0x7ff03c0d07c0 2024-04-24T06:33:29.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.858+0000 7ff033e00640 1 --2- 172.21.15.59:0/2704440151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0d07c0 0x7ff03c0ce620 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0ce310 tx=0x7ff02402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b9875ebfe8d5a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.858+0000 7ff03aa00640 1 -- 172.21.15.59:0/2704440151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024037070 con 0x7ff03c0d07c0 2024-04-24T06:33:29.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.858+0000 7ff03aa00640 1 -- 172.21.15.59:0/2704440151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff024032440 con 0x7ff03c0d07c0 2024-04-24T06:33:29.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.858+0000 7ff03aa00640 1 -- 172.21.15.59:0/2704440151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024032810 con 0x7ff03c0d07c0 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 -- 172.21.15.59:0/2704440151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0d07c0 msgr2=0x7ff03c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 --2- 172.21.15.59:0/2704440151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0d07c0 0x7ff03c0ce620 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0ce310 tx=0x7ff02402a1f0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 -- 172.21.15.59:0/2704440151 shutdown_connections 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 --2- 172.21.15.59:0/2704440151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0d07c0 0x7ff03c0ce620 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 --2- 172.21.15.59:0/2704440151 >> v2:172.21.15.66:3300/0 conn(0x7ff03c0cce70 0x7ff03c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 -- 172.21.15.59:0/2704440151 >> 172.21.15.59:0/2704440151 conn(0x7ff03c09aa00 msgr2=0x7ff03c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 -- 172.21.15.59:0/2704440151 shutdown_connections 2024-04-24T06:33:29.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.859+0000 7ff040e00640 1 -- 172.21.15.59:0/2704440151 wait complete. 2024-04-24T06:33:29.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.861+0000 7ff040e00640 1 Processor -- start 2024-04-24T06:33:29.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.861+0000 7ff040e00640 1 -- start start 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff03c0b1f60 0x7ff03c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0d07c0 0x7ff03c150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c0d0f80 con 0x7ff03c0cce70 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c0cde20 con 0x7ff03c0d07c0 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff040e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c0cd250 con 0x7ff03c0b1f60 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff033e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff033e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54178/0 (socket says 172.21.15.59:54178) 2024-04-24T06:33:29.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.862+0000 7ff033e00640 1 -- 172.21.15.59:0/257499145 learned_addr learned my addr 172.21.15.59:0/257499145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:29.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 -- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff03c0b1f60 msgr2=0x7ff03c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff03c0b1f60 0x7ff03c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 -- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0d07c0 msgr2=0x7ff03c150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0d07c0 0x7ff03c150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 -- 172.21.15.59:0/257499145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0ca650 con 0x7ff03c0cce70 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff024000c00 tx=0x7ff0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024041020 con 0x7ff03c0cce70 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff024030970 con 0x7ff03c0cce70 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024037040 con 0x7ff03c0cce70 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c0c9070 con 0x7ff03c0cce70 2024-04-24T06:33:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.863+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c0c9630 con 0x7ff03c0cce70 2024-04-24T06:33:29.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.864+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c04fd90 con 0x7ff03c0cce70 2024-04-24T06:33:29.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.865+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff024044020 con 0x7ff03c0cce70 2024-04-24T06:33:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.866+0000 7ff033400640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 0x7ff01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:29.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.867+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff0240ac2f0 con 0x7ff03c0cce70 2024-04-24T06:33:29.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.867+0000 7ff03b400640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 0x7ff01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:29.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.868+0000 7ff03b400640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 0x7ff01806edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff02c006fd0 tx=0x7ff02c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:29.872+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff02404b070 con 0x7ff03c0cce70 2024-04-24T06:33:30.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.114+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff03c0ae8f0 con 0x7ff03c0cce70 2024-04-24T06:33:30.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.115+0000 7ff033400640 1 -- 172.21.15.59:0/257499145 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7ff02407a690 con 0x7ff03c0cce70 2024-04-24T06:33:30.118 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:30.118 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:32:07.494567+0000","modified":"2024-04-24T06:32:56.100001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6192},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6192":{"gid":6192,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.59:6837/2525582962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2525582962},{"type":"v1","addr":"172.21.15.59:6837","nonce":2525582962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6192,"qdb_cluster":[6192]},"id":4}]} 2024-04-24T06:33:30.118 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-24T06:33:30.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.117+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 msgr2=0x7ff01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.117+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 0x7ff01806edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff02c006fd0 tx=0x7ff02c008040 comp rx=0 tx=0).stop 2024-04-24T06:33:30.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.117+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 msgr2=0x7ff03c0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.117+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff024000c00 tx=0x7ff0240046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 shutdown_connections 2024-04-24T06:33:30.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff01806c8f0 0x7ff01806edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff03c0d07c0 0x7ff03c150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff03c0cce70 0x7ff03c0cae60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 --2- 172.21.15.59:0/257499145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff03c0b1f60 0x7ff03c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 >> 172.21.15.59:0/257499145 conn(0x7ff03c09aa00 msgr2=0x7ff03c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 shutdown_connections 2024-04-24T06:33:30.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.118+0000 7ff040e00640 1 -- 172.21.15.59:0/257499145 wait complete. 2024-04-24T06:33:30.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.345+0000 7f5e3a600640 1 Processor -- start 2024-04-24T06:33:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.345+0000 7f5e3a600640 1 -- start start 2024-04-24T06:33:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e3a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e340b3bb0 0x7f5e3409b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e3a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e340d0840 con 0x7f5e3406dc00 2024-04-24T06:33:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e3a600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e340d0ae0 con 0x7f5e3409bbe0 2024-04-24T06:33:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e3a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5e340cdc60 con 0x7f5e340b3bb0 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e340b3bb0 0x7f5e3409b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e33e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5e3406dc00 0x7f5e3415e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51948/0 (socket says 172.21.15.59:51948) 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e33e00640 1 -- 172.21.15.59:0/3803627095 learned_addr learned my addr 172.21.15.59:0/3803627095 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.346+0000 7f5e33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e340b3bb0 0x7f5e3409b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54184/0 (socket says 172.21.15.59:54184) 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.347+0000 7f5e33400640 1 -- 172.21.15.59:0/3803627095 >> v1:172.21.15.66:6789/0 conn(0x7f5e3409bbe0 legacy=0x7f5e3406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.347+0000 7f5e33400640 1 -- 172.21.15.59:0/3803627095 >> v1:172.21.15.59:6789/0 conn(0x7f5e3406dc00 legacy=0x7f5e3415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.347+0000 7f5e33400640 1 -- 172.21.15.59:0/3803627095 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e340d0dc0 con 0x7f5e340b3bb0 2024-04-24T06:33:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.348+0000 7f5e33400640 1 --2- 172.21.15.59:0/3803627095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e340b3bb0 0x7f5e3409b6a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097a0 tx=0x7f5e2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15327138f0853cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.348+0000 7f5e32000640 1 -- 172.21.15.59:0/3803627095 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24037070 con 0x7f5e340b3bb0 2024-04-24T06:33:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.348+0000 7f5e32000640 1 -- 172.21.15.59:0/3803627095 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24032440 con 0x7f5e340b3bb0 2024-04-24T06:33:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.348+0000 7f5e32000640 1 -- 172.21.15.59:0/3803627095 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24032810 con 0x7f5e340b3bb0 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 -- 172.21.15.59:0/3803627095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e340b3bb0 msgr2=0x7f5e3409b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 --2- 172.21.15.59:0/3803627095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e340b3bb0 0x7f5e3409b6a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097a0 tx=0x7f5e2402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 -- 172.21.15.59:0/3803627095 shutdown_connections 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 --2- 172.21.15.59:0/3803627095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e340b3bb0 0x7f5e3409b6a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 -- 172.21.15.59:0/3803627095 >> 172.21.15.59:0/3803627095 conn(0x7f5e340b40f0 msgr2=0x7f5e340b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 -- 172.21.15.59:0/3803627095 shutdown_connections 2024-04-24T06:33:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.349+0000 7f5e3a600640 1 -- 172.21.15.59:0/3803627095 wait complete. 2024-04-24T06:33:30.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.351+0000 7f5e3a600640 1 Processor -- start 2024-04-24T06:33:30.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.351+0000 7f5e3a600640 1 -- start start 2024-04-24T06:33:30.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e340b3bb0 0x7f5e340cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5e340cc390 0x7f5e34162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d1660 con 0x7f5e3406dc00 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d1380 con 0x7f5e340b3bb0 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e3a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340d1b80 con 0x7f5e340cc390 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54188/0 (socket says 172.21.15.59:54188) 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e33400640 1 -- 172.21.15.59:0/2763997499 learned_addr learned my addr 172.21.15.59:0/2763997499 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e33400640 1 -- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5e340cc390 msgr2=0x7f5e34162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.352+0000 7f5e33400640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5e340cc390 0x7f5e34162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e33400640 1 -- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e340b3bb0 msgr2=0x7f5e340cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e33400640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e340b3bb0 0x7f5e340cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e33400640 1 -- 172.21.15.59:0/2763997499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e340c6f90 con 0x7f5e3406dc00 2024-04-24T06:33:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e33400640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000c00 tx=0x7f5e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24041070 con 0x7f5e3406dc00 2024-04-24T06:33:30.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2403c930 con 0x7f5e3406dc00 2024-04-24T06:33:30.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24037040 con 0x7f5e3406dc00 2024-04-24T06:33:30.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e340c69d0 con 0x7f5e3406dc00 2024-04-24T06:33:30.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.353+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e340ca510 con 0x7f5e3406dc00 2024-04-24T06:33:30.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.355+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e34068a80 con 0x7f5e3406dc00 2024-04-24T06:33:30.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.356+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e24044070 con 0x7f5e3406dc00 2024-04-24T06:33:30.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.356+0000 7f5e2b400640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 0x7f5e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.357+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5e240ac870 con 0x7f5e3406dc00 2024-04-24T06:33:30.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.357+0000 7f5e32a00640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 0x7f5e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.358+0000 7f5e32a00640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 0x7f5e1406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003cc0 tx=0x7f5e200079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.361+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e2407adb0 con 0x7f5e3406dc00 2024-04-24T06:33:30.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.571+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e3409b480 con 0x7f5e3406dc00 2024-04-24T06:33:30.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.573+0000 7f5e2b400640 1 -- 172.21.15.59:0/2763997499 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16581 (secure 0 0 0) 0x7f5e2407a6b0 con 0x7f5e3406dc00 2024-04-24T06:33:30.575 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:30.575 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:32:09.051581+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:32:05.373199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-24T06:32:06.536666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 msgr2=0x7f5e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 0x7f5e1406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003cc0 tx=0x7f5e200079b0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 msgr2=0x7f5e340cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000c00 tx=0x7f5e240046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 shutdown_connections 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5e1406c8f0 0x7f5e1406edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5e340cc390 0x7f5e34162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e340b3bb0 0x7f5e340cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 --2- 172.21.15.59:0/2763997499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e3406dc00 0x7f5e340cb7f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.575+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 >> 172.21.15.59:0/2763997499 conn(0x7f5e340b40f0 msgr2=0x7f5e3415ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.576+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 shutdown_connections 2024-04-24T06:33:30.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.576+0000 7f5e3a600640 1 -- 172.21.15.59:0/2763997499 wait complete. 2024-04-24T06:33:30.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.814+0000 7f8ce2000640 1 Processor -- start 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.814+0000 7f8ce2000640 1 -- start start 2024-04-24T06:33:30.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.816+0000 7f8ce2000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 0x7f8cdc0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.816+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 0x7f8cdc0af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.816+0000 7f8ce2000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc1676a0 con 0x7f8cdc0d0c10 2024-04-24T06:33:30.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.816+0000 7f8ce2000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8cdc166b20 con 0x7f8cdc0d0830 2024-04-24T06:33:30.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.816+0000 7f8ce2000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc166e00 con 0x7f8cdc0af010 2024-04-24T06:33:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.817+0000 7f8cdbe00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f8cdc0d0c10 0x7f8cdc163bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:55960/0 (socket says 172.21.15.59:55960) 2024-04-24T06:33:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.817+0000 7f8cdbe00640 1 -- 172.21.15.59:0/159896529 learned_addr learned my addr 172.21.15.59:0/159896529 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.817+0000 7f8ce1600640 1 --2- 172.21.15.59:0/159896529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 0x7f8cdc0af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.817+0000 7f8ce0c00640 1 --2- 172.21.15.59:0/159896529 >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 0x7f8cdc0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8cdb400640 1 -- 172.21.15.59:0/159896529 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3183317875 0 0) 0x7f8cdc1676a0 con 0x7f8cdc0d0c10 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8cdb400640 1 -- 172.21.15.59:0/159896529 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc4003520 con 0x7f8cdc0d0c10 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8ce1600640 1 -- 172.21.15.59:0/159896529 >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 msgr2=0x7f8cdc0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8ce1600640 1 --2- 172.21.15.59:0/159896529 >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 0x7f8cdc0cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8ce1600640 1 -- 172.21.15.59:0/159896529 >> v1:172.21.15.66:6789/0 conn(0x7f8cdc0d0c10 legacy=0x7f8cdc163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8ce1600640 1 -- 172.21.15.59:0/159896529 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1670e0 con 0x7f8cdc0af010 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.818+0000 7f8ce0c00640 1 --2- 172.21.15.59:0/159896529 >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 0x7f8cdc0cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.819+0000 7f8ce1600640 1 --2- 172.21.15.59:0/159896529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 0x7f8cdc0af450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd805c6e18e1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.820+0000 7f8cdb400640 1 -- 172.21.15.59:0/159896529 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4031600 con 0x7f8cdc0af010 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.820+0000 7f8cdb400640 1 -- 172.21.15.59:0/159896529 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd40317a0 con 0x7f8cdc0af010 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.820+0000 7f8cdb400640 1 -- 172.21.15.59:0/159896529 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4031b70 con 0x7f8cdc0af010 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 -- 172.21.15.59:0/159896529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 msgr2=0x7f8cdc0af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 --2- 172.21.15.59:0/159896529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 0x7f8cdc0af450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402a5d0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 -- 172.21.15.59:0/159896529 shutdown_connections 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 --2- 172.21.15.59:0/159896529 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0af010 0x7f8cdc0af450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 --2- 172.21.15.59:0/159896529 >> v2:172.21.15.66:3300/0 conn(0x7f8cdc0d0830 0x7f8cdc0cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 -- 172.21.15.59:0/159896529 >> 172.21.15.59:0/159896529 conn(0x7f8cdc06bdf0 msgr2=0x7f8cdc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 -- 172.21.15.59:0/159896529 shutdown_connections 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.821+0000 7f8ce2000640 1 -- 172.21.15.59:0/159896529 wait complete. 2024-04-24T06:33:30.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.823+0000 7f8ce2000640 1 Processor -- start 2024-04-24T06:33:30.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.823+0000 7f8ce2000640 1 -- start start 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0d0830 0x7f8cdc163de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc0d0c10 0x7f8cdc09c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0b45f0 con 0x7f8cdc0af010 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0af990 con 0x7f8cdc0d0c10 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce2000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc166740 con 0x7f8cdc0d0830 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54200/0 (socket says 172.21.15.59:54200) 2024-04-24T06:33:30.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.824+0000 7f8ce0c00640 1 -- 172.21.15.59:0/1052359955 learned_addr learned my addr 172.21.15.59:0/1052359955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce1600640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc0d0c10 0x7f8cdc09c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8cdbe00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0d0830 0x7f8cdc163de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 -- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0d0830 msgr2=0x7f8cdc163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0d0830 0x7f8cdc163de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 -- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc0d0c10 msgr2=0x7f8cdc09c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc0d0c10 0x7f8cdc09c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 -- 172.21.15.59:0/1052359955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc164540 con 0x7f8cdc0af010 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce0c00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000ebe0 tx=0x7f8cd000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0017070 con 0x7f8cdc0af010 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd000cc30 con 0x7f8cdc0af010 2024-04-24T06:33:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0013470 con 0x7f8cdc0af010 2024-04-24T06:33:30.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.825+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc164820 con 0x7f8cdc0af010 2024-04-24T06:33:30.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.826+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc164de0 con 0x7f8cdc0af010 2024-04-24T06:33:30.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.827+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc04fd90 con 0x7f8cdc0af010 2024-04-24T06:33:30.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.827+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cd001a070 con 0x7f8cdc0af010 2024-04-24T06:33:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.829+0000 7f8cd8c00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 0x7f8cbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.829+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8cd008be10 con 0x7f8cdc0af010 2024-04-24T06:33:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.829+0000 7f8cdbe00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 0x7f8cbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.830+0000 7f8cdbe00640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 0x7f8cbc06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc1627c0 tx=0x7f8cc8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:30.833+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cd008f030 con 0x7f8cdc0af010 2024-04-24T06:33:31.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.077+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8cdc09e150 con 0x7f8cdc0af010 2024-04-24T06:33:31.079 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:33:31.078+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:33:31.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.081+0000 7f8cd8c00640 1 -- 172.21.15.59:0/1052359955 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f8cd005a230 con 0x7f8cdc0af010 2024-04-24T06:33:31.083 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 msgr2=0x7f8cbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 0x7f8cbc06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc1627c0 tx=0x7f8cc8009040 comp rx=0 tx=0).stop 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 msgr2=0x7f8cdc1638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000ebe0 tx=0x7f8cd000c520 comp rx=0 tx=0).stop 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 shutdown_connections 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8cbc06c8f0 0x7f8cbc06edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cdc0d0c10 0x7f8cdc09c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8cdc0d0830 0x7f8cdc163de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 --2- 172.21.15.59:0/1052359955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cdc0af010 0x7f8cdc1638a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.085+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 >> 172.21.15.59:0/1052359955 conn(0x7f8cdc06bdf0 msgr2=0x7f8cdc0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.086+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 shutdown_connections 2024-04-24T06:33:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.086+0000 7f8ce2000640 1 -- 172.21.15.59:0/1052359955 wait complete. 2024-04-24T06:33:31.105 INFO:tasks.ceph.mds.a.smithi059.stderr: -6678> 2024-04-24T06:32:12.975+0000 7f544b800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:33:31.109 INFO:tasks.ceph.mds.a.smithi059.stderr: -4975> 2024-04-24T06:32:18.746+0000 7f5446800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6351=0-4194304@1} caps={6351=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6351 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55d8a99c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:33:31.112 INFO:tasks.ceph.mds.a.smithi059.stderr: -2983> 2024-04-24T06:32:50.835+0000 7f5446800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-24T06:32:17.158789+0000 1=1+0) caps={6351=pAsLsXsFs/-@2},l=6351(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55d8a99c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:33:31.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:33:31.163 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:33:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.300+0000 7f43c7200640 1 Processor -- start 2024-04-24T06:33:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.301+0000 7f43c7200640 1 -- start start 2024-04-24T06:33:31.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.302+0000 7f43c7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f43c0162c10 0x7f43c00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.302+0000 7f43c7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c00d0fd0 con 0x7f43c00b4670 2024-04-24T06:33:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.302+0000 7f43c7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c00b35d0 con 0x7f43c0072480 2024-04-24T06:33:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.302+0000 7f43c7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f43c00b38b0 con 0x7f43c0162c10 2024-04-24T06:33:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.303+0000 7f43bf400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f43c00b4670 0x7f43c0071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51970/0 (socket says 172.21.15.59:51970) 2024-04-24T06:33:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.303+0000 7f43bf400640 1 -- 172.21.15.59:0/1664349827 learned_addr learned my addr 172.21.15.59:0/1664349827 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:31.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.303+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414547787 0 0) 0x7f43ac001110 con 0x7f43c00b4670 2024-04-24T06:33:31.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.303+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43a4003520 con 0x7f43c00b4670 2024-04-24T06:33:31.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bfe00640 1 --2- 172.21.15.59:0/1664349827 >> v2:172.21.15.66:3300/0 conn(0x7f43c0162c10 0x7f43c00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1644042975 0 0) 0x7f43a4003520 con 0x7f43c00b4670 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 >> v2:172.21.15.66:3300/0 conn(0x7f43c0162c10 msgr2=0x7f43c00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 --2- 172.21.15.59:0/1664349827 >> v2:172.21.15.66:3300/0 conn(0x7f43c0162c10 0x7f43c00d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 >> v1:172.21.15.66:6789/0 conn(0x7f43c0072480 legacy=0x7f43c006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c00b3b90 con 0x7f43c00b4670 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f43ac001840 con 0x7f43c00b4670 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 >> v1:172.21.15.59:6789/0 conn(0x7f43c00b4670 legacy=0x7f43c0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43a40039b0 0x7f43a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 0x7f43a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a40052e0 0x7f43a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b38b0 con 0x7f43a4004800 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400c1e0 con 0x7f43a40039b0 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f43a400c3e0 con 0x7f43a40052e0 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bfe00640 1 --2- 172.21.15.59:0/1664349827 >> v2:172.21.15.66:3300/0 conn(0x7f43c0162c10 0x7f43c00d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.304+0000 7f43bf400640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 0x7f43a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bf400640 1 -- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a40052e0 msgr2=0x7f43a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bf400640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a40052e0 0x7f43a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bfe00640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43a40039b0 0x7f43a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bf400640 1 -- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43a40039b0 msgr2=0x7f43a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bf400640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43a40039b0 0x7f43a4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43bf400640 1 -- 172.21.15.59:0/1664349827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a4003520 con 0x7f43a4004800 2024-04-24T06:33:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.305+0000 7f43c5200640 1 -- 172.21.15.59:0/1664349827 reap_dead start 2024-04-24T06:33:31.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.306+0000 7f43bf400640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 0x7f43a4004c80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00c490 tx=0x7f43a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea4c124ca798a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.306+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00c0e0 con 0x7f43a4004800 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.306+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ac034070 con 0x7f43a4004800 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.306+0000 7f43bea00640 1 -- 172.21.15.59:0/1664349827 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac005d70 con 0x7f43a4004800 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 -- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 msgr2=0x7f43a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 0x7f43a4004c80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00c490 tx=0x7f43a4005830 comp rx=0 tx=0).stop 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 -- 172.21.15.59:0/1664349827 shutdown_connections 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 --2- 172.21.15.59:0/1664349827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a4004800 0x7f43a4004c80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 -- 172.21.15.59:0/1664349827 >> 172.21.15.59:0/1664349827 conn(0x7f43c0067d90 msgr2=0x7f43c006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 -- 172.21.15.59:0/1664349827 shutdown_connections 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.307+0000 7f43c7200640 1 -- 172.21.15.59:0/1664349827 wait complete. 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.309+0000 7f43c7200640 1 Processor -- start 2024-04-24T06:33:31.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.309+0000 7f43c7200640 1 -- start start 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a4004800 0x7f43c014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43c01ee2a0 0x7f43c01ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0162ff0 con 0x7f43c01653c0 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b4150 con 0x7f43c01ee2a0 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f43c0163170 con 0x7f43a4004800 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54242/0 (socket says 172.21.15.59:54242) 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 -- 172.21.15.59:0/4169345901 learned_addr learned my addr 172.21.15.59:0/4169345901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 -- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a4004800 msgr2=0x7f43c014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a4004800 0x7f43c014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 -- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43c01ee2a0 msgr2=0x7f43c01ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43c01ee2a0 0x7f43c01ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.310+0000 7f43c5200640 1 -- 172.21.15.59:0/4169345901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c01eed80 con 0x7f43c01653c0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43c5200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b5ab0 tx=0x7f43b800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80074e0 con 0x7f43c01653c0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8004020 con 0x7f43c01653c0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004410 con 0x7f43c01653c0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c01ef060 con 0x7f43c01653c0 2024-04-24T06:33:31.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.311+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01eb6a0 con 0x7f43c01653c0 2024-04-24T06:33:31.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.313+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c0165d80 con 0x7f43c01653c0 2024-04-24T06:33:31.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.314+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43b80057c0 con 0x7f43c01653c0 2024-04-24T06:33:31.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.315+0000 7f43b7e00640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:31.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.315+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f43b808acf0 con 0x7f43c01653c0 2024-04-24T06:33:31.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.315+0000 7f43bfe00640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:31.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.316+0000 7f43bfe00640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 0x7f43a006ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43c006ad70 tx=0x7f43b0008410 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:31.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.320+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43b8059020 con 0x7f43c01653c0 2024-04-24T06:33:31.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:31.567+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f43c0068a80 con 0x7f43c01653c0 2024-04-24T06:33:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.085+0000 7f43b7e00640 1 -- 172.21.15.59:0/4169345901 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f43b8058920 con 0x7f43c01653c0 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.088+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 msgr2=0x7f43a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.088+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 0x7f43a006ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f43c006ad70 tx=0x7f43b0008410 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.088+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 msgr2=0x7f43c0165840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.088+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b5ab0 tx=0x7f43b800c820 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 shutdown_connections 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43c01653c0 0x7f43c0165840 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43c01ee2a0 0x7f43c01ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 --2- 172.21.15.59:0/4169345901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43a4004800 0x7f43c014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 >> 172.21.15.59:0/4169345901 conn(0x7f43c0067d90 msgr2=0x7f43c00b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 shutdown_connections 2024-04-24T06:33:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.089+0000 7f43c7200640 1 -- 172.21.15.59:0/4169345901 wait complete. 2024-04-24T06:33:32.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:33:32.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.324+0000 7f24d2800640 1 Processor -- start 2024-04-24T06:33:32.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.324+0000 7f24d2800640 1 -- start start 2024-04-24T06:33:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.325+0000 7f24d2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24cc15eb60 0x7f24cc0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.325+0000 7f24d2800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24cc0d5470 con 0x7f24cc0d1930 2024-04-24T06:33:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.325+0000 7f24d2800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24cc0d24c0 con 0x7f24cc0d3560 2024-04-24T06:33:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.325+0000 7f24d2800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f24cc0d30d0 con 0x7f24cc15eb60 2024-04-24T06:33:32.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24cbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f24cc0d1930 0x7f24cc0b76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51974/0 (socket says 172.21.15.59:51974) 2024-04-24T06:33:32.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24cbe00640 1 -- 172.21.15.59:0/2555753725 learned_addr learned my addr 172.21.15.59:0/2555753725 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:32.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24cb400640 1 --2- 172.21.15.59:0/2555753725 >> v2:172.21.15.59:3300/0 conn(0x7f24cc15eb60 0x7f24cc0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24cb400640 1 -- 172.21.15.59:0/2555753725 >> v1:172.21.15.66:6789/0 conn(0x7f24cc0d3560 legacy=0x7f24cc0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24cb400640 1 -- 172.21.15.59:0/2555753725 >> v1:172.21.15.59:6789/0 conn(0x7f24cc0d1930 legacy=0x7f24cc0b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.326+0000 7f24ca000640 1 -- 172.21.15.59:0/2555753725 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910635284 0 0) 0x7f24cc0d5470 con 0x7f24cc0d1930 2024-04-24T06:33:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.327+0000 7f24cb400640 1 -- 172.21.15.59:0/2555753725 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc0d27a0 con 0x7f24cc15eb60 2024-04-24T06:33:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24cb400640 1 --2- 172.21.15.59:0/2555753725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc15eb60 0x7f24cc0cff70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097a0 tx=0x7f24b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35958f0f8571033e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24ca000640 1 -- 172.21.15.59:0/2555753725 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4037070 con 0x7f24cc15eb60 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24ca000640 1 -- 172.21.15.59:0/2555753725 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b4032440 con 0x7f24cc15eb60 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24ca000640 1 -- 172.21.15.59:0/2555753725 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4032810 con 0x7f24cc15eb60 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24d2800640 1 -- 172.21.15.59:0/2555753725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc15eb60 msgr2=0x7f24cc0cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.328+0000 7f24d2800640 1 --2- 172.21.15.59:0/2555753725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc15eb60 0x7f24cc0cff70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097a0 tx=0x7f24b402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.329+0000 7f24d2800640 1 -- 172.21.15.59:0/2555753725 shutdown_connections 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.329+0000 7f24d2800640 1 --2- 172.21.15.59:0/2555753725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc15eb60 0x7f24cc0cff70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.329+0000 7f24d2800640 1 -- 172.21.15.59:0/2555753725 >> 172.21.15.59:0/2555753725 conn(0x7f24cc08bf90 msgr2=0x7f24cc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.329+0000 7f24d2800640 1 -- 172.21.15.59:0/2555753725 shutdown_connections 2024-04-24T06:33:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.329+0000 7f24d2800640 1 -- 172.21.15.59:0/2555753725 wait complete. 2024-04-24T06:33:32.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.330+0000 7f24d2800640 1 Processor -- start 2024-04-24T06:33:32.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 -- start start 2024-04-24T06:33:32.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24cc0d3560 0x7f24cc0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24cc15eb60 0x7f24cc14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc15ef40 con 0x7f24cc0d1930 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc0d2d60 con 0x7f24cc0d3560 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.331+0000 7f24d2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc0d04b0 con 0x7f24cc15eb60 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54256/0 (socket says 172.21.15.59:54256) 2024-04-24T06:33:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 -- 172.21.15.59:0/1208958951 learned_addr learned my addr 172.21.15.59:0/1208958951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 -- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24cc15eb60 msgr2=0x7f24cc14e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24cc15eb60 0x7f24cc14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 -- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24cc0d3560 msgr2=0x7f24cc0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24cc0d3560 0x7f24cc0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.332+0000 7f24cb400640 1 -- 172.21.15.59:0/1208958951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc0c9e10 con 0x7f24cc0d1930 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24cb400640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097a0 tx=0x7f24b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4037040 con 0x7f24cc0d1930 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b4004250 con 0x7f24cc0d1930 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4032e20 con 0x7f24cc0d1930 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc0ca0f0 con 0x7f24cc0d1930 2024-04-24T06:33:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.333+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc0ca6b0 con 0x7f24cc0d1930 2024-04-24T06:33:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.334+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc069190 con 0x7f24cc0d1930 2024-04-24T06:33:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.335+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24b4030770 con 0x7f24cc0d1930 2024-04-24T06:33:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.336+0000 7f24c3400640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.336+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f24b40acfc0 con 0x7f24cc0d1930 2024-04-24T06:33:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.337+0000 7f24caa00640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.337+0000 7f24caa00640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1f8460 tx=0x7f24bc0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.341+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24b407b260 con 0x7f24cc0d1930 2024-04-24T06:33:32.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:32.549+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24cc0b6d80 con 0x7f24cc0d1930 2024-04-24T06:33:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.100+0000 7f24c3400640 1 -- 172.21.15.59:0/1208958951 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f24b407ab60 con 0x7f24cc0d1930 2024-04-24T06:33:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:33:33.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.103+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 msgr2=0x7f24a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.103+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1f8460 tx=0x7f24bc0079b0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.103+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 msgr2=0x7f24cc0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.103+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097a0 tx=0x7f24b40046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 shutdown_connections 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24cc15eb60 0x7f24cc14e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24cc0d3560 0x7f24cc0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 --2- 172.21.15.59:0/1208958951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24cc0d1930 0x7f24cc0cd5c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 >> 172.21.15.59:0/1208958951 conn(0x7f24cc08bf90 msgr2=0x7f24cc0b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 shutdown_connections 2024-04-24T06:33:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.104+0000 7f24d2800640 1 -- 172.21.15.59:0/1208958951 wait complete. 2024-04-24T06:33:33.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:33:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.362+0000 7f5050c00640 1 Processor -- start 2024-04-24T06:33:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.362+0000 7f5050c00640 1 -- start start 2024-04-24T06:33:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f5050c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f504c0cd250 0x7f504c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f5050c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f504c073170 con 0x7f504c162d20 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f5050c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f504c073450 con 0x7f504c162940 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f5050c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f504c070060 con 0x7f504c0cd250 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f504aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f504c162d20 0x7f504c0ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51986/0 (socket says 172.21.15.59:51986) 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.364+0000 7f504aa00640 1 -- 172.21.15.59:0/1871628429 learned_addr learned my addr 172.21.15.59:0/1871628429 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504be00640 1 --2- 172.21.15.59:0/1871628429 >> v2:172.21.15.66:3300/0 conn(0x7f504c0cd250 0x7f504c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27626096 0 0) 0x7f504c073170 con 0x7f504c162d20 2024-04-24T06:33:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5030003520 con 0x7f504c162d20 2024-04-24T06:33:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 528219041 0 0) 0x7f5030003520 con 0x7f504c162d20 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 >> v2:172.21.15.66:3300/0 conn(0x7f504c0cd250 msgr2=0x7f504c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 --2- 172.21.15.59:0/1871628429 >> v2:172.21.15.66:3300/0 conn(0x7f504c0cd250 0x7f504c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 >> v1:172.21.15.66:6789/0 conn(0x7f504c162940 legacy=0x7f504c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c073730 con 0x7f504c162d20 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5038001670 con 0x7f504c162d20 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 >> v1:172.21.15.59:6789/0 conn(0x7f504c162d20 legacy=0x7f504c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030003a00 0x7f5030003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5030004800 0x7f5030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f503000c090 con 0x7f50300051c0 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f503000c240 con 0x7f5030003a00 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.365+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f503000c440 con 0x7f5030004800 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504b400640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030003a00 0x7f5030003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 -- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5030004800 msgr2=0x7f5030004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5030004800 0x7f5030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 -- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030003a00 msgr2=0x7f5030003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030003a00 0x7f5030003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.366+0000 7f504be00640 1 -- 172.21.15.59:0/1871628429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5030003520 con 0x7f50300051c0 2024-04-24T06:33:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.367+0000 7f504be00640 1 -- 172.21.15.59:0/1871628429 reap_dead start 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.367+0000 7f504be00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 0x7f503000b9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5044009a90 tx=0x7f5030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f9fd09cec260e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.367+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044009690 con 0x7f50300051c0 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.367+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5044004020 con 0x7f50300051c0 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.367+0000 7f504a000640 1 -- 172.21.15.59:0/1871628429 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044004410 con 0x7f50300051c0 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 -- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 msgr2=0x7f503000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 0x7f503000b9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5044009a90 tx=0x7f5030005830 comp rx=0 tx=0).stop 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 -- 172.21.15.59:0/1871628429 shutdown_connections 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 --2- 172.21.15.59:0/1871628429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50300051c0 0x7f503000b9c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 -- 172.21.15.59:0/1871628429 >> 172.21.15.59:0/1871628429 conn(0x7f504c0b1f90 msgr2=0x7f504c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 -- 172.21.15.59:0/1871628429 shutdown_connections 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.368+0000 7f5050c00640 1 -- 172.21.15.59:0/1871628429 wait complete. 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.370+0000 7f5050c00640 1 Processor -- start 2024-04-24T06:33:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.370+0000 7f5050c00640 1 -- start start 2024-04-24T06:33:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 0x7f504c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0b0880 0x7f504c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f504c0ce300 con 0x7f504c0b4ea0 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f504c0ce180 con 0x7f504c0b0880 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f5050c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f504c0af970 con 0x7f50300051c0 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f504aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f504aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54260/0 (socket says 172.21.15.59:54260) 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.371+0000 7f504aa00640 1 -- 172.21.15.59:0/1792486601 learned_addr learned my addr 172.21.15.59:0/1792486601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504be00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0b0880 0x7f504c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504b400640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 0x7f504c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 -- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 msgr2=0x7f504c153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 0x7f504c153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 -- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0b0880 msgr2=0x7f504c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0b0880 0x7f504c0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 -- 172.21.15.59:0/1792486601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c163470 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504b400640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 0x7f504c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.372+0000 7f504aa00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f503800c730 tx=0x7f503800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.373+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038018070 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.373+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50380072d0 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.373+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380047e0 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.373+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c163750 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.373+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c163d10 con 0x7f504c0b4ea0 2024-04-24T06:33:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.374+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c04fd90 con 0x7f504c0b4ea0 2024-04-24T06:33:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.375+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5038004980 con 0x7f504c0b4ea0 2024-04-24T06:33:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.376+0000 7f5043400640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 0x7f502c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:33.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.376+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7f503808aea0 con 0x7f504c0b4ea0 2024-04-24T06:33:33.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.376+0000 7f504b400640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 0x7f502c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.377+0000 7f504b400640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 0x7f502c06ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f503c001d40 tx=0x7f503c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.381+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5038004c70 con 0x7f504c0b4ea0 2024-04-24T06:33:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:33.585+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f504c0b1240 con 0x7f504c0b4ea0 2024-04-24T06:33:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.113+0000 7f5043400640 1 -- 172.21.15.59:0/1792486601 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f5038059390 con 0x7f504c0b4ea0 2024-04-24T06:33:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:33:34.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.117+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 msgr2=0x7f502c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.117+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 0x7f502c06ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f503c001d40 tx=0x7f503c009040 comp rx=0 tx=0).stop 2024-04-24T06:33:34.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.117+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 msgr2=0x7f504c0b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.117+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f503800c730 tx=0x7f503800cbf0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 shutdown_connections 2024-04-24T06:33:34.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f502c06c920 0x7f502c06ede0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f504c0b0880 0x7f504c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f504c0b4ea0 0x7f504c0b5320 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 --2- 172.21.15.59:0/1792486601 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50300051c0 0x7f504c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 >> 172.21.15.59:0/1792486601 conn(0x7f504c0b1f90 msgr2=0x7f504c0adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 shutdown_connections 2024-04-24T06:33:34.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.118+0000 7f5050c00640 1 -- 172.21.15.59:0/1792486601 wait complete. 2024-04-24T06:33:34.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:33:34.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.357+0000 7f9614c00640 1 Processor -- start 2024-04-24T06:33:34.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.357+0000 7f9614c00640 1 -- start start 2024-04-24T06:33:34.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.359+0000 7f9614c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f961006f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.359+0000 7f9614c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9610071d70 0x7f9610072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f9614c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96100cfc20 con 0x7f9610145e10 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f9614c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f96100cff00 con 0x7f9610071d70 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f9614c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f96100d01e0 con 0x7f96100d0cc0 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f960de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9610145e10 0x7f96100cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51988/0 (socket says 172.21.15.59:51988) 2024-04-24T06:33:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f960de00640 1 -- 172.21.15.59:0/1400699584 learned_addr learned my addr 172.21.15.59:0/1400699584 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:34.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f960d400640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f961006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f960e800640 1 --2- 172.21.15.59:0/1400699584 >> v2:172.21.15.66:3300/0 conn(0x7f9610071d70 0x7f9610072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.360+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606826420 0 0) 0x7f96100cfc20 con 0x7f9610145e10 2024-04-24T06:33:34.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95f4003520 con 0x7f9610145e10 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3004339083 0 0) 0x7f95f4003520 con 0x7f9610145e10 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 msgr2=0x7f961006f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f961006f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 >> v2:172.21.15.66:3300/0 conn(0x7f9610071d70 msgr2=0x7f9610072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 --2- 172.21.15.59:0/1400699584 >> v2:172.21.15.66:3300/0 conn(0x7f9610071d70 0x7f9610072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96100d04c0 con 0x7f9610145e10 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9600001670 con 0x7f9610145e10 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 >> v1:172.21.15.59:6789/0 conn(0x7f9610145e10 legacy=0x7f96100cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f40039b0 0x7f95f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f95f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f4004d40 0x7f95f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96100cff00 con 0x7f95f40039b0 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96100d01e0 con 0x7f95f4004d40 2024-04-24T06:33:34.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95f400b9c0 con 0x7f96100d0cc0 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.361+0000 7f960de00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f40039b0 0x7f95f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960e800640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f4004d40 0x7f95f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960d400640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f95f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960de00640 1 -- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 msgr2=0x7f95f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960de00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96100d0cc0 0x7f95f4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960de00640 1 -- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f4004d40 msgr2=0x7f95f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960de00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f4004d40 0x7f95f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.362+0000 7f960de00640 1 -- 172.21.15.59:0/1400699584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f4003520 con 0x7f95f40039b0 2024-04-24T06:33:34.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.363+0000 7f960de00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f40039b0 0x7f95f4003dd0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f960000c110 tx=0x7f95f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41597cbaaeb2c70a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.363+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600007c20 con 0x7f95f40039b0 2024-04-24T06:33:34.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.363+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9600005af0 con 0x7f95f40039b0 2024-04-24T06:33:34.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.363+0000 7f960ca00640 1 -- 172.21.15.59:0/1400699584 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600007070 con 0x7f95f40039b0 2024-04-24T06:33:34.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 -- 172.21.15.59:0/1400699584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f40039b0 msgr2=0x7f95f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 --2- 172.21.15.59:0/1400699584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f40039b0 0x7f95f4003dd0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f960000c110 tx=0x7f95f4005830 comp rx=0 tx=0).stop 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f960e800640 1 -- 172.21.15.59:0/1400699584 reap_dead start 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 -- 172.21.15.59:0/1400699584 shutdown_connections 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 -- 172.21.15.59:0/1400699584 >> 172.21.15.59:0/1400699584 conn(0x7f9610067d90 msgr2=0x7f961006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 -- 172.21.15.59:0/1400699584 shutdown_connections 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.364+0000 7f9614c00640 1 -- 172.21.15.59:0/1400699584 wait complete. 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.366+0000 7f9614c00640 1 Processor -- start 2024-04-24T06:33:34.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.366+0000 7f9614c00640 1 -- start start 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95f40039b0 0x7f96101f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9610140810 0x7f9610140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96101461f0 con 0x7f961015c330 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f961006f130 con 0x7f9610140810 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.367+0000 7f9614c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f96100c9bc0 con 0x7f95f40039b0 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54270/0 (socket says 172.21.15.59:54270) 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 -- 172.21.15.59:0/381246537 learned_addr learned my addr 172.21.15.59:0/381246537 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:34.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960e800640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9610140810 0x7f9610140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960de00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95f40039b0 0x7f96101f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 -- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95f40039b0 msgr2=0x7f96101f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95f40039b0 0x7f96101f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 -- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9610140810 msgr2=0x7f9610140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9610140810 0x7f9610140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.368+0000 7f960d400640 1 -- 172.21.15.59:0/381246537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96101f8bd0 con 0x7f961015c330 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f960d400640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f95f8010860 tx=0x7f95f8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f800f040 con 0x7f961015c330 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f800eb60 con 0x7f961015c330 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f801c400 con 0x7f961015c330 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96101f8eb0 con 0x7f961015c330 2024-04-24T06:33:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.369+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101f88a0 con 0x7f961015c330 2024-04-24T06:33:34.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.370+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96101411d0 con 0x7f961015c330 2024-04-24T06:33:34.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.371+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95f801c5a0 con 0x7f961015c330 2024-04-24T06:33:34.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.372+0000 7f9606000640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 0x7f95ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.372+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f95f808bea0 con 0x7f961015c330 2024-04-24T06:33:34.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.373+0000 7f960de00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 0x7f95ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.373+0000 7f960de00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 0x7f95ec06edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9600032000 tx=0x7f9600009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.377+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95f8018030 con 0x7f961015c330 2024-04-24T06:33:34.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.584+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9610068a80 con 0x7f961015c330 2024-04-24T06:33:34.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.592+0000 7f9606000640 1 -- 172.21.15.59:0/381246537 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f95f8059f70 con 0x7f961015c330 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 msgr2=0x7f95ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 0x7f95ec06edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9600032000 tx=0x7f9600009030 comp rx=0 tx=0).stop 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 msgr2=0x7f961013d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f95f8010860 tx=0x7f95f8010d20 comp rx=0 tx=0).stop 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 shutdown_connections 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f95ec06c8f0 0x7f95ec06edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9610140810 0x7f9610140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f961015c330 0x7f961013d3f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 --2- 172.21.15.59:0/381246537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95f40039b0 0x7f96101f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 >> 172.21.15.59:0/381246537 conn(0x7f9610067d90 msgr2=0x7f961006e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 shutdown_connections 2024-04-24T06:33:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.595+0000 7f9614c00640 1 -- 172.21.15.59:0/381246537 wait complete. 2024-04-24T06:33:34.615 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-24T06:33:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.830+0000 7f25d7e00640 1 Processor -- start 2024-04-24T06:33:34.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.831+0000 7f25d7e00640 1 -- start start 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25d815c3d0 0x7f25d80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25d80cd590 0x7f25d80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d8160210 con 0x7f25d80b4690 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f25d81604f0 con 0x7f25d80cd590 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f25d81607d0 con 0x7f25d815c3d0 2024-04-24T06:33:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25cfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25d80cd590 0x7f25d80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f25d80b4690 0x7f25d806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51992/0 (socket says 172.21.15.59:51992) 2024-04-24T06:33:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.832+0000 7f25d7400640 1 -- 172.21.15.59:0/1097530143 learned_addr learned my addr 172.21.15.59:0/1097530143 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25d6a00640 1 --2- 172.21.15.59:0/1097530143 >> v2:172.21.15.66:3300/0 conn(0x7f25d815c3d0 0x7f25d80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25d6000640 1 -- 172.21.15.59:0/1097530143 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 227422509 0 0) 0x7f25d8160210 con 0x7f25d80b4690 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25d6000640 1 -- 172.21.15.59:0/1097530143 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25bc003520 con 0x7f25d80b4690 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25cfe00640 1 -- 172.21.15.59:0/1097530143 >> v2:172.21.15.66:3300/0 conn(0x7f25d815c3d0 msgr2=0x7f25d80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25cfe00640 1 --2- 172.21.15.59:0/1097530143 >> v2:172.21.15.66:3300/0 conn(0x7f25d815c3d0 0x7f25d80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25cfe00640 1 -- 172.21.15.59:0/1097530143 >> v1:172.21.15.59:6789/0 conn(0x7f25d80b4690 legacy=0x7f25d806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.833+0000 7f25cfe00640 1 -- 172.21.15.59:0/1097530143 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d8160ab0 con 0x7f25d80cd590 2024-04-24T06:33:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.834+0000 7f25cfe00640 1 --2- 172.21.15.59:0/1097530143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d80cd590 0x7f25d80b4150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f25d806d9b0 tx=0x7f25c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579739a0b4356bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.834+0000 7f25d6000640 1 -- 172.21.15.59:0/1097530143 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0037070 con 0x7f25d80cd590 2024-04-24T06:33:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.834+0000 7f25d6000640 1 -- 172.21.15.59:0/1097530143 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c0032440 con 0x7f25d80cd590 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.834+0000 7f25d6000640 1 -- 172.21.15.59:0/1097530143 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00327f0 con 0x7f25d80cd590 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 -- 172.21.15.59:0/1097530143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d80cd590 msgr2=0x7f25d80b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 --2- 172.21.15.59:0/1097530143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d80cd590 0x7f25d80b4150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f25d806d9b0 tx=0x7f25c002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 -- 172.21.15.59:0/1097530143 shutdown_connections 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 --2- 172.21.15.59:0/1097530143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d80cd590 0x7f25d80b4150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 --2- 172.21.15.59:0/1097530143 >> v2:172.21.15.66:3300/0 conn(0x7f25d815c3d0 0x7f25d80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 -- 172.21.15.59:0/1097530143 >> 172.21.15.59:0/1097530143 conn(0x7f25d80b21a0 msgr2=0x7f25d80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 -- 172.21.15.59:0/1097530143 shutdown_connections 2024-04-24T06:33:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.835+0000 7f25d7e00640 1 -- 172.21.15.59:0/1097530143 wait complete. 2024-04-24T06:33:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.837+0000 7f25d7e00640 1 Processor -- start 2024-04-24T06:33:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.837+0000 7f25d7e00640 1 -- start start 2024-04-24T06:33:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d80b4690 0x7f25d80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25d80cd590 0x7f25d80cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25d815c7b0 con 0x7f25d815c3d0 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25d80b4ac0 con 0x7f25d80b4690 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.838+0000 7f25d7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25d815c930 con 0x7f25d80cd590 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54292/0 (socket says 172.21.15.59:54292) 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 -- 172.21.15.59:0/3884938046 learned_addr learned my addr 172.21.15.59:0/3884938046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25cfe00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25d80cd590 0x7f25d80cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d6a00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d80b4690 0x7f25d80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 -- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25d80cd590 msgr2=0x7f25d80cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25d80cd590 0x7f25d80cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 -- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d80b4690 msgr2=0x7f25d80c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d80b4690 0x7f25d80c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 -- 172.21.15.59:0/3884938046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d8159250 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.839+0000 7f25d7400640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f25d000cb40 tx=0x7f25d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.840+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0017070 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.840+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d000aca0 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.840+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0012620 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.840+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d80cdf20 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.840+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d80ce4e0 con 0x7f25d815c3d0 2024-04-24T06:33:34.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.841+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d80b11e0 con 0x7f25d815c3d0 2024-04-24T06:33:34.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.843+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25d001a070 con 0x7f25d815c3d0 2024-04-24T06:33:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.844+0000 7f25cea00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 0x7f25b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.844+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f25d008bb30 con 0x7f25d815c3d0 2024-04-24T06:33:34.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.845+0000 7f25d6a00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 0x7f25b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.845+0000 7f25d6a00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 0x7f25b406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f25c8006fd0 tx=0x7f25c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:34.848+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25d80b11e0 con 0x7f25d815c3d0 2024-04-24T06:33:35.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:35.129+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f25d80b4150 con 0x7f25d815c3d0 2024-04-24T06:33:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.127+0000 7f25cea00640 1 -- 172.21.15.59:0/3884938046 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v255) ==== 169+0+0 (secure 0 0 0) 0x7f25d80b4150 con 0x7f25d815c3d0 2024-04-24T06:33:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 msgr2=0x7f25b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 0x7f25b406edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f25c8006fd0 tx=0x7f25c8008040 comp rx=0 tx=0).stop 2024-04-24T06:33:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 msgr2=0x7f25d81595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f25d000cb40 tx=0x7f25d000a060 comp rx=0 tx=0).stop 2024-04-24T06:33:36.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 shutdown_connections 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25b406c8f0 0x7f25b406edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d815c3d0 0x7f25d81595b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25d80cd590 0x7f25d80cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 --2- 172.21.15.59:0/3884938046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25d80b4690 0x7f25d80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 >> 172.21.15.59:0/3884938046 conn(0x7f25d80b21a0 msgr2=0x7f25d80b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 shutdown_connections 2024-04-24T06:33:36.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.131+0000 7f25d7e00640 1 -- 172.21.15.59:0/3884938046 wait complete. 2024-04-24T06:33:36.155 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-24T06:33:36.155 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-24T06:33:36.155 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:33:36.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:36.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.361+0000 7fc71ca00640 1 Processor -- start 2024-04-24T06:33:36.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.361+0000 7fc71ca00640 1 -- start start 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.362+0000 7fc71ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7180cd760 0x7fc7180b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc71ca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 0x7fc7180cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc71ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc718160210 con 0x7fc71815c3d0 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc71ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc7181604f0 con 0x7fc7180cd760 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc71ca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc7181607d0 con 0x7fc7180b4690 2024-04-24T06:33:36.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7180cd760 0x7fc7180b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7180cd760 0x7fc7180b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54306/0 (socket says 172.21.15.59:54306) 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc716800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc71815c3d0 0x7fc7180cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52008/0 (socket says 172.21.15.59:52008) 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 -- 172.21.15.59:0/3598500209 learned_addr learned my addr 172.21.15.59:0/3598500209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc717200640 1 --2- 172.21.15.59:0/3598500209 >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 0x7fc7180cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 -- 172.21.15.59:0/3598500209 >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 msgr2=0x7fc7180cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 --2- 172.21.15.59:0/3598500209 >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 0x7fc7180cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 -- 172.21.15.59:0/3598500209 >> v1:172.21.15.59:6789/0 conn(0x7fc71815c3d0 legacy=0x7fc7180cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc715e00640 1 -- 172.21.15.59:0/3598500209 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc718160ab0 con 0x7fc7180cd760 2024-04-24T06:33:36.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.363+0000 7fc717200640 1 --2- 172.21.15.59:0/3598500209 >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 0x7fc7180cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.364+0000 7fc715e00640 1 --2- 172.21.15.59:0/3598500209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180cd760 0x7fc7180b4150 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc7180b2de0 tx=0x7fc70402a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734d23cd4da0406b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.365+0000 7fc715400640 1 -- 172.21.15.59:0/3598500209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037070 con 0x7fc7180cd760 2024-04-24T06:33:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.365+0000 7fc715400640 1 -- 172.21.15.59:0/3598500209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704032440 con 0x7fc7180cd760 2024-04-24T06:33:36.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.365+0000 7fc715400640 1 -- 172.21.15.59:0/3598500209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704032810 con 0x7fc7180cd760 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.365+0000 7fc71ca00640 1 -- 172.21.15.59:0/3598500209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180cd760 msgr2=0x7fc7180b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.365+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3598500209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180cd760 0x7fc7180b4150 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc7180b2de0 tx=0x7fc70402a280 comp rx=0 tx=0).stop 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 -- 172.21.15.59:0/3598500209 shutdown_connections 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3598500209 >> v2:172.21.15.66:3300/0 conn(0x7fc7180b4690 0x7fc7180cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3598500209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180cd760 0x7fc7180b4150 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 -- 172.21.15.59:0/3598500209 >> 172.21.15.59:0/3598500209 conn(0x7fc7180b21a0 msgr2=0x7fc7180b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 -- 172.21.15.59:0/3598500209 shutdown_connections 2024-04-24T06:33:36.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.366+0000 7fc71ca00640 1 -- 172.21.15.59:0/3598500209 wait complete. 2024-04-24T06:33:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.367+0000 7fc71ca00640 1 Processor -- start 2024-04-24T06:33:36.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 -- start start 2024-04-24T06:33:36.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7180cd760 0x7fc7181f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71815c3d0 0x7fc7180c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc71815c7b0 con 0x7fc7180b4690 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180b4ae0 con 0x7fc71815c3d0 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.368+0000 7fc71ca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc71815c930 con 0x7fc7180cd760 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54320/0 (socket says 172.21.15.59:54320) 2024-04-24T06:33:36.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 -- 172.21.15.59:0/3903949441 learned_addr learned my addr 172.21.15.59:0/3903949441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc717200640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71815c3d0 0x7fc7180c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc715e00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7180cd760 0x7fc7181f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 -- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7180cd760 msgr2=0x7fc7181f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7180cd760 0x7fc7181f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 -- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71815c3d0 msgr2=0x7fc7180c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71815c3d0 0x7fc7180c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.369+0000 7fc716800640 1 -- 172.21.15.59:0/3903949441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7180ae100 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc716800640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c3c0 tx=0x7fc70800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080074e0 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc708007680 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708004a00 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7180ae3e0 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.370+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7180ae9a0 con 0x7fc7180b4690 2024-04-24T06:33:36.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.371+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71804fd90 con 0x7fc7180b4690 2024-04-24T06:33:36.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.372+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc70801a020 con 0x7fc7180b4690 2024-04-24T06:33:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.373+0000 7fc70ea00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 0x7fc6f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.373+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc70808afe0 con 0x7fc7180b4690 2024-04-24T06:33:36.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.374+0000 7fc715e00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 0x7fc6f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.374+0000 7fc715e00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 0x7fc6f806edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc704034000 tx=0x7fc70402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.378+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc708059740 con 0x7fc7180b4690 2024-04-24T06:33:36.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:33:36.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:33:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.586+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc71806e400 con 0x7fc7180b4690 2024-04-24T06:33:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.588+0000 7fc70ea00640 1 -- 172.21.15.59:0/3903949441 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+12936 (secure 0 0 0) 0x7fc708059040 con 0x7fc7180b4690 2024-04-24T06:33:36.590 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:36.590 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:34.102249+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2525582962":"2024-04-25T06:33:31.079192+0000","172.21.15.59:6835/2525582962":"2024-04-25T06:33:31.079192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.590+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 msgr2=0x7fc6f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.590+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 0x7fc6f806edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc704034000 tx=0x7fc70402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.590+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 msgr2=0x7fc7180ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.590+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c3c0 tx=0x7fc70800c880 comp rx=0 tx=0).stop 2024-04-24T06:33:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 shutdown_connections 2024-04-24T06:33:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6f806c8f0 0x7fc6f806edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71815c3d0 0x7fc7180c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7180cd760 0x7fc7181f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 --2- 172.21.15.59:0/3903949441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7180b4690 0x7fc7180ad9d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 >> 172.21.15.59:0/3903949441 conn(0x7fc7180b21a0 msgr2=0x7fc7180d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 shutdown_connections 2024-04-24T06:33:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.591+0000 7fc71ca00640 1 -- 172.21.15.59:0/3903949441 wait complete. 2024-04-24T06:33:36.612 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:36.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.827+0000 7fc9af200640 1 Processor -- start 2024-04-24T06:33:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.828+0000 7fc9af200640 1 -- start start 2024-04-24T06:33:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.829+0000 7fc9af200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc9a815ed20 0x7fc9a80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.829+0000 7fc9af200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc9a80b3600 0x7fc9a80d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.829+0000 7fc9af200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a80b5da0 con 0x7fc9a80b3f10 2024-04-24T06:33:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.829+0000 7fc9af200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a815f100 con 0x7fc9a80b3600 2024-04-24T06:33:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.829+0000 7fc9af200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a80d0a20 con 0x7fc9a815ed20 2024-04-24T06:33:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ad400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc9a80b3600 0x7fc9a80d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ae800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc9a80b3f10 0x7fc9a80b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52012/0 (socket says 172.21.15.59:52012) 2024-04-24T06:33:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ae800640 1 -- 172.21.15.59:0/3062495017 learned_addr learned my addr 172.21.15.59:0/3062495017 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ade00640 1 --2- 172.21.15.59:0/3062495017 >> v2:172.21.15.66:3300/0 conn(0x7fc9a815ed20 0x7fc9a80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ad400640 1 -- 172.21.15.59:0/3062495017 >> v2:172.21.15.66:3300/0 conn(0x7fc9a815ed20 msgr2=0x7fc9a80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ad400640 1 --2- 172.21.15.59:0/3062495017 >> v2:172.21.15.66:3300/0 conn(0x7fc9a815ed20 0x7fc9a80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ad400640 1 -- 172.21.15.59:0/3062495017 >> v1:172.21.15.59:6789/0 conn(0x7fc9a80b3f10 legacy=0x7fc9a80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9ad400640 1 -- 172.21.15.59:0/3062495017 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a80d2ed0 con 0x7fc9a80b3600 2024-04-24T06:33:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.830+0000 7fc9aca00640 1 -- 172.21.15.59:0/3062495017 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001813908 0 0) 0x7fc9a80b5da0 con 0x7fc9a80b3f10 2024-04-24T06:33:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.832+0000 7fc9ad400640 1 --2- 172.21.15.59:0/3062495017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80b3600 0x7fc9a80d13e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc9a808e2a0 tx=0x7fc99802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a26012941955171 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.832+0000 7fc9aca00640 1 -- 172.21.15.59:0/3062495017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998007740 con 0x7fc9a80b3600 2024-04-24T06:33:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.832+0000 7fc9aca00640 1 -- 172.21.15.59:0/3062495017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9980078e0 con 0x7fc9a80b3600 2024-04-24T06:33:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9aca00640 1 -- 172.21.15.59:0/3062495017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998007cb0 con 0x7fc9a80b3600 2024-04-24T06:33:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 -- 172.21.15.59:0/3062495017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80b3600 msgr2=0x7fc9a80d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 --2- 172.21.15.59:0/3062495017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80b3600 0x7fc9a80d13e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc9a808e2a0 tx=0x7fc99802a560 comp rx=0 tx=0).stop 2024-04-24T06:33:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 -- 172.21.15.59:0/3062495017 shutdown_connections 2024-04-24T06:33:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 --2- 172.21.15.59:0/3062495017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80b3600 0x7fc9a80d13e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 --2- 172.21.15.59:0/3062495017 >> v2:172.21.15.66:3300/0 conn(0x7fc9a815ed20 0x7fc9a80d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 -- 172.21.15.59:0/3062495017 >> 172.21.15.59:0/3062495017 conn(0x7fc9a808bf90 msgr2=0x7fc9a80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 -- 172.21.15.59:0/3062495017 shutdown_connections 2024-04-24T06:33:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.833+0000 7fc9af200640 1 -- 172.21.15.59:0/3062495017 wait complete. 2024-04-24T06:33:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.836+0000 7fc9af200640 1 Processor -- start 2024-04-24T06:33:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.836+0000 7fc9af200640 1 -- start start 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9a80b3600 0x7fc9a81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9a80b3f10 0x7fc9a81e5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a80d0290 con 0x7fc9a815ed20 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a80d3490 con 0x7fc9a80b3f10 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9af200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a80d0410 con 0x7fc9a80b3600 2024-04-24T06:33:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9ae800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9ae800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54332/0 (socket says 172.21.15.59:54332) 2024-04-24T06:33:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.837+0000 7fc9ae800640 1 -- 172.21.15.59:0/3809964127 learned_addr learned my addr 172.21.15.59:0/3809964127 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ade00640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9a80b3600 0x7fc9a81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 -- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9a80b3600 msgr2=0x7fc9a81541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ad400640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9a80b3f10 0x7fc9a81e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9a80b3600 0x7fc9a81541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 -- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9a80b3f10 msgr2=0x7fc9a81e5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9a80b3f10 0x7fc9a81e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 -- 172.21.15.59:0/3809964127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a80cc5a0 con 0x7fc9a815ed20 2024-04-24T06:33:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9ae800640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c450 tx=0x7fc9a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4018070 con 0x7fc9a815ed20 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4007500 con 0x7fc9a815ed20 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a80cc880 con 0x7fc9a815ed20 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.838+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4004a20 con 0x7fc9a815ed20 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.839+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a80cce40 con 0x7fc9a815ed20 2024-04-24T06:33:36.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.840+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a806e490 con 0x7fc9a815ed20 2024-04-24T06:33:36.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.841+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9a401b070 con 0x7fc9a815ed20 2024-04-24T06:33:36.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.842+0000 7fc9a2000640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 0x7fc98c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:36.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.842+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc9a408aba0 con 0x7fc9a815ed20 2024-04-24T06:33:36.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.842+0000 7fc9ade00640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 0x7fc98c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:36.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.843+0000 7fc9ade00640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 0x7fc98c06ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:36.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:36.847+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9a806e490 con 0x7fc9a815ed20 2024-04-24T06:33:37.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.052+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9a80d5430 con 0x7fc9a815ed20 2024-04-24T06:33:37.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.053+0000 7fc9a2000640 1 -- 172.21.15.59:0/3809964127 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7fc9a40592f0 con 0x7fc9a815ed20 2024-04-24T06:33:37.055 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 msgr2=0x7fc98c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 0x7fc98c06ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c008040 comp rx=0 tx=0).stop 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 msgr2=0x7fc9a81e57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c450 tx=0x7fc9a400c910 comp rx=0 tx=0).stop 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 shutdown_connections 2024-04-24T06:33:37.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc98c06c920 0x7fc98c06ede0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a815ed20 0x7fc9a81e57d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9a80b3f10 0x7fc9a81e5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 --2- 172.21.15.59:0/3809964127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc9a80b3600 0x7fc9a81541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 >> 172.21.15.59:0/3809964127 conn(0x7fc9a808bf90 msgr2=0x7fc9a8155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.055+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 shutdown_connections 2024-04-24T06:33:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.056+0000 7fc9af200640 1 -- 172.21.15.59:0/3809964127 wait complete. 2024-04-24T06:33:37.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-24T06:33:37.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.290+0000 7f07c4a00640 1 Processor -- start 2024-04-24T06:33:37.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.290+0000 7f07c4a00640 1 -- start start 2024-04-24T06:33:37.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07c4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07c00cd760 0x7f07c00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07c4a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f07c00b4690 0x7f07c00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07c4a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c0160210 con 0x7f07c015c3d0 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07c4a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f07c01604f0 con 0x7f07c00cd760 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07c4a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f07c01607d0 con 0x7f07c00b4690 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07bde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07c00cd760 0x7f07c00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07bde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07c00cd760 0x7f07c00b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54348/0 (socket says 172.21.15.59:54348) 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.292+0000 7f07bde00640 1 -- 172.21.15.59:0/291340676 learned_addr learned my addr 172.21.15.59:0/291340676 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.293+0000 7f07bde00640 1 -- 172.21.15.59:0/291340676 >> v2:172.21.15.66:3300/0 conn(0x7f07c00b4690 msgr2=0x7f07c00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.293+0000 7f07bde00640 1 --2- 172.21.15.59:0/291340676 >> v2:172.21.15.66:3300/0 conn(0x7f07c00b4690 0x7f07c00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.293+0000 7f07bde00640 1 -- 172.21.15.59:0/291340676 >> v1:172.21.15.66:6789/0 conn(0x7f07c015c3d0 legacy=0x7f07c00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:37.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.293+0000 7f07bde00640 1 -- 172.21.15.59:0/291340676 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c0160ab0 con 0x7f07c00cd760 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.294+0000 7f07bde00640 1 --2- 172.21.15.59:0/291340676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00cd760 0x7f07c00b4150 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f07c00b2de0 tx=0x7f07b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780f2a09f6e70849 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.294+0000 7f07bd400640 1 -- 172.21.15.59:0/291340676 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0036070 con 0x7f07c00cd760 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.294+0000 7f07bd400640 1 -- 172.21.15.59:0/291340676 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b0031600 con 0x7f07c00cd760 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.294+0000 7f07bd400640 1 -- 172.21.15.59:0/291340676 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00319d0 con 0x7f07c00cd760 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 -- 172.21.15.59:0/291340676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00cd760 msgr2=0x7f07c00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 --2- 172.21.15.59:0/291340676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00cd760 0x7f07c00b4150 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f07c00b2de0 tx=0x7f07b002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 -- 172.21.15.59:0/291340676 shutdown_connections 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 --2- 172.21.15.59:0/291340676 >> v2:172.21.15.66:3300/0 conn(0x7f07c00b4690 0x7f07c00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 --2- 172.21.15.59:0/291340676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00cd760 0x7f07c00b4150 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 -- 172.21.15.59:0/291340676 >> 172.21.15.59:0/291340676 conn(0x7f07c00b21a0 msgr2=0x7f07c00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 -- 172.21.15.59:0/291340676 shutdown_connections 2024-04-24T06:33:37.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.295+0000 7f07c4a00640 1 -- 172.21.15.59:0/291340676 wait complete. 2024-04-24T06:33:37.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.297+0000 7f07c4a00640 1 Processor -- start 2024-04-24T06:33:37.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.297+0000 7f07c4a00640 1 -- start start 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07c00cd760 0x7f07c0072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 0x7f07c0073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07c015c7b0 con 0x7f07c00b4690 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00b4ae0 con 0x7f07c015c3d0 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07c4a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f07c015c930 con 0x7f07c00cd760 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07be800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07be800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54356/0 (socket says 172.21.15.59:54356) 2024-04-24T06:33:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.298+0000 7f07be800640 1 -- 172.21.15.59:0/3295591977 learned_addr learned my addr 172.21.15.59:0/3295591977 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07bde00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07c00cd760 0x7f07c0072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07bf200640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 0x7f07c0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 -- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07c00cd760 msgr2=0x7f07c0072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07c00cd760 0x7f07c0072f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 -- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 msgr2=0x7f07c0073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 0x7f07c0073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 -- 172.21.15.59:0/3295591977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00c7320 con 0x7f07c00b4690 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07bf200640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 0x7f07c0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07be800640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b410 tx=0x7f07a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8015020 con 0x7f07c00b4690 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8011830 con 0x7f07c00b4690 2024-04-24T06:33:37.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.299+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011c20 con 0x7f07c00b4690 2024-04-24T06:33:37.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.300+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c00c7590 con 0x7f07c00b4690 2024-04-24T06:33:37.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.300+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01f59a0 con 0x7f07c00b4690 2024-04-24T06:33:37.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.301+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c006fda0 con 0x7f07c00b4690 2024-04-24T06:33:37.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.302+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07a8018020 con 0x7f07c00b4690 2024-04-24T06:33:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.303+0000 7f07b6a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 0x7f079c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.303+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f07a808b8a0 con 0x7f07c00b4690 2024-04-24T06:33:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.303+0000 7f07bde00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 0x7f079c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:37.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.304+0000 7f07bde00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 0x7f079c06edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f07b0033000 tx=0x7f07b0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:37.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.308+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07c006fda0 con 0x7f07c00b4690 2024-04-24T06:33:37.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:37.590+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f07c00b4150 con 0x7f07c00b4690 2024-04-24T06:33:38.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.140+0000 7f07b6a00640 1 -- 172.21.15.59:0/3295591977 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v257) ==== 169+0+0 (secure 0 0 0) 0x7f07c00b4150 con 0x7f07c00b4690 2024-04-24T06:33:38.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 msgr2=0x7f079c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 0x7f079c06edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f07b0033000 tx=0x7f07b0009000 comp rx=0 tx=0).stop 2024-04-24T06:33:38.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 msgr2=0x7f07c0159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b410 tx=0x7f07a800b8d0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 shutdown_connections 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f079c06c8f0 0x7f079c06edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07c015c3d0 0x7f07c0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f07c00cd760 0x7f07c0072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 --2- 172.21.15.59:0/3295591977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07c00b4690 0x7f07c0159310 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 >> 172.21.15.59:0/3295591977 conn(0x7f07c00b21a0 msgr2=0x7f07c006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 shutdown_connections 2024-04-24T06:33:38.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.145+0000 7f07c4a00640 1 -- 172.21.15.59:0/3295591977 wait complete. 2024-04-24T06:33:38.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:33:38.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.375+0000 7f1ca6800640 1 Processor -- start 2024-04-24T06:33:38.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.375+0000 7f1ca6800640 1 -- start start 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 0x7f1ca0160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca6800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca015cab0 con 0x7f1ca00d0d10 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca6800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca0160e40 con 0x7f1ca01567e0 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca015cd30 con 0x7f1ca015d810 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca4a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1ca00d0d10 0x7f1ca0160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52018/0 (socket says 172.21.15.59:52018) 2024-04-24T06:33:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca4a00640 1 -- 172.21.15.59:0/1290567227 learned_addr learned my addr 172.21.15.59:0/1290567227 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.377+0000 7f1ca5e00640 1 --2- 172.21.15.59:0/1290567227 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 0x7f1ca0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243594619 0 0) 0x7f1c8c001110 con 0x7f1ca00d0d10 2024-04-24T06:33:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c88003520 con 0x7f1ca00d0d10 2024-04-24T06:33:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368405634 0 0) 0x7f1ca0160e40 con 0x7f1ca01567e0 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c8c001110 con 0x7f1ca01567e0 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2112388629 0 0) 0x7f1c88003520 con 0x7f1ca00d0d10 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1ca5e00640 1 -- 172.21.15.59:0/1290567227 >> v1:172.21.15.66:6789/0 conn(0x7f1ca01567e0 legacy=0x7f1ca00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1ca5e00640 1 -- 172.21.15.59:0/1290567227 >> v1:172.21.15.59:6789/0 conn(0x7f1ca00d0d10 legacy=0x7f1ca0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.378+0000 7f1ca5e00640 1 -- 172.21.15.59:0/1290567227 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca015d010 con 0x7f1ca015d810 2024-04-24T06:33:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.380+0000 7f1ca5e00640 1 --2- 172.21.15.59:0/1290567227 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 0x7f1ca0160820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009dc0 tx=0x7f1c9802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2dc9f67395b2124 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:38.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.380+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c001840 con 0x7f1ca015d810 2024-04-24T06:33:38.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.380+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98007540 con 0x7f1ca015d810 2024-04-24T06:33:38.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.380+0000 7f1c9fe00640 1 -- 172.21.15.59:0/1290567227 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98007910 con 0x7f1ca015d810 2024-04-24T06:33:38.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 -- 172.21.15.59:0/1290567227 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 msgr2=0x7f1ca0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 --2- 172.21.15.59:0/1290567227 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 0x7f1ca0160820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009dc0 tx=0x7f1c9802a660 comp rx=0 tx=0).stop 2024-04-24T06:33:38.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 -- 172.21.15.59:0/1290567227 shutdown_connections 2024-04-24T06:33:38.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 --2- 172.21.15.59:0/1290567227 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca015d810 0x7f1ca0160820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 -- 172.21.15.59:0/1290567227 >> 172.21.15.59:0/1290567227 conn(0x7f1ca006bdf0 msgr2=0x7f1ca006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:38.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 -- 172.21.15.59:0/1290567227 shutdown_connections 2024-04-24T06:33:38.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.381+0000 7f1ca6800640 1 -- 172.21.15.59:0/1290567227 wait complete. 2024-04-24T06:33:38.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.383+0000 7f1ca6800640 1 Processor -- start 2024-04-24T06:33:38.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.383+0000 7f1ca6800640 1 -- start start 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca00d0d10 0x7f1ca01f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca01567e0 0x7f1ca014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0156d40 con 0x7f1ca015d810 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0156bc0 con 0x7f1ca01567e0 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca0156ec0 con 0x7f1ca00d0d10 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.384+0000 7f1ca5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:38.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54364/0 (socket says 172.21.15.59:54364) 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 -- 172.21.15.59:0/3789734328 learned_addr learned my addr 172.21.15.59:0/3789734328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5400640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca00d0d10 0x7f1ca01f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca4a00640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca01567e0 0x7f1ca014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 -- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca00d0d10 msgr2=0x7f1ca01f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca00d0d10 0x7f1ca01f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 -- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca01567e0 msgr2=0x7f1ca014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca01567e0 0x7f1ca014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 -- 172.21.15.59:0/3789734328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca006e090 con 0x7f1ca015d810 2024-04-24T06:33:38.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.385+0000 7f1ca5e00640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1c9802ac50 tx=0x7f1c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.386+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98040070 con 0x7f1ca015d810 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.386+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98003b90 con 0x7f1ca015d810 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.386+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98007e90 con 0x7f1ca015d810 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.386+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca006e370 con 0x7f1ca015d810 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.386+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca006e930 con 0x7f1ca015d810 2024-04-24T06:33:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.387+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca004fd90 con 0x7f1ca015d810 2024-04-24T06:33:38.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.388+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c98039040 con 0x7f1ca015d810 2024-04-24T06:33:38.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.389+0000 7f1c9d600640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 0x7f1c8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:38.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.389+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f1c980acbd0 con 0x7f1ca015d810 2024-04-24T06:33:38.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.389+0000 7f1ca5400640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 0x7f1c8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:38.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.390+0000 7f1ca5400640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 0x7f1c8006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c940015b0 tx=0x7f1c94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.393+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c9807b2f0 con 0x7f1ca015d810 2024-04-24T06:33:38.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.597+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1ca00b5150 con 0x7f1ca015d810 2024-04-24T06:33:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.604+0000 7f1c9d600640 1 -- 172.21.15.59:0/3789734328 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f1c9807abf0 con 0x7f1ca015d810 2024-04-24T06:33:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 msgr2=0x7f1c8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 0x7f1c8006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c940015b0 tx=0x7f1c94009040 comp rx=0 tx=0).stop 2024-04-24T06:33:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 msgr2=0x7f1ca0155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1c9802ac50 tx=0x7f1c98004a40 comp rx=0 tx=0).stop 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 shutdown_connections 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1c8006c8f0 0x7f1c8006edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca015d810 0x7f1ca0155240 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca01567e0 0x7f1ca014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 --2- 172.21.15.59:0/3789734328 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ca00d0d10 0x7f1ca01f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 >> 172.21.15.59:0/3789734328 conn(0x7f1ca006bdf0 msgr2=0x7f1ca015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 shutdown_connections 2024-04-24T06:33:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:38.607+0000 7f1ca6800640 1 -- 172.21.15.59:0/3789734328 wait complete. 2024-04-24T06:33:38.629 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:33:38.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:33:38.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:33:38.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:33:38.869 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:38.869 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:38.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:33:38.930 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:33:38.930 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:38.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:33:38.992 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:38.992 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:39.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:33:39.054 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:33:39.054 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:39.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:33:39.115 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:33:39.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:39.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.333+0000 7fcadaa00640 1 Processor -- start 2024-04-24T06:33:39.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.333+0000 7fcadaa00640 1 -- start start 2024-04-24T06:33:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.335+0000 7fcadaa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 0x7fcad40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.335+0000 7fcadaa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcad40d0d90 0x7fcad40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.335+0000 7fcadaa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad40cf030 con 0x7fcad40d2630 2024-04-24T06:33:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.335+0000 7fcadaa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcad40b7000 con 0x7fcad40d0d90 2024-04-24T06:33:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.335+0000 7fcadaa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40cf620 con 0x7fcad415e7c0 2024-04-24T06:33:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.336+0000 7fcad9600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 0x7fcad40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.336+0000 7fcad9600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 0x7fcad40d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:53432/0 (socket says 172.21.15.59:53432) 2024-04-24T06:33:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.336+0000 7fcad9600640 1 -- 172.21.15.59:0/137388082 learned_addr learned my addr 172.21.15.59:0/137388082 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:39.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.336+0000 7fcada000640 1 --2- 172.21.15.59:0/137388082 >> v2:172.21.15.66:3300/0 conn(0x7fcad40d0d90 0x7fcad40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.337+0000 7fcada000640 1 -- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 msgr2=0x7fcad40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.337+0000 7fcada000640 1 --2- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 0x7fcad40d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.337+0000 7fcada000640 1 -- 172.21.15.59:0/137388082 >> v1:172.21.15.66:6789/0 conn(0x7fcad40d2630 legacy=0x7fcad40b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.337+0000 7fcada000640 1 -- 172.21.15.59:0/137388082 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad40b6a40 con 0x7fcad40d0d90 2024-04-24T06:33:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.338+0000 7fcada000640 1 --2- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40d1220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009d90 tx=0x7fcacc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dd7823d27fdee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcad3e00640 1 -- 172.21.15.59:0/137388082 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007740 con 0x7fcad40d0d90 2024-04-24T06:33:39.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcad3e00640 1 -- 172.21.15.59:0/137388082 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc0078e0 con 0x7fcad40d0d90 2024-04-24T06:33:39.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcad3e00640 1 -- 172.21.15.59:0/137388082 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007cb0 con 0x7fcad40d0d90 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 -- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 msgr2=0x7fcad40d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 --2- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40d1220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009d90 tx=0x7fcacc02a660 comp rx=0 tx=0).stop 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 -- 172.21.15.59:0/137388082 shutdown_connections 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 --2- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40d1220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 --2- 172.21.15.59:0/137388082 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad415e7c0 0x7fcad40d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.339+0000 7fcadaa00640 1 -- 172.21.15.59:0/137388082 >> 172.21.15.59:0/137388082 conn(0x7fcad409d920 msgr2=0x7fcad408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.340+0000 7fcadaa00640 1 -- 172.21.15.59:0/137388082 shutdown_connections 2024-04-24T06:33:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.340+0000 7fcadaa00640 1 -- 172.21.15.59:0/137388082 wait complete. 2024-04-24T06:33:39.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.341+0000 7fcadaa00640 1 Processor -- start 2024-04-24T06:33:39.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.342+0000 7fcadaa00640 1 -- start start 2024-04-24T06:33:39.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.342+0000 7fcadaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.342+0000 7fcadaa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad40d2630 0x7fcad40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcadaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcadaa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40b6700 con 0x7fcad415e7c0 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcadaa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40cf2b0 con 0x7fcad40d0d90 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcadaa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40d1a80 con 0x7fcad40d2630 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46628/0 (socket says 172.21.15.59:46628) 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 -- 172.21.15.59:0/3122778387 learned_addr learned my addr 172.21.15.59:0/3122778387 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcad9600640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcad8c00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad40d2630 0x7fcad40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 -- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad40d2630 msgr2=0x7fcad40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad40d2630 0x7fcad40cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 -- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 msgr2=0x7fcad40b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.343+0000 7fcada000640 1 -- 172.21.15.59:0/3122778387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad40cd090 con 0x7fcad415e7c0 2024-04-24T06:33:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcad9600640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:33:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcada000640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009eb0 tx=0x7fcacc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc040070 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc031bc0 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc009450 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.344+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad40cd370 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.345+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad40cd930 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.346+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad404fd90 con 0x7fcad415e7c0 2024-04-24T06:33:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.347+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcacc039040 con 0x7fcad415e7c0 2024-04-24T06:33:39.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.348+0000 7fcad1600640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 0x7fcab406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.348+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fcacc0ad220 con 0x7fcad415e7c0 2024-04-24T06:33:39.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.348+0000 7fcad9600640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 0x7fcab406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.349+0000 7fcad9600640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 0x7fcab406ede0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009b80 tx=0x7fcac8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.352+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcacc043360 con 0x7fcad415e7c0 2024-04-24T06:33:39.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.600+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcad40cb8a0 con 0x7fcad415e7c0 2024-04-24T06:33:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.601+0000 7fcad1600640 1 -- 172.21.15.59:0/3122778387 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3866 (secure 0 0 0) 0x7fcacc07b820 con 0x7fcad415e7c0 2024-04-24T06:33:39.603 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:39.603 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3038339888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3038339888},{"type":"v1","addr":"172.21.15.59:6836","nonce":3038339888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/444867172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":444867172},{"type":"v1","addr":"172.21.15.66:6833","nonce":444867172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3216966671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3216966671},{"type":"v1","addr":"172.21.15.66:6835","nonce":3216966671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6675,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1472071580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1472071580},{"type":"v1","addr":"172.21.15.59:6837","nonce":1472071580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-24T06:33:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 37 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 msgr2=0x7fcab406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 0x7fcab406ede0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009b80 tx=0x7fcac8008040 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 msgr2=0x7fcad40cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009eb0 tx=0x7fcacc004a40 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 shutdown_connections 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcab406c920 0x7fcab406ede0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad415e7c0 0x7fcad40cfe40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcad40d2630 0x7fcad40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 --2- 172.21.15.59:0/3122778387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcad40d0d90 0x7fcad40b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 >> 172.21.15.59:0/3122778387 conn(0x7fcad409d920 msgr2=0x7fcad415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.604+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 shutdown_connections 2024-04-24T06:33:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.605+0000 7fcadaa00640 1 -- 172.21.15.59:0/3122778387 wait complete. 2024-04-24T06:33:39.625 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:33:39.625 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:33:39.625 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:33:39.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:33:39.626 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:33:39.626 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:33:39.626 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:33:39.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:33:39.626 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:33:39.626 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:33:39.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:33:39.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:33:39.627 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:33:39.627 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:33:39.627 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:33:39.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:33:39.642 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:33:39.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:33:39.650 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:33:39.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:33:39.653 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:33:39.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:33:39.653 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:33:39.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:33:39.694 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:33:39.695 DEBUG:teuthology.parallel:result is None 2024-04-24T06:33:39.698 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:33:39.698 DEBUG:teuthology.parallel:result is None 2024-04-24T06:33:39.701 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:33:39.701 DEBUG:teuthology.parallel:result is None 2024-04-24T06:33:39.707 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:33:39.707 DEBUG:teuthology.parallel:result is None 2024-04-24T06:33:39.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:33:39.843 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:33:39.846 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:33:39.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-24T06:33:39.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-24T06:33:39.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-24T06:33:39.862 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:33:39.865 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:33:39.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.892+0000 7f0b63e00640 1 Processor -- start 2024-04-24T06:33:39.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.892+0000 7f0b63e00640 1 -- start start 2024-04-24T06:33:39.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b63e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b5c15ea90 0x7f0b5c15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c0b26d0 0x7f0b5c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b63e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b5c15f490 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b63e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5c1600a0 con 0x7f0b5c15ea90 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c15f770 con 0x7f0b5c0b26d0 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b61400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b5c15ea90 0x7f0b5c15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b61400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b5c15ea90 0x7f0b5c15ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46680/0 (socket says 172.21.15.59:46680) 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b61400640 1 -- 172.21.15.59:0/3756951479 learned_addr learned my addr 172.21.15.59:0/3756951479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.893+0000 7f0b60a00640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c0b26d0 0x7f0b5c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.894+0000 7f0b61400640 1 -- 172.21.15.59:0/3756951479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c0b26d0 msgr2=0x7f0b5c0b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.894+0000 7f0b61400640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c0b26d0 0x7f0b5c0b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.894+0000 7f0b61400640 1 -- 172.21.15.59:0/3756951479 >> v1:172.21.15.66:6789/0 conn(0x7f0b5c06b9d0 legacy=0x7f0b5c162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.894+0000 7f0b61400640 1 -- 172.21.15.59:0/3756951479 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c15fa50 con 0x7f0b5c15ea90 2024-04-24T06:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.895+0000 7f0b61400640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c15ea90 0x7f0b5c15ee70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009870 tx=0x7f0b5002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b940c440b128c901 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.895+0000 7f0b5be00640 1 -- 172.21.15.59:0/3756951479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50037070 con 0x7f0b5c15ea90 2024-04-24T06:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.895+0000 7f0b5be00640 1 -- 172.21.15.59:0/3756951479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b50032440 con 0x7f0b5c15ea90 2024-04-24T06:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.895+0000 7f0b5be00640 1 -- 172.21.15.59:0/3756951479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50032810 con 0x7f0b5c15ea90 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 -- 172.21.15.59:0/3756951479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c15ea90 msgr2=0x7f0b5c15ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c15ea90 0x7f0b5c15ee70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009870 tx=0x7f0b5002a250 comp rx=0 tx=0).stop 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 -- 172.21.15.59:0/3756951479 shutdown_connections 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c0b26d0 0x7f0b5c0b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 --2- 172.21.15.59:0/3756951479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c15ea90 0x7f0b5c15ee70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 -- 172.21.15.59:0/3756951479 >> 172.21.15.59:0/3756951479 conn(0x7f0b5c09aa00 msgr2=0x7f0b5c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 -- 172.21.15.59:0/3756951479 shutdown_connections 2024-04-24T06:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.896+0000 7f0b63e00640 1 -- 172.21.15.59:0/3756951479 wait complete. 2024-04-24T06:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.898+0000 7f0b63e00640 1 Processor -- start 2024-04-24T06:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.898+0000 7f0b63e00640 1 -- start start 2024-04-24T06:33:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b5c0b26d0 0x7f0b5c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c15ea90 0x7f0b5c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0cce00 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0ccc80 con 0x7f0b5c0b26d0 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c0ccf80 con 0x7f0b5c15ea90 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b61400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b61400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46694/0 (socket says 172.21.15.59:46694) 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b61400640 1 -- 172.21.15.59:0/1585874427 learned_addr learned my addr 172.21.15.59:0/1585874427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b60a00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b5c0b26d0 0x7f0b5c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.899+0000 7f0b61e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c15ea90 0x7f0b5c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 -- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c15ea90 msgr2=0x7f0b5c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c15ea90 0x7f0b5c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 -- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b5c0b26d0 msgr2=0x7f0b5c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b5c0b26d0 0x7f0b5c155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 -- 172.21.15.59:0/1585874427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c0af260 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b61400640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000c00 tx=0x7f0b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.900+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50037040 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.901+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b50042070 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.901+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50030880 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.901+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c0b15c0 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.901+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c0b1b80 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.902+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c068a80 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.903+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b50030a20 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.904+0000 7f0b59600640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 0x7f0b3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.904+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f0b500ac070 con 0x7f0b5c06b9d0 2024-04-24T06:33:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.904+0000 7f0b60a00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 0x7f0b3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.905+0000 7f0b60a00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 0x7f0b3c06ede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0c90e0 tx=0x7f0b48009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:39.909+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b5007a020 con 0x7f0b5c06b9d0 2024-04-24T06:33:40.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.131+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0b5c09af10 con 0x7f0b5c06b9d0 2024-04-24T06:33:40.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.165+0000 7f0b59600640 1 -- 172.21.15.59:0/1585874427 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f0b5007e320 con 0x7f0b5c06b9d0 2024-04-24T06:33:40.167 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:33:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 msgr2=0x7f0b3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 0x7f0b3c06ede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0c90e0 tx=0x7f0b48009190 comp rx=0 tx=0).stop 2024-04-24T06:33:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 msgr2=0x7f0b5c0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000c00 tx=0x7f0b500046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 shutdown_connections 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b3c06c920 0x7f0b3c06ede0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0b5c15ea90 0x7f0b5c0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b5c0b26d0 0x7f0b5c155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 --2- 172.21.15.59:0/1585874427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b5c06b9d0 0x7f0b5c0aed20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 >> 172.21.15.59:0/1585874427 conn(0x7f0b5c09aa00 msgr2=0x7f0b5c0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 shutdown_connections 2024-04-24T06:33:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.168+0000 7f0b63e00640 1 -- 172.21.15.59:0/1585874427 wait complete. 2024-04-24T06:33:40.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:33:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.395+0000 7fd75a400640 1 Processor -- start 2024-04-24T06:33:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.395+0000 7fd75a400640 1 -- start start 2024-04-24T06:33:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.396+0000 7fd75a400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd75415e7c0 0x7fd7540d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.396+0000 7fd75a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7540d2630 0x7fd7540b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.396+0000 7fd75a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7540cf030 con 0x7fd7540b6530 2024-04-24T06:33:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.396+0000 7fd75a400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd7540b7000 con 0x7fd75415e7c0 2024-04-24T06:33:40.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.396+0000 7fd75a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7540cf620 con 0x7fd7540d2630 2024-04-24T06:33:40.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd75415e7c0 0x7fd7540d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd75415e7c0 0x7fd7540d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46708/0 (socket says 172.21.15.59:46708) 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 -- 172.21.15.59:0/918335496 learned_addr learned my addr 172.21.15.59:0/918335496 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd753e00640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7540d2630 0x7fd7540b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 -- 172.21.15.59:0/918335496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7540d2630 msgr2=0x7fd7540b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7540d2630 0x7fd7540b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.397+0000 7fd759000640 1 -- 172.21.15.59:0/918335496 >> v1:172.21.15.66:6789/0 conn(0x7fd7540b6530 legacy=0x7fd7540d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.398+0000 7fd759000640 1 -- 172.21.15.59:0/918335496 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7540b6a40 con 0x7fd75415e7c0 2024-04-24T06:33:40.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.399+0000 7fd759000640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd75415e7c0 0x7fd7540d20f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd744009e50 tx=0x7fd74402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b7697daa16938 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.399+0000 7fd753400640 1 -- 172.21.15.59:0/918335496 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744036070 con 0x7fd75415e7c0 2024-04-24T06:33:40.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.399+0000 7fd753400640 1 -- 172.21.15.59:0/918335496 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd744031600 con 0x7fd75415e7c0 2024-04-24T06:33:40.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.399+0000 7fd753400640 1 -- 172.21.15.59:0/918335496 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440319d0 con 0x7fd75415e7c0 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 -- 172.21.15.59:0/918335496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd75415e7c0 msgr2=0x7fd7540d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd75415e7c0 0x7fd7540d20f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd744009e50 tx=0x7fd74402a6c0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 -- 172.21.15.59:0/918335496 shutdown_connections 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7540d2630 0x7fd7540b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 --2- 172.21.15.59:0/918335496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd75415e7c0 0x7fd7540d20f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 -- 172.21.15.59:0/918335496 >> 172.21.15.59:0/918335496 conn(0x7fd75409d920 msgr2=0x7fd75408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:40.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 -- 172.21.15.59:0/918335496 shutdown_connections 2024-04-24T06:33:40.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.400+0000 7fd75a400640 1 -- 172.21.15.59:0/918335496 wait complete. 2024-04-24T06:33:40.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.402+0000 7fd75a400640 1 Processor -- start 2024-04-24T06:33:40.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.403+0000 7fd75a400640 1 -- start start 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 0x7fd7540cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd75415e7c0 0x7fd7540b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7540d1a80 con 0x7fd7540d2630 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7540cf2b0 con 0x7fd7540b6530 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd75a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7540d1c00 con 0x7fd75415e7c0 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd753e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd753e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46714/0 (socket says 172.21.15.59:46714) 2024-04-24T06:33:40.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.404+0000 7fd753e00640 1 -- 172.21.15.59:0/2679346762 learned_addr learned my addr 172.21.15.59:0/2679346762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd759a00640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd75415e7c0 0x7fd7540b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 -- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd75415e7c0 msgr2=0x7fd7540b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd75415e7c0 0x7fd7540b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 -- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 msgr2=0x7fd7540cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd759000640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 0x7fd7540cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 0x7fd7540cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 -- 172.21.15.59:0/2679346762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7540b1c20 con 0x7fd7540d2630 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd759000640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 0x7fd7540cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd753e00640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00c7f0 tx=0x7fd73c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c015070 con 0x7fd7540d2630 2024-04-24T06:33:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c004220 con 0x7fd7540d2630 2024-04-24T06:33:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.405+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7540b1f00 con 0x7fd7540d2630 2024-04-24T06:33:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.406+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c007620 con 0x7fd7540d2630 2024-04-24T06:33:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.406+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7540b24c0 con 0x7fd7540d2630 2024-04-24T06:33:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.407+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75404fd90 con 0x7fd7540d2630 2024-04-24T06:33:40.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.408+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73c0079d0 con 0x7fd7540d2630 2024-04-24T06:33:40.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.409+0000 7fd750c00640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 0x7fd73006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.409+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fd73c08b9d0 con 0x7fd7540d2630 2024-04-24T06:33:40.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.409+0000 7fd759000640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 0x7fd73006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.410+0000 7fd759000640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 0x7fd73006ede0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd744033000 tx=0x7fd744009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.413+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd73c05a020 con 0x7fd7540d2630 2024-04-24T06:33:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.694+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd7540b42d0 con 0x7fd7540d2630 2024-04-24T06:33:40.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.695+0000 7fd750c00640 1 -- 172.21.15.59:0/2679346762 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fd73c059920 con 0x7fd7540d2630 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:33:40.698 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.699 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:33:40.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.702 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:33:40.705 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:33:40.706 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:33:40.707 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:33:40.708 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:33:40.709 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:33:40.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:40.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:33:40.711 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 msgr2=0x7fd73006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 0x7fd73006ede0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd744033000 tx=0x7fd744009000 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 msgr2=0x7fd7540b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00c7f0 tx=0x7fd73c00ccb0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 shutdown_connections 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd73006c920 0x7fd73006ede0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd75415e7c0 0x7fd7540b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7540d2630 0x7fd7540b14b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 --2- 172.21.15.59:0/2679346762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7540b6530 0x7fd7540cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.698+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 >> 172.21.15.59:0/2679346762 conn(0x7fd75409d920 msgr2=0x7fd75415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.699+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 shutdown_connections 2024-04-24T06:33:40.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.699+0000 7fd75a400640 1 -- 172.21.15.59:0/2679346762 wait complete. 2024-04-24T06:33:40.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.926+0000 7fab0f200640 1 Processor -- start 2024-04-24T06:33:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.926+0000 7fab0f200640 1 -- start start 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7fab0f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab080cd020 0x7fab080b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7fab0f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080cd520 con 0x7fab081454e0 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7fab0f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080cd800 con 0x7fab080b3e40 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7fab0f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fab080cddc0 con 0x7fab080cd020 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7faaff400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab080cd020 0x7fab080b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7faaff400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab080cd020 0x7fab080b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46728/0 (socket says 172.21.15.59:46728) 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7fab07e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fab081454e0 0x7fab08145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33352/0 (socket says 172.21.15.59:33352) 2024-04-24T06:33:40.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.928+0000 7faaff400640 1 -- 172.21.15.59:0/3837162984 learned_addr learned my addr 172.21.15.59:0/3837162984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:40.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.929+0000 7faaff400640 1 -- 172.21.15.59:0/3837162984 >> v1:172.21.15.66:6789/0 conn(0x7fab080b3e40 legacy=0x7fab08160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:40.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.929+0000 7faaff400640 1 -- 172.21.15.59:0/3837162984 >> v1:172.21.15.59:6789/0 conn(0x7fab081454e0 legacy=0x7fab08145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.929+0000 7faaff400640 1 -- 172.21.15.59:0/3837162984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080cdae0 con 0x7fab080cd020 2024-04-24T06:33:40.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.929+0000 7fab07400640 1 -- 172.21.15.59:0/3837162984 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120405417 0 0) 0x7fab080cd520 con 0x7fab081454e0 2024-04-24T06:33:40.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.930+0000 7faaff400640 1 --2- 172.21.15.59:0/3837162984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab080b3900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009870 tx=0x7faaf002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1360ac3b12144f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.930+0000 7fab07400640 1 -- 172.21.15.59:0/3837162984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0037070 con 0x7fab080cd020 2024-04-24T06:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.930+0000 7fab07400640 1 -- 172.21.15.59:0/3837162984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf0032440 con 0x7fab080cd020 2024-04-24T06:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.930+0000 7fab07400640 1 -- 172.21.15.59:0/3837162984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0032810 con 0x7fab080cd020 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 -- 172.21.15.59:0/3837162984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 msgr2=0x7fab080b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 --2- 172.21.15.59:0/3837162984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab080b3900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009870 tx=0x7faaf002a210 comp rx=0 tx=0).stop 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 -- 172.21.15.59:0/3837162984 shutdown_connections 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 --2- 172.21.15.59:0/3837162984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab080b3900 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 -- 172.21.15.59:0/3837162984 >> 172.21.15.59:0/3837162984 conn(0x7fab08067ea0 msgr2=0x7fab0806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 -- 172.21.15.59:0/3837162984 shutdown_connections 2024-04-24T06:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.931+0000 7fab0f200640 1 -- 172.21.15.59:0/3837162984 wait complete. 2024-04-24T06:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.933+0000 7fab0f200640 1 Processor -- start 2024-04-24T06:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.933+0000 7fab0f200640 1 -- start start 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 0x7fab0813edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab081454e0 0x7fab0815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab08161020 con 0x7fab080cd020 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab081462c0 con 0x7fab081454e0 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7fab0f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab081611a0 con 0x7fab080b3e40 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7faaff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7faaff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46736/0 (socket says 172.21.15.59:46736) 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.934+0000 7faaff400640 1 -- 172.21.15.59:0/1659814297 learned_addr learned my addr 172.21.15.59:0/1659814297 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:40.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7fab07e00640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 0x7fab0813edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 -- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 msgr2=0x7fab0813edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 0x7fab0813edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7fab0d200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab081454e0 0x7fab0815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 -- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab081454e0 msgr2=0x7fab0815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab081454e0 0x7fab0815b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 -- 172.21.15.59:0/1659814297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0813f330 con 0x7fab080cd020 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7fab07e00640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 0x7fab0813edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:40.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.935+0000 7faaff400640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009870 tx=0x7faaf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.936+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0041070 con 0x7fab080cd020 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.936+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf003c870 con 0x7fab080cd020 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.936+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0037040 con 0x7fab080cd020 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.936+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0806e9a0 con 0x7fab080cd020 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.936+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab08142ef0 con 0x7fab080cd020 2024-04-24T06:33:40.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.937+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08068a80 con 0x7fab080cd020 2024-04-24T06:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.938+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaf0044070 con 0x7fab080cd020 2024-04-24T06:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.939+0000 7fab04c00640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 0x7faae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.939+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7faaf00ac560 con 0x7fab080cd020 2024-04-24T06:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.939+0000 7fab07e00640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 0x7faae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.940+0000 7fab07e00640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 0x7faae406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faaf8000f60 tx=0x7faaf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:40.944+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faaf007ad60 con 0x7fab080cd020 2024-04-24T06:33:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.148+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab080b56c0 con 0x7fab080cd020 2024-04-24T06:33:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.149+0000 7fab04c00640 1 -- 172.21.15.59:0/1659814297 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+12807 (secure 0 0 0) 0x7faaf007a660 con 0x7fab080cd020 2024-04-24T06:33:41.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:41.152 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":65,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:40.179902+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 msgr2=0x7faae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 0x7faae406edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faaf8000f60 tx=0x7faaf8009040 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 msgr2=0x7fab0815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009870 tx=0x7faaf00046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 shutdown_connections 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faae406c8f0 0x7faae406edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab081454e0 0x7fab0815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080cd020 0x7fab0815d1e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 --2- 172.21.15.59:0/1659814297 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab080b3e40 0x7fab0813edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.152+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 >> 172.21.15.59:0/1659814297 conn(0x7fab08067ea0 msgr2=0x7fab080b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.153+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 shutdown_connections 2024-04-24T06:33:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.153+0000 7fab0f200640 1 -- 172.21.15.59:0/1659814297 wait complete. 2024-04-24T06:33:41.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:41.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:41.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.391+0000 7f72dce00640 1 Processor -- start 2024-04-24T06:33:41.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.391+0000 7f72dce00640 1 -- start start 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72dce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72d806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72dce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f72d80af2a0 0x7f72d80d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72dce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d815ef20 con 0x7f72d80d0600 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72dce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f72d80d0fa0 con 0x7f72d80af2a0 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72dce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f72d815f1c0 con 0x7f72d8155a80 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72cea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f72d80d0600 0x7f72d80b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33368/0 (socket says 172.21.15.59:33368) 2024-04-24T06:33:41.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.393+0000 7f72cea00640 1 -- 172.21.15.59:0/2472456326 learned_addr learned my addr 172.21.15.59:0/2472456326 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d7e00640 1 --2- 172.21.15.59:0/2472456326 >> v2:172.21.15.66:3300/0 conn(0x7f72d80af2a0 0x7f72d80d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72d806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2059722416 0 0) 0x7f72d815ef20 con 0x7f72d80d0600 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72bc003520 con 0x7f72d80d0600 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 985979365 0 0) 0x7f72bc003520 con 0x7f72d80d0600 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 msgr2=0x7f72d806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72d806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 >> v2:172.21.15.66:3300/0 conn(0x7f72d80af2a0 msgr2=0x7f72d80d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 --2- 172.21.15.59:0/2472456326 >> v2:172.21.15.66:3300/0 conn(0x7f72d80af2a0 0x7f72d80d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d815f4a0 con 0x7f72d80d0600 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f72c0001670 con 0x7f72d80d0600 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72d806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.394+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 >> v1:172.21.15.59:6789/0 conn(0x7f72d80d0600 legacy=0x7f72d80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72bc004800 0x7f72bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72d815f1c0 con 0x7f72bc0039b0 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc00dbd0 con 0x7f72bc004800 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc00dd80 con 0x7f72d8155a80 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72cea00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72bc004800 0x7f72bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.395+0000 7f72d7e00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.396+0000 7f72d7400640 1 -- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 msgr2=0x7f72bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.396+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8155a80 0x7f72bc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.396+0000 7f72d7400640 1 -- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72bc004800 msgr2=0x7f72bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.396+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72bc004800 0x7f72bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.396+0000 7f72d7400640 1 -- 172.21.15.59:0/2472456326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc003520 con 0x7f72bc0039b0 2024-04-24T06:33:41.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.397+0000 7f72d7400640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72bc003dd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f72c80098a0 tx=0x7f72bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3518672c55da5220 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.397+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c800f040 con 0x7f72bc0039b0 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.397+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c802ad00 con 0x7f72bc0039b0 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.397+0000 7f72d6a00640 1 -- 172.21.15.59:0/2472456326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8004250 con 0x7f72bc0039b0 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 -- 172.21.15.59:0/2472456326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 msgr2=0x7f72bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 --2- 172.21.15.59:0/2472456326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72bc003dd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f72c80098a0 tx=0x7f72bc005830 comp rx=0 tx=0).stop 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72d7e00640 1 -- 172.21.15.59:0/2472456326 reap_dead start 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 -- 172.21.15.59:0/2472456326 shutdown_connections 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 -- 172.21.15.59:0/2472456326 >> 172.21.15.59:0/2472456326 conn(0x7f72d806bdf0 msgr2=0x7f72d806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 -- 172.21.15.59:0/2472456326 shutdown_connections 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.398+0000 7f72dce00640 1 -- 172.21.15.59:0/2472456326 wait complete. 2024-04-24T06:33:41.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.400+0000 7f72dce00640 1 Processor -- start 2024-04-24T06:33:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.400+0000 7f72dce00640 1 -- start start 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8141a90 0x7f72d815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8142ff0 0x7f72d81434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72d815fd40 con 0x7f72bc0039b0 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f72d815fa60 con 0x7f72d8142ff0 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72dce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f72d806ef80 con 0x7f72d8141a90 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46754/0 (socket says 172.21.15.59:46754) 2024-04-24T06:33:41.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 -- 172.21.15.59:0/1775882973 learned_addr learned my addr 172.21.15.59:0/1775882973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 -- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8141a90 msgr2=0x7f72d815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72cea00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8141a90 0x7f72d815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8141a90 0x7f72d815a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 -- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8142ff0 msgr2=0x7f72d81434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8142ff0 0x7f72d81434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.401+0000 7f72d7400640 1 -- 172.21.15.59:0/1775882973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d8141f50 con 0x7f72bc0039b0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72d7400640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f72c80099c0 tx=0x7f72c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8041070 con 0x7f72bc0039b0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c800f000 con 0x7f72bc0039b0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8003c50 con 0x7f72bc0039b0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d8142230 con 0x7f72bc0039b0 2024-04-24T06:33:41.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.402+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d81427f0 con 0x7f72bc0039b0 2024-04-24T06:33:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.404+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d804fd90 con 0x7f72bc0039b0 2024-04-24T06:33:41.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.404+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72c8044070 con 0x7f72bc0039b0 2024-04-24T06:33:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.405+0000 7f72cfe00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.405+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f72c80abed0 con 0x7f72bc0039b0 2024-04-24T06:33:41.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.405+0000 7f72cea00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.406+0000 7f72cea00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 0x7f72b406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72d80b4560 tx=0x7f72c0000f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.410+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72c807a5c0 con 0x7f72bc0039b0 2024-04-24T06:33:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.612+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72d806eba0 con 0x7f72bc0039b0 2024-04-24T06:33:41.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.613+0000 7f72cfe00640 1 -- 172.21.15.59:0/1775882973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f72c8079ec0 con 0x7f72bc0039b0 2024-04-24T06:33:41.615 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:41.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 msgr2=0x7f72b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 0x7f72b406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72d80b4560 tx=0x7f72c0000f90 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 msgr2=0x7f72d80b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f72c80099c0 tx=0x7f72c8004730 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 shutdown_connections 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72d8142ff0 0x7f72d81434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f72d8141a90 0x7f72d815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 --2- 172.21.15.59:0/1775882973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72bc0039b0 0x7f72d80b1dc0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.616+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 >> 172.21.15.59:0/1775882973 conn(0x7f72d806bdf0 msgr2=0x7f72d80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.617+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 shutdown_connections 2024-04-24T06:33:41.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.617+0000 7f72dce00640 1 -- 172.21.15.59:0/1775882973 wait complete. 2024-04-24T06:33:41.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:33:41.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.840+0000 7f7e1da00640 1 Processor -- start 2024-04-24T06:33:41.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.840+0000 7f7e1da00640 1 -- start start 2024-04-24T06:33:41.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.841+0000 7f7e1da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7e1815c3d0 0x7f7e180cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.841+0000 7f7e1da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e180cd590 0x7f7e180b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e1da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e18160210 con 0x7f7e180b4690 2024-04-24T06:33:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e1da00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7e181604f0 con 0x7f7e180cd590 2024-04-24T06:33:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e1da00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7e181607d0 con 0x7f7e1815c3d0 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e180cd590 0x7f7e180b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e180cd590 0x7f7e180b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46770/0 (socket says 172.21.15.59:46770) 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 -- 172.21.15.59:0/2934802195 learned_addr learned my addr 172.21.15.59:0/2934802195 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e1d000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7e180b4690 0x7f7e1806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33370/0 (socket says 172.21.15.59:33370) 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17e00640 1 --2- 172.21.15.59:0/2934802195 >> v2:172.21.15.66:3300/0 conn(0x7f7e1815c3d0 0x7f7e180cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 -- 172.21.15.59:0/2934802195 >> v2:172.21.15.66:3300/0 conn(0x7f7e1815c3d0 msgr2=0x7f7e180cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 --2- 172.21.15.59:0/2934802195 >> v2:172.21.15.66:3300/0 conn(0x7f7e1815c3d0 0x7f7e180cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e17400640 1 -- 172.21.15.59:0/2934802195 >> v1:172.21.15.59:6789/0 conn(0x7f7e180b4690 legacy=0x7f7e1806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.843+0000 7f7e17400640 1 -- 172.21.15.59:0/2934802195 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18160ab0 con 0x7f7e180cd590 2024-04-24T06:33:41.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.842+0000 7f7e16a00640 1 -- 172.21.15.59:0/2934802195 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250892957 0 0) 0x7f7e18160210 con 0x7f7e180b4690 2024-04-24T06:33:41.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.844+0000 7f7e17400640 1 --2- 172.21.15.59:0/2934802195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e180b4150 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7e1806d9b0 tx=0x7f7e0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc31f08474b23d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.844+0000 7f7e16a00640 1 -- 172.21.15.59:0/2934802195 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00037070 con 0x7f7e180cd590 2024-04-24T06:33:41.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.844+0000 7f7e16a00640 1 -- 172.21.15.59:0/2934802195 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e00032440 con 0x7f7e180cd590 2024-04-24T06:33:41.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.844+0000 7f7e16a00640 1 -- 172.21.15.59:0/2934802195 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00032810 con 0x7f7e180cd590 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 -- 172.21.15.59:0/2934802195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 msgr2=0x7f7e180b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2934802195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e180b4150 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7e1806d9b0 tx=0x7f7e0002a270 comp rx=0 tx=0).stop 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 -- 172.21.15.59:0/2934802195 shutdown_connections 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2934802195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e180b4150 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2934802195 >> v2:172.21.15.66:3300/0 conn(0x7f7e1815c3d0 0x7f7e180cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 -- 172.21.15.59:0/2934802195 >> 172.21.15.59:0/2934802195 conn(0x7f7e180b21a0 msgr2=0x7f7e180b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 -- 172.21.15.59:0/2934802195 shutdown_connections 2024-04-24T06:33:41.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.845+0000 7f7e1da00640 1 -- 172.21.15.59:0/2934802195 wait complete. 2024-04-24T06:33:41.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.847+0000 7f7e1da00640 1 Processor -- start 2024-04-24T06:33:41.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.847+0000 7f7e1da00640 1 -- start start 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e180b4690 0x7f7e18072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e1815c3d0 0x7f7e180bb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1815c7b0 con 0x7f7e180cd590 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180b4ac0 con 0x7f7e1815c3d0 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e1da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1815c930 con 0x7f7e180b4690 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46774/0 (socket says 172.21.15.59:46774) 2024-04-24T06:33:41.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 -- 172.21.15.59:0/2540513186 learned_addr learned my addr 172.21.15.59:0/2540513186 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 -- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e180b4690 msgr2=0x7f7e18072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e180b4690 0x7f7e18072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 -- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e1815c3d0 msgr2=0x7f7e180bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:41.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e1815c3d0 0x7f7e180bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.848+0000 7f7e17400640 1 -- 172.21.15.59:0/2540513186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18072ac0 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e17400640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7e0002a2b0 tx=0x7f7e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00041020 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e000306c0 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00037040 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e18072d90 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.849+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e18073350 con 0x7f7e180cd590 2024-04-24T06:33:41.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.850+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e180adc10 con 0x7f7e180cd590 2024-04-24T06:33:41.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.851+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e00044020 con 0x7f7e180cd590 2024-04-24T06:33:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.852+0000 7f7e0fe00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 0x7f7df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.852+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f7e000ac100 con 0x7f7e180cd590 2024-04-24T06:33:41.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.853+0000 7f7e17e00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 0x7f7df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:41.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.853+0000 7f7e17e00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 0x7f7df406ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006fd0 tx=0x7f7e08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:41.857+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e0004b030 con 0x7f7e180cd590 2024-04-24T06:33:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.062+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7e1804fd90 con 0x7f7e180cd590 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.063+0000 7f7e0fe00640 1 -- 172.21.15.59:0/2540513186 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f7e0007a7e0 con 0x7f7e180cd590 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:42.065 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 msgr2=0x7f7df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 0x7f7df406ede0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006fd0 tx=0x7f7e08008040 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 msgr2=0x7f7e1815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7e0002a2b0 tx=0x7f7e000046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 shutdown_connections 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7df406c920 0x7f7df406ede0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e1815c3d0 0x7f7e180bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e180cd590 0x7f7e1815d5a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 --2- 172.21.15.59:0/2540513186 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7e180b4690 0x7f7e18072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 >> 172.21.15.59:0/2540513186 conn(0x7f7e180b21a0 msgr2=0x7f7e180cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:42.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 shutdown_connections 2024-04-24T06:33:42.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.066+0000 7f7e1da00640 1 -- 172.21.15.59:0/2540513186 wait complete. 2024-04-24T06:33:42.087 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:33:42.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:33:42.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.290+0000 7f57e9800640 1 Processor -- start 2024-04-24T06:33:42.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.290+0000 7f57e9800640 1 -- start start 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e9800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57e41677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e9800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e40cf340 con 0x7f57e40cd7e0 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e9800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e406ef10 con 0x7f57e41431b0 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e9800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57e40cf670 con 0x7f57e406ea10 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f57e40cd7e0 0x7f57e406e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33386/0 (socket says 172.21.15.59:33386) 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.292+0000 7f57e2000640 1 -- 172.21.15.59:0/4262018098 learned_addr learned my addr 172.21.15.59:0/4262018098 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222130244 0 0) 0x7f57e40cf340 con 0x7f57e40cd7e0 2024-04-24T06:33:42.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e3400640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57e41677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57c8003520 con 0x7f57e40cd7e0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 594519340 0 0) 0x7f57c8003520 con 0x7f57e40cd7e0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 msgr2=0x7f57e41677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57e41677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 >> v1:172.21.15.66:6789/0 conn(0x7f57e41431b0 legacy=0x7f57e40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e406f1f0 con 0x7f57e40cd7e0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f57d0001670 con 0x7f57e40cd7e0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 >> v1:172.21.15.59:6789/0 conn(0x7f57e40cd7e0 legacy=0x7f57e406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57c8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57c8004030 0x7f57c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c8004d40 0x7f57c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57e40cf670 con 0x7f57c8004d40 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800bc10 con 0x7f57c8004030 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.293+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57c800be10 con 0x7f57e406ea10 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c8004d40 0x7f57c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e2000640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57c8004030 0x7f57c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 -- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 msgr2=0x7f57c8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e2a00640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57c8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57e406ea10 0x7f57c8003a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 -- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57c8004030 msgr2=0x7f57c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57c8004030 0x7f57c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e3400640 1 -- 172.21.15.59:0/4262018098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c8003520 con 0x7f57c8004d40 2024-04-24T06:33:42.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.294+0000 7f57e2000640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57c8004030 0x7f57c8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.295+0000 7f57e3400640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c8004d40 0x7f57c80051c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f57dc007330 tx=0x7f57c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef876bcf8986e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.295+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc033070 con 0x7f57c8004d40 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.295+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc02f650 con 0x7f57c8004d40 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.295+0000 7f57e1600640 1 -- 172.21.15.59:0/4262018098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc009be0 con 0x7f57c8004d40 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 -- 172.21.15.59:0/4262018098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c8004d40 msgr2=0x7f57c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 --2- 172.21.15.59:0/4262018098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c8004d40 0x7f57c80051c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f57dc007330 tx=0x7f57c8005830 comp rx=0 tx=0).stop 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e3400640 1 -- 172.21.15.59:0/4262018098 reap_dead start 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 -- 172.21.15.59:0/4262018098 shutdown_connections 2024-04-24T06:33:42.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 -- 172.21.15.59:0/4262018098 >> 172.21.15.59:0/4262018098 conn(0x7f57e406b9d0 msgr2=0x7f57e406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:42.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 -- 172.21.15.59:0/4262018098 shutdown_connections 2024-04-24T06:33:42.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.296+0000 7f57e9800640 1 -- 172.21.15.59:0/4262018098 wait complete. 2024-04-24T06:33:42.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.298+0000 7f57e9800640 1 Processor -- start 2024-04-24T06:33:42.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.298+0000 7f57e9800640 1 -- start start 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57c8004d40 0x7f57e415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e41f4820 0x7f57e41f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57e40d0c90 con 0x7f57e413ee20 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57e40d0b10 con 0x7f57e41f4820 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e9800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f57e40d0e10 con 0x7f57c8004d40 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46792/0 (socket says 172.21.15.59:46792) 2024-04-24T06:33:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.299+0000 7f57e3400640 1 -- 172.21.15.59:0/1827756509 learned_addr learned my addr 172.21.15.59:0/1827756509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e2a00640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57c8004d40 0x7f57e415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e2000640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e41f4820 0x7f57e41f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 -- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57c8004d40 msgr2=0x7f57e415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57c8004d40 0x7f57e415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 -- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e41f4820 msgr2=0x7f57e41f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e41f4820 0x7f57e41f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 -- 172.21.15.59:0/1827756509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41f4d20 con 0x7f57e413ee20 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e3400640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f57dc02b030 tx=0x7f57dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc033040 con 0x7f57e413ee20 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc02f450 con 0x7f57e413ee20 2024-04-24T06:33:42.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc009be0 con 0x7f57e413ee20 2024-04-24T06:33:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e41f5000 con 0x7f57e413ee20 2024-04-24T06:33:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.300+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41f55c0 con 0x7f57e413ee20 2024-04-24T06:33:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.302+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e4068a80 con 0x7f57e413ee20 2024-04-24T06:33:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.302+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57dc036070 con 0x7f57e413ee20 2024-04-24T06:33:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.303+0000 7f57daa00640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.303+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f57dc0abdc0 con 0x7f57e413ee20 2024-04-24T06:33:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.304+0000 7f57e2a00640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:42.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.304+0000 7f57e2a00640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 0x7f57c406edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57d4001590 tx=0x7f57d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:42.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.308+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57dc07a530 con 0x7f57e413ee20 2024-04-24T06:33:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:42.513+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f57e40ccc30 con 0x7f57e413ee20 2024-04-24T06:33:43.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.203+0000 7f57daa00640 1 -- 172.21.15.59:0/1827756509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f57dc079e30 con 0x7f57e413ee20 2024-04-24T06:33:43.205 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:33:43.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 msgr2=0x7f57c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 0x7f57c406edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57d4001590 tx=0x7f57d4009040 comp rx=0 tx=0).stop 2024-04-24T06:33:43.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 msgr2=0x7f57e40b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f57dc02b030 tx=0x7f57dc004ab0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 shutdown_connections 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f57c406c8f0 0x7f57c406edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57e413ee20 0x7f57e40b4ac0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57e41f4820 0x7f57e41f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 --2- 172.21.15.59:0/1827756509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f57c8004d40 0x7f57e415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.207+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 >> 172.21.15.59:0/1827756509 conn(0x7f57e406b9d0 msgr2=0x7f57e415f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.208+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 shutdown_connections 2024-04-24T06:33:43.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.208+0000 7f57e9800640 1 -- 172.21.15.59:0/1827756509 wait complete. 2024-04-24T06:33:43.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:33:43.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.455+0000 7fbf5ee00640 1 Processor -- start 2024-04-24T06:33:43.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.456+0000 7fbf5ee00640 1 -- start start 2024-04-24T06:33:43.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.457+0000 7fbf5ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf580aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.457+0000 7fbf5ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf58071d70 0x7fbf58072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.457+0000 7fbf5ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf580adc80 con 0x7fbf5806f6f0 2024-04-24T06:33:43.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.457+0000 7fbf5ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbf580aef40 con 0x7fbf58071d70 2024-04-24T06:33:43.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.457+0000 7fbf5ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf580afd80 con 0x7fbf5815e790 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.458+0000 7fbf5ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf58071d70 0x7fbf58072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.458+0000 7fbf5ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf58071d70 0x7fbf58072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46804/0 (socket says 172.21.15.59:46804) 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.458+0000 7fbf5ce00640 1 -- 172.21.15.59:0/3358910382 learned_addr learned my addr 172.21.15.59:0/3358910382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.458+0000 7fbf57e00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf580aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.459+0000 7fbf5ce00640 1 -- 172.21.15.59:0/3358910382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 msgr2=0x7fbf580aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.459+0000 7fbf5ce00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf580aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.459+0000 7fbf5ce00640 1 -- 172.21.15.59:0/3358910382 >> v1:172.21.15.66:6789/0 conn(0x7fbf5806f6f0 legacy=0x7fbf58162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.459+0000 7fbf5ce00640 1 -- 172.21.15.59:0/3358910382 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf580adf20 con 0x7fbf58071d70 2024-04-24T06:33:43.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.460+0000 7fbf5ce00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf58071d70 0x7fbf58072200 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009e50 tx=0x7fbf5002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ae442659402518 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.460+0000 7fbf57400640 1 -- 172.21.15.59:0/3358910382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50036070 con 0x7fbf58071d70 2024-04-24T06:33:43.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.460+0000 7fbf57400640 1 -- 172.21.15.59:0/3358910382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50031600 con 0x7fbf58071d70 2024-04-24T06:33:43.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.460+0000 7fbf57400640 1 -- 172.21.15.59:0/3358910382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500319d0 con 0x7fbf58071d70 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 -- 172.21.15.59:0/3358910382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf58071d70 msgr2=0x7fbf58072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf58071d70 0x7fbf58072200 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009e50 tx=0x7fbf5002a730 comp rx=0 tx=0).stop 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 -- 172.21.15.59:0/3358910382 shutdown_connections 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf58071d70 0x7fbf58072200 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/3358910382 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf580aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 -- 172.21.15.59:0/3358910382 >> 172.21.15.59:0/3358910382 conn(0x7fbf58067d90 msgr2=0x7fbf5806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 -- 172.21.15.59:0/3358910382 shutdown_connections 2024-04-24T06:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.461+0000 7fbf5ee00640 1 -- 172.21.15.59:0/3358910382 wait complete. 2024-04-24T06:33:43.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.463+0000 7fbf5ee00640 1 Processor -- start 2024-04-24T06:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.463+0000 7fbf5ee00640 1 -- start start 2024-04-24T06:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf58071d70 0x7fbf5815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf5815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5806fbb0 con 0x7fbf5806f6f0 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf580ae4e0 con 0x7fbf58071d70 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf5ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5815eb70 con 0x7fbf5815e790 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46810/0 (socket says 172.21.15.59:46810) 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.464+0000 7fbf57e00640 1 -- 172.21.15.59:0/222680305 learned_addr learned my addr 172.21.15.59:0/222680305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 -- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 msgr2=0x7fbf5815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf5815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 -- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf58071d70 msgr2=0x7fbf5815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf58071d70 0x7fbf5815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 -- 172.21.15.59:0/222680305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5814dc30 con 0x7fbf5806f6f0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf57e00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800d5f0 tx=0x7fbf4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480125f0 con 0x7fbf5806f6f0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48019070 con 0x7fbf5806f6f0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5815f9c0 con 0x7fbf5806f6f0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480129d0 con 0x7fbf5806f6f0 2024-04-24T06:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.465+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5815ff80 con 0x7fbf5806f6f0 2024-04-24T06:33:43.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.467+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58068a80 con 0x7fbf5806f6f0 2024-04-24T06:33:43.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.468+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf48012b70 con 0x7fbf5806f6f0 2024-04-24T06:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.469+0000 7fbf54c00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.469+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7fbf4808bdf0 con 0x7fbf5806f6f0 2024-04-24T06:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.469+0000 7fbf4fe00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.470+0000 7fbf4fe00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 0x7fbf3406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f60 tx=0x7fbf400079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.475+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf5814dc30 con 0x7fbf5806f6f0 2024-04-24T06:33:43.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:43.680+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbf58162190 con 0x7fbf5806f6f0 2024-04-24T06:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.219+0000 7fbf54c00640 1 -- 172.21.15.59:0/222680305 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7fbf4805a360 con 0x7fbf5806f6f0 2024-04-24T06:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:33:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.223+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 msgr2=0x7fbf3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.223+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 0x7fbf3406edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f60 tx=0x7fbf400079d0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.223+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 msgr2=0x7fbf5814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.223+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800d5f0 tx=0x7fbf4800dab0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 shutdown_connections 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf5815e790 0x7fbf5815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf58071d70 0x7fbf5815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 --2- 172.21.15.59:0/222680305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf5806f6f0 0x7fbf5814d6f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 >> 172.21.15.59:0/222680305 conn(0x7fbf58067d90 msgr2=0x7fbf580b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 shutdown_connections 2024-04-24T06:33:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.224+0000 7fbf5ee00640 1 -- 172.21.15.59:0/222680305 wait complete. 2024-04-24T06:33:44.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:33:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.440+0000 7f8f23e00640 1 Processor -- start 2024-04-24T06:33:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.440+0000 7f8f23e00640 1 -- start start 2024-04-24T06:33:44.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8f241567e0 0x7f8f24156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f2415cab0 con 0x7f8f2415d810 2024-04-24T06:33:44.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8f24160e40 con 0x7f8f241567e0 2024-04-24T06:33:44.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2415cd30 con 0x7f8f240d0d10 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f22000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f2415d810 0x7f8f240d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33402/0 (socket says 172.21.15.59:33402) 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f22000640 1 -- 172.21.15.59:0/1113367088 learned_addr learned my addr 172.21.15.59:0/1113367088 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f22a00640 1 --2- 172.21.15.59:0/1113367088 >> v2:172.21.15.66:3300/0 conn(0x7f8f241567e0 0x7f8f24156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f23400640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462859450 0 0) 0x7f8f0c001110 con 0x7f8f2415d810 2024-04-24T06:33:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.442+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f08003520 con 0x7f8f2415d810 2024-04-24T06:33:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.443+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 57704583 0 0) 0x7f8f08003520 con 0x7f8f2415d810 2024-04-24T06:33:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.443+0000 7f8f22a00640 1 -- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 msgr2=0x7f8f24160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.443+0000 7f8f22a00640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.443+0000 7f8f22a00640 1 -- 172.21.15.59:0/1113367088 >> v1:172.21.15.59:6789/0 conn(0x7f8f2415d810 legacy=0x7f8f240d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.443+0000 7f8f22a00640 1 -- 172.21.15.59:0/1113367088 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2415d010 con 0x7f8f241567e0 2024-04-24T06:33:44.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.444+0000 7f8f22a00640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f24156bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097a0 tx=0x7f8f1402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71727f25e22aef99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:44.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.444+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037070 con 0x7f8f241567e0 2024-04-24T06:33:44.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1402aba0 con 0x7f8f241567e0 2024-04-24T06:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f21600640 1 -- 172.21.15.59:0/1113367088 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c001840 con 0x7f8f241567e0 2024-04-24T06:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 -- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 msgr2=0x7f8f24156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f24156bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097a0 tx=0x7f8f1402a110 comp rx=0 tx=0).stop 2024-04-24T06:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 -- 172.21.15.59:0/1113367088 shutdown_connections 2024-04-24T06:33:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 --2- 172.21.15.59:0/1113367088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f24156bc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 -- 172.21.15.59:0/1113367088 >> 172.21.15.59:0/1113367088 conn(0x7f8f2406bdf0 msgr2=0x7f8f2406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 -- 172.21.15.59:0/1113367088 shutdown_connections 2024-04-24T06:33:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.445+0000 7f8f23e00640 1 -- 172.21.15.59:0/1113367088 wait complete. 2024-04-24T06:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.447+0000 7f8f23e00640 1 Processor -- start 2024-04-24T06:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.447+0000 7f8f23e00640 1 -- start start 2024-04-24T06:33:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24099950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f2406f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241607d0 con 0x7f8f2415d810 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24160650 con 0x7f8f241567e0 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f24160950 con 0x7f8f240d0d10 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46818/0 (socket says 172.21.15.59:46818) 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.448+0000 7f8f23400640 1 -- 172.21.15.59:0/4209816691 learned_addr learned my addr 172.21.15.59:0/4209816691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f22a00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24099950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f22000640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f2406f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 -- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 msgr2=0x7f8f24099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24099950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 -- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 msgr2=0x7f8f2406f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f2406f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 -- 172.21.15.59:0/4209816691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24158a30 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f22a00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24099950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f22000640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f2406f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.449+0000 7f8f23400640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d30 tx=0x7f8f1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.450+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c018020 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.450+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c014480 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.450+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c014870 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.450+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2415ef00 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.450+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2414edb0 con 0x7f8f2415d810 2024-04-24T06:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.451+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2404fd90 con 0x7f8f2415d810 2024-04-24T06:33:44.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.452+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f1c005d10 con 0x7f8f2415d810 2024-04-24T06:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.453+0000 7f8f1aa00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 0x7f8f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.453+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f8f1c08cd50 con 0x7f8f2415d810 2024-04-24T06:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.454+0000 7f8f22a00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 0x7f8f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.454+0000 7f8f22a00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 0x7f8f0006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8f14034000 tx=0x7f8f1402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:44.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.457+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f1c022070 con 0x7f8f2415d810 2024-04-24T06:33:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:44.701+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8f2415c090 con 0x7f8f2415d810 2024-04-24T06:33:45.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.248+0000 7f8f1aa00640 1 -- 172.21.15.59:0/4209816691 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) ==== 174+0+0 (secure 0 0 0) 0x7f8f1c010040 con 0x7f8f2415d810 2024-04-24T06:33:45.254 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 10 and data pool 11 2024-04-24T06:33:45.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.255+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 msgr2=0x7f8f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.255+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 0x7f8f0006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8f14034000 tx=0x7f8f1402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.255+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 msgr2=0x7f8f24157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.255+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d30 tx=0x7f8f1c00c490 comp rx=0 tx=0).stop 2024-04-24T06:33:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 shutdown_connections 2024-04-24T06:33:45.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0006c8f0 0x7f8f0006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2415d810 0x7f8f24157150 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f241567e0 0x7f8f2406f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 --2- 172.21.15.59:0/4209816691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f240d0d10 0x7f8f24099950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.258+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 >> 172.21.15.59:0/4209816691 conn(0x7f8f2406bdf0 msgr2=0x7f8f24072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.259+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 shutdown_connections 2024-04-24T06:33:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.260+0000 7f8f23e00640 1 -- 172.21.15.59:0/4209816691 wait complete. 2024-04-24T06:33:45.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:45.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.471+0000 7f491a600640 1 Processor -- start 2024-04-24T06:33:45.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.471+0000 7f491a600640 1 -- start start 2024-04-24T06:33:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f491a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 0x7f49140af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f491a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49140d08b0 0x7f49141624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f491a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f491415f0f0 con 0x7f49140af920 2024-04-24T06:33:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f491a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f49140afd80 con 0x7f49140d08b0 2024-04-24T06:33:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f491a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f491415fa20 con 0x7f491415c5e0 2024-04-24T06:33:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f4913e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49140d08b0 0x7f49141624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f4913e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49140d08b0 0x7f49141624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46822/0 (socket says 172.21.15.59:46822) 2024-04-24T06:33:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f4913e00640 1 -- 172.21.15.59:0/3421415384 learned_addr learned my addr 172.21.15.59:0/3421415384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.472+0000 7f4912a00640 1 --1- 172.21.15.59:0/3421415384 >> v1:172.21.15.59:6789/0 conn(0x7f49140af920 0x7f49140d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33410/0 (socket says 172.21.15.59:33410) 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913400640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 0x7f49140af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913e00640 1 -- 172.21.15.59:0/3421415384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 msgr2=0x7f49140af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913e00640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 0x7f49140af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913e00640 1 -- 172.21.15.59:0/3421415384 >> v1:172.21.15.59:6789/0 conn(0x7f49140af920 legacy=0x7f49140d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913e00640 1 -- 172.21.15.59:0/3421415384 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491415f3d0 con 0x7f49140d08b0 2024-04-24T06:33:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4912000640 1 -- 172.21.15.59:0/3421415384 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313726189 0 0) 0x7f491415f0f0 con 0x7f49140af920 2024-04-24T06:33:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.473+0000 7f4913400640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 0x7f49140af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.474+0000 7f4913e00640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49140d08b0 0x7f49141624d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cb080989d096ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.474+0000 7f4912000640 1 -- 172.21.15.59:0/3421415384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c036070 con 0x7f49140d08b0 2024-04-24T06:33:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.474+0000 7f4912000640 1 -- 172.21.15.59:0/3421415384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c031600 con 0x7f49140d08b0 2024-04-24T06:33:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.474+0000 7f4912000640 1 -- 172.21.15.59:0/3421415384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0319d0 con 0x7f49140d08b0 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 -- 172.21.15.59:0/3421415384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49140d08b0 msgr2=0x7f49141624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49140d08b0 0x7f49141624d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c02a730 comp rx=0 tx=0).stop 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 -- 172.21.15.59:0/3421415384 shutdown_connections 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49140d08b0 0x7f49141624d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 --2- 172.21.15.59:0/3421415384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f491415c5e0 0x7f49140af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 -- 172.21.15.59:0/3421415384 >> 172.21.15.59:0/3421415384 conn(0x7f4914067ea0 msgr2=0x7f491406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 -- 172.21.15.59:0/3421415384 shutdown_connections 2024-04-24T06:33:45.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.475+0000 7f491a600640 1 -- 172.21.15.59:0/3421415384 wait complete. 2024-04-24T06:33:45.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.477+0000 7f491a600640 1 Processor -- start 2024-04-24T06:33:45.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.477+0000 7f491a600640 1 -- start start 2024-04-24T06:33:45.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 0x7f4914158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49140d08b0 0x7f4914149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f491406e600 con 0x7f491415c5e0 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f491406e480 con 0x7f49140d08b0 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f491a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4914162a10 con 0x7f49140af920 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f4913e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f4913e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46824/0 (socket says 172.21.15.59:46824) 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.478+0000 7f4913e00640 1 -- 172.21.15.59:0/646663331 learned_addr learned my addr 172.21.15.59:0/646663331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4912a00640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49140d08b0 0x7f4914149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913400640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 0x7f4914158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 -- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 msgr2=0x7f4914158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 0x7f4914158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 -- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49140d08b0 msgr2=0x7f4914149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49140d08b0 0x7f4914149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 -- 172.21.15.59:0/646663331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49141550c0 con 0x7f491415c5e0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913400640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 0x7f4914158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.479+0000 7f4913e00640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f490c034040 tx=0x7f490c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.480+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c036050 con 0x7f491415c5e0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.480+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c042070 con 0x7f491415c5e0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.480+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c009510 con 0x7f491415c5e0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.480+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49141553a0 con 0x7f491415c5e0 2024-04-24T06:33:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.480+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4914155960 con 0x7f491415c5e0 2024-04-24T06:33:45.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.481+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4914068a80 con 0x7f491415c5e0 2024-04-24T06:33:45.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.483+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f490c007bf0 con 0x7f491415c5e0 2024-04-24T06:33:45.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.484+0000 7f490b400640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 0x7f48f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.484+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f490c0aca30 con 0x7f491415c5e0 2024-04-24T06:33:45.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.484+0000 7f4913400640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 0x7f48f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.484+0000 7f4913400640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 0x7f48f406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4904006fd0 tx=0x7f4904008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.489+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f490c045360 con 0x7f491415c5e0 2024-04-24T06:33:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.692+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4914160390 con 0x7f491415c5e0 2024-04-24T06:33:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.696+0000 7f490b400640 1 -- 172.21.15.59:0/646663331 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16597 (secure 0 0 0) 0x7f490c07ae50 con 0x7f491415c5e0 2024-04-24T06:33:45.698 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:45.698 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:44.703627+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.698+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 msgr2=0x7f48f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.698+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 0x7f48f406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4904006fd0 tx=0x7f4904008040 comp rx=0 tx=0).stop 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.698+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 msgr2=0x7f4914156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.698+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f490c034040 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 shutdown_connections 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f48f406c8f0 0x7f48f406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f491415c5e0 0x7f4914156160 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49140d08b0 0x7f4914149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 --2- 172.21.15.59:0/646663331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f49140af920 0x7f4914158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 >> 172.21.15.59:0/646663331 conn(0x7f4914067ea0 msgr2=0x7f491415b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 shutdown_connections 2024-04-24T06:33:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.699+0000 7f491a600640 1 -- 172.21.15.59:0/646663331 wait complete. 2024-04-24T06:33:45.719 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.923+0000 7f654f800640 1 Processor -- start 2024-04-24T06:33:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.923+0000 7f654f800640 1 -- start start 2024-04-24T06:33:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.924+0000 7f654f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 0x7f65480d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.924+0000 7f654f800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654815ccc0 con 0x7f65480ce010 2024-04-24T06:33:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.924+0000 7f654f800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65480d1360 con 0x7f654815c8e0 2024-04-24T06:33:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.924+0000 7f654f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65480ccfb0 con 0x7f65480d0a20 2024-04-24T06:33:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f654d800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65480ce010 0x7f65480ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33416/0 (socket says 172.21.15.59:33416) 2024-04-24T06:33:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f654d800640 1 -- 172.21.15.59:0/2971639494 learned_addr learned my addr 172.21.15.59:0/2971639494 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547e00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 0x7f65480d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565148106 0 0) 0x7f654815ccc0 con 0x7f65480ce010 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003520 con 0x7f65480ce010 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3265854148 0 0) 0x7f652c003520 con 0x7f65480ce010 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 msgr2=0x7f65480d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.925+0000 7f6547400640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 0x7f65480d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 >> v1:172.21.15.66:6789/0 conn(0x7f654815c8e0 legacy=0x7f65480b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65480cd250 con 0x7f65480ce010 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6540001670 con 0x7f65480ce010 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 >> v1:172.21.15.59:6789/0 conn(0x7f65480ce010 legacy=0x7f65480ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f652c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 0x7f652c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65480ccfb0 con 0x7f652c0039e0 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f652c00ba70 con 0x7f652c004800 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f652c00bc70 con 0x7f65480d0a20 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f652c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547e00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 -- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 msgr2=0x7f652c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480d0a20 0x7f652c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 -- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f652c004800 msgr2=0x7f652c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f654ce00640 1 -- 172.21.15.59:0/2971639494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c003520 con 0x7f652c0039e0 2024-04-24T06:33:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.926+0000 7f6547e00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f652c004800 0x7f652c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.927+0000 7f654ce00640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f652c003e00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6538001e20 tx=0x7f652c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a678a3d5dad3414 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538001a70 con 0x7f652c0039e0 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653802ae10 con 0x7f652c0039e0 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f6547400640 1 -- 172.21.15.59:0/2971639494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538003bb0 con 0x7f652c0039e0 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654f800640 1 -- 172.21.15.59:0/2971639494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 msgr2=0x7f652c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654f800640 1 --2- 172.21.15.59:0/2971639494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f652c003e00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6538001e20 tx=0x7f652c005830 comp rx=0 tx=0).stop 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654d800640 1 -- 172.21.15.59:0/2971639494 reap_dead start 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654f800640 1 -- 172.21.15.59:0/2971639494 shutdown_connections 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654f800640 1 -- 172.21.15.59:0/2971639494 >> 172.21.15.59:0/2971639494 conn(0x7f65480d25d0 msgr2=0x7f65480d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.928+0000 7f654f800640 1 -- 172.21.15.59:0/2971639494 shutdown_connections 2024-04-24T06:33:45.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.929+0000 7f654f800640 1 -- 172.21.15.59:0/2971639494 wait complete. 2024-04-24T06:33:45.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.930+0000 7f654f800640 1 Processor -- start 2024-04-24T06:33:45.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.930+0000 7f654f800640 1 -- start start 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.931+0000 7f654f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.931+0000 7f654f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480c51e0 0x7f654806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.931+0000 7f654f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f654815f8e0 0x7f65480b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65480cdaf0 con 0x7f652c0039e0 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65480cd810 con 0x7f654815f8e0 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f65480b27f0 con 0x7f65480c51e0 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46842/0 (socket says 172.21.15.59:46842) 2024-04-24T06:33:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 -- 172.21.15.59:0/47498596 learned_addr learned my addr 172.21.15.59:0/47498596 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 -- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480c51e0 msgr2=0x7f654806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480c51e0 0x7f654806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 -- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f654815f8e0 msgr2=0x7f65480b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f654815f8e0 0x7f65480b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 -- 172.21.15.59:0/47498596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654815fd60 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.932+0000 7f654ce00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f653802bfd0 tx=0x7f65380014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.933+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653803f030 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.933+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6538001250 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.933+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654806e900 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.933+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538037040 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.933+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654806d590 con 0x7f652c0039e0 2024-04-24T06:33:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.934+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6548069190 con 0x7f652c0039e0 2024-04-24T06:33:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.935+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6538042070 con 0x7f652c0039e0 2024-04-24T06:33:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.936+0000 7f6544c00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 0x7f652406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.936+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f65380ab7e0 con 0x7f652c0039e0 2024-04-24T06:33:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.936+0000 7f6547e00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 0x7f652406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.937+0000 7f6547e00640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 0x7f652406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65480b3500 tx=0x7f6530008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:45.941+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6538079c80 con 0x7f652c0039e0 2024-04-24T06:33:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.148+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65480b3cb0 con 0x7f652c0039e0 2024-04-24T06:33:46.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.151+0000 7f6544c00640 1 -- 172.21.15.59:0/47498596 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16597 (secure 0 0 0) 0x7f6538079580 con 0x7f652c0039e0 2024-04-24T06:33:46.154 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:46.154 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:44.703627+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 msgr2=0x7f652406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 0x7f652406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65480b3500 tx=0x7f6530008040 comp rx=0 tx=0).stop 2024-04-24T06:33:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 msgr2=0x7f654806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f653802bfd0 tx=0x7f65380014a0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 shutdown_connections 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f652406c8f0 0x7f652406edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f654815f8e0 0x7f65480b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f65480c51e0 0x7f654806cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 --2- 172.21.15.59:0/47498596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c0039e0 0x7f654806dd90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 >> 172.21.15.59:0/47498596 conn(0x7f65480d25d0 msgr2=0x7f6548070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 shutdown_connections 2024-04-24T06:33:46.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.155+0000 7f654f800640 1 -- 172.21.15.59:0/47498596 wait complete. 2024-04-24T06:33:46.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:33:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.399+0000 7f7619800640 1 Processor -- start 2024-04-24T06:33:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.399+0000 7f7619800640 1 -- start start 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7619800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7614072480 0x7f761406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7619800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76140cfc20 con 0x7f76140d0cc0 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7619800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76140cff00 con 0x7f7614145e10 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7619800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f76140d01e0 con 0x7f7614072480 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7612000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f76140d0cc0 0x7f7614071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33428/0 (socket says 172.21.15.59:33428) 2024-04-24T06:33:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7612000640 1 -- 172.21.15.59:0/3027365312 learned_addr learned my addr 172.21.15.59:0/3027365312 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7613400640 1 --2- 172.21.15.59:0/3027365312 >> v2:172.21.15.66:3300/0 conn(0x7f7614072480 0x7f761406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835881321 0 0) 0x7f76140cfc20 con 0x7f76140d0cc0 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.400+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f8003520 con 0x7f76140d0cc0 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2787598543 0 0) 0x7f75f8003520 con 0x7f76140d0cc0 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 >> v2:172.21.15.66:3300/0 conn(0x7f7614072480 msgr2=0x7f761406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 --2- 172.21.15.59:0/3027365312 >> v2:172.21.15.66:3300/0 conn(0x7f7614072480 0x7f761406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 >> v1:172.21.15.66:6789/0 conn(0x7f7614145e10 legacy=0x7f76140cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140d04c0 con 0x7f76140d0cc0 2024-04-24T06:33:46.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7600001670 con 0x7f76140d0cc0 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 >> v1:172.21.15.59:6789/0 conn(0x7f76140d0cc0 legacy=0x7f7614071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 0x7f75f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8004800 0x7f75f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80051c0 0x7f75f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76140d01e0 con 0x7f75f8003a00 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75f800c0c0 con 0x7f75f80051c0 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f75f800c2c0 con 0x7f75f8004800 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 0x7f75f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7613400640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80051c0 0x7f75f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612000640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8004800 0x7f75f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 -- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8004800 msgr2=0x7f75f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8004800 0x7f75f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 -- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80051c0 msgr2=0x7f75f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:46.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80051c0 0x7f75f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7612a00640 1 -- 172.21.15.59:0/3027365312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f8003520 con 0x7f75f8003a00 2024-04-24T06:33:46.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7613400640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75f80051c0 0x7f75f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:46.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.402+0000 7f7613400640 1 -- 172.21.15.59:0/3027365312 reap_dead start 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.403+0000 7f7612a00640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 0x7f75f8003e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7604001750 tx=0x7f75f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f926f83cf658caa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.403+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604001370 con 0x7f75f8003a00 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.403+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76040052d0 con 0x7f75f8003a00 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.403+0000 7f7611600640 1 -- 172.21.15.59:0/3027365312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760402ab50 con 0x7f75f8003a00 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 -- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 msgr2=0x7f75f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 0x7f75f8003e20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7604001750 tx=0x7f75f8005830 comp rx=0 tx=0).stop 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 -- 172.21.15.59:0/3027365312 shutdown_connections 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 --2- 172.21.15.59:0/3027365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f8003a00 0x7f75f8003e20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 -- 172.21.15.59:0/3027365312 >> 172.21.15.59:0/3027365312 conn(0x7f7614067d90 msgr2=0x7f761406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 -- 172.21.15.59:0/3027365312 shutdown_connections 2024-04-24T06:33:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.405+0000 7f7619800640 1 -- 172.21.15.59:0/3027365312 wait complete. 2024-04-24T06:33:46.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.406+0000 7f7619800640 1 Processor -- start 2024-04-24T06:33:46.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.406+0000 7f7619800640 1 -- start start 2024-04-24T06:33:46.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8003a00 0x7f76140ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f761415c290 0x7f76140ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76141461f0 con 0x7f76140aede0 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f761406f130 con 0x7f761415c290 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f76140af780 con 0x7f75f8003a00 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46850/0 (socket says 172.21.15.59:46850) 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 -- 172.21.15.59:0/3621924530 learned_addr learned my addr 172.21.15.59:0/3621924530 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7612a00640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8003a00 0x7f76140ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 -- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8003a00 msgr2=0x7f76140ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8003a00 0x7f76140ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 -- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f761415c290 msgr2=0x7f76140ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f761415c290 0x7f76140ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 -- 172.21.15.59:0/3621924530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140a9940 con 0x7f76140aede0 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7613400640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f76140b3500 tx=0x7f760c0079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c020070 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c018500 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0188b0 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76140acbe0 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.408+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76140ae2c0 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.410+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7614068a80 con 0x7f76140aede0 2024-04-24T06:33:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.410+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f760c018a50 con 0x7f76140aede0 2024-04-24T06:33:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.411+0000 7f760aa00640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 0x7f75f406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.411+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f760c08fbc0 con 0x7f76140aede0 2024-04-24T06:33:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.412+0000 7f7612a00640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 0x7f75f406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:46.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.413+0000 7f7612a00640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 0x7f75f406ee30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7604032000 tx=0x7f760402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:46.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.416+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f760c05e160 con 0x7f76140aede0 2024-04-24T06:33:46.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-24T06:33:46.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-24T06:33:46.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-24T06:33:46.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:46.666+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f76140ace10 con 0x7f76140aede0 2024-04-24T06:33:47.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.266+0000 7f760aa00640 1 -- 172.21.15.59:0/3621924530 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7f760c0930b0 con 0x7f76140aede0 2024-04-24T06:33:47.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.269+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 msgr2=0x7f75f406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.269+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 0x7f75f406ee30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7604032000 tx=0x7f760402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:47.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.269+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 msgr2=0x7f76140af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.269+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f76140b3500 tx=0x7f760c0079f0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 shutdown_connections 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75f406c970 0x7f75f406ee30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76140aede0 0x7f76140af240 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f761415c290 0x7f76140ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 --2- 172.21.15.59:0/3621924530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75f8003a00 0x7f76140ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 >> 172.21.15.59:0/3621924530 conn(0x7f7614067d90 msgr2=0x7f7614071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 shutdown_connections 2024-04-24T06:33:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.270+0000 7f7619800640 1 -- 172.21.15.59:0/3621924530 wait complete. 2024-04-24T06:33:47.294 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:33:47.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:33:47.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.510+0000 7f8240a00640 1 Processor -- start 2024-04-24T06:33:47.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.510+0000 7f8240a00640 1 -- start start 2024-04-24T06:33:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8240a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f823c15ed20 0x7f823c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8240a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 0x7f823c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8240a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c0b5da0 con 0x7f823c0b3600 2024-04-24T06:33:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8240a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f823c15f100 con 0x7f823c15ed20 2024-04-24T06:33:47.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8240a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f823c0d07d0 con 0x7f823c0d13e0 2024-04-24T06:33:47.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8233e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f823c0b3600 0x7f823c0b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33442/0 (socket says 172.21.15.59:33442) 2024-04-24T06:33:47.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f8233e00640 1 -- 172.21.15.59:0/1851047532 learned_addr learned my addr 172.21.15.59:0/1851047532 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:47.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.512+0000 7f823b400640 1 --2- 172.21.15.59:0/1851047532 >> v2:172.21.15.59:3300/0 conn(0x7f823c15ed20 0x7f823c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823be00640 1 --2- 172.21.15.59:0/1851047532 >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 0x7f823c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823aa00640 1 -- 172.21.15.59:0/1851047532 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130727662 0 0) 0x7f823c0b5da0 con 0x7f823c0b3600 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823aa00640 1 -- 172.21.15.59:0/1851047532 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8220003520 con 0x7f823c0b3600 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823b400640 1 -- 172.21.15.59:0/1851047532 >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 msgr2=0x7f823c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823b400640 1 --2- 172.21.15.59:0/1851047532 >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 0x7f823c0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823b400640 1 -- 172.21.15.59:0/1851047532 >> v1:172.21.15.59:6789/0 conn(0x7f823c0b3600 legacy=0x7f823c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823b400640 1 -- 172.21.15.59:0/1851047532 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c0d0a20 con 0x7f823c15ed20 2024-04-24T06:33:47.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.513+0000 7f823be00640 1 --2- 172.21.15.59:0/1851047532 >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 0x7f823c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:33:47.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.515+0000 7f823b400640 1 --2- 172.21.15.59:0/1851047532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c0d0ea0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f823c08e630 tx=0x7f822c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c323dbbbc2fd14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:47.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.515+0000 7f823aa00640 1 -- 172.21.15.59:0/1851047532 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c037070 con 0x7f823c15ed20 2024-04-24T06:33:47.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.515+0000 7f823aa00640 1 -- 172.21.15.59:0/1851047532 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c02add0 con 0x7f823c15ed20 2024-04-24T06:33:47.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.515+0000 7f823aa00640 1 -- 172.21.15.59:0/1851047532 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c032660 con 0x7f823c15ed20 2024-04-24T06:33:47.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 -- 172.21.15.59:0/1851047532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 msgr2=0x7f823c0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 --2- 172.21.15.59:0/1851047532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c0d0ea0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f823c08e630 tx=0x7f822c02a080 comp rx=0 tx=0).stop 2024-04-24T06:33:47.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 -- 172.21.15.59:0/1851047532 shutdown_connections 2024-04-24T06:33:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 --2- 172.21.15.59:0/1851047532 >> v2:172.21.15.66:3300/0 conn(0x7f823c0d13e0 0x7f823c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 --2- 172.21.15.59:0/1851047532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c0d0ea0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 -- 172.21.15.59:0/1851047532 >> 172.21.15.59:0/1851047532 conn(0x7f823c08bf90 msgr2=0x7f823c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.516+0000 7f8240a00640 1 -- 172.21.15.59:0/1851047532 shutdown_connections 2024-04-24T06:33:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.517+0000 7f8240a00640 1 -- 172.21.15.59:0/1851047532 wait complete. 2024-04-24T06:33:47.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.518+0000 7f8240a00640 1 Processor -- start 2024-04-24T06:33:47.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.518+0000 7f8240a00640 1 -- start start 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 0x7f823c1f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 0x7f823c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0d3490 con 0x7f823c15ed20 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0d31b0 con 0x7f823c0d13e0 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.519+0000 7f8240a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f823c0b4620 con 0x7f823c0b3600 2024-04-24T06:33:47.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46868/0 (socket says 172.21.15.59:46868) 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 -- 172.21.15.59:0/1129455760 learned_addr learned my addr 172.21.15.59:0/1129455760 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823b400640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 0x7f823c1f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f8233e00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 0x7f823c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 -- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 msgr2=0x7f823c1f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 0x7f823c1f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 -- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 msgr2=0x7f823c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 0x7f823c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823be00640 1 -- 172.21.15.59:0/1129455760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1f8dd0 con 0x7f823c15ed20 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f823b400640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 0x7f823c1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:47.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.520+0000 7f8233e00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 0x7f823c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:47.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f823be00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8234007ef0 tx=0x7f823400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:47.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823401a070 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8234003b90 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c159400 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234004030 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.521+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c15c540 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.522+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c04fd90 con 0x7f823c15ed20 2024-04-24T06:33:47.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.523+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f823401d070 con 0x7f823c15ed20 2024-04-24T06:33:47.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.525+0000 7f8233400640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:47.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.525+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f823408bdf0 con 0x7f823c15ed20 2024-04-24T06:33:47.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.525+0000 7f823b400640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:47.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.526+0000 7f823b400640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f822c034000 tx=0x7f822c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:47.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.529+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f823c1f8dd0 con 0x7f823c15ed20 2024-04-24T06:33:47.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:47.778+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f823c0b33c0 con 0x7f823c15ed20 2024-04-24T06:33:48.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.269+0000 7f8233400640 1 -- 172.21.15.59:0/1129455760 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f8234010070 con 0x7f823c15ed20 2024-04-24T06:33:48.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 msgr2=0x7f821806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f822c034000 tx=0x7f822c02e000 comp rx=0 tx=0).stop 2024-04-24T06:33:48.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 msgr2=0x7f823c09f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8234007ef0 tx=0x7f823400d4a0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 shutdown_connections 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.272+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f823c15ed20 0x7f823c09f550 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.273+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c0d13e0 0x7f823c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.273+0000 7f8240a00640 1 --2- 172.21.15.59:0/1129455760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f823c0b3600 0x7f823c1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.273+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 >> 172.21.15.59:0/1129455760 conn(0x7f823c08bf90 msgr2=0x7f823c06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.273+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 shutdown_connections 2024-04-24T06:33:48.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.273+0000 7f8240a00640 1 -- 172.21.15.59:0/1129455760 wait complete. 2024-04-24T06:33:48.300 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:33:48.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:48.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.511+0000 7f2412400640 1 Processor -- start 2024-04-24T06:33:48.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.512+0000 7f2412400640 1 -- start start 2024-04-24T06:33:48.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f2412400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 0x7f240c06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f2412400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f240c15ece0 0x7f240c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f2412400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c0cd970 con 0x7f240c0b3080 2024-04-24T06:33:48.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f2412400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f240c0ced20 con 0x7f240c15ece0 2024-04-24T06:33:48.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f2412400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f240c15fcd0 con 0x7f240c0cd050 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f240be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f240c15ece0 0x7f240c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f240be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f240c15ece0 0x7f240c0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46874/0 (socket says 172.21.15.59:46874) 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.513+0000 7f240be00640 1 -- 172.21.15.59:0/2829860224 learned_addr learned my addr 172.21.15.59:0/2829860224 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f2411000640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 0x7f240c06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f240be00640 1 -- 172.21.15.59:0/2829860224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 msgr2=0x7f240c06f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f240be00640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 0x7f240c06f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f240be00640 1 -- 172.21.15.59:0/2829860224 >> v1:172.21.15.66:6789/0 conn(0x7f240c0b3080 legacy=0x7f240c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f240be00640 1 -- 172.21.15.59:0/2829860224 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c15f160 con 0x7f240c15ece0 2024-04-24T06:33:48.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.514+0000 7f2411000640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 0x7f240c06f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:48.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.515+0000 7f240be00640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c15ece0 0x7f240c0cd430 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009870 tx=0x7f23f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3a928e7fefb790 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.515+0000 7f240b400640 1 -- 172.21.15.59:0/2829860224 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8037070 con 0x7f240c15ece0 2024-04-24T06:33:48.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.515+0000 7f240b400640 1 -- 172.21.15.59:0/2829860224 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f8032440 con 0x7f240c15ece0 2024-04-24T06:33:48.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.515+0000 7f240b400640 1 -- 172.21.15.59:0/2829860224 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8032810 con 0x7f240c15ece0 2024-04-24T06:33:48.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 -- 172.21.15.59:0/2829860224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c15ece0 msgr2=0x7f240c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c15ece0 0x7f240c0cd430 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009870 tx=0x7f23f802a180 comp rx=0 tx=0).stop 2024-04-24T06:33:48.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 -- 172.21.15.59:0/2829860224 shutdown_connections 2024-04-24T06:33:48.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c15ece0 0x7f240c0cd430 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 --2- 172.21.15.59:0/2829860224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c0cd050 0x7f240c06f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 -- 172.21.15.59:0/2829860224 >> 172.21.15.59:0/2829860224 conn(0x7f240c0b21a0 msgr2=0x7f240c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:48.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 -- 172.21.15.59:0/2829860224 shutdown_connections 2024-04-24T06:33:48.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.516+0000 7f2412400640 1 -- 172.21.15.59:0/2829860224 wait complete. 2024-04-24T06:33:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.518+0000 7f2412400640 1 Processor -- start 2024-04-24T06:33:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.518+0000 7f2412400640 1 -- start start 2024-04-24T06:33:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 0x7f240c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 0x7f240c09d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f240c15f9f0 con 0x7f240c0b3080 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f240c15f710 con 0x7f240c0cd050 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.519+0000 7f2412400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f240c0cef50 con 0x7f240c15ece0 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46884/0 (socket says 172.21.15.59:46884) 2024-04-24T06:33:48.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 -- 172.21.15.59:0/1583829105 learned_addr learned my addr 172.21.15.59:0/1583829105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:48.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f240be00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 0x7f240c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411a00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 0x7f240c09d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 -- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 msgr2=0x7f240c09d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 0x7f240c09d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 -- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 msgr2=0x7f240c0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 0x7f240c0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411000640 1 -- 172.21.15.59:0/1583829105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c072ab0 con 0x7f240c0b3080 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f240be00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 0x7f240c0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.520+0000 7f2411a00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 0x7f240c09d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2411000640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f240000c7f0 tx=0x7f240000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400015070 con 0x7f240c0b3080 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2400007a30 con 0x7f240c0b3080 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400007e00 con 0x7f240c0b3080 2024-04-24T06:33:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c072d90 con 0x7f240c0b3080 2024-04-24T06:33:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.521+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c073350 con 0x7f240c0b3080 2024-04-24T06:33:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.523+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c04fd90 con 0x7f240c0b3080 2024-04-24T06:33:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.523+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2400018070 con 0x7f240c0b3080 2024-04-24T06:33:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.524+0000 7f2408c00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.524+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f240008bc00 con 0x7f240c0b3080 2024-04-24T06:33:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.525+0000 7f240be00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.525+0000 7f240be00640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.529+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2400059f90 con 0x7f240c0b3080 2024-04-24T06:33:48.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.774+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f240c0b46f0 con 0x7f240c0b3080 2024-04-24T06:33:48.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.775+0000 7f2408c00640 1 -- 172.21.15.59:0/1583829105 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5174 (secure 0 0 0) 0x7f2400059890 con 0x7f240c0b3080 2024-04-24T06:33:48.776 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:48.776 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6759,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:33:47.780106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2419732942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2419732942},{"type":"v1","addr":"172.21.15.66:6835","nonce":2419732942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6771,"qdb_cluster":[6771]},"id":5}]} 2024-04-24T06:33:48.776 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 44 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.777+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 msgr2=0x7f23ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.777+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23f8034000 tx=0x7f23f802e000 comp rx=0 tx=0).stop 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 msgr2=0x7f240c0724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f240000c7f0 tx=0x7f240000cbf0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 shutdown_connections 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f240c15ece0 0x7f240c09d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f240c0cd050 0x7f240c0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 --2- 172.21.15.59:0/1583829105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f240c0b3080 0x7f240c0724c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 >> 172.21.15.59:0/1583829105 conn(0x7f240c0b21a0 msgr2=0x7f240c0ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 shutdown_connections 2024-04-24T06:33:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:48.778+0000 7f2412400640 1 -- 172.21.15.59:0/1583829105 wait complete. 2024-04-24T06:33:48.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:49.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.011+0000 7f1e38200640 1 Processor -- start 2024-04-24T06:33:49.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.011+0000 7f1e38200640 1 -- start start 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e38200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e3015b750 0x7f1e300cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e38200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e300b2f00 con 0x7f1e300b1d60 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e38200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e300b2380 con 0x7f1e300cfe40 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e38200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1e300d02a0 con 0x7f1e3015b750 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e35800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e3015b750 0x7f1e300cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e35800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e3015b750 0x7f1e300cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46378/0 (socket says 172.21.15.59:46378) 2024-04-24T06:33:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.012+0000 7f1e35800640 1 -- 172.21.15.59:0/3948062115 learned_addr learned my addr 172.21.15.59:0/3948062115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.013+0000 7f1e35800640 1 -- 172.21.15.59:0/3948062115 >> v1:172.21.15.66:6789/0 conn(0x7f1e300cfe40 legacy=0x7f1e300b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.013+0000 7f1e35800640 1 -- 172.21.15.59:0/3948062115 >> v1:172.21.15.59:6789/0 conn(0x7f1e300b1d60 legacy=0x7f1e300ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.013+0000 7f1e35800640 1 -- 172.21.15.59:0/3948062115 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e300b2660 con 0x7f1e3015b750 2024-04-24T06:33:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.014+0000 7f1e35800640 1 --2- 172.21.15.59:0/3948062115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e300cf900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d48d66dfcfac7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.014+0000 7f1e2fe00640 1 -- 172.21.15.59:0/3948062115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20037070 con 0x7f1e3015b750 2024-04-24T06:33:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.014+0000 7f1e2fe00640 1 -- 172.21.15.59:0/3948062115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e20032440 con 0x7f1e3015b750 2024-04-24T06:33:49.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.014+0000 7f1e2fe00640 1 -- 172.21.15.59:0/3948062115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20032810 con 0x7f1e3015b750 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 -- 172.21.15.59:0/3948062115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 msgr2=0x7f1e300cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 --2- 172.21.15.59:0/3948062115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e300cf900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2002a210 comp rx=0 tx=0).stop 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 -- 172.21.15.59:0/3948062115 shutdown_connections 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 --2- 172.21.15.59:0/3948062115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e300cf900 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 -- 172.21.15.59:0/3948062115 >> 172.21.15.59:0/3948062115 conn(0x7f1e30073450 msgr2=0x7f1e3009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:49.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 -- 172.21.15.59:0/3948062115 shutdown_connections 2024-04-24T06:33:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.015+0000 7f1e38200640 1 -- 172.21.15.59:0/3948062115 wait complete. 2024-04-24T06:33:49.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.017+0000 7f1e38200640 1 Processor -- start 2024-04-24T06:33:49.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.017+0000 7f1e38200640 1 -- start start 2024-04-24T06:33:49.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e300b1d60 0x7f1e300c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 0x7f1e300c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3015bb30 con 0x7f1e3015b750 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e300b2c20 con 0x7f1e300cfe40 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e38200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1e300aeda0 con 0x7f1e300b1d60 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e36200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e36200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46382/0 (socket says 172.21.15.59:46382) 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.018+0000 7f1e36200640 1 -- 172.21.15.59:0/775014330 learned_addr learned my addr 172.21.15.59:0/775014330 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 -- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e300b1d60 msgr2=0x7f1e300c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e35800640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e300b1d60 0x7f1e300c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e300b1d60 0x7f1e300c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 -- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 msgr2=0x7f1e300c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e34e00640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 0x7f1e300c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 0x7f1e300c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 -- 172.21.15.59:0/775014330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e300c6ae0 con 0x7f1e3015b750 2024-04-24T06:33:49.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e34e00640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 0x7f1e300c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e36200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c7a0 tx=0x7f1e2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003b70 con 0x7f1e3015b750 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e28003d30 con 0x7f1e3015b750 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2800aa90 con 0x7f1e3015b750 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e300c6dc0 con 0x7f1e3015b750 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.019+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e300c7380 con 0x7f1e3015b750 2024-04-24T06:33:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.021+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e30069190 con 0x7f1e3015b750 2024-04-24T06:33:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.022+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e2800ac30 con 0x7f1e3015b750 2024-04-24T06:33:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.022+0000 7f1e2d600640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.023+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f1e2808bb90 con 0x7f1e3015b750 2024-04-24T06:33:49.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.023+0000 7f1e35800640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.024+0000 7f1e35800640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 0x7f1e1006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e20034000 tx=0x7f1e2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.027+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e28059f20 con 0x7f1e3015b750 2024-04-24T06:33:49.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.236+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e3015e8c0 con 0x7f1e3015b750 2024-04-24T06:33:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.238+0000 7f1e2d600640 1 -- 172.21.15.59:0/775014330 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16584 (secure 0 0 0) 0x7f1e28059820 con 0x7f1e3015b750 2024-04-24T06:33:49.239 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:49.240 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:46.242082+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.240+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 msgr2=0x7f1e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.240+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 0x7f1e1006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1e20034000 tx=0x7f1e2002e000 comp rx=0 tx=0).stop 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.240+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 msgr2=0x7f1e301f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.240+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c7a0 tx=0x7f1e2800cc60 comp rx=0 tx=0).stop 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 shutdown_connections 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3015b750 0x7f1e301f47d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e300cfe40 0x7f1e300c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 --2- 172.21.15.59:0/775014330 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1e300b1d60 0x7f1e300c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 >> 172.21.15.59:0/775014330 conn(0x7f1e30073450 msgr2=0x7f1e3006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 shutdown_connections 2024-04-24T06:33:49.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.241+0000 7f1e38200640 1 -- 172.21.15.59:0/775014330 wait complete. 2024-04-24T06:33:49.261 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:33:49.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:33:49.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.472+0000 7f97e6800640 1 Processor -- start 2024-04-24T06:33:49.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.472+0000 7f97e6800640 1 -- start start 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97e6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f97e00cd020 0x7f97e00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97e6800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e00cd520 con 0x7f97e01454e0 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97e6800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e00cd800 con 0x7f97e00b3e40 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97e6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f97e00cddc0 con 0x7f97e00cd020 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97d7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f97e00cd020 0x7f97e00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97df400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f97e01454e0 0x7f97e0145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43960/0 (socket says 172.21.15.59:43960) 2024-04-24T06:33:49.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.474+0000 7f97df400640 1 -- 172.21.15.59:0/998070638 learned_addr learned my addr 172.21.15.59:0/998070638 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.475+0000 7f97d7e00640 1 -- 172.21.15.59:0/998070638 >> v1:172.21.15.66:6789/0 conn(0x7f97e00b3e40 legacy=0x7f97e0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.475+0000 7f97d7e00640 1 -- 172.21.15.59:0/998070638 >> v1:172.21.15.59:6789/0 conn(0x7f97e01454e0 legacy=0x7f97e0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.475+0000 7f97d7e00640 1 -- 172.21.15.59:0/998070638 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e00cdae0 con 0x7f97e00cd020 2024-04-24T06:33:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.475+0000 7f97dea00640 1 -- 172.21.15.59:0/998070638 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324176257 0 0) 0x7f97e00cd520 con 0x7f97e01454e0 2024-04-24T06:33:49.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.476+0000 7f97d7e00640 1 --2- 172.21.15.59:0/998070638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e00cd020 0x7f97e00b3900 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097a0 tx=0x7f97c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a4b96b0080aa04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.476+0000 7f97dea00640 1 -- 172.21.15.59:0/998070638 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8037070 con 0x7f97e00cd020 2024-04-24T06:33:49.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.476+0000 7f97dea00640 1 -- 172.21.15.59:0/998070638 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c802ae50 con 0x7f97e00cd020 2024-04-24T06:33:49.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.476+0000 7f97dea00640 1 -- 172.21.15.59:0/998070638 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8032660 con 0x7f97e00cd020 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 -- 172.21.15.59:0/998070638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e00cd020 msgr2=0x7f97e00b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 --2- 172.21.15.59:0/998070638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e00cd020 0x7f97e00b3900 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097a0 tx=0x7f97c802a140 comp rx=0 tx=0).stop 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 -- 172.21.15.59:0/998070638 shutdown_connections 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 --2- 172.21.15.59:0/998070638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e00cd020 0x7f97e00b3900 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 -- 172.21.15.59:0/998070638 >> 172.21.15.59:0/998070638 conn(0x7f97e0067ea0 msgr2=0x7f97e006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 -- 172.21.15.59:0/998070638 shutdown_connections 2024-04-24T06:33:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.477+0000 7f97e6800640 1 -- 172.21.15.59:0/998070638 wait complete. 2024-04-24T06:33:49.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.479+0000 7f97e6800640 1 Processor -- start 2024-04-24T06:33:49.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.479+0000 7f97e6800640 1 -- start start 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97e00b3e40 0x7f97e0131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97e00cd020 0x7f97e00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0161020 con 0x7f97e01454e0 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01462c0 con 0x7f97e00cd020 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97e6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97e01611a0 con 0x7f97e00b3e40 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46404/0 (socket says 172.21.15.59:46404) 2024-04-24T06:33:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 -- 172.21.15.59:0/3051397885 learned_addr learned my addr 172.21.15.59:0/3051397885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:49.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97df400640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97e00b3e40 0x7f97e0131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 -- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97e00b3e40 msgr2=0x7f97e0131fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97e00b3e40 0x7f97e0131fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.480+0000 7f97dfe00640 1 -- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97e00cd020 msgr2=0x7f97e00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97dfe00640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97e00cd020 0x7f97e00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97dfe00640 1 -- 172.21.15.59:0/3051397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e015fc30 con 0x7f97e01454e0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97dfe00640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c780 tx=0x7f97d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8004b60 con 0x7f97e01454e0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d8004d00 con 0x7f97e01454e0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e0133660 con 0x7f97e01454e0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d800aa90 con 0x7f97e01454e0 2024-04-24T06:33:49.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.481+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e0132b80 con 0x7f97e01454e0 2024-04-24T06:33:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.482+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e0068a80 con 0x7f97e01454e0 2024-04-24T06:33:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.483+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97d800ac30 con 0x7f97e01454e0 2024-04-24T06:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.485+0000 7f97d7400640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.485+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f97d808beb0 con 0x7f97e01454e0 2024-04-24T06:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.485+0000 7f97df400640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.486+0000 7f97df400640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000f60 tx=0x7f97d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.489+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97d8020070 con 0x7f97e01454e0 2024-04-24T06:33:49.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.687+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97e0160030 con 0x7f97bc06c8f0 2024-04-24T06:33:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.706+0000 7f97d7400640 1 -- 172.21.15.59:0/3051397885 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252068 (secure 0 0 0) 0x7f97e0160030 con 0x7f97bc06c8f0 2024-04-24T06:33:49.711 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:49.712 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:33:49.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.710+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 msgr2=0x7f97bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.710+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000f60 tx=0x7f97d0009040 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.710+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 msgr2=0x7f97e00af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.710+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c780 tx=0x7f97d800cc40 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 shutdown_connections 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97e01454e0 0x7f97e00af310 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97e00cd020 0x7f97e00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 --2- 172.21.15.59:0/3051397885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97e00b3e40 0x7f97e0131fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 >> 172.21.15.59:0/3051397885 conn(0x7f97e0067ea0 msgr2=0x7f97e00724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 shutdown_connections 2024-04-24T06:33:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.711+0000 7f97e6800640 1 -- 172.21.15.59:0/3051397885 wait complete. 2024-04-24T06:33:49.732 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-24T06:33:49.290795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257240,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717480,"statfs":{"total":773094113280,"available":772830699520,"internally_reserved":0,"allocated":7970816,"data_stored":4014876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001581"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103328+0000","last_change":"2024-04-24T06:33:45.278721+0000","last_active":"2024-04-24T06:33:47.103328+0000","last_peered":"2024-04-24T06:33:47.103328+0000","last_clean":"2024-04-24T06:33:47.103328+0000","last_became_active":"2024-04-24T06:33:45.275658+0000","last_became_peered":"2024-04-24T06:33:45.275658+0000","last_unstale":"2024-04-24T06:33:47.103328+0000","last_undegraded":"2024-04-24T06:33:47.103328+0000","last_fullsized":"2024-04-24T06:33:47.103328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:37:30.945343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101628+0000","last_change":"2024-04-24T06:33:45.284399+0000","last_active":"2024-04-24T06:33:47.101628+0000","last_peered":"2024-04-24T06:33:47.101628+0000","last_clean":"2024-04-24T06:33:47.101628+0000","last_became_active":"2024-04-24T06:33:45.280645+0000","last_became_peered":"2024-04-24T06:33:45.280645+0000","last_unstale":"2024-04-24T06:33:47.101628+0000","last_undegraded":"2024-04-24T06:33:47.101628+0000","last_fullsized":"2024-04-24T06:33:47.101628+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:15.426306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572275+0000","last_change":"2024-04-24T06:33:45.272543+0000","last_active":"2024-04-24T06:33:46.572275+0000","last_peered":"2024-04-24T06:33:46.572275+0000","last_clean":"2024-04-24T06:33:46.572275+0000","last_became_active":"2024-04-24T06:33:45.272264+0000","last_became_peered":"2024-04-24T06:33:45.272264+0000","last_unstale":"2024-04-24T06:33:46.572275+0000","last_undegraded":"2024-04-24T06:33:46.572275+0000","last_fullsized":"2024-04-24T06:33:46.572275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:23.351151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970036+0000","last_change":"2024-04-24T06:33:45.271249+0000","last_active":"2024-04-24T06:33:46.970036+0000","last_peered":"2024-04-24T06:33:46.970036+0000","last_clean":"2024-04-24T06:33:46.970036+0000","last_became_active":"2024-04-24T06:33:45.268312+0000","last_became_peered":"2024-04-24T06:33:45.268312+0000","last_unstale":"2024-04-24T06:33:46.970036+0000","last_undegraded":"2024-04-24T06:33:46.970036+0000","last_fullsized":"2024-04-24T06:33:46.970036+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:40.753734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257489+0000","last_change":"2024-04-24T06:33:45.270127+0000","last_active":"2024-04-24T06:33:46.257489+0000","last_peered":"2024-04-24T06:33:46.257489+0000","last_clean":"2024-04-24T06:33:46.257489+0000","last_became_active":"2024-04-24T06:33:45.269551+0000","last_became_peered":"2024-04-24T06:33:45.269551+0000","last_unstale":"2024-04-24T06:33:46.257489+0000","last_undegraded":"2024-04-24T06:33:46.257489+0000","last_fullsized":"2024-04-24T06:33:46.257489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:32:32.566961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101100+0000","last_change":"2024-04-24T06:33:45.271505+0000","last_active":"2024-04-24T06:33:47.101100+0000","last_peered":"2024-04-24T06:33:47.101100+0000","last_clean":"2024-04-24T06:33:47.101100+0000","last_became_active":"2024-04-24T06:33:45.271079+0000","last_became_peered":"2024-04-24T06:33:45.271079+0000","last_unstale":"2024-04-24T06:33:47.101100+0000","last_undegraded":"2024-04-24T06:33:47.101100+0000","last_fullsized":"2024-04-24T06:33:47.101100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:47.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265035+0000","last_change":"2024-04-24T06:33:45.261095+0000","last_active":"2024-04-24T06:33:46.265035+0000","last_peered":"2024-04-24T06:33:46.265035+0000","last_clean":"2024-04-24T06:33:46.265035+0000","last_became_active":"2024-04-24T06:33:45.260808+0000","last_became_peered":"2024-04-24T06:33:45.260808+0000","last_unstale":"2024-04-24T06:33:46.265035+0000","last_undegraded":"2024-04-24T06:33:46.265035+0000","last_fullsized":"2024-04-24T06:33:46.265035+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101917+0000","last_change":"2024-04-24T06:33:45.258389+0000","last_active":"2024-04-24T06:33:47.101917+0000","last_peered":"2024-04-24T06:33:47.101917+0000","last_clean":"2024-04-24T06:33:47.101917+0000","last_became_active":"2024-04-24T06:33:45.254715+0000","last_became_peered":"2024-04-24T06:33:45.254715+0000","last_unstale":"2024-04-24T06:33:47.101917+0000","last_undegraded":"2024-04-24T06:33:47.101917+0000","last_fullsized":"2024-04-24T06:33:47.101917+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:50:08.517777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258046+0000","last_change":"2024-04-24T06:33:45.267762+0000","last_active":"2024-04-24T06:33:46.258046+0000","last_peered":"2024-04-24T06:33:46.258046+0000","last_clean":"2024-04-24T06:33:46.258046+0000","last_became_active":"2024-04-24T06:33:45.267069+0000","last_became_peered":"2024-04-24T06:33:45.267069+0000","last_unstale":"2024-04-24T06:33:46.258046+0000","last_undegraded":"2024-04-24T06:33:46.258046+0000","last_fullsized":"2024-04-24T06:33:46.258046+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:34.128584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260652+0000","last_change":"2024-04-24T06:33:45.273706+0000","last_active":"2024-04-24T06:33:46.260652+0000","last_peered":"2024-04-24T06:33:46.260652+0000","last_clean":"2024-04-24T06:33:46.260652+0000","last_became_active":"2024-04-24T06:33:45.271271+0000","last_became_peered":"2024-04-24T06:33:45.271271+0000","last_unstale":"2024-04-24T06:33:46.260652+0000","last_undegraded":"2024-04-24T06:33:46.260652+0000","last_fullsized":"2024-04-24T06:33:46.260652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102776+0000","last_change":"2024-04-24T06:33:45.282134+0000","last_active":"2024-04-24T06:33:47.102776+0000","last_peered":"2024-04-24T06:33:47.102776+0000","last_clean":"2024-04-24T06:33:47.102776+0000","last_became_active":"2024-04-24T06:33:45.281185+0000","last_became_peered":"2024-04-24T06:33:45.281185+0000","last_unstale":"2024-04-24T06:33:47.102776+0000","last_undegraded":"2024-04-24T06:33:47.102776+0000","last_fullsized":"2024-04-24T06:33:47.102776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:13.511841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970096+0000","last_change":"2024-04-24T06:33:45.259322+0000","last_active":"2024-04-24T06:33:46.970096+0000","last_peered":"2024-04-24T06:33:46.970096+0000","last_clean":"2024-04-24T06:33:46.970096+0000","last_became_active":"2024-04-24T06:33:45.258962+0000","last_became_peered":"2024-04-24T06:33:45.258962+0000","last_unstale":"2024-04-24T06:33:46.970096+0000","last_undegraded":"2024-04-24T06:33:46.970096+0000","last_fullsized":"2024-04-24T06:33:46.970096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:52.370033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270556+0000","last_change":"2024-04-24T06:33:45.259803+0000","last_active":"2024-04-24T06:33:46.270556+0000","last_peered":"2024-04-24T06:33:46.270556+0000","last_clean":"2024-04-24T06:33:46.270556+0000","last_became_active":"2024-04-24T06:33:45.258411+0000","last_became_peered":"2024-04-24T06:33:45.258411+0000","last_unstale":"2024-04-24T06:33:46.270556+0000","last_undegraded":"2024-04-24T06:33:46.270556+0000","last_fullsized":"2024-04-24T06:33:46.270556+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258042+0000","last_change":"2024-04-24T06:33:45.260324+0000","last_active":"2024-04-24T06:33:46.258042+0000","last_peered":"2024-04-24T06:33:46.258042+0000","last_clean":"2024-04-24T06:33:46.258042+0000","last_became_active":"2024-04-24T06:33:45.260025+0000","last_became_peered":"2024-04-24T06:33:45.260025+0000","last_unstale":"2024-04-24T06:33:46.258042+0000","last_undegraded":"2024-04-24T06:33:46.258042+0000","last_fullsized":"2024-04-24T06:33:46.258042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:15.543240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.271061+0000","last_change":"2024-04-24T06:33:45.255831+0000","last_active":"2024-04-24T06:33:46.271061+0000","last_peered":"2024-04-24T06:33:46.271061+0000","last_clean":"2024-04-24T06:33:46.271061+0000","last_became_active":"2024-04-24T06:33:45.255530+0000","last_became_peered":"2024-04-24T06:33:45.255530+0000","last_unstale":"2024-04-24T06:33:46.271061+0000","last_undegraded":"2024-04-24T06:33:46.271061+0000","last_fullsized":"2024-04-24T06:33:46.271061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:17:53.778790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259694+0000","last_change":"2024-04-24T06:33:45.273709+0000","last_active":"2024-04-24T06:33:46.259694+0000","last_peered":"2024-04-24T06:33:46.259694+0000","last_clean":"2024-04-24T06:33:46.259694+0000","last_became_active":"2024-04-24T06:33:45.273348+0000","last_became_peered":"2024-04-24T06:33:45.273348+0000","last_unstale":"2024-04-24T06:33:46.259694+0000","last_undegraded":"2024-04-24T06:33:46.259694+0000","last_fullsized":"2024-04-24T06:33:46.259694+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:16.918794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258254+0000","last_change":"2024-04-24T06:33:45.267392+0000","last_active":"2024-04-24T06:33:46.258254+0000","last_peered":"2024-04-24T06:33:46.258254+0000","last_clean":"2024-04-24T06:33:46.258254+0000","last_became_active":"2024-04-24T06:33:45.266895+0000","last_became_peered":"2024-04-24T06:33:45.266895+0000","last_unstale":"2024-04-24T06:33:46.258254+0000","last_undegraded":"2024-04-24T06:33:46.258254+0000","last_fullsized":"2024-04-24T06:33:46.258254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:22:41.989654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969951+0000","last_change":"2024-04-24T06:33:45.264820+0000","last_active":"2024-04-24T06:33:46.969951+0000","last_peered":"2024-04-24T06:33:46.969951+0000","last_clean":"2024-04-24T06:33:46.969951+0000","last_became_active":"2024-04-24T06:33:45.263855+0000","last_became_peered":"2024-04-24T06:33:45.263855+0000","last_unstale":"2024-04-24T06:33:46.969951+0000","last_undegraded":"2024-04-24T06:33:46.969951+0000","last_fullsized":"2024-04-24T06:33:46.969951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:57.985877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103442+0000","last_change":"2024-04-24T06:33:45.257278+0000","last_active":"2024-04-24T06:33:47.103442+0000","last_peered":"2024-04-24T06:33:47.103442+0000","last_clean":"2024-04-24T06:33:47.103442+0000","last_became_active":"2024-04-24T06:33:45.256984+0000","last_became_peered":"2024-04-24T06:33:45.256984+0000","last_unstale":"2024-04-24T06:33:47.103442+0000","last_undegraded":"2024-04-24T06:33:47.103442+0000","last_fullsized":"2024-04-24T06:33:47.103442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:47.797889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270731+0000","last_change":"2024-04-24T06:33:45.261637+0000","last_active":"2024-04-24T06:33:46.270731+0000","last_peered":"2024-04-24T06:33:46.270731+0000","last_clean":"2024-04-24T06:33:46.270731+0000","last_became_active":"2024-04-24T06:33:45.261314+0000","last_became_peered":"2024-04-24T06:33:45.261314+0000","last_unstale":"2024-04-24T06:33:46.270731+0000","last_undegraded":"2024-04-24T06:33:46.270731+0000","last_fullsized":"2024-04-24T06:33:46.270731+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259792+0000","last_change":"2024-04-24T06:33:45.279300+0000","last_active":"2024-04-24T06:33:46.259792+0000","last_peered":"2024-04-24T06:33:46.259792+0000","last_clean":"2024-04-24T06:33:46.259792+0000","last_became_active":"2024-04-24T06:33:45.278698+0000","last_became_peered":"2024-04-24T06:33:45.278698+0000","last_unstale":"2024-04-24T06:33:46.259792+0000","last_undegraded":"2024-04-24T06:33:46.259792+0000","last_fullsized":"2024-04-24T06:33:46.259792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:23.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259801+0000","last_change":"2024-04-24T06:33:45.268393+0000","last_active":"2024-04-24T06:33:46.259801+0000","last_peered":"2024-04-24T06:33:46.259801+0000","last_clean":"2024-04-24T06:33:46.259801+0000","last_became_active":"2024-04-24T06:33:45.268057+0000","last_became_peered":"2024-04-24T06:33:45.268057+0000","last_unstale":"2024-04-24T06:33:46.259801+0000","last_undegraded":"2024-04-24T06:33:46.259801+0000","last_fullsized":"2024-04-24T06:33:46.259801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:12:24.381678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101159+0000","last_change":"2024-04-24T06:33:45.266906+0000","last_active":"2024-04-24T06:33:47.101159+0000","last_peered":"2024-04-24T06:33:47.101159+0000","last_clean":"2024-04-24T06:33:47.101159+0000","last_became_active":"2024-04-24T06:33:45.266580+0000","last_became_peered":"2024-04-24T06:33:45.266580+0000","last_unstale":"2024-04-24T06:33:47.101159+0000","last_undegraded":"2024-04-24T06:33:47.101159+0000","last_fullsized":"2024-04-24T06:33:47.101159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:13.995575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100101+0000","last_change":"2024-04-24T06:33:45.284150+0000","last_active":"2024-04-24T06:33:47.100101+0000","last_peered":"2024-04-24T06:33:47.100101+0000","last_clean":"2024-04-24T06:33:47.100101+0000","last_became_active":"2024-04-24T06:33:45.283865+0000","last_became_peered":"2024-04-24T06:33:45.283865+0000","last_unstale":"2024-04-24T06:33:47.100101+0000","last_undegraded":"2024-04-24T06:33:47.100101+0000","last_fullsized":"2024-04-24T06:33:47.100101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:21:05.963617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260679+0000","last_change":"2024-04-24T06:33:45.274121+0000","last_active":"2024-04-24T06:33:46.260679+0000","last_peered":"2024-04-24T06:33:46.260679+0000","last_clean":"2024-04-24T06:33:46.260679+0000","last_became_active":"2024-04-24T06:33:45.273202+0000","last_became_peered":"2024-04-24T06:33:45.273202+0000","last_unstale":"2024-04-24T06:33:46.260679+0000","last_undegraded":"2024-04-24T06:33:46.260679+0000","last_fullsized":"2024-04-24T06:33:46.260679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265744+0000","last_change":"2024-04-24T06:33:45.276246+0000","last_active":"2024-04-24T06:33:46.265744+0000","last_peered":"2024-04-24T06:33:46.265744+0000","last_clean":"2024-04-24T06:33:46.265744+0000","last_became_active":"2024-04-24T06:33:45.269723+0000","last_became_peered":"2024-04-24T06:33:45.269723+0000","last_unstale":"2024-04-24T06:33:46.265744+0000","last_undegraded":"2024-04-24T06:33:46.265744+0000","last_fullsized":"2024-04-24T06:33:46.265744+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:30.304735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572563+0000","last_change":"2024-04-24T06:33:45.273920+0000","last_active":"2024-04-24T06:33:46.572563+0000","last_peered":"2024-04-24T06:33:46.572563+0000","last_clean":"2024-04-24T06:33:46.572563+0000","last_became_active":"2024-04-24T06:33:45.273603+0000","last_became_peered":"2024-04-24T06:33:45.273603+0000","last_unstale":"2024-04-24T06:33:46.572563+0000","last_undegraded":"2024-04-24T06:33:46.572563+0000","last_fullsized":"2024-04-24T06:33:46.572563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:22.433498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260228+0000","last_change":"2024-04-24T06:33:45.271766+0000","last_active":"2024-04-24T06:33:46.260228+0000","last_peered":"2024-04-24T06:33:46.260228+0000","last_clean":"2024-04-24T06:33:46.260228+0000","last_became_active":"2024-04-24T06:33:45.271351+0000","last_became_peered":"2024-04-24T06:33:45.271351+0000","last_unstale":"2024-04-24T06:33:46.260228+0000","last_undegraded":"2024-04-24T06:33:46.260228+0000","last_fullsized":"2024-04-24T06:33:46.260228+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572144+0000","last_change":"2024-04-24T06:33:45.266267+0000","last_active":"2024-04-24T06:33:46.572144+0000","last_peered":"2024-04-24T06:33:46.572144+0000","last_clean":"2024-04-24T06:33:46.572144+0000","last_became_active":"2024-04-24T06:33:45.265581+0000","last_became_peered":"2024-04-24T06:33:45.265581+0000","last_unstale":"2024-04-24T06:33:46.572144+0000","last_undegraded":"2024-04-24T06:33:46.572144+0000","last_fullsized":"2024-04-24T06:33:46.572144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:52.823391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265278+0000","last_change":"2024-04-24T06:33:45.270105+0000","last_active":"2024-04-24T06:33:46.265278+0000","last_peered":"2024-04-24T06:33:46.265278+0000","last_clean":"2024-04-24T06:33:46.265278+0000","last_became_active":"2024-04-24T06:33:45.269462+0000","last_became_peered":"2024-04-24T06:33:45.269462+0000","last_unstale":"2024-04-24T06:33:46.265278+0000","last_undegraded":"2024-04-24T06:33:46.265278+0000","last_fullsized":"2024-04-24T06:33:46.265278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265385+0000","last_change":"2024-04-24T06:33:45.259599+0000","last_active":"2024-04-24T06:33:46.265385+0000","last_peered":"2024-04-24T06:33:46.265385+0000","last_clean":"2024-04-24T06:33:46.265385+0000","last_became_active":"2024-04-24T06:33:45.258490+0000","last_became_peered":"2024-04-24T06:33:45.258490+0000","last_unstale":"2024-04-24T06:33:46.265385+0000","last_undegraded":"2024-04-24T06:33:46.265385+0000","last_fullsized":"2024-04-24T06:33:46.265385+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:57.581510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102355+0000","last_change":"2024-04-24T06:33:45.262083+0000","last_active":"2024-04-24T06:33:47.102355+0000","last_peered":"2024-04-24T06:33:47.102355+0000","last_clean":"2024-04-24T06:33:47.102355+0000","last_became_active":"2024-04-24T06:33:45.261767+0000","last_became_peered":"2024-04-24T06:33:45.261767+0000","last_unstale":"2024-04-24T06:33:47.102355+0000","last_undegraded":"2024-04-24T06:33:47.102355+0000","last_fullsized":"2024-04-24T06:33:47.102355+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:54:59.436903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103233+0000","last_change":"2024-04-24T06:33:45.290790+0000","last_active":"2024-04-24T06:33:47.103233+0000","last_peered":"2024-04-24T06:33:47.103233+0000","last_clean":"2024-04-24T06:33:47.103233+0000","last_became_active":"2024-04-24T06:33:45.290379+0000","last_became_peered":"2024-04-24T06:33:45.290379+0000","last_unstale":"2024-04-24T06:33:47.103233+0000","last_undegraded":"2024-04-24T06:33:47.103233+0000","last_fullsized":"2024-04-24T06:33:47.103233+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:21.675900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103873+0000","last_change":"2024-04-24T06:33:44.266860+0000","last_active":"2024-04-24T06:33:47.103873+0000","last_peered":"2024-04-24T06:33:47.103873+0000","last_clean":"2024-04-24T06:33:47.103873+0000","last_became_active":"2024-04-24T06:33:44.266534+0000","last_became_peered":"2024-04-24T06:33:44.266534+0000","last_unstale":"2024-04-24T06:33:47.103873+0000","last_undegraded":"2024-04-24T06:33:47.103873+0000","last_fullsized":"2024-04-24T06:33:47.103873+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:19.836312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258541+0000","last_change":"2024-04-24T06:33:45.267173+0000","last_active":"2024-04-24T06:33:46.258541+0000","last_peered":"2024-04-24T06:33:46.258541+0000","last_clean":"2024-04-24T06:33:46.258541+0000","last_became_active":"2024-04-24T06:33:45.266532+0000","last_became_peered":"2024-04-24T06:33:45.266532+0000","last_unstale":"2024-04-24T06:33:46.258541+0000","last_undegraded":"2024-04-24T06:33:46.258541+0000","last_fullsized":"2024-04-24T06:33:46.258541+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:06.906420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270835+0000","last_change":"2024-04-24T06:33:44.252531+0000","last_active":"2024-04-24T06:33:46.270835+0000","last_peered":"2024-04-24T06:33:46.270835+0000","last_clean":"2024-04-24T06:33:46.270835+0000","last_became_active":"2024-04-24T06:33:44.251855+0000","last_became_peered":"2024-04-24T06:33:44.251855+0000","last_unstale":"2024-04-24T06:33:46.270835+0000","last_undegraded":"2024-04-24T06:33:46.270835+0000","last_fullsized":"2024-04-24T06:33:46.270835+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:45:18.118468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102770+0000","last_change":"2024-04-24T06:33:45.282852+0000","last_active":"2024-04-24T06:33:47.102770+0000","last_peered":"2024-04-24T06:33:47.102770+0000","last_clean":"2024-04-24T06:33:47.102770+0000","last_became_active":"2024-04-24T06:33:45.281556+0000","last_became_peered":"2024-04-24T06:33:45.281556+0000","last_unstale":"2024-04-24T06:33:47.102770+0000","last_undegraded":"2024-04-24T06:33:47.102770+0000","last_fullsized":"2024-04-24T06:33:47.102770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:31:37.535227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260357+0000","last_change":"2024-04-24T06:33:44.266047+0000","last_active":"2024-04-24T06:33:46.260357+0000","last_peered":"2024-04-24T06:33:46.260357+0000","last_clean":"2024-04-24T06:33:46.260357+0000","last_became_active":"2024-04-24T06:33:44.264688+0000","last_became_peered":"2024-04-24T06:33:44.264688+0000","last_unstale":"2024-04-24T06:33:46.260357+0000","last_undegraded":"2024-04-24T06:33:46.260357+0000","last_fullsized":"2024-04-24T06:33:46.260357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:14.409639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102843+0000","last_change":"2024-04-24T06:33:45.275637+0000","last_active":"2024-04-24T06:33:47.102843+0000","last_peered":"2024-04-24T06:33:47.102843+0000","last_clean":"2024-04-24T06:33:47.102843+0000","last_became_active":"2024-04-24T06:33:45.275244+0000","last_became_peered":"2024-04-24T06:33:45.275244+0000","last_unstale":"2024-04-24T06:33:47.102843+0000","last_undegraded":"2024-04-24T06:33:47.102843+0000","last_fullsized":"2024-04-24T06:33:47.102843+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:55:27.775478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103336+0000","last_change":"2024-04-24T06:33:44.263749+0000","last_active":"2024-04-24T06:33:47.103336+0000","last_peered":"2024-04-24T06:33:47.103336+0000","last_clean":"2024-04-24T06:33:47.103336+0000","last_became_active":"2024-04-24T06:33:44.263434+0000","last_became_peered":"2024-04-24T06:33:44.263434+0000","last_unstale":"2024-04-24T06:33:47.103336+0000","last_undegraded":"2024-04-24T06:33:47.103336+0000","last_fullsized":"2024-04-24T06:33:47.103336+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:23:05.090830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265650+0000","last_change":"2024-04-24T06:33:45.273462+0000","last_active":"2024-04-24T06:33:46.265650+0000","last_peered":"2024-04-24T06:33:46.265650+0000","last_clean":"2024-04-24T06:33:46.265650+0000","last_became_active":"2024-04-24T06:33:45.273148+0000","last_became_peered":"2024-04-24T06:33:45.273148+0000","last_unstale":"2024-04-24T06:33:46.265650+0000","last_undegraded":"2024-04-24T06:33:46.265650+0000","last_fullsized":"2024-04-24T06:33:46.265650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:17:53.778790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571992+0000","last_change":"2024-04-24T06:33:44.252064+0000","last_active":"2024-04-24T06:33:46.571992+0000","last_peered":"2024-04-24T06:33:46.571992+0000","last_clean":"2024-04-24T06:33:46.571992+0000","last_became_active":"2024-04-24T06:33:44.251738+0000","last_became_peered":"2024-04-24T06:33:44.251738+0000","last_unstale":"2024-04-24T06:33:46.571992+0000","last_undegraded":"2024-04-24T06:33:46.571992+0000","last_fullsized":"2024-04-24T06:33:46.571992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:44.971036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260047+0000","last_change":"2024-04-24T06:33:45.267452+0000","last_active":"2024-04-24T06:33:46.260047+0000","last_peered":"2024-04-24T06:33:46.260047+0000","last_clean":"2024-04-24T06:33:46.260047+0000","last_became_active":"2024-04-24T06:33:45.266997+0000","last_became_peered":"2024-04-24T06:33:45.266997+0000","last_unstale":"2024-04-24T06:33:46.260047+0000","last_undegraded":"2024-04-24T06:33:46.260047+0000","last_fullsized":"2024-04-24T06:33:46.260047+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:39.619240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101257+0000","last_change":"2024-04-24T06:33:44.258497+0000","last_active":"2024-04-24T06:33:47.101257+0000","last_peered":"2024-04-24T06:33:47.101257+0000","last_clean":"2024-04-24T06:33:47.101257+0000","last_became_active":"2024-04-24T06:33:44.256889+0000","last_became_peered":"2024-04-24T06:33:44.256889+0000","last_unstale":"2024-04-24T06:33:47.101257+0000","last_undegraded":"2024-04-24T06:33:47.101257+0000","last_fullsized":"2024-04-24T06:33:47.101257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:15.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257865+0000","last_change":"2024-04-24T06:33:45.267148+0000","last_active":"2024-04-24T06:33:46.257865+0000","last_peered":"2024-04-24T06:33:46.257865+0000","last_clean":"2024-04-24T06:33:46.257865+0000","last_became_active":"2024-04-24T06:33:45.264815+0000","last_became_peered":"2024-04-24T06:33:45.264815+0000","last_unstale":"2024-04-24T06:33:46.257865+0000","last_undegraded":"2024-04-24T06:33:46.257865+0000","last_fullsized":"2024-04-24T06:33:46.257865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:22.950993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969109+0000","last_change":"2024-04-24T06:33:44.254391+0000","last_active":"2024-04-24T06:33:46.969109+0000","last_peered":"2024-04-24T06:33:46.969109+0000","last_clean":"2024-04-24T06:33:46.969109+0000","last_became_active":"2024-04-24T06:33:44.254127+0000","last_became_peered":"2024-04-24T06:33:44.254127+0000","last_unstale":"2024-04-24T06:33:46.969109+0000","last_undegraded":"2024-04-24T06:33:46.969109+0000","last_fullsized":"2024-04-24T06:33:46.969109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:39:22.504664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270467+0000","last_change":"2024-04-24T06:33:45.263533+0000","last_active":"2024-04-24T06:33:46.270467+0000","last_peered":"2024-04-24T06:33:46.270467+0000","last_clean":"2024-04-24T06:33:46.270467+0000","last_became_active":"2024-04-24T06:33:45.263131+0000","last_became_peered":"2024-04-24T06:33:45.263131+0000","last_unstale":"2024-04-24T06:33:46.270467+0000","last_undegraded":"2024-04-24T06:33:46.270467+0000","last_fullsized":"2024-04-24T06:33:46.270467+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257292+0000","last_change":"2024-04-24T06:33:44.248872+0000","last_active":"2024-04-24T06:33:46.257292+0000","last_peered":"2024-04-24T06:33:46.257292+0000","last_clean":"2024-04-24T06:33:46.257292+0000","last_became_active":"2024-04-24T06:33:44.248584+0000","last_became_peered":"2024-04-24T06:33:44.248584+0000","last_unstale":"2024-04-24T06:33:46.257292+0000","last_undegraded":"2024-04-24T06:33:46.257292+0000","last_fullsized":"2024-04-24T06:33:46.257292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:57.686898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265752+0000","last_change":"2024-04-24T06:33:45.277803+0000","last_active":"2024-04-24T06:33:46.265752+0000","last_peered":"2024-04-24T06:33:46.265752+0000","last_clean":"2024-04-24T06:33:46.265752+0000","last_became_active":"2024-04-24T06:33:45.276196+0000","last_became_peered":"2024-04-24T06:33:45.276196+0000","last_unstale":"2024-04-24T06:33:46.265752+0000","last_undegraded":"2024-04-24T06:33:46.265752+0000","last_fullsized":"2024-04-24T06:33:46.265752+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:21:05.963617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260537+0000","last_change":"2024-04-24T06:33:44.244072+0000","last_active":"2024-04-24T06:33:46.260537+0000","last_peered":"2024-04-24T06:33:46.260537+0000","last_clean":"2024-04-24T06:33:46.260537+0000","last_became_active":"2024-04-24T06:33:44.243459+0000","last_became_peered":"2024-04-24T06:33:44.243459+0000","last_unstale":"2024-04-24T06:33:46.260537+0000","last_undegraded":"2024-04-24T06:33:46.260537+0000","last_fullsized":"2024-04-24T06:33:46.260537+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:46.781222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103723+0000","last_change":"2024-04-24T06:33:45.289439+0000","last_active":"2024-04-24T06:33:47.103723+0000","last_peered":"2024-04-24T06:33:47.103723+0000","last_clean":"2024-04-24T06:33:47.103723+0000","last_became_active":"2024-04-24T06:33:45.288880+0000","last_became_peered":"2024-04-24T06:33:45.288880+0000","last_unstale":"2024-04-24T06:33:47.103723+0000","last_undegraded":"2024-04-24T06:33:47.103723+0000","last_fullsized":"2024-04-24T06:33:47.103723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:24.944322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101955+0000","last_change":"2024-04-24T06:33:44.262780+0000","last_active":"2024-04-24T06:33:47.101955+0000","last_peered":"2024-04-24T06:33:47.101955+0000","last_clean":"2024-04-24T06:33:47.101955+0000","last_became_active":"2024-04-24T06:33:44.262383+0000","last_became_peered":"2024-04-24T06:33:44.262383+0000","last_unstale":"2024-04-24T06:33:47.101955+0000","last_undegraded":"2024-04-24T06:33:47.101955+0000","last_fullsized":"2024-04-24T06:33:47.101955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:06:05.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103229+0000","last_change":"2024-04-24T06:33:45.279688+0000","last_active":"2024-04-24T06:33:47.103229+0000","last_peered":"2024-04-24T06:33:47.103229+0000","last_clean":"2024-04-24T06:33:47.103229+0000","last_became_active":"2024-04-24T06:33:45.276726+0000","last_became_peered":"2024-04-24T06:33:45.276726+0000","last_unstale":"2024-04-24T06:33:47.103229+0000","last_undegraded":"2024-04-24T06:33:47.103229+0000","last_fullsized":"2024-04-24T06:33:47.103229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:24.899489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572381+0000","last_change":"2024-04-24T06:33:44.255196+0000","last_active":"2024-04-24T06:33:46.572381+0000","last_peered":"2024-04-24T06:33:46.572381+0000","last_clean":"2024-04-24T06:33:46.572381+0000","last_became_active":"2024-04-24T06:33:44.254124+0000","last_became_peered":"2024-04-24T06:33:44.254124+0000","last_unstale":"2024-04-24T06:33:46.572381+0000","last_undegraded":"2024-04-24T06:33:46.572381+0000","last_fullsized":"2024-04-24T06:33:46.572381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:23.927655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258089+0000","last_change":"2024-04-24T06:33:45.254225+0000","last_active":"2024-04-24T06:33:46.258089+0000","last_peered":"2024-04-24T06:33:46.258089+0000","last_clean":"2024-04-24T06:33:46.258089+0000","last_became_active":"2024-04-24T06:33:45.253464+0000","last_became_peered":"2024-04-24T06:33:45.253464+0000","last_unstale":"2024-04-24T06:33:46.258089+0000","last_undegraded":"2024-04-24T06:33:46.258089+0000","last_fullsized":"2024-04-24T06:33:46.258089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:01.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970239+0000","last_change":"2024-04-24T06:33:44.253343+0000","last_active":"2024-04-24T06:33:46.970239+0000","last_peered":"2024-04-24T06:33:46.970239+0000","last_clean":"2024-04-24T06:33:46.970239+0000","last_became_active":"2024-04-24T06:33:44.253004+0000","last_became_peered":"2024-04-24T06:33:44.253004+0000","last_unstale":"2024-04-24T06:33:46.970239+0000","last_undegraded":"2024-04-24T06:33:46.970239+0000","last_fullsized":"2024-04-24T06:33:46.970239+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:58:37.073804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101631+0000","last_change":"2024-04-24T06:33:45.275985+0000","last_active":"2024-04-24T06:33:47.101631+0000","last_peered":"2024-04-24T06:33:47.101631+0000","last_clean":"2024-04-24T06:33:47.101631+0000","last_became_active":"2024-04-24T06:33:45.275564+0000","last_became_peered":"2024-04-24T06:33:45.275564+0000","last_unstale":"2024-04-24T06:33:47.101631+0000","last_undegraded":"2024-04-24T06:33:47.101631+0000","last_fullsized":"2024-04-24T06:33:47.101631+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:57:57.176251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970007+0000","last_change":"2024-04-24T06:33:44.265007+0000","last_active":"2024-04-24T06:33:46.970007+0000","last_peered":"2024-04-24T06:33:46.970007+0000","last_clean":"2024-04-24T06:33:46.970007+0000","last_became_active":"2024-04-24T06:33:44.264636+0000","last_became_peered":"2024-04-24T06:33:44.264636+0000","last_unstale":"2024-04-24T06:33:46.970007+0000","last_undegraded":"2024-04-24T06:33:46.970007+0000","last_fullsized":"2024-04-24T06:33:46.970007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:06:05.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265377+0000","last_change":"2024-04-24T06:33:45.276693+0000","last_active":"2024-04-24T06:33:46.265377+0000","last_peered":"2024-04-24T06:33:46.265377+0000","last_clean":"2024-04-24T06:33:46.265377+0000","last_became_active":"2024-04-24T06:33:45.276397+0000","last_became_peered":"2024-04-24T06:33:45.276397+0000","last_unstale":"2024-04-24T06:33:46.265377+0000","last_undegraded":"2024-04-24T06:33:46.265377+0000","last_fullsized":"2024-04-24T06:33:46.265377+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101610+0000","last_change":"2024-04-24T06:33:44.261485+0000","last_active":"2024-04-24T06:33:47.101610+0000","last_peered":"2024-04-24T06:33:47.101610+0000","last_clean":"2024-04-24T06:33:47.101610+0000","last_became_active":"2024-04-24T06:33:44.260284+0000","last_became_peered":"2024-04-24T06:33:44.260284+0000","last_unstale":"2024-04-24T06:33:47.101610+0000","last_undegraded":"2024-04-24T06:33:47.101610+0000","last_fullsized":"2024-04-24T06:33:47.101610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:34:46.718006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270627+0000","last_change":"2024-04-24T06:33:45.257746+0000","last_active":"2024-04-24T06:33:46.270627+0000","last_peered":"2024-04-24T06:33:46.270627+0000","last_clean":"2024-04-24T06:33:46.270627+0000","last_became_active":"2024-04-24T06:33:45.257463+0000","last_became_peered":"2024-04-24T06:33:45.257463+0000","last_unstale":"2024-04-24T06:33:46.270627+0000","last_undegraded":"2024-04-24T06:33:46.270627+0000","last_fullsized":"2024-04-24T06:33:46.270627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265105+0000","last_change":"2024-04-24T06:33:44.259184+0000","last_active":"2024-04-24T06:33:46.265105+0000","last_peered":"2024-04-24T06:33:46.265105+0000","last_clean":"2024-04-24T06:33:46.265105+0000","last_became_active":"2024-04-24T06:33:44.258801+0000","last_became_peered":"2024-04-24T06:33:44.258801+0000","last_unstale":"2024-04-24T06:33:46.265105+0000","last_undegraded":"2024-04-24T06:33:46.265105+0000","last_fullsized":"2024-04-24T06:33:46.265105+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:23.882822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257721+0000","last_change":"2024-04-24T06:33:45.262676+0000","last_active":"2024-04-24T06:33:46.257721+0000","last_peered":"2024-04-24T06:33:46.257721+0000","last_clean":"2024-04-24T06:33:46.257721+0000","last_became_active":"2024-04-24T06:33:45.262306+0000","last_became_peered":"2024-04-24T06:33:45.262306+0000","last_unstale":"2024-04-24T06:33:46.257721+0000","last_undegraded":"2024-04-24T06:33:46.257721+0000","last_fullsized":"2024-04-24T06:33:46.257721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:26:07.882700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259473+0000","last_change":"2024-04-24T06:33:44.266404+0000","last_active":"2024-04-24T06:33:46.259473+0000","last_peered":"2024-04-24T06:33:46.259473+0000","last_clean":"2024-04-24T06:33:46.259473+0000","last_became_active":"2024-04-24T06:33:44.266098+0000","last_became_peered":"2024-04-24T06:33:44.266098+0000","last_unstale":"2024-04-24T06:33:46.259473+0000","last_undegraded":"2024-04-24T06:33:46.259473+0000","last_fullsized":"2024-04-24T06:33:46.259473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:51.353366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260762+0000","last_change":"2024-04-24T06:33:45.274710+0000","last_active":"2024-04-24T06:33:46.260762+0000","last_peered":"2024-04-24T06:33:46.260762+0000","last_clean":"2024-04-24T06:33:46.260762+0000","last_became_active":"2024-04-24T06:33:45.274127+0000","last_became_peered":"2024-04-24T06:33:45.274127+0000","last_unstale":"2024-04-24T06:33:46.260762+0000","last_undegraded":"2024-04-24T06:33:46.260762+0000","last_fullsized":"2024-04-24T06:33:46.260762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101693+0000","last_change":"2024-04-24T06:33:44.258870+0000","last_active":"2024-04-24T06:33:47.101693+0000","last_peered":"2024-04-24T06:33:47.101693+0000","last_clean":"2024-04-24T06:33:47.101693+0000","last_became_active":"2024-04-24T06:33:44.258575+0000","last_became_peered":"2024-04-24T06:33:44.258575+0000","last_unstale":"2024-04-24T06:33:47.101693+0000","last_undegraded":"2024-04-24T06:33:47.101693+0000","last_fullsized":"2024-04-24T06:33:47.101693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:00.397778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103771+0000","last_change":"2024-04-24T06:33:45.278975+0000","last_active":"2024-04-24T06:33:47.103771+0000","last_peered":"2024-04-24T06:33:47.103771+0000","last_clean":"2024-04-24T06:33:47.103771+0000","last_became_active":"2024-04-24T06:33:45.277425+0000","last_became_peered":"2024-04-24T06:33:45.277425+0000","last_unstale":"2024-04-24T06:33:47.103771+0000","last_undegraded":"2024-04-24T06:33:47.103771+0000","last_fullsized":"2024-04-24T06:33:47.103771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:45.987702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260747+0000","last_change":"2024-04-24T06:33:44.260055+0000","last_active":"2024-04-24T06:33:46.260747+0000","last_peered":"2024-04-24T06:33:46.260747+0000","last_clean":"2024-04-24T06:33:46.260747+0000","last_became_active":"2024-04-24T06:33:44.259623+0000","last_became_peered":"2024-04-24T06:33:44.259623+0000","last_unstale":"2024-04-24T06:33:46.260747+0000","last_undegraded":"2024-04-24T06:33:46.260747+0000","last_fullsized":"2024-04-24T06:33:46.260747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:02.627841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970456+0000","last_change":"2024-04-24T06:33:45.257168+0000","last_active":"2024-04-24T06:33:46.970456+0000","last_peered":"2024-04-24T06:33:46.970456+0000","last_clean":"2024-04-24T06:33:46.970456+0000","last_became_active":"2024-04-24T06:33:45.256681+0000","last_became_peered":"2024-04-24T06:33:45.256681+0000","last_unstale":"2024-04-24T06:33:46.970456+0000","last_undegraded":"2024-04-24T06:33:46.970456+0000","last_fullsized":"2024-04-24T06:33:46.970456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:47.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103778+0000","last_change":"2024-04-24T06:33:44.263585+0000","last_active":"2024-04-24T06:33:47.103778+0000","last_peered":"2024-04-24T06:33:47.103778+0000","last_clean":"2024-04-24T06:33:47.103778+0000","last_became_active":"2024-04-24T06:33:44.263169+0000","last_became_peered":"2024-04-24T06:33:44.263169+0000","last_unstale":"2024-04-24T06:33:47.103778+0000","last_undegraded":"2024-04-24T06:33:47.103778+0000","last_fullsized":"2024-04-24T06:33:47.103778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:39:22.504664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258362+0000","last_change":"2024-04-24T06:33:45.273407+0000","last_active":"2024-04-24T06:33:46.258362+0000","last_peered":"2024-04-24T06:33:46.258362+0000","last_clean":"2024-04-24T06:33:46.258362+0000","last_became_active":"2024-04-24T06:33:45.273008+0000","last_became_peered":"2024-04-24T06:33:45.273008+0000","last_unstale":"2024-04-24T06:33:46.258362+0000","last_undegraded":"2024-04-24T06:33:46.258362+0000","last_fullsized":"2024-04-24T06:33:46.258362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:16.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100848+0000","last_change":"2024-04-24T06:33:44.268287+0000","last_active":"2024-04-24T06:33:47.100848+0000","last_peered":"2024-04-24T06:33:47.100848+0000","last_clean":"2024-04-24T06:33:47.100848+0000","last_became_active":"2024-04-24T06:33:44.267922+0000","last_became_peered":"2024-04-24T06:33:44.267922+0000","last_unstale":"2024-04-24T06:33:47.100848+0000","last_undegraded":"2024-04-24T06:33:47.100848+0000","last_fullsized":"2024-04-24T06:33:47.100848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:00:50.117855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571931+0000","last_change":"2024-04-24T06:33:45.266570+0000","last_active":"2024-04-24T06:33:46.571931+0000","last_peered":"2024-04-24T06:33:46.571931+0000","last_clean":"2024-04-24T06:33:46.571931+0000","last_became_active":"2024-04-24T06:33:45.264552+0000","last_became_peered":"2024-04-24T06:33:45.264552+0000","last_unstale":"2024-04-24T06:33:46.571931+0000","last_undegraded":"2024-04-24T06:33:46.571931+0000","last_fullsized":"2024-04-24T06:33:46.571931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:30:57.410126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102014+0000","last_change":"2024-04-24T06:33:44.256611+0000","last_active":"2024-04-24T06:33:47.102014+0000","last_peered":"2024-04-24T06:33:47.102014+0000","last_clean":"2024-04-24T06:33:47.102014+0000","last_became_active":"2024-04-24T06:33:44.256303+0000","last_became_peered":"2024-04-24T06:33:44.256303+0000","last_unstale":"2024-04-24T06:33:47.102014+0000","last_undegraded":"2024-04-24T06:33:47.102014+0000","last_fullsized":"2024-04-24T06:33:47.102014+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:05.889753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572140+0000","last_change":"2024-04-24T06:33:45.260300+0000","last_active":"2024-04-24T06:33:46.572140+0000","last_peered":"2024-04-24T06:33:46.572140+0000","last_clean":"2024-04-24T06:33:46.572140+0000","last_became_active":"2024-04-24T06:33:45.258860+0000","last_became_peered":"2024-04-24T06:33:45.258860+0000","last_unstale":"2024-04-24T06:33:46.572140+0000","last_undegraded":"2024-04-24T06:33:46.572140+0000","last_fullsized":"2024-04-24T06:33:46.572140+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:06:21.828845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265458+0000","last_change":"2024-04-24T06:33:44.257380+0000","last_active":"2024-04-24T06:33:46.265458+0000","last_peered":"2024-04-24T06:33:46.265458+0000","last_clean":"2024-04-24T06:33:46.265458+0000","last_became_active":"2024-04-24T06:33:44.256966+0000","last_became_peered":"2024-04-24T06:33:44.256966+0000","last_unstale":"2024-04-24T06:33:46.265458+0000","last_undegraded":"2024-04-24T06:33:46.265458+0000","last_fullsized":"2024-04-24T06:33:46.265458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.195266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571678+0000","last_change":"2024-04-24T06:33:45.260565+0000","last_active":"2024-04-24T06:33:46.571678+0000","last_peered":"2024-04-24T06:33:46.571678+0000","last_clean":"2024-04-24T06:33:46.571678+0000","last_became_active":"2024-04-24T06:33:45.260141+0000","last_became_peered":"2024-04-24T06:33:45.260141+0000","last_unstale":"2024-04-24T06:33:46.571678+0000","last_undegraded":"2024-04-24T06:33:46.571678+0000","last_fullsized":"2024-04-24T06:33:46.571678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:35:22.766755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270129+0000","last_change":"2024-04-24T06:33:44.255503+0000","last_active":"2024-04-24T06:33:46.270129+0000","last_peered":"2024-04-24T06:33:46.270129+0000","last_clean":"2024-04-24T06:33:46.270129+0000","last_became_active":"2024-04-24T06:33:44.254397+0000","last_became_peered":"2024-04-24T06:33:44.254397+0000","last_unstale":"2024-04-24T06:33:46.270129+0000","last_undegraded":"2024-04-24T06:33:46.270129+0000","last_fullsized":"2024-04-24T06:33:46.270129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.195266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270934+0000","last_change":"2024-04-24T06:33:45.258877+0000","last_active":"2024-04-24T06:33:46.270934+0000","last_peered":"2024-04-24T06:33:46.270934+0000","last_clean":"2024-04-24T06:33:46.270934+0000","last_became_active":"2024-04-24T06:33:45.258529+0000","last_became_peered":"2024-04-24T06:33:45.258529+0000","last_unstale":"2024-04-24T06:33:46.270934+0000","last_undegraded":"2024-04-24T06:33:46.270934+0000","last_fullsized":"2024-04-24T06:33:46.270934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:57.581510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102036+0000","last_change":"2024-04-24T06:33:44.259767+0000","last_active":"2024-04-24T06:33:47.102036+0000","last_peered":"2024-04-24T06:33:47.102036+0000","last_clean":"2024-04-24T06:33:47.102036+0000","last_became_active":"2024-04-24T06:33:44.258750+0000","last_became_peered":"2024-04-24T06:33:44.258750+0000","last_unstale":"2024-04-24T06:33:47.102036+0000","last_undegraded":"2024-04-24T06:33:47.102036+0000","last_fullsized":"2024-04-24T06:33:47.102036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:22:40.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265357+0000","last_change":"2024-04-24T06:33:45.260239+0000","last_active":"2024-04-24T06:33:46.265357+0000","last_peered":"2024-04-24T06:33:46.265357+0000","last_clean":"2024-04-24T06:33:46.265357+0000","last_became_active":"2024-04-24T06:33:45.258989+0000","last_became_peered":"2024-04-24T06:33:45.258989+0000","last_unstale":"2024-04-24T06:33:46.265357+0000","last_undegraded":"2024-04-24T06:33:46.265357+0000","last_fullsized":"2024-04-24T06:33:46.265357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265887+0000","last_change":"2024-04-24T06:33:44.252340+0000","last_active":"2024-04-24T06:33:46.265887+0000","last_peered":"2024-04-24T06:33:46.265887+0000","last_clean":"2024-04-24T06:33:46.265887+0000","last_became_active":"2024-04-24T06:33:44.251970+0000","last_became_peered":"2024-04-24T06:33:44.251970+0000","last_unstale":"2024-04-24T06:33:46.265887+0000","last_undegraded":"2024-04-24T06:33:46.265887+0000","last_fullsized":"2024-04-24T06:33:46.265887+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:46.781222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100703+0000","last_change":"2024-04-24T06:33:45.288763+0000","last_active":"2024-04-24T06:33:47.100703+0000","last_peered":"2024-04-24T06:33:47.100703+0000","last_clean":"2024-04-24T06:33:47.100703+0000","last_became_active":"2024-04-24T06:33:45.287535+0000","last_became_peered":"2024-04-24T06:33:45.287535+0000","last_unstale":"2024-04-24T06:33:47.100703+0000","last_undegraded":"2024-04-24T06:33:47.100703+0000","last_fullsized":"2024-04-24T06:33:47.100703+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:39:35.827262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258232+0000","last_change":"2024-04-24T06:33:44.250982+0000","last_active":"2024-04-24T06:33:46.258232+0000","last_peered":"2024-04-24T06:33:46.258232+0000","last_clean":"2024-04-24T06:33:46.258232+0000","last_became_active":"2024-04-24T06:33:44.250666+0000","last_became_peered":"2024-04-24T06:33:44.250666+0000","last_unstale":"2024-04-24T06:33:46.258232+0000","last_undegraded":"2024-04-24T06:33:46.258232+0000","last_fullsized":"2024-04-24T06:33:46.258232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:37:57.733050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103049+0000","last_change":"2024-04-24T06:33:45.285931+0000","last_active":"2024-04-24T06:33:47.103049+0000","last_peered":"2024-04-24T06:33:47.103049+0000","last_clean":"2024-04-24T06:33:47.103049+0000","last_became_active":"2024-04-24T06:33:45.285578+0000","last_became_peered":"2024-04-24T06:33:45.285578+0000","last_unstale":"2024-04-24T06:33:47.103049+0000","last_undegraded":"2024-04-24T06:33:47.103049+0000","last_fullsized":"2024-04-24T06:33:47.103049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:53.723991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270892+0000","last_change":"2024-04-24T06:33:44.253357+0000","last_active":"2024-04-24T06:33:46.270892+0000","last_peered":"2024-04-24T06:33:46.270892+0000","last_clean":"2024-04-24T06:33:46.270892+0000","last_became_active":"2024-04-24T06:33:44.253043+0000","last_became_peered":"2024-04-24T06:33:44.253043+0000","last_unstale":"2024-04-24T06:33:46.270892+0000","last_undegraded":"2024-04-24T06:33:46.270892+0000","last_fullsized":"2024-04-24T06:33:46.270892+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:02.627841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258431+0000","last_change":"2024-04-24T06:33:45.264024+0000","last_active":"2024-04-24T06:33:46.258431+0000","last_peered":"2024-04-24T06:33:46.258431+0000","last_clean":"2024-04-24T06:33:46.258431+0000","last_became_active":"2024-04-24T06:33:45.262733+0000","last_became_peered":"2024-04-24T06:33:45.262733+0000","last_unstale":"2024-04-24T06:33:46.258431+0000","last_undegraded":"2024-04-24T06:33:46.258431+0000","last_fullsized":"2024-04-24T06:33:46.258431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:34:47.734673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257574+0000","last_change":"2024-04-24T06:33:44.254527+0000","last_active":"2024-04-24T06:33:46.257574+0000","last_peered":"2024-04-24T06:33:46.257574+0000","last_clean":"2024-04-24T06:33:46.257574+0000","last_became_active":"2024-04-24T06:33:44.253190+0000","last_became_peered":"2024-04-24T06:33:44.253190+0000","last_unstale":"2024-04-24T06:33:46.257574+0000","last_undegraded":"2024-04-24T06:33:46.257574+0000","last_fullsized":"2024-04-24T06:33:46.257574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:33.210407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103518+0000","last_change":"2024-04-24T06:33:45.282664+0000","last_active":"2024-04-24T06:33:47.103518+0000","last_peered":"2024-04-24T06:33:47.103518+0000","last_clean":"2024-04-24T06:33:47.103518+0000","last_became_active":"2024-04-24T06:33:45.280883+0000","last_became_peered":"2024-04-24T06:33:45.280883+0000","last_unstale":"2024-04-24T06:33:47.103518+0000","last_undegraded":"2024-04-24T06:33:47.103518+0000","last_fullsized":"2024-04-24T06:33:47.103518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:01.211932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969025+0000","last_change":"2024-04-24T06:33:44.253348+0000","last_active":"2024-04-24T06:33:46.969025+0000","last_peered":"2024-04-24T06:33:46.969025+0000","last_clean":"2024-04-24T06:33:46.969025+0000","last_became_active":"2024-04-24T06:33:44.251208+0000","last_became_peered":"2024-04-24T06:33:44.251208+0000","last_unstale":"2024-04-24T06:33:46.969025+0000","last_undegraded":"2024-04-24T06:33:46.969025+0000","last_fullsized":"2024-04-24T06:33:46.969025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:58.618644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969721+0000","last_change":"2024-04-24T06:33:45.267112+0000","last_active":"2024-04-24T06:33:46.969721+0000","last_peered":"2024-04-24T06:33:46.969721+0000","last_clean":"2024-04-24T06:33:46.969721+0000","last_became_active":"2024-04-24T06:33:45.266466+0000","last_became_peered":"2024-04-24T06:33:45.266466+0000","last_unstale":"2024-04-24T06:33:46.969721+0000","last_undegraded":"2024-04-24T06:33:46.969721+0000","last_fullsized":"2024-04-24T06:33:46.969721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:21.890534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259423+0000","last_change":"2024-04-24T06:33:44.261959+0000","last_active":"2024-04-24T06:33:46.259423+0000","last_peered":"2024-04-24T06:33:46.259423+0000","last_clean":"2024-04-24T06:33:46.259423+0000","last_became_active":"2024-04-24T06:33:44.261655+0000","last_became_peered":"2024-04-24T06:33:44.261655+0000","last_unstale":"2024-04-24T06:33:46.259423+0000","last_undegraded":"2024-04-24T06:33:46.259423+0000","last_fullsized":"2024-04-24T06:33:46.259423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:46.359172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100868+0000","last_change":"2024-04-24T06:33:45.269195+0000","last_active":"2024-04-24T06:33:47.100868+0000","last_peered":"2024-04-24T06:33:47.100868+0000","last_clean":"2024-04-24T06:33:47.100868+0000","last_became_active":"2024-04-24T06:33:45.268893+0000","last_became_peered":"2024-04-24T06:33:45.268893+0000","last_unstale":"2024-04-24T06:33:47.100868+0000","last_undegraded":"2024-04-24T06:33:47.100868+0000","last_fullsized":"2024-04-24T06:33:47.100868+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:52.370033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102533+0000","last_change":"2024-04-24T06:33:44.262023+0000","last_active":"2024-04-24T06:33:47.102533+0000","last_peered":"2024-04-24T06:33:47.102533+0000","last_clean":"2024-04-24T06:33:47.102533+0000","last_became_active":"2024-04-24T06:33:44.261729+0000","last_became_peered":"2024-04-24T06:33:44.261729+0000","last_unstale":"2024-04-24T06:33:47.102533+0000","last_undegraded":"2024-04-24T06:33:47.102533+0000","last_fullsized":"2024-04-24T06:33:47.102533+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:33.111917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572152+0000","last_change":"2024-04-24T06:33:45.267029+0000","last_active":"2024-04-24T06:33:46.572152+0000","last_peered":"2024-04-24T06:33:46.572152+0000","last_clean":"2024-04-24T06:33:46.572152+0000","last_became_active":"2024-04-24T06:33:45.266658+0000","last_became_peered":"2024-04-24T06:33:45.266658+0000","last_unstale":"2024-04-24T06:33:46.572152+0000","last_undegraded":"2024-04-24T06:33:46.572152+0000","last_fullsized":"2024-04-24T06:33:46.572152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:02.942459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102245+0000","last_change":"2024-04-24T06:33:44.261630+0000","last_active":"2024-04-24T06:33:47.102245+0000","last_peered":"2024-04-24T06:33:47.102245+0000","last_clean":"2024-04-24T06:33:47.102245+0000","last_became_active":"2024-04-24T06:33:44.261285+0000","last_became_peered":"2024-04-24T06:33:44.261285+0000","last_unstale":"2024-04-24T06:33:47.102245+0000","last_undegraded":"2024-04-24T06:33:47.102245+0000","last_fullsized":"2024-04-24T06:33:47.102245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:14.526573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.099685+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:33:47.099685+0000","last_peered":"2024-04-24T06:33:47.099685+0000","last_clean":"2024-04-24T06:33:47.099685+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:33:47.099685+0000","last_undegraded":"2024-04-24T06:33:47.099685+0000","last_fullsized":"2024-04-24T06:33:47.099685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":944649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":355277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":355893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":944033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:33:49.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:33:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.992+0000 7fddd7000640 1 Processor -- start 2024-04-24T06:33:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.992+0000 7fddd7000640 1 -- start start 2024-04-24T06:33:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd7000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fddd015e9d0 0x7fddd00d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd7000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fddd00d02e0 0x7fddd00d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd7000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd00cf520 con 0x7fddd00d0f70 2024-04-24T06:33:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd7000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fddd00cf800 con 0x7fddd00d02e0 2024-04-24T06:33:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd7000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fddd00cfae0 con 0x7fddd015e9d0 2024-04-24T06:33:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fddd00d02e0 0x7fddd00d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fddd00d02e0 0x7fddd00d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46420/0 (socket says 172.21.15.59:46420) 2024-04-24T06:33:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd5200640 1 -- 172.21.15.59:0/496393731 learned_addr learned my addr 172.21.15.59:0/496393731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.994+0000 7fddd6600640 1 --1- 172.21.15.59:0/496393731 >> v1:172.21.15.59:6789/0 conn(0x7fddd00d0f70 0x7fddd00d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43968/0 (socket says 172.21.15.59:43968) 2024-04-24T06:33:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.995+0000 7fddd5c00640 1 --2- 172.21.15.59:0/496393731 >> v2:172.21.15.66:3300/0 conn(0x7fddd015e9d0 0x7fddd00d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.995+0000 7fddd5200640 1 -- 172.21.15.59:0/496393731 >> v2:172.21.15.66:3300/0 conn(0x7fddd015e9d0 msgr2=0x7fddd00d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.995+0000 7fddd5200640 1 --2- 172.21.15.59:0/496393731 >> v2:172.21.15.66:3300/0 conn(0x7fddd015e9d0 0x7fddd00d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.995+0000 7fddd5200640 1 -- 172.21.15.59:0/496393731 >> v1:172.21.15.59:6789/0 conn(0x7fddd00d0f70 legacy=0x7fddd00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.995+0000 7fddd5200640 1 -- 172.21.15.59:0/496393731 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00cfdc0 con 0x7fddd00d02e0 2024-04-24T06:33:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.996+0000 7fddd5200640 1 --2- 172.21.15.59:0/496393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d02e0 0x7fddd00d3090 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fddd008cdd0 tx=0x7fddc002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e292a7d08ac306 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.996+0000 7fddcfe00640 1 -- 172.21.15.59:0/496393731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0036070 con 0x7fddd00d02e0 2024-04-24T06:33:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.996+0000 7fddcfe00640 1 -- 172.21.15.59:0/496393731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc0031600 con 0x7fddd00d02e0 2024-04-24T06:33:49.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.996+0000 7fddcfe00640 1 -- 172.21.15.59:0/496393731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00319d0 con 0x7fddd00d02e0 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 -- 172.21.15.59:0/496393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d02e0 msgr2=0x7fddd00d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 --2- 172.21.15.59:0/496393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d02e0 0x7fddd00d3090 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fddd008cdd0 tx=0x7fddc002a740 comp rx=0 tx=0).stop 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 -- 172.21.15.59:0/496393731 shutdown_connections 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 --2- 172.21.15.59:0/496393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d02e0 0x7fddd00d3090 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 --2- 172.21.15.59:0/496393731 >> v2:172.21.15.66:3300/0 conn(0x7fddd015e9d0 0x7fddd00d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 -- 172.21.15.59:0/496393731 >> 172.21.15.59:0/496393731 conn(0x7fddd008bf90 msgr2=0x7fddd00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 -- 172.21.15.59:0/496393731 shutdown_connections 2024-04-24T06:33:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.997+0000 7fddd7000640 1 -- 172.21.15.59:0/496393731 wait complete. 2024-04-24T06:33:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.999+0000 7fddd7000640 1 Processor -- start 2024-04-24T06:33:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:49.999+0000 7fddd7000640 1 -- start start 2024-04-24T06:33:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 0x7fddd00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddd0159d40 0x7fddd00b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddd006cc70 con 0x7fddd00d0f70 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fddd006caf0 con 0x7fddd00d02e0 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd7000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fddd006cdf0 con 0x7fddd0159d40 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46432/0 (socket says 172.21.15.59:46432) 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.000+0000 7fddd5200640 1 -- 172.21.15.59:0/1007975448 learned_addr learned my addr 172.21.15.59:0/1007975448 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5c00640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 0x7fddd00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd6600640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddd0159d40 0x7fddd00b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 -- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddd0159d40 msgr2=0x7fddd00b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddd0159d40 0x7fddd00b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 -- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 msgr2=0x7fddd00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 0x7fddd00b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 -- 172.21.15.59:0/1007975448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00b46e0 con 0x7fddd00d0f70 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5c00640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 0x7fddd00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddd5200640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fddc0034040 tx=0x7fddc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.001+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0036050 con 0x7fddd00d0f70 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.002+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc0042070 con 0x7fddd00d0f70 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.002+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd00b49c0 con 0x7fddd00d0f70 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.002+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0009510 con 0x7fddd00d0f70 2024-04-24T06:33:50.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.002+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd00b4f80 con 0x7fddd00d0f70 2024-04-24T06:33:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.003+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd00b2f10 con 0x7fddd00d0f70 2024-04-24T06:33:50.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.004+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddc0007c30 con 0x7fddd00d0f70 2024-04-24T06:33:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.005+0000 7fddcd600640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 0x7fddb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.005+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fddc00acb70 con 0x7fddd00d0f70 2024-04-24T06:33:50.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.006+0000 7fddd5c00640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 0x7fddb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.007+0000 7fddd5c00640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 0x7fddb406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.010+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddd00b2f10 con 0x7fddd00d0f70 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.217+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddd01f58b0 con 0x7fddb406c8f0 2024-04-24T06:33:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.236+0000 7fddcd600640 1 -- 172.21.15.59:0/1007975448 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252068 (secure 0 0 0) 0x7fddc4009d90 con 0x7fddb406c8f0 2024-04-24T06:33:50.241 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 msgr2=0x7fddb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 0x7fddb406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 msgr2=0x7fddd01f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fddc0034040 tx=0x7fddc0004a40 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 shutdown_connections 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fddb406c8f0 0x7fddb406edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddd0159d40 0x7fddd00b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddd00d0f70 0x7fddd01f5370 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 --2- 172.21.15.59:0/1007975448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddd00d02e0 0x7fddd00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.243+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 >> 172.21.15.59:0/1007975448 conn(0x7fddd008bf90 msgr2=0x7fddd015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:50.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.244+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 shutdown_connections 2024-04-24T06:33:50.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.244+0000 7fddd7000640 1 -- 172.21.15.59:0/1007975448 wait complete. 2024-04-24T06:33:50.264 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-24T06:33:49.290795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257240,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717480,"statfs":{"total":773094113280,"available":772830699520,"internally_reserved":0,"allocated":7970816,"data_stored":4014876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001581"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103328+0000","last_change":"2024-04-24T06:33:45.278721+0000","last_active":"2024-04-24T06:33:47.103328+0000","last_peered":"2024-04-24T06:33:47.103328+0000","last_clean":"2024-04-24T06:33:47.103328+0000","last_became_active":"2024-04-24T06:33:45.275658+0000","last_became_peered":"2024-04-24T06:33:45.275658+0000","last_unstale":"2024-04-24T06:33:47.103328+0000","last_undegraded":"2024-04-24T06:33:47.103328+0000","last_fullsized":"2024-04-24T06:33:47.103328+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:37:30.945343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101628+0000","last_change":"2024-04-24T06:33:45.284399+0000","last_active":"2024-04-24T06:33:47.101628+0000","last_peered":"2024-04-24T06:33:47.101628+0000","last_clean":"2024-04-24T06:33:47.101628+0000","last_became_active":"2024-04-24T06:33:45.280645+0000","last_became_peered":"2024-04-24T06:33:45.280645+0000","last_unstale":"2024-04-24T06:33:47.101628+0000","last_undegraded":"2024-04-24T06:33:47.101628+0000","last_fullsized":"2024-04-24T06:33:47.101628+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:15.426306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572275+0000","last_change":"2024-04-24T06:33:45.272543+0000","last_active":"2024-04-24T06:33:46.572275+0000","last_peered":"2024-04-24T06:33:46.572275+0000","last_clean":"2024-04-24T06:33:46.572275+0000","last_became_active":"2024-04-24T06:33:45.272264+0000","last_became_peered":"2024-04-24T06:33:45.272264+0000","last_unstale":"2024-04-24T06:33:46.572275+0000","last_undegraded":"2024-04-24T06:33:46.572275+0000","last_fullsized":"2024-04-24T06:33:46.572275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:45:23.351151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970036+0000","last_change":"2024-04-24T06:33:45.271249+0000","last_active":"2024-04-24T06:33:46.970036+0000","last_peered":"2024-04-24T06:33:46.970036+0000","last_clean":"2024-04-24T06:33:46.970036+0000","last_became_active":"2024-04-24T06:33:45.268312+0000","last_became_peered":"2024-04-24T06:33:45.268312+0000","last_unstale":"2024-04-24T06:33:46.970036+0000","last_undegraded":"2024-04-24T06:33:46.970036+0000","last_fullsized":"2024-04-24T06:33:46.970036+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:45:40.753734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257489+0000","last_change":"2024-04-24T06:33:45.270127+0000","last_active":"2024-04-24T06:33:46.257489+0000","last_peered":"2024-04-24T06:33:46.257489+0000","last_clean":"2024-04-24T06:33:46.257489+0000","last_became_active":"2024-04-24T06:33:45.269551+0000","last_became_peered":"2024-04-24T06:33:45.269551+0000","last_unstale":"2024-04-24T06:33:46.257489+0000","last_undegraded":"2024-04-24T06:33:46.257489+0000","last_fullsized":"2024-04-24T06:33:46.257489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:32:32.566961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101100+0000","last_change":"2024-04-24T06:33:45.271505+0000","last_active":"2024-04-24T06:33:47.101100+0000","last_peered":"2024-04-24T06:33:47.101100+0000","last_clean":"2024-04-24T06:33:47.101100+0000","last_became_active":"2024-04-24T06:33:45.271079+0000","last_became_peered":"2024-04-24T06:33:45.271079+0000","last_unstale":"2024-04-24T06:33:47.101100+0000","last_undegraded":"2024-04-24T06:33:47.101100+0000","last_fullsized":"2024-04-24T06:33:47.101100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:47.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265035+0000","last_change":"2024-04-24T06:33:45.261095+0000","last_active":"2024-04-24T06:33:46.265035+0000","last_peered":"2024-04-24T06:33:46.265035+0000","last_clean":"2024-04-24T06:33:46.265035+0000","last_became_active":"2024-04-24T06:33:45.260808+0000","last_became_peered":"2024-04-24T06:33:45.260808+0000","last_unstale":"2024-04-24T06:33:46.265035+0000","last_undegraded":"2024-04-24T06:33:46.265035+0000","last_fullsized":"2024-04-24T06:33:46.265035+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101917+0000","last_change":"2024-04-24T06:33:45.258389+0000","last_active":"2024-04-24T06:33:47.101917+0000","last_peered":"2024-04-24T06:33:47.101917+0000","last_clean":"2024-04-24T06:33:47.101917+0000","last_became_active":"2024-04-24T06:33:45.254715+0000","last_became_peered":"2024-04-24T06:33:45.254715+0000","last_unstale":"2024-04-24T06:33:47.101917+0000","last_undegraded":"2024-04-24T06:33:47.101917+0000","last_fullsized":"2024-04-24T06:33:47.101917+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:50:08.517777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258046+0000","last_change":"2024-04-24T06:33:45.267762+0000","last_active":"2024-04-24T06:33:46.258046+0000","last_peered":"2024-04-24T06:33:46.258046+0000","last_clean":"2024-04-24T06:33:46.258046+0000","last_became_active":"2024-04-24T06:33:45.267069+0000","last_became_peered":"2024-04-24T06:33:45.267069+0000","last_unstale":"2024-04-24T06:33:46.258046+0000","last_undegraded":"2024-04-24T06:33:46.258046+0000","last_fullsized":"2024-04-24T06:33:46.258046+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:34.128584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260652+0000","last_change":"2024-04-24T06:33:45.273706+0000","last_active":"2024-04-24T06:33:46.260652+0000","last_peered":"2024-04-24T06:33:46.260652+0000","last_clean":"2024-04-24T06:33:46.260652+0000","last_became_active":"2024-04-24T06:33:45.271271+0000","last_became_peered":"2024-04-24T06:33:45.271271+0000","last_unstale":"2024-04-24T06:33:46.260652+0000","last_undegraded":"2024-04-24T06:33:46.260652+0000","last_fullsized":"2024-04-24T06:33:46.260652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102776+0000","last_change":"2024-04-24T06:33:45.282134+0000","last_active":"2024-04-24T06:33:47.102776+0000","last_peered":"2024-04-24T06:33:47.102776+0000","last_clean":"2024-04-24T06:33:47.102776+0000","last_became_active":"2024-04-24T06:33:45.281185+0000","last_became_peered":"2024-04-24T06:33:45.281185+0000","last_unstale":"2024-04-24T06:33:47.102776+0000","last_undegraded":"2024-04-24T06:33:47.102776+0000","last_fullsized":"2024-04-24T06:33:47.102776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:13.511841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970096+0000","last_change":"2024-04-24T06:33:45.259322+0000","last_active":"2024-04-24T06:33:46.970096+0000","last_peered":"2024-04-24T06:33:46.970096+0000","last_clean":"2024-04-24T06:33:46.970096+0000","last_became_active":"2024-04-24T06:33:45.258962+0000","last_became_peered":"2024-04-24T06:33:45.258962+0000","last_unstale":"2024-04-24T06:33:46.970096+0000","last_undegraded":"2024-04-24T06:33:46.970096+0000","last_fullsized":"2024-04-24T06:33:46.970096+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:52.370033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270556+0000","last_change":"2024-04-24T06:33:45.259803+0000","last_active":"2024-04-24T06:33:46.270556+0000","last_peered":"2024-04-24T06:33:46.270556+0000","last_clean":"2024-04-24T06:33:46.270556+0000","last_became_active":"2024-04-24T06:33:45.258411+0000","last_became_peered":"2024-04-24T06:33:45.258411+0000","last_unstale":"2024-04-24T06:33:46.270556+0000","last_undegraded":"2024-04-24T06:33:46.270556+0000","last_fullsized":"2024-04-24T06:33:46.270556+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258042+0000","last_change":"2024-04-24T06:33:45.260324+0000","last_active":"2024-04-24T06:33:46.258042+0000","last_peered":"2024-04-24T06:33:46.258042+0000","last_clean":"2024-04-24T06:33:46.258042+0000","last_became_active":"2024-04-24T06:33:45.260025+0000","last_became_peered":"2024-04-24T06:33:45.260025+0000","last_unstale":"2024-04-24T06:33:46.258042+0000","last_undegraded":"2024-04-24T06:33:46.258042+0000","last_fullsized":"2024-04-24T06:33:46.258042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:15.543240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.271061+0000","last_change":"2024-04-24T06:33:45.255831+0000","last_active":"2024-04-24T06:33:46.271061+0000","last_peered":"2024-04-24T06:33:46.271061+0000","last_clean":"2024-04-24T06:33:46.271061+0000","last_became_active":"2024-04-24T06:33:45.255530+0000","last_became_peered":"2024-04-24T06:33:45.255530+0000","last_unstale":"2024-04-24T06:33:46.271061+0000","last_undegraded":"2024-04-24T06:33:46.271061+0000","last_fullsized":"2024-04-24T06:33:46.271061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:17:53.778790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259694+0000","last_change":"2024-04-24T06:33:45.273709+0000","last_active":"2024-04-24T06:33:46.259694+0000","last_peered":"2024-04-24T06:33:46.259694+0000","last_clean":"2024-04-24T06:33:46.259694+0000","last_became_active":"2024-04-24T06:33:45.273348+0000","last_became_peered":"2024-04-24T06:33:45.273348+0000","last_unstale":"2024-04-24T06:33:46.259694+0000","last_undegraded":"2024-04-24T06:33:46.259694+0000","last_fullsized":"2024-04-24T06:33:46.259694+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:16.918794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258254+0000","last_change":"2024-04-24T06:33:45.267392+0000","last_active":"2024-04-24T06:33:46.258254+0000","last_peered":"2024-04-24T06:33:46.258254+0000","last_clean":"2024-04-24T06:33:46.258254+0000","last_became_active":"2024-04-24T06:33:45.266895+0000","last_became_peered":"2024-04-24T06:33:45.266895+0000","last_unstale":"2024-04-24T06:33:46.258254+0000","last_undegraded":"2024-04-24T06:33:46.258254+0000","last_fullsized":"2024-04-24T06:33:46.258254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:22:41.989654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969951+0000","last_change":"2024-04-24T06:33:45.264820+0000","last_active":"2024-04-24T06:33:46.969951+0000","last_peered":"2024-04-24T06:33:46.969951+0000","last_clean":"2024-04-24T06:33:46.969951+0000","last_became_active":"2024-04-24T06:33:45.263855+0000","last_became_peered":"2024-04-24T06:33:45.263855+0000","last_unstale":"2024-04-24T06:33:46.969951+0000","last_undegraded":"2024-04-24T06:33:46.969951+0000","last_fullsized":"2024-04-24T06:33:46.969951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:57.985877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103442+0000","last_change":"2024-04-24T06:33:45.257278+0000","last_active":"2024-04-24T06:33:47.103442+0000","last_peered":"2024-04-24T06:33:47.103442+0000","last_clean":"2024-04-24T06:33:47.103442+0000","last_became_active":"2024-04-24T06:33:45.256984+0000","last_became_peered":"2024-04-24T06:33:45.256984+0000","last_unstale":"2024-04-24T06:33:47.103442+0000","last_undegraded":"2024-04-24T06:33:47.103442+0000","last_fullsized":"2024-04-24T06:33:47.103442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:47.797889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270731+0000","last_change":"2024-04-24T06:33:45.261637+0000","last_active":"2024-04-24T06:33:46.270731+0000","last_peered":"2024-04-24T06:33:46.270731+0000","last_clean":"2024-04-24T06:33:46.270731+0000","last_became_active":"2024-04-24T06:33:45.261314+0000","last_became_peered":"2024-04-24T06:33:45.261314+0000","last_unstale":"2024-04-24T06:33:46.270731+0000","last_undegraded":"2024-04-24T06:33:46.270731+0000","last_fullsized":"2024-04-24T06:33:46.270731+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259792+0000","last_change":"2024-04-24T06:33:45.279300+0000","last_active":"2024-04-24T06:33:46.259792+0000","last_peered":"2024-04-24T06:33:46.259792+0000","last_clean":"2024-04-24T06:33:46.259792+0000","last_became_active":"2024-04-24T06:33:45.278698+0000","last_became_peered":"2024-04-24T06:33:45.278698+0000","last_unstale":"2024-04-24T06:33:46.259792+0000","last_undegraded":"2024-04-24T06:33:46.259792+0000","last_fullsized":"2024-04-24T06:33:46.259792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:23.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259801+0000","last_change":"2024-04-24T06:33:45.268393+0000","last_active":"2024-04-24T06:33:46.259801+0000","last_peered":"2024-04-24T06:33:46.259801+0000","last_clean":"2024-04-24T06:33:46.259801+0000","last_became_active":"2024-04-24T06:33:45.268057+0000","last_became_peered":"2024-04-24T06:33:45.268057+0000","last_unstale":"2024-04-24T06:33:46.259801+0000","last_undegraded":"2024-04-24T06:33:46.259801+0000","last_fullsized":"2024-04-24T06:33:46.259801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:12:24.381678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101159+0000","last_change":"2024-04-24T06:33:45.266906+0000","last_active":"2024-04-24T06:33:47.101159+0000","last_peered":"2024-04-24T06:33:47.101159+0000","last_clean":"2024-04-24T06:33:47.101159+0000","last_became_active":"2024-04-24T06:33:45.266580+0000","last_became_peered":"2024-04-24T06:33:45.266580+0000","last_unstale":"2024-04-24T06:33:47.101159+0000","last_undegraded":"2024-04-24T06:33:47.101159+0000","last_fullsized":"2024-04-24T06:33:47.101159+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:13.995575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100101+0000","last_change":"2024-04-24T06:33:45.284150+0000","last_active":"2024-04-24T06:33:47.100101+0000","last_peered":"2024-04-24T06:33:47.100101+0000","last_clean":"2024-04-24T06:33:47.100101+0000","last_became_active":"2024-04-24T06:33:45.283865+0000","last_became_peered":"2024-04-24T06:33:45.283865+0000","last_unstale":"2024-04-24T06:33:47.100101+0000","last_undegraded":"2024-04-24T06:33:47.100101+0000","last_fullsized":"2024-04-24T06:33:47.100101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:21:05.963617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260679+0000","last_change":"2024-04-24T06:33:45.274121+0000","last_active":"2024-04-24T06:33:46.260679+0000","last_peered":"2024-04-24T06:33:46.260679+0000","last_clean":"2024-04-24T06:33:46.260679+0000","last_became_active":"2024-04-24T06:33:45.273202+0000","last_became_peered":"2024-04-24T06:33:45.273202+0000","last_unstale":"2024-04-24T06:33:46.260679+0000","last_undegraded":"2024-04-24T06:33:46.260679+0000","last_fullsized":"2024-04-24T06:33:46.260679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265744+0000","last_change":"2024-04-24T06:33:45.276246+0000","last_active":"2024-04-24T06:33:46.265744+0000","last_peered":"2024-04-24T06:33:46.265744+0000","last_clean":"2024-04-24T06:33:46.265744+0000","last_became_active":"2024-04-24T06:33:45.269723+0000","last_became_peered":"2024-04-24T06:33:45.269723+0000","last_unstale":"2024-04-24T06:33:46.265744+0000","last_undegraded":"2024-04-24T06:33:46.265744+0000","last_fullsized":"2024-04-24T06:33:46.265744+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:30.304735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572563+0000","last_change":"2024-04-24T06:33:45.273920+0000","last_active":"2024-04-24T06:33:46.572563+0000","last_peered":"2024-04-24T06:33:46.572563+0000","last_clean":"2024-04-24T06:33:46.572563+0000","last_became_active":"2024-04-24T06:33:45.273603+0000","last_became_peered":"2024-04-24T06:33:45.273603+0000","last_unstale":"2024-04-24T06:33:46.572563+0000","last_undegraded":"2024-04-24T06:33:46.572563+0000","last_fullsized":"2024-04-24T06:33:46.572563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:22.433498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260228+0000","last_change":"2024-04-24T06:33:45.271766+0000","last_active":"2024-04-24T06:33:46.260228+0000","last_peered":"2024-04-24T06:33:46.260228+0000","last_clean":"2024-04-24T06:33:46.260228+0000","last_became_active":"2024-04-24T06:33:45.271351+0000","last_became_peered":"2024-04-24T06:33:45.271351+0000","last_unstale":"2024-04-24T06:33:46.260228+0000","last_undegraded":"2024-04-24T06:33:46.260228+0000","last_fullsized":"2024-04-24T06:33:46.260228+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572144+0000","last_change":"2024-04-24T06:33:45.266267+0000","last_active":"2024-04-24T06:33:46.572144+0000","last_peered":"2024-04-24T06:33:46.572144+0000","last_clean":"2024-04-24T06:33:46.572144+0000","last_became_active":"2024-04-24T06:33:45.265581+0000","last_became_peered":"2024-04-24T06:33:45.265581+0000","last_unstale":"2024-04-24T06:33:46.572144+0000","last_undegraded":"2024-04-24T06:33:46.572144+0000","last_fullsized":"2024-04-24T06:33:46.572144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:52.823391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265278+0000","last_change":"2024-04-24T06:33:45.270105+0000","last_active":"2024-04-24T06:33:46.265278+0000","last_peered":"2024-04-24T06:33:46.265278+0000","last_clean":"2024-04-24T06:33:46.265278+0000","last_became_active":"2024-04-24T06:33:45.269462+0000","last_became_peered":"2024-04-24T06:33:45.269462+0000","last_unstale":"2024-04-24T06:33:46.265278+0000","last_undegraded":"2024-04-24T06:33:46.265278+0000","last_fullsized":"2024-04-24T06:33:46.265278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:18:03.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265385+0000","last_change":"2024-04-24T06:33:45.259599+0000","last_active":"2024-04-24T06:33:46.265385+0000","last_peered":"2024-04-24T06:33:46.265385+0000","last_clean":"2024-04-24T06:33:46.265385+0000","last_became_active":"2024-04-24T06:33:45.258490+0000","last_became_peered":"2024-04-24T06:33:45.258490+0000","last_unstale":"2024-04-24T06:33:46.265385+0000","last_undegraded":"2024-04-24T06:33:46.265385+0000","last_fullsized":"2024-04-24T06:33:46.265385+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:57.581510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102355+0000","last_change":"2024-04-24T06:33:45.262083+0000","last_active":"2024-04-24T06:33:47.102355+0000","last_peered":"2024-04-24T06:33:47.102355+0000","last_clean":"2024-04-24T06:33:47.102355+0000","last_became_active":"2024-04-24T06:33:45.261767+0000","last_became_peered":"2024-04-24T06:33:45.261767+0000","last_unstale":"2024-04-24T06:33:47.102355+0000","last_undegraded":"2024-04-24T06:33:47.102355+0000","last_fullsized":"2024-04-24T06:33:47.102355+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:54:59.436903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103233+0000","last_change":"2024-04-24T06:33:45.290790+0000","last_active":"2024-04-24T06:33:47.103233+0000","last_peered":"2024-04-24T06:33:47.103233+0000","last_clean":"2024-04-24T06:33:47.103233+0000","last_became_active":"2024-04-24T06:33:45.290379+0000","last_became_peered":"2024-04-24T06:33:45.290379+0000","last_unstale":"2024-04-24T06:33:47.103233+0000","last_undegraded":"2024-04-24T06:33:47.103233+0000","last_fullsized":"2024-04-24T06:33:47.103233+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:21.675900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103873+0000","last_change":"2024-04-24T06:33:44.266860+0000","last_active":"2024-04-24T06:33:47.103873+0000","last_peered":"2024-04-24T06:33:47.103873+0000","last_clean":"2024-04-24T06:33:47.103873+0000","last_became_active":"2024-04-24T06:33:44.266534+0000","last_became_peered":"2024-04-24T06:33:44.266534+0000","last_unstale":"2024-04-24T06:33:47.103873+0000","last_undegraded":"2024-04-24T06:33:47.103873+0000","last_fullsized":"2024-04-24T06:33:47.103873+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:11:19.836312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258541+0000","last_change":"2024-04-24T06:33:45.267173+0000","last_active":"2024-04-24T06:33:46.258541+0000","last_peered":"2024-04-24T06:33:46.258541+0000","last_clean":"2024-04-24T06:33:46.258541+0000","last_became_active":"2024-04-24T06:33:45.266532+0000","last_became_peered":"2024-04-24T06:33:45.266532+0000","last_unstale":"2024-04-24T06:33:46.258541+0000","last_undegraded":"2024-04-24T06:33:46.258541+0000","last_fullsized":"2024-04-24T06:33:46.258541+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:06.906420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270835+0000","last_change":"2024-04-24T06:33:44.252531+0000","last_active":"2024-04-24T06:33:46.270835+0000","last_peered":"2024-04-24T06:33:46.270835+0000","last_clean":"2024-04-24T06:33:46.270835+0000","last_became_active":"2024-04-24T06:33:44.251855+0000","last_became_peered":"2024-04-24T06:33:44.251855+0000","last_unstale":"2024-04-24T06:33:46.270835+0000","last_undegraded":"2024-04-24T06:33:46.270835+0000","last_fullsized":"2024-04-24T06:33:46.270835+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:45:18.118468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102770+0000","last_change":"2024-04-24T06:33:45.282852+0000","last_active":"2024-04-24T06:33:47.102770+0000","last_peered":"2024-04-24T06:33:47.102770+0000","last_clean":"2024-04-24T06:33:47.102770+0000","last_became_active":"2024-04-24T06:33:45.281556+0000","last_became_peered":"2024-04-24T06:33:45.281556+0000","last_unstale":"2024-04-24T06:33:47.102770+0000","last_undegraded":"2024-04-24T06:33:47.102770+0000","last_fullsized":"2024-04-24T06:33:47.102770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:31:37.535227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260357+0000","last_change":"2024-04-24T06:33:44.266047+0000","last_active":"2024-04-24T06:33:46.260357+0000","last_peered":"2024-04-24T06:33:46.260357+0000","last_clean":"2024-04-24T06:33:46.260357+0000","last_became_active":"2024-04-24T06:33:44.264688+0000","last_became_peered":"2024-04-24T06:33:44.264688+0000","last_unstale":"2024-04-24T06:33:46.260357+0000","last_undegraded":"2024-04-24T06:33:46.260357+0000","last_fullsized":"2024-04-24T06:33:46.260357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:14.409639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102843+0000","last_change":"2024-04-24T06:33:45.275637+0000","last_active":"2024-04-24T06:33:47.102843+0000","last_peered":"2024-04-24T06:33:47.102843+0000","last_clean":"2024-04-24T06:33:47.102843+0000","last_became_active":"2024-04-24T06:33:45.275244+0000","last_became_peered":"2024-04-24T06:33:45.275244+0000","last_unstale":"2024-04-24T06:33:47.102843+0000","last_undegraded":"2024-04-24T06:33:47.102843+0000","last_fullsized":"2024-04-24T06:33:47.102843+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:55:27.775478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103336+0000","last_change":"2024-04-24T06:33:44.263749+0000","last_active":"2024-04-24T06:33:47.103336+0000","last_peered":"2024-04-24T06:33:47.103336+0000","last_clean":"2024-04-24T06:33:47.103336+0000","last_became_active":"2024-04-24T06:33:44.263434+0000","last_became_peered":"2024-04-24T06:33:44.263434+0000","last_unstale":"2024-04-24T06:33:47.103336+0000","last_undegraded":"2024-04-24T06:33:47.103336+0000","last_fullsized":"2024-04-24T06:33:47.103336+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:23:05.090830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265650+0000","last_change":"2024-04-24T06:33:45.273462+0000","last_active":"2024-04-24T06:33:46.265650+0000","last_peered":"2024-04-24T06:33:46.265650+0000","last_clean":"2024-04-24T06:33:46.265650+0000","last_became_active":"2024-04-24T06:33:45.273148+0000","last_became_peered":"2024-04-24T06:33:45.273148+0000","last_unstale":"2024-04-24T06:33:46.265650+0000","last_undegraded":"2024-04-24T06:33:46.265650+0000","last_fullsized":"2024-04-24T06:33:46.265650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:17:53.778790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571992+0000","last_change":"2024-04-24T06:33:44.252064+0000","last_active":"2024-04-24T06:33:46.571992+0000","last_peered":"2024-04-24T06:33:46.571992+0000","last_clean":"2024-04-24T06:33:46.571992+0000","last_became_active":"2024-04-24T06:33:44.251738+0000","last_became_peered":"2024-04-24T06:33:44.251738+0000","last_unstale":"2024-04-24T06:33:46.571992+0000","last_undegraded":"2024-04-24T06:33:46.571992+0000","last_fullsized":"2024-04-24T06:33:46.571992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:44.971036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260047+0000","last_change":"2024-04-24T06:33:45.267452+0000","last_active":"2024-04-24T06:33:46.260047+0000","last_peered":"2024-04-24T06:33:46.260047+0000","last_clean":"2024-04-24T06:33:46.260047+0000","last_became_active":"2024-04-24T06:33:45.266997+0000","last_became_peered":"2024-04-24T06:33:45.266997+0000","last_unstale":"2024-04-24T06:33:46.260047+0000","last_undegraded":"2024-04-24T06:33:46.260047+0000","last_fullsized":"2024-04-24T06:33:46.260047+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:39.619240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101257+0000","last_change":"2024-04-24T06:33:44.258497+0000","last_active":"2024-04-24T06:33:47.101257+0000","last_peered":"2024-04-24T06:33:47.101257+0000","last_clean":"2024-04-24T06:33:47.101257+0000","last_became_active":"2024-04-24T06:33:44.256889+0000","last_became_peered":"2024-04-24T06:33:44.256889+0000","last_unstale":"2024-04-24T06:33:47.101257+0000","last_undegraded":"2024-04-24T06:33:47.101257+0000","last_fullsized":"2024-04-24T06:33:47.101257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:15.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257865+0000","last_change":"2024-04-24T06:33:45.267148+0000","last_active":"2024-04-24T06:33:46.257865+0000","last_peered":"2024-04-24T06:33:46.257865+0000","last_clean":"2024-04-24T06:33:46.257865+0000","last_became_active":"2024-04-24T06:33:45.264815+0000","last_became_peered":"2024-04-24T06:33:45.264815+0000","last_unstale":"2024-04-24T06:33:46.257865+0000","last_undegraded":"2024-04-24T06:33:46.257865+0000","last_fullsized":"2024-04-24T06:33:46.257865+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:22.950993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969109+0000","last_change":"2024-04-24T06:33:44.254391+0000","last_active":"2024-04-24T06:33:46.969109+0000","last_peered":"2024-04-24T06:33:46.969109+0000","last_clean":"2024-04-24T06:33:46.969109+0000","last_became_active":"2024-04-24T06:33:44.254127+0000","last_became_peered":"2024-04-24T06:33:44.254127+0000","last_unstale":"2024-04-24T06:33:46.969109+0000","last_undegraded":"2024-04-24T06:33:46.969109+0000","last_fullsized":"2024-04-24T06:33:46.969109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:39:22.504664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270467+0000","last_change":"2024-04-24T06:33:45.263533+0000","last_active":"2024-04-24T06:33:46.270467+0000","last_peered":"2024-04-24T06:33:46.270467+0000","last_clean":"2024-04-24T06:33:46.270467+0000","last_became_active":"2024-04-24T06:33:45.263131+0000","last_became_peered":"2024-04-24T06:33:45.263131+0000","last_unstale":"2024-04-24T06:33:46.270467+0000","last_undegraded":"2024-04-24T06:33:46.270467+0000","last_fullsized":"2024-04-24T06:33:46.270467+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:12:45.728301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257292+0000","last_change":"2024-04-24T06:33:44.248872+0000","last_active":"2024-04-24T06:33:46.257292+0000","last_peered":"2024-04-24T06:33:46.257292+0000","last_clean":"2024-04-24T06:33:46.257292+0000","last_became_active":"2024-04-24T06:33:44.248584+0000","last_became_peered":"2024-04-24T06:33:44.248584+0000","last_unstale":"2024-04-24T06:33:46.257292+0000","last_undegraded":"2024-04-24T06:33:46.257292+0000","last_fullsized":"2024-04-24T06:33:46.257292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:57.686898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265752+0000","last_change":"2024-04-24T06:33:45.277803+0000","last_active":"2024-04-24T06:33:46.265752+0000","last_peered":"2024-04-24T06:33:46.265752+0000","last_clean":"2024-04-24T06:33:46.265752+0000","last_became_active":"2024-04-24T06:33:45.276196+0000","last_became_peered":"2024-04-24T06:33:45.276196+0000","last_unstale":"2024-04-24T06:33:46.265752+0000","last_undegraded":"2024-04-24T06:33:46.265752+0000","last_fullsized":"2024-04-24T06:33:46.265752+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:21:05.963617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260537+0000","last_change":"2024-04-24T06:33:44.244072+0000","last_active":"2024-04-24T06:33:46.260537+0000","last_peered":"2024-04-24T06:33:46.260537+0000","last_clean":"2024-04-24T06:33:46.260537+0000","last_became_active":"2024-04-24T06:33:44.243459+0000","last_became_peered":"2024-04-24T06:33:44.243459+0000","last_unstale":"2024-04-24T06:33:46.260537+0000","last_undegraded":"2024-04-24T06:33:46.260537+0000","last_fullsized":"2024-04-24T06:33:46.260537+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:46.781222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103723+0000","last_change":"2024-04-24T06:33:45.289439+0000","last_active":"2024-04-24T06:33:47.103723+0000","last_peered":"2024-04-24T06:33:47.103723+0000","last_clean":"2024-04-24T06:33:47.103723+0000","last_became_active":"2024-04-24T06:33:45.288880+0000","last_became_peered":"2024-04-24T06:33:45.288880+0000","last_unstale":"2024-04-24T06:33:47.103723+0000","last_undegraded":"2024-04-24T06:33:47.103723+0000","last_fullsized":"2024-04-24T06:33:47.103723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:24.944322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101955+0000","last_change":"2024-04-24T06:33:44.262780+0000","last_active":"2024-04-24T06:33:47.101955+0000","last_peered":"2024-04-24T06:33:47.101955+0000","last_clean":"2024-04-24T06:33:47.101955+0000","last_became_active":"2024-04-24T06:33:44.262383+0000","last_became_peered":"2024-04-24T06:33:44.262383+0000","last_unstale":"2024-04-24T06:33:47.101955+0000","last_undegraded":"2024-04-24T06:33:47.101955+0000","last_fullsized":"2024-04-24T06:33:47.101955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:06:05.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103229+0000","last_change":"2024-04-24T06:33:45.279688+0000","last_active":"2024-04-24T06:33:47.103229+0000","last_peered":"2024-04-24T06:33:47.103229+0000","last_clean":"2024-04-24T06:33:47.103229+0000","last_became_active":"2024-04-24T06:33:45.276726+0000","last_became_peered":"2024-04-24T06:33:45.276726+0000","last_unstale":"2024-04-24T06:33:47.103229+0000","last_undegraded":"2024-04-24T06:33:47.103229+0000","last_fullsized":"2024-04-24T06:33:47.103229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:24.899489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572381+0000","last_change":"2024-04-24T06:33:44.255196+0000","last_active":"2024-04-24T06:33:46.572381+0000","last_peered":"2024-04-24T06:33:46.572381+0000","last_clean":"2024-04-24T06:33:46.572381+0000","last_became_active":"2024-04-24T06:33:44.254124+0000","last_became_peered":"2024-04-24T06:33:44.254124+0000","last_unstale":"2024-04-24T06:33:46.572381+0000","last_undegraded":"2024-04-24T06:33:46.572381+0000","last_fullsized":"2024-04-24T06:33:46.572381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:23.927655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258089+0000","last_change":"2024-04-24T06:33:45.254225+0000","last_active":"2024-04-24T06:33:46.258089+0000","last_peered":"2024-04-24T06:33:46.258089+0000","last_clean":"2024-04-24T06:33:46.258089+0000","last_became_active":"2024-04-24T06:33:45.253464+0000","last_became_peered":"2024-04-24T06:33:45.253464+0000","last_unstale":"2024-04-24T06:33:46.258089+0000","last_undegraded":"2024-04-24T06:33:46.258089+0000","last_fullsized":"2024-04-24T06:33:46.258089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:01.414445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970239+0000","last_change":"2024-04-24T06:33:44.253343+0000","last_active":"2024-04-24T06:33:46.970239+0000","last_peered":"2024-04-24T06:33:46.970239+0000","last_clean":"2024-04-24T06:33:46.970239+0000","last_became_active":"2024-04-24T06:33:44.253004+0000","last_became_peered":"2024-04-24T06:33:44.253004+0000","last_unstale":"2024-04-24T06:33:46.970239+0000","last_undegraded":"2024-04-24T06:33:46.970239+0000","last_fullsized":"2024-04-24T06:33:46.970239+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:58:37.073804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101631+0000","last_change":"2024-04-24T06:33:45.275985+0000","last_active":"2024-04-24T06:33:47.101631+0000","last_peered":"2024-04-24T06:33:47.101631+0000","last_clean":"2024-04-24T06:33:47.101631+0000","last_became_active":"2024-04-24T06:33:45.275564+0000","last_became_peered":"2024-04-24T06:33:45.275564+0000","last_unstale":"2024-04-24T06:33:47.101631+0000","last_undegraded":"2024-04-24T06:33:47.101631+0000","last_fullsized":"2024-04-24T06:33:47.101631+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:57:57.176251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970007+0000","last_change":"2024-04-24T06:33:44.265007+0000","last_active":"2024-04-24T06:33:46.970007+0000","last_peered":"2024-04-24T06:33:46.970007+0000","last_clean":"2024-04-24T06:33:46.970007+0000","last_became_active":"2024-04-24T06:33:44.264636+0000","last_became_peered":"2024-04-24T06:33:44.264636+0000","last_unstale":"2024-04-24T06:33:46.970007+0000","last_undegraded":"2024-04-24T06:33:46.970007+0000","last_fullsized":"2024-04-24T06:33:46.970007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:06:05.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265377+0000","last_change":"2024-04-24T06:33:45.276693+0000","last_active":"2024-04-24T06:33:46.265377+0000","last_peered":"2024-04-24T06:33:46.265377+0000","last_clean":"2024-04-24T06:33:46.265377+0000","last_became_active":"2024-04-24T06:33:45.276397+0000","last_became_peered":"2024-04-24T06:33:45.276397+0000","last_unstale":"2024-04-24T06:33:46.265377+0000","last_undegraded":"2024-04-24T06:33:46.265377+0000","last_fullsized":"2024-04-24T06:33:46.265377+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101610+0000","last_change":"2024-04-24T06:33:44.261485+0000","last_active":"2024-04-24T06:33:47.101610+0000","last_peered":"2024-04-24T06:33:47.101610+0000","last_clean":"2024-04-24T06:33:47.101610+0000","last_became_active":"2024-04-24T06:33:44.260284+0000","last_became_peered":"2024-04-24T06:33:44.260284+0000","last_unstale":"2024-04-24T06:33:47.101610+0000","last_undegraded":"2024-04-24T06:33:47.101610+0000","last_fullsized":"2024-04-24T06:33:47.101610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:34:46.718006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270627+0000","last_change":"2024-04-24T06:33:45.257746+0000","last_active":"2024-04-24T06:33:46.270627+0000","last_peered":"2024-04-24T06:33:46.270627+0000","last_clean":"2024-04-24T06:33:46.270627+0000","last_became_active":"2024-04-24T06:33:45.257463+0000","last_became_peered":"2024-04-24T06:33:45.257463+0000","last_unstale":"2024-04-24T06:33:46.270627+0000","last_undegraded":"2024-04-24T06:33:46.270627+0000","last_fullsized":"2024-04-24T06:33:46.270627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265105+0000","last_change":"2024-04-24T06:33:44.259184+0000","last_active":"2024-04-24T06:33:46.265105+0000","last_peered":"2024-04-24T06:33:46.265105+0000","last_clean":"2024-04-24T06:33:46.265105+0000","last_became_active":"2024-04-24T06:33:44.258801+0000","last_became_peered":"2024-04-24T06:33:44.258801+0000","last_unstale":"2024-04-24T06:33:46.265105+0000","last_undegraded":"2024-04-24T06:33:46.265105+0000","last_fullsized":"2024-04-24T06:33:46.265105+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:16:23.882822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257721+0000","last_change":"2024-04-24T06:33:45.262676+0000","last_active":"2024-04-24T06:33:46.257721+0000","last_peered":"2024-04-24T06:33:46.257721+0000","last_clean":"2024-04-24T06:33:46.257721+0000","last_became_active":"2024-04-24T06:33:45.262306+0000","last_became_peered":"2024-04-24T06:33:45.262306+0000","last_unstale":"2024-04-24T06:33:46.257721+0000","last_undegraded":"2024-04-24T06:33:46.257721+0000","last_fullsized":"2024-04-24T06:33:46.257721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:26:07.882700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259473+0000","last_change":"2024-04-24T06:33:44.266404+0000","last_active":"2024-04-24T06:33:46.259473+0000","last_peered":"2024-04-24T06:33:46.259473+0000","last_clean":"2024-04-24T06:33:46.259473+0000","last_became_active":"2024-04-24T06:33:44.266098+0000","last_became_peered":"2024-04-24T06:33:44.266098+0000","last_unstale":"2024-04-24T06:33:46.259473+0000","last_undegraded":"2024-04-24T06:33:46.259473+0000","last_fullsized":"2024-04-24T06:33:46.259473+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:51.353366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260762+0000","last_change":"2024-04-24T06:33:45.274710+0000","last_active":"2024-04-24T06:33:46.260762+0000","last_peered":"2024-04-24T06:33:46.260762+0000","last_clean":"2024-04-24T06:33:46.260762+0000","last_became_active":"2024-04-24T06:33:45.274127+0000","last_became_peered":"2024-04-24T06:33:45.274127+0000","last_unstale":"2024-04-24T06:33:46.260762+0000","last_undegraded":"2024-04-24T06:33:46.260762+0000","last_fullsized":"2024-04-24T06:33:46.260762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:01.157617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.101693+0000","last_change":"2024-04-24T06:33:44.258870+0000","last_active":"2024-04-24T06:33:47.101693+0000","last_peered":"2024-04-24T06:33:47.101693+0000","last_clean":"2024-04-24T06:33:47.101693+0000","last_became_active":"2024-04-24T06:33:44.258575+0000","last_became_peered":"2024-04-24T06:33:44.258575+0000","last_unstale":"2024-04-24T06:33:47.101693+0000","last_undegraded":"2024-04-24T06:33:47.101693+0000","last_fullsized":"2024-04-24T06:33:47.101693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:20:00.397778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103771+0000","last_change":"2024-04-24T06:33:45.278975+0000","last_active":"2024-04-24T06:33:47.103771+0000","last_peered":"2024-04-24T06:33:47.103771+0000","last_clean":"2024-04-24T06:33:47.103771+0000","last_became_active":"2024-04-24T06:33:45.277425+0000","last_became_peered":"2024-04-24T06:33:45.277425+0000","last_unstale":"2024-04-24T06:33:47.103771+0000","last_undegraded":"2024-04-24T06:33:47.103771+0000","last_fullsized":"2024-04-24T06:33:47.103771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:23:45.987702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.260747+0000","last_change":"2024-04-24T06:33:44.260055+0000","last_active":"2024-04-24T06:33:46.260747+0000","last_peered":"2024-04-24T06:33:46.260747+0000","last_clean":"2024-04-24T06:33:46.260747+0000","last_became_active":"2024-04-24T06:33:44.259623+0000","last_became_peered":"2024-04-24T06:33:44.259623+0000","last_unstale":"2024-04-24T06:33:46.260747+0000","last_undegraded":"2024-04-24T06:33:46.260747+0000","last_fullsized":"2024-04-24T06:33:46.260747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:02.627841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.970456+0000","last_change":"2024-04-24T06:33:45.257168+0000","last_active":"2024-04-24T06:33:46.970456+0000","last_peered":"2024-04-24T06:33:46.970456+0000","last_clean":"2024-04-24T06:33:46.970456+0000","last_became_active":"2024-04-24T06:33:45.256681+0000","last_became_peered":"2024-04-24T06:33:45.256681+0000","last_unstale":"2024-04-24T06:33:46.970456+0000","last_undegraded":"2024-04-24T06:33:46.970456+0000","last_fullsized":"2024-04-24T06:33:46.970456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:47.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103778+0000","last_change":"2024-04-24T06:33:44.263585+0000","last_active":"2024-04-24T06:33:47.103778+0000","last_peered":"2024-04-24T06:33:47.103778+0000","last_clean":"2024-04-24T06:33:47.103778+0000","last_became_active":"2024-04-24T06:33:44.263169+0000","last_became_peered":"2024-04-24T06:33:44.263169+0000","last_unstale":"2024-04-24T06:33:47.103778+0000","last_undegraded":"2024-04-24T06:33:47.103778+0000","last_fullsized":"2024-04-24T06:33:47.103778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:39:22.504664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258362+0000","last_change":"2024-04-24T06:33:45.273407+0000","last_active":"2024-04-24T06:33:46.258362+0000","last_peered":"2024-04-24T06:33:46.258362+0000","last_clean":"2024-04-24T06:33:46.258362+0000","last_became_active":"2024-04-24T06:33:45.273008+0000","last_became_peered":"2024-04-24T06:33:45.273008+0000","last_unstale":"2024-04-24T06:33:46.258362+0000","last_undegraded":"2024-04-24T06:33:46.258362+0000","last_fullsized":"2024-04-24T06:33:46.258362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:16.898231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100848+0000","last_change":"2024-04-24T06:33:44.268287+0000","last_active":"2024-04-24T06:33:47.100848+0000","last_peered":"2024-04-24T06:33:47.100848+0000","last_clean":"2024-04-24T06:33:47.100848+0000","last_became_active":"2024-04-24T06:33:44.267922+0000","last_became_peered":"2024-04-24T06:33:44.267922+0000","last_unstale":"2024-04-24T06:33:47.100848+0000","last_undegraded":"2024-04-24T06:33:47.100848+0000","last_fullsized":"2024-04-24T06:33:47.100848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:00:50.117855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571931+0000","last_change":"2024-04-24T06:33:45.266570+0000","last_active":"2024-04-24T06:33:46.571931+0000","last_peered":"2024-04-24T06:33:46.571931+0000","last_clean":"2024-04-24T06:33:46.571931+0000","last_became_active":"2024-04-24T06:33:45.264552+0000","last_became_peered":"2024-04-24T06:33:45.264552+0000","last_unstale":"2024-04-24T06:33:46.571931+0000","last_undegraded":"2024-04-24T06:33:46.571931+0000","last_fullsized":"2024-04-24T06:33:46.571931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:30:57.410126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102014+0000","last_change":"2024-04-24T06:33:44.256611+0000","last_active":"2024-04-24T06:33:47.102014+0000","last_peered":"2024-04-24T06:33:47.102014+0000","last_clean":"2024-04-24T06:33:47.102014+0000","last_became_active":"2024-04-24T06:33:44.256303+0000","last_became_peered":"2024-04-24T06:33:44.256303+0000","last_unstale":"2024-04-24T06:33:47.102014+0000","last_undegraded":"2024-04-24T06:33:47.102014+0000","last_fullsized":"2024-04-24T06:33:47.102014+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:23:05.889753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572140+0000","last_change":"2024-04-24T06:33:45.260300+0000","last_active":"2024-04-24T06:33:46.572140+0000","last_peered":"2024-04-24T06:33:46.572140+0000","last_clean":"2024-04-24T06:33:46.572140+0000","last_became_active":"2024-04-24T06:33:45.258860+0000","last_became_peered":"2024-04-24T06:33:45.258860+0000","last_unstale":"2024-04-24T06:33:46.572140+0000","last_undegraded":"2024-04-24T06:33:46.572140+0000","last_fullsized":"2024-04-24T06:33:46.572140+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:06:21.828845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265458+0000","last_change":"2024-04-24T06:33:44.257380+0000","last_active":"2024-04-24T06:33:46.265458+0000","last_peered":"2024-04-24T06:33:46.265458+0000","last_clean":"2024-04-24T06:33:46.265458+0000","last_became_active":"2024-04-24T06:33:44.256966+0000","last_became_peered":"2024-04-24T06:33:44.256966+0000","last_unstale":"2024-04-24T06:33:46.265458+0000","last_undegraded":"2024-04-24T06:33:46.265458+0000","last_fullsized":"2024-04-24T06:33:46.265458+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.195266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.571678+0000","last_change":"2024-04-24T06:33:45.260565+0000","last_active":"2024-04-24T06:33:46.571678+0000","last_peered":"2024-04-24T06:33:46.571678+0000","last_clean":"2024-04-24T06:33:46.571678+0000","last_became_active":"2024-04-24T06:33:45.260141+0000","last_became_peered":"2024-04-24T06:33:45.260141+0000","last_unstale":"2024-04-24T06:33:46.571678+0000","last_undegraded":"2024-04-24T06:33:46.571678+0000","last_fullsized":"2024-04-24T06:33:46.571678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:35:22.766755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270129+0000","last_change":"2024-04-24T06:33:44.255503+0000","last_active":"2024-04-24T06:33:46.270129+0000","last_peered":"2024-04-24T06:33:46.270129+0000","last_clean":"2024-04-24T06:33:46.270129+0000","last_became_active":"2024-04-24T06:33:44.254397+0000","last_became_peered":"2024-04-24T06:33:44.254397+0000","last_unstale":"2024-04-24T06:33:46.270129+0000","last_undegraded":"2024-04-24T06:33:46.270129+0000","last_fullsized":"2024-04-24T06:33:46.270129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:00.195266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270934+0000","last_change":"2024-04-24T06:33:45.258877+0000","last_active":"2024-04-24T06:33:46.270934+0000","last_peered":"2024-04-24T06:33:46.270934+0000","last_clean":"2024-04-24T06:33:46.270934+0000","last_became_active":"2024-04-24T06:33:45.258529+0000","last_became_peered":"2024-04-24T06:33:45.258529+0000","last_unstale":"2024-04-24T06:33:46.270934+0000","last_undegraded":"2024-04-24T06:33:46.270934+0000","last_fullsized":"2024-04-24T06:33:46.270934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:42:57.581510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102036+0000","last_change":"2024-04-24T06:33:44.259767+0000","last_active":"2024-04-24T06:33:47.102036+0000","last_peered":"2024-04-24T06:33:47.102036+0000","last_clean":"2024-04-24T06:33:47.102036+0000","last_became_active":"2024-04-24T06:33:44.258750+0000","last_became_peered":"2024-04-24T06:33:44.258750+0000","last_unstale":"2024-04-24T06:33:47.102036+0000","last_undegraded":"2024-04-24T06:33:47.102036+0000","last_fullsized":"2024-04-24T06:33:47.102036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:22:40.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265357+0000","last_change":"2024-04-24T06:33:45.260239+0000","last_active":"2024-04-24T06:33:46.265357+0000","last_peered":"2024-04-24T06:33:46.265357+0000","last_clean":"2024-04-24T06:33:46.265357+0000","last_became_active":"2024-04-24T06:33:45.258989+0000","last_became_peered":"2024-04-24T06:33:45.258989+0000","last_unstale":"2024-04-24T06:33:46.265357+0000","last_undegraded":"2024-04-24T06:33:46.265357+0000","last_fullsized":"2024-04-24T06:33:46.265357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:01.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.265887+0000","last_change":"2024-04-24T06:33:44.252340+0000","last_active":"2024-04-24T06:33:46.265887+0000","last_peered":"2024-04-24T06:33:46.265887+0000","last_clean":"2024-04-24T06:33:46.265887+0000","last_became_active":"2024-04-24T06:33:44.251970+0000","last_became_peered":"2024-04-24T06:33:44.251970+0000","last_unstale":"2024-04-24T06:33:46.265887+0000","last_undegraded":"2024-04-24T06:33:46.265887+0000","last_fullsized":"2024-04-24T06:33:46.265887+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:46.781222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100703+0000","last_change":"2024-04-24T06:33:45.288763+0000","last_active":"2024-04-24T06:33:47.100703+0000","last_peered":"2024-04-24T06:33:47.100703+0000","last_clean":"2024-04-24T06:33:47.100703+0000","last_became_active":"2024-04-24T06:33:45.287535+0000","last_became_peered":"2024-04-24T06:33:45.287535+0000","last_unstale":"2024-04-24T06:33:47.100703+0000","last_undegraded":"2024-04-24T06:33:47.100703+0000","last_fullsized":"2024-04-24T06:33:47.100703+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:39:35.827262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258232+0000","last_change":"2024-04-24T06:33:44.250982+0000","last_active":"2024-04-24T06:33:46.258232+0000","last_peered":"2024-04-24T06:33:46.258232+0000","last_clean":"2024-04-24T06:33:46.258232+0000","last_became_active":"2024-04-24T06:33:44.250666+0000","last_became_peered":"2024-04-24T06:33:44.250666+0000","last_unstale":"2024-04-24T06:33:46.258232+0000","last_undegraded":"2024-04-24T06:33:46.258232+0000","last_fullsized":"2024-04-24T06:33:46.258232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:37:57.733050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103049+0000","last_change":"2024-04-24T06:33:45.285931+0000","last_active":"2024-04-24T06:33:47.103049+0000","last_peered":"2024-04-24T06:33:47.103049+0000","last_clean":"2024-04-24T06:33:47.103049+0000","last_became_active":"2024-04-24T06:33:45.285578+0000","last_became_peered":"2024-04-24T06:33:45.285578+0000","last_unstale":"2024-04-24T06:33:47.103049+0000","last_undegraded":"2024-04-24T06:33:47.103049+0000","last_fullsized":"2024-04-24T06:33:47.103049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:53.723991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.270892+0000","last_change":"2024-04-24T06:33:44.253357+0000","last_active":"2024-04-24T06:33:46.270892+0000","last_peered":"2024-04-24T06:33:46.270892+0000","last_clean":"2024-04-24T06:33:46.270892+0000","last_became_active":"2024-04-24T06:33:44.253043+0000","last_became_peered":"2024-04-24T06:33:44.253043+0000","last_unstale":"2024-04-24T06:33:46.270892+0000","last_undegraded":"2024-04-24T06:33:46.270892+0000","last_fullsized":"2024-04-24T06:33:46.270892+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:02.627841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.258431+0000","last_change":"2024-04-24T06:33:45.264024+0000","last_active":"2024-04-24T06:33:46.258431+0000","last_peered":"2024-04-24T06:33:46.258431+0000","last_clean":"2024-04-24T06:33:46.258431+0000","last_became_active":"2024-04-24T06:33:45.262733+0000","last_became_peered":"2024-04-24T06:33:45.262733+0000","last_unstale":"2024-04-24T06:33:46.258431+0000","last_undegraded":"2024-04-24T06:33:46.258431+0000","last_fullsized":"2024-04-24T06:33:46.258431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:34:47.734673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.257574+0000","last_change":"2024-04-24T06:33:44.254527+0000","last_active":"2024-04-24T06:33:46.257574+0000","last_peered":"2024-04-24T06:33:46.257574+0000","last_clean":"2024-04-24T06:33:46.257574+0000","last_became_active":"2024-04-24T06:33:44.253190+0000","last_became_peered":"2024-04-24T06:33:44.253190+0000","last_unstale":"2024-04-24T06:33:46.257574+0000","last_undegraded":"2024-04-24T06:33:46.257574+0000","last_fullsized":"2024-04-24T06:33:46.257574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:33.210407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.103518+0000","last_change":"2024-04-24T06:33:45.282664+0000","last_active":"2024-04-24T06:33:47.103518+0000","last_peered":"2024-04-24T06:33:47.103518+0000","last_clean":"2024-04-24T06:33:47.103518+0000","last_became_active":"2024-04-24T06:33:45.280883+0000","last_became_peered":"2024-04-24T06:33:45.280883+0000","last_unstale":"2024-04-24T06:33:47.103518+0000","last_undegraded":"2024-04-24T06:33:47.103518+0000","last_fullsized":"2024-04-24T06:33:47.103518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:01.211932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969025+0000","last_change":"2024-04-24T06:33:44.253348+0000","last_active":"2024-04-24T06:33:46.969025+0000","last_peered":"2024-04-24T06:33:46.969025+0000","last_clean":"2024-04-24T06:33:46.969025+0000","last_became_active":"2024-04-24T06:33:44.251208+0000","last_became_peered":"2024-04-24T06:33:44.251208+0000","last_unstale":"2024-04-24T06:33:46.969025+0000","last_undegraded":"2024-04-24T06:33:46.969025+0000","last_fullsized":"2024-04-24T06:33:46.969025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:58.618644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.969721+0000","last_change":"2024-04-24T06:33:45.267112+0000","last_active":"2024-04-24T06:33:46.969721+0000","last_peered":"2024-04-24T06:33:46.969721+0000","last_clean":"2024-04-24T06:33:46.969721+0000","last_became_active":"2024-04-24T06:33:45.266466+0000","last_became_peered":"2024-04-24T06:33:45.266466+0000","last_unstale":"2024-04-24T06:33:46.969721+0000","last_undegraded":"2024-04-24T06:33:46.969721+0000","last_fullsized":"2024-04-24T06:33:46.969721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:21.890534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.259423+0000","last_change":"2024-04-24T06:33:44.261959+0000","last_active":"2024-04-24T06:33:46.259423+0000","last_peered":"2024-04-24T06:33:46.259423+0000","last_clean":"2024-04-24T06:33:46.259423+0000","last_became_active":"2024-04-24T06:33:44.261655+0000","last_became_peered":"2024-04-24T06:33:44.261655+0000","last_unstale":"2024-04-24T06:33:46.259423+0000","last_undegraded":"2024-04-24T06:33:46.259423+0000","last_fullsized":"2024-04-24T06:33:46.259423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:15:46.359172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.100868+0000","last_change":"2024-04-24T06:33:45.269195+0000","last_active":"2024-04-24T06:33:47.100868+0000","last_peered":"2024-04-24T06:33:47.100868+0000","last_clean":"2024-04-24T06:33:47.100868+0000","last_became_active":"2024-04-24T06:33:45.268893+0000","last_became_peered":"2024-04-24T06:33:45.268893+0000","last_unstale":"2024-04-24T06:33:47.100868+0000","last_undegraded":"2024-04-24T06:33:47.100868+0000","last_fullsized":"2024-04-24T06:33:47.100868+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:52.370033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102533+0000","last_change":"2024-04-24T06:33:44.262023+0000","last_active":"2024-04-24T06:33:47.102533+0000","last_peered":"2024-04-24T06:33:47.102533+0000","last_clean":"2024-04-24T06:33:47.102533+0000","last_became_active":"2024-04-24T06:33:44.261729+0000","last_became_peered":"2024-04-24T06:33:44.261729+0000","last_unstale":"2024-04-24T06:33:47.102533+0000","last_undegraded":"2024-04-24T06:33:47.102533+0000","last_fullsized":"2024-04-24T06:33:47.102533+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:33.111917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:46.572152+0000","last_change":"2024-04-24T06:33:45.267029+0000","last_active":"2024-04-24T06:33:46.572152+0000","last_peered":"2024-04-24T06:33:46.572152+0000","last_clean":"2024-04-24T06:33:46.572152+0000","last_became_active":"2024-04-24T06:33:45.266658+0000","last_became_peered":"2024-04-24T06:33:45.266658+0000","last_unstale":"2024-04-24T06:33:46.572152+0000","last_undegraded":"2024-04-24T06:33:46.572152+0000","last_fullsized":"2024-04-24T06:33:46.572152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:44.205974+0000","last_clean_scrub_stamp":"2024-04-24T06:33:44.205974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:14:02.942459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.102245+0000","last_change":"2024-04-24T06:33:44.261630+0000","last_active":"2024-04-24T06:33:47.102245+0000","last_peered":"2024-04-24T06:33:47.102245+0000","last_clean":"2024-04-24T06:33:47.102245+0000","last_became_active":"2024-04-24T06:33:44.261285+0000","last_became_peered":"2024-04-24T06:33:44.261285+0000","last_unstale":"2024-04-24T06:33:47.102245+0000","last_undegraded":"2024-04-24T06:33:47.102245+0000","last_fullsized":"2024-04-24T06:33:47.102245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:33:43.189308+0000","last_clean_scrub_stamp":"2024-04-24T06:33:43.189308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:14.526573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-24T06:33:47.099685+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:33:47.099685+0000","last_peered":"2024-04-24T06:33:47.099685+0000","last_clean":"2024-04-24T06:33:47.099685+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:33:47.099685+0000","last_undegraded":"2024-04-24T06:33:47.099685+0000","last_fullsized":"2024-04-24T06:33:47.099685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":944649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":355277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":355893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":944033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:33:50.268 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:33:50.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:33:50.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.530+0000 7fbf63e00640 1 Processor -- start 2024-04-24T06:33:50.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.530+0000 7fbf63e00640 1 -- start start 2024-04-24T06:33:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.531+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf640d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.531+0000 7fbf63e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbf640d1040 0x7fbf640d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.531+0000 7fbf63e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf640d3660 con 0x7fbf640cfcb0 2024-04-24T06:33:50.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.531+0000 7fbf63e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbf640d2220 con 0x7fbf640d1040 2024-04-24T06:33:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.531+0000 7fbf63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf640d2ac0 con 0x7fbf6415e840 2024-04-24T06:33:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.532+0000 7fbf62000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbf640cfcb0 0x7fbf640d2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43972/0 (socket says 172.21.15.59:43972) 2024-04-24T06:33:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.532+0000 7fbf62000640 1 -- 172.21.15.59:0/2806273870 learned_addr learned my addr 172.21.15.59:0/2806273870 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.532+0000 7fbf62a00640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf640d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.532+0000 7fbf63400640 1 --2- 172.21.15.59:0/2806273870 >> v2:172.21.15.66:3300/0 conn(0x7fbf640d1040 0x7fbf640d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654397325 0 0) 0x7fbf640d3660 con 0x7fbf640cfcb0 2024-04-24T06:33:50.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf48003520 con 0x7fbf640cfcb0 2024-04-24T06:33:50.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1973271321 0 0) 0x7fbf48003520 con 0x7fbf640cfcb0 2024-04-24T06:33:50.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 msgr2=0x7fbf640d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:50.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf640d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 >> v2:172.21.15.66:3300/0 conn(0x7fbf640d1040 msgr2=0x7fbf640d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 --2- 172.21.15.59:0/2806273870 >> v2:172.21.15.66:3300/0 conn(0x7fbf640d1040 0x7fbf640d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf640d0470 con 0x7fbf640cfcb0 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbf50001670 con 0x7fbf640cfcb0 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 >> v1:172.21.15.59:6789/0 conn(0x7fbf640cfcb0 legacy=0x7fbf640d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf48003f80 0x7fbf48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf48004d40 0x7fbf480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf640d2220 con 0x7fbf48004d40 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf640d2ac0 con 0x7fbf48003f80 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.533+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4800b9c0 con 0x7fbf6415e840 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf48004d40 0x7fbf480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf62a00640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf62000640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf48003f80 0x7fbf48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 -- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 msgr2=0x7fbf480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf6415e840 0x7fbf480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 -- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf48003f80 msgr2=0x7fbf48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf48003f80 0x7fbf48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.534+0000 7fbf63400640 1 -- 172.21.15.59:0/2806273870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf48003520 con 0x7fbf48004d40 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.536+0000 7fbf63400640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf48004d40 0x7fbf480051c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00e170 tx=0x7fbf48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8bce80e91a933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.536+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c034020 con 0x7fbf48004d40 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.536+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c030430 con 0x7fbf48004d40 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.536+0000 7fbf61600640 1 -- 172.21.15.59:0/2806273870 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c009c10 con 0x7fbf48004d40 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 -- 172.21.15.59:0/2806273870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf48004d40 msgr2=0x7fbf480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2806273870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf48004d40 0x7fbf480051c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00e170 tx=0x7fbf48005830 comp rx=0 tx=0).stop 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63400640 1 -- 172.21.15.59:0/2806273870 reap_dead start 2024-04-24T06:33:50.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 -- 172.21.15.59:0/2806273870 shutdown_connections 2024-04-24T06:33:50.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 -- 172.21.15.59:0/2806273870 >> 172.21.15.59:0/2806273870 conn(0x7fbf6408bf90 msgr2=0x7fbf640d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:50.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 -- 172.21.15.59:0/2806273870 shutdown_connections 2024-04-24T06:33:50.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.537+0000 7fbf63e00640 1 -- 172.21.15.59:0/2806273870 wait complete. 2024-04-24T06:33:50.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.539+0000 7fbf63e00640 1 Processor -- start 2024-04-24T06:33:50.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.539+0000 7fbf63e00640 1 -- start start 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf48004d40 0x7fbf6415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64156800 0x7fbf64156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415ec20 con 0x7fbf640b7b80 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf640d2750 con 0x7fbf64156800 2024-04-24T06:33:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf63e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf640d0110 con 0x7fbf48004d40 2024-04-24T06:33:50.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf62000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf62000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46440/0 (socket says 172.21.15.59:46440) 2024-04-24T06:33:50.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.540+0000 7fbf62000640 1 -- 172.21.15.59:0/2475618230 learned_addr learned my addr 172.21.15.59:0/2475618230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf63400640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64156800 0x7fbf64156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 -- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf48004d40 msgr2=0x7fbf6415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf48004d40 0x7fbf6415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 -- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64156800 msgr2=0x7fbf64156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64156800 0x7fbf64156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 -- 172.21.15.59:0/2475618230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641571c0 con 0x7fbf640b7b80 2024-04-24T06:33:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.541+0000 7fbf62000640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e7e0 tx=0x7fbf5000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.542+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000c8d0 con 0x7fbf640b7b80 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.542+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50018070 con 0x7fbf640b7b80 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.542+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf640b8660 con 0x7fbf640b7b80 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.542+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000ccd0 con 0x7fbf640b7b80 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.542+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641548f0 con 0x7fbf640b7b80 2024-04-24T06:33:50.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.543+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf640b8b80 con 0x7fbf640b7b80 2024-04-24T06:33:50.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.544+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf50013420 con 0x7fbf640b7b80 2024-04-24T06:33:50.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.545+0000 7fbf5aa00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 0x7fbf4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.545+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fbf5008b830 con 0x7fbf640b7b80 2024-04-24T06:33:50.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.545+0000 7fbf62a00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 0x7fbf4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.546+0000 7fbf62a00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 0x7fbf4406ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009910 tx=0x7fbf54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.549+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf640b8b80 con 0x7fbf640b7b80 2024-04-24T06:33:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.829+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbf6404fd90 con 0x7fbf640b7b80 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.835+0000 7fbf5aa00640 1 -- 172.21.15.59:0/2475618230 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fbf641571c0 con 0x7fbf640b7b80 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:33:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 msgr2=0x7fbf4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 0x7fbf4406ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009910 tx=0x7fbf54008040 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 msgr2=0x7fbf640b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e7e0 tx=0x7fbf5000eca0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 shutdown_connections 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4406c920 0x7fbf4406ede0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf64156800 0x7fbf64156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf640b7b80 0x7fbf640b7fe0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 --2- 172.21.15.59:0/2475618230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf48004d40 0x7fbf6415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 >> 172.21.15.59:0/2475618230 conn(0x7fbf6408bf90 msgr2=0x7fbf640b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 shutdown_connections 2024-04-24T06:33:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:50.838+0000 7fbf63e00640 1 -- 172.21.15.59:0/2475618230 wait complete. 2024-04-24T06:33:50.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:33:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.061+0000 7fedf6600640 1 Processor -- start 2024-04-24T06:33:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.062+0000 7fedf6600640 1 -- start start 2024-04-24T06:33:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.063+0000 7fedf6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedf00ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.063+0000 7fedf6600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fedf015eb30 0x7fedf00cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.063+0000 7fedf6600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf00b8fc0 con 0x7fedf00cfa20 2024-04-24T06:33:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.063+0000 7fedf6600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fedf00cfe80 con 0x7fedf015eb30 2024-04-24T06:33:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.063+0000 7fedf6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedf00b92a0 con 0x7fedf015e750 2024-04-24T06:33:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedefe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fedf00cfa20 0x7fedf00d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43986/0 (socket says 172.21.15.59:43986) 2024-04-24T06:33:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedefe00640 1 -- 172.21.15.59:0/3082223210 learned_addr learned my addr 172.21.15.59:0/3082223210 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedef400640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedf00ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedeea00640 1 --2- 172.21.15.59:0/3082223210 >> v2:172.21.15.66:3300/0 conn(0x7fedf015eb30 0x7fedf00cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311959714 0 0) 0x7fedf00b8fc0 con 0x7fedf00cfa20 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.064+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd4003520 con 0x7fedf00cfa20 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2408175414 0 0) 0x7fedd4003520 con 0x7fedf00cfa20 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 msgr2=0x7fedf00ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedf00ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 >> v2:172.21.15.66:3300/0 conn(0x7fedf015eb30 msgr2=0x7fedf00cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 --2- 172.21.15.59:0/3082223210 >> v2:172.21.15.66:3300/0 conn(0x7fedf015eb30 0x7fedf00cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf00b9580 con 0x7fedf00cfa20 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fede8001670 con 0x7fedf00cfa20 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 >> v1:172.21.15.59:6789/0 conn(0x7fedf00cfa20 legacy=0x7fedf00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedd40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4003f80 0x7fedd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4004d40 0x7fedd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b92a0 con 0x7fedd4004d40 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedd400b9f0 con 0x7fedd4003f80 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedd400bbf0 con 0x7fedf015e750 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedefe00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4004d40 0x7fedd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedef400640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedd40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedeea00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4003f80 0x7fedd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedefe00640 1 -- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 msgr2=0x7fedd40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.065+0000 7fedefe00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf015e750 0x7fedd40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.066+0000 7fedefe00640 1 -- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4003f80 msgr2=0x7fedd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.066+0000 7fedefe00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4003f80 0x7fedd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.066+0000 7fedefe00640 1 -- 172.21.15.59:0/3082223210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd4003520 con 0x7fedd4004d40 2024-04-24T06:33:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.066+0000 7fedeea00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4003f80 0x7fedd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedefe00640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4004d40 0x7fedd40051c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fede800c160 tx=0x7fedd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2395ef1804ad50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8007c80 con 0x7fedd4004d40 2024-04-24T06:33:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede8005ad0 con 0x7fedd4004d40 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedee000640 1 -- 172.21.15.59:0/3082223210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8007070 con 0x7fedd4004d40 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedf6600640 1 -- 172.21.15.59:0/3082223210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4004d40 msgr2=0x7fedd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedf6600640 1 --2- 172.21.15.59:0/3082223210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4004d40 0x7fedd40051c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fede800c160 tx=0x7fedd4005830 comp rx=0 tx=0).stop 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.067+0000 7fedefe00640 1 -- 172.21.15.59:0/3082223210 reap_dead start 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.068+0000 7fedf6600640 1 -- 172.21.15.59:0/3082223210 shutdown_connections 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.068+0000 7fedf6600640 1 -- 172.21.15.59:0/3082223210 >> 172.21.15.59:0/3082223210 conn(0x7fedf0067b50 msgr2=0x7fedf0067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.068+0000 7fedf6600640 1 -- 172.21.15.59:0/3082223210 shutdown_connections 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.068+0000 7fedf6600640 1 -- 172.21.15.59:0/3082223210 wait complete. 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.070+0000 7fedf6600640 1 Processor -- start 2024-04-24T06:33:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.070+0000 7fedf6600640 1 -- start start 2024-04-24T06:33:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4004d40 0x7fedf015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf00ce4d0 0x7fedf00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b8950 con 0x7fedf014aa00 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b9b40 con 0x7fedd4004d40 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedf6600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fedf00b8ad0 con 0x7fedf00ce4d0 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46464/0 (socket says 172.21.15.59:46464) 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedefe00640 1 -- 172.21.15.59:0/1177463915 learned_addr learned my addr 172.21.15.59:0/1177463915 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedeea00640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf00ce4d0 0x7fedf00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.071+0000 7fedef400640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4004d40 0x7fedf015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 -- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf00ce4d0 msgr2=0x7fedf00cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf00ce4d0 0x7fedf00cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 -- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4004d40 msgr2=0x7fedf015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4004d40 0x7fedf015ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 -- 172.21.15.59:0/1177463915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01608a0 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedefe00640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fede802ad60 tx=0x7fedf00b9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8037040 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede8041070 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8007290 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf0160b80 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.072+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf0161140 con 0x7fedf014aa00 2024-04-24T06:33:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.074+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0068a80 con 0x7fedf014aa00 2024-04-24T06:33:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.075+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fede8007430 con 0x7fedf014aa00 2024-04-24T06:33:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.075+0000 7fede7400640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 0x7fedcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.076+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fede80ab8d0 con 0x7fedf014aa00 2024-04-24T06:33:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.076+0000 7fedef400640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 0x7fedcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.077+0000 7fedef400640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 0x7fedcc06edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fede0007520 tx=0x7fede000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.080+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fede8001630 con 0x7fedf014aa00 2024-04-24T06:33:51.189 INFO:tasks.ceph.mds.b.smithi066.stderr:2024-04-24T06:33:51.187+0000 7f4176a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:33:51.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.327+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedf0161940 con 0x7fedf014aa00 2024-04-24T06:33:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.328+0000 7fede7400640 1 -- 172.21.15.59:0/1177463915 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5174 (secure 0 0 0) 0x7fede8079cf0 con 0x7fedf014aa00 2024-04-24T06:33:51.329 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:51.329 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6759,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:33:47.780106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2419732942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2419732942},{"type":"v1","addr":"172.21.15.66:6835","nonce":2419732942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6771,"qdb_cluster":[6771]},"id":5}]} 2024-04-24T06:33:51.330 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 44 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 msgr2=0x7fedcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 0x7fedcc06edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fede0007520 tx=0x7fede000b040 comp rx=0 tx=0).stop 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 msgr2=0x7fedf014ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fede802ad60 tx=0x7fedf00b9750 comp rx=0 tx=0).stop 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 shutdown_connections 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fedcc06c8f0 0x7fedcc06edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedf014aa00 0x7fedf014ae80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fedf00ce4d0 0x7fedf00cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 --2- 172.21.15.59:0/1177463915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fedd4004d40 0x7fedf015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 >> 172.21.15.59:0/1177463915 conn(0x7fedf0067b50 msgr2=0x7fedf015ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 shutdown_connections 2024-04-24T06:33:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.331+0000 7fedf6600640 1 -- 172.21.15.59:0/1177463915 wait complete. 2024-04-24T06:33:51.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:33:44.703546+0000', 'modified': '2024-04-24T06:33:47.780106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6771': {'gid': 6771, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/2419732942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 2419732942}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2419732942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6771, 'qdb_cluster': [6771]} 2024-04-24T06:33:51.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:33:51.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:33:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.977+0000 7f9812600640 1 Processor -- start 2024-04-24T06:33:51.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.978+0000 7f9812600640 1 -- start start 2024-04-24T06:33:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.979+0000 7f9812600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f980c15ea40 0x7f980c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.979+0000 7f9812600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.979+0000 7f9812600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f980c15fd30 con 0x7f980c0d0f80 2024-04-24T06:33:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.979+0000 7f9812600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f980c0cf930 con 0x7f980c15ea40 2024-04-24T06:33:51.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.979+0000 7f9812600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f980c0cfc10 con 0x7f980c15f880 2024-04-24T06:33:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.980+0000 7f9811200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f980c0d0f80 0x7f980c0d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:60172/0 (socket says 172.21.15.59:60172) 2024-04-24T06:33:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.980+0000 7f9811c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.980+0000 7f9811200640 1 -- 172.21.15.59:0/3625923107 learned_addr learned my addr 172.21.15.59:0/3625923107 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:51.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.980+0000 7f980be00640 1 --2- 172.21.15.59:0/3625923107 >> v2:172.21.15.66:3300/0 conn(0x7f980c15ea40 0x7f980c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.981+0000 7f980be00640 1 -- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 msgr2=0x7f980c0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.981+0000 7f980be00640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c0d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.981+0000 7f980be00640 1 -- 172.21.15.59:0/3625923107 >> v1:172.21.15.66:6789/0 conn(0x7f980c0d0f80 legacy=0x7f980c0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.981+0000 7f980be00640 1 -- 172.21.15.59:0/3625923107 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c0cfef0 con 0x7f980c15ea40 2024-04-24T06:33:51.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.981+0000 7f9811c00640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:33:51.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.982+0000 7f980be00640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0cf250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f72229156ae9b059 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.983+0000 7f980b400640 1 -- 172.21.15.59:0/3625923107 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc031600 con 0x7f980c15ea40 2024-04-24T06:33:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.983+0000 7f980b400640 1 -- 172.21.15.59:0/3625923107 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0317a0 con 0x7f980c15ea40 2024-04-24T06:33:51.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.983+0000 7f980b400640 1 -- 172.21.15.59:0/3625923107 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc031b70 con 0x7f980c15ea40 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 -- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 msgr2=0x7f980c0cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0cf250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc02a5d0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 -- 172.21.15.59:0/3625923107 shutdown_connections 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 --2- 172.21.15.59:0/3625923107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0cf250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 -- 172.21.15.59:0/3625923107 >> 172.21.15.59:0/3625923107 conn(0x7f980c08bf90 msgr2=0x7f980c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 -- 172.21.15.59:0/3625923107 shutdown_connections 2024-04-24T06:33:51.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.984+0000 7f9812600640 1 -- 172.21.15.59:0/3625923107 wait complete. 2024-04-24T06:33:51.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.986+0000 7f9812600640 1 Processor -- start 2024-04-24T06:33:51.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.986+0000 7f9812600640 1 -- start start 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.987+0000 7f9812600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.987+0000 7f9812600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9812600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c1f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9812600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0d2850 con 0x7f980c0d0f80 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9812600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0d04b0 con 0x7f980c15ea40 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9812600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f980c0d29d0 con 0x7f980c15f880 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46470/0 (socket says 172.21.15.59:46470) 2024-04-24T06:33:51.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 -- 172.21.15.59:0/970420100 learned_addr learned my addr 172.21.15.59:0/970420100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f980be00640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811c00640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c1f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 -- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 msgr2=0x7f980c1f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c1f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 -- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 msgr2=0x7f980c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.988+0000 7f9811200640 1 -- 172.21.15.59:0/970420100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c0ba420 con 0x7f980c0d0f80 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9811200640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f97f400c400 tx=0x7f97f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9808c00640 1 -- 172.21.15.59:0/970420100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4017070 con 0x7f980c0d0f80 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9808c00640 1 -- 172.21.15.59:0/970420100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f4005ce0 con 0x7f980c0d0f80 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9808c00640 1 -- 172.21.15.59:0/970420100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40072f0 con 0x7f980c0d0f80 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c0b9b80 con 0x7f980c0d0f80 2024-04-24T06:33:51.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.989+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c0ba140 con 0x7f980c0d0f80 2024-04-24T06:33:51.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.992+0000 7f9808c00640 1 -- 172.21.15.59:0/970420100 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97f401a070 con 0x7f980c0d0f80 2024-04-24T06:33:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.994+0000 7f9808c00640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 0x7f97e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.994+0000 7f9808c00640 1 -- 172.21.15.59:0/970420100 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f97f408af70 con 0x7f980c0d0f80 2024-04-24T06:33:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.994+0000 7f980be00640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 0x7f97e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.994+0000 7f9816b6b740 1 Processor -- start 2024-04-24T06:33:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.994+0000 7f9816b6b740 1 -- start start 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f980be00640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 0x7f97e806edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034000 tx=0x7f97fc02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92746310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92775e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca926b0b10 con 0x55ca92776360 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca926572b0 con 0x55ca92793980 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.995+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ca926acad0 con 0x55ca92776aa0 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46486/0 (socket says 172.21.15.59:46486) 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 -- 172.21.15.59:0/1894391993 learned_addr learned my addr 172.21.15.59:0/1894391993 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811200640 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92746310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f980be00640 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92775e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 -- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 msgr2=0x55ca92775e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92775e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 -- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 msgr2=0x55ca92746310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92746310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 -- 172.21.15.59:0/1894391993 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca923c32d0 con 0x55ca92776360 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.996+0000 7f9811c00640 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f98040099e0 tx=0x7f9804009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deef169149ac6fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.997+0000 7f9801600640 1 -- 172.21.15.59:0/1894391993 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804018070 con 0x55ca92776360 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.997+0000 7f9801600640 1 -- 172.21.15.59:0/1894391993 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804010ca0 con 0x55ca92776360 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.997+0000 7f9801600640 1 -- 172.21.15.59:0/1894391993 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980400f040 con 0x55ca92776360 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.997+0000 7f9816b6b740 1 -- 172.21.15.59:0/1894391993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 msgr2=0x55ca9274b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.997+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f98040099e0 tx=0x7f9804009190 comp rx=0 tx=0).stop 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 -- 172.21.15.59:0/1894391993 shutdown_connections 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776360 0x55ca9274b170 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92775e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1894391993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92746310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 -- 172.21.15.59:0/1894391993 >> 172.21.15.59:0/1894391993 conn(0x55ca92748580 msgr2=0x55ca92790e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 -- 172.21.15.59:0/1894391993 shutdown_connections 2024-04-24T06:33:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 -- 172.21.15.59:0/1894391993 wait complete. 2024-04-24T06:33:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.998+0000 7f9816b6b740 1 -- start start 2024-04-24T06:33:52.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:51.999+0000 7f9816b6b740 1 Processor -- start 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca92776f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 0x55ca927783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca9274a730 con 0x55ca92776aa0 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ca9266a590 con 0x55ca92777f30 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ca92749b20 con 0x55ca92793980 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9816b6b740 10 client.0 _ll_register_callbacks cb 0x55ca9264d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:52.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9811200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9811200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46490/0 (socket says 172.21.15.59:46490) 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9811200640 1 -- 172.21.15.59:0/1456859156 learned_addr learned my addr 172.21.15.59:0/1456859156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9811c00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 0x55ca927783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.000+0000 7f9811200640 1 -- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 msgr2=0x55ca92776f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f980be00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca92776f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811200640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca92776f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811200640 1 -- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 msgr2=0x55ca927783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811200640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 0x55ca927783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811200640 1 -- 172.21.15.59:0/1456859156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9248d6b0 con 0x55ca92776aa0 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811c00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 0x55ca927783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f980be00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca92776f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9811200640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f97f4001eb0 tx=0x7f97f4058dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e4517bb5b26850 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f97fa000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40076a0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f97f40711c0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9804010ca0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.001+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f97f4089120 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 10 client.6909 fetch_fsmap learned FSMap version 44 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ca927741f0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804010ca0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f97f40580e0 con 0x55ca92776aa0 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 10 client.6909 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 10 client.6909 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:33:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 20 client.6909 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca927a4e70 0x55ca92774fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 4 client.6909 mds_command: new command op to 6771 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:33:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.002+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 --> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca926be530 con 0x55ca927a4e70 2024-04-24T06:33:52.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.003+0000 7f980be00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca927a4e70 0x55ca92774fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.004+0000 7f980be00640 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca927a4e70 0x55ca92774fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9c3b44d77f2899 server_cookie=7eca6c9e3c1de2c in_seq=0 out_seq=0 2024-04-24T06:33:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.004+0000 7f97fa000640 10 client.6909 ms_handle_connect on v2:172.21.15.66:6833/2419732942 2024-04-24T06:33:52.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.006+0000 7f97fa000640 1 -- 172.21.15.59:0/1456859156 <== mds.0 v2:172.21.15.66:6833/2419732942 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca926be530 con 0x55ca927a4e70 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.006+0000 7f97fa000640 10 client.6909 handle_command_reply: tid=0 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.006+0000 7f9816b6b740 1 client.6909 shutdown 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 msgr2=0x55ca92762840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f97f4001eb0 tx=0x7f97f4058dd0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 shutdown_connections 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca927a4e70 0x55ca92774fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92777f30 0x55ca927783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca92776f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 --2- 172.21.15.59:0/1456859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92776aa0 0x55ca92762840 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f97fa000640 0 client.6909 ms_handle_reset on v2:172.21.15.66:6833/2419732942 2024-04-24T06:33:52.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 >> 172.21.15.59:0/1456859156 conn(0x55ca92776360 msgr2=0x55ca927724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 shutdown_connections 2024-04-24T06:33:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 1 -- 172.21.15.59:0/1456859156 wait complete. 2024-04-24T06:33:52.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.007+0000 7f9816b6b740 20 client.6909 trim_cache size 0 max 16384 2024-04-24T06:33:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.027+0000 7f9816b6b740 1 Processor -- start 2024-04-24T06:33:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.027+0000 7f9816b6b740 1 -- start start 2024-04-24T06:33:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92778460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92771820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f980c200d10 con 0x55ca92777f30 2024-04-24T06:33:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0c8870 con 0x55ca92793980 2024-04-24T06:33:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.028+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f980c0b2210 con 0x55ca92776aa0 2024-04-24T06:33:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 -- 172.21.15.59:0/2542916064 learned_addr learned my addr 172.21.15.59:0/2542916064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f9811c00640 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92771820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 -- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 msgr2=0x55ca92778460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92778460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 -- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 msgr2=0x55ca92771820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92771820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 -- 172.21.15.59:0/2542916064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca92777300 con 0x55ca92777f30 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.029+0000 7f980be00640 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009eb0 tx=0x7f97fc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c9243b83b25881 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.030+0000 7f9800c00640 1 -- 172.21.15.59:0/2542916064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc051c10 con 0x55ca92777f30 2024-04-24T06:33:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.030+0000 7f9800c00640 1 -- 172.21.15.59:0/2542916064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc051db0 con 0x55ca92777f30 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.030+0000 7f9800c00640 1 -- 172.21.15.59:0/2542916064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0514d0 con 0x55ca92777f30 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.030+0000 7f9816b6b740 1 -- 172.21.15.59:0/2542916064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 msgr2=0x55ca92745620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.030+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009eb0 tx=0x7f97fc030000 comp rx=0 tx=0).stop 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 -- 172.21.15.59:0/2542916064 shutdown_connections 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92793980 0x55ca92771820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92745620 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2542916064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92776aa0 0x55ca92778460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 -- 172.21.15.59:0/2542916064 >> 172.21.15.59:0/2542916064 conn(0x55ca92776360 msgr2=0x55ca92774f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 -- 172.21.15.59:0/2542916064 shutdown_connections 2024-04-24T06:33:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 -- 172.21.15.59:0/2542916064 wait complete. 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.031+0000 7f9816b6b740 1 -- start start 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.032+0000 7f9816b6b740 1 Processor -- start 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92776aa0 0x55ca927b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca9279fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ca927a4cf0 con 0x55ca92777f30 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0c8220 con 0x55ca92776aa0 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ca927a5340 con 0x55ca92793980 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f9816b6b740 10 client.0 _ll_register_callbacks cb 0x7f97f4059480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46508/0 (socket says 172.21.15.59:46508) 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 -- 172.21.15.59:0/2786558708 learned_addr learned my addr 172.21.15.59:0/2786558708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 -- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 msgr2=0x55ca9279fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca9279fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 -- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92776aa0 msgr2=0x55ca927b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92776aa0 0x55ca927b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.033+0000 7f980be00640 1 -- 172.21.15.59:0/2786558708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca92792dc0 con 0x55ca92777f30 2024-04-24T06:33:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f980be00640 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f97fc031910 tx=0x7f97fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585a3577b3713dea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f97fa000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc061060 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97fc051db0 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f97fc049ab0 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f97fc049cb0 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f9816b6b740 10 client.6915 fetch_fsmap learned FSMap version 44 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.034+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55ca92771350 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc051db0 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f97fc0512d0 con 0x55ca92777f30 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 10 client.6915 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 10 client.6915 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 20 client.6915 populate_metadata read hostname 'smithi059' 2024-04-24T06:33:52.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca92773370 0x55ca927867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 4 client.6915 mds_command: new command op to 6771 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:33:52.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.035+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 --> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] -- command(tid 0: {"prefix": "status"}) -- 0x55ca926b83a0 con 0x55ca92773370 2024-04-24T06:33:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.036+0000 7f9811200640 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca92773370 0x55ca927867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.037+0000 7f9811200640 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca92773370 0x55ca927867b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de66f68379ec0121 server_cookie=e867cbc14b486950 in_seq=0 out_seq=0 2024-04-24T06:33:52.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.037+0000 7f97fa000640 10 client.6915 ms_handle_connect on v2:172.21.15.66:6833/2419732942 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.037+0000 7f97fa000640 1 -- 172.21.15.59:0/2786558708 <== mds.0 v2:172.21.15.66:6833/2419732942 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ca926b83a0 con 0x55ca92773370 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f97fa000640 10 client.6915 handle_command_reply: tid=0 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 client.6915 shutdown 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 msgr2=0x55ca92813490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f97fc031910 tx=0x7f97fc004ab0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 shutdown_connections 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:6833/2419732942,v1:172.21.15.66:6835/2419732942] conn(0x55ca92773370 0x55ca927867b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ca92793980 0x55ca9279fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ca92777f30 0x55ca92813490 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 --2- 172.21.15.59:0/2786558708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca92776aa0 0x55ca927b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f97fa000640 0 client.6915 ms_handle_reset on v2:172.21.15.66:6833/2419732942 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 >> 172.21.15.59:0/2786558708 conn(0x55ca92776360 msgr2=0x55ca92741800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 shutdown_connections 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 1 -- 172.21.15.59:0/2786558708 wait complete. 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.038+0000 7f9816b6b740 20 client.6915 trim_cache size 0 max 16384 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:33:52.042 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6771, 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7857316729999999, 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 44, 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 69, 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 69, 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.168562088 2024-04-24T06:33:52.043 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 msgr2=0x7f97e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 0x7f97e806edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f97fc034000 tx=0x7f97fc02f000 comp rx=0 tx=0).stop 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 msgr2=0x7f980c158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f97f400c400 tx=0x7f97f400c8c0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 shutdown_connections 2024-04-24T06:33:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f97e806c8f0 0x7f97e806edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f980c15f880 0x7f980c1f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f980c15ea40 0x7f980c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 --2- 172.21.15.59:0/970420100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f980c0d0f80 0x7f980c158590 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 >> 172.21.15.59:0/970420100 conn(0x7f980c08bf90 msgr2=0x7f980c161960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 shutdown_connections 2024-04-24T06:33:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.039+0000 7f9812600640 1 -- 172.21.15.59:0/970420100 wait complete. 2024-04-24T06:33:52.059 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:33:52.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.258+0000 7f407ae00640 1 Processor -- start 2024-04-24T06:33:52.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.258+0000 7f407ae00640 1 -- start start 2024-04-24T06:33:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.259+0000 7f407ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f407415eb80 0x7f40740b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f407ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40740cf540 0x7f40740d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f407ae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40740d0740 con 0x7f407406ccf0 2024-04-24T06:33:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f407ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f407415ef60 con 0x7f407415eb80 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f407ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40740b2a50 con 0x7f40740cf540 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f407415eb80 0x7f40740b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f407415eb80 0x7f40740b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46516/0 (socket says 172.21.15.59:46516) 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 -- 172.21.15.59:0/1771839716 learned_addr learned my addr 172.21.15.59:0/1771839716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4078e00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40740cf540 0x7f40740d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 -- 172.21.15.59:0/1771839716 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40740cf540 msgr2=0x7f40740d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40740cf540 0x7f40740d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 -- 172.21.15.59:0/1771839716 >> v1:172.21.15.66:6789/0 conn(0x7f407406ccf0 legacy=0x7f40740cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:52.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.260+0000 7f4073e00640 1 -- 172.21.15.59:0/1771839716 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40740b2d30 con 0x7f407415eb80 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.262+0000 7f4073e00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f40740b2220 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f40740b5100 tx=0x7f406402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b1012be6c63aae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.262+0000 7f4072a00640 1 -- 172.21.15.59:0/1771839716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064037070 con 0x7f407415eb80 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.262+0000 7f4072a00640 1 -- 172.21.15.59:0/1771839716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406402ae50 con 0x7f407415eb80 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.262+0000 7f4072a00640 1 -- 172.21.15.59:0/1771839716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064032660 con 0x7f407415eb80 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 -- 172.21.15.59:0/1771839716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 msgr2=0x7f40740b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f40740b2220 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f40740b5100 tx=0x7f406402a140 comp rx=0 tx=0).stop 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 -- 172.21.15.59:0/1771839716 shutdown_connections 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40740cf540 0x7f40740d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 --2- 172.21.15.59:0/1771839716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f40740b2220 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 -- 172.21.15.59:0/1771839716 >> 172.21.15.59:0/1771839716 conn(0x7f40740d25b0 msgr2=0x7f40740d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 -- 172.21.15.59:0/1771839716 shutdown_connections 2024-04-24T06:33:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.263+0000 7f407ae00640 1 -- 172.21.15.59:0/1771839716 wait complete. 2024-04-24T06:33:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.265+0000 7f407ae00640 1 Processor -- start 2024-04-24T06:33:52.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.265+0000 7f407ae00640 1 -- start start 2024-04-24T06:33:52.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f407406ccf0 0x7f407415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40740cf540 0x7f407415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40740b2760 con 0x7f407415eb80 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40740b32f0 con 0x7f40740cf540 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f407ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40740d1140 con 0x7f407406ccf0 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4078e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4078e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46530/0 (socket says 172.21.15.59:46530) 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4078e00640 1 -- 172.21.15.59:0/326538506 learned_addr learned my addr 172.21.15.59:0/326538506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4073e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f407406ccf0 0x7f407415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4073400640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40740cf540 0x7f407415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.266+0000 7f4078e00640 1 -- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f407406ccf0 msgr2=0x7f407415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f4078e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f407406ccf0 0x7f407415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f4078e00640 1 -- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40740cf540 msgr2=0x7f407415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f4078e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40740cf540 0x7f407415a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f4078e00640 1 -- 172.21.15.59:0/326538506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407415a970 con 0x7f407415eb80 2024-04-24T06:33:52.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f4078e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f406c00cd40 tx=0x7f406c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c003b90 con 0x7f407415eb80 2024-04-24T06:33:52.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406c003d30 con 0x7f407415eb80 2024-04-24T06:33:52.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407414d010 con 0x7f407415eb80 2024-04-24T06:33:52.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c004240 con 0x7f407415eb80 2024-04-24T06:33:52.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.267+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40740b6700 con 0x7f407415eb80 2024-04-24T06:33:52.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.268+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4074069190 con 0x7f407415eb80 2024-04-24T06:33:52.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.269+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f406c0043e0 con 0x7f407415eb80 2024-04-24T06:33:52.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.270+0000 7f406be00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 0x7f405406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:52.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.271+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f406c08c6b0 con 0x7f407415eb80 2024-04-24T06:33:52.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.271+0000 7f4073e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 0x7f405406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:52.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.271+0000 7f4073e00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 0x7f405406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f406402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:52.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.275+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f406c05aad0 con 0x7f407415eb80 2024-04-24T06:33:52.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.520+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4074070450 con 0x7f407415eb80 2024-04-24T06:33:52.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.521+0000 7f406be00640 1 -- 172.21.15.59:0/326538506 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f406c05a3d0 con 0x7f407415eb80 2024-04-24T06:33:52.522 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 msgr2=0x7f405406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 0x7f405406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f406402e000 comp rx=0 tx=0).stop 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 msgr2=0x7f407414cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f406c00cd40 tx=0x7f406c00a660 comp rx=0 tx=0).stop 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 shutdown_connections 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f405406c8f0 0x7f405406edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f407415eb80 0x7f407414cad0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40740cf540 0x7f407415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 --2- 172.21.15.59:0/326538506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f407406ccf0 0x7f407415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:52.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 >> 172.21.15.59:0/326538506 conn(0x7f40740d25b0 msgr2=0x7f407409b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:52.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 shutdown_connections 2024-04-24T06:33:52.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52.524+0000 7f407ae00640 1 -- 172.21.15.59:0/326538506 wait complete. 2024-04-24T06:33:52.545 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:33:52.545 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:33:52.545 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:33:52.546 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:33:52.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:52.546 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.599 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:33:52.600 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:33:52.601 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:33:52.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:52.603 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:33:52.684 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:33:52.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:52.689 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:33:52.743 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:33:52.743 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:33:52.743 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:33:52.743 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:33:52.743 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:33:52.744 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:33:52.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:33:52.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:33:52.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:52.967 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:33:53.015 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:33:53.015 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:33:53.016 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:33:53.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:53.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:33:53.106 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:33:53.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:33:53.106 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:53.161 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:33:53.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:33:53.162 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:53.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:33:53.313 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:33:53.369 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:33:53.369 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:33:53.370 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:33:53.371 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:33:53.372 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:33:53.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:53.373 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:33:53.424 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:33:53.424 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:33:53.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:33:53.494 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:33:53.540 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:33:53.541 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:33:53.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:53.542 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:33:53.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:33:53.598+0000 7f31e3e0d480 -1 init, newargv = 0x55eb44ce6400 newargc=15 2024-04-24T06:33:53.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57511]: starting ceph client 2024-04-24T06:33:53.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57511]: starting fuse 2024-04-24T06:33:53.628 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:33:53.628 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:33:53.628 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:33:53.629 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:33:53.630 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:33:53.826 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.57511.asok 2024-04-24T06:33:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:33:53.843 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57511.asok 2024-04-24T06:33:53.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:53.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57511.asok status 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:33:54.081 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "57511", 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6927, 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 6927 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:33:54.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1560896381 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1560896381 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.6927 192.168.0.1:0/1560896381", 2024-04-24T06:33:54.083 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1560896381", 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 44, 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 69, 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:33:54.084 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:33:54.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:54.096 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:33:54.096 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:54.151 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:33:54.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:54.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:54.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:54.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:54.242 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-24T06:33:54.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:54.316 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-24T06:33:54.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:54.383 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-24T06:33:54.444 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:33:54.444 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:33:54.444 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:33:54.445 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:33:54.445 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-24T06:33:54.445 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:33:54.593 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:33:54.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:33:54.608 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:54.608 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:54.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:54.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.921+0000 7f5423e00640 1 Processor -- start 2024-04-24T06:33:54.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.922+0000 7f5423e00640 1 -- start start 2024-04-24T06:33:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f5423e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f542415e7c0 0x7f54240d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f5423e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54240cf030 con 0x7f54240d2630 2024-04-24T06:33:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f5423e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54240b6a40 con 0x7f54240d0d90 2024-04-24T06:33:54.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f5423e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f54240b7000 con 0x7f542415e7c0 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f541be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f54240d2630 0x7f54240b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43988/0 (socket says 172.21.15.59:43988) 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f5422a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f542415e7c0 0x7f54240d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.923+0000 7f541be00640 1 -- 172.21.15.59:0/38899355 learned_addr learned my addr 172.21.15.59:0/38899355 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.924+0000 7f5422a00640 1 -- 172.21.15.59:0/38899355 >> v1:172.21.15.66:6789/0 conn(0x7f54240d0d90 legacy=0x7f54240cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.924+0000 7f5422a00640 1 -- 172.21.15.59:0/38899355 >> v1:172.21.15.59:6789/0 conn(0x7f54240d2630 legacy=0x7f54240b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.924+0000 7f5422a00640 1 -- 172.21.15.59:0/38899355 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54240b6d20 con 0x7f542415e7c0 2024-04-24T06:33:54.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.924+0000 7f5422000640 1 -- 172.21.15.59:0/38899355 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760705483 0 0) 0x7f540c001110 con 0x7f54240d2630 2024-04-24T06:33:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.925+0000 7f5422a00640 1 --2- 172.21.15.59:0/38899355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f542415e7c0 0x7f54240d20f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54140097a0 tx=0x7f541402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ac1c405e39508 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.925+0000 7f5422000640 1 -- 172.21.15.59:0/38899355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414037070 con 0x7f542415e7c0 2024-04-24T06:33:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.925+0000 7f5422000640 1 -- 172.21.15.59:0/38899355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5414032440 con 0x7f542415e7c0 2024-04-24T06:33:54.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.925+0000 7f5422000640 1 -- 172.21.15.59:0/38899355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414032810 con 0x7f542415e7c0 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 -- 172.21.15.59:0/38899355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f542415e7c0 msgr2=0x7f54240d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 --2- 172.21.15.59:0/38899355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f542415e7c0 0x7f54240d20f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54140097a0 tx=0x7f541402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 -- 172.21.15.59:0/38899355 shutdown_connections 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 --2- 172.21.15.59:0/38899355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f542415e7c0 0x7f54240d20f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 -- 172.21.15.59:0/38899355 >> 172.21.15.59:0/38899355 conn(0x7f542409d920 msgr2=0x7f542408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 -- 172.21.15.59:0/38899355 shutdown_connections 2024-04-24T06:33:54.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.926+0000 7f5423e00640 1 -- 172.21.15.59:0/38899355 wait complete. 2024-04-24T06:33:54.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.928+0000 7f5423e00640 1 Processor -- start 2024-04-24T06:33:54.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.928+0000 7f5423e00640 1 -- start start 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54240d2630 0x7f54241481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f542415e7c0 0x7f542414a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f54240b6700 con 0x7f54240d0d90 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f54240cf620 con 0x7f542415e7c0 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5423e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f54240d1a80 con 0x7f54240d2630 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46554/0 (socket says 172.21.15.59:46554) 2024-04-24T06:33:54.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 -- 172.21.15.59:0/4106737466 learned_addr learned my addr 172.21.15.59:0/4106737466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 -- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54240d2630 msgr2=0x7f54241481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54240d2630 0x7f54241481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 -- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f542415e7c0 msgr2=0x7f542414a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f542415e7c0 0x7f542414a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.929+0000 7f5422a00640 1 -- 172.21.15.59:0/4106737466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54240cde30 con 0x7f54240d0d90 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f5422a00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5414000c00 tx=0x7f54140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414037040 con 0x7f54240d0d90 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5414004230 con 0x7f54240d0d90 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414032e40 con 0x7f54240d0d90 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54240cad00 con 0x7f54240d0d90 2024-04-24T06:33:54.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.930+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54240ce390 con 0x7f54240d0d90 2024-04-24T06:33:54.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.931+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542404fd90 con 0x7f54240d0d90 2024-04-24T06:33:54.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.932+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5414030770 con 0x7f54240d0d90 2024-04-24T06:33:54.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.933+0000 7f541b400640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 0x7f540006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:54.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.934+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f54140acf30 con 0x7f54240d0d90 2024-04-24T06:33:54.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.934+0000 7f541be00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 0x7f540006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:54.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.934+0000 7f541be00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 0x7f540006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f54241607c0 tx=0x7f540c001310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:54.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:54.944+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54240cde30 con 0x7f54240d0d90 2024-04-24T06:33:55.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.149+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5424161090 con 0x7f54240d0d90 2024-04-24T06:33:55.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.153+0000 7f541b400640 1 -- 172.21.15.59:0/4106737466 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16584 (secure 0 0 0) 0x7f541407b2c0 con 0x7f54240d0d90 2024-04-24T06:33:55.154 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:55.154 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:46.242082+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:55.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 msgr2=0x7f540006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 0x7f540006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f54241607c0 tx=0x7f540c001310 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 msgr2=0x7f54240cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5414000c00 tx=0x7f54140046c0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 shutdown_connections 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f540006c8f0 0x7f540006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f542415e7c0 0x7f542414a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f54240d2630 0x7f54241481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 --2- 172.21.15.59:0/4106737466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54240d0d90 0x7f54240cb5f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 >> 172.21.15.59:0/4106737466 conn(0x7f542409d920 msgr2=0x7f54240d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.156+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 shutdown_connections 2024-04-24T06:33:55.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.157+0000 7f5423e00640 1 -- 172.21.15.59:0/4106737466 wait complete. 2024-04-24T06:33:55.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:55.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.394+0000 7f833e800640 1 Processor -- start 2024-04-24T06:33:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.394+0000 7f833e800640 1 -- start start 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f833e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f83380b4670 0x7f8338071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f833e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83380cfba0 0x7f83380722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f833e800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83380b35d0 con 0x7f8338162c10 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f833e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f83380b38b0 con 0x7f83380cfba0 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f833e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f83380b3b90 con 0x7f83380b4670 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83380cfba0 0x7f83380722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83380cfba0 0x7f83380722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 -- 172.21.15.59:0/2009914112 learned_addr learned my addr 172.21.15.59:0/2009914112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 -- 172.21.15.59:0/2009914112 >> v2:172.21.15.66:3300/0 conn(0x7f83380b4670 msgr2=0x7f8338071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 --2- 172.21.15.59:0/2009914112 >> v2:172.21.15.66:3300/0 conn(0x7f83380b4670 0x7f8338071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 -- 172.21.15.59:0/2009914112 >> v1:172.21.15.66:6789/0 conn(0x7f8338162c10 legacy=0x7f83380d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.396+0000 7f8337e00640 1 -- 172.21.15.59:0/2009914112 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83380b3e70 con 0x7f83380cfba0 2024-04-24T06:33:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.398+0000 7f8337e00640 1 --2- 172.21.15.59:0/2009914112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380cfba0 0x7f83380722b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f83380b5b20 tx=0x7f833002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353338a116c97b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.398+0000 7f8336000640 1 -- 172.21.15.59:0/2009914112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330036070 con 0x7f83380cfba0 2024-04-24T06:33:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.398+0000 7f8336000640 1 -- 172.21.15.59:0/2009914112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8330031600 con 0x7f83380cfba0 2024-04-24T06:33:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.398+0000 7f8336000640 1 -- 172.21.15.59:0/2009914112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300319d0 con 0x7f83380cfba0 2024-04-24T06:33:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 -- 172.21.15.59:0/2009914112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380cfba0 msgr2=0x7f83380722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 --2- 172.21.15.59:0/2009914112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380cfba0 0x7f83380722b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f83380b5b20 tx=0x7f833002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 -- 172.21.15.59:0/2009914112 shutdown_connections 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 --2- 172.21.15.59:0/2009914112 >> v2:172.21.15.66:3300/0 conn(0x7f83380b4670 0x7f8338071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 --2- 172.21.15.59:0/2009914112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380cfba0 0x7f83380722b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 -- 172.21.15.59:0/2009914112 >> 172.21.15.59:0/2009914112 conn(0x7f8338067d90 msgr2=0x7f833806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 -- 172.21.15.59:0/2009914112 shutdown_connections 2024-04-24T06:33:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.399+0000 7f833e800640 1 -- 172.21.15.59:0/2009914112 wait complete. 2024-04-24T06:33:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.401+0000 7f833e800640 1 Processor -- start 2024-04-24T06:33:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.401+0000 7f833e800640 1 -- start start 2024-04-24T06:33:55.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83380cfba0 0x7f83380b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338162c10 0x7f833815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8338163170 con 0x7f83380b4670 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8338162ff0 con 0x7f8338162c10 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f833e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f833806ddd0 con 0x7f83380cfba0 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46584/0 (socket says 172.21.15.59:46584) 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 -- 172.21.15.59:0/4142561359 learned_addr learned my addr 172.21.15.59:0/4142561359 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337e00640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338162c10 0x7f833815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 -- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83380cfba0 msgr2=0x7f83380b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83380cfba0 0x7f83380b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 -- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338162c10 msgr2=0x7f833815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338162c10 0x7f833815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.402+0000 7f8337400640 1 -- 172.21.15.59:0/4142561359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338164870 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f8337400640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f832800b3e0 tx=0x7f832800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328015020 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8328011850 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8338164b30 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328011c40 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.403+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381650f0 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.405+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338068a80 con 0x7f83380b4670 2024-04-24T06:33:55.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.406+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8328018020 con 0x7f83380b4670 2024-04-24T06:33:55.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.407+0000 7f832f400640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 0x7f831806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.408+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f832808ba70 con 0x7f83380b4670 2024-04-24T06:33:55.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.408+0000 7f8336a00640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 0x7f831806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.408+0000 7f8336a00640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 0x7f831806edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83381ef420 tx=0x7f8324006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.412+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f832801f020 con 0x7f83380b4670 2024-04-24T06:33:55.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.614+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f833815ea70 con 0x7f83380b4670 2024-04-24T06:33:55.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.614+0000 7f832f400640 1 -- 172.21.15.59:0/4142561359 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f83280172c0 con 0x7f83380b4670 2024-04-24T06:33:55.615 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 msgr2=0x7f831806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 0x7f831806edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83381ef420 tx=0x7f8324006c90 comp rx=0 tx=0).stop 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 msgr2=0x7f83380cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f832800b3e0 tx=0x7f832800b8a0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 shutdown_connections 2024-04-24T06:33:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f831806c8f0 0x7f831806edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338162c10 0x7f833815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83380cfba0 0x7f83380b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 --2- 172.21.15.59:0/4142561359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83380b4670 0x7f83380cc840 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 >> 172.21.15.59:0/4142561359 conn(0x7f8338067d90 msgr2=0x7f833806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 shutdown_connections 2024-04-24T06:33:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.617+0000 7f833e800640 1 -- 172.21.15.59:0/4142561359 wait complete. 2024-04-24T06:33:55.637 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:33:55.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.855+0000 7fa887200640 1 Processor -- start 2024-04-24T06:33:55.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.855+0000 7fa887200640 1 -- start start 2024-04-24T06:33:55.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.856+0000 7fa887200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa8880d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.856+0000 7fa887200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa8880d09e0 0x7fa8880b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.856+0000 7fa887200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa88815cae0 con 0x7fa88815c700 2024-04-24T06:33:55.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.856+0000 7fa887200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa8880b2df0 con 0x7fa8880d09e0 2024-04-24T06:33:55.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.856+0000 7fa887200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa8880d0e70 con 0x7fa8880ce180 2024-04-24T06:33:55.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa88815c700 0x7fa8880cda70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43992/0 (socket says 172.21.15.59:43992) 2024-04-24T06:33:55.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885e00640 1 -- 172.21.15.59:0/3436593852 learned_addr learned my addr 172.21.15.59:0/3436593852 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa87d400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa8880d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa886800640 1 --2- 172.21.15.59:0/3436593852 >> v2:172.21.15.66:3300/0 conn(0x7fa8880d09e0 0x7fa8880b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484279250 0 0) 0x7fa88815cae0 con 0x7fa88815c700 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa86c003520 con 0x7fa88815c700 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3277966414 0 0) 0x7fa86c003520 con 0x7fa88815c700 2024-04-24T06:33:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 msgr2=0x7fa8880d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa8880d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 >> v2:172.21.15.66:3300/0 conn(0x7fa8880d09e0 msgr2=0x7fa8880b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 --2- 172.21.15.59:0/3436593852 >> v2:172.21.15.66:3300/0 conn(0x7fa8880d09e0 0x7fa8880b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.857+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88809aba0 con 0x7fa88815c700 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa878001670 con 0x7fa88815c700 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 >> v1:172.21.15.59:6789/0 conn(0x7fa88815c700 legacy=0x7fa8880cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86c0039e0 0x7fa86c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa86c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c004d40 0x7fa86c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880d0e70 con 0x7fa86c0039e0 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c00b9c0 con 0x7fa86c004d40 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa86c00bb40 con 0x7fa8880ce180 2024-04-24T06:33:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885e00640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86c0039e0 0x7fa86c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa886800640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c004d40 0x7fa86c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa87d400640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa86c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.858+0000 7fa885e00640 1 -- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 msgr2=0x7fa86c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.859+0000 7fa885e00640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880ce180 0x7fa86c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.859+0000 7fa885e00640 1 -- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c004d40 msgr2=0x7fa86c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.859+0000 7fa885e00640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c004d40 0x7fa86c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.859+0000 7fa885e00640 1 -- 172.21.15.59:0/3436593852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c003520 con 0x7fa86c0039e0 2024-04-24T06:33:55.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa885e00640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86c0039e0 0x7fa86c003e00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c190 tx=0x7fa86c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de5990fe1b33802 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878007c60 con 0x7fa86c0039e0 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa878005af0 con 0x7fa86c0039e0 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa885400640 1 -- 172.21.15.59:0/3436593852 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878007070 con 0x7fa86c0039e0 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa887200640 1 -- 172.21.15.59:0/3436593852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86c0039e0 msgr2=0x7fa86c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa887200640 1 --2- 172.21.15.59:0/3436593852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86c0039e0 0x7fa86c003e00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c190 tx=0x7fa86c005830 comp rx=0 tx=0).stop 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.860+0000 7fa886800640 1 -- 172.21.15.59:0/3436593852 reap_dead start 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.861+0000 7fa887200640 1 -- 172.21.15.59:0/3436593852 shutdown_connections 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.861+0000 7fa887200640 1 -- 172.21.15.59:0/3436593852 >> 172.21.15.59:0/3436593852 conn(0x7fa8880b3f70 msgr2=0x7fa8880b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.861+0000 7fa887200640 1 -- 172.21.15.59:0/3436593852 shutdown_connections 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.861+0000 7fa887200640 1 -- 172.21.15.59:0/3436593852 wait complete. 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.862+0000 7fa887200640 1 Processor -- start 2024-04-24T06:33:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.863+0000 7fa887200640 1 -- start start 2024-04-24T06:33:55.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 0x7fa888150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880cf3d0 0x7fa8880b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa88809b440 con 0x7fa8880cef70 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa88809b160 con 0x7fa86c0039e0 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa887200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa88809b720 con 0x7fa8880cf3d0 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46604/0 (socket says 172.21.15.59:46604) 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 -- 172.21.15.59:0/3183154440 learned_addr learned my addr 172.21.15.59:0/3183154440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa885e00640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 0x7fa888150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 -- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880cf3d0 msgr2=0x7fa8880b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880cf3d0 0x7fa8880b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 -- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 msgr2=0x7fa888150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 0x7fa888150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa87d400640 1 -- 172.21.15.59:0/3183154440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8880cf9a0 con 0x7fa8880cef70 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.864+0000 7fa885e00640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 0x7fa888150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa87d400640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa870011520 tx=0x7fa8700119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870015930 con 0x7fa8880cef70 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870015ad0 con 0x7fa8880cef70 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8880cfc80 con 0x7fa8880cef70 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87001d400 con 0x7fa8880cef70 2024-04-24T06:33:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.865+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8880c6950 con 0x7fa8880cef70 2024-04-24T06:33:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.867+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8880c5b30 con 0x7fa8880cef70 2024-04-24T06:33:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.868+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa87001d5a0 con 0x7fa8880cef70 2024-04-24T06:33:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.869+0000 7fa87ea00640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 0x7fa86406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.869+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fa87008c460 con 0x7fa8880cef70 2024-04-24T06:33:55.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.869+0000 7fa885e00640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 0x7fa86406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:55.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.870+0000 7fa885e00640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 0x7fa86406ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c160 tx=0x7fa878009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:55.873+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8880c5b30 con 0x7fa8880cef70 2024-04-24T06:33:56.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.078+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa88804fd90 con 0x7fa8880cef70 2024-04-24T06:33:56.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.078+0000 7fa87ea00640 1 -- 172.21.15.59:0/3183154440 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7fa870019030 con 0x7fa8880cef70 2024-04-24T06:33:56.079 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:33:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 msgr2=0x7fa86406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 0x7fa86406ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c160 tx=0x7fa878009030 comp rx=0 tx=0).stop 2024-04-24T06:33:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 msgr2=0x7fa8880b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa870011520 tx=0x7fa8700119e0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 shutdown_connections 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa86406c920 0x7fa86406ede0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa8880cf3d0 0x7fa8880b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8880cef70 0x7fa8880b5070 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 --2- 172.21.15.59:0/3183154440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa86c0039e0 0x7fa888150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 >> 172.21.15.59:0/3183154440 conn(0x7fa8880b3f70 msgr2=0x7fa888071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.081+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 shutdown_connections 2024-04-24T06:33:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.082+0000 7fa887200640 1 -- 172.21.15.59:0/3183154440 wait complete. 2024-04-24T06:33:56.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:33:56.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.314+0000 7f003e000640 1 Processor -- start 2024-04-24T06:33:56.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.314+0000 7f003e000640 1 -- start start 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.316+0000 7f003e000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0038075370 0x7f0038072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.316+0000 7f003e000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003815f240 con 0x7f0038072590 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.316+0000 7f003e000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003806ea30 con 0x7f003806fad0 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.316+0000 7f003e000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f003806f640 con 0x7f0038075370 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.317+0000 7f003d600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0038072590 0x7f003815eb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44000/0 (socket says 172.21.15.59:44000) 2024-04-24T06:33:56.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.317+0000 7f003d600640 1 -- 172.21.15.59:0/2997491531 learned_addr learned my addr 172.21.15.59:0/2997491531 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.317+0000 7f003cc00640 1 --2- 172.21.15.59:0/2997491531 >> v2:172.21.15.66:3300/0 conn(0x7f0038075370 0x7f0038072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.317+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505297291 0 0) 0x7f003815f240 con 0x7f0038072590 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.317+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0020003520 con 0x7f0038072590 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1834893038 0 0) 0x7f0020003520 con 0x7f0038072590 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 >> v2:172.21.15.66:3300/0 conn(0x7f0038075370 msgr2=0x7f0038072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 --2- 172.21.15.59:0/2997491531 >> v2:172.21.15.66:3300/0 conn(0x7f0038075370 0x7f0038072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 >> v1:172.21.15.66:6789/0 conn(0x7f003806fad0 legacy=0x7f003815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003806ed10 con 0x7f0038072590 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f00300019f0 con 0x7f0038072590 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 >> v1:172.21.15.59:6789/0 conn(0x7f0038072590 legacy=0x7f003815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0020003a00 0x7f0020003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f00200051c0 0x7f0020005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f003806f640 con 0x7f0020004800 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00200060e0 con 0x7f0020003a00 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f00200062e0 con 0x7f00200051c0 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f0037e00640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.318+0000 7f003cc00640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0020003a00 0x7f0020003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f003d600640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f00200051c0 0x7f0020005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f0037e00640 1 -- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f00200051c0 msgr2=0x7f0020005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f0037e00640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f00200051c0 0x7f0020005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f0037e00640 1 -- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0020003a00 msgr2=0x7f0020003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f0037e00640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0020003a00 0x7f0020003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f0037e00640 1 -- 172.21.15.59:0/2997491531 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020003520 con 0x7f0020004800 2024-04-24T06:33:56.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.319+0000 7f003d600640 1 -- 172.21.15.59:0/2997491531 reap_dead start 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.320+0000 7f0037e00640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f0020004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f002c001750 tx=0x7f0020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db7cf46c633b012 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.320+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c001370 con 0x7f0020004800 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.320+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0052d0 con 0x7f0020004800 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.320+0000 7f0037400640 1 -- 172.21.15.59:0/2997491531 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c02abc0 con 0x7f0020004800 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 -- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 msgr2=0x7f0020004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f0020004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f002c001750 tx=0x7f0020005830 comp rx=0 tx=0).stop 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 -- 172.21.15.59:0/2997491531 shutdown_connections 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 --2- 172.21.15.59:0/2997491531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 -- 172.21.15.59:0/2997491531 >> 172.21.15.59:0/2997491531 conn(0x7f00380b2310 msgr2=0x7f00380b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 -- 172.21.15.59:0/2997491531 shutdown_connections 2024-04-24T06:33:56.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.321+0000 7f003e000640 1 -- 172.21.15.59:0/2997491531 wait complete. 2024-04-24T06:33:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.323+0000 7f003e000640 1 Processor -- start 2024-04-24T06:33:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.323+0000 7f003e000640 1 -- start start 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0038139420 0x7f0038139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f003815d120 0x7f003815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00380b5460 con 0x7f0020004800 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f003806f2d0 con 0x7f003815d120 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f00380cff30 con 0x7f0038139420 2024-04-24T06:33:56.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46632/0 (socket says 172.21.15.59:46632) 2024-04-24T06:33:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.324+0000 7f003cc00640 1 -- 172.21.15.59:0/3750536996 learned_addr learned my addr 172.21.15.59:0/3750536996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003d600640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f003815d120 0x7f003815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 -- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0038139420 msgr2=0x7f0038139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0038139420 0x7f0038139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 -- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f003815d120 msgr2=0x7f003815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f003815d120 0x7f003815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 -- 172.21.15.59:0/3750536996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0038138d60 con 0x7f0020004800 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f003cc00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f00240116d0 tx=0x7f0024011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002400f9c0 con 0x7f0020004800 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.325+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002400fb60 con 0x7f0020004800 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.326+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024015460 con 0x7f0020004800 2024-04-24T06:33:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.326+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003815db10 con 0x7f0020004800 2024-04-24T06:33:56.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.326+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003815ac10 con 0x7f0020004800 2024-04-24T06:33:56.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.327+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003806c960 con 0x7f0020004800 2024-04-24T06:33:56.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.328+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0024015600 con 0x7f0020004800 2024-04-24T06:33:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.329+0000 7f0034c00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.329+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f002408c1c0 con 0x7f0020004800 2024-04-24T06:33:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.329+0000 7f0037e00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.330+0000 7f0037e00640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 0x7f001806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f002c032000 tx=0x7f002c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.335+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0024019030 con 0x7f0020004800 2024-04-24T06:33:56.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.539+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f003815e1e0 con 0x7f0020004800 2024-04-24T06:33:56.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.540+0000 7f0034c00640 1 -- 172.21.15.59:0/3750536996 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7f002405a5f0 con 0x7f0020004800 2024-04-24T06:33:56.541 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.542+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 msgr2=0x7f001806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.542+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 0x7f001806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f002c032000 tx=0x7f002c02c000 comp rx=0 tx=0).stop 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.542+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 msgr2=0x7f00380ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.542+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f00240116d0 tx=0x7f0024011b90 comp rx=0 tx=0).stop 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 shutdown_connections 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f001806c8f0 0x7f001806edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f003815d120 0x7f003815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0038139420 0x7f0038139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 --2- 172.21.15.59:0/3750536996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0020004800 0x7f00380ce8a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 >> 172.21.15.59:0/3750536996 conn(0x7f00380b2310 msgr2=0x7f003815e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:56.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 shutdown_connections 2024-04-24T06:33:56.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.543+0000 7f003e000640 1 -- 172.21.15.59:0/3750536996 wait complete. 2024-04-24T06:33:56.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:33:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.776+0000 7f29d7e00640 1 Processor -- start 2024-04-24T06:33:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.776+0000 7f29d7e00640 1 -- start start 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 0x7f29d80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80cf930 con 0x7f29d80d0f80 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80cfc10 con 0x7f29d815f880 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80cfef0 con 0x7f29d815ea40 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f29d80d0f80 0x7f29d80d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44006/0 (socket says 172.21.15.59:44006) 2024-04-24T06:33:56.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.778+0000 7f29d6a00640 1 -- 172.21.15.59:0/3595697464 learned_addr learned my addr 172.21.15.59:0/3595697464 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29cfe00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 0x7f29d80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 105860160 0 0) 0x7f29d80cf930 con 0x7f29d80d0f80 2024-04-24T06:33:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29bc003520 con 0x7f29d80d0f80 2024-04-24T06:33:56.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254531939 0 0) 0x7f29bc003520 con 0x7f29d80d0f80 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 msgr2=0x7f29d80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 0x7f29d80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 >> v1:172.21.15.66:6789/0 conn(0x7f29d815f880 legacy=0x7f29d80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d80d01d0 con 0x7f29d80d0f80 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f29c8001670 con 0x7f29d80d0f80 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 >> v1:172.21.15.59:6789/0 conn(0x7f29d80d0f80 legacy=0x7f29d80d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29bc0039e0 0x7f29bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 0x7f29bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29bc004d40 0x7f29bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80cfef0 con 0x7f29bc0039e0 2024-04-24T06:33:56.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc00baf0 con 0x7f29bc004d40 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.779+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc00bcf0 con 0x7f29d815ea40 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29bc0039e0 0x7f29bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 -- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 msgr2=0x7f29bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29d815ea40 0x7f29bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 -- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29bc004d40 msgr2=0x7f29bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29bc004d40 0x7f29bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.780+0000 7f29d6a00640 1 -- 172.21.15.59:0/3595697464 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc003520 con 0x7f29bc0039e0 2024-04-24T06:33:56.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.781+0000 7f29d6a00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29bc0039e0 0x7f29bc003e00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c210 tx=0x7f29bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7d17343d0b123a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.781+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007d30 con 0x7f29bc0039e0 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.781+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c8005af0 con 0x7f29bc0039e0 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.781+0000 7f29d6000640 1 -- 172.21.15.59:0/3595697464 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8007070 con 0x7f29bc0039e0 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 -- 172.21.15.59:0/3595697464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29bc0039e0 msgr2=0x7f29bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 --2- 172.21.15.59:0/3595697464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29bc0039e0 0x7f29bc003e00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c210 tx=0x7f29bc005830 comp rx=0 tx=0).stop 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7400640 1 -- 172.21.15.59:0/3595697464 reap_dead start 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 -- 172.21.15.59:0/3595697464 shutdown_connections 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 -- 172.21.15.59:0/3595697464 >> 172.21.15.59:0/3595697464 conn(0x7f29d808bf90 msgr2=0x7f29d80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 -- 172.21.15.59:0/3595697464 shutdown_connections 2024-04-24T06:33:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.782+0000 7f29d7e00640 1 -- 172.21.15.59:0/3595697464 wait complete. 2024-04-24T06:33:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.784+0000 7f29d7e00640 1 Processor -- start 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.784+0000 7f29d7e00640 1 -- start start 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29bc0039e0 0x7f29d80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815a6a0 0x7f29d815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80cf250 con 0x7f29d80ccb10 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29d815fd00 con 0x7f29d815a6a0 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29d7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29d80cf3d0 con 0x7f29bc0039e0 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.785+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46648/0 (socket says 172.21.15.59:46648) 2024-04-24T06:33:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 -- 172.21.15.59:0/1528790895 learned_addr learned my addr 172.21.15.59:0/1528790895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 -- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29bc0039e0 msgr2=0x7f29d80cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29bc0039e0 0x7f29d80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 -- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815a6a0 msgr2=0x7f29d815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815a6a0 0x7f29d815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 -- 172.21.15.59:0/1528790895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d81f6000 con 0x7f29d80ccb10 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cfe00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e7c0 tx=0x7f29c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.786+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0017070 con 0x7f29d80ccb10 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.787+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c000ee70 con 0x7f29d80ccb10 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.787+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000cb40 con 0x7f29d80ccb10 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.787+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d8158fe0 con 0x7f29d80ccb10 2024-04-24T06:33:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.787+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d8158740 con 0x7f29d80ccb10 2024-04-24T06:33:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.788+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d804fd90 con 0x7f29d80ccb10 2024-04-24T06:33:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.789+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29c001a070 con 0x7f29d80ccb10 2024-04-24T06:33:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.790+0000 7f29cea00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 0x7f29b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.790+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f29c008b920 con 0x7f29d80ccb10 2024-04-24T06:33:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.790+0000 7f29d6a00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 0x7f29b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.791+0000 7f29d6a00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 0x7f29b406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29c8032000 tx=0x7f29c8007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:56.794+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29d81f6000 con 0x7f29d80ccb10 2024-04-24T06:33:57.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.013+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29d80d1ec0 con 0x7f29d80ccb10 2024-04-24T06:33:57.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.014+0000 7f29cea00640 1 -- 172.21.15.59:0/1528790895 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f29c0059d40 con 0x7f29d80ccb10 2024-04-24T06:33:57.015 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 msgr2=0x7f29b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 0x7f29b406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29c8032000 tx=0x7f29c8007000 comp rx=0 tx=0).stop 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 msgr2=0x7f29d81f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f29c000e7c0 tx=0x7f29c000ec80 comp rx=0 tx=0).stop 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 shutdown_connections 2024-04-24T06:33:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29b406c8f0 0x7f29b406edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29d815a6a0 0x7f29d815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29d80ccb10 0x7f29d81f5ac0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 --2- 172.21.15.59:0/1528790895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29bc0039e0 0x7f29d80cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 >> 172.21.15.59:0/1528790895 conn(0x7f29d808bf90 msgr2=0x7f29d80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 shutdown_connections 2024-04-24T06:33:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.018+0000 7f29d7e00640 1 -- 172.21.15.59:0/1528790895 wait complete. 2024-04-24T06:33:57.038 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:33:57.039 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:33:57.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:33:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.341+0000 7fa822600640 1 Processor -- start 2024-04-24T06:33:57.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.342+0000 7fa822600640 1 -- start start 2024-04-24T06:33:57.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.343+0000 7fa822600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0cd020 0x7fa81c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.343+0000 7fa822600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa81c0b3e40 0x7fa81c160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.343+0000 7fa822600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81c0cd520 con 0x7fa81c1454e0 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.343+0000 7fa822600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa81c0cddc0 con 0x7fa81c0b3e40 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.343+0000 7fa822600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c1462c0 con 0x7fa81c0cd020 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa81c1454e0 0x7fa81c145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44014/0 (socket says 172.21.15.59:44014) 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa81c0b3e40 0x7fa81c160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81b400640 1 -- 172.21.15.59:0/413457945 learned_addr learned my addr 172.21.15.59:0/413457945 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa813e00640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0cd020 0x7fa81c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81be00640 1 -- 172.21.15.59:0/413457945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0cd020 msgr2=0x7fa81c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81be00640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0cd020 0x7fa81c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81be00640 1 -- 172.21.15.59:0/413457945 >> v1:172.21.15.59:6789/0 conn(0x7fa81c1454e0 legacy=0x7fa81c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81be00640 1 -- 172.21.15.59:0/413457945 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c0cd800 con 0x7fa81c0b3e40 2024-04-24T06:33:57.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.344+0000 7fa81aa00640 1 -- 172.21.15.59:0/413457945 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718185804 0 0) 0x7fa81c0cd520 con 0x7fa81c1454e0 2024-04-24T06:33:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.345+0000 7fa81be00640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c0b3e40 0x7fa81c160910 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa814009d90 tx=0x7fa81402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc9c749fd58256 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.346+0000 7fa81aa00640 1 -- 172.21.15.59:0/413457945 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814031600 con 0x7fa81c0b3e40 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.346+0000 7fa81aa00640 1 -- 172.21.15.59:0/413457945 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8140317a0 con 0x7fa81c0b3e40 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.346+0000 7fa81aa00640 1 -- 172.21.15.59:0/413457945 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814031b70 con 0x7fa81c0b3e40 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.346+0000 7fa822600640 1 -- 172.21.15.59:0/413457945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c0b3e40 msgr2=0x7fa81c160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.346+0000 7fa822600640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c0b3e40 0x7fa81c160910 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa814009d90 tx=0x7fa81402a5d0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 -- 172.21.15.59:0/413457945 shutdown_connections 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c0b3e40 0x7fa81c160910 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 --2- 172.21.15.59:0/413457945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0cd020 0x7fa81c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 -- 172.21.15.59:0/413457945 >> 172.21.15.59:0/413457945 conn(0x7fa81c067ea0 msgr2=0x7fa81c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 -- 172.21.15.59:0/413457945 shutdown_connections 2024-04-24T06:33:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.347+0000 7fa822600640 1 -- 172.21.15.59:0/413457945 wait complete. 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.348+0000 7fa822600640 1 Processor -- start 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.349+0000 7fa822600640 1 -- start start 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.349+0000 7fa822600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0b3e40 0x7fa81c1ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.349+0000 7fa822600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa81c0cd020 0x7fa81c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.349+0000 7fa822600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.349+0000 7fa822600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c160e50 con 0x7fa81c1454e0 2024-04-24T06:33:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa822600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1603f0 con 0x7fa81c0cd020 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa822600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c160fd0 con 0x7fa81c0b3e40 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46662/0 (socket says 172.21.15.59:46662) 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 -- 172.21.15.59:0/2078323341 learned_addr learned my addr 172.21.15.59:0/2078323341 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81b400640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0b3e40 0x7fa81c1ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa813e00640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa81c0cd020 0x7fa81c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 -- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0b3e40 msgr2=0x7fa81c1ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0b3e40 0x7fa81c1ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 -- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa81c0cd020 msgr2=0x7fa81c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa81c0cd020 0x7fa81c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.350+0000 7fa81be00640 1 -- 172.21.15.59:0/2078323341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c15df30 con 0x7fa81c1454e0 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa81be00640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa814009eb0 tx=0x7fa814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814040070 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa814030b60 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814009450 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c15e210 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.351+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c15e7d0 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.352+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c068a80 con 0x7fa81c1454e0 2024-04-24T06:33:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.353+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa814039040 con 0x7fa81c1454e0 2024-04-24T06:33:57.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.355+0000 7fa813400640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.355+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fa8140ad340 con 0x7fa81c1454e0 2024-04-24T06:33:57.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.355+0000 7fa81b400640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.356+0000 7fa81b400640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 0x7fa7f806edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000f60 tx=0x7fa80c007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.360+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa81404d350 con 0x7fa81c1454e0 2024-04-24T06:33:57.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.564+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa81c160030 con 0x7fa81c1454e0 2024-04-24T06:33:57.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.567+0000 7fa813400640 1 -- 172.21.15.59:0/2078323341 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16584 (secure 0 0 0) 0x7fa81407b7f0 con 0x7fa81c1454e0 2024-04-24T06:33:57.569 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:33:57.569 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:33:46.242082+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:33:57.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 msgr2=0x7fa7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 0x7fa7f806edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000f60 tx=0x7fa80c007c00 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 msgr2=0x7fa81c1404b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa814009eb0 tx=0x7fa814004a40 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 shutdown_connections 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa7f806c8f0 0x7fa7f806edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa81c1454e0 0x7fa81c1404b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa81c0cd020 0x7fa81c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 --2- 172.21.15.59:0/2078323341 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa81c0b3e40 0x7fa81c1ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 >> 172.21.15.59:0/2078323341 conn(0x7fa81c067ea0 msgr2=0x7fa81c0af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.570+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 shutdown_connections 2024-04-24T06:33:57.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.571+0000 7fa822600640 1 -- 172.21.15.59:0/2078323341 wait complete. 2024-04-24T06:33:57.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:33:57.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:33:57.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.805+0000 7f258b600640 1 Processor -- start 2024-04-24T06:33:57.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.805+0000 7f258b600640 1 -- start start 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.807+0000 7f258b600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2584072480 0x7f25840aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.807+0000 7f258b600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25840adc80 con 0x7f258406f6f0 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.807+0000 7f258b600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25840adf20 con 0x7f258415e790 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.807+0000 7f258b600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f25840aef40 con 0x7f2584072480 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.808+0000 7f2583e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f258406f6f0 0x7f2584071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44030/0 (socket says 172.21.15.59:44030) 2024-04-24T06:33:57.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.808+0000 7f2583e00640 1 -- 172.21.15.59:0/2201574691 learned_addr learned my addr 172.21.15.59:0/2201574691 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.808+0000 7f2589600640 1 --2- 172.21.15.59:0/2201574691 >> v2:172.21.15.66:3300/0 conn(0x7f2584072480 0x7f25840aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.808+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58900737 0 0) 0x7f25840adc80 con 0x7f258406f6f0 2024-04-24T06:33:57.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.808+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f256c003520 con 0x7f258406f6f0 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1935409134 0 0) 0x7f256c003520 con 0x7f258406f6f0 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 >> v2:172.21.15.66:3300/0 conn(0x7f2584072480 msgr2=0x7f25840aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 --2- 172.21.15.59:0/2201574691 >> v2:172.21.15.66:3300/0 conn(0x7f2584072480 0x7f25840aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 >> v1:172.21.15.66:6789/0 conn(0x7f258415e790 legacy=0x7f2584162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25840ae200 con 0x7f258406f6f0 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2570001670 con 0x7f258406f6f0 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 >> v1:172.21.15.59:6789/0 conn(0x7f258406f6f0 legacy=0x7f2584071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c003a00 0x7f256c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f256c004800 0x7f256c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f256c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25840aef40 con 0x7f256c0051c0 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f256c00c0c0 con 0x7f256c003a00 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f256c00c2c0 con 0x7f256c004800 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2589600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f256c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.809+0000 7f2588c00640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c003a00 0x7f256c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2583e00640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f256c004800 0x7f256c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2589600640 1 -- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f256c004800 msgr2=0x7f256c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2589600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f256c004800 0x7f256c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2589600640 1 -- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c003a00 msgr2=0x7f256c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2589600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f256c003a00 0x7f256c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.810+0000 7f2589600640 1 -- 172.21.15.59:0/2201574691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c003520 con 0x7f256c0051c0 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.811+0000 7f2589600640 1 -- 172.21.15.59:0/2201574691 reap_dead start 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.811+0000 7f2589600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f256c00b9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f257c009fc0 tx=0x7f256c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac5f259c83cdb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c035020 con 0x7f256c0051c0 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c003bd0 con 0x7f256c0051c0 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f2583400640 1 -- 172.21.15.59:0/2201574691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c009c10 con 0x7f256c0051c0 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f258b600640 1 -- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 msgr2=0x7f256c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f258b600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f256c00b9c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f257c009fc0 tx=0x7f256c005830 comp rx=0 tx=0).stop 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f258b600640 1 -- 172.21.15.59:0/2201574691 shutdown_connections 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f258b600640 1 --2- 172.21.15.59:0/2201574691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f256c00b9c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.812+0000 7f258b600640 1 -- 172.21.15.59:0/2201574691 >> 172.21.15.59:0/2201574691 conn(0x7f2584067d90 msgr2=0x7f258406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.813+0000 7f258b600640 1 -- 172.21.15.59:0/2201574691 shutdown_connections 2024-04-24T06:33:57.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.813+0000 7f258b600640 1 -- 172.21.15.59:0/2201574691 wait complete. 2024-04-24T06:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.814+0000 7f258b600640 1 Processor -- start 2024-04-24T06:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.814+0000 7f258b600640 1 -- start start 2024-04-24T06:33:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2584161040 0x7f25841614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 0x7f25841f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f258406fbb0 con 0x7f256c0051c0 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f258415eb70 con 0x7f25841f4930 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.815+0000 7f258b600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f258415ecf0 con 0x7f2584161040 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46682/0 (socket says 172.21.15.59:46682) 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 -- 172.21.15.59:0/2996611324 learned_addr learned my addr 172.21.15.59:0/2996611324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2589600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 0x7f25841f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2583e00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2584161040 0x7f25841614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 -- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2584161040 msgr2=0x7f25841614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2584161040 0x7f25841614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 -- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 msgr2=0x7f25841f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 0x7f25841f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 -- 172.21.15.59:0/2996611324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25840cefa0 con 0x7f256c0051c0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2589600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 0x7f25841f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.816+0000 7f2588c00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f257800f5f0 tx=0x7f257800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.817+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25780052e0 con 0x7f256c0051c0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.817+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2578018070 con 0x7f256c0051c0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.817+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578003810 con 0x7f256c0051c0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.817+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25840ce1e0 con 0x7f256c0051c0 2024-04-24T06:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.817+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25840ce7a0 con 0x7f256c0051c0 2024-04-24T06:33:57.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.818+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2584161a00 con 0x7f256c0051c0 2024-04-24T06:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.819+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25780039b0 con 0x7f256c0051c0 2024-04-24T06:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.820+0000 7f2580c00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 0x7f256406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.820+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f257808b910 con 0x7f256c0051c0 2024-04-24T06:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.820+0000 7f2583e00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 0x7f256406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.821+0000 7f2583e00640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 0x7f256406ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f258406eee0 tx=0x7f2570000f10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:57.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:57.824+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2578059d30 con 0x7f256c0051c0 2024-04-24T06:33:58.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.027+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25840b09b0 con 0x7f256c0051c0 2024-04-24T06:33:58.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.029+0000 7f2580c00640 1 -- 172.21.15.59:0/2996611324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f25840b09b0 con 0x7f256c0051c0 2024-04-24T06:33:58.030 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:33:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 msgr2=0x7f256406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 0x7f256406ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f258406eee0 tx=0x7f2570000f10 comp rx=0 tx=0).stop 2024-04-24T06:33:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 msgr2=0x7f2584149e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f257800f5f0 tx=0x7f257800fab0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 shutdown_connections 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f256406c920 0x7f256406ede0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25841f4930 0x7f25841f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2584161040 0x7f25841614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 --2- 172.21.15.59:0/2996611324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0051c0 0x7f2584149e30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.031+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 >> 172.21.15.59:0/2996611324 conn(0x7f2584067d90 msgr2=0x7f25840b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.032+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 shutdown_connections 2024-04-24T06:33:58.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.032+0000 7f258b600640 1 -- 172.21.15.59:0/2996611324 wait complete. 2024-04-24T06:33:58.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:33:58.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.276+0000 7f55bfa00640 1 Processor -- start 2024-04-24T06:33:58.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.276+0000 7f55bfa00640 1 -- start start 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.277+0000 7f55bfa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f55b80d0cc0 0x7f55b806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.277+0000 7f55bfa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55b8071d70 0x7f55b8072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bfa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b80cfc20 con 0x7f55b8145e10 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bfa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f55b80cff00 con 0x7f55b8071d70 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bfa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f55b80d01e0 con 0x7f55b80d0cc0 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bd000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f55b8145e10 0x7f55b80cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44032/0 (socket says 172.21.15.59:44032) 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bd000640 1 -- 172.21.15.59:0/3580806315 learned_addr learned my addr 172.21.15.59:0/3580806315 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:58.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55bda00640 1 --2- 172.21.15.59:0/3580806315 >> v2:172.21.15.59:3300/0 conn(0x7f55b8071d70 0x7f55b8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55afe00640 1 --2- 172.21.15.59:0/3580806315 >> v2:172.21.15.66:3300/0 conn(0x7f55b80d0cc0 0x7f55b806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.278+0000 7f55b7e00640 1 -- 172.21.15.59:0/3580806315 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223412966 0 0) 0x7f55b80cfc20 con 0x7f55b8145e10 2024-04-24T06:33:58.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.279+0000 7f55b7e00640 1 -- 172.21.15.59:0/3580806315 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f559c003520 con 0x7f55b8145e10 2024-04-24T06:33:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.279+0000 7f55bda00640 1 -- 172.21.15.59:0/3580806315 >> v2:172.21.15.66:3300/0 conn(0x7f55b80d0cc0 msgr2=0x7f55b806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.279+0000 7f55bda00640 1 --2- 172.21.15.59:0/3580806315 >> v2:172.21.15.66:3300/0 conn(0x7f55b80d0cc0 0x7f55b806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.279+0000 7f55bda00640 1 -- 172.21.15.59:0/3580806315 >> v1:172.21.15.59:6789/0 conn(0x7f55b8145e10 legacy=0x7f55b80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.279+0000 7f55bda00640 1 -- 172.21.15.59:0/3580806315 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b80d04c0 con 0x7f55b8071d70 2024-04-24T06:33:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.280+0000 7f55bda00640 1 --2- 172.21.15.59:0/3580806315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b8071d70 0x7f55b8072200 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009d90 tx=0x7f55b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4e3ae542c2f113 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.280+0000 7f55b7e00640 1 -- 172.21.15.59:0/3580806315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0007740 con 0x7f55b8071d70 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.280+0000 7f55b7e00640 1 -- 172.21.15.59:0/3580806315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b00078e0 con 0x7f55b8071d70 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.280+0000 7f55b7e00640 1 -- 172.21.15.59:0/3580806315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0007cb0 con 0x7f55b8071d70 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 -- 172.21.15.59:0/3580806315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b8071d70 msgr2=0x7f55b8072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 --2- 172.21.15.59:0/3580806315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b8071d70 0x7f55b8072200 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009d90 tx=0x7f55b002a6a0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 -- 172.21.15.59:0/3580806315 shutdown_connections 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 --2- 172.21.15.59:0/3580806315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b8071d70 0x7f55b8072200 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 --2- 172.21.15.59:0/3580806315 >> v2:172.21.15.66:3300/0 conn(0x7f55b80d0cc0 0x7f55b806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 -- 172.21.15.59:0/3580806315 >> 172.21.15.59:0/3580806315 conn(0x7f55b8067d90 msgr2=0x7f55b806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 -- 172.21.15.59:0/3580806315 shutdown_connections 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.281+0000 7f55bfa00640 1 -- 172.21.15.59:0/3580806315 wait complete. 2024-04-24T06:33:58.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.283+0000 7f55bfa00640 1 Processor -- start 2024-04-24T06:33:58.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.283+0000 7f55bfa00640 1 -- start start 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 0x7f55b81e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55b8145e10 0x7f55b815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55b81461f0 con 0x7f55b80d0cc0 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55b815cdd0 con 0x7f55b8071d70 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55bfa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f55b81eb100 con 0x7f55b8145e10 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46710/0 (socket says 172.21.15.59:46710) 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.284+0000 7f55afe00640 1 -- 172.21.15.59:0/1626488990 learned_addr learned my addr 172.21.15.59:0/1626488990 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55bd000640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 0x7f55b81e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 -- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55b8145e10 msgr2=0x7f55b815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55bda00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55b8145e10 0x7f55b815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55b8145e10 0x7f55b815e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 -- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 msgr2=0x7f55b81e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 0x7f55b81e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 -- 172.21.15.59:0/1626488990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81e6550 con 0x7f55b80d0cc0 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55bd000640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 0x7f55b81e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:58.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55afe00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f55a000d800 tx=0x7f55a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.285+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a000b6c0 con 0x7f55b80d0cc0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.286+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a0019070 con 0x7f55b80d0cc0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.286+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81e6b10 con 0x7f55b80d0cc0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.286+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a000ba80 con 0x7f55b80d0cc0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.286+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81e70d0 con 0x7f55b80d0cc0 2024-04-24T06:33:58.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.287+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b81e62e0 con 0x7f55b80d0cc0 2024-04-24T06:33:58.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.288+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55a000bc20 con 0x7f55b80d0cc0 2024-04-24T06:33:58.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.289+0000 7f55b5600640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 0x7f559406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.289+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f55a008c040 con 0x7f55b80d0cc0 2024-04-24T06:33:58.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.289+0000 7f55bd000640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 0x7f559406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.290+0000 7f55bd000640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 0x7f559406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005880 tx=0x7f55a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.294+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55a001c360 con 0x7f55b80d0cc0 2024-04-24T06:33:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.497+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55b81e6830 con 0x7f55b80d0cc0 2024-04-24T06:33:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.499+0000 7f55b5600640 1 -- 172.21.15.59:0/1626488990 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7f55a005a470 con 0x7f55b80d0cc0 2024-04-24T06:33:58.500 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:33:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 msgr2=0x7f559406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 0x7f559406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005880 tx=0x7f55a8009040 comp rx=0 tx=0).stop 2024-04-24T06:33:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 msgr2=0x7f55b81eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f55a000d800 tx=0x7f55a000dcc0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 shutdown_connections 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f559406c8f0 0x7f559406edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f55b8145e10 0x7f55b815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55b80d0cc0 0x7f55b81eabc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 --2- 172.21.15.59:0/1626488990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b8071d70 0x7f55b81e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 >> 172.21.15.59:0/1626488990 conn(0x7f55b8067d90 msgr2=0x7f55b815d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.501+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 shutdown_connections 2024-04-24T06:33:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.502+0000 7f55bfa00640 1 -- 172.21.15.59:0/1626488990 wait complete. 2024-04-24T06:33:58.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:33:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.737+0000 7fdf11000640 1 Processor -- start 2024-04-24T06:33:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.737+0000 7fdf11000640 1 -- start start 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf11000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf0c1567e0 0x7fdf0c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf11000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 0x7fdf0c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf11000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0c15cab0 con 0x7fdf0c0d0b40 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf11000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdf0c0d0f70 con 0x7fdf0c1567e0 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf11000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c160e40 con 0x7fdf0c15d810 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf0b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf0c1567e0 0x7fdf0c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf0b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf0c1567e0 0x7fdf0c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46724/0 (socket says 172.21.15.59:46724) 2024-04-24T06:33:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.739+0000 7fdf0b400640 1 -- 172.21.15.59:0/3134818283 learned_addr learned my addr 172.21.15.59:0/3134818283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf03e00640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 0x7fdf0c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf0b400640 1 -- 172.21.15.59:0/3134818283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 msgr2=0x7fdf0c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 0x7fdf0c0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf0b400640 1 -- 172.21.15.59:0/3134818283 >> v1:172.21.15.66:6789/0 conn(0x7fdf0c0d0b40 legacy=0x7fdf0c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf0b400640 1 -- 172.21.15.59:0/3134818283 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c15cd30 con 0x7fdf0c1567e0 2024-04-24T06:33:58.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.740+0000 7fdf03e00640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 0x7fdf0c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.741+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156bc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009870 tx=0x7fdefc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3275d1d6e0ace94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.741+0000 7fdf0aa00640 1 -- 172.21.15.59:0/3134818283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc037070 con 0x7fdf0c1567e0 2024-04-24T06:33:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.741+0000 7fdf0aa00640 1 -- 172.21.15.59:0/3134818283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdefc032440 con 0x7fdf0c1567e0 2024-04-24T06:33:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.741+0000 7fdf0aa00640 1 -- 172.21.15.59:0/3134818283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc032810 con 0x7fdf0c1567e0 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 -- 172.21.15.59:0/3134818283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c1567e0 msgr2=0x7fdf0c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156bc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009870 tx=0x7fdefc02a210 comp rx=0 tx=0).stop 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 -- 172.21.15.59:0/3134818283 shutdown_connections 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c15d810 0x7fdf0c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 --2- 172.21.15.59:0/3134818283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156bc0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 -- 172.21.15.59:0/3134818283 >> 172.21.15.59:0/3134818283 conn(0x7fdf0c06bdf0 msgr2=0x7fdf0c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 -- 172.21.15.59:0/3134818283 shutdown_connections 2024-04-24T06:33:58.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.742+0000 7fdf11000640 1 -- 172.21.15.59:0/3134818283 wait complete. 2024-04-24T06:33:58.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.744+0000 7fdf11000640 1 Processor -- start 2024-04-24T06:33:58.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.744+0000 7fdf11000640 1 -- start start 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c0d0b40 0x7fdf0c1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c160820 con 0x7fdf0c15d810 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c15d2f0 con 0x7fdf0c1567e0 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf11000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c1609a0 con 0x7fdf0c0d0b40 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46726/0 (socket says 172.21.15.59:46726) 2024-04-24T06:33:58.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.745+0000 7fdf0be00640 1 -- 172.21.15.59:0/3135022479 learned_addr learned my addr 172.21.15.59:0/3135022479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c0d0b40 0x7fdf0c1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf03e00640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 -- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c0d0b40 msgr2=0x7fdf0c1f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c0d0b40 0x7fdf0c1f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 -- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 msgr2=0x7fdf0c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 -- 172.21.15.59:0/3135022479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c0b4870 con 0x7fdf0c15d810 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf03e00640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf0be00640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b970 tx=0x7fdf0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04019070 con 0x7fdf0c15d810 2024-04-24T06:33:58.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0400cb40 con 0x7fdf0c15d810 2024-04-24T06:33:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04014440 con 0x7fdf0c15d810 2024-04-24T06:33:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.746+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c159ea0 con 0x7fdf0c15d810 2024-04-24T06:33:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.747+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1f5ad0 con 0x7fdf0c15d810 2024-04-24T06:33:58.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.748+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c04fd90 con 0x7fdf0c15d810 2024-04-24T06:33:58.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.749+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf0401c070 con 0x7fdf0c15d810 2024-04-24T06:33:58.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.750+0000 7fdf03400640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 0x7fdee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:58.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.751+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fdf0408b6f0 con 0x7fdf0c15d810 2024-04-24T06:33:58.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.751+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 0x7fdee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:58.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.751+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 0x7fdee806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdefc034000 tx=0x7fdefc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:58.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.755+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf04059bb0 con 0x7fdf0c15d810 2024-04-24T06:33:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.956+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf0c0b5150 con 0x7fdf0c15d810 2024-04-24T06:33:58.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.958+0000 7fdf03400640 1 -- 172.21.15.59:0/3135022479 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7fdf040594b0 con 0x7fdf0c15d810 2024-04-24T06:33:58.959 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:33:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 msgr2=0x7fdee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 0x7fdee806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdefc034000 tx=0x7fdefc02e000 comp rx=0 tx=0).stop 2024-04-24T06:33:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 msgr2=0x7fdf0c1573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b970 tx=0x7fdf0400be30 comp rx=0 tx=0).stop 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 shutdown_connections 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdee806c8f0 0x7fdee806edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf0c15d810 0x7fdf0c1573e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1567e0 0x7fdf0c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 --2- 172.21.15.59:0/3135022479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdf0c0d0b40 0x7fdf0c1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 >> 172.21.15.59:0/3135022479 conn(0x7fdf0c06bdf0 msgr2=0x7fdf0c15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 shutdown_connections 2024-04-24T06:33:58.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:58.960+0000 7fdf11000640 1 -- 172.21.15.59:0/3135022479 wait complete. 2024-04-24T06:33:58.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:33:59.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.200+0000 7f9a8a200640 1 Processor -- start 2024-04-24T06:33:59.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.200+0000 7f9a8a200640 1 -- start start 2024-04-24T06:33:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.201+0000 7f9a8a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 0x7f9a841416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.201+0000 7f9a8a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8415ba70 con 0x7f9a8415f060 2024-04-24T06:33:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.201+0000 7f9a8a200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8415f800 con 0x7f9a8415c250 2024-04-24T06:33:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.201+0000 7f9a8a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9a84141c20 con 0x7f9a840d05c0 2024-04-24T06:33:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.202+0000 7f9a83e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9a8415f060 0x7f9a840d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59934/0 (socket says 172.21.15.59:59934) 2024-04-24T06:33:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.202+0000 7f9a83e00640 1 -- 172.21.15.59:0/341531945 learned_addr learned my addr 172.21.15.59:0/341531945 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.202+0000 7f9a88e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 0x7f9a841416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.202+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302598213 0 0) 0x7f9a8415ba70 con 0x7f9a8415f060 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.202+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a6c003520 con 0x7f9a8415f060 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 809485288 0 0) 0x7f9a6c003520 con 0x7f9a8415f060 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 msgr2=0x7f9a841416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 0x7f9a841416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 >> v1:172.21.15.66:6789/0 conn(0x7f9a8415c250 legacy=0x7f9a841555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8415fae0 con 0x7f9a8415f060 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9a70001670 con 0x7f9a8415f060 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 >> v1:172.21.15.59:6789/0 conn(0x7f9a8415f060 legacy=0x7f9a840d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a6c0039e0 0x7f9a6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a6c004800 0x7f9a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 0x7f9a6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84141c20 con 0x7f9a6c0039e0 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c00ba70 con 0x7f9a6c004800 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c00de30 con 0x7f9a840d05c0 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.203+0000 7f9a88e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a6c0039e0 0x7f9a6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a83e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a6c004800 0x7f9a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a88e00640 1 -- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 msgr2=0x7f9a6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a88e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a840d05c0 0x7f9a6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a88e00640 1 -- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a6c004800 msgr2=0x7f9a6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a88e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a6c004800 0x7f9a6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a88e00640 1 -- 172.21.15.59:0/341531945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c003520 con 0x7f9a6c0039e0 2024-04-24T06:33:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.204+0000 7f9a83e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a6c004800 0x7f9a6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.205+0000 7f9a88e00640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a6c0039e0 0x7f9a6c003dc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009a90 tx=0x7f9a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f203ef19f88135e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.205+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78036070 con 0x7f9a6c0039e0 2024-04-24T06:33:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.205+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a78031400 con 0x7f9a6c0039e0 2024-04-24T06:33:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.205+0000 7f9a83400640 1 -- 172.21.15.59:0/341531945 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78009690 con 0x7f9a6c0039e0 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 -- 172.21.15.59:0/341531945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a6c0039e0 msgr2=0x7f9a6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 --2- 172.21.15.59:0/341531945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a6c0039e0 0x7f9a6c003dc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009a90 tx=0x7f9a6c005830 comp rx=0 tx=0).stop 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a89800640 1 -- 172.21.15.59:0/341531945 reap_dead start 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 -- 172.21.15.59:0/341531945 shutdown_connections 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 -- 172.21.15.59:0/341531945 >> 172.21.15.59:0/341531945 conn(0x7f9a8406bdf0 msgr2=0x7f9a8406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 -- 172.21.15.59:0/341531945 shutdown_connections 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.206+0000 7f9a8a200640 1 -- 172.21.15.59:0/341531945 wait complete. 2024-04-24T06:33:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.208+0000 7f9a8a200640 1 Processor -- start 2024-04-24T06:33:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.208+0000 7f9a8a200640 1 -- start start 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a6c0039e0 0x7f9a8409b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 0x7f9a8406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8415f4c0 con 0x7f9a84158050 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8415bdc0 con 0x7f9a841565b0 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a8a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9a84155ce0 con 0x7f9a6c0039e0 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55884/0 (socket says 172.21.15.59:55884) 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.209+0000 7f9a83e00640 1 -- 172.21.15.59:0/3405943528 learned_addr learned my addr 172.21.15.59:0/3405943528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a88e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a6c0039e0 0x7f9a8409b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 -- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a6c0039e0 msgr2=0x7f9a8409b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a89800640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 0x7f9a8406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a6c0039e0 0x7f9a8409b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 -- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 msgr2=0x7f9a8406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 0x7f9a8406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 -- 172.21.15.59:0/3405943528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8409ba40 con 0x7f9a84158050 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a89800640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 0x7f9a8406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:33:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a83e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9a70000fd0 tx=0x7f9a7000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70019070 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a700074e0 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007690 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.210+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a84158920 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.211+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a84158ee0 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.212+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a84141a10 con 0x7f9a84158050 2024-04-24T06:33:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.213+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a700049e0 con 0x7f9a84158050 2024-04-24T06:33:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.214+0000 7f9a80c00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 0x7f9a6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.214+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f9a7008b660 con 0x7f9a84158050 2024-04-24T06:33:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.214+0000 7f9a88e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 0x7f9a6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.215+0000 7f9a88e00640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 0x7f9a6406ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a78033000 tx=0x7f9a78008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:59.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.218+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a84141a10 con 0x7f9a84158050 2024-04-24T06:33:59.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.437+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a840cfec0 con 0x7f9a84158050 2024-04-24T06:33:59.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.438+0000 7f9a80c00640 1 -- 172.21.15.59:0/3405943528 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f9a70059a80 con 0x7f9a84158050 2024-04-24T06:33:59.438 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:33:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 msgr2=0x7f9a6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 0x7f9a6406ede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a78033000 tx=0x7f9a78008030 comp rx=0 tx=0).stop 2024-04-24T06:33:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 msgr2=0x7f9a84156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9a70000fd0 tx=0x7f9a7000dd40 comp rx=0 tx=0).stop 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 shutdown_connections 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9a6406c920 0x7f9a6406ede0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a841565b0 0x7f9a8406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a84158050 0x7f9a84156070 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 --2- 172.21.15.59:0/3405943528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9a6c0039e0 0x7f9a8409b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 >> 172.21.15.59:0/3405943528 conn(0x7f9a8406bdf0 msgr2=0x7f9a840b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 shutdown_connections 2024-04-24T06:33:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:33:59.440+0000 7f9a8a200640 1 -- 172.21.15.59:0/3405943528 wait complete. 2024-04-24T06:33:59.462 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:33:59.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:33:59.462 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:33:59.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57511]: fuse finished with error 0 and tester_r 0 2024-04-24T06:33:59.548 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:34:05.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:34:05.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:05.650 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:34:05.712 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:34:05.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.925+0000 7febc7a00640 1 Processor -- start 2024-04-24T06:34:05.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.926+0000 7febc7a00640 1 -- start start 2024-04-24T06:34:05.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc7a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febc015b840 0x7febc00b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc7a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7febc006df00 0x7febc006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc7a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc006ec60 con 0x7febc006f7c0 2024-04-24T06:34:05.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc7a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7febc00b46e0 con 0x7febc015b840 2024-04-24T06:34:05.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc7a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7febc00b49c0 con 0x7febc006df00 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febc015b840 0x7febc00b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febc015b840 0x7febc00b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55892/0 (socket says 172.21.15.59:55892) 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febbfe00640 1 -- 172.21.15.59:0/2063467923 learned_addr learned my addr 172.21.15.59:0/2063467923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.928+0000 7febc5000640 1 --1- 172.21.15.59:0/2063467923 >> v1:172.21.15.59:6789/0 conn(0x7febc006f7c0 0x7febc006fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59944/0 (socket says 172.21.15.59:59944) 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.929+0000 7febc5a00640 1 --2- 172.21.15.59:0/2063467923 >> v2:172.21.15.66:3300/0 conn(0x7febc006df00 0x7febc006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.929+0000 7febbfe00640 1 -- 172.21.15.59:0/2063467923 >> v2:172.21.15.66:3300/0 conn(0x7febc006df00 msgr2=0x7febc006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:05.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.929+0000 7febbfe00640 1 --2- 172.21.15.59:0/2063467923 >> v2:172.21.15.66:3300/0 conn(0x7febc006df00 0x7febc006e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:05.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.929+0000 7febbfe00640 1 -- 172.21.15.59:0/2063467923 >> v1:172.21.15.59:6789/0 conn(0x7febc006f7c0 legacy=0x7febc006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:05.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.929+0000 7febbfe00640 1 -- 172.21.15.59:0/2063467923 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc00b4ca0 con 0x7febc015b840 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.930+0000 7febbfe00640 1 --2- 172.21.15.59:0/2063467923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc015b840 0x7febc00b54a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7febc00731b0 tx=0x7feba802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0fa2883cb350fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.930+0000 7febbf400640 1 -- 172.21.15.59:0/2063467923 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800b2e0 con 0x7febc015b840 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.930+0000 7febbf400640 1 -- 172.21.15.59:0/2063467923 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba8037070 con 0x7febc015b840 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.930+0000 7febbf400640 1 -- 172.21.15.59:0/2063467923 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800b480 con 0x7febc015b840 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 -- 172.21.15.59:0/2063467923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc015b840 msgr2=0x7febc00b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:05.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 --2- 172.21.15.59:0/2063467923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc015b840 0x7febc00b54a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7febc00731b0 tx=0x7feba802c940 comp rx=0 tx=0).stop 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 -- 172.21.15.59:0/2063467923 shutdown_connections 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 --2- 172.21.15.59:0/2063467923 >> v2:172.21.15.66:3300/0 conn(0x7febc006df00 0x7febc006e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 --2- 172.21.15.59:0/2063467923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc015b840 0x7febc00b54a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 -- 172.21.15.59:0/2063467923 >> 172.21.15.59:0/2063467923 conn(0x7febc0067ea0 msgr2=0x7febc006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 -- 172.21.15.59:0/2063467923 shutdown_connections 2024-04-24T06:34:05.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.931+0000 7febc7a00640 1 -- 172.21.15.59:0/2063467923 wait complete. 2024-04-24T06:34:05.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.933+0000 7febc7a00640 1 Processor -- start 2024-04-24T06:34:05.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.933+0000 7febc7a00640 1 -- start start 2024-04-24T06:34:05.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7febc006df00 0x7febc0145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febc006f7c0 0x7febc01493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7febc00ce1c0 con 0x7febc0151c70 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febc006e8d0 con 0x7febc006f7c0 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.934+0000 7febc7a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7febc00ce340 con 0x7febc006df00 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55902/0 (socket says 172.21.15.59:55902) 2024-04-24T06:34:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 -- 172.21.15.59:0/3287185548 learned_addr learned my addr 172.21.15.59:0/3287185548 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5000640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7febc006df00 0x7febc0145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febbfe00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febc006f7c0 0x7febc01493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 -- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7febc006df00 msgr2=0x7febc0145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7febc006df00 0x7febc0145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 -- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febc006f7c0 msgr2=0x7febc01493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febc006f7c0 0x7febc01493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 -- 172.21.15.59:0/3287185548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0153830 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.935+0000 7febc5a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7febb800d800 tx=0x7febb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.936+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb800b6c0 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.936+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb8019070 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.936+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb800bac0 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.936+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc0153ab0 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.936+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc0154070 con 0x7febc0151c70 2024-04-24T06:34:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.937+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc0068a80 con 0x7febc0151c70 2024-04-24T06:34:05.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.938+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febb800bc60 con 0x7febc0151c70 2024-04-24T06:34:05.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.939+0000 7febbcc00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 0x7feb9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:05.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.939+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7febb808c060 con 0x7febc0151c70 2024-04-24T06:34:05.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.939+0000 7febc5000640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 0x7feb9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:05.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.940+0000 7febc5000640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 0x7feb9c06edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f60 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:05.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:05.944+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febc0153830 con 0x7febc0151c70 2024-04-24T06:34:06.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.187+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc0158ed0 con 0x7febc0151c70 2024-04-24T06:34:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.187+0000 7febbcc00640 1 -- 172.21.15.59:0/3287185548 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5174 (secure 0 0 0) 0x7febb805a3f0 con 0x7febc0151c70 2024-04-24T06:34:06.189 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:06.189 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6759,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:33:47.780106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2419732942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2419732942},{"type":"v1","addr":"172.21.15.66:6835","nonce":2419732942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6771,"qdb_cluster":[6771]},"id":5}]} 2024-04-24T06:34:06.189 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 44 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 msgr2=0x7feb9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 0x7feb9c06edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f60 tx=0x7febb0009040 comp rx=0 tx=0).stop 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 msgr2=0x7febc0150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7febb800d800 tx=0x7febb800dcc0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 shutdown_connections 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb9c06c8f0 0x7feb9c06edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febc0151c70 0x7febc0150a20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febc006f7c0 0x7febc01493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 --2- 172.21.15.59:0/3287185548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7febc006df00 0x7febc0145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.190+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 >> 172.21.15.59:0/3287185548 conn(0x7febc0067ea0 msgr2=0x7febc0072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.191+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 shutdown_connections 2024-04-24T06:34:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.191+0000 7febc7a00640 1 -- 172.21.15.59:0/3287185548 wait complete. 2024-04-24T06:34:06.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-24T06:34:06.478 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:34:06.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:34:06.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:34:06.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:34:06.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:34:06.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.692+0000 7fefb3200640 1 Processor -- start 2024-04-24T06:34:06.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.692+0000 7fefb3200640 1 -- start start 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb41416e0 0x7fefb415f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefb415c250 0x7fefb41555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb415ba70 con 0x7fefb40d05c0 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fefb415bdc0 con 0x7fefb415c250 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fefb415f5a0 con 0x7fefb41416e0 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefb415c250 0x7fefb41555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefb415c250 0x7fefb41555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55912/0 (socket says 172.21.15.59:55912) 2024-04-24T06:34:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefb2800640 1 -- 172.21.15.59:0/977292990 learned_addr learned my addr 172.21.15.59:0/977292990 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:06.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.694+0000 7fefabe00640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb41416e0 0x7fefb415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.695+0000 7fefb2800640 1 -- 172.21.15.59:0/977292990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb41416e0 msgr2=0x7fefb415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:06.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.695+0000 7fefb2800640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb41416e0 0x7fefb415f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.695+0000 7fefb2800640 1 -- 172.21.15.59:0/977292990 >> v1:172.21.15.66:6789/0 conn(0x7fefb40d05c0 legacy=0x7fefb40d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:06.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.695+0000 7fefb2800640 1 -- 172.21.15.59:0/977292990 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb415f800 con 0x7fefb415c250 2024-04-24T06:34:06.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.696+0000 7fefb2800640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb415c250 0x7fefb41555d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6773262fd7d2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:06.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.696+0000 7fefb1400640 1 -- 172.21.15.59:0/977292990 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac036070 con 0x7fefb415c250 2024-04-24T06:34:06.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.696+0000 7fefb1400640 1 -- 172.21.15.59:0/977292990 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefac031600 con 0x7fefb415c250 2024-04-24T06:34:06.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.696+0000 7fefb1400640 1 -- 172.21.15.59:0/977292990 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac0319d0 con 0x7fefb415c250 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 -- 172.21.15.59:0/977292990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb415c250 msgr2=0x7fefb41555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb415c250 0x7fefb41555d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac02a6d0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 -- 172.21.15.59:0/977292990 shutdown_connections 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb41416e0 0x7fefb415f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 --2- 172.21.15.59:0/977292990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb415c250 0x7fefb41555d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 -- 172.21.15.59:0/977292990 >> 172.21.15.59:0/977292990 conn(0x7fefb406bdf0 msgr2=0x7fefb406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 -- 172.21.15.59:0/977292990 shutdown_connections 2024-04-24T06:34:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.697+0000 7fefb3200640 1 -- 172.21.15.59:0/977292990 wait complete. 2024-04-24T06:34:06.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.699+0000 7fefb3200640 1 Processor -- start 2024-04-24T06:34:06.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.700+0000 7fefb3200640 1 -- start start 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefb41416e0 0x7fefb4153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb415c250 0x7fefb41f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4141b40 con 0x7fefb40d05c0 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fefb415fdc0 con 0x7fefb41416e0 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fefb415c6b0 con 0x7fefb415c250 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55922/0 (socket says 172.21.15.59:55922) 2024-04-24T06:34:06.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.701+0000 7fefb1e00640 1 -- 172.21.15.59:0/689517108 learned_addr learned my addr 172.21.15.59:0/689517108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefabe00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefb41416e0 0x7fefb4153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb2800640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb415c250 0x7fefb41f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 -- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb415c250 msgr2=0x7fefb41f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb415c250 0x7fefb41f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 -- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefb41416e0 msgr2=0x7fefb4153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefb41416e0 0x7fefb4153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 -- 172.21.15.59:0/689517108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb416a410 con 0x7fefb40d05c0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb1e00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c430 tx=0x7fefa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4011790 con 0x7fefb40d05c0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa4017070 con 0x7fefb40d05c0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb4169e50 con 0x7fefb40d05c0 2024-04-24T06:34:06.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4011b70 con 0x7fefb40d05c0 2024-04-24T06:34:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.702+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb4154940 con 0x7fefb40d05c0 2024-04-24T06:34:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.703+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb404fd90 con 0x7fefb40d05c0 2024-04-24T06:34:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.705+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefa4011d10 con 0x7fefb40d05c0 2024-04-24T06:34:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.706+0000 7fefaa000640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.706+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fefa408ba20 con 0x7fefb40d05c0 2024-04-24T06:34:06.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.706+0000 7fefabe00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:06.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.707+0000 7fefabe00640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fefb409dbb0 tx=0x7fef9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:06.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.710+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefa4059e40 con 0x7fefb40d05c0 2024-04-24T06:34:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.951+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fefb40cfec0 con 0x7fefb40d05c0 2024-04-24T06:34:06.953 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:34:06.953+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:34:06.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.957+0000 7fefaa000640 1 -- 172.21.15.59:0/689517108 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fefa4059740 con 0x7fefb40d05c0 2024-04-24T06:34:06.958 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.959+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 msgr2=0x7fef9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fefb409dbb0 tx=0x7fef9c008040 comp rx=0 tx=0).stop 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 msgr2=0x7fefb409b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c430 tx=0x7fefa400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 shutdown_connections 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fefb415c250 0x7fefb41f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fefb41416e0 0x7fefb4153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 --2- 172.21.15.59:0/689517108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb40d05c0 0x7fefb409b770 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:06.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 >> 172.21.15.59:0/689517108 conn(0x7fefb406bdf0 msgr2=0x7fefb4151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:06.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 shutdown_connections 2024-04-24T06:34:06.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:06.960+0000 7fefb3200640 1 -- 172.21.15.59:0/689517108 wait complete. 2024-04-24T06:34:06.982 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-24T06:34:06.992 INFO:tasks.ceph.mds.b.smithi066.stderr: -3601> 2024-04-24T06:33:51.187+0000 7f4176a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:34:07.074 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:34:07.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.078+0000 7f067391df40 1 Processor -- start 2024-04-24T06:34:07.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.078+0000 7f067391df40 1 -- start start 2024-04-24T06:34:07.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23e2bc90 0x55ee23e26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee23e42d60 con 0x55ee23d691d0 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee23d12cc0 con 0x55ee23e2bc90 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.079+0000 7f067391df40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ee23d69b20 con 0x55ee23e42980 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55934/0 (socket says 172.21.15.59:55934) 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 -- 172.21.15.59:0/1830036808 learned_addr learned my addr 172.21.15.59:0/1830036808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670200640 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23e2bc90 0x55ee23e26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0671600640 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 -- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 msgr2=0x55ee23e414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 -- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23e2bc90 msgr2=0x55ee23e26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23e2bc90 0x55ee23e26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 -- 172.21.15.59:0/1830036808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee23e27ee0 con 0x55ee23d691d0 2024-04-24T06:34:07.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f0670c00640 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0660001660 tx=0x7f0660001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7f9d0c60417b08 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.080+0000 7f066ee00640 1 -- 172.21.15.59:0/1830036808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660004520 con 0x55ee23d691d0 2024-04-24T06:34:07.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f066ee00640 1 -- 172.21.15.59:0/1830036808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06600046c0 con 0x55ee23d691d0 2024-04-24T06:34:07.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f066ee00640 1 -- 172.21.15.59:0/1830036808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660004ab0 con 0x55ee23d691d0 2024-04-24T06:34:07.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 -- 172.21.15.59:0/1830036808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 msgr2=0x55ee23e3f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0660001660 tx=0x7f0660001b20 comp rx=0 tx=0).stop 2024-04-24T06:34:07.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 -- 172.21.15.59:0/1830036808 shutdown_connections 2024-04-24T06:34:07.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23e2bc90 0x55ee23e26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 --2- 172.21.15.59:0/1830036808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23d691d0 0x55ee23e3f6d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 -- 172.21.15.59:0/1830036808 >> 172.21.15.59:0/1830036808 conn(0x55ee23d9af40 msgr2=0x55ee23db40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 -- 172.21.15.59:0/1830036808 shutdown_connections 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.081+0000 7f067391df40 1 -- 172.21.15.59:0/1830036808 wait complete. 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.082+0000 7f067391df40 1 Processor -- start 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.082+0000 7f067391df40 1 -- start start 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23d691d0 0x55ee23ed12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 0x55ee23e23690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e23bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ee23e2b780 con 0x55ee23e2bc90 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ee23e2ba20 con 0x55ee23d691d0 2024-04-24T06:34:07.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f067391df40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ee23e2af40 con 0x55ee23e42980 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f0670200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 0x55ee23e23690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f0670200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 0x55ee23e23690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55938/0 (socket says 172.21.15.59:55938) 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f0670200640 1 -- 172.21.15.59:0/2648001895 learned_addr learned my addr 172.21.15.59:0/2648001895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.083+0000 7f0670c00640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23d691d0 0x55ee23ed12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0671600640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e23bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 -- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 msgr2=0x55ee23e23bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ee23e42980 0x55ee23e23bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 -- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23d691d0 msgr2=0x55ee23ed12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23d691d0 0x55ee23ed12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 -- 172.21.15.59:0/2648001895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee23e24110 con 0x55ee23e2bc90 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670c00640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee23d691d0 0x55ee23ed12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f0670200640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 0x55ee23e23690 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0658008700 tx=0x7f06580087f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f066da00640 1 -- 172.21.15.59:0/2648001895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580148a0 con 0x55ee23e2bc90 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f066da00640 1 -- 172.21.15.59:0/2648001895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065801a070 con 0x55ee23e2bc90 2024-04-24T06:34:07.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f066da00640 1 -- 172.21.15.59:0/2648001895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658014c40 con 0x55ee23e2bc90 2024-04-24T06:34:07.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.084+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee23d98980 con 0x55ee23e2bc90 2024-04-24T06:34:07.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.085+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee23d98f40 con 0x55ee23e2bc90 2024-04-24T06:34:07.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.087+0000 7f066da00640 1 -- 172.21.15.59:0/2648001895 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0658015470 con 0x55ee23e2bc90 2024-04-24T06:34:07.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.087+0000 7f066da00640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f064c06b2d0 0x7f064c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.087+0000 7f066da00640 1 -- 172.21.15.59:0/2648001895 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f065808b250 con 0x55ee23e2bc90 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.088+0000 7f0670c00640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f064c06b2d0 0x7f064c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.088+0000 7f067391df40 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55ee23ed17f0 0x55ee23ed38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.088+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) -- 0x55ee23ed3e30 con 0x55ee23ed17f0 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.088+0000 7f0670c00640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f064c06b2d0 0x7f064c06d790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0660001660 tx=0x7f0660015040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.088+0000 7f0671600640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55ee23ed17f0 0x55ee23ed38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.089+0000 7f0671600640 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55ee23ed17f0 0x55ee23ed38f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f0671600640 1 -- 172.21.15.59:0/2648001895 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v70'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f066805c040 con 0x55ee23ed17f0 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55ee23ed17f0 msgr2=0x55ee23ed38f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55ee23ed17f0 0x55ee23ed38f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f064c06b2d0 msgr2=0x7f064c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f064c06b2d0 0x7f064c06d790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0660001660 tx=0x7f0660015040 comp rx=0 tx=0).stop 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 msgr2=0x55ee23e23690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.091+0000 7f067391df40 1 --2- 172.21.15.59:0/2648001895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ee23e2bc90 0x55ee23e23690 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0658008700 tx=0x7f06580087f0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.092+0000 7f0671600640 1 -- 172.21.15.59:0/2648001895 reap_dead start 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.092+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 shutdown_connections 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.092+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 >> 172.21.15.59:0/2648001895 conn(0x55ee23d9af40 msgr2=0x55ee23db40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.092+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 shutdown_connections 2024-04-24T06:34:07.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.092+0000 7f067391df40 1 -- 172.21.15.59:0/2648001895 wait complete. 2024-04-24T06:34:07.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:34:07.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.286+0000 7fc48f200640 1 Processor -- start 2024-04-24T06:34:07.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.286+0000 7fc48f200640 1 -- start start 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc48815ed20 0x7fc4880d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4880b3600 0x7fc4880d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880b5da0 con 0x7fc4880b3f10 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc48815f100 con 0x7fc4880b3600 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc4880d0a20 con 0x7fc48815ed20 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4880b3600 0x7fc4880d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4880b3600 0x7fc4880d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55946/0 (socket says 172.21.15.59:55946) 2024-04-24T06:34:07.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.288+0000 7fc48d400640 1 -- 172.21.15.59:0/4109273604 learned_addr learned my addr 172.21.15.59:0/4109273604 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:07.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.289+0000 7fc48d400640 1 -- 172.21.15.59:0/4109273604 >> v2:172.21.15.66:3300/0 conn(0x7fc48815ed20 msgr2=0x7fc4880d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:07.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.289+0000 7fc48d400640 1 --2- 172.21.15.59:0/4109273604 >> v2:172.21.15.66:3300/0 conn(0x7fc48815ed20 0x7fc4880d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.289+0000 7fc48d400640 1 -- 172.21.15.59:0/4109273604 >> v1:172.21.15.66:6789/0 conn(0x7fc4880b3f10 legacy=0x7fc4880b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:07.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.289+0000 7fc48d400640 1 -- 172.21.15.59:0/4109273604 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880d2ed0 con 0x7fc4880b3600 2024-04-24T06:34:07.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.290+0000 7fc48d400640 1 --2- 172.21.15.59:0/4109273604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4880b3600 0x7fc4880d13e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc48808e2a0 tx=0x7fc47802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b9da5d3677b3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.290+0000 7fc48ca00640 1 -- 172.21.15.59:0/4109273604 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478037070 con 0x7fc4880b3600 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.290+0000 7fc48ca00640 1 -- 172.21.15.59:0/4109273604 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47802ae20 con 0x7fc4880b3600 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.290+0000 7fc48ca00640 1 -- 172.21.15.59:0/4109273604 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478032660 con 0x7fc4880b3600 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 -- 172.21.15.59:0/4109273604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4880b3600 msgr2=0x7fc4880d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 --2- 172.21.15.59:0/4109273604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4880b3600 0x7fc4880d13e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc48808e2a0 tx=0x7fc47802a110 comp rx=0 tx=0).stop 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 -- 172.21.15.59:0/4109273604 shutdown_connections 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 --2- 172.21.15.59:0/4109273604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4880b3600 0x7fc4880d13e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 --2- 172.21.15.59:0/4109273604 >> v2:172.21.15.66:3300/0 conn(0x7fc48815ed20 0x7fc4880d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 -- 172.21.15.59:0/4109273604 >> 172.21.15.59:0/4109273604 conn(0x7fc48808bf90 msgr2=0x7fc4880d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:07.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 -- 172.21.15.59:0/4109273604 shutdown_connections 2024-04-24T06:34:07.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.291+0000 7fc48f200640 1 -- 172.21.15.59:0/4109273604 wait complete. 2024-04-24T06:34:07.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.292+0000 7fc48f200640 1 Processor -- start 2024-04-24T06:34:07.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.293+0000 7fc48f200640 1 -- start start 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.293+0000 7fc48f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4880b3600 0x7fc488156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4880b3f10 0x7fc4880b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880d0290 con 0x7fc48815ed20 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880d3490 con 0x7fc4880b3600 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880d0410 con 0x7fc4880b3f10 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55950/0 (socket says 172.21.15.59:55950) 2024-04-24T06:34:07.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 -- 172.21.15.59:0/1380315093 learned_addr learned my addr 172.21.15.59:0/1380315093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48de00640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4880b3600 0x7fc488156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48d400640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4880b3f10 0x7fc4880b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 -- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4880b3f10 msgr2=0x7fc4880b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4880b3f10 0x7fc4880b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 -- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4880b3600 msgr2=0x7fc488156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4880b3600 0x7fc488156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.294+0000 7fc48e800640 1 -- 172.21.15.59:0/1380315093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc488156fc0 con 0x7fc48815ed20 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc48e800640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc48400c7b0 tx=0x7fc48400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484017070 con 0x7fc48815ed20 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484003da0 con 0x7fc48815ed20 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4840052b0 con 0x7fc48815ed20 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4881f81d0 con 0x7fc48815ed20 2024-04-24T06:34:07.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.295+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4881f8790 con 0x7fc48815ed20 2024-04-24T06:34:07.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.296+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4880d1f80 con 0x7fc48815ed20 2024-04-24T06:34:07.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.297+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc484005450 con 0x7fc48815ed20 2024-04-24T06:34:07.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.298+0000 7fc482000640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:07.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.298+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fc48408b1b0 con 0x7fc48815ed20 2024-04-24T06:34:07.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.298+0000 7fc48de00640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:07.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.299+0000 7fc48de00640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00a770 tx=0x7fc47c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:07.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.303+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc484059570 con 0x7fc48815ed20 2024-04-24T06:34:07.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.558+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc48804fd90 con 0x7fc48815ed20 2024-04-24T06:34:07.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.970+0000 7fc482000640 1 -- 172.21.15.59:0/1380315093 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v46) ==== 162+0+0 (secure 0 0 0) 0x7fc484058e70 con 0x7fc48815ed20 2024-04-24T06:34:07.971 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:34:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.973+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 msgr2=0x7fc46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.973+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 0x7fc46c06edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00a770 tx=0x7fc47c008040 comp rx=0 tx=0).stop 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 msgr2=0x7fc4880b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc48400c7b0 tx=0x7fc48400cc70 comp rx=0 tx=0).stop 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 shutdown_connections 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46c06c8f0 0x7fc46c06edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48815ed20 0x7fc4880b2800 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc4880b3f10 0x7fc4880b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 --2- 172.21.15.59:0/1380315093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4880b3600 0x7fc488156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:07.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 >> 172.21.15.59:0/1380315093 conn(0x7fc48808bf90 msgr2=0x7fc48806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:07.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 shutdown_connections 2024-04-24T06:34:07.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:07.974+0000 7fc48f200640 1 -- 172.21.15.59:0/1380315093 wait complete. 2024-04-24T06:34:08.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:34:08.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.192+0000 7fc227200640 1 Processor -- start 2024-04-24T06:34:08.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.193+0000 7fc227200640 1 -- start start 2024-04-24T06:34:08.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.194+0000 7fc227200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc2281455c0 0x7fc22806c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.194+0000 7fc227200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2280b24a0 0x7fc2280b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.194+0000 7fc227200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2280b1400 con 0x7fc22806b9d0 2024-04-24T06:34:08.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.194+0000 7fc227200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc2280b2010 con 0x7fc2280b24a0 2024-04-24T06:34:08.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.194+0000 7fc227200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc2280b16e0 con 0x7fc2281455c0 2024-04-24T06:34:08.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.195+0000 7fc225400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2280b24a0 0x7fc2280b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.195+0000 7fc225e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc2281455c0 0x7fc22806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.195+0000 7fc225400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc2280b24a0 0x7fc2280b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:47344/0 (socket says 172.21.15.59:47344) 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.195+0000 7fc225400640 1 -- 172.21.15.59:0/1831461100 learned_addr learned my addr 172.21.15.59:0/1831461100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.196+0000 7fc225400640 1 -- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc2281455c0 msgr2=0x7fc22806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.196+0000 7fc225400640 1 --2- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc2281455c0 0x7fc22806c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.196+0000 7fc225400640 1 -- 172.21.15.59:0/1831461100 >> v1:172.21.15.66:6789/0 conn(0x7fc22806b9d0 legacy=0x7fc22815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.196+0000 7fc225400640 1 -- 172.21.15.59:0/1831461100 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280b19c0 con 0x7fc2280b24a0 2024-04-24T06:34:08.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.197+0000 7fc225400640 1 --2- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2280b28e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc210009870 tx=0x7fc21002a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d67fb6595b3deee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:08.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc224a00640 1 -- 172.21.15.59:0/1831461100 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210037070 con 0x7fc2280b24a0 2024-04-24T06:34:08.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc224a00640 1 -- 172.21.15.59:0/1831461100 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210032640 con 0x7fc2280b24a0 2024-04-24T06:34:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc224a00640 1 -- 172.21.15.59:0/1831461100 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210032a10 con 0x7fc2280b24a0 2024-04-24T06:34:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 -- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 msgr2=0x7fc2280b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 --2- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2280b28e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc210009870 tx=0x7fc21002a2e0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 -- 172.21.15.59:0/1831461100 shutdown_connections 2024-04-24T06:34:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 --2- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2280b28e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 --2- 172.21.15.59:0/1831461100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc2281455c0 0x7fc22806c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.198+0000 7fc227200640 1 -- 172.21.15.59:0/1831461100 >> 172.21.15.59:0/1831461100 conn(0x7fc2280b6750 msgr2=0x7fc2280b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.199+0000 7fc227200640 1 -- 172.21.15.59:0/1831461100 shutdown_connections 2024-04-24T06:34:08.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.199+0000 7fc227200640 1 -- 172.21.15.59:0/1831461100 wait complete. 2024-04-24T06:34:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.200+0000 7fc227200640 1 Processor -- start 2024-04-24T06:34:08.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.200+0000 7fc227200640 1 -- start start 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.201+0000 7fc227200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc22806b9d0 0x7fc22813cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.201+0000 7fc227200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2281e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.201+0000 7fc227200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.201+0000 7fc227200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815d570 con 0x7fc2281455c0 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc227200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815d3f0 con 0x7fc2280b24a0 2024-04-24T06:34:08.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc227200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc22815d6f0 con 0x7fc22806b9d0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55964/0 (socket says 172.21.15.59:55964) 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 -- 172.21.15.59:0/4240626212 learned_addr learned my addr 172.21.15.59:0/4240626212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc225e00640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc22806b9d0 0x7fc22813cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc225400640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2281e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 -- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc22806b9d0 msgr2=0x7fc22813cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc22806b9d0 0x7fc22813cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 -- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 msgr2=0x7fc2281e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2281e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.202+0000 7fc226800640 1 -- 172.21.15.59:0/4240626212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281e4380 con 0x7fc2281455c0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc226800640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc220001eb0 tx=0x7fc22000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:08.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220017070 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2200052d0 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220003810 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281e4660 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.203+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281e4f00 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.204+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2281e4830 con 0x7fc2281455c0 2024-04-24T06:34:08.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.205+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc22001a070 con 0x7fc2281455c0 2024-04-24T06:34:08.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.206+0000 7fc21e000640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 0x7fc20406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:08.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.206+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fc22008aff0 con 0x7fc2281455c0 2024-04-24T06:34:08.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.207+0000 7fc225e00640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 0x7fc20406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:08.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.207+0000 7fc225e00640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 0x7fc20406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc2180073f0 tx=0x7fc21800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:08.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.211+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc220059320 con 0x7fc2281455c0 2024-04-24T06:34:08.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.466+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc22804fd90 con 0x7fc2281455c0 2024-04-24T06:34:08.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.467+0000 7fc21e000640 1 -- 172.21.15.59:0/4240626212 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5168 (secure 0 0 0) 0x7fc220058c20 con 0x7fc2281455c0 2024-04-24T06:34:08.468 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:08.468 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:34:07.975745+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-24T06:34:08.468 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 47 2024-04-24T06:34:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 msgr2=0x7fc20406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 0x7fc20406edb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc2180073f0 tx=0x7fc21800b040 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 msgr2=0x7fc2281f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc220001eb0 tx=0x7fc22000c890 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 shutdown_connections 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc20406c8f0 0x7fc20406edb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281455c0 0x7fc2281f4f30 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2280b24a0 0x7fc2281e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 --2- 172.21.15.59:0/4240626212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc22806b9d0 0x7fc22813cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 >> 172.21.15.59:0/4240626212 conn(0x7fc2280b6750 msgr2=0x7fc22806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 shutdown_connections 2024-04-24T06:34:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:08.470+0000 7fc227200640 1 -- 172.21.15.59:0/4240626212 wait complete. 2024-04-24T06:34:08.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:33:44.703546+0000', 'modified': '2024-04-24T06:34:07.975745+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/209407257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 209407257}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 209407257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:34:08.894 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6759:up:replay 2024-04-24T06:34:09.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:34:10.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.115+0000 7fb42cc00640 1 Processor -- start 2024-04-24T06:34:10.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.116+0000 7fb42cc00640 1 -- start start 2024-04-24T06:34:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb42cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb428160960 0x7fb428163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb42cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4280b4550 0x7fb42806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb42cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb428160d40 con 0x7fb4281644b0 2024-04-24T06:34:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb42cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb428164270 con 0x7fb4280b4550 2024-04-24T06:34:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb42cc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb428166de0 con 0x7fb428160960 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.117+0000 7fb426800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4280b4550 0x7fb42806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4280b4550 0x7fb42806e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58632/0 (socket says 172.21.15.59:58632) 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 -- 172.21.15.59:0/927975712 learned_addr learned my addr 172.21.15.59:0/927975712 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb425e00640 1 --2- 172.21.15.59:0/927975712 >> v2:172.21.15.66:3300/0 conn(0x7fb428160960 0x7fb428163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 -- 172.21.15.59:0/927975712 >> v2:172.21.15.66:3300/0 conn(0x7fb428160960 msgr2=0x7fb428163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 --2- 172.21.15.59:0/927975712 >> v2:172.21.15.66:3300/0 conn(0x7fb428160960 0x7fb428163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 -- 172.21.15.59:0/927975712 >> v1:172.21.15.59:6789/0 conn(0x7fb4281644b0 legacy=0x7fb4280b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.118+0000 7fb426800640 1 -- 172.21.15.59:0/927975712 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4281670c0 con 0x7fb4280b4550 2024-04-24T06:34:10.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.119+0000 7fb426800640 1 --2- 172.21.15.59:0/927975712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4280b4550 0x7fb42806e8d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb428072e60 tx=0x7fb42002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48853c2d24773ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.119+0000 7fb424a00640 1 -- 172.21.15.59:0/927975712 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420037070 con 0x7fb4280b4550 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb424a00640 1 -- 172.21.15.59:0/927975712 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb420032440 con 0x7fb4280b4550 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb424a00640 1 -- 172.21.15.59:0/927975712 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420032810 con 0x7fb4280b4550 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 -- 172.21.15.59:0/927975712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4280b4550 msgr2=0x7fb42806e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 --2- 172.21.15.59:0/927975712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4280b4550 0x7fb42806e8d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb428072e60 tx=0x7fb42002a270 comp rx=0 tx=0).stop 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 -- 172.21.15.59:0/927975712 shutdown_connections 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 --2- 172.21.15.59:0/927975712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4280b4550 0x7fb42806e8d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 --2- 172.21.15.59:0/927975712 >> v2:172.21.15.66:3300/0 conn(0x7fb428160960 0x7fb428163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 -- 172.21.15.59:0/927975712 >> 172.21.15.59:0/927975712 conn(0x7fb428067ea0 msgr2=0x7fb42806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 -- 172.21.15.59:0/927975712 shutdown_connections 2024-04-24T06:34:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.120+0000 7fb42cc00640 1 -- 172.21.15.59:0/927975712 wait complete. 2024-04-24T06:34:10.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.122+0000 7fb42cc00640 1 Processor -- start 2024-04-24T06:34:10.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.122+0000 7fb42cc00640 1 -- start start 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4280b4550 0x7fb4280cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 0x7fb4280d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281668e0 con 0x7fb428160960 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb428167680 con 0x7fb4281644b0 2024-04-24T06:34:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb42cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb428166a60 con 0x7fb4280b4550 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.123+0000 7fb425400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58638/0 (socket says 172.21.15.59:58638) 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 -- 172.21.15.59:0/19519446 learned_addr learned my addr 172.21.15.59:0/19519446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425e00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4280b4550 0x7fb4280cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb426800640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 0x7fb4280d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 -- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4280b4550 msgr2=0x7fb4280cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4280b4550 0x7fb4280cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 -- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 msgr2=0x7fb4280d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 0x7fb4280d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 -- 172.21.15.59:0/19519446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42814fb70 con 0x7fb428160960 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb426800640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 0x7fb4280d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.124+0000 7fb425400640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c460 tx=0x7fb41000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:10.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.125+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100052d0 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.125+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb410012630 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.125+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100054b0 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.125+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4280d0ff0 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.125+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4280cfcf0 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.126+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428068a80 con 0x7fb428160960 2024-04-24T06:34:10.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.127+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb410012a20 con 0x7fb428160960 2024-04-24T06:34:10.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.128+0000 7fb41e000640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 0x7fb40406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:10.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.128+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fb41008b880 con 0x7fb428160960 2024-04-24T06:34:10.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.128+0000 7fb425e00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 0x7fb40406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:10.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.129+0000 7fb425e00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 0x7fb40406edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb4180099c0 tx=0x7fb418003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:10.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.135+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb42814fb70 con 0x7fb428160960 2024-04-24T06:34:10.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.378+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb428165fe0 con 0x7fb428160960 2024-04-24T06:34:10.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.380+0000 7fb41e000640 1 -- 172.21.15.59:0/19519446 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5169 (secure 0 0 0) 0x7fb410059b20 con 0x7fb428160960 2024-04-24T06:34:10.381 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:10.381 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:34:08.988810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-24T06:34:10.381 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 49 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.382+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 msgr2=0x7fb40406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.382+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 0x7fb40406edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb4180099c0 tx=0x7fb418003990 comp rx=0 tx=0).stop 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.382+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 msgr2=0x7fb42814f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.382+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c460 tx=0x7fb41000c920 comp rx=0 tx=0).stop 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.382+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 shutdown_connections 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb40406c8f0 0x7fb40406edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4281644b0 0x7fb4280d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb428160960 0x7fb42814f630 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 --2- 172.21.15.59:0/19519446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4280b4550 0x7fb4280cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:10.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 >> 172.21.15.59:0/19519446 conn(0x7fb428067ea0 msgr2=0x7fb428161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:10.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 shutdown_connections 2024-04-24T06:34:10.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:10.383+0000 7fb42cc00640 1 -- 172.21.15.59:0/19519446 wait complete. 2024-04-24T06:34:10.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:33:44.703546+0000', 'modified': '2024-04-24T06:34:08.988810+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.59:6837/209407257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 209407257}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 209407257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:34:10.804 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6759:up:rejoin 2024-04-24T06:34:11.185 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:34:11.184+0000 7fbc83c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:34:11.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:34:12.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.028+0000 7faa47c00640 1 Processor -- start 2024-04-24T06:34:12.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.029+0000 7faa47c00640 1 -- start start 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.030+0000 7faa47c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa400cd020 0x7faa400b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.030+0000 7faa47c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faa400b3e40 0x7faa40160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.030+0000 7faa47c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa400cd520 con 0x7faa401454e0 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa47c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faa400cddc0 con 0x7faa400cd020 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa47c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faa401462c0 con 0x7faa400b3e40 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa400cd020 0x7faa400b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa400cd020 0x7faa400b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58646/0 (socket says 172.21.15.59:58646) 2024-04-24T06:34:12.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 -- 172.21.15.59:0/1567533698 learned_addr learned my addr 172.21.15.59:0/1567533698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 -- 172.21.15.59:0/1567533698 >> v2:172.21.15.66:3300/0 conn(0x7faa400b3e40 msgr2=0x7faa40160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa45c00640 1 --2- 172.21.15.59:0/1567533698 >> v2:172.21.15.66:3300/0 conn(0x7faa400b3e40 0x7faa40160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 --2- 172.21.15.59:0/1567533698 >> v2:172.21.15.66:3300/0 conn(0x7faa400b3e40 0x7faa40160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 -- 172.21.15.59:0/1567533698 >> v1:172.21.15.66:6789/0 conn(0x7faa401454e0 legacy=0x7faa40145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.031+0000 7faa3fe00640 1 -- 172.21.15.59:0/1567533698 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400cd800 con 0x7faa400cd020 2024-04-24T06:34:12.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.032+0000 7faa3fe00640 1 --2- 172.21.15.59:0/1567533698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa400b3900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7faa400702d0 tx=0x7faa2c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e737a4d14f7056b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.033+0000 7faa3f400640 1 -- 172.21.15.59:0/1567533698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c037070 con 0x7faa400cd020 2024-04-24T06:34:12.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.033+0000 7faa3f400640 1 -- 172.21.15.59:0/1567533698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa2c032440 con 0x7faa400cd020 2024-04-24T06:34:12.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.033+0000 7faa3f400640 1 -- 172.21.15.59:0/1567533698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c032810 con 0x7faa400cd020 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.033+0000 7faa47c00640 1 -- 172.21.15.59:0/1567533698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 msgr2=0x7faa400b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.033+0000 7faa47c00640 1 --2- 172.21.15.59:0/1567533698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa400b3900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7faa400702d0 tx=0x7faa2c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 -- 172.21.15.59:0/1567533698 shutdown_connections 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 --2- 172.21.15.59:0/1567533698 >> v2:172.21.15.66:3300/0 conn(0x7faa400b3e40 0x7faa40160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 --2- 172.21.15.59:0/1567533698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa400b3900 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 -- 172.21.15.59:0/1567533698 >> 172.21.15.59:0/1567533698 conn(0x7faa40067ea0 msgr2=0x7faa4006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 -- 172.21.15.59:0/1567533698 shutdown_connections 2024-04-24T06:34:12.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.034+0000 7faa47c00640 1 -- 172.21.15.59:0/1567533698 wait complete. 2024-04-24T06:34:12.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.035+0000 7faa47c00640 1 Processor -- start 2024-04-24T06:34:12.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.035+0000 7faa47c00640 1 -- start start 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa400b3e40 0x7faa4013e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 0x7faa4015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa40160e50 con 0x7faa400cd020 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa401603f0 con 0x7faa401454e0 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.036+0000 7faa47c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa40160fd0 con 0x7faa400b3e40 2024-04-24T06:34:12.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58650/0 (socket says 172.21.15.59:58650) 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 -- 172.21.15.59:0/3044382917 learned_addr learned my addr 172.21.15.59:0/3044382917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa45c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 0x7faa4015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa45200640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa400b3e40 0x7faa4013e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 -- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa400b3e40 msgr2=0x7faa4013e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa400b3e40 0x7faa4013e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 -- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 msgr2=0x7faa4015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 0x7faa4015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa3fe00640 1 -- 172.21.15.59:0/3044382917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa401402d0 con 0x7faa400cd020 2024-04-24T06:34:12.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.037+0000 7faa45c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 0x7faa4015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:12.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa3fe00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7faa2c000c00 tx=0x7faa2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c041020 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa2c030970 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c037040 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa401405b0 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.038+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa40140b70 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.039+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa40068a80 con 0x7faa400cd020 2024-04-24T06:34:12.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.040+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa2c044020 con 0x7faa400cd020 2024-04-24T06:34:12.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.041+0000 7faa3cc00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.041+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7faa2c0ac5b0 con 0x7faa400cd020 2024-04-24T06:34:12.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.042+0000 7faa45200640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.042+0000 7faa45200640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.046+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa2c04b030 con 0x7faa400cd020 2024-04-24T06:34:12.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.290+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa40160030 con 0x7faa400cd020 2024-04-24T06:34:12.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.291+0000 7faa3cc00640 1 -- 172.21.15.59:0/3044382917 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5176 (secure 0 0 0) 0x7faa2c07a970 con 0x7faa400cd020 2024-04-24T06:34:12.293 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:12.293 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:34:10.995637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":11,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:34:12.293 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 50 2024-04-24T06:34:12.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 msgr2=0x7faa2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30008040 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 msgr2=0x7faa401432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7faa2c000c00 tx=0x7faa2c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 shutdown_connections 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa401454e0 0x7faa4015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa400cd020 0x7faa401432c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 --2- 172.21.15.59:0/3044382917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa400b3e40 0x7faa4013e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.294+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 >> 172.21.15.59:0/3044382917 conn(0x7faa40067ea0 msgr2=0x7faa40145bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.295+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 shutdown_connections 2024-04-24T06:34:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.295+0000 7faa47c00640 1 -- 172.21.15.59:0/3044382917 wait complete. 2024-04-24T06:34:12.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:33:44.703546+0000', 'modified': '2024-04-24T06:34:10.995637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.59:6837/209407257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 209407257}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 209407257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6759, 'qdb_cluster': [6759]} 2024-04-24T06:34:12.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:34:12.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T06:34:12.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.942+0000 7f8905c00640 1 Processor -- start 2024-04-24T06:34:12.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.942+0000 7f8905c00640 1 -- start start 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f8905c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f89000b2900 0x7f89000cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f8905c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89000b0f90 con 0x7f890015fbd0 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f8905c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89000b1270 con 0x7f890006b9d0 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f8905c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f89000b1550 con 0x7f89000b2900 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f88ffe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f890015fbd0 0x7f89000b21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60926/0 (socket says 172.21.15.59:60926) 2024-04-24T06:34:12.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.944+0000 7f88ffe00640 1 -- 172.21.15.59:0/2360568904 learned_addr learned my addr 172.21.15.59:0/2360568904 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:12.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> v2:172.21.15.66:3300/0 conn(0x7f89000b2900 0x7f89000cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343599757 0 0) 0x7f89000b0f90 con 0x7f890015fbd0 2024-04-24T06:34:12.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88e4003520 con 0x7f890015fbd0 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191133841 0 0) 0x7f88e4003520 con 0x7f890015fbd0 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 >> v2:172.21.15.66:3300/0 conn(0x7f89000b2900 msgr2=0x7f89000cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 --2- 172.21.15.59:0/2360568904 >> v2:172.21.15.66:3300/0 conn(0x7f89000b2900 0x7f89000cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 >> v1:172.21.15.66:6789/0 conn(0x7f890006b9d0 legacy=0x7f89000cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89000b1830 con 0x7f890015fbd0 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f88f0001670 con 0x7f890015fbd0 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 >> v1:172.21.15.59:6789/0 conn(0x7f890015fbd0 legacy=0x7f89000b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e40039b0 0x7f88e4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 0x7f88e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88e40051c0 0x7f88e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89000b1550 con 0x7f88e4004800 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88e400c0f0 con 0x7f88e40039b0 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f88e400c2f0 con 0x7f88e40051c0 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.945+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> v2:172.21.15.66:3300/0 conn(0x7f89000b2900 0x7f89000cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 0x7f88e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e40039b0 0x7f88e4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 -- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88e40051c0 msgr2=0x7f88e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f8905200640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88e40051c0 0x7f88e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f88e40051c0 0x7f88e400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 -- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e40039b0 msgr2=0x7f88e4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e40039b0 0x7f88e4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f88f7e00640 1 -- 172.21.15.59:0/2360568904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e4003520 con 0x7f88e4004800 2024-04-24T06:34:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.946+0000 7f8905200640 1 -- 172.21.15.59:0/2360568904 reap_dead start 2024-04-24T06:34:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.947+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 0x7f88e4004c80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f88e8006fd0 tx=0x7f88e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faea9bd315921927 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.947+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8037070 con 0x7f88e4004800 2024-04-24T06:34:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e802ac50 con 0x7f88e4004800 2024-04-24T06:34:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f88ff400640 1 -- 172.21.15.59:0/2360568904 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e800f040 con 0x7f88e4004800 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 -- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 msgr2=0x7f88e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 0x7f88e4004c80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f88e8006fd0 tx=0x7f88e4005830 comp rx=0 tx=0).stop 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 -- 172.21.15.59:0/2360568904 shutdown_connections 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 --2- 172.21.15.59:0/2360568904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e4004800 0x7f88e4004c80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 -- 172.21.15.59:0/2360568904 >> 172.21.15.59:0/2360568904 conn(0x7f89000d12c0 msgr2=0x7f89000d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 -- 172.21.15.59:0/2360568904 shutdown_connections 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.948+0000 7f8905c00640 1 -- 172.21.15.59:0/2360568904 wait complete. 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.950+0000 7f8905c00640 1 Processor -- start 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.950+0000 7f8905c00640 1 -- start start 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e4004800 0x7f89000c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89001f48e0 0x7f89001f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89000b0c20 con 0x7f89001f4460 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f890015ffb0 con 0x7f88e4004800 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f8905c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f89000c4950 con 0x7f89001f48e0 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.951+0000 7f88f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58658/0 (socket says 172.21.15.59:58658) 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 -- 172.21.15.59:0/2107238105 learned_addr learned my addr 172.21.15.59:0/2107238105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e4004800 0x7f89000c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f8905200640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89001f48e0 0x7f89001f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 -- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89001f48e0 msgr2=0x7f89001f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89001f48e0 0x7f89001f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 -- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e4004800 msgr2=0x7f89000c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e4004800 0x7f89000c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 -- 172.21.15.59:0/2107238105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89000ca9e0 con 0x7f89001f4460 2024-04-24T06:34:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.952+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009990 tx=0x7f88e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.953+0000 7f88fcc00640 1 -- 172.21.15.59:0/2107238105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8040020 con 0x7f89001f4460 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.953+0000 7f88fcc00640 1 -- 172.21.15.59:0/2107238105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e8038040 con 0x7f89001f4460 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.953+0000 7f88fcc00640 1 -- 172.21.15.59:0/2107238105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8037070 con 0x7f89001f4460 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.953+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89000cacc0 con 0x7f89001f4460 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.953+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89000b66e0 con 0x7f89001f4460 2024-04-24T06:34:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.955+0000 7f88fcc00640 1 -- 172.21.15.59:0/2107238105 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88e8043020 con 0x7f89001f4460 2024-04-24T06:34:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.957+0000 7f88fcc00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 0x7f88dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.957+0000 7f88fcc00640 1 -- 172.21.15.59:0/2107238105 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f88e80abab0 con 0x7f89001f4460 2024-04-24T06:34:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.957+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 0x7f88dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.958+0000 7f890a02b740 1 Processor -- start 2024-04-24T06:34:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.958+0000 7f890a02b740 1 -- start start 2024-04-24T06:34:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.958+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 0x7f88dc06ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88f0001eb0 tx=0x7f88f0001c50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f5554ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554f360 0x556f55524170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f55489b10 con 0x556f5556c980 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556f554302b0 con 0x556f5554faa0 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556f55485ad0 con 0x556f5554f360 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58666/0 (socket says 172.21.15.59:58666) 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 -- 172.21.15.59:0/3619188639 learned_addr learned my addr 172.21.15.59:0/3619188639 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 -- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554f360 msgr2=0x556f55524170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554f360 0x556f55524170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 -- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 msgr2=0x556f5554ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88f7e00640 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f5554ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f5554ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.959+0000 7f88ffe00640 1 -- 172.21.15.59:0/3619188639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5519c2d0 con 0x556f5556c980 2024-04-24T06:34:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.960+0000 7f88f7e00640 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f5554ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.960+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f88f000f9e0 tx=0x7f88f0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2432e3a30132de8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.960+0000 7f88f4c00640 1 -- 172.21.15.59:0/3619188639 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0010040 con 0x556f5556c980 2024-04-24T06:34:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.960+0000 7f88f4c00640 1 -- 172.21.15.59:0/3619188639 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f00169d0 con 0x556f5556c980 2024-04-24T06:34:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.960+0000 7f88f4c00640 1 -- 172.21.15.59:0/3619188639 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f000fea0 con 0x556f5556c980 2024-04-24T06:34:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 -- 172.21.15.59:0/3619188639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 msgr2=0x556f5551f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f88f000f9e0 tx=0x7f88f0012000 comp rx=0 tx=0).stop 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 -- 172.21.15.59:0/3619188639 shutdown_connections 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554f360 0x556f55524170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f5554ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 --2- 172.21.15.59:0/3619188639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5551f310 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 -- 172.21.15.59:0/3619188639 >> 172.21.15.59:0/3619188639 conn(0x556f55521580 msgr2=0x556f55569e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 -- 172.21.15.59:0/3619188639 shutdown_connections 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.961+0000 7f890a02b740 1 -- 172.21.15.59:0/3619188639 wait complete. 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.962+0000 7f890a02b740 1 -- start start 2024-04-24T06:34:12.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.962+0000 7f890a02b740 1 Processor -- start 2024-04-24T06:34:12.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f55568b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f55523730 con 0x556f5554faa0 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556f55443590 con 0x556f5556c980 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556f55522b20 con 0x556f55569e20 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f890a02b740 10 client.0 _ll_register_callbacks cb 0x556f554263a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58668/0 (socket says 172.21.15.59:58668) 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.963+0000 7f88ffe00640 1 -- 172.21.15.59:0/2410693996 learned_addr learned my addr 172.21.15.59:0/2410693996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 -- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 msgr2=0x556f5556a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f55568b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 -- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 msgr2=0x556f55568b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f55568b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 -- 172.21.15.59:0/2410693996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f552666b0 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ffe00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f88f0001cc0 tx=0x7f89000b1b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5978691a8ef92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ed600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0014430 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f88f002db30 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.964+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f00169d0 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f88f0034050 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 10 client.7062 fetch_fsmap learned FSMap version 50 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x556f555529b0 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f00169d0 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f88f0037020 con 0x556f5554faa0 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 10 client.7062 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 10 client.7062 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 20 client.7062 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f55586030 0x556f55588420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 4 client.7062 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:12.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.965+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f55497530 con 0x556f55586030 2024-04-24T06:34:12.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.966+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f55586030 0x556f55588420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.966+0000 7f88f7e00640 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f55586030 0x556f55588420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa074d7962c44d38 server_cookie=2066d28d7b77a84a in_seq=0 out_seq=0 2024-04-24T06:34:12.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.966+0000 7f88ed600640 10 client.7062 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f88ed600640 1 -- 172.21.15.59:0/2410693996 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f55497530 con 0x556f55586030 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f88ed600640 10 client.7062 handle_command_reply: tid=0 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 client.7062 shutdown 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 msgr2=0x556f5554a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f88f0001cc0 tx=0x7f89000b1b10 comp rx=0 tx=0).stop 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 shutdown_connections 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f55586030 0x556f55588420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f55568b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 --2- 172.21.15.59:0/2410693996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5554faa0 0x556f5554a6e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 >> 172.21.15.59:0/2410693996 conn(0x556f5554f360 msgr2=0x556f5554c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.967+0000 7f88ed600640 0 client.7062 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:12.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.968+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 shutdown_connections 2024-04-24T06:34:12.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.968+0000 7f890a02b740 1 -- 172.21.15.59:0/2410693996 wait complete. 2024-04-24T06:34:12.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.968+0000 7f890a02b740 20 client.7062 trim_cache size 0 max 16384 2024-04-24T06:34:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.988+0000 7f890a02b740 1 Processor -- start 2024-04-24T06:34:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.988+0000 7f890a02b740 1 -- start start 2024-04-24T06:34:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554faa0 0x556f55561e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89002005c0 con 0x556f55569e20 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89000b49d0 con 0x556f5556c980 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f89000b5020 con 0x556f5554faa0 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.989+0000 7f88f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58678/0 (socket says 172.21.15.59:58678) 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 -- 172.21.15.59:0/535912273 learned_addr learned my addr 172.21.15.59:0/535912273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88ffe00640 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554faa0 0x556f55561e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f8905200640 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 -- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554faa0 msgr2=0x556f55561e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554faa0 0x556f55561e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 -- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 msgr2=0x556f5556c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 -- 172.21.15.59:0/535912273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5557c5d0 con 0x556f55569e20 2024-04-24T06:34:12.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.990+0000 7f88f7e00640 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f88e8081d90 tx=0x7f88e803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddd1cf207bbfd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f88efe00640 1 -- 172.21.15.59:0/535912273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e80822c0 con 0x556f55569e20 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f88efe00640 1 -- 172.21.15.59:0/535912273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e80776f0 con 0x556f55569e20 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f88efe00640 1 -- 172.21.15.59:0/535912273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8077950 con 0x556f55569e20 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 -- 172.21.15.59:0/535912273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 msgr2=0x556f5556be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f88e8081d90 tx=0x7f88e803e040 comp rx=0 tx=0).stop 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 -- 172.21.15.59:0/535912273 shutdown_connections 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 --2- 172.21.15.59:0/535912273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f5554faa0 0x556f55561e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 -- 172.21.15.59:0/535912273 >> 172.21.15.59:0/535912273 conn(0x556f5554f360 msgr2=0x556f5556ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 -- 172.21.15.59:0/535912273 shutdown_connections 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.991+0000 7f890a02b740 1 -- 172.21.15.59:0/535912273 wait complete. 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.992+0000 7f890a02b740 1 -- start start 2024-04-24T06:34:12.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.992+0000 7f890a02b740 1 Processor -- start 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f555601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556f5557d9f0 con 0x556f5556c980 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556f55551c40 con 0x556f5554faa0 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556f5557e010 con 0x556f55569e20 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f890a02b740 10 client.0 _ll_register_callbacks cb 0x7f88f000f6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.993+0000 7f8905200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58686/0 (socket says 172.21.15.59:58686) 2024-04-24T06:34:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 -- 172.21.15.59:0/3095858910 learned_addr learned my addr 172.21.15.59:0/3095858910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f555601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f88f7e00640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 -- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 msgr2=0x556f5556be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 -- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 msgr2=0x556f555601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f555601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 -- 172.21.15.59:0/3095858910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f55551590 con 0x556f5556c980 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f555601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f8905200640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f89000b0960 tx=0x7f88f80106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304f086968271fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f88ed600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.994+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8018070 con 0x556f5556c980 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f88f8014830 con 0x556f5556c980 2024-04-24T06:34:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e80776f0 con 0x556f5556c980 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f88f8023050 con 0x556f5556c980 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 10 client.7068 fetch_fsmap learned FSMap version 50 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x556f55560760 con 0x556f5556c980 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e80776f0 con 0x556f5556c980 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f88f8026020 con 0x556f5556c980 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 10 client.7068 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 10 client.7068 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 20 client.7068 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f5551e350 0x556f5551e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 4 client.7068 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.995+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "status"}) -- 0x556f554913a0 con 0x556f5551e350 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f5551e350 0x556f5551e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f88ffe00640 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f5551e350 0x556f5551e730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581aa301991e43e7 server_cookie=91d8a5d7057f564b in_seq=0 out_seq=0 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f88ed600640 10 client.7068 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:13.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f88ed600640 1 -- 172.21.15.59:0/3095858910 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556f554913a0 con 0x556f5551e350 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f88ed600640 10 client.7068 handle_command_reply: tid=0 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.996+0000 7f890a02b740 1 client.7068 shutdown 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 msgr2=0x556f5556c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f89000b0960 tx=0x7f88f80106d0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 shutdown_connections 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x556f5551e350 0x556f5551e730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556f5556c980 0x556f5556c350 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556f55569e20 0x556f5556be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 --2- 172.21.15.59:0/3095858910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556f5554faa0 0x556f555601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f88ed600640 0 client.7068 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 >> 172.21.15.59:0/3095858910 conn(0x556f5554f360 msgr2=0x556f55562d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 shutdown_connections 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 1 -- 172.21.15.59:0/3095858910 wait complete. 2024-04-24T06:34:13.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.997+0000 7f890a02b740 20 client.7068 trim_cache size 0 max 16384 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6759, 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0154550230000003, 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 50, 2024-04-24T06:34:13.002 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 70, 2024-04-24T06:34:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 70, 2024-04-24T06:34:13.003 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 33.144732443999999 2024-04-24T06:34:13.003 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 msgr2=0x7f88dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 0x7f88dc06ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88f0001eb0 tx=0x7f88f0001c50 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 msgr2=0x7f89001f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009990 tx=0x7f88e8004730 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 shutdown_connections 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f88dc06c920 0x7f88dc06ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89001f48e0 0x7f89001f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89001f4460 0x7f89001f81e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 --2- 172.21.15.59:0/2107238105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88e4004800 0x7f89000c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 >> 172.21.15.59:0/2107238105 conn(0x7f89000d12c0 msgr2=0x7f89000aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 shutdown_connections 2024-04-24T06:34:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:12.998+0000 7f8905c00640 1 -- 172.21.15.59:0/2107238105 wait complete. 2024-04-24T06:34:13.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:34:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.223+0000 7f2164200640 1 Processor -- start 2024-04-24T06:34:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.223+0000 7f2164200640 1 -- start start 2024-04-24T06:34:13.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.225+0000 7f2164200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f215c1454e0 0x7f215c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.225+0000 7f2164200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0cd020 0x7f215c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.225+0000 7f2164200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c0cd520 con 0x7f215c0b3e40 2024-04-24T06:34:13.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.225+0000 7f2164200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f215c0cddc0 con 0x7f215c1454e0 2024-04-24T06:34:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.225+0000 7f2164200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f215c0cd800 con 0x7f215c0cd020 2024-04-24T06:34:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.226+0000 7f2160e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0cd020 0x7f215c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.226+0000 7f2161800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f215c1454e0 0x7f215c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.226+0000 7f2162200640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f215c0b3e40 0x7f215c160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:38752/0 (socket says 172.21.15.59:38752) 2024-04-24T06:34:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.226+0000 7f2162200640 1 -- 172.21.15.59:0/3272932012 learned_addr learned my addr 172.21.15.59:0/3272932012 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:13.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.227+0000 7f2161800640 1 -- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0cd020 msgr2=0x7f215c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:13.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.227+0000 7f2161800640 1 --2- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0cd020 0x7f215c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.227+0000 7f2161800640 1 -- 172.21.15.59:0/3272932012 >> v1:172.21.15.66:6789/0 conn(0x7f215c0b3e40 legacy=0x7f215c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.227+0000 7f2161800640 1 -- 172.21.15.59:0/3272932012 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0cdae0 con 0x7f215c1454e0 2024-04-24T06:34:13.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.229+0000 7f2161800640 1 --2- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c145bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f21500097a0 tx=0x7f215002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed0bd82d9539160 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:13.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.230+0000 7f215be00640 1 -- 172.21.15.59:0/3272932012 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150037070 con 0x7f215c1454e0 2024-04-24T06:34:13.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.230+0000 7f215be00640 1 -- 172.21.15.59:0/3272932012 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215002ae50 con 0x7f215c1454e0 2024-04-24T06:34:13.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.230+0000 7f215be00640 1 -- 172.21.15.59:0/3272932012 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150032660 con 0x7f215c1454e0 2024-04-24T06:34:13.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.230+0000 7f2164200640 1 -- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 msgr2=0x7f215c145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.230+0000 7f2164200640 1 --2- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c145bb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f21500097a0 tx=0x7f215002a100 comp rx=0 tx=0).stop 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 -- 172.21.15.59:0/3272932012 shutdown_connections 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 --2- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0cd020 0x7f215c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 --2- 172.21.15.59:0/3272932012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c145bb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 -- 172.21.15.59:0/3272932012 >> 172.21.15.59:0/3272932012 conn(0x7f215c067ea0 msgr2=0x7f215c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 -- 172.21.15.59:0/3272932012 shutdown_connections 2024-04-24T06:34:13.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.231+0000 7f2164200640 1 -- 172.21.15.59:0/3272932012 wait complete. 2024-04-24T06:34:13.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.233+0000 7f2164200640 1 Processor -- start 2024-04-24T06:34:13.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.233+0000 7f2164200640 1 -- start start 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0b3e40 0x7f215c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c1f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f215c146270 con 0x7f215c0cd020 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1460f0 con 0x7f215c1454e0 2024-04-24T06:34:13.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2164200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f215c161020 con 0x7f215c0b3e40 2024-04-24T06:34:13.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2160e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2160e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58688/0 (socket says 172.21.15.59:58688) 2024-04-24T06:34:13.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2160e00640 1 -- 172.21.15.59:0/2588916534 learned_addr learned my addr 172.21.15.59:0/2588916534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:13.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2161800640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0b3e40 0x7f215c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.234+0000 7f2162200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c1f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 -- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0b3e40 msgr2=0x7f215c143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0b3e40 0x7f215c143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 -- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 msgr2=0x7f215c1f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c1f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 -- 172.21.15.59:0/2588916534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1f92a0 con 0x7f215c0cd020 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2160e00640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9b0 tx=0x7f214800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148017070 con 0x7f215c0cd020 2024-04-24T06:34:13.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214800c9a0 con 0x7f215c0cd020 2024-04-24T06:34:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c142ca0 con 0x7f215c0cd020 2024-04-24T06:34:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.235+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800cd90 con 0x7f215c0cd020 2024-04-24T06:34:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.236+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c143260 con 0x7f215c0cd020 2024-04-24T06:34:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.237+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c068a80 con 0x7f215c0cd020 2024-04-24T06:34:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.238+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f214801a070 con 0x7f215c0cd020 2024-04-24T06:34:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.239+0000 7f2159600640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 0x7f213c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.239+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f214808bc60 con 0x7f215c0cd020 2024-04-24T06:34:13.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.239+0000 7f2161800640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 0x7f213c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.240+0000 7f2161800640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 0x7f213c06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2150034000 tx=0x7f215002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.245+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f215c1f92a0 con 0x7f215c0cd020 2024-04-24T06:34:13.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.493+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f215c160030 con 0x7f215c0cd020 2024-04-24T06:34:13.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.494+0000 7f2159600640 1 -- 172.21.15.59:0/2588916534 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7f2148059f90 con 0x7f215c0cd020 2024-04-24T06:34:13.495 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:34:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 msgr2=0x7f213c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 0x7f213c06edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2150034000 tx=0x7f215002e000 comp rx=0 tx=0).stop 2024-04-24T06:34:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 msgr2=0x7f215c1f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9b0 tx=0x7f214800ee70 comp rx=0 tx=0).stop 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 shutdown_connections 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213c06c8f0 0x7f213c06edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f215c1454e0 0x7f215c1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f215c0cd020 0x7f215c1f8d60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 --2- 172.21.15.59:0/2588916534 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215c0b3e40 0x7f215c143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 >> 172.21.15.59:0/2588916534 conn(0x7f215c067ea0 msgr2=0x7f215c0aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 shutdown_connections 2024-04-24T06:34:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13.497+0000 7f2164200640 1 -- 172.21.15.59:0/2588916534 wait complete. 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:34:13.517 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:34:13.518 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:34:13.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:13.518 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:34:13.567 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:34:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:34:13.567 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:34:13.568 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:34:13.569 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:34:13.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:13.571 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:34:13.643 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:34:13.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:13.648 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:34:13.702 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:34:13.703 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:34:13.703 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:34:13.703 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:34:13.703 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:34:13.703 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:34:13.703 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:34:13.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:34:13.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:34:13.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:34:13.704 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:34:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:34:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:34:13.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:13.933 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:34:13.982 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:34:13.982 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:34:13.983 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:34:13.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:13.983 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:34:14.063 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:34:14.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:34:14.064 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:14.120 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:34:14.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:34:14.120 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:14.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:34:14.260 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:34:14.317 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:34:14.318 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:34:14.319 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:34:14.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:14.320 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:34:14.373 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:34:14.373 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:34:14.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:34:14.453 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:34:14.495 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:34:14.496 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:34:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:34:14.498 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:34:14.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:14.499 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:34:14.563 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:34:14.562+0000 7f5c1971a480 -1 init, newargv = 0x55b88b6f43b0 newargc=15 2024-04-24T06:34:14.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[58872]: starting ceph client 2024-04-24T06:34:14.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[58872]: starting fuse 2024-04-24T06:34:14.588 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:34:14.588 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:34:14.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:34:14.588 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:34:14.588 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:34:14.589 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:34:14.590 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:34:14.776 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.58872.asok 2024-04-24T06:34:14.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:34:14.793 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58872.asok 2024-04-24T06:34:14.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:14.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58872.asok status 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "58872", 2024-04-24T06:34:15.031 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7080, 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7080 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3499934595 2024-04-24T06:34:15.032 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3499934595 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7080 192.168.0.1:0/3499934595", 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3499934595", 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 50, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 70, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:34:15.033 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:34:15.034 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:15.047 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:34:15.047 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:15.105 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:34:15.105 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:15.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:15.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:15.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:15.195 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-24T06:34:15.260 INFO:teuthology.orchestra.run.smithi059.stdout:file_undamaged 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> import json 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> import sys 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> try: 2024-04-24T06:34:15.262 DEBUG:teuthology.orchestra.run.smithi059:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> except OSError as e: 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> sys.exit(e.errno) 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> print(json.dumps( 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> indent=2)) 2024-04-24T06:34:15.263 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:34:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-24T06:34:15.432 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-24T06:34:15.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:15.433 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-24T06:34:15.537 INFO:teuthology.orchestra.run.smithi059.stdout:2 2024-04-24T06:34:15.537 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:34:15.537 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:34:15.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:34:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.797+0000 7efedf200640 1 Processor -- start 2024-04-24T06:34:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.797+0000 7efedf200640 1 -- start start 2024-04-24T06:34:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.798+0000 7efedf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efee015b450 0x7efee00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.798+0000 7efedf200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efee00cd760 0x7efee00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.798+0000 7efedf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee00b3870 con 0x7efee006f3f0 2024-04-24T06:34:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.798+0000 7efedf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efee00b41a0 con 0x7efee015b450 2024-04-24T06:34:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.798+0000 7efedf200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efee00b3b50 con 0x7efee00cd760 2024-04-24T06:34:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efee015b450 0x7efee00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedd400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efee006f3f0 0x7efee00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60938/0 (socket says 172.21.15.59:60938) 2024-04-24T06:34:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efee015b450 0x7efee00b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58694/0 (socket says 172.21.15.59:58694) 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedd400640 1 -- 172.21.15.59:0/2141655964 learned_addr learned my addr 172.21.15.59:0/2141655964 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efede800640 1 --2- 172.21.15.59:0/2141655964 >> v2:172.21.15.66:3300/0 conn(0x7efee00cd760 0x7efee00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 -- 172.21.15.59:0/2141655964 >> v2:172.21.15.66:3300/0 conn(0x7efee00cd760 msgr2=0x7efee00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 --2- 172.21.15.59:0/2141655964 >> v2:172.21.15.66:3300/0 conn(0x7efee00cd760 0x7efee00ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 -- 172.21.15.59:0/2141655964 >> v1:172.21.15.59:6789/0 conn(0x7efee006f3f0 legacy=0x7efee00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:15.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.799+0000 7efedde00640 1 -- 172.21.15.59:0/2141655964 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee00b3e30 con 0x7efee015b450 2024-04-24T06:34:15.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.800+0000 7efedde00640 1 --2- 172.21.15.59:0/2141655964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b4630 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efee00af020 tx=0x7efed002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb0c780ab343297 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:15.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.801+0000 7efedca00640 1 -- 172.21.15.59:0/2141655964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0037070 con 0x7efee015b450 2024-04-24T06:34:15.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.801+0000 7efedca00640 1 -- 172.21.15.59:0/2141655964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed0032440 con 0x7efee015b450 2024-04-24T06:34:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.801+0000 7efedca00640 1 -- 172.21.15.59:0/2141655964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0032810 con 0x7efee015b450 2024-04-24T06:34:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.801+0000 7efedf200640 1 -- 172.21.15.59:0/2141655964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 msgr2=0x7efee00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.801+0000 7efedf200640 1 --2- 172.21.15.59:0/2141655964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b4630 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efee00af020 tx=0x7efed002a250 comp rx=0 tx=0).stop 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 -- 172.21.15.59:0/2141655964 shutdown_connections 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 --2- 172.21.15.59:0/2141655964 >> v2:172.21.15.66:3300/0 conn(0x7efee00cd760 0x7efee00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 --2- 172.21.15.59:0/2141655964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b4630 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 -- 172.21.15.59:0/2141655964 >> 172.21.15.59:0/2141655964 conn(0x7efee00b21a0 msgr2=0x7efee00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 -- 172.21.15.59:0/2141655964 shutdown_connections 2024-04-24T06:34:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.802+0000 7efedf200640 1 -- 172.21.15.59:0/2141655964 wait complete. 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.803+0000 7efedf200640 1 Processor -- start 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.804+0000 7efedf200640 1 -- start start 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.804+0000 7efedf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efee006f3f0 0x7efee00927c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.804+0000 7efedf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee00cd760 0x7efee00b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.804+0000 7efedf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efedf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efee00b4ff0 con 0x7efee015b450 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efedf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efee015b830 con 0x7efee00cd760 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efedf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efee00b3080 con 0x7efee006f3f0 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58704/0 (socket says 172.21.15.59:58704) 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 -- 172.21.15.59:0/376888864 learned_addr learned my addr 172.21.15.59:0/376888864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 -- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efee006f3f0 msgr2=0x7efee00927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efee006f3f0 0x7efee00927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 -- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee00cd760 msgr2=0x7efee00b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee00cd760 0x7efee00b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.805+0000 7efede800640 1 -- 172.21.15.59:0/376888864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01f4520 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efede800640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efed8007e40 tx=0x7efed800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8014620 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed8019070 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8014a20 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee00c3e10 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.806+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee00c43d0 con 0x7efee015b450 2024-04-24T06:34:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.807+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee004fd90 con 0x7efee015b450 2024-04-24T06:34:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.808+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efed8014bc0 con 0x7efee015b450 2024-04-24T06:34:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.809+0000 7efed6000640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.810+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7efed808c4d0 con 0x7efee015b450 2024-04-24T06:34:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.810+0000 7efedde00640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.811+0000 7efedde00640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 0x7efebc06edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:15.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:15.815+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efed805a890 con 0x7efee015b450 2024-04-24T06:34:16.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.025+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efee01592b0 con 0x7efee015b450 2024-04-24T06:34:16.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.028+0000 7efed6000640 1 -- 172.21.15.59:0/376888864 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16713 (secure 0 0 0) 0x7efed805a190 con 0x7efee015b450 2024-04-24T06:34:16.029 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:16.029 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:06.952943+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:34:16.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 msgr2=0x7efebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 0x7efebc06edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efed0034000 tx=0x7efed002c000 comp rx=0 tx=0).stop 2024-04-24T06:34:16.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 msgr2=0x7efee00b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efed8007e40 tx=0x7efed800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 shutdown_connections 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee015b450 0x7efee00b0820 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee00cd760 0x7efee00b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 --2- 172.21.15.59:0/376888864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efee006f3f0 0x7efee00927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 >> 172.21.15.59:0/376888864 conn(0x7efee00b21a0 msgr2=0x7efee00aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.031+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 shutdown_connections 2024-04-24T06:34:16.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.032+0000 7efedf200640 1 -- 172.21.15.59:0/376888864 wait complete. 2024-04-24T06:34:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:34:16.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:34:16.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.264+0000 7fdb57200640 1 Processor -- start 2024-04-24T06:34:16.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.265+0000 7fdb57200640 1 -- start start 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb57200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb58162c80 0x7fdb5806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb57200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb58166760 0x7fdb580b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb57200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5806ec00 con 0x7fdb5806fca0 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb57200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdb58163060 con 0x7fdb58162c80 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb57200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdb5806eee0 con 0x7fdb58166760 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb56800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdb5806fca0 0x7fdb580b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60952/0 (socket says 172.21.15.59:60952) 2024-04-24T06:34:16.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb56800640 1 -- 172.21.15.59:0/2294012415 learned_addr learned my addr 172.21.15.59:0/2294012415 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb55400640 1 --2- 172.21.15.59:0/2294012415 >> v2:172.21.15.66:3300/0 conn(0x7fdb58166760 0x7fdb580b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709010719 0 0) 0x7fdb5806ec00 con 0x7fdb5806fca0 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb3c003520 con 0x7fdb5806fca0 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.267+0000 7fdb55e00640 1 --2- 172.21.15.59:0/2294012415 >> v2:172.21.15.59:3300/0 conn(0x7fdb58162c80 0x7fdb5806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289058825 0 0) 0x7fdb3c003520 con 0x7fdb5806fca0 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 >> v2:172.21.15.66:3300/0 conn(0x7fdb58166760 msgr2=0x7fdb580b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 --2- 172.21.15.59:0/2294012415 >> v2:172.21.15.66:3300/0 conn(0x7fdb58166760 0x7fdb580b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 >> v2:172.21.15.59:3300/0 conn(0x7fdb58162c80 msgr2=0x7fdb5806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 --2- 172.21.15.59:0/2294012415 >> v2:172.21.15.59:3300/0 conn(0x7fdb58162c80 0x7fdb5806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5806f1c0 con 0x7fdb5806fca0 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fdb500019f0 con 0x7fdb5806fca0 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 >> v1:172.21.15.59:6789/0 conn(0x7fdb5806fca0 legacy=0x7fdb580b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 0x7fdb3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c004800 0x7fdb3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3c0051c0 0x7fdb3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58163060 con 0x7fdb3c0039b0 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5806eee0 con 0x7fdb3c0051c0 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.268+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c00c000 con 0x7fdb3c004800 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 0x7fdb3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55400640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c004800 0x7fdb3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb56800640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3c0051c0 0x7fdb3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 -- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c004800 msgr2=0x7fdb3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c004800 0x7fdb3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 -- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3c0051c0 msgr2=0x7fdb3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb3c0051c0 0x7fdb3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb56800640 1 -- 172.21.15.59:0/2294012415 reap_dead start 2024-04-24T06:34:16.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.269+0000 7fdb55e00640 1 -- 172.21.15.59:0/2294012415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c003520 con 0x7fdb3c0039b0 2024-04-24T06:34:16.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.271+0000 7fdb55e00640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 0x7fdb3c003dd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ba30 tx=0x7fdb3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9dd9fa362e4d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.271+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48030070 con 0x7fdb3c0039b0 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.271+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4802ad10 con 0x7fdb3c0039b0 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.271+0000 7fdb54a00640 1 -- 172.21.15.59:0/2294012415 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800b600 con 0x7fdb3c0039b0 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 -- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 msgr2=0x7fdb3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 0x7fdb3c003dd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ba30 tx=0x7fdb3c005830 comp rx=0 tx=0).stop 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 -- 172.21.15.59:0/2294012415 shutdown_connections 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 --2- 172.21.15.59:0/2294012415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb3c0039b0 0x7fdb3c003dd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 -- 172.21.15.59:0/2294012415 >> 172.21.15.59:0/2294012415 conn(0x7fdb580733e0 msgr2=0x7fdb5806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 -- 172.21.15.59:0/2294012415 shutdown_connections 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.272+0000 7fdb57200640 1 -- 172.21.15.59:0/2294012415 wait complete. 2024-04-24T06:34:16.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.274+0000 7fdb57200640 1 Processor -- start 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.274+0000 7fdb57200640 1 -- start start 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c0039b0 0x7fdb5815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815f5c0 0x7fdb5815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb580b5b80 con 0x7fdb58156e40 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5806f780 con 0x7fdb5815f5c0 2024-04-24T06:34:16.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.275+0000 7fdb57200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5806e570 con 0x7fdb3c0039b0 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58742/0 (socket says 172.21.15.59:58742) 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 -- 172.21.15.59:0/3626381148 learned_addr learned my addr 172.21.15.59:0/3626381148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55e00640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c0039b0 0x7fdb5815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb56800640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815f5c0 0x7fdb5815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 -- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c0039b0 msgr2=0x7fdb5815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c0039b0 0x7fdb5815a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 -- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815f5c0 msgr2=0x7fdb5815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815f5c0 0x7fdb5815fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.276+0000 7fdb55400640 1 -- 172.21.15.59:0/3626381148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb58159410 con 0x7fdb58156e40 2024-04-24T06:34:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb55400640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fdb44011700 tx=0x7fdb44011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400f9c0 con 0x7fdb58156e40 2024-04-24T06:34:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4400fb60 con 0x7fdb58156e40 2024-04-24T06:34:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb581596f0 con 0x7fdb58156e40 2024-04-24T06:34:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44015460 con 0x7fdb58156e40 2024-04-24T06:34:16.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.277+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb58159cb0 con 0x7fdb58156e40 2024-04-24T06:34:16.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.278+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5804fd90 con 0x7fdb58156e40 2024-04-24T06:34:16.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.279+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb44015600 con 0x7fdb58156e40 2024-04-24T06:34:16.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.280+0000 7fdb4e000640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 0x7fdb3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.280+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fdb4408c690 con 0x7fdb58156e40 2024-04-24T06:34:16.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.281+0000 7fdb55e00640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 0x7fdb3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.281+0000 7fdb55e00640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 0x7fdb3806edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ba00 tx=0x7fdb48030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.285+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb44019030 con 0x7fdb58156e40 2024-04-24T06:34:16.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.487+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb58067b50 con 0x7fdb58156e40 2024-04-24T06:34:16.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.488+0000 7fdb4e000640 1 -- 172.21.15.59:0/3626381148 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fdb4405a9c0 con 0x7fdb58156e40 2024-04-24T06:34:16.489 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:34:16.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 msgr2=0x7fdb3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 0x7fdb3806edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ba00 tx=0x7fdb48030000 comp rx=0 tx=0).stop 2024-04-24T06:34:16.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 msgr2=0x7fdb58165e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fdb44011700 tx=0x7fdb44011bc0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 shutdown_connections 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb3806c8f0 0x7fdb3806edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb5815f5c0 0x7fdb5815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb58156e40 0x7fdb58165e80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 --2- 172.21.15.59:0/3626381148 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb3c0039b0 0x7fdb5815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.490+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 >> 172.21.15.59:0/3626381148 conn(0x7fdb580733e0 msgr2=0x7fdb580b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.491+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 shutdown_connections 2024-04-24T06:34:16.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.491+0000 7fdb57200640 1 -- 172.21.15.59:0/3626381148 wait complete. 2024-04-24T06:34:16.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:34:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.727+0000 7f61e7200640 1 Processor -- start 2024-04-24T06:34:16.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.727+0000 7f61e7200640 1 -- start start 2024-04-24T06:34:16.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.729+0000 7f61e7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61e81567e0 0x7f61e8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.729+0000 7f61e7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e815cab0 con 0x7f61e80d0d10 2024-04-24T06:34:16.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.729+0000 7f61e7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e8160e40 con 0x7f61e815d810 2024-04-24T06:34:16.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.729+0000 7f61e7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f61e815cd30 con 0x7f61e81567e0 2024-04-24T06:34:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e6800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f61e80d0d10 0x7f61e8160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60968/0 (socket says 172.21.15.59:60968) 2024-04-24T06:34:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e6800640 1 -- 172.21.15.59:0/2187530979 learned_addr learned my addr 172.21.15.59:0/2187530979 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5e00640 1 --2- 172.21.15.59:0/2187530979 >> v2:172.21.15.66:3300/0 conn(0x7f61e81567e0 0x7f61e8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739893245 0 0) 0x7f61e0001490 con 0x7f61e80d0d10 2024-04-24T06:34:16.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61cc003520 con 0x7f61e80d0d10 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094787119 0 0) 0x7f61cc003520 con 0x7f61e80d0d10 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 >> v2:172.21.15.66:3300/0 conn(0x7f61e81567e0 msgr2=0x7f61e8156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 --2- 172.21.15.59:0/2187530979 >> v2:172.21.15.66:3300/0 conn(0x7f61e81567e0 0x7f61e8156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 >> v1:172.21.15.66:6789/0 conn(0x7f61e815d810 legacy=0x7f61e80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.730+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e815d010 con 0x7f61e80d0d10 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f61e0001bc0 con 0x7f61e80d0d10 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 >> v1:172.21.15.59:6789/0 conn(0x7f61e80d0d10 legacy=0x7f61e8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc004800 0x7f61cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61cc0051c0 0x7f61cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61e815cd30 con 0x7f61cc003a00 2024-04-24T06:34:16.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc00c0c0 con 0x7f61cc004800 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc00c2c0 con 0x7f61cc0051c0 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 -- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61cc0051c0 msgr2=0x7f61cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61cc0051c0 0x7f61cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 -- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc004800 msgr2=0x7f61cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61cc004800 0x7f61cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e5e00640 1 -- 172.21.15.59:0/2187530979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc003520 con 0x7f61cc003a00 2024-04-24T06:34:16.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.731+0000 7f61e6800640 1 -- 172.21.15.59:0/2187530979 reap_dead start 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.732+0000 7f61e5e00640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61cc003e20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009ac0 tx=0x7f61cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aaa29cbe895f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8036070 con 0x7f61cc003a00 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8031400 con 0x7f61cc003a00 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e5400640 1 -- 172.21.15.59:0/2187530979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8009690 con 0x7f61cc003a00 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 -- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 msgr2=0x7f61cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61cc003e20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009ac0 tx=0x7f61cc005830 comp rx=0 tx=0).stop 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 -- 172.21.15.59:0/2187530979 shutdown_connections 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 --2- 172.21.15.59:0/2187530979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61cc003e20 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 -- 172.21.15.59:0/2187530979 >> 172.21.15.59:0/2187530979 conn(0x7f61e806bdf0 msgr2=0x7f61e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 -- 172.21.15.59:0/2187530979 shutdown_connections 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.733+0000 7f61e7200640 1 -- 172.21.15.59:0/2187530979 wait complete. 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.735+0000 7f61e7200640 1 Processor -- start 2024-04-24T06:34:16.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.735+0000 7f61e7200640 1 -- start start 2024-04-24T06:34:16.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61e8099820 0x7f61e81e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61e81e63d0 0x7f61e8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8156d40 con 0x7f61cc003a00 2024-04-24T06:34:16.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8156bc0 con 0x7f61e8099820 2024-04-24T06:34:16.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.736+0000 7f61e7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f61e8156ec0 con 0x7f61e81e63d0 2024-04-24T06:34:16.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58758/0 (socket says 172.21.15.59:58758) 2024-04-24T06:34:16.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 -- 172.21.15.59:0/3576160565 learned_addr learned my addr 172.21.15.59:0/3576160565 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61dfe00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61e8099820 0x7f61e81e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 -- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61e81e63d0 msgr2=0x7f61e8159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61e81e63d0 0x7f61e8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 -- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61e8099820 msgr2=0x7f61e81e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61e8099820 0x7f61e81e5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 -- 172.21.15.59:0/3576160565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e806e7f0 con 0x7f61cc003a00 2024-04-24T06:34:16.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.737+0000 7f61e5e00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f61d8034040 tx=0x7f61d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.738+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d803f020 con 0x7f61cc003a00 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.738+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8037040 con 0x7f61cc003a00 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.738+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8036070 con 0x7f61cc003a00 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.738+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e8099ca0 con 0x7f61cc003a00 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.738+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e806ed70 con 0x7f61cc003a00 2024-04-24T06:34:16.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.739+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e81e67f0 con 0x7f61cc003a00 2024-04-24T06:34:16.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.740+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61d8042020 con 0x7f61cc003a00 2024-04-24T06:34:16.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.741+0000 7f61de000640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 0x7f61c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:16.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.741+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f61d80abc40 con 0x7f61cc003a00 2024-04-24T06:34:16.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.741+0000 7f61dfe00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 0x7f61c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:16.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.742+0000 7f61dfe00640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 0x7f61c406ede0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f61e815bb40 tx=0x7f61d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:16.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.746+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61d8049030 con 0x7f61cc003a00 2024-04-24T06:34:16.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.951+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61e80b5150 con 0x7f61cc003a00 2024-04-24T06:34:16.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.952+0000 7f61de000640 1 -- 172.21.15.59:0/3576160565 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f61d807a030 con 0x7f61cc003a00 2024-04-24T06:34:16.953 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:34:16.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 msgr2=0x7f61c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 0x7f61c406ede0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f61e815bb40 tx=0x7f61d0009040 comp rx=0 tx=0).stop 2024-04-24T06:34:16.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 msgr2=0x7f61e806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:16.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f61d8034040 tx=0x7f61d8004730 comp rx=0 tx=0).stop 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 shutdown_connections 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61c406c920 0x7f61c406ede0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61e81e63d0 0x7f61e8159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61e8099820 0x7f61e81e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 --2- 172.21.15.59:0/3576160565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc003a00 0x7f61e806f850 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 >> 172.21.15.59:0/3576160565 conn(0x7f61e806bdf0 msgr2=0x7f61e8071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 shutdown_connections 2024-04-24T06:34:16.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:16.954+0000 7f61e7200640 1 -- 172.21.15.59:0/3576160565 wait complete. 2024-04-24T06:34:16.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:34:17.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.183+0000 7f2ce6800640 1 Processor -- start 2024-04-24T06:34:17.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.183+0000 7f2ce6800640 1 -- start start 2024-04-24T06:34:17.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.184+0000 7f2ce6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ce01567e0 0x7f2ce0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce00d0d10 0x7f2ce0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce6800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce015cab0 con 0x7f2ce015d810 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce0160e40 con 0x7f2ce01567e0 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce015cd30 con 0x7f2ce00d0d10 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ce01567e0 0x7f2ce0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ce01567e0 0x7f2ce0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58762/0 (socket says 172.21.15.59:58762) 2024-04-24T06:34:17.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 -- 172.21.15.59:0/3978918705 learned_addr learned my addr 172.21.15.59:0/3978918705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 -- 172.21.15.59:0/3978918705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce00d0d10 msgr2=0x7f2ce0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 --2- 172.21.15.59:0/3978918705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce00d0d10 0x7f2ce0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 -- 172.21.15.59:0/3978918705 >> v1:172.21.15.66:6789/0 conn(0x7f2ce015d810 legacy=0x7f2ce00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.185+0000 7f2ce5400640 1 -- 172.21.15.59:0/3978918705 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce015d010 con 0x7f2ce01567e0 2024-04-24T06:34:17.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2ce5400640 1 --2- 172.21.15.59:0/3978918705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce01567e0 0x7f2ce0156bc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a0c200abbff9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2cdfe00640 1 -- 172.21.15.59:0/3978918705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4037070 con 0x7f2ce01567e0 2024-04-24T06:34:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2cdfe00640 1 -- 172.21.15.59:0/3978918705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd4032440 con 0x7f2ce01567e0 2024-04-24T06:34:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2cdfe00640 1 -- 172.21.15.59:0/3978918705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4032810 con 0x7f2ce01567e0 2024-04-24T06:34:17.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2ce6800640 1 -- 172.21.15.59:0/3978918705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce01567e0 msgr2=0x7f2ce0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.187+0000 7f2ce6800640 1 --2- 172.21.15.59:0/3978918705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce01567e0 0x7f2ce0156bc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd402a210 comp rx=0 tx=0).stop 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 -- 172.21.15.59:0/3978918705 shutdown_connections 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 --2- 172.21.15.59:0/3978918705 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce00d0d10 0x7f2ce0160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 --2- 172.21.15.59:0/3978918705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce01567e0 0x7f2ce0156bc0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 -- 172.21.15.59:0/3978918705 >> 172.21.15.59:0/3978918705 conn(0x7f2ce006bdf0 msgr2=0x7f2ce006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 -- 172.21.15.59:0/3978918705 shutdown_connections 2024-04-24T06:34:17.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.188+0000 7f2ce6800640 1 -- 172.21.15.59:0/3978918705 wait complete. 2024-04-24T06:34:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.189+0000 7f2ce6800640 1 Processor -- start 2024-04-24T06:34:17.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.189+0000 7f2ce6800640 1 -- start start 2024-04-24T06:34:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 0x7f2ce014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce015d810 0x7f2ce0158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01607d0 con 0x7f2ce00d0d10 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0160650 con 0x7f2ce01567e0 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.190+0000 7f2ce6800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce0160950 con 0x7f2ce015d810 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58764/0 (socket says 172.21.15.59:58764) 2024-04-24T06:34:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 -- 172.21.15.59:0/4272814309 learned_addr learned my addr 172.21.15.59:0/4272814309 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce4a00640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 0x7f2ce014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5e00640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce015d810 0x7f2ce0158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 -- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce015d810 msgr2=0x7f2ce0158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce015d810 0x7f2ce0158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 -- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 msgr2=0x7f2ce014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 0x7f2ce014fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce5400640 1 -- 172.21.15.59:0/4272814309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce0150250 con 0x7f2ce00d0d10 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.191+0000 7f2ce4a00640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 0x7f2ce014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2ce5400640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000c00 tx=0x7f2cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4041070 con 0x7f2ce00d0d10 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd403c870 con 0x7f2ce00d0d10 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4037040 con 0x7f2ce00d0d10 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce0159330 con 0x7f2ce00d0d10 2024-04-24T06:34:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.192+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce0151af0 con 0x7f2ce00d0d10 2024-04-24T06:34:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.193+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce004fd90 con 0x7f2ce00d0d10 2024-04-24T06:34:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.194+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cd4044070 con 0x7f2ce00d0d10 2024-04-24T06:34:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.195+0000 7f2cdd600640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 0x7f2cc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.195+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f2cd40ac780 con 0x7f2ce00d0d10 2024-04-24T06:34:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.195+0000 7f2ce4a00640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 0x7f2cc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.196+0000 7f2ce4a00640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 0x7f2cc006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0099230 tx=0x7f2ccc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.200+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ce0150250 con 0x7f2ce00d0d10 2024-04-24T06:34:17.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.398+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ce00b5150 con 0x7f2ce00d0d10 2024-04-24T06:34:17.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.399+0000 7f2cdd600640 1 -- 172.21.15.59:0/4272814309 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f2ce015cab0 con 0x7f2ce00d0d10 2024-04-24T06:34:17.400 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:34:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 msgr2=0x7f2cc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 0x7f2cc006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0099230 tx=0x7f2ccc009040 comp rx=0 tx=0).stop 2024-04-24T06:34:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 msgr2=0x7f2ce0098cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000c00 tx=0x7f2cd40046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 shutdown_connections 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2cc006c920 0x7f2cc006ede0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2ce015d810 0x7f2ce0158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ce01567e0 0x7f2ce014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 --2- 172.21.15.59:0/4272814309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ce00d0d10 0x7f2ce0098cf0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 >> 172.21.15.59:0/4272814309 conn(0x7f2ce006bdf0 msgr2=0x7f2ce0071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.402+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 shutdown_connections 2024-04-24T06:34:17.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.403+0000 7f2ce6800640 1 -- 172.21.15.59:0/4272814309 wait complete. 2024-04-24T06:34:17.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:34:17.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.625+0000 7f6dbb200640 1 Processor -- start 2024-04-24T06:34:17.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.626+0000 7f6dbb200640 1 -- start start 2024-04-24T06:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6dbb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6dbb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6dbc15d420 0x7f6dbc0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6dbb200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dbc06e9c0 con 0x7f6dbc13f750 2024-04-24T06:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6dbb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6dbc13fb30 con 0x7f6dbc15d420 2024-04-24T06:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6dbb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc06eca0 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.628+0000 7f6db9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f6dbc13f750 0x7f6dbc0b5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:38786/0 (socket says 172.21.15.59:38786) 2024-04-24T06:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:48318/0 (socket says 172.21.15.59:48318) 2024-04-24T06:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9e00640 1 -- 172.21.15.59:0/509618184 learned_addr learned my addr 172.21.15.59:0/509618184 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6dba800640 1 --2- 172.21.15.59:0/509618184 >> v2:172.21.15.66:3300/0 conn(0x7f6dbc15d420 0x7f6dbc0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9400640 1 -- 172.21.15.59:0/509618184 >> v2:172.21.15.66:3300/0 conn(0x7f6dbc15d420 msgr2=0x7f6dbc0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9400640 1 --2- 172.21.15.59:0/509618184 >> v2:172.21.15.66:3300/0 conn(0x7f6dbc15d420 0x7f6dbc0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9400640 1 -- 172.21.15.59:0/509618184 >> v1:172.21.15.66:6789/0 conn(0x7f6dbc13f750 legacy=0x7f6dbc0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.629+0000 7f6db9400640 1 -- 172.21.15.59:0/509618184 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc06ef80 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.631+0000 7f6db9400640 1 --2- 172.21.15.59:0/509618184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009d90 tx=0x7f6da402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64854279bf8b12f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.631+0000 7f6db8a00640 1 -- 172.21.15.59:0/509618184 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4007740 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.631+0000 7f6db8a00640 1 -- 172.21.15.59:0/509618184 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da40078e0 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.631+0000 7f6db8a00640 1 -- 172.21.15.59:0/509618184 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4007cb0 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 -- 172.21.15.59:0/509618184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 msgr2=0x7f6dbc15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 --2- 172.21.15.59:0/509618184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009d90 tx=0x7f6da402a660 comp rx=0 tx=0).stop 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 -- 172.21.15.59:0/509618184 shutdown_connections 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 --2- 172.21.15.59:0/509618184 >> v2:172.21.15.66:3300/0 conn(0x7f6dbc15d420 0x7f6dbc0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 --2- 172.21.15.59:0/509618184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc0b65c0 0x7f6dbc15cee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 -- 172.21.15.59:0/509618184 >> 172.21.15.59:0/509618184 conn(0x7f6dbc06b9d0 msgr2=0x7f6dbc06bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.632+0000 7f6dbb200640 1 -- 172.21.15.59:0/509618184 shutdown_connections 2024-04-24T06:34:17.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.633+0000 7f6dbb200640 1 -- 172.21.15.59:0/509618184 wait complete. 2024-04-24T06:34:17.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.635+0000 7f6dbb200640 1 Processor -- start 2024-04-24T06:34:17.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.635+0000 7f6dbb200640 1 -- start start 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.636+0000 7f6dbb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b65c0 0x7f6dbc0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dbb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc13f750 0x7f6dbc155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dbb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dbb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc06f780 con 0x7f6dbc15d420 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dbb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc15d890 con 0x7f6dbc0b65c0 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dbb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc154de0 con 0x7f6dbc13f750 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58770/0 (socket says 172.21.15.59:58770) 2024-04-24T06:34:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dba800640 1 -- 172.21.15.59:0/3362841742 learned_addr learned my addr 172.21.15.59:0/3362841742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6db9e00640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b65c0 0x7f6dbc0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dba800640 1 -- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc13f750 msgr2=0x7f6dbc155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.637+0000 7f6dba800640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc13f750 0x7f6dbc155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dba800640 1 -- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b65c0 msgr2=0x7f6dbc0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dba800640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b65c0 0x7f6dbc0b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dba800640 1 -- 172.21.15.59:0/3362841742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc1541d0 con 0x7f6dbc15d420 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dba800640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e7f0 tx=0x7f6db400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4017070 con 0x7f6dbc15d420 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db400ee40 con 0x7f6dbc15d420 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400cb80 con 0x7f6dbc15d420 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc1544b0 con 0x7f6dbc15d420 2024-04-24T06:34:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.638+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc154a70 con 0x7f6dbc15d420 2024-04-24T06:34:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.639+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc04fd90 con 0x7f6dbc15d420 2024-04-24T06:34:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.640+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6db401a070 con 0x7f6dbc15d420 2024-04-24T06:34:17.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.641+0000 7f6db2000640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 0x7f6d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:17.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.641+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f6db408be90 con 0x7f6dbc15d420 2024-04-24T06:34:17.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.642+0000 7f6db9e00640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 0x7f6d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:17.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.642+0000 7f6db9e00640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 0x7f6d9806edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000f60 tx=0x7f6dac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:17.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.647+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6db405a1c0 con 0x7f6dbc15d420 2024-04-24T06:34:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.863+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6dbc06f900 con 0x7f6dbc15d420 2024-04-24T06:34:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.864+0000 7f6db2000640 1 -- 172.21.15.59:0/3362841742 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7f6db4059ac0 con 0x7f6dbc15d420 2024-04-24T06:34:17.865 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:34:17.865 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:34:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.866+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 msgr2=0x7f6d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 0x7f6d9806edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000f60 tx=0x7f6dac009040 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 msgr2=0x7f6dbc15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e7f0 tx=0x7f6db400ebf0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 shutdown_connections 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d9806c8f0 0x7f6d9806edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dbc15d420 0x7f6dbc15ec40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6dbc13f750 0x7f6dbc155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 --2- 172.21.15.59:0/3362841742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dbc0b65c0 0x7f6dbc0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:17.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 >> 172.21.15.59:0/3362841742 conn(0x7f6dbc06b9d0 msgr2=0x7f6dbc15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 shutdown_connections 2024-04-24T06:34:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:17.867+0000 7f6dbb200640 1 -- 172.21.15.59:0/3362841742 wait complete. 2024-04-24T06:34:17.888 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:34:17.888 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:34:17.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:34:18.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.151+0000 7f46b8a00640 1 Processor -- start 2024-04-24T06:34:18.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.151+0000 7f46b8a00640 1 -- start start 2024-04-24T06:34:18.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.152+0000 7f46b8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f46b40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.152+0000 7f46b8a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b40cf030 con 0x7f46b40d2630 2024-04-24T06:34:18.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.152+0000 7f46b8a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b40b6a40 con 0x7f46b40d0d90 2024-04-24T06:34:18.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.152+0000 7f46b8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46b40b7000 con 0x7f46b415e7c0 2024-04-24T06:34:18.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f46b40d2630 0x7f46b40b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60978/0 (socket says 172.21.15.59:60978) 2024-04-24T06:34:18.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b2a00640 1 -- 172.21.15.59:0/3686679245 learned_addr learned my addr 172.21.15.59:0/3686679245 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:18.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125233001 0 0) 0x7f46a0001110 con 0x7f46b40d2630 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b3400640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f46b40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4698003520 con 0x7f46b40d2630 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.153+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 67823233 0 0) 0x7f4698003520 con 0x7f46b40d2630 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 msgr2=0x7f46b40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f46b40d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 >> v1:172.21.15.66:6789/0 conn(0x7f46b40d0d90 legacy=0x7f46b40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b40b6d20 con 0x7f46b40d2630 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f46a0001840 con 0x7f46b40d2630 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 >> v1:172.21.15.59:6789/0 conn(0x7f46b40d2630 legacy=0x7f46b40b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46980039e0 0x7f4698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f4698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698004d40 0x7f46980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40b7000 con 0x7f46980039e0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f469800baf0 con 0x7f4698004d40 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f469800bcf0 con 0x7f46b415e7c0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.154+0000 7f46b3400640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46980039e0 0x7f4698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b2a00640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f4698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3400640 1 -- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 msgr2=0x7f4698004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3e00640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698004d40 0x7f46980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3400640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f4698004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3400640 1 -- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698004d40 msgr2=0x7f46980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3400640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698004d40 0x7f46980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3400640 1 -- 172.21.15.59:0/3686679245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4698003520 con 0x7f46980039e0 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b2a00640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b415e7c0 0x7f4698004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.155+0000 7f46b3e00640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4698004d40 0x7f46980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.156+0000 7f46b3400640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46980039e0 0x7f4698003de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f46a4007360 tx=0x7f4698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ebceed9723ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.156+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4034020 con 0x7f46980039e0 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.156+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a4030660 con 0x7f46980039e0 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.156+0000 7f46b2000640 1 -- 172.21.15.59:0/3686679245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4009be0 con 0x7f46980039e0 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 -- 172.21.15.59:0/3686679245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46980039e0 msgr2=0x7f4698003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3686679245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46980039e0 0x7f4698003de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f46a4007360 tx=0x7f4698005830 comp rx=0 tx=0).stop 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b3e00640 1 -- 172.21.15.59:0/3686679245 reap_dead start 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 -- 172.21.15.59:0/3686679245 shutdown_connections 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 -- 172.21.15.59:0/3686679245 >> 172.21.15.59:0/3686679245 conn(0x7f46b409d920 msgr2=0x7f46b408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 -- 172.21.15.59:0/3686679245 shutdown_connections 2024-04-24T06:34:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.157+0000 7f46b8a00640 1 -- 172.21.15.59:0/3686679245 wait complete. 2024-04-24T06:34:18.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.159+0000 7f46b8a00640 1 Processor -- start 2024-04-24T06:34:18.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.159+0000 7f46b8a00640 1 -- start start 2024-04-24T06:34:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 0x7f46b41576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b40cdce0 0x7f46b406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40b6700 con 0x7f46b406da40 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40cf620 con 0x7f46980039e0 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46b40d1a80 con 0x7f46b40cdce0 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58790/0 (socket says 172.21.15.59:58790) 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.160+0000 7f46b3e00640 1 -- 172.21.15.59:0/3609282160 learned_addr learned my addr 172.21.15.59:0/3609282160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b2a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b40cdce0 0x7f46b406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3400640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 0x7f46b41576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 -- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b40cdce0 msgr2=0x7f46b406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b40cdce0 0x7f46b406d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 -- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 msgr2=0x7f46b41576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 0x7f46b41576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 -- 172.21.15.59:0/3609282160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b40ccc40 con 0x7f46b406da40 2024-04-24T06:34:18.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3400640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 0x7f46b41576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:18.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b3e00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f46b40cb580 tx=0x7f46ac00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac016070 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac005230 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b40ccf20 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.161+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac007020 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.162+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b40cd4e0 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.163+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b406dec0 con 0x7f46b406da40 2024-04-24T06:34:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.164+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46ac0071c0 con 0x7f46b406da40 2024-04-24T06:34:18.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.165+0000 7f46ab400640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.165+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f46ac08b5a0 con 0x7f46b406da40 2024-04-24T06:34:18.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.165+0000 7f46b3400640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.166+0000 7f46b3400640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f46a4035000 tx=0x7f46a400b030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.170+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46ac0074b0 con 0x7f46b406da40 2024-04-24T06:34:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.377+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46b404fd90 con 0x7f46b406da40 2024-04-24T06:34:18.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.379+0000 7f46ab400640 1 -- 172.21.15.59:0/3609282160 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16713 (secure 0 0 0) 0x7f46ac059960 con 0x7f46b406da40 2024-04-24T06:34:18.380 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:18.381 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:06.952943+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:34:18.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 msgr2=0x7f469406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f46a4035000 tx=0x7f46a400b030 comp rx=0 tx=0).stop 2024-04-24T06:34:18.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 msgr2=0x7f46b41e8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f46b40cb580 tx=0x7f46ac00d890 comp rx=0 tx=0).stop 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 shutdown_connections 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46b406da40 0x7f46b41e8a50 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46b40cdce0 0x7f46b406d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 --2- 172.21.15.59:0/3609282160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46980039e0 0x7f46b41576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 >> 172.21.15.59:0/3609282160 conn(0x7f46b409d920 msgr2=0x7f46b415f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 shutdown_connections 2024-04-24T06:34:18.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.382+0000 7f46b8a00640 1 -- 172.21.15.59:0/3609282160 wait complete. 2024-04-24T06:34:18.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:34:18.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:34:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.620+0000 7f1811200640 1 Processor -- start 2024-04-24T06:34:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.620+0000 7f1811200640 1 -- start start 2024-04-24T06:34:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f180c06dcb0 0x7f180c162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f180c0d09e0 0x7f180c06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f180c072a20 con 0x7f180c0d09e0 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f180c163240 con 0x7f180c06dcb0 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.621+0000 7f1811200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f180c071e10 con 0x7f180c0d0600 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f180c0d09e0 0x7f180c06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f180c0d09e0 0x7f180c06e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58804/0 (socket says 172.21.15.59:58804) 2024-04-24T06:34:18.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 -- 172.21.15.59:0/1782505248 learned_addr learned my addr 172.21.15.59:0/1782505248 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180b400640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180aa00640 1 --2- 172.21.15.59:0/1782505248 >> v2:172.21.15.66:3300/0 conn(0x7f180c06dcb0 0x7f180c162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 -- 172.21.15.59:0/1782505248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 msgr2=0x7f180c0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 -- 172.21.15.59:0/1782505248 >> v2:172.21.15.66:3300/0 conn(0x7f180c06dcb0 msgr2=0x7f180c162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 --2- 172.21.15.59:0/1782505248 >> v2:172.21.15.66:3300/0 conn(0x7f180c06dcb0 0x7f180c162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.622+0000 7f180be00640 1 -- 172.21.15.59:0/1782505248 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c0720f0 con 0x7f180c0d09e0 2024-04-24T06:34:18.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.624+0000 7f180be00640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c0d09e0 0x7f180c06e5c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f180c0b5660 tx=0x7f180402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfb769aa462acb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.624+0000 7f180a000640 1 -- 172.21.15.59:0/1782505248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804036070 con 0x7f180c0d09e0 2024-04-24T06:34:18.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.624+0000 7f180a000640 1 -- 172.21.15.59:0/1782505248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804031600 con 0x7f180c0d09e0 2024-04-24T06:34:18.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.624+0000 7f180a000640 1 -- 172.21.15.59:0/1782505248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040319d0 con 0x7f180c0d09e0 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.624+0000 7f1811200640 1 -- 172.21.15.59:0/1782505248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c0d09e0 msgr2=0x7f180c06e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c0d09e0 0x7f180c06e5c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f180c0b5660 tx=0x7f180402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 -- 172.21.15.59:0/1782505248 shutdown_connections 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c0d09e0 0x7f180c06e5c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 --2- 172.21.15.59:0/1782505248 >> v2:172.21.15.66:3300/0 conn(0x7f180c06dcb0 0x7f180c162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 --2- 172.21.15.59:0/1782505248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 -- 172.21.15.59:0/1782505248 >> 172.21.15.59:0/1782505248 conn(0x7f180c06bdf0 msgr2=0x7f180c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 -- 172.21.15.59:0/1782505248 shutdown_connections 2024-04-24T06:34:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.625+0000 7f1811200640 1 -- 172.21.15.59:0/1782505248 wait complete. 2024-04-24T06:34:18.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.627+0000 7f1811200640 1 Processor -- start 2024-04-24T06:34:18.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.627+0000 7f1811200640 1 -- start start 2024-04-24T06:34:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0d09e0 0x7f180c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f180c15eaf0 con 0x7f180c06dcb0 2024-04-24T06:34:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f180c0726b0 con 0x7f180c0d09e0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f1811200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f180c0d0e70 con 0x7f180c0d0600 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f180b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f180b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58818/0 (socket says 172.21.15.59:58818) 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f180b400640 1 -- 172.21.15.59:0/969268889 learned_addr learned my addr 172.21.15.59:0/969268889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f180aa00640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.628+0000 7f180be00640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0d09e0 0x7f180c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 -- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 msgr2=0x7f180c14f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c14f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 -- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0d09e0 msgr2=0x7f180c0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0d09e0 0x7f180c0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 -- 172.21.15.59:0/969268889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c161810 con 0x7f180c06dcb0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f180b400640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00d800 tx=0x7f17fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00b6c0 con 0x7f180c06dcb0 2024-04-24T06:34:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17fc019070 con 0x7f180c06dcb0 2024-04-24T06:34:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00ba80 con 0x7f180c06dcb0 2024-04-24T06:34:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c15bb20 con 0x7f180c06dcb0 2024-04-24T06:34:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.629+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c14f910 con 0x7f180c06dcb0 2024-04-24T06:34:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.630+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c04fd90 con 0x7f180c06dcb0 2024-04-24T06:34:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.631+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17fc00bc20 con 0x7f180c06dcb0 2024-04-24T06:34:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.632+0000 7f1803400640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 0x7f17ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.633+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f17fc08c070 con 0x7f180c06dcb0 2024-04-24T06:34:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.633+0000 7f180aa00640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 0x7f17ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.634+0000 7f180aa00640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 0x7f17ec06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c98f0 tx=0x7f17f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.637+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17fc011340 con 0x7f180c06dcb0 2024-04-24T06:34:18.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.842+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f180c0cff50 con 0x7f180c06dcb0 2024-04-24T06:34:18.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.843+0000 7f1803400640 1 -- 172.21.15.59:0/969268889 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f17fc05a430 con 0x7f180c06dcb0 2024-04-24T06:34:18.844 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 msgr2=0x7f17ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 0x7f17ec06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c98f0 tx=0x7f17f8008040 comp rx=0 tx=0).stop 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 msgr2=0x7f180c1530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00d800 tx=0x7f17fc00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 shutdown_connections 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f17ec06c8f0 0x7f17ec06edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0d09e0 0x7f180c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f180c0d0600 0x7f180c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 --2- 172.21.15.59:0/969268889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f180c06dcb0 0x7f180c1530f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:18.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 >> 172.21.15.59:0/969268889 conn(0x7f180c06bdf0 msgr2=0x7f180c06f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:18.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 shutdown_connections 2024-04-24T06:34:18.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:18.846+0000 7f1811200640 1 -- 172.21.15.59:0/969268889 wait complete. 2024-04-24T06:34:18.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:34:19.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.086+0000 7f9211c00640 1 Processor -- start 2024-04-24T06:34:19.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.086+0000 7f9211c00640 1 -- start start 2024-04-24T06:34:19.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.087+0000 7f9211c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f920c145e10 0x7f920c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.087+0000 7f9211c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f920c0cfc20 con 0x7f920c071d70 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.087+0000 7f9211c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f920c0cff00 con 0x7f920c0d0cc0 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.087+0000 7f9211c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f920c0d01e0 con 0x7f920c145e10 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f920c145e10 0x7f920c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f920c145e10 0x7f920c06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43920/0 (socket says 172.21.15.59:43920) 2024-04-24T06:34:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 -- 172.21.15.59:0/3956523696 learned_addr learned my addr 172.21.15.59:0/3956523696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 -- 172.21.15.59:0/3956523696 >> v1:172.21.15.66:6789/0 conn(0x7f920c0d0cc0 legacy=0x7f920c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 -- 172.21.15.59:0/3956523696 >> v1:172.21.15.59:6789/0 conn(0x7f920c071d70 legacy=0x7f920c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920aa00640 1 -- 172.21.15.59:0/3956523696 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c0d04c0 con 0x7f920c145e10 2024-04-24T06:34:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.088+0000 7f920a000640 1 -- 172.21.15.59:0/3956523696 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905213652 0 0) 0x7f920c0cfc20 con 0x7f920c071d70 2024-04-24T06:34:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f920aa00640 1 --2- 172.21.15.59:0/3956523696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c145e10 0x7f920c06f130 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0098a0 tx=0x7f91fc02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47266d373b00067a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f920a000640 1 -- 172.21.15.59:0/3956523696 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc037070 con 0x7f920c145e10 2024-04-24T06:34:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f920a000640 1 -- 172.21.15.59:0/3956523696 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91fc032440 con 0x7f920c145e10 2024-04-24T06:34:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f920a000640 1 -- 172.21.15.59:0/3956523696 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc032810 con 0x7f920c145e10 2024-04-24T06:34:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f9211c00640 1 -- 172.21.15.59:0/3956523696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c145e10 msgr2=0x7f920c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.090+0000 7f9211c00640 1 --2- 172.21.15.59:0/3956523696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c145e10 0x7f920c06f130 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0098a0 tx=0x7f91fc02a2e0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.091+0000 7f9211c00640 1 -- 172.21.15.59:0/3956523696 shutdown_connections 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.091+0000 7f9211c00640 1 --2- 172.21.15.59:0/3956523696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c145e10 0x7f920c06f130 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.091+0000 7f9211c00640 1 -- 172.21.15.59:0/3956523696 >> 172.21.15.59:0/3956523696 conn(0x7f920c067d90 msgr2=0x7f920c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.091+0000 7f9211c00640 1 -- 172.21.15.59:0/3956523696 shutdown_connections 2024-04-24T06:34:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.091+0000 7f9211c00640 1 -- 172.21.15.59:0/3956523696 wait complete. 2024-04-24T06:34:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.092+0000 7f9211c00640 1 Processor -- start 2024-04-24T06:34:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 -- start start 2024-04-24T06:34:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f920c071d70 0x7f920c1e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f920c145e10 0x7f920c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1461f0 con 0x7f920c0d0cc0 2024-04-24T06:34:19.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f920c072200 con 0x7f920c145e10 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.093+0000 7f9211c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f920c12eb20 con 0x7f920c071d70 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43922/0 (socket says 172.21.15.59:43922) 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 -- 172.21.15.59:0/4292129496 learned_addr learned my addr 172.21.15.59:0/4292129496 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f920aa00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f920c071d70 0x7f920c1e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f920b400640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f920c145e10 0x7f920c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 -- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f920c071d70 msgr2=0x7f920c1e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f920c071d70 0x7f920c1e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 -- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f920c145e10 msgr2=0x7f920c0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f920c145e10 0x7f920c0ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.094+0000 7f9203e00640 1 -- 172.21.15.59:0/4292129496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c0b1090 con 0x7f920c0d0cc0 2024-04-24T06:34:19.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9203e00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c410 tx=0x7f91f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4017070 con 0x7f920c0d0cc0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f40035f0 con 0x7f920c0d0cc0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4003a20 con 0x7f920c0d0cc0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c0b1370 con 0x7f920c0d0cc0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.095+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c1e8800 con 0x7f920c0d0cc0 2024-04-24T06:34:19.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.096+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c068a80 con 0x7f920c0d0cc0 2024-04-24T06:34:19.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.097+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91f401a070 con 0x7f920c0d0cc0 2024-04-24T06:34:19.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.098+0000 7f9202a00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.099+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f91f408b060 con 0x7f920c0d0cc0 2024-04-24T06:34:19.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.099+0000 7f920aa00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.100+0000 7f920aa00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 0x7f91e806edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f91fc034000 tx=0x7f91fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.103+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91f4059430 con 0x7f920c0d0cc0 2024-04-24T06:34:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.309+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f920c0d1190 con 0x7f920c0d0cc0 2024-04-24T06:34:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.310+0000 7f9202a00640 1 -- 172.21.15.59:0/4292129496 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f91f4058d30 con 0x7f920c0d0cc0 2024-04-24T06:34:19.311 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 msgr2=0x7f91e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 0x7f91e806edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f91fc034000 tx=0x7f91fc02e000 comp rx=0 tx=0).stop 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 msgr2=0x7f920c0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c410 tx=0x7f91f400c8d0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 shutdown_connections 2024-04-24T06:34:19.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f920c145e10 0x7f920c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f920c0d0cc0 0x7f920c0ae3f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 --2- 172.21.15.59:0/4292129496 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f920c071d70 0x7f920c1e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 >> 172.21.15.59:0/4292129496 conn(0x7f920c067d90 msgr2=0x7f920c15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 shutdown_connections 2024-04-24T06:34:19.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.313+0000 7f9211c00640 1 -- 172.21.15.59:0/4292129496 wait complete. 2024-04-24T06:34:19.335 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:34:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.543+0000 7fe7afe00640 1 Processor -- start 2024-04-24T06:34:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.543+0000 7fe7afe00640 1 -- start start 2024-04-24T06:34:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe7b01567e0 0x7fe7b0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7b015d810 0x7fe7b00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b00d0f60 con 0x7fe7b015d810 2024-04-24T06:34:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b015cab0 con 0x7fe7b01567e0 2024-04-24T06:34:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.544+0000 7fe7afe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b0160e40 con 0x7fe7b00d0b40 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7b015d810 0x7fe7b00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7b015d810 0x7fe7b00d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43932/0 (socket says 172.21.15.59:43932) 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 -- 172.21.15.59:0/725139618 learned_addr learned my addr 172.21.15.59:0/725139618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7aea00640 1 --2- 172.21.15.59:0/725139618 >> v2:172.21.15.66:3300/0 conn(0x7fe7b01567e0 0x7fe7b0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7af400640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 -- 172.21.15.59:0/725139618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 msgr2=0x7fe7b0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b0160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 -- 172.21.15.59:0/725139618 >> v2:172.21.15.66:3300/0 conn(0x7fe7b01567e0 msgr2=0x7fe7b0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 --2- 172.21.15.59:0/725139618 >> v2:172.21.15.66:3300/0 conn(0x7fe7b01567e0 0x7fe7b0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.545+0000 7fe7ae000640 1 -- 172.21.15.59:0/725139618 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b015cd30 con 0x7fe7b015d810 2024-04-24T06:34:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.547+0000 7fe7ae000640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b015d810 0x7fe7b00d0600 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00b6360 tx=0x7fe79c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80c0c1f7db2fd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.547+0000 7fe7ad600640 1 -- 172.21.15.59:0/725139618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c037070 con 0x7fe7b015d810 2024-04-24T06:34:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.547+0000 7fe7ad600640 1 -- 172.21.15.59:0/725139618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c032440 con 0x7fe7b015d810 2024-04-24T06:34:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.547+0000 7fe7ad600640 1 -- 172.21.15.59:0/725139618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c032810 con 0x7fe7b015d810 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 -- 172.21.15.59:0/725139618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b015d810 msgr2=0x7fe7b00d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b015d810 0x7fe7b00d0600 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00b6360 tx=0x7fe79c02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 -- 172.21.15.59:0/725139618 shutdown_connections 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b0160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 --2- 172.21.15.59:0/725139618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b015d810 0x7fe7b00d0600 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 --2- 172.21.15.59:0/725139618 >> v2:172.21.15.66:3300/0 conn(0x7fe7b01567e0 0x7fe7b0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 -- 172.21.15.59:0/725139618 >> 172.21.15.59:0/725139618 conn(0x7fe7b006bdf0 msgr2=0x7fe7b006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 -- 172.21.15.59:0/725139618 shutdown_connections 2024-04-24T06:34:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.548+0000 7fe7afe00640 1 -- 172.21.15.59:0/725139618 wait complete. 2024-04-24T06:34:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.550+0000 7fe7afe00640 1 Processor -- start 2024-04-24T06:34:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.550+0000 7fe7afe00640 1 -- start start 2024-04-24T06:34:19.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b01f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7b015d810 0x7fe7b0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0160650 con 0x7fe7b01567e0 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b015d2f0 con 0x7fe7b015d810 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7afe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b01607d0 con 0x7fe7b00d0b40 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7ae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7ae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43948/0 (socket says 172.21.15.59:43948) 2024-04-24T06:34:19.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.551+0000 7fe7ae000640 1 -- 172.21.15.59:0/395271591 learned_addr learned my addr 172.21.15.59:0/395271591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7aea00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b01f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 -- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 msgr2=0x7fe7b01f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b01f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7af400640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7b015d810 0x7fe7b0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 -- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7b015d810 msgr2=0x7fe7b0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7b015d810 0x7fe7b0158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 -- 172.21.15.59:0/395271591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01f44d0 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7ae000640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe79c02a220 tx=0x7fe79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c041020 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.552+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c0306c0 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.553+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c037040 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.553+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b0148a10 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.553+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b0148fd0 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.554+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b004fd90 con 0x7fe7b01567e0 2024-04-24T06:34:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.555+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe79c044020 con 0x7fe7b01567e0 2024-04-24T06:34:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.555+0000 7fe7a6a00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 0x7fe79006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.556+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fe79c0ac420 con 0x7fe7b01567e0 2024-04-24T06:34:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.556+0000 7fe7aea00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 0x7fe79006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:19.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.557+0000 7fe7aea00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 0x7fe79006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:19.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.560+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe79c04b030 con 0x7fe7b01567e0 2024-04-24T06:34:19.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.763+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe7b00b5150 con 0x7fe7b01567e0 2024-04-24T06:34:19.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.764+0000 7fe7a6a00640 1 -- 172.21.15.59:0/395271591 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7fe79c07a7e0 con 0x7fe7b01567e0 2024-04-24T06:34:19.765 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 msgr2=0x7fe79006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 0x7fe79006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0008040 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 msgr2=0x7fe7b01e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe79c02a220 tx=0x7fe79c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 shutdown_connections 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe79006c920 0x7fe79006ede0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7b015d810 0x7fe7b0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7b01567e0 0x7fe7b01e6880 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 --2- 172.21.15.59:0/395271591 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe7b00d0b40 0x7fe7b01f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:19.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.766+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 >> 172.21.15.59:0/395271591 conn(0x7fe7b006bdf0 msgr2=0x7fe7b015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:19.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.767+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 shutdown_connections 2024-04-24T06:34:19.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:19.767+0000 7fe7afe00640 1 -- 172.21.15.59:0/395271591 wait complete. 2024-04-24T06:34:19.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:34:20.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.002+0000 7f11cac00640 1 Processor -- start 2024-04-24T06:34:20.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.003+0000 7f11cac00640 1 -- start start 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.004+0000 7f11cac00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f11c40cd8b0 0x7f11c40d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.004+0000 7f11cac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40d0b00 0x7f11c406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.004+0000 7f11cac00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c40b4cf0 con 0x7f11c415c670 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.004+0000 7f11cac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f11c415ca50 con 0x7f11c40d0b00 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.004+0000 7f11cac00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f11c40ccb80 con 0x7f11c40cd8b0 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40d0b00 0x7f11c406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40d0b00 0x7f11c406ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43958/0 (socket says 172.21.15.59:43958) 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 -- 172.21.15.59:0/1678237538 learned_addr learned my addr 172.21.15.59:0/1678237538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:20.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11c8e00640 1 --2- 172.21.15.59:0/1678237538 >> v2:172.21.15.66:3300/0 conn(0x7f11c40cd8b0 0x7f11c40d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 -- 172.21.15.59:0/1678237538 >> v2:172.21.15.66:3300/0 conn(0x7f11c40cd8b0 msgr2=0x7f11c40d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 --2- 172.21.15.59:0/1678237538 >> v2:172.21.15.66:3300/0 conn(0x7f11c40cd8b0 0x7f11c40d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 -- 172.21.15.59:0/1678237538 >> v1:172.21.15.66:6789/0 conn(0x7f11c415c670 legacy=0x7f11c40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.005+0000 7f11ca200640 1 -- 172.21.15.59:0/1678237538 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c40ccdd0 con 0x7f11c40d0b00 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.006+0000 7f11ca200640 1 --2- 172.21.15.59:0/1678237538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c406ec50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f11c40ce7e0 tx=0x7f11bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569fba1346652397 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:20.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.007+0000 7f11c3e00640 1 -- 172.21.15.59:0/1678237538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc036070 con 0x7f11c40d0b00 2024-04-24T06:34:20.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.007+0000 7f11c3e00640 1 -- 172.21.15.59:0/1678237538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc031600 con 0x7f11c40d0b00 2024-04-24T06:34:20.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.007+0000 7f11c3e00640 1 -- 172.21.15.59:0/1678237538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0319d0 con 0x7f11c40d0b00 2024-04-24T06:34:20.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.007+0000 7f11cac00640 1 -- 172.21.15.59:0/1678237538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 msgr2=0x7f11c406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.007+0000 7f11cac00640 1 --2- 172.21.15.59:0/1678237538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c406ec50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f11c40ce7e0 tx=0x7f11bc02a6f0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 -- 172.21.15.59:0/1678237538 shutdown_connections 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 --2- 172.21.15.59:0/1678237538 >> v2:172.21.15.66:3300/0 conn(0x7f11c40cd8b0 0x7f11c40d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 --2- 172.21.15.59:0/1678237538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c406ec50 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 -- 172.21.15.59:0/1678237538 >> 172.21.15.59:0/1678237538 conn(0x7f11c406bdf0 msgr2=0x7f11c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 -- 172.21.15.59:0/1678237538 shutdown_connections 2024-04-24T06:34:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.008+0000 7f11cac00640 1 -- 172.21.15.59:0/1678237538 wait complete. 2024-04-24T06:34:20.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.010+0000 7f11cac00640 1 Processor -- start 2024-04-24T06:34:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.010+0000 7f11cac00640 1 -- start start 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c40cd8b0 0x7f11c41f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11c415c670 0x7f11c40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11c40b3e60 con 0x7f11c40d0b00 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11c40cd390 con 0x7f11c40cd8b0 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11cac00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f11c40b3fe0 con 0x7f11c415c670 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43974/0 (socket says 172.21.15.59:43974) 2024-04-24T06:34:20.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.011+0000 7f11c8e00640 1 -- 172.21.15.59:0/2762113849 learned_addr learned my addr 172.21.15.59:0/2762113849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11ca200640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11c415c670 0x7f11c40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 -- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11c415c670 msgr2=0x7f11c40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11c415c670 0x7f11c40b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 -- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c40cd8b0 msgr2=0x7f11c41f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c40cd8b0 0x7f11c41f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 -- 172.21.15.59:0/2762113849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c40b2450 con 0x7f11c40d0b00 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c8e00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f11b000d800 tx=0x7f11b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0004020 con 0x7f11c40d0b00 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b00041c0 con 0x7f11c40d0b00 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c40b2730 con 0x7f11c40d0b00 2024-04-24T06:34:20.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0004390 con 0x7f11c40d0b00 2024-04-24T06:34:20.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.012+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c4154c80 con 0x7f11c40d0b00 2024-04-24T06:34:20.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.014+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c41f0c80 con 0x7f11c40d0b00 2024-04-24T06:34:20.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.015+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11b001b020 con 0x7f11c40d0b00 2024-04-24T06:34:20.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.015+0000 7f11c1600640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 0x7f11a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:20.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.016+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f11b008b9c0 con 0x7f11c40d0b00 2024-04-24T06:34:20.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.016+0000 7f11c9800640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 0x7f11a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:20.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.017+0000 7f11c9800640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 0x7f11a406edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11b8006fd0 tx=0x7f11b8006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:20.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.020+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11b0059d80 con 0x7f11c40d0b00 2024-04-24T06:34:20.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.238+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f11c404fd90 con 0x7f11c40d0b00 2024-04-24T06:34:20.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.239+0000 7f11c1600640 1 -- 172.21.15.59:0/2762113849 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7f11b0059680 con 0x7f11c40d0b00 2024-04-24T06:34:20.241 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:34:20.241 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:34:20.241 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 msgr2=0x7f11a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 0x7f11a406edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11b8006fd0 tx=0x7f11b8006bc0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 msgr2=0x7f11c41516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f11b000d800 tx=0x7f11b000dcc0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 shutdown_connections 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f11a406c8f0 0x7f11a406edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11c415c670 0x7f11c40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40d0b00 0x7f11c41516c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 --2- 172.21.15.59:0/2762113849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11c40cd8b0 0x7f11c41f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 >> 172.21.15.59:0/2762113849 conn(0x7f11c406bdf0 msgr2=0x7f11c415b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 shutdown_connections 2024-04-24T06:34:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:20.242+0000 7f11cac00640 1 -- 172.21.15.59:0/2762113849 wait complete. 2024-04-24T06:34:20.261 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:34:20.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:20.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:34:20.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[58872]: fuse finished with error 0 and tester_r 0 2024-04-24T06:34:20.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:34:22.064 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:34:22.063+0000 7f6490400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T06:34:26.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:34:26.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:26.443 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:34:26.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:26.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-24T06:34:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.733+0000 7f118f200640 1 Processor -- start 2024-04-24T06:34:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.733+0000 7f118f200640 1 -- start start 2024-04-24T06:34:26.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.734+0000 7f118f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1190156c20 0x7f11900aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.734+0000 7f118f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1190157000 con 0x7f119006c500 2024-04-24T06:34:26.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.734+0000 7f118f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11900afb00 con 0x7f11900af4c0 2024-04-24T06:34:26.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.734+0000 7f118f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f119015cc20 con 0x7f1190156c20 2024-04-24T06:34:26.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.735+0000 7f118e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f119006c500 0x7f1190160100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53312/0 (socket says 172.21.15.59:53312) 2024-04-24T06:34:26.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.735+0000 7f118e800640 1 -- 172.21.15.59:0/2594748220 learned_addr learned my addr 172.21.15.59:0/2594748220 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:26.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.735+0000 7f118de00640 1 --2- 172.21.15.59:0/2594748220 >> v2:172.21.15.66:3300/0 conn(0x7f1190156c20 0x7f11900aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.735+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781356142 0 0) 0x7f1190157000 con 0x7f119006c500 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1174003520 con 0x7f119006c500 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2719228837 0 0) 0x7f1174003520 con 0x7f119006c500 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 >> v2:172.21.15.66:3300/0 conn(0x7f1190156c20 msgr2=0x7f11900aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 --2- 172.21.15.59:0/2594748220 >> v2:172.21.15.66:3300/0 conn(0x7f1190156c20 0x7f11900aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 >> v1:172.21.15.66:6789/0 conn(0x7f11900af4c0 legacy=0x7f119006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119015cf00 con 0x7f119006c500 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f11880019f0 con 0x7f119006c500 2024-04-24T06:34:26.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 >> v1:172.21.15.59:6789/0 conn(0x7f119006c500 legacy=0x7f1190160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1174004800 0x7f1174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 0x7f117400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f119015cc20 con 0x7f11740051c0 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f117400c0c0 con 0x7f1174004800 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f117400c2c0 con 0x7f1174003a00 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.736+0000 7f118e800640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 0x7f117400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118de00640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118d400640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1174004800 0x7f1174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118e800640 1 -- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1174003a00 msgr2=0x7f1174003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118e800640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118e800640 1 -- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1174004800 msgr2=0x7f1174004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118e800640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1174004800 0x7f1174004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.737+0000 7f118e800640 1 -- 172.21.15.59:0/2594748220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1174003520 con 0x7f11740051c0 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.738+0000 7f118e800640 1 -- 172.21.15.59:0/2594748220 reap_dead start 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.738+0000 7f118e800640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 0x7f117400b9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f118800a790 tx=0x7f1174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4515b9cb965c24c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.738+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118800a3b0 con 0x7f11740051c0 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.738+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802f070 con 0x7f11740051c0 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.738+0000 7f118ca00640 1 -- 172.21.15.59:0/2594748220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118802bc00 con 0x7f11740051c0 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 -- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 msgr2=0x7f117400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 0x7f117400b9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f118800a790 tx=0x7f1174005830 comp rx=0 tx=0).stop 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 -- 172.21.15.59:0/2594748220 shutdown_connections 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 --2- 172.21.15.59:0/2594748220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11740051c0 0x7f117400b9c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 -- 172.21.15.59:0/2594748220 >> 172.21.15.59:0/2594748220 conn(0x7f11900b2200 msgr2=0x7f11900b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 -- 172.21.15.59:0/2594748220 shutdown_connections 2024-04-24T06:34:26.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.739+0000 7f118f200640 1 -- 172.21.15.59:0/2594748220 wait complete. 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.741+0000 7f118f200640 1 Processor -- start 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.741+0000 7f118f200640 1 -- start start 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11740051c0 0x7f11900cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1190155ec0 0x7f119015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f119015d9e0 con 0x7f1190148980 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f119015d4c0 con 0x7f11740051c0 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1190073ba0 con 0x7f1190155ec0 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43998/0 (socket says 172.21.15.59:43998) 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.742+0000 7f118e800640 1 -- 172.21.15.59:0/3743593108 learned_addr learned my addr 172.21.15.59:0/3743593108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118d400640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1190155ec0 0x7f119015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118de00640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11740051c0 0x7f11900cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 -- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1190155ec0 msgr2=0x7f119015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1190155ec0 0x7f119015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 -- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11740051c0 msgr2=0x7f11900cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11740051c0 0x7f11900cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 -- 172.21.15.59:0/3743593108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11900cca80 con 0x7f1190148980 2024-04-24T06:34:26.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f118e800640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f118802b450 tx=0x7f118802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.743+0000 7f1186000640 1 -- 172.21.15.59:0/3743593108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118802b7a0 con 0x7f1190148980 2024-04-24T06:34:26.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.744+0000 7f1186000640 1 -- 172.21.15.59:0/3743593108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802b940 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.744+0000 7f1186000640 1 -- 172.21.15.59:0/3743593108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118802bcf0 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.744+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11900ccd60 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.744+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119006e0c0 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.746+0000 7f1186000640 1 -- 172.21.15.59:0/3743593108 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1188034470 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.747+0000 7f1186000640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.747+0000 7f1186000640 1 -- 172.21.15.59:0/3743593108 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f11880ab760 con 0x7f1190148980 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.747+0000 7f118de00640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.747+0000 7f1197dd1740 1 Processor -- start 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 -- start start 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f118de00640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 0x7f117006edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e71b50 0x55c849e720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c849d8d3a0 con 0x55c849e29f90 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c849d33240 con 0x55c849e542b0 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.748+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c849d89360 con 0x55c849e71b50 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44006/0 (socket says 172.21.15.59:44006) 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 -- 172.21.15.59:0/344446961 learned_addr learned my addr 172.21.15.59:0/344446961 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118de00640 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 -- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e71b50 msgr2=0x55c849e720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e71b50 0x55c849e720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 -- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 msgr2=0x55c849e54690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e54690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 -- 172.21.15.59:0/344446961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c849b6ba20 con 0x55c849e29f90 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.749+0000 7f118e800640 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1188079120 tx=0x7f11880757f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dfd93600ded467 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.750+0000 7f117aa00640 1 -- 172.21.15.59:0/344446961 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118807d7d0 con 0x55c849e29f90 2024-04-24T06:34:26.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.750+0000 7f117aa00640 1 -- 172.21.15.59:0/344446961 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802f070 con 0x55c849e29f90 2024-04-24T06:34:26.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.750+0000 7f117aa00640 1 -- 172.21.15.59:0/344446961 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188082000 con 0x55c849e29f90 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.750+0000 7f1197dd1740 1 -- 172.21.15.59:0/344446961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 msgr2=0x55c849e70210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.750+0000 7f1197dd1740 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1188079120 tx=0x7f11880757f0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 -- 172.21.15.59:0/344446961 shutdown_connections 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e29f90 0x55c849e70210 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e71b50 0x55c849e720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 --2- 172.21.15.59:0/344446961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e54690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 -- 172.21.15.59:0/344446961 >> 172.21.15.59:0/344446961 conn(0x55c849e268c0 msgr2=0x55c849e6f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 -- 172.21.15.59:0/344446961 shutdown_connections 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 -- 172.21.15.59:0/344446961 wait complete. 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.751+0000 7f1197dd1740 1 -- start start 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.752+0000 7f1197dd1740 1 Processor -- start 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e4ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 0x55c849e7f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c849d33240 con 0x55c849e23500 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c849d467e0 con 0x55c849e71b50 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c849d8d3a0 con 0x55c849e542b0 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f1197dd1740 10 client.0 _ll_register_callbacks cb 0x55c849d83b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44016/0 (socket says 172.21.15.59:44016) 2024-04-24T06:34:26.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 -- 172.21.15.59:0/2935675720 learned_addr learned my addr 172.21.15.59:0/2935675720 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118de00640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e4ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118d400640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 0x55c849e7f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 -- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 msgr2=0x55c849e4ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e4ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 -- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 msgr2=0x55c849e7f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 0x55c849e7f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118e800640 1 -- 172.21.15.59:0/2935675720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c849a5f310 con 0x55c849e23500 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.753+0000 7f118d400640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 0x55c849e7f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f118e800640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1188075880 tx=0x7f1188004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176c3689253079ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f116f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880a8060 con 0x55c849e23500 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f1188075a90 con 0x55c849e23500 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118802f070 con 0x55c849e23500 2024-04-24T06:34:26.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1188096050 con 0x55c849e23500 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f1197dd1740 10 client.7158 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55c849bcc1b0 con 0x55c849e23500 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.754+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802f070 con 0x55c849e23500 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f1188078870 con 0x55c849e23500 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 10 client.7158 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 10 client.7158 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 20 client.7158 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e55440 0x55c849e55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 4 client.7158 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c849d9adc0 con 0x55c849e55440 2024-04-24T06:34:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.755+0000 7f118de00640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e55440 0x55c849e55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.756+0000 7f118de00640 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e55440 0x55c849e55850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de17d19663260d4e server_cookie=178bd76bdda8e306 in_seq=0 out_seq=0 2024-04-24T06:34:26.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.756+0000 7f116f400640 10 client.7158 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:26.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.756+0000 7f116f400640 1 -- 172.21.15.59:0/2935675720 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c849d9adc0 con 0x55c849e55440 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.756+0000 7f116f400640 10 client.7158 handle_command_reply: tid=0 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.756+0000 7f1197dd1740 1 client.7158 shutdown 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 msgr2=0x55c849e23980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1188075880 tx=0x7f1188004f70 comp rx=0 tx=0).stop 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 shutdown_connections 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e55440 0x55c849e55850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e23500 0x55c849e23980 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e71b50 0x55c849e7f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 --2- 172.21.15.59:0/2935675720 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e4ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 >> 172.21.15.59:0/2935675720 conn(0x55c849e29f90 msgr2=0x55c849e72640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f116f400640 0 client.7158 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 shutdown_connections 2024-04-24T06:34:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 1 -- 172.21.15.59:0/2935675720 wait complete. 2024-04-24T06:34:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.757+0000 7f1197dd1740 20 client.7158 trim_cache size 0 max 16384 2024-04-24T06:34:26.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.777+0000 7f1197dd1740 1 Processor -- start 2024-04-24T06:34:26.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.777+0000 7f1197dd1740 1 -- start start 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e23500 0x55c849e50470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1190201370 con 0x55c849e71b50 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11902003d0 con 0x55c849e542b0 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f119006f530 con 0x55c849e23500 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44030/0 (socket says 172.21.15.59:44030) 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.778+0000 7f118e800640 1 -- 172.21.15.59:0/324939758 learned_addr learned my addr 172.21.15.59:0/324939758 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 -- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e23500 msgr2=0x55c849e50470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e23500 0x55c849e50470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 -- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 msgr2=0x55c849e6edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 -- 172.21.15.59:0/324939758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c849e29dc0 con 0x55c849e71b50 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f118e800640 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1188000f80 tx=0x7f1188042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97590cd4d1511b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f117a000640 1 -- 172.21.15.59:0/324939758 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188078f60 con 0x55c849e71b50 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f117a000640 1 -- 172.21.15.59:0/324939758 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802f070 con 0x55c849e71b50 2024-04-24T06:34:26.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.779+0000 7f117a000640 1 -- 172.21.15.59:0/324939758 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188075a90 con 0x55c849e71b50 2024-04-24T06:34:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 -- 172.21.15.59:0/324939758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 msgr2=0x55c849e6f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1188000f80 tx=0x7f1188042040 comp rx=0 tx=0).stop 2024-04-24T06:34:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 -- 172.21.15.59:0/324939758 shutdown_connections 2024-04-24T06:34:26.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 --2- 172.21.15.59:0/324939758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e23500 0x55c849e50470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 -- 172.21.15.59:0/324939758 >> 172.21.15.59:0/324939758 conn(0x55c849e29f90 msgr2=0x55c849e6d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 -- 172.21.15.59:0/324939758 shutdown_connections 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.780+0000 7f1197dd1740 1 -- 172.21.15.59:0/324939758 wait complete. 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.781+0000 7f1197dd1740 1 -- start start 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.781+0000 7f1197dd1740 1 Processor -- start 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e23500 0x55c849e24830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c849e23d60 con 0x55c849e71b50 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ffdb0 con 0x55c849e23500 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c849a5f310 con 0x55c849e542b0 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f1197dd1740 10 client.0 _ll_register_callbacks cb 0x7f1188043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f118e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44044/0 (socket says 172.21.15.59:44044) 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.782+0000 7f118e800640 1 -- 172.21.15.59:0/1631098694 learned_addr learned my addr 172.21.15.59:0/1631098694 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118d400640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118de00640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e23500 0x55c849e24830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 -- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 msgr2=0x55c849e6edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 -- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e23500 msgr2=0x55c849e24830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:26.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e23500 0x55c849e24830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 -- 172.21.15.59:0/1631098694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c849a38490 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f118e800640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f11880a9c20 tx=0x7f1188002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb8fbcf2a546a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f116f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188078f60 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f1188075380 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118802f070 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f118807d7d0 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.783+0000 7f1197dd1740 10 client.7164 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55c849a38490 con 0x55c849e71b50 2024-04-24T06:34:26.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118802f070 con 0x55c849e71b50 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f11880782e0 con 0x55c849e71b50 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 10 client.7164 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 10 client.7164 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 20 client.7164 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e538f0 0x55c849ee8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 4 client.7164 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55c849d94c30 con 0x55c849e538f0 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.784+0000 7f118de00640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e538f0 0x55c849ee8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.785+0000 7f118de00640 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e538f0 0x55c849ee8180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10fe35faff1b5ea3 server_cookie=8651e450bb838617 in_seq=0 out_seq=0 2024-04-24T06:34:26.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.785+0000 7f116f400640 10 client.7164 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:26.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.785+0000 7f116f400640 1 -- 172.21.15.59:0/1631098694 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c849d94c30 con 0x55c849e538f0 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.785+0000 7f116f400640 10 client.7164 handle_command_reply: tid=0 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.785+0000 7f1197dd1740 1 client.7164 shutdown 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 msgr2=0x55c849e6f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f11880a9c20 tx=0x7f1188002150 comp rx=0 tx=0).stop 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 shutdown_connections 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c849e538f0 0x55c849ee8180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c849e71b50 0x55c849e6f2f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c849e542b0 0x55c849e6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 --2- 172.21.15.59:0/1631098694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c849e23500 0x55c849e24830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f116f400640 0 client.7164 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 >> 172.21.15.59:0/1631098694 conn(0x55c849e29f90 msgr2=0x55c849e50470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 shutdown_connections 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 1 -- 172.21.15.59:0/1631098694 wait complete. 2024-04-24T06:34:26.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.786+0000 7f1197dd1740 20 client.7164 trim_cache size 0 max 16384 2024-04-24T06:34:26.793 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:34:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "0cd676bf-ed73-4c06-bbfc-9cdad057c976", 2024-04-24T06:34:26.793 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:34:26.793 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 msgr2=0x7f117006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 0x7f117006edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180008040 comp rx=0 tx=0).stop 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 msgr2=0x7f119015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f118802b450 tx=0x7f118802d070 comp rx=0 tx=0).stop 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 shutdown_connections 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f117006c8f0 0x7f117006edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1190148980 0x7f119015b950 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1190155ec0 0x7f119015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 --2- 172.21.15.59:0/3743593108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11740051c0 0x7f11900cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:26.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 >> 172.21.15.59:0/3743593108 conn(0x7f11900b2200 msgr2=0x7f119015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:26.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 shutdown_connections 2024-04-24T06:34:26.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:26.787+0000 7f118f200640 1 -- 172.21.15.59:0/3743593108 wait complete. 2024-04-24T06:34:26.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:26.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-24T06:34:27.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.016+0000 7f0520400640 1 Processor -- start 2024-04-24T06:34:27.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.017+0000 7f0520400640 1 -- start start 2024-04-24T06:34:27.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.018+0000 7f0520400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05180d42c0 0x7f051815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.018+0000 7f0520400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05180d1670 0x7f05180d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.018+0000 7f0520400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051815edb0 con 0x7f051815e9d0 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.018+0000 7f0520400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f05180d3e30 con 0x7f05180d1670 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.018+0000 7f0520400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f051815ff20 con 0x7f05180d42c0 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05180d1670 0x7f05180d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f051815e9d0 0x7f05180d0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53320/0 (socket says 172.21.15.59:53320) 2024-04-24T06:34:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051da00640 1 -- 172.21.15.59:0/4019769469 learned_addr learned my addr 172.21.15.59:0/4019769469 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051d000640 1 --2- 172.21.15.59:0/4019769469 >> v2:172.21.15.66:3300/0 conn(0x7f05180d42c0 0x7f051815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051e400640 1 -- 172.21.15.59:0/4019769469 >> v2:172.21.15.66:3300/0 conn(0x7f05180d42c0 msgr2=0x7f051815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051e400640 1 --2- 172.21.15.59:0/4019769469 >> v2:172.21.15.66:3300/0 conn(0x7f05180d42c0 0x7f051815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051e400640 1 -- 172.21.15.59:0/4019769469 >> v1:172.21.15.59:6789/0 conn(0x7f051815e9d0 legacy=0x7f05180d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f051e400640 1 -- 172.21.15.59:0/4019769469 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05180ba630 con 0x7f05180d1670 2024-04-24T06:34:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.019+0000 7f0517e00640 1 -- 172.21.15.59:0/4019769469 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783592354 0 0) 0x7f051815edb0 con 0x7f051815e9d0 2024-04-24T06:34:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.021+0000 7f051e400640 1 --2- 172.21.15.59:0/4019769469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d1670 0x7f05180d2390 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f05180d3fb0 tx=0x7f051002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bd4d9bc76fab75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.021+0000 7f0517e00640 1 -- 172.21.15.59:0/4019769469 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510037070 con 0x7f05180d1670 2024-04-24T06:34:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.021+0000 7f0517e00640 1 -- 172.21.15.59:0/4019769469 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510032440 con 0x7f05180d1670 2024-04-24T06:34:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.021+0000 7f0517e00640 1 -- 172.21.15.59:0/4019769469 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510032810 con 0x7f05180d1670 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 -- 172.21.15.59:0/4019769469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d1670 msgr2=0x7f05180d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 --2- 172.21.15.59:0/4019769469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d1670 0x7f05180d2390 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f05180d3fb0 tx=0x7f051002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 -- 172.21.15.59:0/4019769469 shutdown_connections 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 --2- 172.21.15.59:0/4019769469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d1670 0x7f05180d2390 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 --2- 172.21.15.59:0/4019769469 >> v2:172.21.15.66:3300/0 conn(0x7f05180d42c0 0x7f051815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 -- 172.21.15.59:0/4019769469 >> 172.21.15.59:0/4019769469 conn(0x7f051806cbd0 msgr2=0x7f051806cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 -- 172.21.15.59:0/4019769469 shutdown_connections 2024-04-24T06:34:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.022+0000 7f0520400640 1 -- 172.21.15.59:0/4019769469 wait complete. 2024-04-24T06:34:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.024+0000 7f0520400640 1 Processor -- start 2024-04-24T06:34:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.024+0000 7f0520400640 1 -- start start 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05180d1670 0x7f05180b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f051815e9d0 0x7f05180b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05180cfd70 con 0x7f05180d42c0 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05180cfa90 con 0x7f051815e9d0 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f0520400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05180ba0e0 con 0x7f05180d1670 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44066/0 (socket says 172.21.15.59:44066) 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f051d000640 1 -- 172.21.15.59:0/466969050 learned_addr learned my addr 172.21.15.59:0/466969050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f051da00640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05180d1670 0x7f05180b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.025+0000 7f051e400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f051815e9d0 0x7f05180b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 -- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05180d1670 msgr2=0x7f05180b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05180d1670 0x7f05180b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 -- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f051815e9d0 msgr2=0x7f05180b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f051815e9d0 0x7f05180b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 -- 172.21.15.59:0/466969050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05180b5b90 con 0x7f05180d42c0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f051d000640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f050400d800 tx=0x7f050400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f0515600640 1 -- 172.21.15.59:0/466969050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400b6c0 con 0x7f05180d42c0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f0515600640 1 -- 172.21.15.59:0/466969050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504019070 con 0x7f05180d42c0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05180b5640 con 0x7f05180d42c0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f0515600640 1 -- 172.21.15.59:0/466969050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400ba80 con 0x7f05180d42c0 2024-04-24T06:34:27.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.026+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05180c83e0 con 0x7f05180d42c0 2024-04-24T06:34:27.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.029+0000 7f0515600640 1 -- 172.21.15.59:0/466969050 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f050400bc20 con 0x7f05180d42c0 2024-04-24T06:34:27.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.030+0000 7f0515600640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 0x7f04f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.030+0000 7f0515600640 1 -- 172.21.15.59:0/466969050 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f050408c0d0 con 0x7f05180d42c0 2024-04-24T06:34:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.030+0000 7f051da00640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 0x7f04f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.030+0000 7f0523368740 1 Processor -- start 2024-04-24T06:34:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.031+0000 7f051da00640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 0x7f04f806edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f050c007ed0 tx=0x7f050c007e60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.031+0000 7f0523368740 1 -- start start 2024-04-24T06:34:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.031+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f0523368740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f0523368740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558fe1a861f0 con 0x558fe1b4c610 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558fe1a2c350 con 0x558fe1b69c30 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x558fe1a821b0 con 0x558fe1b4cd50 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44068/0 (socket says 172.21.15.59:44068) 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 -- 172.21.15.59:0/3585646191 learned_addr learned my addr 172.21.15.59:0/3585646191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051da00640 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051d000640 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 -- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 msgr2=0x558fe1b4c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b4c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 -- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 msgr2=0x558fe1b1c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b1c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.032+0000 7f051e400640 1 -- 172.21.15.59:0/3585646191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe1a80f50 con 0x558fe1b4c610 2024-04-24T06:34:27.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.033+0000 7f051e400640 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f051002a790 tx=0x7f051002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ab98a274bf7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.033+0000 7f050a000640 1 -- 172.21.15.59:0/3585646191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510037040 con 0x558fe1b4c610 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.033+0000 7f050a000640 1 -- 172.21.15.59:0/3585646191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510042070 con 0x558fe1b4c610 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.033+0000 7f050a000640 1 -- 172.21.15.59:0/3585646191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051003c860 con 0x558fe1b4c610 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- 172.21.15.59:0/3585646191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 msgr2=0x558fe1b21420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f051002a790 tx=0x7f051002c000 comp rx=0 tx=0).stop 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- 172.21.15.59:0/3585646191 shutdown_connections 2024-04-24T06:34:27.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4c610 0x558fe1b21420 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 --2- 172.21.15.59:0/3585646191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b1c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- 172.21.15.59:0/3585646191 >> 172.21.15.59:0/3585646191 conn(0x558fe1b1e830 msgr2=0x558fe1b670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- 172.21.15.59:0/3585646191 shutdown_connections 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- 172.21.15.59:0/3585646191 wait complete. 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.034+0000 7f0523368740 1 -- start start 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.035+0000 7f0523368740 1 Processor -- start 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b63850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558fe1b209e0 con 0x558fe1b48ea0 2024-04-24T06:34:27.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558fe1a3f690 con 0x558fe1b69c30 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x558fe1b1fdd0 con 0x558fe1b4cd50 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f0523368740 10 client.0 _ll_register_callbacks cb 0x558fe1a7b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44082/0 (socket says 172.21.15.59:44082) 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 -- 172.21.15.59:0/249731114 learned_addr learned my addr 172.21.15.59:0/249731114 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 -- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 msgr2=0x558fe1b47340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 -- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 msgr2=0x558fe1b63850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b63850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.036+0000 7f051e400640 1 -- 172.21.15.59:0/249731114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe1a3e920 con 0x558fe1b48ea0 2024-04-24T06:34:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f051e400640 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0510004280 tx=0x7f0510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d768a036d01ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f04fea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510037040 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f0510003a20 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0510042070 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0510003c20 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f0523368740 10 client.7179 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.037+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x558fe18627d0 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510042070 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0510005990 con 0x558fe1b48ea0 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 10 client.7179 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 10 client.7179 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 20 client.7179 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b4ddd0 0x558fe1b4e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 4 client.7179 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fe1a93c10 con 0x558fe1b4ddd0 2024-04-24T06:34:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.038+0000 7f051da00640 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b4ddd0 0x558fe1b4e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.039+0000 7f051da00640 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b4ddd0 0x558fe1b4e200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b1e7cd6f0de10e server_cookie=fe5d24e230d06806 in_seq=0 out_seq=0 2024-04-24T06:34:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.039+0000 7f04fea00640 10 client.7179 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.039+0000 7f04fea00640 1 -- 172.21.15.59:0/249731114 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fe1a93c10 con 0x558fe1b4ddd0 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.039+0000 7f04fea00640 10 client.7179 handle_command_reply: tid=0 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 client.7179 shutdown 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 msgr2=0x558fe1b49320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0510004280 tx=0x7f0510004730 comp rx=0 tx=0).stop 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 shutdown_connections 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b4ddd0 0x558fe1b4e200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b48ea0 0x558fe1b49320 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b63850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 --2- 172.21.15.59:0/249731114 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b4cd50 0x558fe1b47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f04fea00640 0 client.7179 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 >> 172.21.15.59:0/249731114 conn(0x558fe1b4c610 msgr2=0x558fe1b48750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 shutdown_connections 2024-04-24T06:34:27.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 1 -- 172.21.15.59:0/249731114 wait complete. 2024-04-24T06:34:27.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.040+0000 7f0523368740 20 client.7179 trim_cache size 0 max 16384 2024-04-24T06:34:27.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.060+0000 7f0523368740 1 Processor -- start 2024-04-24T06:34:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.060+0000 7f0523368740 1 -- start start 2024-04-24T06:34:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b47150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0518200ca0 con 0x558fe1b4cd50 2024-04-24T06:34:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05181ffd00 con 0x558fe1b69c30 2024-04-24T06:34:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05181ff060 con 0x558fe1b48ea0 2024-04-24T06:34:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44084/0 (socket says 172.21.15.59:44084) 2024-04-24T06:34:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f051d000640 1 -- 172.21.15.59:0/3872699157 learned_addr learned my addr 172.21.15.59:0/3872699157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.061+0000 7f051e400640 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051da00640 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b47150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 -- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 msgr2=0x558fe1b47150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b47150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 -- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 msgr2=0x558fe1b675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 -- 172.21.15.59:0/3872699157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe1b505a0 con 0x558fe1b4cd50 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f051d000640 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0504062440 tx=0x7f0504059a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596eeb549c7de7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f0509600640 1 -- 172.21.15.59:0/3872699157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504059d90 con 0x558fe1b4cd50 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f0509600640 1 -- 172.21.15.59:0/3872699157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050405e140 con 0x558fe1b4cd50 2024-04-24T06:34:27.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.062+0000 7f0509600640 1 -- 172.21.15.59:0/3872699157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504062970 con 0x558fe1b4cd50 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 -- 172.21.15.59:0/3872699157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 msgr2=0x558fe1b67060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0504062440 tx=0x7f0504059a00 comp rx=0 tx=0).stop 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 -- 172.21.15.59:0/3872699157 shutdown_connections 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1b67060 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 --2- 172.21.15.59:0/3872699157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b47150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 -- 172.21.15.59:0/3872699157 >> 172.21.15.59:0/3872699157 conn(0x558fe1b4c610 msgr2=0x558fe1b4fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 -- 172.21.15.59:0/3872699157 shutdown_connections 2024-04-24T06:34:27.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.063+0000 7f0523368740 1 -- 172.21.15.59:0/3872699157 wait complete. 2024-04-24T06:34:27.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.064+0000 7f0523368740 1 -- start start 2024-04-24T06:34:27.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.064+0000 7f0523368740 1 Processor -- start 2024-04-24T06:34:27.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05181ff6b0 con 0x558fe1b4cd50 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558fe1b20110 con 0x558fe1b69c30 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x558fe1b68fb0 con 0x558fe1b48ea0 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f0523368740 10 client.0 _ll_register_callbacks cb 0x7f0510009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.065+0000 7f051d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44090/0 (socket says 172.21.15.59:44090) 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 -- 172.21.15.59:0/389373 learned_addr learned my addr 172.21.15.59:0/389373 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051da00640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051e400640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 -- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 msgr2=0x558fe1b219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 -- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 msgr2=0x558fe1b79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:27.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 -- 172.21.15.59:0/389373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe17983f0 con 0x558fe1b4cd50 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f051d000640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0504004790 tx=0x7f0504020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaeff818b55d31e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.066+0000 7f04fea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504059d90 con 0x558fe1b4cd50 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f0504058090 con 0x558fe1b4cd50 2024-04-24T06:34:27.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 1 -- 172.21.15.59:0/389373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050405e140 con 0x558fe1b4cd50 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0504056400 con 0x558fe1b4cd50 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 10 client.7185 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 1 -- 172.21.15.59:0/389373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x558fe17580f0 con 0x558fe1b4cd50 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050405e140 con 0x558fe1b4cd50 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0504013070 con 0x558fe1b4cd50 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 10 client.7185 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 10 client.7185 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.067+0000 7f0523368740 20 client.7185 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.068+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b6aee0 0x558fe1b79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.068+0000 7f0523368740 4 client.7185 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:34:27.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.068+0000 7f0523368740 1 -- 172.21.15.59:0/389373 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fe1a8da80 con 0x558fe1b6aee0 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.068+0000 7f051da00640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b6aee0 0x558fe1b79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f051da00640 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b6aee0 0x558fe1b79f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f83f48b6248d746 server_cookie=13cb04fe1c3dc160 in_seq=0 out_seq=0 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f04fea00640 10 client.7185 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f04fea00640 1 -- 172.21.15.59:0/389373 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558fe1a8da80 con 0x558fe1b6aee0 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f04fea00640 10 client.7185 handle_command_reply: tid=0 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 client.7185 shutdown 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 -- 172.21.15.59:0/389373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 msgr2=0x558fe1be8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0504004790 tx=0x7f0504020000 comp rx=0 tx=0).stop 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 -- 172.21.15.59:0/389373 shutdown_connections 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x558fe1b6aee0 0x558fe1b79f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558fe1b69c30 0x558fe1b79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558fe1b4cd50 0x558fe1be8320 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 --2- 172.21.15.59:0/389373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x558fe1b48ea0 0x558fe1b219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f04fea00640 0 client.7185 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:27.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 -- 172.21.15.59:0/389373 >> 172.21.15.59:0/389373 conn(0x558fe1b4c610 msgr2=0x558fe1b47150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 -- 172.21.15.59:0/389373 shutdown_connections 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.069+0000 7f0523368740 1 -- 172.21.15.59:0/389373 wait complete. 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0523368740 20 client.7185 trim_cache size 0 max 16384 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "0cd676bf-ed73-4c06-bbfc-9cdad057c976": { 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir", 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "0cd676bf-ed73-4c06-bbfc-9cdad057c976", 2024-04-24T06:34:27.074 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-24T06:34:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:27.075 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:27.075 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 msgr2=0x7f04f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 0x7f04f806edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f050c007ed0 tx=0x7f050c007e60 comp rx=0 tx=0).stop 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 msgr2=0x7f05180b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f050400d800 tx=0x7f050400dcc0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 shutdown_connections 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04f806c8f0 0x7f04f806edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f051815e9d0 0x7f05180b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05180d42c0 0x7f05180b5100 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 --2- 172.21.15.59:0/466969050 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05180d1670 0x7f05180b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.070+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 >> 172.21.15.59:0/466969050 conn(0x7f051806cbd0 msgr2=0x7f05181613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.071+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 shutdown_connections 2024-04-24T06:34:27.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:27.071+0000 7f0520400640 1 -- 172.21.15.59:0/466969050 wait complete. 2024-04-24T06:34:27.091 INFO:tasks.cephfs.filesystem:scrub status for tag:0cd676bf-ed73-4c06-bbfc-9cdad057c976 - {'path': '/subdir', 'tag': '0cd676bf-ed73-4c06-bbfc-9cdad057c976', 'options': 'recursive'} 2024-04-24T06:34:36.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:34:36.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:34:57.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:57.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-24T06:34:57.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.327+0000 7ff3f2800640 1 Processor -- start 2024-04-24T06:34:57.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.328+0000 7ff3f2800640 1 -- start start 2024-04-24T06:34:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.329+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3ec0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.329+0000 7ff3f2800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec160120 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.329+0000 7ff3f2800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec160400 con 0x7ff3ec0af790 2024-04-24T06:34:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.329+0000 7ff3f2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec0d0f70 con 0x7ff3ec15c410 2024-04-24T06:34:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.330+0000 7ff3f0a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff3ec0d0b10 0x7ff3ec0af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50614/0 (socket says 172.21.15.59:50614) 2024-04-24T06:34:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.330+0000 7ff3f0a00640 1 -- 172.21.15.59:0/2049014954 learned_addr learned my addr 172.21.15.59:0/2049014954 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.330+0000 7ff3f1400640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3ec0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.330+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2514943703 0 0) 0x7ff3ec160120 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.330+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3d4003520 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 948954392 0 0) 0x7ff3d4003520 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 msgr2=0x7ff3ec0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3ec0d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 >> v1:172.21.15.66:6789/0 conn(0x7ff3ec0af790 legacy=0x7ff3ec071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec1606e0 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff3d8001670 con 0x7ff3ec0d0b10 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 >> v1:172.21.15.59:6789/0 conn(0x7ff3ec0d0b10 legacy=0x7ff3ec0af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d40039e0 0x7ff3d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3d4004d40 0x7ff3d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d400ba70 con 0x7ff3d4004d40 2024-04-24T06:34:57.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d400bc20 con 0x7ff3d40039e0 2024-04-24T06:34:57.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d400be20 con 0x7ff3ec15c410 2024-04-24T06:34:57.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.331+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3d4004d40 0x7ff3d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1400640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d40039e0 0x7ff3d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1e00640 1 -- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 msgr2=0x7ff3d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec15c410 0x7ff3d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1e00640 1 -- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d40039e0 msgr2=0x7ff3d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d40039e0 0x7ff3d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.332+0000 7ff3f1e00640 1 -- 172.21.15.59:0/2049014954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d4003520 con 0x7ff3d4004d40 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.333+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3d4004d40 0x7ff3d40051c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4001e20 tx=0x7ff3d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef4ddc680488689 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.333+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4001a70 con 0x7ff3d4004d40 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.333+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e402ae40 con 0x7ff3d4004d40 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.333+0000 7ff3ebe00640 1 -- 172.21.15.59:0/2049014954 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4003bb0 con 0x7ff3d4004d40 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 -- 172.21.15.59:0/2049014954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3d4004d40 msgr2=0x7ff3d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 --2- 172.21.15.59:0/2049014954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3d4004d40 0x7ff3d40051c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4001e20 tx=0x7ff3d4005830 comp rx=0 tx=0).stop 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f1e00640 1 -- 172.21.15.59:0/2049014954 reap_dead start 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 -- 172.21.15.59:0/2049014954 shutdown_connections 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 -- 172.21.15.59:0/2049014954 >> 172.21.15.59:0/2049014954 conn(0x7ff3ec06bdf0 msgr2=0x7ff3ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 -- 172.21.15.59:0/2049014954 shutdown_connections 2024-04-24T06:34:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.334+0000 7ff3f2800640 1 -- 172.21.15.59:0/2049014954 wait complete. 2024-04-24T06:34:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.336+0000 7ff3f2800640 1 Processor -- start 2024-04-24T06:34:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.336+0000 7ff3f2800640 1 -- start start 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d4004d40 0x7ff3ec15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec159170 0x7ff3ec1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec15c7f0 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec0d05d0 con 0x7ff3d4004d40 2024-04-24T06:34:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f2800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec159950 con 0x7ff3ec159170 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36276/0 (socket says 172.21.15.59:36276) 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.337+0000 7ff3f1e00640 1 -- 172.21.15.59:0/355475886 learned_addr learned my addr 172.21.15.59:0/355475886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1400640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d4004d40 0x7ff3ec15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec159170 0x7ff3ec1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 -- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec159170 msgr2=0x7ff3ec1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec159170 0x7ff3ec1553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 -- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d4004d40 msgr2=0x7ff3ec15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d4004d40 0x7ff3ec15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 -- 172.21.15.59:0/355475886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec159c30 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.338+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005040 tx=0x7ff3e40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.339+0000 7ff3e9600640 1 -- 172.21.15.59:0/355475886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4007db0 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.339+0000 7ff3e9600640 1 -- 172.21.15.59:0/355475886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e402ac40 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.339+0000 7ff3e9600640 1 -- 172.21.15.59:0/355475886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40050d0 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.339+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec159f10 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.339+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec1595f0 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.341+0000 7ff3e9600640 1 -- 172.21.15.59:0/355475886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3e4043020 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.342+0000 7ff3e9600640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 0x7ff3cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.342+0000 7ff3e9600640 1 -- 172.21.15.59:0/355475886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ff3e40ab170 con 0x7ff3ec1f5d60 2024-04-24T06:34:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.343+0000 7ff3f1400640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 0x7ff3cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.343+0000 7ff3f6c1f740 1 Processor -- start 2024-04-24T06:34:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.343+0000 7ff3f6c1f740 1 -- start start 2024-04-24T06:34:57.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.343+0000 7ff3f1400640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 0x7ff3cc06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009a10 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d417e0 0x55c825d165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c825c7b1f0 con 0x55c825d41f20 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c825c21350 con 0x55c825d5ee00 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c825c771b0 con 0x55c825d417e0 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36278/0 (socket says 172.21.15.59:36278) 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.344+0000 7ff3f0a00640 1 -- 172.21.15.59:0/797108483 learned_addr learned my addr 172.21.15.59:0/797108483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f1400640 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 -- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d417e0 msgr2=0x55c825d165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d417e0 0x55c825d165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 -- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 msgr2=0x55c825d11790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d11790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 -- 172.21.15.59:0/797108483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c825c75f50 con 0x55c825d41f20 2024-04-24T06:34:57.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f1400640 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80059e0 tx=0x7ff3d80017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eb63311dce826a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3ddc00640 1 -- 172.21.15.59:0/797108483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8010a80 con 0x55c825d41f20 2024-04-24T06:34:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3ddc00640 1 -- 172.21.15.59:0/797108483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d8019070 con 0x55c825d41f20 2024-04-24T06:34:57.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.345+0000 7ff3ddc00640 1 -- 172.21.15.59:0/797108483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8010e40 con 0x55c825d41f20 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/797108483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 msgr2=0x55c825d412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80059e0 tx=0x7ff3d80017a0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/797108483 shutdown_connections 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d417e0 0x55c825d165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d412a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/797108483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/797108483 >> 172.21.15.59:0/797108483 conn(0x55c825d13a00 msgr2=0x55c825d5c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/797108483 shutdown_connections 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.346+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/797108483 wait complete. 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.347+0000 7ff3f6c1f740 1 -- start start 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.347+0000 7ff3f6c1f740 1 Processor -- start 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c825d15bb0 con 0x55c825d41f20 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c825c34690 con 0x55c825d5ee00 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c825d14fa0 con 0x55c825d5d3d0 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.348+0000 7ff3f6c1f740 10 client.0 _ll_register_callbacks cb 0x55c825c70650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36288/0 (socket says 172.21.15.59:36288) 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 -- 172.21.15.59:0/3362005796 learned_addr learned my addr 172.21.15.59:0/3362005796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 -- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 msgr2=0x55c825d5d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d5d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 -- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 msgr2=0x55c825d40ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.349+0000 7ff3f1400640 1 -- 172.21.15.59:0/3362005796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c825c33920 con 0x55c825d41f20 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3f1400640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00164c0 tx=0x7ff3e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd085f006d24250 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0027070 con 0x55c825d41f20 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ff3e0023820 con 0x55c825d41f20 2024-04-24T06:34:57.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3d8019070 con 0x55c825d41f20 2024-04-24T06:34:57.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff3e0032050 con 0x55c825d41f20 2024-04-24T06:34:57.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3f6c1f740 10 client.7197 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:57.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.350+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55c825a577d0 con 0x55c825d41f20 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d8019070 con 0x55c825d41f20 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7ff3e0035020 con 0x55c825d41f20 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 10 client.7197 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 10 client.7197 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 20 client.7197 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d3f730 0x55c825d3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 4 client.7197 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c825c88c10 con 0x55c825d3f730 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.351+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d3f730 0x55c825d3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.352+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d3f730 0x55c825d3fb90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19acf7f6ef1b142 server_cookie=60f841f991876a55 in_seq=0 out_seq=0 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.352+0000 7ff3d2000640 10 client.7197 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3d2000640 1 -- 172.21.15.59:0/3362005796 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c825c88c10 con 0x55c825d3f730 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3d2000640 10 client.7197 handle_command_reply: tid=0 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 client.7197 shutdown 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 msgr2=0x55c825d32030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff3e00164c0 tx=0x7ff3e0004730 comp rx=0 tx=0).stop 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 shutdown_connections 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d3f730 0x55c825d3fb90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3d2000640 0 client.7197 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.353+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3362005796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825d32030 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.354+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 >> 172.21.15.59:0/3362005796 conn(0x55c825d417e0 msgr2=0x55c825d5c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.354+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 shutdown_connections 2024-04-24T06:34:57.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.354+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3362005796 wait complete. 2024-04-24T06:34:57.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.354+0000 7ff3f6c1f740 20 client.7197 trim_cache size 0 max 16384 2024-04-24T06:34:57.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.373+0000 7ff3f6c1f740 1 Processor -- start 2024-04-24T06:34:57.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.374+0000 7ff3f6c1f740 1 -- start start 2024-04-24T06:34:57.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.374+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d41f20 0x55c825d3be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.374+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5d3d0 0x55c825d3c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.374+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1ff290 con 0x55c825d5ee00 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1fe2b0 con 0x55c825d5d3d0 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec0ce5f0 con 0x55c825d41f20 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36300/0 (socket says 172.21.15.59:36300) 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 -- 172.21.15.59:0/3943817350 learned_addr learned my addr 172.21.15.59:0/3943817350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 -- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d41f20 msgr2=0x55c825d3be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d41f20 0x55c825d3be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5d3d0 0x55c825d3c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 -- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5d3d0 msgr2=0x55c825d3c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5d3d0 0x55c825d3c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.375+0000 7ff3f1e00640 1 -- 172.21.15.59:0/3943817350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82598d3f0 con 0x55c825d5ee00 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.376+0000 7ff3f1e00640 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4075250 tx=0x7ff3e4078cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca3686385aadc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.376+0000 7ff3dd200640 1 -- 172.21.15.59:0/3943817350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40a7070 con 0x55c825d5ee00 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.376+0000 7ff3dd200640 1 -- 172.21.15.59:0/3943817350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e4078df0 con 0x55c825d5ee00 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.376+0000 7ff3dd200640 1 -- 172.21.15.59:0/3943817350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e407d1a0 con 0x55c825d5ee00 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3943817350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 msgr2=0x55c825d10740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4075250 tx=0x7ff3e4078cf0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3943817350 shutdown_connections 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d5ee00 0x55c825d10740 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5d3d0 0x55c825d3c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/3943817350 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d41f20 0x55c825d3be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3943817350 >> 172.21.15.59:0/3943817350 conn(0x55c825d417e0 msgr2=0x55c825d0ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3943817350 shutdown_connections 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/3943817350 wait complete. 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.377+0000 7ff3f6c1f740 1 -- start start 2024-04-24T06:34:57.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.378+0000 7ff3f6c1f740 1 Processor -- start 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d58ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec0cec40 con 0x55c825d41f20 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c825d030f0 con 0x55c825d5ee00 2024-04-24T06:34:57.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c825d3e500 con 0x55c825d5d3d0 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f6c1f740 10 client.0 _ll_register_callbacks cb 0x7ff3e0013960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36302/0 (socket says 172.21.15.59:36302) 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 -- 172.21.15.59:0/2964106294 learned_addr learned my addr 172.21.15.59:0/2964106294 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 -- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 msgr2=0x55c825d6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 -- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 msgr2=0x55c825d58ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d58ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.379+0000 7ff3f1400640 1 -- 172.21.15.59:0/2964106294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82594d0f0 con 0x55c825d41f20 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3f1400640 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009350 tx=0x7ff3e0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af23c09febea4f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0027040 con 0x55c825d41f20 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ff3e003fb40 con 0x55c825d41f20 2024-04-24T06:34:57.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e4078df0 con 0x55c825d41f20 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff3e0016d50 con 0x55c825d41f20 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3f6c1f740 10 client.7203 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55c825c40810 con 0x55c825d41f20 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.380+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e4078df0 con 0x55c825d41f20 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7ff3e0035050 con 0x55c825d41f20 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 10 client.7203 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 10 client.7203 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 20 client.7203 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d59430 0x55c825d3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 4 client.7203 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c825c82a80 con 0x55c825d59430 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d59430 0x55c825d3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3f0a00640 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d59430 0x55c825d3d800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d644ed0d36f085 server_cookie=bc8d4d9bcf9211b1 in_seq=0 out_seq=0 2024-04-24T06:34:57.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.381+0000 7ff3d2000640 10 client.7203 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3d2000640 1 -- 172.21.15.59:0/2964106294 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c825c82a80 con 0x55c825d59430 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3d2000640 10 client.7203 handle_command_reply: tid=0 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 client.7203 shutdown 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 msgr2=0x55c825ddaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009350 tx=0x7ff3e0009190 comp rx=0 tx=0).stop 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 shutdown_connections 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c825d59430 0x55c825d3d800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c825d5ee00 0x55c825d58ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3d2000640 0 client.7203 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c825d5d3d0 0x55c825d6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 --2- 172.21.15.59:0/2964106294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c825d41f20 0x55c825ddaa60 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 >> 172.21.15.59:0/2964106294 conn(0x55c825d417e0 msgr2=0x55c825d0ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 shutdown_connections 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.382+0000 7ff3f6c1f740 1 -- 172.21.15.59:0/2964106294 wait complete. 2024-04-24T06:34:57.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f6c1f740 20 client.7203 trim_cache size 0 max 16384 2024-04-24T06:34:57.388 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:57.388 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:34:57.388 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:34:57.388 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 msgr2=0x7ff3cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 0x7ff3cc06ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009a10 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 msgr2=0x7ff3ec1f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005040 tx=0x7ff3e40014a0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 shutdown_connections 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff3cc06c920 0x7ff3cc06ede0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3ec1f5d60 0x7ff3ec1f61e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff3ec159170 0x7ff3ec1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 --2- 172.21.15.59:0/355475886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3d4004d40 0x7ff3ec15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.383+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 >> 172.21.15.59:0/355475886 conn(0x7ff3ec06bdf0 msgr2=0x7ff3ec071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.384+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 shutdown_connections 2024-04-24T06:34:57.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.384+0000 7ff3f2800640 1 -- 172.21.15.59:0/355475886 wait complete. 2024-04-24T06:34:57.402 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:34:57.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:34:57.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.602+0000 7ff742a00640 1 Processor -- start 2024-04-24T06:34:57.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.602+0000 7ff742a00640 1 -- start start 2024-04-24T06:34:57.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.603+0000 7ff742a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c15c3d0 0x7ff73c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.603+0000 7ff742a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff73c0cd590 0x7ff73c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.604+0000 7ff742a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff73c160210 con 0x7ff73c0b4690 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.604+0000 7ff742a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff73c0cd9f0 con 0x7ff73c0cd590 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.604+0000 7ff742a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c1604f0 con 0x7ff73c15c3d0 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.604+0000 7ff740c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff73c0cd590 0x7ff73c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.604+0000 7ff741600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c15c3d0 0x7ff73c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff73c0cd590 0x7ff73c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:54406/0 (socket says 172.21.15.59:54406) 2024-04-24T06:34:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 -- 172.21.15.59:0/2793810051 learned_addr learned my addr 172.21.15.59:0/2793810051 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 -- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c15c3d0 msgr2=0x7ff73c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 --2- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c15c3d0 0x7ff73c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 -- 172.21.15.59:0/2793810051 >> v1:172.21.15.66:6789/0 conn(0x7ff73c0b4690 legacy=0x7ff73c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.605+0000 7ff740c00640 1 -- 172.21.15.59:0/2793810051 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c1607d0 con 0x7ff73c0cd590 2024-04-24T06:34:57.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.607+0000 7ff740c00640 1 --2- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c0cd590 0x7ff73c0b4150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff728009870 tx=0x7ff72802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b36c0a6ba692cace server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.607+0000 7ff73be00640 1 -- 172.21.15.59:0/2793810051 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728037070 con 0x7ff73c0cd590 2024-04-24T06:34:57.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.607+0000 7ff73be00640 1 -- 172.21.15.59:0/2793810051 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728032440 con 0x7ff73c0cd590 2024-04-24T06:34:57.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.607+0000 7ff73be00640 1 -- 172.21.15.59:0/2793810051 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728032810 con 0x7ff73c0cd590 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 -- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c0cd590 msgr2=0x7ff73c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 --2- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c0cd590 0x7ff73c0b4150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff728009870 tx=0x7ff72802a250 comp rx=0 tx=0).stop 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 -- 172.21.15.59:0/2793810051 shutdown_connections 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 --2- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c0cd590 0x7ff73c0b4150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 --2- 172.21.15.59:0/2793810051 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c15c3d0 0x7ff73c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 -- 172.21.15.59:0/2793810051 >> 172.21.15.59:0/2793810051 conn(0x7ff73c0b21a0 msgr2=0x7ff73c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 -- 172.21.15.59:0/2793810051 shutdown_connections 2024-04-24T06:34:57.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.608+0000 7ff742a00640 1 -- 172.21.15.59:0/2793810051 wait complete. 2024-04-24T06:34:57.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.610+0000 7ff742a00640 1 Processor -- start 2024-04-24T06:34:57.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.610+0000 7ff742a00640 1 -- start start 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c0b4690 0x7ff73c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 0x7ff73c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c0b4ae0 con 0x7ff73c0cd590 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c160d90 con 0x7ff73c15c3d0 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff742a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c15c7b0 con 0x7ff73c0b4690 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff740c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff740c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36308/0 (socket says 172.21.15.59:36308) 2024-04-24T06:34:57.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.611+0000 7ff740c00640 1 -- 172.21.15.59:0/2244240837 learned_addr learned my addr 172.21.15.59:0/2244240837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff741600640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c0b4690 0x7ff73c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff742000640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 0x7ff73c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 -- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c0b4690 msgr2=0x7ff73c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c0b4690 0x7ff73c0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 -- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 msgr2=0x7ff73c1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 0x7ff73c1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 -- 172.21.15.59:0/2244240837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c0ce6a0 con 0x7ff73c0cd590 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff742000640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 0x7ff73c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.612+0000 7ff740c00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff728000c00 tx=0x7ff7280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.613+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728037040 con 0x7ff73c0cd590 2024-04-24T06:34:57.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.613+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728042070 con 0x7ff73c0cd590 2024-04-24T06:34:57.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.613+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728030940 con 0x7ff73c0cd590 2024-04-24T06:34:57.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.613+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c0ce980 con 0x7ff73c0cd590 2024-04-24T06:34:57.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.613+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c0cef40 con 0x7ff73c0cd590 2024-04-24T06:34:57.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.614+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c0cca20 con 0x7ff73c0cd590 2024-04-24T06:34:57.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.615+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff728030ae0 con 0x7ff73c0cd590 2024-04-24T06:34:57.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.616+0000 7ff739600640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 0x7ff71c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:57.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.616+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ff7280ac460 con 0x7ff73c0cd590 2024-04-24T06:34:57.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.616+0000 7ff741600640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 0x7ff71c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:57.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.617+0000 7ff741600640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 0x7ff71c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0af2f0 tx=0x7ff73000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:57.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.621+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff73c0cca20 con 0x7ff73c0cd590 2024-04-24T06:34:57.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.867+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff73c04fd90 con 0x7ff73c0cd590 2024-04-24T06:34:57.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.868+0000 7ff739600640 1 -- 172.21.15.59:0/2244240837 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5176 (secure 0 0 0) 0x7ff72807a800 con 0x7ff73c0cd590 2024-04-24T06:34:57.870 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:57.871 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:34:21.996172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:34:57.871 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 msgr2=0x7ff71c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 0x7ff71c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0af2f0 tx=0x7ff73000b040 comp rx=0 tx=0).stop 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 msgr2=0x7ff73c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff728000c00 tx=0x7ff7280046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 shutdown_connections 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff71c06c920 0x7ff71c06ede0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff73c15c3d0 0x7ff73c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff73c0cd590 0x7ff73c159110 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 --2- 172.21.15.59:0/2244240837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff73c0b4690 0x7ff73c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:57.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 >> 172.21.15.59:0/2244240837 conn(0x7ff73c0b21a0 msgr2=0x7ff73c0af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:57.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 shutdown_connections 2024-04-24T06:34:57.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:57.872+0000 7ff742a00640 1 -- 172.21.15.59:0/2244240837 wait complete. 2024-04-24T06:34:57.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-24T06:34:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.107+0000 7fe1cb200640 1 Processor -- start 2024-04-24T06:34:58.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.107+0000 7fe1cb200640 1 -- start start 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.108+0000 7fe1cb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.108+0000 7fe1cb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe1cc06edd0 0x7fe1cc06f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.108+0000 7fe1cb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc06c0b0 con 0x7fe1cc15a360 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.108+0000 7fe1cb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe1cc0d1150 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.108+0000 7fe1cb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe1cc15ce90 con 0x7fe1cc06edd0 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.109+0000 7fe1c9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.109+0000 7fe1c9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36314/0 (socket says 172.21.15.59:36314) 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.109+0000 7fe1c9400640 1 -- 172.21.15.59:0/763576591 learned_addr learned my addr 172.21.15.59:0/763576591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.109+0000 7fe1c9e00640 1 --1- 172.21.15.59:0/763576591 >> v1:172.21.15.59:6789/0 conn(0x7fe1cc15a360 0x7fe1cc15d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50618/0 (socket says 172.21.15.59:50618) 2024-04-24T06:34:58.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.109+0000 7fe1ca800640 1 --2- 172.21.15.59:0/763576591 >> v2:172.21.15.66:3300/0 conn(0x7fe1cc06edd0 0x7fe1cc06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.110+0000 7fe1c9400640 1 -- 172.21.15.59:0/763576591 >> v2:172.21.15.66:3300/0 conn(0x7fe1cc06edd0 msgr2=0x7fe1cc06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.110+0000 7fe1c9400640 1 --2- 172.21.15.59:0/763576591 >> v2:172.21.15.66:3300/0 conn(0x7fe1cc06edd0 0x7fe1cc06f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.110+0000 7fe1c9400640 1 -- 172.21.15.59:0/763576591 >> v1:172.21.15.59:6789/0 conn(0x7fe1cc15a360 legacy=0x7fe1cc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.110+0000 7fe1c9400640 1 -- 172.21.15.59:0/763576591 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc06c390 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.111+0000 7fe1c9400640 1 --2- 172.21.15.59:0/763576591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc06da90 tx=0x7fe1b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c522b02fbc3296ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.111+0000 7fe1c8a00640 1 -- 172.21.15.59:0/763576591 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8037070 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.111+0000 7fe1c8a00640 1 -- 172.21.15.59:0/763576591 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b8032440 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.111+0000 7fe1c8a00640 1 -- 172.21.15.59:0/763576591 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8032810 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 -- 172.21.15.59:0/763576591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 msgr2=0x7fe1cc06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 --2- 172.21.15.59:0/763576591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc06da90 tx=0x7fe1b802a270 comp rx=0 tx=0).stop 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 -- 172.21.15.59:0/763576591 shutdown_connections 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 --2- 172.21.15.59:0/763576591 >> v2:172.21.15.66:3300/0 conn(0x7fe1cc06edd0 0x7fe1cc06f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 --2- 172.21.15.59:0/763576591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc06b9d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 -- 172.21.15.59:0/763576591 >> 172.21.15.59:0/763576591 conn(0x7fe1cc0b2f30 msgr2=0x7fe1cc0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 -- 172.21.15.59:0/763576591 shutdown_connections 2024-04-24T06:34:58.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.112+0000 7fe1cb200640 1 -- 172.21.15.59:0/763576591 wait complete. 2024-04-24T06:34:58.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.114+0000 7fe1cb200640 1 Processor -- start 2024-04-24T06:34:58.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.114+0000 7fe1cb200640 1 -- start start 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1cc06edd0 0x7fe1cc14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1cc15a360 0x7fe1cc1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc15dbb0 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc15da30 con 0x7fe1cc06edd0 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1cb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc070420 con 0x7fe1cc15a360 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36330/0 (socket says 172.21.15.59:36330) 2024-04-24T06:34:58.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 -- 172.21.15.59:0/841154201 learned_addr learned my addr 172.21.15.59:0/841154201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 -- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1cc15a360 msgr2=0x7fe1cc1f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1cc15a360 0x7fe1cc1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9400640 1 -- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1cc06edd0 msgr2=0x7fe1cc14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.115+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1cc06edd0 0x7fe1cc14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c9400640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1cc06edd0 0x7fe1cc14aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c9400640 1 -- 172.21.15.59:0/841154201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc143bb0 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c9400640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c2000640 1 -- 172.21.15.59:0/841154201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8041020 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c2000640 1 -- 172.21.15.59:0/841154201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b80306c0 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc14b6f0 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.116+0000 7fe1c2000640 1 -- 172.21.15.59:0/841154201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8037040 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.117+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc15b680 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.119+0000 7fe1c2000640 1 -- 172.21.15.59:0/841154201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1b8044020 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.120+0000 7fe1c2000640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 0x7fe1ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.120+0000 7fe1c2000640 1 -- 172.21.15.59:0/841154201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe1b80ac440 con 0x7fe1cc0b3a50 2024-04-24T06:34:58.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.120+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 0x7fe1ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.121+0000 7fe1d37c4740 1 Processor -- start 2024-04-24T06:34:58.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.121+0000 7fe1d37c4740 1 -- start start 2024-04-24T06:34:58.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.121+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 0x7fe1ac06edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc1504d0 tx=0x7fe1bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae05600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499addaf00 0x56499ae21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56499ad3f190 con 0x56499ae22ac0 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56499ace5030 con 0x56499ae05220 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56499ad3b150 con 0x56499addaf00 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36334/0 (socket says 172.21.15.59:36334) 2024-04-24T06:34:58.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.122+0000 7fe1c9400640 1 -- 172.21.15.59:0/3131257594 learned_addr learned my addr 172.21.15.59:0/3131257594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1ca800640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499addaf00 0x56499ae21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae05600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 -- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499addaf00 msgr2=0x56499ae21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499addaf00 0x56499ae21180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 -- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 msgr2=0x56499ae05600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae05600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 -- 172.21.15.59:0/3131257594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499accdb60 con 0x56499ae22ac0 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae05600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80aa840 tx=0x7fe1b8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d4df9735f4933e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1b6600640 1 -- 172.21.15.59:0/3131257594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b807a0d0 con 0x56499ae22ac0 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1b6600640 1 -- 172.21.15.59:0/3131257594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b807e480 con 0x56499ae22ac0 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.123+0000 7fe1b6600640 1 -- 172.21.15.59:0/3131257594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8082cb0 con 0x56499ae22ac0 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3131257594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 msgr2=0x56499ae23010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80aa840 tx=0x7fe1b8009240 comp rx=0 tx=0).stop 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3131257594 shutdown_connections 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499addaf00 0x56499ae21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae23010 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3131257594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae05600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3131257594 >> 172.21.15.59:0/3131257594 conn(0x56499add7830 msgr2=0x56499ae20030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3131257594 shutdown_connections 2024-04-24T06:34:58.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.124+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3131257594 wait complete. 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.125+0000 7fe1d37c4740 1 -- start start 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.125+0000 7fe1d37c4740 1 Processor -- start 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae01ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56499ad3f190 con 0x56499ae22ac0 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56499acf8500 con 0x56499ae05220 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56499addbca0 con 0x56499ae30410 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1d37c4740 10 client.0 _ll_register_callbacks cb 0x56499ad35270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36336/0 (socket says 172.21.15.59:36336) 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.126+0000 7fe1c9400640 1 -- 172.21.15.59:0/4081888150 learned_addr learned my addr 172.21.15.59:0/4081888150 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae01ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 -- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 msgr2=0x56499ae235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 -- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 msgr2=0x56499ae01ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae01ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 -- 172.21.15.59:0/4081888150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499ab1d7c0 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1c9400640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80aa810 tx=0x7fe1b8007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bc3502ef8a3ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b807a0d0 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe1b8076740 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.127+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b807e480 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe1b8082820 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 10 client.7221 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x56499ab7df50 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b807e480 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fe1b803a030 con 0x56499ae22ac0 2024-04-24T06:34:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 10 client.7221 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 10 client.7221 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 20 client.7221 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae21190 0x56499ae21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 4 client.7221 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499ad4cbb0 con 0x56499ae21190 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.128+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae21190 0x56499ae21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.129+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae21190 0x56499ae21570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a25cb98063092c server_cookie=fc6d77acfbe8c64 in_seq=0 out_seq=0 2024-04-24T06:34:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.129+0000 7fe1aaa00640 10 client.7221 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1aaa00640 1 -- 172.21.15.59:0/4081888150 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56499ad4cbb0 con 0x56499ae21190 2024-04-24T06:34:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1aaa00640 10 client.7221 handle_command_reply: tid=0 2024-04-24T06:34:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 client.7221 shutdown 2024-04-24T06:34:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 msgr2=0x56499ae08cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80aa810 tx=0x7fe1b8007000 comp rx=0 tx=0).stop 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 shutdown_connections 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae21190 0x56499ae21570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae08cd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/4081888150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae01ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1aaa00640 0 client.7221 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 >> 172.21.15.59:0/4081888150 conn(0x56499addaf00 msgr2=0x56499ae042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 shutdown_connections 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 1 -- 172.21.15.59:0/4081888150 wait complete. 2024-04-24T06:34:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.130+0000 7fe1d37c4740 20 client.7221 trim_cache size 0 max 16384 2024-04-24T06:34:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.145+0000 7fe1d37c4740 1 Processor -- start 2024-04-24T06:34:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.145+0000 7fe1d37c4740 1 -- start start 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae05220 0x56499ae223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae22ac0 0x56499ae20030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc2000e0 con 0x56499ae30410 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0af040 con 0x56499ae22ac0 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.146+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ffac0 con 0x56499ae05220 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36338/0 (socket says 172.21.15.59:36338) 2024-04-24T06:34:58.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 -- 172.21.15.59:0/3077729715 learned_addr learned my addr 172.21.15.59:0/3077729715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae05220 0x56499ae223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae22ac0 0x56499ae20030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 -- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae05220 msgr2=0x56499ae223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae05220 0x56499ae223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 -- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae22ac0 msgr2=0x56499ae20030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae22ac0 0x56499ae20030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.147+0000 7fe1ca800640 1 -- 172.21.15.59:0/3077729715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499aa110b0 con 0x56499ae30410 2024-04-24T06:34:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.148+0000 7fe1ca800640 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009f10 tx=0x7fe1c4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c042f0b7d826c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.148+0000 7fe1b5c00640 1 -- 172.21.15.59:0/3077729715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4013780 con 0x56499ae30410 2024-04-24T06:34:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.148+0000 7fe1b5c00640 1 -- 172.21.15.59:0/3077729715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4019070 con 0x56499ae30410 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.148+0000 7fe1b5c00640 1 -- 172.21.15.59:0/3077729715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4013b40 con 0x56499ae30410 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3077729715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 msgr2=0x56499ae20570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009f10 tx=0x7fe1c4007700 comp rx=0 tx=0).stop 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3077729715 shutdown_connections 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae30410 0x56499ae20570 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae22ac0 0x56499ae20030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3077729715 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae05220 0x56499ae223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3077729715 >> 172.21.15.59:0/3077729715 conn(0x56499addaf00 msgr2=0x56499ae1f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3077729715 shutdown_connections 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.149+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3077729715 wait complete. 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.150+0000 7fe1d37c4740 1 -- start start 2024-04-24T06:34:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.150+0000 7fe1d37c4740 1 Processor -- start 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae99aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56499ad3f190 con 0x56499ae22ac0 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56499addbca0 con 0x56499ae05220 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56499acf8500 con 0x56499ae30410 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1d37c4740 10 client.0 _ll_register_callbacks cb 0x7fe1b8079c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36348/0 (socket says 172.21.15.59:36348) 2024-04-24T06:34:58.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 -- 172.21.15.59:0/3431083926 learned_addr learned my addr 172.21.15.59:0/3431083926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 -- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 msgr2=0x56499ae165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 -- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 msgr2=0x56499ae99aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae99aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.151+0000 7fe1c9400640 1 -- 172.21.15.59:0/3431083926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499ae33510 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1c9400640 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe1b807a010 tx=0x7fe1b80ac040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775990a7d4eb8b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b807a0d0 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe1b8079520 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c4019070 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe1b808c050 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1d37c4740 10 client.7227 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x56499ae33510 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.152+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4019070 con 0x56499ae22ac0 2024-04-24T06:34:58.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fe1b8075e20 con 0x56499ae22ac0 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 10 client.7227 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 10 client.7227 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 20 client.7227 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae2edf0 0x56499ae432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 4 client.7227 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56499ad46a20 con 0x56499ae2edf0 2024-04-24T06:34:58.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.153+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae2edf0 0x56499ae432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1c9e00640 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae2edf0 0x56499ae432d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455635fca88d1e35 server_cookie=1d87e6e1146e81a1 in_seq=0 out_seq=0 2024-04-24T06:34:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1aaa00640 10 client.7227 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1aaa00640 1 -- 172.21.15.59:0/3431083926 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x56499ad46a20 con 0x56499ae2edf0 2024-04-24T06:34:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1aaa00640 10 client.7227 handle_command_reply: tid=0 2024-04-24T06:34:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 client.7227 shutdown 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 msgr2=0x56499ae160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe1b807a010 tx=0x7fe1b80ac040 comp rx=0 tx=0).stop 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 shutdown_connections 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x56499ae2edf0 0x56499ae432d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56499ae30410 0x56499ae165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56499ae22ac0 0x56499ae160b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1aaa00640 0 client.7227 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 --2- 172.21.15.59:0/3431083926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499ae05220 0x56499ae99aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 >> 172.21.15.59:0/3431083926 conn(0x56499addaf00 msgr2=0x56499add0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 shutdown_connections 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.154+0000 7fe1d37c4740 1 -- 172.21.15.59:0/3431083926 wait complete. 2024-04-24T06:34:58.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1d37c4740 20 client.7227 trim_cache size 0 max 16384 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "dentry", 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 948498329, 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "frag": "*", 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "file_to_be_damaged", 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_id": "head", 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:58.160 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 msgr2=0x7fe1ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 0x7fe1ac06edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc1504d0 tx=0x7fe1bc009040 comp rx=0 tx=0).stop 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 msgr2=0x7fe1cc14af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b80046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 shutdown_connections 2024-04-24T06:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe1ac06c8f0 0x7fe1ac06edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1cc15a360 0x7fe1cc1f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1cc0b3a50 0x7fe1cc14af90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 --2- 172.21.15.59:0/841154201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1cc06edd0 0x7fe1cc14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 >> 172.21.15.59:0/841154201 conn(0x7fe1cc0b2f30 msgr2=0x7fe1cc14d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.155+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 shutdown_connections 2024-04-24T06:34:58.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.156+0000 7fe1cb200640 1 -- 172.21.15.59:0/841154201 wait complete. 2024-04-24T06:34:58.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:34:58.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.358+0000 7f0baca00640 1 Processor -- start 2024-04-24T06:34:58.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.358+0000 7f0baca00640 1 -- start start 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0baca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0ba815b450 0x7f0ba80b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0baca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 0x7f0ba80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0baca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba80b3870 con 0x7f0ba80b3080 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0baca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba80b41a0 con 0x7f0ba815b450 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0baca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80b3b50 con 0x7f0ba806f3f0 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0ba7200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0ba80b3080 0x7f0ba80ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50620/0 (socket says 172.21.15.59:50620) 2024-04-24T06:34:58.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.360+0000 7f0ba7200640 1 -- 172.21.15.59:0/2074502389 learned_addr learned my addr 172.21.15.59:0/2074502389 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5e00640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 0x7f0ba80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba6800640 1 --2- 172.21.15.59:0/2074502389 >> v2:172.21.15.66:3300/0 conn(0x7f0ba815b450 0x7f0ba80b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107434465 0 0) 0x7f0ba80b3870 con 0x7f0ba80b3080 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b8c003520 con 0x7f0ba80b3080 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293457556 0 0) 0x7f0b8c003520 con 0x7f0ba80b3080 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 msgr2=0x7f0ba80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 0x7f0ba80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 >> v2:172.21.15.66:3300/0 conn(0x7f0ba815b450 msgr2=0x7f0ba80b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 --2- 172.21.15.59:0/2074502389 >> v2:172.21.15.66:3300/0 conn(0x7f0ba815b450 0x7f0ba80b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80b3e30 con 0x7f0ba80b3080 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0ba0001670 con 0x7f0ba80b3080 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.361+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 >> v1:172.21.15.59:6789/0 conn(0x7f0ba80b3080 legacy=0x7f0ba80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 0x7f0b8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80b41a0 con 0x7f0b8c0039b0 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80b3b50 con 0x7f0b8c004d40 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c00dbd0 con 0x7f0ba806f3f0 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 -- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 msgr2=0x7f0b8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba806f3f0 0x7f0b8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 -- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c004d40 msgr2=0x7f0b8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba7200640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.362+0000 7f0ba6800640 1 -- 172.21.15.59:0/2074502389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c003520 con 0x7f0b8c0039b0 2024-04-24T06:34:58.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.363+0000 7f0ba6800640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fc0 tx=0x7f0b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c29df4a9177ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b98009c10 con 0x7f0b8c0039b0 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b98036070 con 0x7f0b8c0039b0 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0ba5400640 1 -- 172.21.15.59:0/2074502389 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9802d070 con 0x7f0b8c0039b0 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0baca00640 1 -- 172.21.15.59:0/2074502389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8c0039b0 msgr2=0x7f0b8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0baca00640 1 --2- 172.21.15.59:0/2074502389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fc0 tx=0x7f0b8c005830 comp rx=0 tx=0).stop 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0ba7200640 1 -- 172.21.15.59:0/2074502389 reap_dead start 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0baca00640 1 -- 172.21.15.59:0/2074502389 shutdown_connections 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.364+0000 7f0baca00640 1 -- 172.21.15.59:0/2074502389 >> 172.21.15.59:0/2074502389 conn(0x7f0ba80b21a0 msgr2=0x7f0ba80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.365+0000 7f0baca00640 1 -- 172.21.15.59:0/2074502389 shutdown_connections 2024-04-24T06:34:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.365+0000 7f0baca00640 1 -- 172.21.15.59:0/2074502389 wait complete. 2024-04-24T06:34:58.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.366+0000 7f0baca00640 1 Processor -- start 2024-04-24T06:34:58.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.367+0000 7f0baca00640 1 -- start start 2024-04-24T06:34:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0039b0 0x7f0ba8156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba815e640 0x7f0ba815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80b4ff0 con 0x7f0ba815d650 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba815b830 con 0x7f0b8c0039b0 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0baca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80ad770 con 0x7f0ba815e640 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36362/0 (socket says 172.21.15.59:36362) 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 -- 172.21.15.59:0/3175393731 learned_addr learned my addr 172.21.15.59:0/3175393731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba5e00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba815e640 0x7f0ba815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba6800640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0039b0 0x7f0ba8156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 -- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba815e640 msgr2=0x7f0ba815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba815e640 0x7f0ba815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 -- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0039b0 msgr2=0x7f0ba8156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0039b0 0x7f0ba8156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.368+0000 7f0ba7200640 1 -- 172.21.15.59:0/3175393731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80c4070 con 0x7f0ba815d650 2024-04-24T06:34:58.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0ba7200640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000d4c0 tx=0x7f0ba000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0007440 con 0x7f0ba815d650 2024-04-24T06:34:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba00075e0 con 0x7f0ba815d650 2024-04-24T06:34:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba80c4350 con 0x7f0ba815d650 2024-04-24T06:34:58.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0004a20 con 0x7f0ba815d650 2024-04-24T06:34:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.369+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba815dff0 con 0x7f0ba815d650 2024-04-24T06:34:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.370+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba804fd90 con 0x7f0ba815d650 2024-04-24T06:34:58.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.371+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ba001b020 con 0x7f0ba815d650 2024-04-24T06:34:58.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.372+0000 7f0b9ea00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 0x7f0b8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.373+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0ba008b6d0 con 0x7f0ba815d650 2024-04-24T06:34:58.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.373+0000 7f0ba6800640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 0x7f0b8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.373+0000 7f0ba6800640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 0x7f0b8406edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b98035000 tx=0x7f0b9802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.377+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ba00599d0 con 0x7f0ba815d650 2024-04-24T06:34:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.624+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0ba806e870 con 0x7f0ba815d650 2024-04-24T06:34:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.625+0000 7f0b9ea00640 1 -- 172.21.15.59:0/3175393731 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f0ba00592d0 con 0x7f0ba815d650 2024-04-24T06:34:58.627 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:34:58.627 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 msgr2=0x7f0b8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 0x7f0b8406edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b98035000 tx=0x7f0b9802f000 comp rx=0 tx=0).stop 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 msgr2=0x7f0ba815dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000d4c0 tx=0x7f0ba000d980 comp rx=0 tx=0).stop 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 shutdown_connections 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0b8406c8f0 0x7f0b8406edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ba815d650 0x7f0ba815dab0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0ba815e640 0x7f0ba815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 --2- 172.21.15.59:0/3175393731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b8c0039b0 0x7f0ba8156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 >> 172.21.15.59:0/3175393731 conn(0x7f0ba80b21a0 msgr2=0x7f0ba81585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 shutdown_connections 2024-04-24T06:34:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.628+0000 7f0baca00640 1 -- 172.21.15.59:0/3175393731 wait complete. 2024-04-24T06:34:58.651 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:34:58.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:58.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-24T06:34:58.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.865+0000 7f0a5c600640 1 Processor -- start 2024-04-24T06:34:58.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.865+0000 7f0a5c600640 1 -- start start 2024-04-24T06:34:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a5c600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a540cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a5c600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a540cd610 0x7f0a5406e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a5c600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a540cda30 con 0x7f0a5406e840 2024-04-24T06:34:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a5c600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0a540cf670 con 0x7f0a540cd610 2024-04-24T06:34:58.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a5c600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540cf340 con 0x7f0a541431b0 2024-04-24T06:34:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.867+0000 7f0a59200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a540cd610 0x7f0a5406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a540cd610 0x7f0a5406e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36372/0 (socket says 172.21.15.59:36372) 2024-04-24T06:34:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 -- 172.21.15.59:0/2107154984 learned_addr learned my addr 172.21.15.59:0/2107154984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 -- 172.21.15.59:0/2107154984 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 msgr2=0x7f0a540cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a540cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a540cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 -- 172.21.15.59:0/2107154984 >> v1:172.21.15.66:6789/0 conn(0x7f0a5406e840 legacy=0x7f0a54071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59200640 1 -- 172.21.15.59:0/2107154984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5406eca0 con 0x7f0a540cd610 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.868+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a540cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.869+0000 7f0a59200640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a5406e300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009870 tx=0x7f0a4002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b67a93f379431af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.870+0000 7f0a53e00640 1 -- 172.21.15.59:0/2107154984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037070 con 0x7f0a540cd610 2024-04-24T06:34:58.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.870+0000 7f0a53e00640 1 -- 172.21.15.59:0/2107154984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a40032440 con 0x7f0a540cd610 2024-04-24T06:34:58.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.870+0000 7f0a53e00640 1 -- 172.21.15.59:0/2107154984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40032810 con 0x7f0a540cd610 2024-04-24T06:34:58.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.870+0000 7f0a5c600640 1 -- 172.21.15.59:0/2107154984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 msgr2=0x7f0a5406e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.870+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a5406e300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009870 tx=0x7f0a4002a2a0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 -- 172.21.15.59:0/2107154984 shutdown_connections 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a5406e300 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2107154984 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a540cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 -- 172.21.15.59:0/2107154984 >> 172.21.15.59:0/2107154984 conn(0x7f0a5406b9d0 msgr2=0x7f0a5406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 -- 172.21.15.59:0/2107154984 shutdown_connections 2024-04-24T06:34:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.871+0000 7f0a5c600640 1 -- 172.21.15.59:0/2107154984 wait complete. 2024-04-24T06:34:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.873+0000 7f0a5c600640 1 Processor -- start 2024-04-24T06:34:58.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.873+0000 7f0a5c600640 1 -- start start 2024-04-24T06:34:58.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a5406e840 0x7f0a541423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a54159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5406f4d0 con 0x7f0a540cd610 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5406f1f0 con 0x7f0a5406e840 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.874+0000 7f0a5c600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540d0b10 con 0x7f0a541431b0 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46200/0 (socket says 172.21.15.59:46200) 2024-04-24T06:34:58.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 -- 172.21.15.59:0/2718951072 learned_addr learned my addr 172.21.15.59:0/2718951072 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a5406e840 0x7f0a541423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 -- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 msgr2=0x7f0a54159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a5a600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a54159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a54159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 -- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a5406e840 msgr2=0x7f0a541423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a5406e840 0x7f0a541423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.875+0000 7f0a59200640 1 -- 172.21.15.59:0/2718951072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540c54d0 con 0x7f0a540cd610 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a59200640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0a40000c00 tx=0x7f0a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a51600640 1 -- 172.21.15.59:0/2718951072 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037040 con 0x7f0a540cd610 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a51600640 1 -- 172.21.15.59:0/2718951072 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a40042070 con 0x7f0a540cd610 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a51600640 1 -- 172.21.15.59:0/2718951072 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40030bb0 con 0x7f0a540cd610 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540c57b0 con 0x7f0a540cd610 2024-04-24T06:34:58.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.876+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a540c5d70 con 0x7f0a540cd610 2024-04-24T06:34:58.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.878+0000 7f0a51600640 1 -- 172.21.15.59:0/2718951072 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a40047460 con 0x7f0a540cd610 2024-04-24T06:34:58.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.879+0000 7f0a51600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 0x7f0a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.879+0000 7f0a51600640 1 -- 172.21.15.59:0/2718951072 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0a400ac840 con 0x7f0a540cd610 2024-04-24T06:34:58.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.880+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 0x7f0a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.880+0000 7f0a5f5ac740 1 Processor -- start 2024-04-24T06:34:58.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.881+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 0x7f0a3406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800a950 tx=0x7f0a48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.881+0000 7f0a5f5ac740 1 -- start start 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.881+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c58d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c2cc10 0x563ff2c552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563ff2b92070 con 0x563ff2c55820 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563ff2b38580 con 0x563ff2c2cc10 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x563ff2b8e030 con 0x563ff2c54f00 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46216/0 (socket says 172.21.15.59:46216) 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 -- 172.21.15.59:0/3366742239 learned_addr learned my addr 172.21.15.59:0/3366742239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a59200640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c2cc10 0x563ff2c552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c58d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 -- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 msgr2=0x563ff2c58d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c58d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 -- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c2cc10 msgr2=0x563ff2c552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c2cc10 0x563ff2c552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a5a600640 1 -- 172.21.15.59:0/3366742239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff296ca50 con 0x563ff2c55820 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.882+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c58d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.883+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001dc0 tx=0x7f0a4c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b46e06a678dd15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.883+0000 7f0a46000640 1 -- 172.21.15.59:0/3366742239 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c018070 con 0x563ff2c55820 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.883+0000 7f0a46000640 1 -- 172.21.15.59:0/3366742239 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00ddd0 con 0x563ff2c55820 2024-04-24T06:34:58.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.883+0000 7f0a46000640 1 -- 172.21.15.59:0/3366742239 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c003b90 con 0x563ff2c55820 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3366742239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 msgr2=0x563ff2c580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c001dc0 tx=0x7f0a4c00d520 comp rx=0 tx=0).stop 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3366742239 shutdown_connections 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c580b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c2cc10 0x563ff2c552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3366742239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c58d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3366742239 >> 172.21.15.59:0/3366742239 conn(0x563ff2c2a810 msgr2=0x563ff2c730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3366742239 shutdown_connections 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.884+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3366742239 wait complete. 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.885+0000 7f0a5f5ac740 1 -- start start 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.885+0000 7f0a5f5ac740 1 Processor -- start 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c73aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c74bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563ff2c28b60 con 0x563ff2c75110 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563ff2b4b860 con 0x563ff2c55820 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x563ff2c285a0 con 0x563ff2c54f00 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5f5ac740 10 client.0 _ll_register_callbacks cb 0x563ff2b39ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46224/0 (socket says 172.21.15.59:46224) 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.886+0000 7f0a5a600640 1 -- 172.21.15.59:0/3197073313 learned_addr learned my addr 172.21.15.59:0/3197073313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 -- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 msgr2=0x563ff2c73aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c73aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 -- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 msgr2=0x563ff2c74bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c74bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 -- 172.21.15.59:0/3197073313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff28643e0 con 0x563ff2c75110 2024-04-24T06:34:58.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5a600640 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c012000 tx=0x7f0a4c001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cfdb185934487c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c023070 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0a4c0050d0 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.887+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4c00ddd0 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c02e050 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 10 client.7245 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x563ff29cd1e0 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00ddd0 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0a4c01e040 con 0x563ff2c75110 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 10 client.7245 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 10 client.7245 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 20 client.7245 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2c8b100 0x563ff2c8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 4 client.7245 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ff2b9fa90 con 0x563ff2c8b100 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.888+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2c8b100 0x563ff2c8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.889+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2c8b100 0x563ff2c8b590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4857c6e35426f76b server_cookie=968a5528cd92fed0 in_seq=0 out_seq=0 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.889+0000 7f0a3aa00640 10 client.7245 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a3aa00640 1 -- 172.21.15.59:0/3197073313 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ff2b9fa90 con 0x563ff2c8b100 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a3aa00640 10 client.7245 handle_command_reply: tid=0 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 client.7245 shutdown 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 msgr2=0x563ff2c4b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c012000 tx=0x7f0a4c001330 comp rx=0 tx=0).stop 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 shutdown_connections 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2c8b100 0x563ff2c8b590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c75110 0x563ff2c4b5a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c74bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3197073313 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c54f00 0x563ff2c73aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a3aa00640 0 client.7245 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 >> 172.21.15.59:0/3197073313 conn(0x563ff2c2cc10 msgr2=0x563ff2c2dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 shutdown_connections 2024-04-24T06:34:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.890+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3197073313 wait complete. 2024-04-24T06:34:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.891+0000 7f0a5f5ac740 20 client.7245 trim_cache size 0 max 16384 2024-04-24T06:34:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.910+0000 7f0a5f5ac740 1 Processor -- start 2024-04-24T06:34:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.910+0000 7f0a5f5ac740 1 -- start start 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c2d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2c2bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b54a0 con 0x563ff2c54f00 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a541f3b70 con 0x563ff2c55820 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54157450 con 0x563ff2c75110 2024-04-24T06:34:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46226/0 (socket says 172.21.15.59:46226) 2024-04-24T06:34:58.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.911+0000 7f0a59c00640 1 -- 172.21.15.59:0/3041754856 learned_addr learned my addr 172.21.15.59:0/3041754856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 -- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 msgr2=0x563ff2c2bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2c2bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 -- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 msgr2=0x563ff2c2d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c2d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 -- 172.21.15.59:0/3041754856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff283d4a0 con 0x563ff2c54f00 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a59c00640 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800afd0 tx=0x7f0a48011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927782dd7e850b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a45600640 1 -- 172.21.15.59:0/3041754856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4802b070 con 0x563ff2c54f00 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a45600640 1 -- 172.21.15.59:0/3041754856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a48013db0 con 0x563ff2c54f00 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.912+0000 7f0a45600640 1 -- 172.21.15.59:0/3041754856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480253b0 con 0x563ff2c54f00 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3041754856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 msgr2=0x563ff2c9b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800afd0 tx=0x7f0a48011000 comp rx=0 tx=0).stop 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3041754856 shutdown_connections 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2c2bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c55820 0x563ff2c2d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/3041754856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c54f00 0x563ff2c9b4b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3041754856 >> 172.21.15.59:0/3041754856 conn(0x563ff2c2cc10 msgr2=0x563ff2c2f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3041754856 shutdown_connections 2024-04-24T06:34:58.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.913+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/3041754856 wait complete. 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.914+0000 7f0a5f5ac740 1 -- start start 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.914+0000 7f0a5f5ac740 1 Processor -- start 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c54f00 0x563ff2c5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2cf1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563ff2c2d1d0 con 0x563ff2c55820 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a541f3520 con 0x563ff2c54f00 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x563ff2c23e30 con 0x563ff2c75110 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a5f5ac740 10 client.0 _ll_register_callbacks cb 0x7f0a4c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a59200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a59200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46238/0 (socket says 172.21.15.59:46238) 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.915+0000 7f0a59200640 1 -- 172.21.15.59:0/2064471218 learned_addr learned my addr 172.21.15.59:0/2064471218 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 -- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 msgr2=0x563ff2cf1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2cf1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 -- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c54f00 msgr2=0x563ff2c5b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c54f00 0x563ff2c5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c54f00 0x563ff2c5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 -- 172.21.15.59:0/2064471218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff2c89d70 con 0x563ff2c55820 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a59200640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0a4007a460 tx=0x7f0a40045420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aacf240b5f6ad66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.916+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40092020 con 0x563ff2c55820 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0a400a5d70 con 0x563ff2c55820 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a48013db0 con 0x563ff2c55820 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0a4007e880 con 0x563ff2c55820 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 10 client.7251 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:58.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x563ff2cf3420 con 0x563ff2c55820 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a48013db0 con 0x563ff2c55820 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0a4003a030 con 0x563ff2c55820 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 10 client.7251 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 10 client.7251 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 20 client.7251 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2cf1940 0x563ff2c4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 4 client.7251 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x563ff2b99900 con 0x563ff2cf1940 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.917+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2cf1940 0x563ff2c4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.918+0000 7f0a59c00640 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2cf1940 0x563ff2c4be70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70083e329a759c3 server_cookie=910b5da4be36e105 in_seq=0 out_seq=0 2024-04-24T06:34:58.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.918+0000 7f0a3aa00640 10 client.7251 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a3aa00640 1 -- 172.21.15.59:0/2064471218 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ff2b99900 con 0x563ff2cf1940 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a3aa00640 10 client.7251 handle_command_reply: tid=0 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 client.7251 shutdown 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 msgr2=0x563ff2c74960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0a4007a460 tx=0x7f0a40045420 comp rx=0 tx=0).stop 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 shutdown_connections 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x563ff2cf1940 0x563ff2c4be70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x563ff2c75110 0x563ff2cf1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563ff2c55820 0x563ff2c74960 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 --2- 172.21.15.59:0/2064471218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ff2c54f00 0x563ff2c5b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 >> 172.21.15.59:0/2064471218 conn(0x563ff2c2cc10 msgr2=0x563ff2c9b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.919+0000 7f0a3aa00640 0 client.7251 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 shutdown_connections 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5f5ac740 1 -- 172.21.15.59:0/2064471218 wait complete. 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5f5ac740 20 client.7251 trim_cache size 0 max 16384 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "86bdc30e-5058-4cb7-9a63-0e50c3d0da72", 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:34:58.925 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:58.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 msgr2=0x7f0a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 0x7f0a3406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800a950 tx=0x7f0a48008040 comp rx=0 tx=0).stop 2024-04-24T06:34:58.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 msgr2=0x7f0a54159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0a40000c00 tx=0x7f0a400046c0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 shutdown_connections 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a3406c8f0 0x7f0a3406edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0a541431b0 0x7f0a54159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a540cd610 0x7f0a54159120 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.920+0000 7f0a5c600640 1 --2- 172.21.15.59:0/2718951072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a5406e840 0x7f0a541423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.921+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 >> 172.21.15.59:0/2718951072 conn(0x7f0a5406b9d0 msgr2=0x7f0a54071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.921+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 shutdown_connections 2024-04-24T06:34:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:58.921+0000 7f0a5c600640 1 -- 172.21.15.59:0/2718951072 wait complete. 2024-04-24T06:34:58.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:34:58.941 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-24T06:34:59.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.152+0000 7f5b0ec00640 1 Processor -- start 2024-04-24T06:34:59.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.152+0000 7f5b0ec00640 1 -- start start 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5b080af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ec00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b08160120 con 0x7f5b080d0ce0 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b08160400 con 0x7f5b0815c410 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5b081606e0 con 0x7f5b08072c80 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ce00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5b080d0ce0 0x7f5b080af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50148/0 (socket says 172.21.15.59:50148) 2024-04-24T06:34:59.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.154+0000 7f5b0ce00640 1 -- 172.21.15.59:0/2954206493 learned_addr learned my addr 172.21.15.59:0/2954206493 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:34:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b0e200640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5b080af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284442144 0 0) 0x7f5b08160120 con 0x7f5b080d0ce0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af0003520 con 0x7f5b080d0ce0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275508131 0 0) 0x7f5af0003520 con 0x7f5b080d0ce0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 msgr2=0x7f5b080af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5b080af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 >> v1:172.21.15.66:6789/0 conn(0x7f5b0815c410 legacy=0x7f5b080d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b081609c0 con 0x7f5b080d0ce0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5af4001670 con 0x7f5b080d0ce0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 >> v1:172.21.15.59:6789/0 conn(0x7f5b080d0ce0 legacy=0x7f5b080af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af00039e0 0x7f5af0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5af0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.155+0000 7f5b07e00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0004d40 0x7f5af00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b081606e0 con 0x7f5af00039e0 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000baa0 con 0x7f5af0004d40 2024-04-24T06:34:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5af000bca0 con 0x7f5b08072c80 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af00039e0 0x7f5af0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0e200640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0004d40 0x7f5af00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5af0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 -- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 msgr2=0x7f5af0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b08072c80 0x7f5af0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 -- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0004d40 msgr2=0x7f5af00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0004d40 0x7f5af00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0d800640 1 -- 172.21.15.59:0/2954206493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af0003520 con 0x7f5af00039e0 2024-04-24T06:34:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.156+0000 7f5b0e200640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0004d40 0x7f5af00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.157+0000 7f5b0d800640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af00039e0 0x7f5af0003e00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001e20 tx=0x7f5af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca204a63f36a0cba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc001a70 con 0x7f5af00039e0 2024-04-24T06:34:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc02ae40 con 0x7f5af00039e0 2024-04-24T06:34:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b07e00640 1 -- 172.21.15.59:0/2954206493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc003b90 con 0x7f5af00039e0 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 -- 172.21.15.59:0/2954206493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af00039e0 msgr2=0x7f5af0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/2954206493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5af00039e0 0x7f5af0003e00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001e20 tx=0x7f5af0005830 comp rx=0 tx=0).stop 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0e200640 1 -- 172.21.15.59:0/2954206493 reap_dead start 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 -- 172.21.15.59:0/2954206493 shutdown_connections 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 -- 172.21.15.59:0/2954206493 >> 172.21.15.59:0/2954206493 conn(0x7f5b0806bdf0 msgr2=0x7f5b0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 -- 172.21.15.59:0/2954206493 shutdown_connections 2024-04-24T06:34:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.158+0000 7f5b0ec00640 1 -- 172.21.15.59:0/2954206493 wait complete. 2024-04-24T06:34:59.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.160+0000 7f5b0ec00640 1 Processor -- start 2024-04-24T06:34:59.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.160+0000 7f5b0ec00640 1 -- start start 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 0x7f5b0815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b080b2490 0x7f5b081593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080af910 con 0x7f5b0815d930 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080af790 con 0x7f5af00039e0 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.161+0000 7f5b0ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5b080afa90 con 0x7f5b080b2490 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46258/0 (socket says 172.21.15.59:46258) 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 -- 172.21.15.59:0/3569688261 learned_addr learned my addr 172.21.15.59:0/3569688261 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 0x7f5b0815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b080b2490 0x7f5b081593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 -- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b080b2490 msgr2=0x7f5b081593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b080b2490 0x7f5b081593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 -- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 msgr2=0x7f5b0815ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 0x7f5b0815ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 -- 172.21.15.59:0/3569688261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0815ddf0 con 0x7f5b0815d930 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 0x7f5b0815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.162+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5b080b5140 tx=0x7f5b00010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.163+0000 7f5b05600640 1 -- 172.21.15.59:0/3569688261 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000f040 con 0x7f5b0815d930 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.163+0000 7f5b05600640 1 -- 172.21.15.59:0/3569688261 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0000e8a0 con 0x7f5b0815d930 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.163+0000 7f5b05600640 1 -- 172.21.15.59:0/3569688261 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000ec70 con 0x7f5b0815d930 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.163+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0815e0d0 con 0x7f5b0815d930 2024-04-24T06:34:59.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.163+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0815e690 con 0x7f5b0815d930 2024-04-24T06:34:59.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.165+0000 7f5b05600640 1 -- 172.21.15.59:0/3569688261 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b000143f0 con 0x7f5b0815d930 2024-04-24T06:34:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.166+0000 7f5b05600640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 0x7f5ae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.166+0000 7f5b05600640 1 -- 172.21.15.59:0/3569688261 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f5b0008be60 con 0x7f5b0815d930 2024-04-24T06:34:59.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.166+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 0x7f5ae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.167+0000 7f5b13168740 1 Processor -- start 2024-04-24T06:34:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.167+0000 7f5b13168740 1 -- start start 2024-04-24T06:34:59.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.167+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 0x7f5ae806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5afc032000 tx=0x7f5afc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585ee9fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585ee9651f0 con 0x5585eea2b7e0 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5585ee90b350 con 0x5585eea2bf20 2024-04-24T06:34:59.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5585ee9611b0 con 0x5585eea48e00 2024-04-24T06:34:59.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46274/0 (socket says 172.21.15.59:46274) 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.168+0000 7f5b0e200640 1 -- 172.21.15.59:0/3268135273 learned_addr learned my addr 172.21.15.59:0/3268135273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585ee9fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 -- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 msgr2=0x5585ee9fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585ee9fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 -- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 msgr2=0x5585eea2b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea2b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 -- 172.21.15.59:0/3268135273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ee95ff50 con 0x5585eea2b7e0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea2b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5b0e200640 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5b0008a230 tx=0x7f5b00059d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba8fcfbb87cfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5afa000640 1 -- 172.21.15.59:0/3268135273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00021040 con 0x5585eea2b7e0 2024-04-24T06:34:59.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5afa000640 1 -- 172.21.15.59:0/3268135273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00057ea0 con 0x5585eea2b7e0 2024-04-24T06:34:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.169+0000 7f5afa000640 1 -- 172.21.15.59:0/3268135273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00062240 con 0x5585eea2b7e0 2024-04-24T06:34:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 -- 172.21.15.59:0/3268135273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 msgr2=0x5585eea005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5b0008a230 tx=0x7f5b00059d20 comp rx=0 tx=0).stop 2024-04-24T06:34:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 -- 172.21.15.59:0/3268135273 shutdown_connections 2024-04-24T06:34:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2b7e0 0x5585eea005f0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea2b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 --2- 172.21.15.59:0/3268135273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585ee9fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 -- 172.21.15.59:0/3268135273 >> 172.21.15.59:0/3268135273 conn(0x5585ee9fda00 msgr2=0x5585eea462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 -- 172.21.15.59:0/3268135273 shutdown_connections 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.170+0000 7f5b13168740 1 -- 172.21.15.59:0/3268135273 wait complete. 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.171+0000 7f5b13168740 1 -- start start 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.171+0000 7f5b13168740 1 Processor -- start 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea56320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5585ee9ffbb0 con 0x5585eea45760 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5585ee91e690 con 0x5585eea2bf20 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5585ee9fefa0 con 0x5585eea48e00 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b13168740 10 client.0 _ll_register_callbacks cb 0x5585ee95a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.172+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46288/0 (socket says 172.21.15.59:46288) 2024-04-24T06:34:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 -- 172.21.15.59:0/1428724980 learned_addr learned my addr 172.21.15.59:0/1428724980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0d800640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 -- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 msgr2=0x5585eea56320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea56320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 -- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 msgr2=0x5585eea26af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea26af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 -- 172.21.15.59:0/1428724980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ee91d920 con 0x5585eea45760 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5b0e200640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5b0005a200 tx=0x7f5b0001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04dc2ae611ca60a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00021040 con 0x5585eea45760 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.173+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f5b00058ff0 con 0x5585eea45760 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b00057ea0 con 0x5585eea45760 2024-04-24T06:34:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5b00000e60 con 0x5585eea45760 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 10 client.7263 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5585ee7417d0 con 0x5585eea45760 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00057ea0 con 0x5585eea45760 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f5b000600b0 con 0x5585eea45760 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 10 client.7263 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 10 client.7263 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 20 client.7263 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea462a0 0x5585eea466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 4 client.7263 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.174+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ee972c10 con 0x5585eea462a0 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.175+0000 7f5b0d800640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea462a0 0x5585eea466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.175+0000 7f5b0d800640 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea462a0 0x5585eea466d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75f81f80120d6af server_cookie=269bc22fdd651b21 in_seq=0 out_seq=0 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.175+0000 7f5aeea00640 10 client.7263 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5aeea00640 1 -- 172.21.15.59:0/1428724980 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585ee972c10 con 0x5585eea462a0 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5aeea00640 10 client.7263 handle_command_reply: tid=0 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 client.7263 shutdown 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 msgr2=0x5585eea5a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5b0005a200 tx=0x7f5b0001d000 comp rx=0 tx=0).stop 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 shutdown_connections 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea462a0 0x5585eea466d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea5a020 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea56320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5aeea00640 0 client.7263 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 --2- 172.21.15.59:0/1428724980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea2bf20 0x5585eea26af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.176+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 >> 172.21.15.59:0/1428724980 conn(0x5585eea2b7e0 msgr2=0x5585ee9fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.177+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 shutdown_connections 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.177+0000 7f5b13168740 1 -- 172.21.15.59:0/1428724980 wait complete. 2024-04-24T06:34:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.177+0000 7f5b13168740 20 client.7263 trim_cache size 0 max 16384 2024-04-24T06:34:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.196+0000 7f5b13168740 1 Processor -- start 2024-04-24T06:34:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.196+0000 7f5b13168740 1 -- start start 2024-04-24T06:34:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea2bf20 0x5585eea56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea48e00 0x5585eea3df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b082016b0 con 0x5585eea45760 2024-04-24T06:34:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08200690 con 0x5585eea48e00 2024-04-24T06:34:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5b080ceb30 con 0x5585eea2bf20 2024-04-24T06:34:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.197+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46300/0 (socket says 172.21.15.59:46300) 2024-04-24T06:34:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 -- 172.21.15.59:0/1136735814 learned_addr learned my addr 172.21.15.59:0/1136735814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:59.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0e200640 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea48e00 0x5585eea3df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0d800640 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea2bf20 0x5585eea56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 -- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea2bf20 msgr2=0x5585eea56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea2bf20 0x5585eea56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 -- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea48e00 msgr2=0x5585eea3df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea48e00 0x5585eea3df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 -- 172.21.15.59:0/1136735814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eea01cd0 con 0x5585eea45760 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.198+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001200 tx=0x7f5af4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baac00fdfa61fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5af9600640 1 -- 172.21.15.59:0/1136735814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af40072e0 con 0x5585eea45760 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5af9600640 1 -- 172.21.15.59:0/1136735814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af40074c0 con 0x5585eea45760 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5af9600640 1 -- 172.21.15.59:0/1136735814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4007670 con 0x5585eea45760 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 -- 172.21.15.59:0/1136735814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 msgr2=0x5585eea475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001200 tx=0x7f5af4007dd0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 -- 172.21.15.59:0/1136735814 shutdown_connections 2024-04-24T06:34:59.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea48e00 0x5585eea3df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea45760 0x5585eea475c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 --2- 172.21.15.59:0/1136735814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea2bf20 0x5585eea56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 -- 172.21.15.59:0/1136735814 >> 172.21.15.59:0/1136735814 conn(0x5585eea2b7e0 msgr2=0x5585eea56220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.199+0000 7f5b13168740 1 -- 172.21.15.59:0/1136735814 shutdown_connections 2024-04-24T06:34:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.200+0000 7f5b13168740 1 -- 172.21.15.59:0/1136735814 wait complete. 2024-04-24T06:34:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.200+0000 7f5b13168740 1 -- start start 2024-04-24T06:34:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.200+0000 7f5b13168740 1 Processor -- start 2024-04-24T06:34:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea45760 0x5585eeaa6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea00ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08200020 con 0x5585eea2bf20 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5585ee93b8b0 con 0x5585eea45760 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5585eea011e0 con 0x5585eea48e00 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.201+0000 7f5b13168740 10 client.0 _ll_register_callbacks cb 0x7f5b00059ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46312/0 (socket says 172.21.15.59:46312) 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 -- 172.21.15.59:0/3648287187 learned_addr learned my addr 172.21.15.59:0/3648287187 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 -- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 msgr2=0x5585eea00ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea00ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 -- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea45760 msgr2=0x5585eeaa6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea45760 0x5585eeaa6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 -- 172.21.15.59:0/3648287187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ee6773f0 con 0x5585eea2bf20 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5b0d800640 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001cc0 tx=0x7f5afc001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae491567c720636 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.202+0000 7f5aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc046040 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f5afc003e00 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af40074c0 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5afc05d050 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 10 client.7269 fetch_fsmap learned FSMap version 51 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5585ee6370f0 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af40074c0 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f5afc060020 con 0x5585eea2bf20 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 10 client.7269 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 10 client.7269 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 20 client.7269 populate_metadata read hostname 'smithi059' 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea2e620 0x5585eea2eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 4 client.7269 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:34:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.203+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585eea59870 con 0x5585eea2e620 2024-04-24T06:34:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea2e620 0x5585eea2eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5b0ce00640 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea2e620 0x5585eea2eaa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4059dae894ff8950 server_cookie=3e57e99201effd73 in_seq=0 out_seq=0 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5aeea00640 10 client.7269 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5aeea00640 1 -- 172.21.15.59:0/3648287187 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5585eea59870 con 0x5585eea2e620 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5aeea00640 10 client.7269 handle_command_reply: tid=0 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.204+0000 7f5b13168740 1 client.7269 shutdown 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 msgr2=0x5585eeaa6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001cc0 tx=0x7f5afc001510 comp rx=0 tx=0).stop 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 shutdown_connections 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x5585eea2e620 0x5585eea2eaa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5585eea48e00 0x5585eea00ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5585eea45760 0x5585eeaa6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 --2- 172.21.15.59:0/3648287187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585eea2bf20 0x5585eeaa6100 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5aeea00640 0 client.7269 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 >> 172.21.15.59:0/3648287187 conn(0x5585eea2b7e0 msgr2=0x5585eea2f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 shutdown_connections 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 1 -- 172.21.15.59:0/3648287187 wait complete. 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.205+0000 7f5b13168740 20 client.7269 trim_cache size 0 max 16384 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:34:59.210 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: "86bdc30e-5058-4cb7-9a63-0e50c3d0da72": { 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir", 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "86bdc30e-5058-4cb7-9a63-0e50c3d0da72", 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:34:59.211 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 msgr2=0x7f5ae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 0x7f5ae806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5afc032000 tx=0x7f5afc02e000 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 msgr2=0x7f5b08159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5b080b5140 tx=0x7f5b00010c20 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 shutdown_connections 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5ae806c8f0 0x7f5ae806edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0815d930 0x7f5b08159920 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5b080b2490 0x7f5b081593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 --2- 172.21.15.59:0/3569688261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af00039e0 0x7f5b0815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 >> 172.21.15.59:0/3569688261 conn(0x7f5b0806bdf0 msgr2=0x7f5b080c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 shutdown_connections 2024-04-24T06:34:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:34:59.206+0000 7f5b0ec00640 1 -- 172.21.15.59:0/3569688261 wait complete. 2024-04-24T06:34:59.225 INFO:tasks.cephfs.filesystem:scrub status for tag:86bdc30e-5058-4cb7-9a63-0e50c3d0da72 - {'path': '/subdir', 'tag': '86bdc30e-5058-4cb7-9a63-0e50c3d0da72', 'options': 'recursive,repair'} 2024-04-24T06:35:06.789 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:06.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:29.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:29.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-24T06:35:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.448+0000 7fcca9c00640 1 Processor -- start 2024-04-24T06:35:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.448+0000 7fcca9c00640 1 -- start start 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca4145e10 0x7fcca406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcca40d0cc0 0x7fcca40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca4071d70 0x7fcca4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcca40cfc20 con 0x7fcca4071d70 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcca40cff00 con 0x7fcca40d0cc0 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.450+0000 7fcca9c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcca40d01e0 con 0x7fcca4145e10 2024-04-24T06:35:29.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca4071d70 0x7fcca4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca4071d70 0x7fcca4072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34902/0 (socket says 172.21.15.59:34902) 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 -- 172.21.15.59:0/2040597311 learned_addr learned my addr 172.21.15.59:0/2040597311 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca2a00640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca4145e10 0x7fcca406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca2000640 1 --2- 172.21.15.59:0/2040597311 >> v2:172.21.15.66:3300/0 conn(0x7fcca40d0cc0 0x7fcca40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 -- 172.21.15.59:0/2040597311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca4145e10 msgr2=0x7fcca406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca4145e10 0x7fcca406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 -- 172.21.15.59:0/2040597311 >> v2:172.21.15.66:3300/0 conn(0x7fcca40d0cc0 msgr2=0x7fcca40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 --2- 172.21.15.59:0/2040597311 >> v2:172.21.15.66:3300/0 conn(0x7fcca40d0cc0 0x7fcca40cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.451+0000 7fcca3400640 1 -- 172.21.15.59:0/2040597311 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca40d04c0 con 0x7fcca4071d70 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.452+0000 7fcca3400640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4072200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009870 tx=0x7fcc9c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa461d7139596a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.452+0000 7fcca1600640 1 -- 172.21.15.59:0/2040597311 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c037070 con 0x7fcca4071d70 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca1600640 1 -- 172.21.15.59:0/2040597311 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c032440 con 0x7fcca4071d70 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca1600640 1 -- 172.21.15.59:0/2040597311 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c032810 con 0x7fcca4071d70 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 -- 172.21.15.59:0/2040597311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 msgr2=0x7fcca4072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4072200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009870 tx=0x7fcc9c02a210 comp rx=0 tx=0).stop 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 -- 172.21.15.59:0/2040597311 shutdown_connections 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4072200 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 --2- 172.21.15.59:0/2040597311 >> v2:172.21.15.66:3300/0 conn(0x7fcca40d0cc0 0x7fcca40cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 --2- 172.21.15.59:0/2040597311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca4145e10 0x7fcca406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 -- 172.21.15.59:0/2040597311 >> 172.21.15.59:0/2040597311 conn(0x7fcca4067d90 msgr2=0x7fcca406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 -- 172.21.15.59:0/2040597311 shutdown_connections 2024-04-24T06:35:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.453+0000 7fcca9c00640 1 -- 172.21.15.59:0/2040597311 wait complete. 2024-04-24T06:35:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.455+0000 7fcca9c00640 1 Processor -- start 2024-04-24T06:35:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.455+0000 7fcca9c00640 1 -- start start 2024-04-24T06:35:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca40d0cc0 0x7fcca413ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcca4145e10 0x7fcca413cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40cfa00 con 0x7fcca4071d70 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40cf880 con 0x7fcca4145e10 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca9c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcca41461f0 con 0x7fcca40d0cc0 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34914/0 (socket says 172.21.15.59:34914) 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.456+0000 7fcca2a00640 1 -- 172.21.15.59:0/819603157 learned_addr learned my addr 172.21.15.59:0/819603157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2000640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca40d0cc0 0x7fcca413ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 -- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca40d0cc0 msgr2=0x7fcca413ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca40d0cc0 0x7fcca413ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca3400640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcca4145e10 0x7fcca413cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 -- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcca4145e10 msgr2=0x7fcca413cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcca4145e10 0x7fcca413cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 -- 172.21.15.59:0/819603157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41f7c70 con 0x7fcca4071d70 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcca2a00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d830 tx=0x7fcc9400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.457+0000 7fcc9aa00640 1 -- 172.21.15.59:0/819603157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400b6c0 con 0x7fcca4071d70 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.458+0000 7fcc9aa00640 1 -- 172.21.15.59:0/819603157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94019070 con 0x7fcca4071d70 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.458+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41f7f50 con 0x7fcca4071d70 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.458+0000 7fcc9aa00640 1 -- 172.21.15.59:0/819603157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400ba80 con 0x7fcca4071d70 2024-04-24T06:35:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.458+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41f8510 con 0x7fcca4071d70 2024-04-24T06:35:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.460+0000 7fcc9aa00640 1 -- 172.21.15.59:0/819603157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9401e460 con 0x7fcca4071d70 2024-04-24T06:35:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.461+0000 7fcc9aa00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 0x7fcc8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.461+0000 7fcc9aa00640 1 -- 172.21.15.59:0/819603157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fcc9408c360 con 0x7fcca4071d70 2024-04-24T06:35:29.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.462+0000 7fcca2000640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 0x7fcc8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.462+0000 7fccac9c7740 1 Processor -- start 2024-04-24T06:35:29.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.462+0000 7fccac9c7740 1 -- start start 2024-04-24T06:35:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.462+0000 7fcca2000640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 0x7fcc8006edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcca4143400 tx=0x7fcc8c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d863d50 0x55c11d8630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863610 0x55c11d838420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c11d79d1f0 con 0x55c11d880c30 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c11d743350 con 0x55c11d863d50 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c11d7991b0 con 0x55c11d863610 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fcca2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34924/0 (socket says 172.21.15.59:34924) 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.463+0000 7fcca2a00640 1 -- 172.21.15.59:0/57734961 learned_addr learned my addr 172.21.15.59:0/57734961 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 -- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863610 msgr2=0x55c11d838420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863610 0x55c11d838420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 -- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d863d50 msgr2=0x55c11d8630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d863d50 0x55c11d8630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 -- 172.21.15.59:0/57734961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d797f50 con 0x55c11d880c30 2024-04-24T06:35:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcca2a00640 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcc9408a6a0 tx=0x7fcc9400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed1cd0f0497cada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcc93400640 1 -- 172.21.15.59:0/57734961 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94072020 con 0x55c11d880c30 2024-04-24T06:35:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcc93400640 1 -- 172.21.15.59:0/57734961 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94062b40 con 0x55c11d880c30 2024-04-24T06:35:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.464+0000 7fcc93400640 1 -- 172.21.15.59:0/57734961 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9408c070 con 0x55c11d880c30 2024-04-24T06:35:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 -- 172.21.15.59:0/57734961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 msgr2=0x55c11d8335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcc9408a6a0 tx=0x7fcc9400bf70 comp rx=0 tx=0).stop 2024-04-24T06:35:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 -- 172.21.15.59:0/57734961 shutdown_connections 2024-04-24T06:35:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863610 0x55c11d838420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d863d50 0x55c11d8630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 --2- 172.21.15.59:0/57734961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d8335c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 -- 172.21.15.59:0/57734961 >> 172.21.15.59:0/57734961 conn(0x55c11d835830 msgr2=0x55c11d87e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 -- 172.21.15.59:0/57734961 shutdown_connections 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.465+0000 7fccac9c7740 1 -- 172.21.15.59:0/57734961 wait complete. 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.466+0000 7fccac9c7740 1 -- start start 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.466+0000 7fccac9c7740 1 Processor -- start 2024-04-24T06:35:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d87e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c11d8379e0 con 0x55c11d8927f0 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c11d756690 con 0x55c11d880c30 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c11d836dd0 con 0x55c11d863d50 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fccac9c7740 10 client.0 _ll_register_callbacks cb 0x55c11d792650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fcca3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fcca3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34926/0 (socket says 172.21.15.59:34926) 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fcca3400640 1 -- 172.21.15.59:0/743881801 learned_addr learned my addr 172.21.15.59:0/743881801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fcca3400640 1 -- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 msgr2=0x55c11d8662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.467+0000 7fcca3400640 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcca3400640 1 -- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 msgr2=0x55c11d87e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcca3400640 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d87e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcca3400640 1 -- 172.21.15.59:0/743881801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11d755920 con 0x55c11d8927f0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcca3400640 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009990 tx=0x7fcc9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd2bd9135f0170e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcc8be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c037040 con 0x55c11d8927f0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fcc9c032c90 con 0x55c11d8927f0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc94062b40 con 0x55c11d8927f0 2024-04-24T06:35:29.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c047050 con 0x55c11d8927f0 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fccac9c7740 10 client.7281 fetch_fsmap learned FSMap version 52 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.468+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55c11d5797d0 con 0x55c11d8927f0 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94062b40 con 0x55c11d8927f0 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fcc9c04a020 con 0x55c11d8927f0 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 10 client.7281 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 10 client.7281 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 20 client.7281 populate_metadata read hostname 'smithi059' 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d865760 0x55c11d865b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 4 client.7281 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:35:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11d7aac10 con 0x55c11d865760 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.469+0000 7fcca2a00640 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d865760 0x55c11d865b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.470+0000 7fcca2a00640 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d865760 0x55c11d865b60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd31e3cfb80b966 server_cookie=70a426a60f3bd9c5 in_seq=0 out_seq=0 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.470+0000 7fcc8be00640 10 client.7281 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fcc8be00640 1 -- 172.21.15.59:0/743881801 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c11d7aac10 con 0x55c11d865760 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fcc8be00640 10 client.7281 handle_command_reply: tid=0 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 client.7281 shutdown 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 msgr2=0x55c11d87e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009990 tx=0x7fcc9c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 shutdown_connections 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d865760 0x55c11d865b60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d87e610 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d87e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 --2- 172.21.15.59:0/743881801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fcc8be00640 0 client.7281 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 >> 172.21.15.59:0/743881801 conn(0x55c11d863610 msgr2=0x55c11d832c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 shutdown_connections 2024-04-24T06:35:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 1 -- 172.21.15.59:0/743881801 wait complete. 2024-04-24T06:35:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.471+0000 7fccac9c7740 20 client.7281 trim_cache size 0 max 16384 2024-04-24T06:35:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.491+0000 7fccac9c7740 1 Processor -- start 2024-04-24T06:35:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.491+0000 7fccac9c7740 1 -- start start 2024-04-24T06:35:29.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.491+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.491+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d8927f0 0x55c11d87e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fccac9c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c11d878400 con 0x55c11d880c30 2024-04-24T06:35:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4201050 con 0x55c11d8927f0 2024-04-24T06:35:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcca4200050 con 0x55c11d863d50 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34940/0 (socket says 172.21.15.59:34940) 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 -- 172.21.15.59:0/352232996 learned_addr learned my addr 172.21.15.59:0/352232996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2a00640 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca3400640 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d8927f0 0x55c11d87e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 -- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 msgr2=0x55c11d8a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 -- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d8927f0 msgr2=0x55c11d87e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.492+0000 7fcca2000640 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d8927f0 0x55c11d87e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.493+0000 7fcca2000640 1 -- 172.21.15.59:0/352232996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d4af3f0 con 0x55c11d880c30 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.493+0000 7fcca2000640 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c016480 tx=0x7fcc8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1da5580c30a196 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.493+0000 7fcc92a00640 1 -- 172.21.15.59:0/352232996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c026070 con 0x55c11d880c30 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.493+0000 7fcc92a00640 1 -- 172.21.15.59:0/352232996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c0147e0 con 0x55c11d880c30 2024-04-24T06:35:29.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.493+0000 7fcc92a00640 1 -- 172.21.15.59:0/352232996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c014bb0 con 0x55c11d880c30 2024-04-24T06:35:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 -- 172.21.15.59:0/352232996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 msgr2=0x55c11d87ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c016480 tx=0x7fcc8c004a40 comp rx=0 tx=0).stop 2024-04-24T06:35:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 -- 172.21.15.59:0/352232996 shutdown_connections 2024-04-24T06:35:29.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d8927f0 0x55c11d87e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d880c30 0x55c11d87ded0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 --2- 172.21.15.59:0/352232996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d8a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 -- 172.21.15.59:0/352232996 >> 172.21.15.59:0/352232996 conn(0x55c11d863610 msgr2=0x55c11d87cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 -- 172.21.15.59:0/352232996 shutdown_connections 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.494+0000 7fccac9c7740 1 -- 172.21.15.59:0/352232996 wait complete. 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.495+0000 7fccac9c7740 1 -- start start 2024-04-24T06:35:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.495+0000 7fccac9c7740 1 Processor -- start 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d9001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d8a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40ca710 con 0x55c11d8927f0 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40ca0c0 con 0x55c11d880c30 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.496+0000 7fccac9c7740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55c11d8355b0 con 0x55c11d863d50 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fccac9c7740 10 client.0 _ll_register_callbacks cb 0x7fcc9c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34946/0 (socket says 172.21.15.59:34946) 2024-04-24T06:35:29.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 -- 172.21.15.59:0/3548234072 learned_addr learned my addr 172.21.15.59:0/3548234072 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca2000640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d8a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca2a00640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d9001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 -- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 msgr2=0x55c11d9001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d9001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 -- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 msgr2=0x55c11d8a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d8a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 -- 172.21.15.59:0/3548234072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11d46f0f0 con 0x55c11d8927f0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca2000640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d8a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcca3400640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009210 tx=0x7fcc9c038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2635ca969275043 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.497+0000 7fcc8be00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c037070 con 0x55c11d8927f0 2024-04-24T06:35:29.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fcc9c004220 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8c0147e0 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c02a850 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fccac9c7740 10 client.7287 fetch_fsmap learned FSMap version 52 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55c11d762810 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.498+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c0147e0 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fcc9c043070 con 0x55c11d8927f0 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 10 client.7287 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 10 client.7287 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-24T06:35:29.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 20 client.7287 populate_metadata read hostname 'smithi059' 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d88b9a0 0x55c11d832390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 4 client.7287 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c11d7a4a80 con 0x55c11d88b9a0 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fcca2a00640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d88b9a0 0x55c11d832390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fcca2a00640 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d88b9a0 0x55c11d832390 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d2373dcf0abff5 server_cookie=acae3a99125bf98b in_seq=0 out_seq=0 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.499+0000 7fcc8be00640 10 client.7287 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:29.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fcc8be00640 1 -- 172.21.15.59:0/3548234072 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c11d7a4a80 con 0x55c11d88b9a0 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fcc8be00640 10 client.7287 handle_command_reply: tid=0 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 client.7287 shutdown 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 msgr2=0x55c11d8a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009210 tx=0x7fcc9c038000 comp rx=0 tx=0).stop 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 shutdown_connections 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55c11d88b9a0 0x55c11d832390 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c11d8927f0 0x55c11d8a23e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c11d880c30 0x55c11d8a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 --2- 172.21.15.59:0/3548234072 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55c11d863d50 0x55c11d9001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fcc8be00640 0 client.7287 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:29.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 >> 172.21.15.59:0/3548234072 conn(0x55c11d863610 msgr2=0x55c11d87ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 shutdown_connections 2024-04-24T06:35:29.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 1 -- 172.21.15.59:0/3548234072 wait complete. 2024-04-24T06:35:29.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.500+0000 7fccac9c7740 20 client.7287 trim_cache size 0 max 16384 2024-04-24T06:35:29.508 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:35:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:35:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:35:29.509 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:35:29.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 msgr2=0x7fcc8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 0x7fcc8006edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcca4143400 tx=0x7fcc8c00b040 comp rx=0 tx=0).stop 2024-04-24T06:35:29.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 msgr2=0x7fcca4140b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d830 tx=0x7fcc9400dcf0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 shutdown_connections 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc8006c8f0 0x7fcc8006edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcca4145e10 0x7fcca413cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcca40d0cc0 0x7fcca413ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 --2- 172.21.15.59:0/819603157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca4071d70 0x7fcca4140b40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.501+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 >> 172.21.15.59:0/819603157 conn(0x7fcca4067d90 msgr2=0x7fcca415dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.502+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 shutdown_connections 2024-04-24T06:35:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.502+0000 7fcca9c00640 1 -- 172.21.15.59:0/819603157 wait complete. 2024-04-24T06:35:29.522 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:35:29.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:29.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.732+0000 7f8386e00640 1 Processor -- start 2024-04-24T06:35:29.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.733+0000 7f8386e00640 1 -- start start 2024-04-24T06:35:29.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f8386e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f838006c9e0 0x7f838006ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f8386e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f838006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f8386e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838015f2e0 con 0x7f838015c450 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f8386e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8380067e70 con 0x7f838006c9e0 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f8386e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8380073a80 con 0x7f838006e9d0 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f837f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f838006c9e0 0x7f838006ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f837f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f838006c9e0 0x7f838006ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34954/0 (socket says 172.21.15.59:34954) 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f837fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f838015c450 0x7f8380067760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40184/0 (socket says 172.21.15.59:40184) 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.734+0000 7f837f400640 1 -- 172.21.15.59:0/1996710645 learned_addr learned my addr 172.21.15.59:0/1996710645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f8384e00640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f838006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f837f400640 1 -- 172.21.15.59:0/1996710645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 msgr2=0x7f838006ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:29.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f837f400640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f838006ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f837f400640 1 -- 172.21.15.59:0/1996710645 >> v1:172.21.15.59:6789/0 conn(0x7f838015c450 legacy=0x7f8380067760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f837f400640 1 -- 172.21.15.59:0/1996710645 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838015f5c0 con 0x7f838006c9e0 2024-04-24T06:35:29.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.735+0000 7f837ea00640 1 -- 172.21.15.59:0/1996710645 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740024875 0 0) 0x7f838015f2e0 con 0x7f838015c450 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.736+0000 7f837f400640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f838006ce90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098a0 tx=0x7f836c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2789025d89d13361 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f837ea00640 1 -- 172.21.15.59:0/1996710645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c037070 con 0x7f838006c9e0 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f837ea00640 1 -- 172.21.15.59:0/1996710645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c032440 con 0x7f838006c9e0 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f837ea00640 1 -- 172.21.15.59:0/1996710645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c032810 con 0x7f838006c9e0 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 -- 172.21.15.59:0/1996710645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 msgr2=0x7f838006ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f838006ce90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098a0 tx=0x7f836c02a290 comp rx=0 tx=0).stop 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 -- 172.21.15.59:0/1996710645 shutdown_connections 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f838006ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 --2- 172.21.15.59:0/1996710645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f838006ce90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 -- 172.21.15.59:0/1996710645 >> 172.21.15.59:0/1996710645 conn(0x7f8380072650 msgr2=0x7f8380072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 -- 172.21.15.59:0/1996710645 shutdown_connections 2024-04-24T06:35:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.737+0000 7f8386e00640 1 -- 172.21.15.59:0/1996710645 wait complete. 2024-04-24T06:35:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.739+0000 7f8386e00640 1 Processor -- start 2024-04-24T06:35:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.739+0000 7f8386e00640 1 -- start start 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f83800c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f838015c450 0x7f838015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f838015c830 con 0x7f838006c9e0 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f838015fb80 con 0x7f838015c450 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.740+0000 7f8386e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f83800c77b0 con 0x7f838006e9d0 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34964/0 (socket says 172.21.15.59:34964) 2024-04-24T06:35:29.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 -- 172.21.15.59:0/2005361190 learned_addr learned my addr 172.21.15.59:0/2005361190 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837f400640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f83800c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f8384e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f838015c450 0x7f838015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 -- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 msgr2=0x7f83800c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f83800c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 -- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f838015c450 msgr2=0x7f838015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f838015c450 0x7f838015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 -- 172.21.15.59:0/2005361190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83800ce560 con 0x7f838006c9e0 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837f400640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f83800c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:29.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.741+0000 7f837fe00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f837000c460 tx=0x7f837000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.742+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370018070 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.742+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370007500 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.742+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370004a20 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.742+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83800cea80 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.742+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83800cf040 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.743+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800d0800 con 0x7f838006c9e0 2024-04-24T06:35:29.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.744+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8370004bc0 con 0x7f838006c9e0 2024-04-24T06:35:29.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.745+0000 7f8377e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 0x7f836006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:29.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.745+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f837008af80 con 0x7f838006c9e0 2024-04-24T06:35:29.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.745+0000 7f837f400640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 0x7f836006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:29.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.746+0000 7f837f400640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 0x7f836006edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f836c034000 tx=0x7f836c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:29.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.750+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83800d0800 con 0x7f838006c9e0 2024-04-24T06:35:29.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.992+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83800b4dd0 con 0x7f838006c9e0 2024-04-24T06:35:29.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.994+0000 7f8377e00640 1 -- 172.21.15.59:0/2005361190 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f83800b4dd0 con 0x7f838006c9e0 2024-04-24T06:35:29.995 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:29.995 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:35:02.002833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":24,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:35:29.995 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 52 2024-04-24T06:35:29.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.996+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 msgr2=0x7f836006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.996+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 0x7f836006edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f836c034000 tx=0x7f836c02e000 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.996+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 msgr2=0x7f83800cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.996+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f837000c460 tx=0x7f837000c920 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 shutdown_connections 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f836006c8f0 0x7f836006edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f838015c450 0x7f838015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f838006e9d0 0x7f83800c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 --2- 172.21.15.59:0/2005361190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f838006c9e0 0x7f83800cde30 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 >> 172.21.15.59:0/2005361190 conn(0x7f8380072650 msgr2=0x7f83800b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 shutdown_connections 2024-04-24T06:35:29.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:29.997+0000 7f8386e00640 1 -- 172.21.15.59:0/2005361190 wait complete. 2024-04-24T06:35:30.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-24T06:35:30.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.235+0000 7fe71a200640 1 Processor -- start 2024-04-24T06:35:30.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.235+0000 7fe71a200640 1 -- start start 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.237+0000 7fe71a200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe714163040 0x7fe7140b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.237+0000 7fe71a200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7140b6080 0x7fe71406b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.237+0000 7fe71a200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7140af470 con 0x7fe71406bf10 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.237+0000 7fe71a200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe7140afda0 con 0x7fe7140b6080 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.237+0000 7fe71a200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe7140af750 con 0x7fe714163040 2024-04-24T06:35:30.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7140b6080 0x7fe71406b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7140b6080 0x7fe71406b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34966/0 (socket says 172.21.15.59:34966) 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 -- 172.21.15.59:0/2130991892 learned_addr learned my addr 172.21.15.59:0/2130991892 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe718e00640 1 --2- 172.21.15.59:0/2130991892 >> v2:172.21.15.66:3300/0 conn(0x7fe714163040 0x7fe7140b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 -- 172.21.15.59:0/2130991892 >> v2:172.21.15.66:3300/0 conn(0x7fe714163040 msgr2=0x7fe7140b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 --2- 172.21.15.59:0/2130991892 >> v2:172.21.15.66:3300/0 conn(0x7fe714163040 0x7fe7140b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 -- 172.21.15.59:0/2130991892 >> v1:172.21.15.66:6789/0 conn(0x7fe71406bf10 legacy=0x7fe7140ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.238+0000 7fe713e00640 1 -- 172.21.15.59:0/2130991892 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7140afa30 con 0x7fe7140b6080 2024-04-24T06:35:30.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.240+0000 7fe713e00640 1 --2- 172.21.15.59:0/2130991892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71406b9d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe7140cd7a0 tx=0x7fe6fc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b65b63617aae36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.240+0000 7fe713400640 1 -- 172.21.15.59:0/2130991892 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc036070 con 0x7fe7140b6080 2024-04-24T06:35:30.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.240+0000 7fe713400640 1 -- 172.21.15.59:0/2130991892 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc031600 con 0x7fe7140b6080 2024-04-24T06:35:30.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.240+0000 7fe713400640 1 -- 172.21.15.59:0/2130991892 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0319d0 con 0x7fe7140b6080 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 -- 172.21.15.59:0/2130991892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 msgr2=0x7fe71406b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 --2- 172.21.15.59:0/2130991892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71406b9d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe7140cd7a0 tx=0x7fe6fc02a6f0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 -- 172.21.15.59:0/2130991892 shutdown_connections 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 --2- 172.21.15.59:0/2130991892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71406b9d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 --2- 172.21.15.59:0/2130991892 >> v2:172.21.15.66:3300/0 conn(0x7fe714163040 0x7fe7140b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 -- 172.21.15.59:0/2130991892 >> 172.21.15.59:0/2130991892 conn(0x7fe7140b69e0 msgr2=0x7fe7140b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 -- 172.21.15.59:0/2130991892 shutdown_connections 2024-04-24T06:35:30.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.241+0000 7fe71a200640 1 -- 172.21.15.59:0/2130991892 wait complete. 2024-04-24T06:35:30.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.243+0000 7fe71a200640 1 Processor -- start 2024-04-24T06:35:30.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.243+0000 7fe71a200640 1 -- start start 2024-04-24T06:35:30.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 0x7fe7141487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe714163040 0x7fe714164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7140b13a0 con 0x7fe7140b6080 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7140b1220 con 0x7fe71406bf10 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe71a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe7140b1520 con 0x7fe714163040 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34980/0 (socket says 172.21.15.59:34980) 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe713e00640 1 -- 172.21.15.59:0/1371066962 learned_addr learned my addr 172.21.15.59:0/1371066962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.244+0000 7fe718e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 0x7fe7141487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 -- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe714163040 msgr2=0x7fe714164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe714163040 0x7fe714164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 -- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 msgr2=0x7fe7141487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 0x7fe7141487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 -- 172.21.15.59:0/1371066962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe714156ae0 con 0x7fe7140b6080 2024-04-24T06:35:30.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe718e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 0x7fe7141487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe713e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc034040 tx=0x7fe6fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe710c00640 1 -- 172.21.15.59:0/1371066962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc036050 con 0x7fe7140b6080 2024-04-24T06:35:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe710c00640 1 -- 172.21.15.59:0/1371066962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc042070 con 0x7fe7140b6080 2024-04-24T06:35:30.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe710c00640 1 -- 172.21.15.59:0/1371066962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc009510 con 0x7fe7140b6080 2024-04-24T06:35:30.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.245+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe714156dc0 con 0x7fe7140b6080 2024-04-24T06:35:30.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.246+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe714148ce0 con 0x7fe7140b6080 2024-04-24T06:35:30.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.248+0000 7fe710c00640 1 -- 172.21.15.59:0/1371066962 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6fc007c30 con 0x7fe7140b6080 2024-04-24T06:35:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.249+0000 7fe710c00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 0x7fe6f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.249+0000 7fe710c00640 1 -- 172.21.15.59:0/1371066962 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe6fc0acb20 con 0x7fe7140b6080 2024-04-24T06:35:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.249+0000 7fe718e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 0x7fe6f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.249+0000 7fe71e62c740 1 Processor -- start 2024-04-24T06:35:30.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.250+0000 7fe71e62c740 1 -- start start 2024-04-24T06:35:30.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.250+0000 7fe718e00640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 0x7fe6f006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75952600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75927f00 0x55fb7596e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fb7588c190 con 0x55fb7596fac0 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fb75832030 con 0x55fb75927f00 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55fb75888150 con 0x55fb75952220 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34990/0 (socket says 172.21.15.59:34990) 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 -- 172.21.15.59:0/1785284171 learned_addr learned my addr 172.21.15.59:0/1785284171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe718e00640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75952600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 -- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 msgr2=0x55fb75952600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe719800640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75927f00 0x55fb7596e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75952600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 -- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75927f00 msgr2=0x55fb7596e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75927f00 0x55fb7596e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe713e00640 1 -- 172.21.15.59:0/1785284171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7581ab60 con 0x55fb7596fac0 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.251+0000 7fe718e00640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75952600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.252+0000 7fe713e00640 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0aae90 tx=0x7fe6fc07a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d774c71909e245 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.252+0000 7fe709600640 1 -- 172.21.15.59:0/1785284171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc07ae20 con 0x55fb7596fac0 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.252+0000 7fe709600640 1 -- 172.21.15.59:0/1785284171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc037040 con 0x55fb7596fac0 2024-04-24T06:35:30.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.252+0000 7fe709600640 1 -- 172.21.15.59:0/1785284171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc07a720 con 0x55fb7596fac0 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 -- 172.21.15.59:0/1785284171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 msgr2=0x55fb75970010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0aae90 tx=0x7fe6fc07a5f0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 -- 172.21.15.59:0/1785284171 shutdown_connections 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75927f00 0x55fb7596e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75970010 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1785284171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75952600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 -- 172.21.15.59:0/1785284171 >> 172.21.15.59:0/1785284171 conn(0x55fb75924830 msgr2=0x55fb7596d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 -- 172.21.15.59:0/1785284171 shutdown_connections 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.253+0000 7fe71e62c740 1 -- 172.21.15.59:0/1785284171 wait complete. 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.254+0000 7fe71e62c740 1 -- start start 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.254+0000 7fe71e62c740 1 Processor -- start 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75941070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb75964bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fb7588c190 con 0x55fb7596fac0 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fb75845500 con 0x55fb75940b70 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55fb75928ca0 con 0x55fb75952220 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe71e62c740 10 client.0 _ll_register_callbacks cb 0x55fb75882270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe713e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34992/0 (socket says 172.21.15.59:34992) 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe713e00640 1 -- 172.21.15.59:0/3660419956 learned_addr learned my addr 172.21.15.59:0/3660419956 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe718e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75941070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe719800640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb75964bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.255+0000 7fe713e00640 1 -- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 msgr2=0x55fb75941070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe713e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75941070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe713e00640 1 -- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 msgr2=0x55fb75964bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe713e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb75964bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe713e00640 1 -- 172.21.15.59:0/3660419956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb7566a7c0 con 0x55fb7596fac0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe719800640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb75964bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe713e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc07a6b0 tx=0x7fe6fc009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba99c74da1735325 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe702000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc098070 con 0x55fb7596fac0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe6fc079d70 con 0x55fb7596fac0 2024-04-24T06:35:30.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc037040 con 0x55fb7596fac0 2024-04-24T06:35:30.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc07ead0 con 0x55fb7596fac0 2024-04-24T06:35:30.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe71e62c740 10 client.7308 fetch_fsmap learned FSMap version 52 2024-04-24T06:35:30.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.256+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55fb756caf50 con 0x55fb7596fac0 2024-04-24T06:35:30.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc037040 con 0x55fb7596fac0 2024-04-24T06:35:30.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fe6fc076470 con 0x55fb7596fac0 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 10 client.7308 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 10 client.7308 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 20 client.7308 populate_metadata read hostname 'smithi059' 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb7596e030 0x55fb7596e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 4 client.7308 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb75899bb0 con 0x55fb7596e030 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.257+0000 7fe718e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb7596e030 0x55fb7596e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.258+0000 7fe718e00640 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb7596e030 0x55fb7596e410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76887d4a1b94a2d server_cookie=f7c6a14ac3f7d0f9 in_seq=0 out_seq=0 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.258+0000 7fe702000640 10 client.7308 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:30.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.258+0000 7fe702000640 1 -- 172.21.15.59:0/3660419956 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb75899bb0 con 0x55fb7596e030 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.258+0000 7fe702000640 10 client.7308 handle_command_reply: tid=0 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.258+0000 7fe71e62c740 1 client.7308 shutdown 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 msgr2=0x55fb75955230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc07a6b0 tx=0x7fe6fc009810 comp rx=0 tx=0).stop 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 shutdown_connections 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb7596e030 0x55fb7596e410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb75964bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb75955230 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 --2- 172.21.15.59:0/3660419956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb75941070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe702000640 0 client.7308 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 >> 172.21.15.59:0/3660419956 conn(0x55fb75927f00 msgr2=0x55fb75927480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 shutdown_connections 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 1 -- 172.21.15.59:0/3660419956 wait complete. 2024-04-24T06:35:30.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.259+0000 7fe71e62c740 20 client.7308 trim_cache size 0 max 16384 2024-04-24T06:35:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.274+0000 7fe71e62c740 1 Processor -- start 2024-04-24T06:35:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.274+0000 7fe71e62c740 1 -- start start 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 0x55fb7596d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 0x55fb7596bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe714204510 con 0x55fb75940b70 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141fc670 con 0x55fb75952220 2024-04-24T06:35:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.275+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe7142040d0 con 0x55fb7596fac0 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35004/0 (socket says 172.21.15.59:35004) 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 -- 172.21.15.59:0/2432721446 learned_addr learned my addr 172.21.15.59:0/2432721446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe713e00640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 0x55fb7596d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 -- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 msgr2=0x55fb7596bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe719800640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 0x55fb7596bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 0x55fb7596bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 -- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 msgr2=0x55fb7596d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 0x55fb7596d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe718e00640 1 -- 172.21.15.59:0/2432721446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7555e0b0 con 0x55fb75940b70 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe713e00640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 0x55fb7596d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.276+0000 7fe719800640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 0x55fb7596bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.277+0000 7fe718e00640 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55fb75970a30 tx=0x7fe704014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f2c3ac1a4ee62b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.277+0000 7fe708c00640 1 -- 172.21.15.59:0/2432721446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70400f040 con 0x55fb75940b70 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.277+0000 7fe708c00640 1 -- 172.21.15.59:0/2432721446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70402a030 con 0x55fb75940b70 2024-04-24T06:35:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.277+0000 7fe708c00640 1 -- 172.21.15.59:0/2432721446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040044a0 con 0x55fb75940b70 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- 172.21.15.59:0/2432721446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 msgr2=0x55fb759551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55fb75970a30 tx=0x7fe704014000 comp rx=0 tx=0).stop 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- 172.21.15.59:0/2432721446 shutdown_connections 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb7596fac0 0x55fb7596bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75952220 0x55fb7596d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 --2- 172.21.15.59:0/2432721446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb75940b70 0x55fb759551f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- 172.21.15.59:0/2432721446 >> 172.21.15.59:0/2432721446 conn(0x55fb75927f00 msgr2=0x55fb759233c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- 172.21.15.59:0/2432721446 shutdown_connections 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- 172.21.15.59:0/2432721446 wait complete. 2024-04-24T06:35:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.278+0000 7fe71e62c740 1 -- start start 2024-04-24T06:35:30.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.279+0000 7fe71e62c740 1 Processor -- start 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb7591dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb7596bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fb7588c190 con 0x55fb7596fac0 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55fb75928ca0 con 0x55fb75940b70 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55fb75845500 con 0x55fb75952220 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe71e62c740 10 client.0 _ll_register_callbacks cb 0x7fe6fc07a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:35:30.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35012/0 (socket says 172.21.15.59:35012) 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 -- 172.21.15.59:0/1463547250 learned_addr learned my addr 172.21.15.59:0/1463547250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe718e00640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb7591dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe713e00640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb7596bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 -- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 msgr2=0x55fb7596bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb7596bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 -- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 msgr2=0x55fb7591dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb7591dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.280+0000 7fe719800640 1 -- 172.21.15.59:0/1463547250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb75990680 con 0x55fb7596fac0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe719800640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x55fb7596d000 tx=0x7fe70c00f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8b404ccc285d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe702000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c017070 con 0x55fb7596fac0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fe70c013630 con 0x55fb7596fac0 2024-04-24T06:35:30.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70402a030 con 0x55fb7596fac0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe70c022050 con 0x55fb7596fac0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe71e62c740 10 client.7314 fetch_fsmap learned FSMap version 52 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.281+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55fb759e0760 con 0x55fb7596fac0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70402a030 con 0x55fb7596fac0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fe70c025020 con 0x55fb7596fac0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 10 client.7314 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 10 client.7314 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 20 client.7314 populate_metadata read hostname 'smithi059' 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb75925dd0 0x55fb759261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 4 client.7314 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 --> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55fb759677f0 con 0x55fb75925dd0 2024-04-24T06:35:30.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.282+0000 7fe718e00640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb75925dd0 0x55fb759261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe718e00640 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb75925dd0 0x55fb759261b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a382f62b47d5de57 server_cookie=3784d8ca87570648 in_seq=0 out_seq=0 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe702000640 10 client.7314 ms_handle_connect on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe702000640 1 -- 172.21.15.59:0/1463547250 <== mds.0 v2:172.21.15.59:6836/209407257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb759677f0 con 0x55fb75925dd0 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe702000640 10 client.7314 handle_command_reply: tid=0 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 client.7314 shutdown 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 msgr2=0x55fb7596c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x55fb7596d000 tx=0x7fe70c00f600 comp rx=0 tx=0).stop 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 shutdown_connections 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:6836/209407257,v1:172.21.15.59:6837/209407257] conn(0x55fb75925dd0 0x55fb759261b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb7596fac0 0x55fb7596c1e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55fb75952220 0x55fb7596bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 --2- 172.21.15.59:0/1463547250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb75940b70 0x55fb7591dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 >> 172.21.15.59:0/1463547250 conn(0x55fb75927f00 msgr2=0x55fb7596d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe702000640 0 client.7314 ms_handle_reset on v2:172.21.15.59:6836/209407257 2024-04-24T06:35:30.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.283+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 shutdown_connections 2024-04-24T06:35:30.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71e62c740 1 -- 172.21.15.59:0/1463547250 wait complete. 2024-04-24T06:35:30.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71e62c740 20 client.7314 trim_cache size 0 max 16384 2024-04-24T06:35:30.288 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:30.288 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 msgr2=0x7fe6f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 0x7fe6f006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 msgr2=0x7fe71415aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc034040 tx=0x7fe6fc004a40 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 shutdown_connections 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe6f006c8f0 0x7fe6f006edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe714163040 0x7fe714164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7140b6080 0x7fe71415aa40 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 --2- 172.21.15.59:0/1371066962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe71406bf10 0x7fe7141487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.284+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 >> 172.21.15.59:0/1371066962 conn(0x7fe7140b69e0 msgr2=0x7fe7140aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.285+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 shutdown_connections 2024-04-24T06:35:30.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.285+0000 7fe71a200640 1 -- 172.21.15.59:0/1371066962 wait complete. 2024-04-24T06:35:30.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:35:30.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.510+0000 7f5066c00640 1 Processor -- start 2024-04-24T06:35:30.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.511+0000 7f5066c00640 1 -- start start 2024-04-24T06:35:30.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.512+0000 7f5066c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f506015c3d0 0x7f50600cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.512+0000 7f5066c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5060160210 con 0x7f50600b4860 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.512+0000 7f5066c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50601604f0 con 0x7f50600cd590 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.512+0000 7f5066c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f50601607d0 con 0x7f506015c3d0 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.513+0000 7f5066200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50600b4860 0x7f50600cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40198/0 (socket says 172.21.15.59:40198) 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.513+0000 7f5066200640 1 -- 172.21.15.59:0/1190663007 learned_addr learned my addr 172.21.15.59:0/1190663007 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:30.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.513+0000 7f5065800640 1 --2- 172.21.15.59:0/1190663007 >> v2:172.21.15.66:3300/0 conn(0x7f506015c3d0 0x7f50600cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.513+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718742948 0 0) 0x7f5060160210 con 0x7f50600b4860 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.513+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5048003520 con 0x7f50600b4860 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 981381505 0 0) 0x7f5048003520 con 0x7f50600b4860 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 >> v2:172.21.15.66:3300/0 conn(0x7f506015c3d0 msgr2=0x7f50600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 --2- 172.21.15.59:0/1190663007 >> v2:172.21.15.66:3300/0 conn(0x7f506015c3d0 0x7f50600cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 >> v1:172.21.15.66:6789/0 conn(0x7f50600cd590 legacy=0x7f50600b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5060160ab0 con 0x7f50600b4860 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f50580019f0 con 0x7f50600b4860 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 >> v1:172.21.15.59:6789/0 conn(0x7f50600b4860 legacy=0x7f50600cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50480039b0 0x7f5048003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 0x7f5048004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50480051c0 0x7f504800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50601607d0 con 0x7f5048004800 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f504800c060 con 0x7f50480039b0 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f504800c260 con 0x7f50480051c0 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5064e00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 0x7f5048004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5065800640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50480039b0 0x7f5048003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5066200640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50480051c0 0x7f504800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5064e00640 1 -- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50480051c0 msgr2=0x7f504800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5064e00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50480051c0 0x7f504800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5064e00640 1 -- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50480039b0 msgr2=0x7f5048003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.514+0000 7f5064e00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50480039b0 0x7f5048003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.515+0000 7f5064e00640 1 -- 172.21.15.59:0/1190663007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5048003520 con 0x7f5048004800 2024-04-24T06:35:30.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.515+0000 7f5066200640 1 -- 172.21.15.59:0/1190663007 reap_dead start 2024-04-24T06:35:30.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f5064e00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 0x7f5048004c80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f504c001750 tx=0x7f5048005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c908bbd9972ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c001370 con 0x7f5048004800 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504c0052d0 con 0x7f5048004800 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f505fe00640 1 -- 172.21.15.59:0/1190663007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c02abc0 con 0x7f5048004800 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f5066c00640 1 -- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 msgr2=0x7f5048004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.516+0000 7f5066c00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 0x7f5048004c80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f504c001750 tx=0x7f5048005830 comp rx=0 tx=0).stop 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.517+0000 7f5066c00640 1 -- 172.21.15.59:0/1190663007 shutdown_connections 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.517+0000 7f5066c00640 1 --2- 172.21.15.59:0/1190663007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5048004800 0x7f5048004c80 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.517+0000 7f5066c00640 1 -- 172.21.15.59:0/1190663007 >> 172.21.15.59:0/1190663007 conn(0x7f50600b21a0 msgr2=0x7f50600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.517+0000 7f5066c00640 1 -- 172.21.15.59:0/1190663007 shutdown_connections 2024-04-24T06:35:30.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.517+0000 7f5066c00640 1 -- 172.21.15.59:0/1190663007 wait complete. 2024-04-24T06:35:30.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.519+0000 7f5066c00640 1 Processor -- start 2024-04-24T06:35:30.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.519+0000 7f5066c00640 1 -- start start 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5048004800 0x7f50601e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50600c5e60 0x7f50600ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f506015c930 con 0x7f5060154870 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f506015c7b0 con 0x7f50600c5e60 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5066c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50600cd050 con 0x7f5048004800 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35032/0 (socket says 172.21.15.59:35032) 2024-04-24T06:35:30.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 -- 172.21.15.59:0/3786284557 learned_addr learned my addr 172.21.15.59:0/3786284557 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 -- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5048004800 msgr2=0x7f50601e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5048004800 0x7f50601e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 -- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50600c5e60 msgr2=0x7f50600ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50600c5e60 0x7f50600ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.520+0000 7f5064e00640 1 -- 172.21.15.59:0/3786284557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5060154f00 con 0x7f5060154870 2024-04-24T06:35:30.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f5064e00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f504c001870 tx=0x7f504c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c037040 con 0x7f5060154870 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504c040070 con 0x7f5060154870 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c007a30 con 0x7f5060154870 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601551e0 con 0x7f5060154870 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.521+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601557a0 con 0x7f5060154870 2024-04-24T06:35:30.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.522+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506004fd90 con 0x7f5060154870 2024-04-24T06:35:30.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.523+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f504c0052d0 con 0x7f5060154870 2024-04-24T06:35:30.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.525+0000 7f505d600640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 0x7f504006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:30.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.525+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f504c0ab970 con 0x7f5060154870 2024-04-24T06:35:30.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.525+0000 7f5065800640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 0x7f504006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:30.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.526+0000 7f5065800640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 0x7f504006edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5054009990 tx=0x7f5054008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.530+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f504c079be0 con 0x7f5060154870 2024-04-24T06:35:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.774+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f50600cbcc0 con 0x7f5060154870 2024-04-24T06:35:30.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.775+0000 7f505d600640 1 -- 172.21.15.59:0/3786284557 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f504c03ae10 con 0x7f5060154870 2024-04-24T06:35:30.776 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:30.776 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 msgr2=0x7f504006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 0x7f504006edb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5054009990 tx=0x7f5054008040 comp rx=0 tx=0).stop 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 msgr2=0x7f506015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f504c001870 tx=0x7f504c001120 comp rx=0 tx=0).stop 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 shutdown_connections 2024-04-24T06:35:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f504006c8f0 0x7f504006edb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50600c5e60 0x7f50600ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5060154870 0x7f506015cab0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 --2- 172.21.15.59:0/3786284557 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5048004800 0x7f50601e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 >> 172.21.15.59:0/3786284557 conn(0x7f50600b21a0 msgr2=0x7f50600b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 shutdown_connections 2024-04-24T06:35:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:30.778+0000 7f5066c00640 1 -- 172.21.15.59:0/3786284557 wait complete. 2024-04-24T06:35:30.800 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:35:30.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:35:31.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.012+0000 7f56fe000640 1 Processor -- start 2024-04-24T06:35:31.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.013+0000 7f56fe000640 1 -- start start 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f815c5e0 0x7f56f80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56f80af920 0x7f56f80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56f80d06e0 0x7f56f8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f56f80afd80 con 0x7f56f80d06e0 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f56f815fa20 con 0x7f56f80af920 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56fe000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f56f815f0f0 con 0x7f56f815c5e0 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56f7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56f80d06e0 0x7f56f8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56f7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56f80d06e0 0x7f56f8071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35040/0 (socket says 172.21.15.59:35040) 2024-04-24T06:35:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.015+0000 7f56f7e00640 1 -- 172.21.15.59:0/4034692865 learned_addr learned my addr 172.21.15.59:0/4034692865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7400640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f815c5e0 0x7f56f80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f6a00640 1 --2- 172.21.15.59:0/4034692865 >> v2:172.21.15.66:3300/0 conn(0x7f56f80af920 0x7f56f80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7e00640 1 -- 172.21.15.59:0/4034692865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f815c5e0 msgr2=0x7f56f80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7e00640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f815c5e0 0x7f56f80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7e00640 1 -- 172.21.15.59:0/4034692865 >> v2:172.21.15.66:3300/0 conn(0x7f56f80af920 msgr2=0x7f56f80d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7e00640 1 --2- 172.21.15.59:0/4034692865 >> v2:172.21.15.66:3300/0 conn(0x7f56f80af920 0x7f56f80d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.016+0000 7f56f7e00640 1 -- 172.21.15.59:0/4034692865 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f815f3d0 con 0x7f56f80d06e0 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.017+0000 7f56f7e00640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f80d06e0 0x7f56f8071de0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f56f806c790 tx=0x7f56f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87b5dc954a9b11c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.017+0000 7f56f6000640 1 -- 172.21.15.59:0/4034692865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0037070 con 0x7f56f80d06e0 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.017+0000 7f56f6000640 1 -- 172.21.15.59:0/4034692865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f0032440 con 0x7f56f80d06e0 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.017+0000 7f56f6000640 1 -- 172.21.15.59:0/4034692865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0032810 con 0x7f56f80d06e0 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 -- 172.21.15.59:0/4034692865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f80d06e0 msgr2=0x7f56f8071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f80d06e0 0x7f56f8071de0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f56f806c790 tx=0x7f56f002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 -- 172.21.15.59:0/4034692865 shutdown_connections 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f80d06e0 0x7f56f8071de0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 --2- 172.21.15.59:0/4034692865 >> v2:172.21.15.66:3300/0 conn(0x7f56f80af920 0x7f56f80d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 --2- 172.21.15.59:0/4034692865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f815c5e0 0x7f56f80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 -- 172.21.15.59:0/4034692865 >> 172.21.15.59:0/4034692865 conn(0x7f56f8067ea0 msgr2=0x7f56f806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 -- 172.21.15.59:0/4034692865 shutdown_connections 2024-04-24T06:35:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.018+0000 7f56fe000640 1 -- 172.21.15.59:0/4034692865 wait complete. 2024-04-24T06:35:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.020+0000 7f56fe000640 1 Processor -- start 2024-04-24T06:35:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.020+0000 7f56fe000640 1 -- start start 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f80af920 0x7f56f80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f80d06e0 0x7f56f81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f80d0cf0 con 0x7f56f815c5e0 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56f80d0b70 con 0x7f56f80af920 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56fe000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f56f806e480 con 0x7f56f80d06e0 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35046/0 (socket says 172.21.15.59:35046) 2024-04-24T06:35:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.021+0000 7f56f7e00640 1 -- 172.21.15.59:0/1942760073 learned_addr learned my addr 172.21.15.59:0/1942760073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 -- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f80d06e0 msgr2=0x7f56f81593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f80d06e0 0x7f56f81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 -- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f80af920 msgr2=0x7f56f80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7400640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f80af920 0x7f56f80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f80af920 0x7f56f80aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 -- 172.21.15.59:0/1942760073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f8155670 con 0x7f56f815c5e0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56f7e00640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f56f002a220 tx=0x7f56f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0037040 con 0x7f56f815c5e0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f0042070 con 0x7f56f815c5e0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0030920 con 0x7f56f815c5e0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.022+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f8155950 con 0x7f56f815c5e0 2024-04-24T06:35:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.023+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f8155f10 con 0x7f56f815c5e0 2024-04-24T06:35:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.024+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f8068a80 con 0x7f56f815c5e0 2024-04-24T06:35:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.025+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56f0030ac0 con 0x7f56f815c5e0 2024-04-24T06:35:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.026+0000 7f56ef400640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.026+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f56f00ac580 con 0x7f56f815c5e0 2024-04-24T06:35:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.026+0000 7f56f7400640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.027+0000 7f56f7400640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 0x7f56d806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009e10 tx=0x7f56e800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:31.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.030+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56f007a230 con 0x7f56f815c5e0 2024-04-24T06:35:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.282+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56f8162c00 con 0x7f56f815c5e0 2024-04-24T06:35:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.282+0000 7f56ef400640 1 -- 172.21.15.59:0/1942760073 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f56f007e530 con 0x7f56f815c5e0 2024-04-24T06:35:31.283 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:35:31.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 msgr2=0x7f56d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 0x7f56d806edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009e10 tx=0x7f56e800b040 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 msgr2=0x7f56f8159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f56f002a220 tx=0x7f56f00046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 shutdown_connections 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f815c5e0 0x7f56f8159930 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56f80d06e0 0x7f56f81593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 --2- 172.21.15.59:0/1942760073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f80af920 0x7f56f80aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.285+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 >> 172.21.15.59:0/1942760073 conn(0x7f56f8067ea0 msgr2=0x7f56f815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.286+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 shutdown_connections 2024-04-24T06:35:31.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31.286+0000 7f56fe000640 1 -- 172.21.15.59:0/1942760073 wait complete. 2024-04-24T06:35:31.306 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:35:31.307 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:35:31.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:31.307 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:35:31.374 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42213sec preferred_lft 42213sec 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:35:31.375 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:35:31.376 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:35:31.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:31.377 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:35:31.512 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:35:31.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:31.517 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:35:31.577 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:35:31.578 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:35:31.578 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:35:31.578 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:35:31.578 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:35:31.579 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:35:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:35:31.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:35:31.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:31.813 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:35:31.867 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:35:31.867 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:35:31.868 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:35:31.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:31.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:35:31.963 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:35:31.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:35:31.963 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:32.024 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:35:32.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:35:32.024 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:32.079 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:35:32.170 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:35:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:35:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:35:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:35:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:35:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:35:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:35:32.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:35:32.229 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:35:32.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:32.230 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:35:32.285 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:35:32.286 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:35:32.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:35:32.371 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:35:32.414 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:35:32.415 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:35:32.416 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:35:32.417 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:35:32.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:32.418 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:35:32.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:35:32.483+0000 7f2e93d96480 -1 init, newargv = 0x564fc8371560 newargc=15 2024-04-24T06:35:32.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60561]: starting ceph client 2024-04-24T06:35:32.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60561]: starting fuse 2024-04-24T06:35:32.507 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:35:32.507 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:35:32.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:35:32.508 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:35:32.509 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:35:32.510 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:35:32.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.60561.asok 2024-04-24T06:35:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:35:32.722 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60561.asok 2024-04-24T06:35:32.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:32.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60561.asok status 2024-04-24T06:35:32.960 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "60561", 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7332, 2024-04-24T06:35:32.961 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7332 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 779491364 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:35:32.962 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 779491364 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7332 192.168.0.1:0/779491364", 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/779491364", 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 52, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 72, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:35:32.963 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:35:32.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:32.977 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:35:32.977 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:33.035 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:35:33.036 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:33.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:33.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:33.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:33.124 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-24T06:35:33.190 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:35:33.191 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-24T06:35:33.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.306+0000 7f7bcdef6f40 1 Processor -- start 2024-04-24T06:35:33.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.306+0000 7f7bcdef6f40 1 -- start start 2024-04-24T06:35:33.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cbe30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc71980 0x560f6cc59c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f6cc71d60 con 0x560f6cc5ac90 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f6cb41cc0 con 0x560f6cb981d0 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.307+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x560f6cb98b20 con 0x560f6cc71980 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35050/0 (socket says 172.21.15.59:35050) 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 -- 172.21.15.59:0/1960585978 learned_addr learned my addr 172.21.15.59:0/1960585978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bcb000640 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cbe30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bcba00640 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc71980 0x560f6cc59c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 -- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc71980 msgr2=0x560f6cc59c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc71980 0x560f6cc59c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 -- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 msgr2=0x560f6cbe30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cbe30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.308+0000 7f7bca600640 1 -- 172.21.15.59:0/1960585978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6cc57370 con 0x560f6cc5ac90 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bca600640 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8025ca0 tx=0x7f7bb8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e95bff618ab898 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bc9200640 1 -- 172.21.15.59:0/1960585978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8062070 con 0x560f6cc5ac90 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bc9200640 1 -- 172.21.15.59:0/1960585978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8054970 con 0x560f6cc5ac90 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bc9200640 1 -- 172.21.15.59:0/1960585978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8054d20 con 0x560f6cc5ac90 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/1960585978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 msgr2=0x560f6cbe3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.309+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8025ca0 tx=0x7f7bb8051000 comp rx=0 tx=0).stop 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/1960585978 shutdown_connections 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc71980 0x560f6cc59c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc5ac90 0x560f6cbe3610 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/1960585978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cbe30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/1960585978 >> 172.21.15.59:0/1960585978 conn(0x560f6cbc9f40 msgr2=0x560f6cb9a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:33.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/1960585978 shutdown_connections 2024-04-24T06:35:33.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/1960585978 wait complete. 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.310+0000 7f7bcdef6f40 1 Processor -- start 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.311+0000 7f7bcdef6f40 1 -- start start 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.311+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cc6d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc5ac90 0x560f6cbcab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcdef6f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 0x560f6cc59c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f6cc6e990 con 0x560f6cc71980 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f6cc6e6f0 con 0x560f6cb981d0 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcdef6f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x560f6cc57610 con 0x560f6cc5ac90 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 0x560f6cc59c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 0x560f6cc59c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35056/0 (socket says 172.21.15.59:35056) 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 -- 172.21.15.59:0/566236669 learned_addr learned my addr 172.21.15.59:0/566236669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcb000640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cc6d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bca600640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc5ac90 0x560f6cbcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 -- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc5ac90 msgr2=0x560f6cbcab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560f6cc5ac90 0x560f6cbcab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 -- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 msgr2=0x560f6cc6d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f6cb981d0 0x560f6cc6d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.312+0000 7f7bcba00640 1 -- 172.21.15.59:0/566236669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6cbcb400 con 0x560f6cc71980 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bcba00640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 0x560f6cc59c60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c1f0 tx=0x7f7bc400c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bc3400640 1 -- 172.21.15.59:0/566236669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4017020 con 0x560f6cc71980 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bc3400640 1 -- 172.21.15.59:0/566236669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4013670 con 0x560f6cc71980 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f6cbcb6e0 con 0x560f6cc71980 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bc3400640 1 -- 172.21.15.59:0/566236669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4013a40 con 0x560f6cc71980 2024-04-24T06:35:33.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.313+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6cbcbca0 con 0x560f6cc71980 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.315+0000 7f7bc3400640 1 -- 172.21.15.59:0/566236669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bc401a020 con 0x560f6cc71980 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.316+0000 7f7bc3400640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bac06b300 0x7f7bac06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.316+0000 7f7bc3400640 1 -- 172.21.15.59:0/566236669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f7bc408adc0 con 0x560f6cc71980 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.317+0000 7f7bcb000640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bac06b300 0x7f7bac06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.317+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x560f6cc56c30 0x560f6cc4b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.317+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72) -- 0x560f6cc4bd40 con 0x560f6cc56c30 2024-04-24T06:35:33.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.317+0000 7f7bcb000640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bac06b300 0x7f7bac06d7c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc003bb0 tx=0x7f7bbc0030d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.317+0000 7f7bca600640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x560f6cc56c30 0x560f6cc4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.319+0000 7f7bca600640 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x560f6cc56c30 0x560f6cc4b800 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bca600640 1 -- 172.21.15.59:0/566236669 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v72'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bb8054ba0 con 0x560f6cc56c30 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x560f6cc56c30 msgr2=0x560f6cc4b800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x560f6cc56c30 0x560f6cc4b800 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bac06b300 msgr2=0x7f7bac06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bac06b300 0x7f7bac06d7c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc003bb0 tx=0x7f7bbc0030d0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 msgr2=0x560f6cc59c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 --2- 172.21.15.59:0/566236669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f6cc71980 0x560f6cc59c60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c1f0 tx=0x7f7bc400c6b0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcba00640 1 -- 172.21.15.59:0/566236669 reap_dead start 2024-04-24T06:35:33.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 shutdown_connections 2024-04-24T06:35:33.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 >> 172.21.15.59:0/566236669 conn(0x560f6cbc9f40 msgr2=0x560f6cc70a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:33.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 shutdown_connections 2024-04-24T06:35:33.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.323+0000 7f7bcdef6f40 1 -- 172.21.15.59:0/566236669 wait complete. 2024-04-24T06:35:33.329 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:35:33.329 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:35:33.428 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:35:33.548 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:35:33.549 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:35:33.549 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:35:33.549 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:33.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:33.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.824+0000 7f0c1d400640 1 Processor -- start 2024-04-24T06:35:33.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.824+0000 7f0c1d400640 1 -- start start 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c1815b450 0x7f0c180b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c1806f3f0 0x7f0c180cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c180b3870 con 0x7f0c180b3080 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0c180b41a0 con 0x7f0c1815b450 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0c180b3b50 con 0x7f0c1806f3f0 2024-04-24T06:35:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c17e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c1815b450 0x7f0c180b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0c180b3080 0x7f0c180ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40214/0 (socket says 172.21.15.59:40214) 2024-04-24T06:35:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.826+0000 7f0c1ca00640 1 -- 172.21.15.59:0/1047253915 learned_addr learned my addr 172.21.15.59:0/1047253915 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.827+0000 7f0c17e00640 1 -- 172.21.15.59:0/1047253915 >> v2:172.21.15.66:3300/0 conn(0x7f0c1806f3f0 msgr2=0x7f0c180cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.827+0000 7f0c17e00640 1 --2- 172.21.15.59:0/1047253915 >> v2:172.21.15.66:3300/0 conn(0x7f0c1806f3f0 0x7f0c180cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.827+0000 7f0c17e00640 1 -- 172.21.15.59:0/1047253915 >> v1:172.21.15.59:6789/0 conn(0x7f0c180b3080 legacy=0x7f0c180ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.827+0000 7f0c17e00640 1 -- 172.21.15.59:0/1047253915 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c180b3e30 con 0x7f0c1815b450 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.828+0000 7f0c17e00640 1 --2- 172.21.15.59:0/1047253915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1815b450 0x7f0c180b4630 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0c180af4a0 tx=0x7f0c0802c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64f22a2519c44bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.828+0000 7f0c17400640 1 -- 172.21.15.59:0/1047253915 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08038070 con 0x7f0c1815b450 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.828+0000 7f0c17400640 1 -- 172.21.15.59:0/1047253915 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08031710 con 0x7f0c1815b450 2024-04-24T06:35:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.828+0000 7f0c17400640 1 -- 172.21.15.59:0/1047253915 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08031ae0 con 0x7f0c1815b450 2024-04-24T06:35:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 -- 172.21.15.59:0/1047253915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1815b450 msgr2=0x7f0c180b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 --2- 172.21.15.59:0/1047253915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1815b450 0x7f0c180b4630 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0c180af4a0 tx=0x7f0c0802c460 comp rx=0 tx=0).stop 2024-04-24T06:35:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 -- 172.21.15.59:0/1047253915 shutdown_connections 2024-04-24T06:35:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 --2- 172.21.15.59:0/1047253915 >> v2:172.21.15.66:3300/0 conn(0x7f0c1806f3f0 0x7f0c180cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 --2- 172.21.15.59:0/1047253915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1815b450 0x7f0c180b4630 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 -- 172.21.15.59:0/1047253915 >> 172.21.15.59:0/1047253915 conn(0x7f0c180b21a0 msgr2=0x7f0c180b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 -- 172.21.15.59:0/1047253915 shutdown_connections 2024-04-24T06:35:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.829+0000 7f0c1d400640 1 -- 172.21.15.59:0/1047253915 wait complete. 2024-04-24T06:35:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.831+0000 7f0c1d400640 1 Processor -- start 2024-04-24T06:35:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.831+0000 7f0c1d400640 1 -- start start 2024-04-24T06:35:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c1806f3f0 0x7f0c181f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c1815b450 0x7f0c180cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c180b4ff0 con 0x7f0c180b3080 2024-04-24T06:35:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1815b830 con 0x7f0c1806f3f0 2024-04-24T06:35:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c1d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c180cd590 con 0x7f0c1815b450 2024-04-24T06:35:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35062/0 (socket says 172.21.15.59:35062) 2024-04-24T06:35:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.832+0000 7f0c0fe00640 1 -- 172.21.15.59:0/2331218775 learned_addr learned my addr 172.21.15.59:0/2331218775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 -- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c1815b450 msgr2=0x7f0c180cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c1815b450 0x7f0c180cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 -- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c1806f3f0 msgr2=0x7f0c181f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c1806f3f0 0x7f0c181f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 -- 172.21.15.59:0/2331218775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c181e4780 con 0x7f0c180b3080 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c0fe00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000b4a0 tx=0x7f0c0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00014070 con 0x7f0c180b3080 2024-04-24T06:35:33.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c00010610 con 0x7f0c180b3080 2024-04-24T06:35:33.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.833+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00010a00 con 0x7f0c180b3080 2024-04-24T06:35:33.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.834+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181f5610 con 0x7f0c180b3080 2024-04-24T06:35:33.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.834+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1815c150 con 0x7f0c180b3080 2024-04-24T06:35:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.835+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1804fd90 con 0x7f0c180b3080 2024-04-24T06:35:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.836+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c00017070 con 0x7f0c180b3080 2024-04-24T06:35:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.837+0000 7f0c14c00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.837+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0c0008b630 con 0x7f0c180b3080 2024-04-24T06:35:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.837+0000 7f0c17e00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:33.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.838+0000 7f0c17e00640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 0x7f0bf406edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0c08035000 tx=0x7f0c08030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:33.841+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c181f5610 con 0x7f0c180b3080 2024-04-24T06:35:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.047+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c180adbf0 con 0x7f0c180b3080 2024-04-24T06:35:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.050+0000 7f0c14c00640 1 -- 172.21.15.59:0/2331218775 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16811 (secure 0 0 0) 0x7f0c181e4780 con 0x7f0c180b3080 2024-04-24T06:35:34.051 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:34.052 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:28.366221+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 msgr2=0x7f0bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 0x7f0bf406edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0c08035000 tx=0x7f0c08030000 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 msgr2=0x7f0c180cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000b4a0 tx=0x7f0c0000b960 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 shutdown_connections 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.053+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c1815b450 0x7f0c180cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.054+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c180b3080 0x7f0c180cb690 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.054+0000 7f0c1d400640 1 --2- 172.21.15.59:0/2331218775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c1806f3f0 0x7f0c181f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.054+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 >> 172.21.15.59:0/2331218775 conn(0x7f0c180b21a0 msgr2=0x7f0c18159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.054+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 shutdown_connections 2024-04-24T06:35:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.054+0000 7f0c1d400640 1 -- 172.21.15.59:0/2331218775 wait complete. 2024-04-24T06:35:34.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:35:34.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:35:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.289+0000 7ff42a200640 1 Processor -- start 2024-04-24T06:35:34.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.289+0000 7ff42a200640 1 -- start start 2024-04-24T06:35:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.290+0000 7ff42a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff424156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.290+0000 7ff42a200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff42415d810 0x7ff424160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.290+0000 7ff42a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42415cab0 con 0x7ff4241567e0 2024-04-24T06:35:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.290+0000 7ff42a200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff424160e40 con 0x7ff42415d810 2024-04-24T06:35:34.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.290+0000 7ff42a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff42415cd30 con 0x7ff4240d0d10 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff428e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff4241567e0 0x7ff4240d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40224/0 (socket says 172.21.15.59:40224) 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff428e00640 1 -- 172.21.15.59:0/3477378465 learned_addr learned my addr 172.21.15.59:0/3477378465 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff429800640 1 --2- 172.21.15.59:0/3477378465 >> v2:172.21.15.66:3300/0 conn(0x7ff42415d810 0x7ff424160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff423e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff424156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695084955 0 0) 0x7ff418001110 con 0x7ff4241567e0 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.291+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff40c003520 con 0x7ff4241567e0 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3396910612 0 0) 0x7ff40c003520 con 0x7ff4241567e0 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 msgr2=0x7ff424156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff424156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 >> v2:172.21.15.66:3300/0 conn(0x7ff42415d810 msgr2=0x7ff424160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 --2- 172.21.15.59:0/3477378465 >> v2:172.21.15.66:3300/0 conn(0x7ff42415d810 0x7ff424160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42415d010 con 0x7ff4241567e0 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff418001840 con 0x7ff4241567e0 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 >> v1:172.21.15.59:6789/0 conn(0x7ff4241567e0 legacy=0x7ff4240d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff40c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff40c003f30 0x7ff40c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff424160e40 con 0x7ff40c003f30 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff42415cd30 con 0x7ff40c004d40 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff40c00b9c0 con 0x7ff4240d0d10 2024-04-24T06:35:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff40c003f30 0x7ff40c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff428e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff40c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff429800640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 -- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 msgr2=0x7ff40c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4240d0d10 0x7ff40c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 -- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c004d40 msgr2=0x7ff40c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.292+0000 7ff423e00640 1 -- 172.21.15.59:0/3477378465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c003520 con 0x7ff40c003f30 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff423e00640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff40c003f30 0x7ff40c004800 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ac0 tx=0x7ff40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb2b332d15e5aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414036070 con 0x7ff40c003f30 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff414031400 con 0x7ff40c003f30 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff423400640 1 -- 172.21.15.59:0/3477378465 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414009690 con 0x7ff40c003f30 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff42a200640 1 -- 172.21.15.59:0/3477378465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff40c003f30 msgr2=0x7ff40c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff42a200640 1 --2- 172.21.15.59:0/3477378465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff40c003f30 0x7ff40c004800 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ac0 tx=0x7ff40c005830 comp rx=0 tx=0).stop 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.294+0000 7ff429800640 1 -- 172.21.15.59:0/3477378465 reap_dead start 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.295+0000 7ff42a200640 1 -- 172.21.15.59:0/3477378465 shutdown_connections 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.295+0000 7ff42a200640 1 -- 172.21.15.59:0/3477378465 >> 172.21.15.59:0/3477378465 conn(0x7ff42406bdf0 msgr2=0x7ff42406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.295+0000 7ff42a200640 1 -- 172.21.15.59:0/3477378465 shutdown_connections 2024-04-24T06:35:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.295+0000 7ff42a200640 1 -- 172.21.15.59:0/3477378465 wait complete. 2024-04-24T06:35:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.296+0000 7ff42a200640 1 Processor -- start 2024-04-24T06:35:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.297+0000 7ff42a200640 1 -- start start 2024-04-24T06:35:34.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.297+0000 7ff42a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 0x7ff4241f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff42a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff42a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff424158f50 0x7ff4241593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff42a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff424156d40 con 0x7ff424157b40 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff42a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff424156bc0 con 0x7ff40c003f30 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff42a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff424156ec0 con 0x7ff424158f50 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35084/0 (socket says 172.21.15.59:35084) 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 -- 172.21.15.59:0/222373291 learned_addr learned my addr 172.21.15.59:0/222373291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff429800640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff424158f50 0x7ff4241593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff428e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 0x7ff4241f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 -- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff424158f50 msgr2=0x7ff4241593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff424158f50 0x7ff4241593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 -- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 msgr2=0x7ff4241f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 0x7ff4241f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff423e00640 1 -- 172.21.15.59:0/222373291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff424158640 con 0x7ff424157b40 2024-04-24T06:35:34.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.298+0000 7ff428e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 0x7ff4241f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:34.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff423e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff414004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414036070 con 0x7ff424157b40 2024-04-24T06:35:34.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff414041070 con 0x7ff424157b40 2024-04-24T06:35:34.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4140315b0 con 0x7ff424157b40 2024-04-24T06:35:34.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4240b0030 con 0x7ff424157b40 2024-04-24T06:35:34.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.299+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4240b05f0 con 0x7ff424157b40 2024-04-24T06:35:34.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.301+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff424070520 con 0x7ff424157b40 2024-04-24T06:35:34.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.301+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff414031770 con 0x7ff424157b40 2024-04-24T06:35:34.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.302+0000 7ff420c00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.302+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ff4140ab960 con 0x7ff424157b40 2024-04-24T06:35:34.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.303+0000 7ff428e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.303+0000 7ff428e00640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 0x7ff40806edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff418001af0 tx=0x7ff418009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.307+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff424070520 con 0x7ff424157b40 2024-04-24T06:35:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.511+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4240afad0 con 0x7ff424157b40 2024-04-24T06:35:34.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.512+0000 7ff420c00640 1 -- 172.21.15.59:0/222373291 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7ff41404a020 con 0x7ff424157b40 2024-04-24T06:35:34.513 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 msgr2=0x7ff40806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 0x7ff40806edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff418001af0 tx=0x7ff418009040 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 msgr2=0x7ff424157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff414004730 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 shutdown_connections 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40806c8f0 0x7ff40806edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff424158f50 0x7ff4241593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff424157b40 0x7ff424157fc0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 --2- 172.21.15.59:0/222373291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff40c003f30 0x7ff4241f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 >> 172.21.15.59:0/222373291 conn(0x7ff42406bdf0 msgr2=0x7ff424072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 shutdown_connections 2024-04-24T06:35:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.515+0000 7ff42a200640 1 -- 172.21.15.59:0/222373291 wait complete. 2024-04-24T06:35:34.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:35:34.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.748+0000 7fbc37e00640 1 Processor -- start 2024-04-24T06:35:34.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.748+0000 7fbc37e00640 1 -- start start 2024-04-24T06:35:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc37e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc380b4f20 0x7fbc380cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc37e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc380b5da0 con 0x7fbc380d15b0 2024-04-24T06:35:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc37e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc380d0a20 con 0x7fbc3815ed20 2024-04-24T06:35:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc37e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3815f100 con 0x7fbc380b4f20 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc2e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc380d15b0 0x7fbc380cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40236/0 (socket says 172.21.15.59:40236) 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.750+0000 7fbc2e000640 1 -- 172.21.15.59:0/1133040361 learned_addr learned my addr 172.21.15.59:0/1133040361 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc37400640 1 --2- 172.21.15.59:0/1133040361 >> v2:172.21.15.66:3300/0 conn(0x7fbc380b4f20 0x7fbc380cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246562363 0 0) 0x7fbc380b5da0 con 0x7fbc380d15b0 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc1c003520 con 0x7fbc380d15b0 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2548586691 0 0) 0x7fbc1c003520 con 0x7fbc380d15b0 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 >> v2:172.21.15.66:3300/0 conn(0x7fbc380b4f20 msgr2=0x7fbc380cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 --2- 172.21.15.59:0/1133040361 >> v2:172.21.15.66:3300/0 conn(0x7fbc380b4f20 0x7fbc380cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 >> v1:172.21.15.66:6789/0 conn(0x7fbc3815ed20 legacy=0x7fbc380d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc380d2ed0 con 0x7fbc380d15b0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbc20001670 con 0x7fbc380d15b0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 >> v1:172.21.15.59:6789/0 conn(0x7fbc380d15b0 legacy=0x7fbc380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc1c003a00 0x7fbc1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc1c004800 0x7fbc1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c00c1b0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.751+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c00c360 con 0x7fbc1c004800 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c00c560 con 0x7fbc1c003a00 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc36a00640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc1c003a00 0x7fbc1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc2e000640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc1c004800 0x7fbc1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 -- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc1c003a00 msgr2=0x7fbc1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc1c003a00 0x7fbc1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 -- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc1c004800 msgr2=0x7fbc1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc1c004800 0x7fbc1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc37400640 1 -- 172.21.15.59:0/1133040361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c003520 con 0x7fbc1c0052e0 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc36a00640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc1c003a00 0x7fbc1c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.752+0000 7fbc2e000640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc1c004800 0x7fbc1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.753+0000 7fbc37400640 1 -- 172.21.15.59:0/1133040361 reap_dead start 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.753+0000 7fbc37400640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc1c00b9c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009ac0 tx=0x7fbc1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b1bb53ae3fef6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30009690 con 0x7fbc1c0052e0 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc30004020 con 0x7fbc1c0052e0 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc36000640 1 -- 172.21.15.59:0/1133040361 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004410 con 0x7fbc1c0052e0 2024-04-24T06:35:34.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc37e00640 1 -- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 msgr2=0x7fbc1c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc37e00640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc1c00b9c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009ac0 tx=0x7fbc1c005830 comp rx=0 tx=0).stop 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc37e00640 1 -- 172.21.15.59:0/1133040361 shutdown_connections 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc37e00640 1 --2- 172.21.15.59:0/1133040361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc1c00b9c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.754+0000 7fbc37e00640 1 -- 172.21.15.59:0/1133040361 >> 172.21.15.59:0/1133040361 conn(0x7fbc3808bf90 msgr2=0x7fbc380d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.755+0000 7fbc37e00640 1 -- 172.21.15.59:0/1133040361 shutdown_connections 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.755+0000 7fbc37e00640 1 -- 172.21.15.59:0/1133040361 wait complete. 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.756+0000 7fbc37e00640 1 Processor -- start 2024-04-24T06:35:34.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.757+0000 7fbc37e00640 1 -- start start 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc380ce130 0x7fbc3809e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc380b1260 0x7fbc380b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc380b3600 con 0x7fbc1c0052e0 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc380d3490 con 0x7fbc380b1260 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc37e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc380b3780 con 0x7fbc380ce130 2024-04-24T06:35:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35110/0 (socket says 172.21.15.59:35110) 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 -- 172.21.15.59:0/2742521856 learned_addr learned my addr 172.21.15.59:0/2742521856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 -- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc380ce130 msgr2=0x7fbc3809e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc380ce130 0x7fbc3809e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 -- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc380b1260 msgr2=0x7fbc380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc380b1260 0x7fbc380b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.758+0000 7fbc36a00640 1 -- 172.21.15.59:0/2742521856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3809ec60 con 0x7fbc1c0052e0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc36a00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800dae0 tx=0x7fbc280020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28003da0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc28017070 con 0x7fbc1c0052e0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800b950 con 0x7fbc1c0052e0 2024-04-24T06:35:34.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3809ef40 con 0x7fbc1c0052e0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.759+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3809f500 con 0x7fbc1c0052e0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.761+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3809f8e0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.761+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc2800baf0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.762+0000 7fbc2f400640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 0x7fbc1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.762+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fbc2808bf80 con 0x7fbc1c0052e0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.763+0000 7fbc2e000640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 0x7fbc1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:34.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.763+0000 7fbc2e000640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 0x7fbc1406ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:34.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.767+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc3809f8e0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.973+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbc3804fd90 con 0x7fbc1c0052e0 2024-04-24T06:35:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.974+0000 7fbc2f400640 1 -- 172.21.15.59:0/2742521856 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7fbc2805a1f0 con 0x7fbc1c0052e0 2024-04-24T06:35:34.975 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 msgr2=0x7fbc1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 0x7fbc1406ede0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc20009040 comp rx=0 tx=0).stop 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 msgr2=0x7fbc381568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800dae0 tx=0x7fbc280020a0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 shutdown_connections 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc1406c920 0x7fbc1406ede0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc380b1260 0x7fbc380b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc380ce130 0x7fbc3809e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.976+0000 7fbc37e00640 1 --2- 172.21.15.59:0/2742521856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc1c0052e0 0x7fbc381568e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.977+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 >> 172.21.15.59:0/2742521856 conn(0x7fbc3808bf90 msgr2=0x7fbc380cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.977+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 shutdown_connections 2024-04-24T06:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:34.977+0000 7fbc37e00640 1 -- 172.21.15.59:0/2742521856 wait complete. 2024-04-24T06:35:34.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:35:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.209+0000 7f6461800640 1 Processor -- start 2024-04-24T06:35:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.209+0000 7f6461800640 1 -- start start 2024-04-24T06:35:35.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c15d300 0x7f645c071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f645c167180 0x7f645c163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f645c0d0600 0x7f645c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f645c163650 con 0x7f645c167180 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f645c163320 con 0x7f645c0d0600 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.211+0000 7f6461800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f645c072380 con 0x7f645c15d300 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f645c167180 0x7f645c163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f645c167180 0x7f645c163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35112/0 (socket says 172.21.15.59:35112) 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 -- 172.21.15.59:0/1049610269 learned_addr learned my addr 172.21.15.59:0/1049610269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645be00640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c15d300 0x7f645c071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f6460e00640 1 --2- 172.21.15.59:0/1049610269 >> v2:172.21.15.66:3300/0 conn(0x7f645c0d0600 0x7f645c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 -- 172.21.15.59:0/1049610269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c15d300 msgr2=0x7f645c071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c15d300 0x7f645c071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 -- 172.21.15.59:0/1049610269 >> v2:172.21.15.66:3300/0 conn(0x7f645c0d0600 msgr2=0x7f645c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 --2- 172.21.15.59:0/1049610269 >> v2:172.21.15.66:3300/0 conn(0x7f645c0d0600 0x7f645c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.212+0000 7f645b400640 1 -- 172.21.15.59:0/1049610269 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c072660 con 0x7f645c167180 2024-04-24T06:35:35.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.213+0000 7f645b400640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c167180 0x7f645c163ae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6444009870 tx=0x7f644402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06d53e7518c2785 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.214+0000 7f645aa00640 1 -- 172.21.15.59:0/1049610269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444037070 con 0x7f645c167180 2024-04-24T06:35:35.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.214+0000 7f645aa00640 1 -- 172.21.15.59:0/1049610269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6444032440 con 0x7f645c167180 2024-04-24T06:35:35.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.214+0000 7f645aa00640 1 -- 172.21.15.59:0/1049610269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444032810 con 0x7f645c167180 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.214+0000 7f6461800640 1 -- 172.21.15.59:0/1049610269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c167180 msgr2=0x7f645c163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.214+0000 7f6461800640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c167180 0x7f645c163ae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6444009870 tx=0x7f644402a210 comp rx=0 tx=0).stop 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 -- 172.21.15.59:0/1049610269 shutdown_connections 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 --2- 172.21.15.59:0/1049610269 >> v2:172.21.15.66:3300/0 conn(0x7f645c0d0600 0x7f645c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c167180 0x7f645c163ae0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 --2- 172.21.15.59:0/1049610269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c15d300 0x7f645c071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 -- 172.21.15.59:0/1049610269 >> 172.21.15.59:0/1049610269 conn(0x7f645c06bdf0 msgr2=0x7f645c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 -- 172.21.15.59:0/1049610269 shutdown_connections 2024-04-24T06:35:35.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.215+0000 7f6461800640 1 -- 172.21.15.59:0/1049610269 wait complete. 2024-04-24T06:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.217+0000 7f6461800640 1 Processor -- start 2024-04-24T06:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.217+0000 7f6461800640 1 -- start start 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c0d0600 0x7f645c09bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c09d420 0x7f645c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f645c164020 con 0x7f645c15d300 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1675e0 con 0x7f645c0d0600 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6461800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f645c06e1e0 con 0x7f645c09d420 2024-04-24T06:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f645b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f645b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35124/0 (socket says 172.21.15.59:35124) 2024-04-24T06:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f645b400640 1 -- 172.21.15.59:0/3718056926 learned_addr learned my addr 172.21.15.59:0/3718056926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f645be00640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c0d0600 0x7f645c09bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.218+0000 7f6460e00640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c09d420 0x7f645c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 -- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c09d420 msgr2=0x7f645c0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c09d420 0x7f645c0afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 -- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c0d0600 msgr2=0x7f645c09bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c0d0600 0x7f645c09bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 -- 172.21.15.59:0/3718056926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0b3cc0 con 0x7f645c15d300 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.219+0000 7f645b400640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f64440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.220+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444037040 con 0x7f645c15d300 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.220+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6444042070 con 0x7f645c15d300 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.220+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444030900 con 0x7f645c15d300 2024-04-24T06:35:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.220+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c0b3fa0 con 0x7f645c15d300 2024-04-24T06:35:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.220+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c0b0420 con 0x7f645c15d300 2024-04-24T06:35:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.221+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c04fd90 con 0x7f645c15d300 2024-04-24T06:35:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.222+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6444030aa0 con 0x7f645c15d300 2024-04-24T06:35:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.223+0000 7f6453e00640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.223+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f64440ac3b0 con 0x7f645c15d300 2024-04-24T06:35:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.224+0000 7f645be00640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.224+0000 7f645be00640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f645c09dc80 tx=0x7f644c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.228+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f644407a7d0 con 0x7f645c15d300 2024-04-24T06:35:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.435+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f645c09cd60 con 0x7f645c15d300 2024-04-24T06:35:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.436+0000 7f6453e00640 1 -- 172.21.15.59:0/3718056926 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f644407a0d0 con 0x7f645c15d300 2024-04-24T06:35:35.437 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 msgr2=0x7f643806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f645c09dc80 tx=0x7f644c00b040 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 msgr2=0x7f645c1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f64440046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 shutdown_connections 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f645c09d420 0x7f645c0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f645c15d300 0x7f645c1565b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 --2- 172.21.15.59:0/3718056926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f645c0d0600 0x7f645c09bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.438+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 >> 172.21.15.59:0/3718056926 conn(0x7f645c06bdf0 msgr2=0x7f645c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.439+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 shutdown_connections 2024-04-24T06:35:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.439+0000 7f6461800640 1 -- 172.21.15.59:0/3718056926 wait complete. 2024-04-24T06:35:35.458 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:35:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.672+0000 7f0d6cc00640 1 Processor -- start 2024-04-24T06:35:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.673+0000 7f0d6cc00640 1 -- start start 2024-04-24T06:35:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.674+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d680cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.674+0000 7f0d6cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d680b4230 0x7f0d680b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.674+0000 7f0d6cc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d680b4cf0 con 0x7f0d680d05c0 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.674+0000 7f0d6cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6815ca50 con 0x7f0d680b4230 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.674+0000 7f0d6cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680ccb80 con 0x7f0d6815c670 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d680b4230 0x7f0d680b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d680b4230 0x7f0d680b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35128/0 (socket says 172.21.15.59:35128) 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 -- 172.21.15.59:0/4188093016 learned_addr learned my addr 172.21.15.59:0/4188093016 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 -- 172.21.15.59:0/4188093016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 msgr2=0x7f0d680cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 --2- 172.21.15.59:0/4188093016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d680cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 -- 172.21.15.59:0/4188093016 >> v1:172.21.15.66:6789/0 conn(0x7f0d680d05c0 legacy=0x7f0d68160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.675+0000 7f0d66a00640 1 -- 172.21.15.59:0/4188093016 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d680ccdd0 con 0x7f0d680b4230 2024-04-24T06:35:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.676+0000 7f0d66a00640 1 --2- 172.21.15.59:0/4188093016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680b4230 0x7f0d680b4690 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d5402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c306db38e3a971 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d66000640 1 -- 172.21.15.59:0/4188093016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54036070 con 0x7f0d680b4230 2024-04-24T06:35:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d66000640 1 -- 172.21.15.59:0/4188093016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d54031600 con 0x7f0d680b4230 2024-04-24T06:35:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d66000640 1 -- 172.21.15.59:0/4188093016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540319d0 con 0x7f0d680b4230 2024-04-24T06:35:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 -- 172.21.15.59:0/4188093016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680b4230 msgr2=0x7f0d680b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/4188093016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680b4230 0x7f0d680b4690 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d5402a6c0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 -- 172.21.15.59:0/4188093016 shutdown_connections 2024-04-24T06:35:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/4188093016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680b4230 0x7f0d680b4690 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/4188093016 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d680cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.677+0000 7f0d6cc00640 1 -- 172.21.15.59:0/4188093016 >> 172.21.15.59:0/4188093016 conn(0x7f0d6806bdf0 msgr2=0x7f0d6806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.678+0000 7f0d6cc00640 1 -- 172.21.15.59:0/4188093016 shutdown_connections 2024-04-24T06:35:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.678+0000 7f0d6cc00640 1 -- 172.21.15.59:0/4188093016 wait complete. 2024-04-24T06:35:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.680+0000 7f0d6cc00640 1 Processor -- start 2024-04-24T06:35:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.680+0000 7f0d6cc00640 1 -- start start 2024-04-24T06:35:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680b4230 0x7f0d6815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d68147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680b3e60 con 0x7f0d680d05c0 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680cd390 con 0x7f0d680b4230 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.681+0000 7f0d6cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680b3fe0 con 0x7f0d6815c670 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35134/0 (socket says 172.21.15.59:35134) 2024-04-24T06:35:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 -- 172.21.15.59:0/3013711160 learned_addr learned my addr 172.21.15.59:0/3013711160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d67400640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680b4230 0x7f0d6815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d67e00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d68147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 -- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 msgr2=0x7f0d68147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d68147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 -- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680b4230 msgr2=0x7f0d6815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680b4230 0x7f0d6815b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.682+0000 7f0d66a00640 1 -- 172.21.15.59:0/3013711160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68145f90 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.683+0000 7f0d66a00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.683+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54041020 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.683+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d54007800 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.683+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54036040 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.683+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d681462d0 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.684+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68146b10 con 0x7f0d680d05c0 2024-04-24T06:35:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.685+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6804fd90 con 0x7f0d680d05c0 2024-04-24T06:35:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.686+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d54044020 con 0x7f0d680d05c0 2024-04-24T06:35:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.687+0000 7f0d5f400640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 0x7f0d4806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.687+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0d540aca50 con 0x7f0d680d05c0 2024-04-24T06:35:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.687+0000 7f0d67400640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 0x7f0d4806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.688+0000 7f0d67400640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 0x7f0d4806ee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b810 tx=0x7f0d58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:35.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.691+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d5404b030 con 0x7f0d680d05c0 2024-04-24T06:35:35.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.907+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0d680ceab0 con 0x7f0d680d05c0 2024-04-24T06:35:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.908+0000 7f0d5f400640 1 -- 172.21.15.59:0/3013711160 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v72) ==== 80+0+122 (secure 0 0 0) 0x7f0d681462d0 con 0x7f0d680d05c0 2024-04-24T06:35:35.909 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:35:35.909 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2419732942 2024-04-25T06:34:06.952885+0000 2024-04-24T06:35:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:35:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.910+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 msgr2=0x7f0d4806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 0x7f0d4806ee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b810 tx=0x7f0d58008040 comp rx=0 tx=0).stop 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 msgr2=0x7f0d68147e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d54004a40 comp rx=0 tx=0).stop 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 shutdown_connections 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d4806c940 0x7f0d4806ee00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d6815c670 0x7f0d68147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d680d05c0 0x7f0d68147e20 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 --2- 172.21.15.59:0/3013711160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d680b4230 0x7f0d6815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 >> 172.21.15.59:0/3013711160 conn(0x7f0d6806bdf0 msgr2=0x7f0d6815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 shutdown_connections 2024-04-24T06:35:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:35.911+0000 7f0d6cc00640 1 -- 172.21.15.59:0/3013711160 wait complete. 2024-04-24T06:35:35.932 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:35:35.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:35.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:35:36.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60561]: fuse finished with error 0 and tester_r 0 2024-04-24T06:35:36.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:35:36.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:36.021 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:35:36.072 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:35:36.073 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:35:36.073 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:36.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:35:36.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:35:36.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:36.136 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:35:36.193 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:35:36.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:35:36.194 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:35:36.194 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:35:36.194 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:36.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:35:36.259 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:35:36.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:36.259 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:35:36.316 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:35:36.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:35:36.318 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:35:36.318 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:35:36.318 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:36.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:35:36.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:35:36.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:35:36.380 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:35:36.438 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:35:36.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:35:36.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:36.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.659+0000 7f7397200640 1 Processor -- start 2024-04-24T06:35:36.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.659+0000 7f7397200640 1 -- start start 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7397200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f73980d28b0 0x7f739815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7397200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f739815fa20 con 0x7f73980cff10 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7397200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73980cf430 con 0x7f73980d0ec0 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7397200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f73980b3b60 con 0x7f73980d28b0 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7395400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f73980cff10 0x7f73980d21a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40240/0 (socket says 172.21.15.59:40240) 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7395400640 1 -- 172.21.15.59:0/1217675120 learned_addr learned my addr 172.21.15.59:0/1217675120 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:36.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.661+0000 7f7396800640 1 --2- 172.21.15.59:0/1217675120 >> v2:172.21.15.59:3300/0 conn(0x7f73980d28b0 0x7f739815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.662+0000 7f7394a00640 1 -- 172.21.15.59:0/1217675120 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181653207 0 0) 0x7f739815fa20 con 0x7f73980cff10 2024-04-24T06:35:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.662+0000 7f7396800640 1 -- 172.21.15.59:0/1217675120 >> v1:172.21.15.66:6789/0 conn(0x7f73980d0ec0 legacy=0x7f73980d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.662+0000 7f7396800640 1 -- 172.21.15.59:0/1217675120 >> v1:172.21.15.59:6789/0 conn(0x7f73980cff10 legacy=0x7f73980d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.662+0000 7f7396800640 1 -- 172.21.15.59:0/1217675120 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73980cf710 con 0x7f73980d28b0 2024-04-24T06:35:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.663+0000 7f7396800640 1 --2- 172.21.15.59:0/1217675120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d28b0 0x7f739815eb10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7390009d90 tx=0x7f739002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3068bccd16e42f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:36.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.663+0000 7f7394a00640 1 -- 172.21.15.59:0/1217675120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390007740 con 0x7f73980d28b0 2024-04-24T06:35:36.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.663+0000 7f7394a00640 1 -- 172.21.15.59:0/1217675120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73900078e0 con 0x7f73980d28b0 2024-04-24T06:35:36.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.663+0000 7f7394a00640 1 -- 172.21.15.59:0/1217675120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390007cb0 con 0x7f73980d28b0 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 -- 172.21.15.59:0/1217675120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d28b0 msgr2=0x7f739815eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 --2- 172.21.15.59:0/1217675120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d28b0 0x7f739815eb10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7390009d90 tx=0x7f739002a730 comp rx=0 tx=0).stop 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 -- 172.21.15.59:0/1217675120 shutdown_connections 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 --2- 172.21.15.59:0/1217675120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d28b0 0x7f739815eb10 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 -- 172.21.15.59:0/1217675120 >> 172.21.15.59:0/1217675120 conn(0x7f739808bf90 msgr2=0x7f73980d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 -- 172.21.15.59:0/1217675120 shutdown_connections 2024-04-24T06:35:36.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.664+0000 7f7397200640 1 -- 172.21.15.59:0/1217675120 wait complete. 2024-04-24T06:35:36.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.666+0000 7f7397200640 1 Processor -- start 2024-04-24T06:35:36.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.666+0000 7f7397200640 1 -- start start 2024-04-24T06:35:36.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73980cff10 0x7f739814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:36.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73980d28b0 0x7f73980a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73980d0330 con 0x7f73980d0ec0 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f739815fd00 con 0x7f73980d28b0 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7397200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73980d04b0 con 0x7f73980cff10 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7395400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7395400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35154/0 (socket says 172.21.15.59:35154) 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7395400640 1 -- 172.21.15.59:0/4257420159 learned_addr learned my addr 172.21.15.59:0/4257420159 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7396800640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73980d28b0 0x7f73980a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.667+0000 7f7395e00640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73980cff10 0x7f739814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 -- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73980cff10 msgr2=0x7f739814f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73980cff10 0x7f739814f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 -- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73980d28b0 msgr2=0x7f73980a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73980d28b0 0x7f73980a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 -- 172.21.15.59:0/4257420159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73980cb8f0 con 0x7f73980d0ec0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7395400640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f738000c3f0 tx=0x7f738000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800074e0 con 0x7f73980d0ec0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73800047e0 con 0x7f73980d0ec0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380004c10 con 0x7f73980d0ec0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73980cbbd0 con 0x7f73980d0ec0 2024-04-24T06:35:36.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.668+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73980cc190 con 0x7f73980d0ec0 2024-04-24T06:35:36.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.670+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739804fd90 con 0x7f73980d0ec0 2024-04-24T06:35:36.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.671+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7380011070 con 0x7f73980d0ec0 2024-04-24T06:35:36.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.672+0000 7f738e000640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 0x7f737406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:36.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.672+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f738008b7c0 con 0x7f73980d0ec0 2024-04-24T06:35:36.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.672+0000 7f7395e00640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 0x7f737406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:36.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.673+0000 7f7395e00640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 0x7f737406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f73880015b0 tx=0x7f7388009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:36.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.676+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f738001a2b0 con 0x7f73980d0ec0 2024-04-24T06:35:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.920+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f739806ead0 con 0x7f73980d0ec0 2024-04-24T06:35:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.921+0000 7f738e000640 1 -- 172.21.15.59:0/4257420159 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f7380059b50 con 0x7f73980d0ec0 2024-04-24T06:35:36.922 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:36.922 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:35:02.002833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":24,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:35:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 52 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.923+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 msgr2=0x7f737406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.923+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 0x7f737406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f73880015b0 tx=0x7f7388009040 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 msgr2=0x7f73980cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f738000c3f0 tx=0x7f738000c8b0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 shutdown_connections 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f737406c8f0 0x7f737406edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73980d28b0 0x7f73980a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73980d0ec0 0x7f73980cc990 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 --2- 172.21.15.59:0/4257420159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73980cff10 0x7f739814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 >> 172.21.15.59:0/4257420159 conn(0x7f739808bf90 msgr2=0x7f73980b9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 shutdown_connections 2024-04-24T06:35:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:36.924+0000 7f7397200640 1 -- 172.21.15.59:0/4257420159 wait complete. 2024-04-24T06:35:36.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:36.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:36.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:37.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.184+0000 7f835b200640 1 Processor -- start 2024-04-24T06:35:37.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.184+0000 7f835b200640 1 -- start start 2024-04-24T06:35:37.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.185+0000 7f835b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c15c3d0 0x7f835c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.185+0000 7f835b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835c0cd590 0x7f835c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.185+0000 7f835b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c160210 con 0x7f835c0b4690 2024-04-24T06:35:37.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.185+0000 7f835b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0cd9f0 con 0x7f835c0cd590 2024-04-24T06:35:37.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.185+0000 7f835b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f835c1604f0 con 0x7f835c15c3d0 2024-04-24T06:35:37.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835c0cd590 0x7f835c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835c0cd590 0x7f835c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 -- 172.21.15.59:0/3961822106 learned_addr learned my addr 172.21.15.59:0/3961822106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359e00640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c15c3d0 0x7f835c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 -- 172.21.15.59:0/3961822106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c15c3d0 msgr2=0x7f835c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c15c3d0 0x7f835c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.186+0000 7f8359400640 1 -- 172.21.15.59:0/3961822106 >> v1:172.21.15.66:6789/0 conn(0x7f835c0b4690 legacy=0x7f835c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.187+0000 7f8359400640 1 -- 172.21.15.59:0/3961822106 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1607d0 con 0x7f835c0cd590 2024-04-24T06:35:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.188+0000 7f8359400640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c0cd590 0x7f835c0b4150 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf1a7091c563eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.188+0000 7f8358a00640 1 -- 172.21.15.59:0/3961822106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344037070 con 0x7f835c0cd590 2024-04-24T06:35:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f8358a00640 1 -- 172.21.15.59:0/3961822106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344032440 con 0x7f835c0cd590 2024-04-24T06:35:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f8358a00640 1 -- 172.21.15.59:0/3961822106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344032810 con 0x7f835c0cd590 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 -- 172.21.15.59:0/3961822106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c0cd590 msgr2=0x7f835c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c0cd590 0x7f835c0b4150 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8344009870 tx=0x7f834402a270 comp rx=0 tx=0).stop 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 -- 172.21.15.59:0/3961822106 shutdown_connections 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c0cd590 0x7f835c0b4150 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 --2- 172.21.15.59:0/3961822106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c15c3d0 0x7f835c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 -- 172.21.15.59:0/3961822106 >> 172.21.15.59:0/3961822106 conn(0x7f835c0b21a0 msgr2=0x7f835c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:37.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 -- 172.21.15.59:0/3961822106 shutdown_connections 2024-04-24T06:35:37.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.189+0000 7f835b200640 1 -- 172.21.15.59:0/3961822106 wait complete. 2024-04-24T06:35:37.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.191+0000 7f835b200640 1 Processor -- start 2024-04-24T06:35:37.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.191+0000 7f835b200640 1 -- start start 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c0b4690 0x7f835c0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835c0cd590 0x7f835c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0b4ae0 con 0x7f835c15c3d0 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f835c160d90 con 0x7f835c0cd590 2024-04-24T06:35:37.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.192+0000 7f835b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f835c15c7b0 con 0x7f835c0b4690 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35168/0 (socket says 172.21.15.59:35168) 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 -- 172.21.15.59:0/2947044301 learned_addr learned my addr 172.21.15.59:0/2947044301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f8359e00640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c0b4690 0x7f835c0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f8359400640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835c0cd590 0x7f835c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 -- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c0b4690 msgr2=0x7f835c0c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c0b4690 0x7f835c0c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 -- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835c0cd590 msgr2=0x7f835c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835c0cd590 0x7f835c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.193+0000 7f835a800640 1 -- 172.21.15.59:0/2947044301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0cdfb0 con 0x7f835c15c3d0 2024-04-24T06:35:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f835a800640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f835400c7a0 tx=0x7f835400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354003b70 con 0x7f835c15c3d0 2024-04-24T06:35:37.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8354003d30 con 0x7f835c15c3d0 2024-04-24T06:35:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835400aa90 con 0x7f835c15c3d0 2024-04-24T06:35:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c0ce290 con 0x7f835c15c3d0 2024-04-24T06:35:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.194+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c0ce850 con 0x7f835c15c3d0 2024-04-24T06:35:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.195+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c04fd90 con 0x7f835c15c3d0 2024-04-24T06:35:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.196+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f835400ac30 con 0x7f835c15c3d0 2024-04-24T06:35:37.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.197+0000 7f8352000640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 0x7f833806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.197+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f835408bb90 con 0x7f835c15c3d0 2024-04-24T06:35:37.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.197+0000 7f8359e00640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 0x7f833806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.198+0000 7f8359e00640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 0x7f833806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.202+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8354059f20 con 0x7f835c15c3d0 2024-04-24T06:35:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.415+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f835c06e400 con 0x7f835c15c3d0 2024-04-24T06:35:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.417+0000 7f8352000640 1 -- 172.21.15.59:0/2947044301 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16811 (secure 0 0 0) 0x7f8354059820 con 0x7f835c15c3d0 2024-04-24T06:35:37.418 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:37.418 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:28.366221+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 msgr2=0x7f833806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 0x7f833806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f834c006fd0 tx=0x7f834c008040 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 msgr2=0x7f835c1f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f835400c7a0 tx=0x7f835400cc60 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 shutdown_connections 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f833806c8f0 0x7f833806edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835c15c3d0 0x7f835c1f7ff0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f835c0cd590 0x7f835c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 --2- 172.21.15.59:0/2947044301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f835c0b4690 0x7f835c0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 >> 172.21.15.59:0/2947044301 conn(0x7f835c0b21a0 msgr2=0x7f835c0ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 shutdown_connections 2024-04-24T06:35:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.420+0000 7f835b200640 1 -- 172.21.15.59:0/2947044301 wait complete. 2024-04-24T06:35:37.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:33:42.531956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:33:43.698690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:35:37.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:35:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.654+0000 7f4566000640 1 Processor -- start 2024-04-24T06:35:37.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.655+0000 7f4566000640 1 -- start start 2024-04-24T06:35:37.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f456015ed20 0x7f45600d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45600b3600 0x7f45600d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45600b3f10 0x7f45600b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f456015f100 con 0x7f45600b3600 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f45600b5da0 con 0x7f456015ed20 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.656+0000 7f4566000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f45600d0a20 con 0x7f45600b3f10 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45600b3600 0x7f45600d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45600b3600 0x7f45600d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35182/0 (socket says 172.21.15.59:35182) 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 -- 172.21.15.59:0/7948978 learned_addr learned my addr 172.21.15.59:0/7948978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:37.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f4565600640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45600b3f10 0x7f45600b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f4564c00640 1 --2- 172.21.15.59:0/7948978 >> v2:172.21.15.66:3300/0 conn(0x7f456015ed20 0x7f45600d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 -- 172.21.15.59:0/7948978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45600b3f10 msgr2=0x7f45600b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45600b3f10 0x7f45600b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 -- 172.21.15.59:0/7948978 >> v2:172.21.15.66:3300/0 conn(0x7f456015ed20 msgr2=0x7f45600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 --2- 172.21.15.59:0/7948978 >> v2:172.21.15.66:3300/0 conn(0x7f456015ed20 0x7f45600d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.657+0000 7f455fe00640 1 -- 172.21.15.59:0/7948978 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45600d2ed0 con 0x7f45600b3600 2024-04-24T06:35:37.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.658+0000 7f455fe00640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600d13e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f456008e2a0 tx=0x7f454c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eb60db889b0c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.659+0000 7f455f400640 1 -- 172.21.15.59:0/7948978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c037070 con 0x7f45600b3600 2024-04-24T06:35:37.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.659+0000 7f455f400640 1 -- 172.21.15.59:0/7948978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c02ae00 con 0x7f45600b3600 2024-04-24T06:35:37.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.659+0000 7f455f400640 1 -- 172.21.15.59:0/7948978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c032660 con 0x7f45600b3600 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.659+0000 7f4566000640 1 -- 172.21.15.59:0/7948978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 msgr2=0x7f45600d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.659+0000 7f4566000640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600d13e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f456008e2a0 tx=0x7f454c02a0b0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 -- 172.21.15.59:0/7948978 shutdown_connections 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45600b3f10 0x7f45600b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 --2- 172.21.15.59:0/7948978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600d13e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 --2- 172.21.15.59:0/7948978 >> v2:172.21.15.66:3300/0 conn(0x7f456015ed20 0x7f45600d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 -- 172.21.15.59:0/7948978 >> 172.21.15.59:0/7948978 conn(0x7f456008bf90 msgr2=0x7f45600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 -- 172.21.15.59:0/7948978 shutdown_connections 2024-04-24T06:35:37.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.660+0000 7f4566000640 1 -- 172.21.15.59:0/7948978 wait complete. 2024-04-24T06:35:37.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.662+0000 7f4566000640 1 Processor -- start 2024-04-24T06:35:37.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.662+0000 7f4566000640 1 -- start start 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45600b3f10 0x7f45600c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f456015ed20 0x7f45600c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45600b48e0 con 0x7f45600b3600 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45600d3490 con 0x7f45600b3f10 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4566000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f45600b4a60 con 0x7f456015ed20 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4564c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4564c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35186/0 (socket says 172.21.15.59:35186) 2024-04-24T06:35:37.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.663+0000 7f4564c00640 1 -- 172.21.15.59:0/546330533 learned_addr learned my addr 172.21.15.59:0/546330533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 -- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f456015ed20 msgr2=0x7f45600c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4565600640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f456015ed20 0x7f45600c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f456015ed20 0x7f45600c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 -- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45600b3f10 msgr2=0x7f45600c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45600b3f10 0x7f45600c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 -- 172.21.15.59:0/546330533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45600b8500 con 0x7f45600b3600 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4564c00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f455400d620 tx=0x7f455400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400b620 con 0x7f45600b3600 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554019070 con 0x7f45600b3600 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400ba00 con 0x7f45600b3600 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.664+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45600b87e0 con 0x7f45600b3600 2024-04-24T06:35:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.665+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45600b8da0 con 0x7f45600b3600 2024-04-24T06:35:37.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.666+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456004fd90 con 0x7f45600b3600 2024-04-24T06:35:37.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.667+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f455400bba0 con 0x7f45600b3600 2024-04-24T06:35:37.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.668+0000 7f455cc00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:37.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.668+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f455408bfe0 con 0x7f45600b3600 2024-04-24T06:35:37.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.668+0000 7f455fe00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:37.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.669+0000 7f455fe00640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f454c034000 tx=0x7f454c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:37.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.673+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f455405a370 con 0x7f45600b3600 2024-04-24T06:35:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.876+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4560074f60 con 0x7f45600b3600 2024-04-24T06:35:37.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.877+0000 7f455cc00640 1 -- 172.21.15.59:0/546330533 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f4554060030 con 0x7f45600b3600 2024-04-24T06:35:37.878 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 msgr2=0x7f454006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f454c034000 tx=0x7f454c02c000 comp rx=0 tx=0).stop 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 msgr2=0x7f45600b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f455400d620 tx=0x7f455400dae0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 shutdown_connections 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f456015ed20 0x7f45600c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45600b3f10 0x7f45600c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 --2- 172.21.15.59:0/546330533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45600b3600 0x7f45600b95a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 >> 172.21.15.59:0/546330533 conn(0x7f456008bf90 msgr2=0x7f45600cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 shutdown_connections 2024-04-24T06:35:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:37.880+0000 7f4566000640 1 -- 172.21.15.59:0/546330533 wait complete. 2024-04-24T06:35:37.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:35:38.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.120+0000 7f6010000640 1 Processor -- start 2024-04-24T06:35:38.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.121+0000 7f6010000640 1 -- start start 2024-04-24T06:35:38.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008145e10 0x7f600806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60080d0cc0 0x7f60080cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6008071d70 0x7f6008072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f60080cfc20 con 0x7f60080d0cc0 2024-04-24T06:35:38.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f60080cff00 con 0x7f6008071d70 2024-04-24T06:35:38.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.122+0000 7f6010000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60080d01e0 con 0x7f6008145e10 2024-04-24T06:35:38.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.123+0000 7f600cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60080d0cc0 0x7f60080cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.123+0000 7f600cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60080d0cc0 0x7f60080cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35188/0 (socket says 172.21.15.59:35188) 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.123+0000 7f600cc00640 1 -- 172.21.15.59:0/383054850 learned_addr learned my addr 172.21.15.59:0/383054850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.123+0000 7f600e000640 1 --2- 172.21.15.59:0/383054850 >> v2:172.21.15.66:3300/0 conn(0x7f6008071d70 0x7f6008072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.123+0000 7f600d600640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008145e10 0x7f600806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.124+0000 7f600cc00640 1 -- 172.21.15.59:0/383054850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008145e10 msgr2=0x7f600806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.124+0000 7f600cc00640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008145e10 0x7f600806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.124+0000 7f600cc00640 1 -- 172.21.15.59:0/383054850 >> v2:172.21.15.66:3300/0 conn(0x7f6008071d70 msgr2=0x7f6008072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.124+0000 7f600cc00640 1 --2- 172.21.15.59:0/383054850 >> v2:172.21.15.66:3300/0 conn(0x7f6008071d70 0x7f6008072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.124+0000 7f600cc00640 1 -- 172.21.15.59:0/383054850 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60080d04c0 con 0x7f60080d0cc0 2024-04-24T06:35:38.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.125+0000 7f600cc00640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60080cf340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009870 tx=0x7f5ff802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7982066ea91e2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6007e00640 1 -- 172.21.15.59:0/383054850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037070 con 0x7f60080d0cc0 2024-04-24T06:35:38.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6007e00640 1 -- 172.21.15.59:0/383054850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff8032440 con 0x7f60080d0cc0 2024-04-24T06:35:38.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6007e00640 1 -- 172.21.15.59:0/383054850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032810 con 0x7f60080d0cc0 2024-04-24T06:35:38.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 -- 172.21.15.59:0/383054850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 msgr2=0x7f60080cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60080cf340 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009870 tx=0x7f5ff802a210 comp rx=0 tx=0).stop 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 -- 172.21.15.59:0/383054850 shutdown_connections 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 --2- 172.21.15.59:0/383054850 >> v2:172.21.15.66:3300/0 conn(0x7f6008071d70 0x7f6008072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60080cf340 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 --2- 172.21.15.59:0/383054850 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008145e10 0x7f600806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.126+0000 7f6010000640 1 -- 172.21.15.59:0/383054850 >> 172.21.15.59:0/383054850 conn(0x7f6008067d90 msgr2=0x7f600806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.127+0000 7f6010000640 1 -- 172.21.15.59:0/383054850 shutdown_connections 2024-04-24T06:35:38.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.127+0000 7f6010000640 1 -- 172.21.15.59:0/383054850 wait complete. 2024-04-24T06:35:38.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.128+0000 7f6010000640 1 Processor -- start 2024-04-24T06:35:38.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.128+0000 7f6010000640 1 -- start start 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008071d70 0x7f60080aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6008145e10 0x7f60081e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60080cfa00 con 0x7f60080d0cc0 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60080cf880 con 0x7f6008145e10 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.129+0000 7f6010000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60081461f0 con 0x7f6008071d70 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35190/0 (socket says 172.21.15.59:35190) 2024-04-24T06:35:38.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 -- 172.21.15.59:0/1137631484 learned_addr learned my addr 172.21.15.59:0/1137631484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600d600640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008071d70 0x7f60080aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600e000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6008145e10 0x7f60081e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 -- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008071d70 msgr2=0x7f60080aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008071d70 0x7f60080aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 -- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6008145e10 msgr2=0x7f60081e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6008145e10 0x7f60081e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.130+0000 7f600cc00640 1 -- 172.21.15.59:0/1137631484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600813f200 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f600cc00640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037040 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff8042070 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8030880 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60080aaf70 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.131+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081e7b60 con 0x7f60080d0cc0 2024-04-24T06:35:38.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.132+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6008068a80 con 0x7f60080d0cc0 2024-04-24T06:35:38.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.133+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ff8030a20 con 0x7f60080d0cc0 2024-04-24T06:35:38.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.135+0000 7f6005600640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.135+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f5ff80ac5f0 con 0x7f60080d0cc0 2024-04-24T06:35:38.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.135+0000 7f600d600640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.136+0000 7f600d600640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006fd0 tx=0x7f5ffc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.139+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f600813f200 con 0x7f60080d0cc0 2024-04-24T06:35:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.341+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f600815e4d0 con 0x7f60080d0cc0 2024-04-24T06:35:38.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.342+0000 7f6005600640 1 -- 172.21.15.59:0/1137631484 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f5ff8045360 con 0x7f60080d0cc0 2024-04-24T06:35:38.343 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:35:38.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 msgr2=0x7f5fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006fd0 tx=0x7f5ffc008040 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 msgr2=0x7f60081e6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8000c00 tx=0x7f5ff80046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 shutdown_connections 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fec06c8f0 0x7f5fec06edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6008145e10 0x7f60081e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080d0cc0 0x7f60081e6510 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 --2- 172.21.15.59:0/1137631484 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6008071d70 0x7f60080aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 >> 172.21.15.59:0/1137631484 conn(0x7f6008067d90 msgr2=0x7f600815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.345+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 shutdown_connections 2024-04-24T06:35:38.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.346+0000 7f6010000640 1 -- 172.21.15.59:0/1137631484 wait complete. 2024-04-24T06:35:38.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:35:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.576+0000 7f7323200640 1 Processor -- start 2024-04-24T06:35:38.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.576+0000 7f7323200640 1 -- start start 2024-04-24T06:35:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7323200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f731c15ed20 0x7f731c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7323200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f731c0d13e0 0x7f731c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7323200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f731c0b5da0 con 0x7f731c0b3600 2024-04-24T06:35:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7323200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f731c15f100 con 0x7f731c0d13e0 2024-04-24T06:35:38.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7323200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f731c0d07d0 con 0x7f731c15ed20 2024-04-24T06:35:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.578+0000 7f7322800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f731c0d13e0 0x7f731c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7321400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f731c0b3600 0x7f731c0b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40248/0 (socket says 172.21.15.59:40248) 2024-04-24T06:35:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7321400640 1 -- 172.21.15.59:0/3571845117 learned_addr learned my addr 172.21.15.59:0/3571845117 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7322800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f731c0d13e0 0x7f731c0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35206/0 (socket says 172.21.15.59:35206) 2024-04-24T06:35:38.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7321e00640 1 --2- 172.21.15.59:0/3571845117 >> v2:172.21.15.66:3300/0 conn(0x7f731c15ed20 0x7f731c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7322800640 1 -- 172.21.15.59:0/3571845117 >> v2:172.21.15.66:3300/0 conn(0x7f731c15ed20 msgr2=0x7f731c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7322800640 1 --2- 172.21.15.59:0/3571845117 >> v2:172.21.15.66:3300/0 conn(0x7f731c15ed20 0x7f731c0d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7322800640 1 -- 172.21.15.59:0/3571845117 >> v1:172.21.15.59:6789/0 conn(0x7f731c0b3600 legacy=0x7f731c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7320a00640 1 -- 172.21.15.59:0/3571845117 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452355200 0 0) 0x7f731c0b5da0 con 0x7f731c0b3600 2024-04-24T06:35:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.579+0000 7f7322800640 1 -- 172.21.15.59:0/3571845117 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c0d0a20 con 0x7f731c0d13e0 2024-04-24T06:35:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.580+0000 7f7322800640 1 --2- 172.21.15.59:0/3571845117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0d13e0 0x7f731c0d0290 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f731c08e2a0 tx=0x7f731802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd66f810f0dffd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.581+0000 7f7320a00640 1 -- 172.21.15.59:0/3571845117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318037070 con 0x7f731c0d13e0 2024-04-24T06:35:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.581+0000 7f7320a00640 1 -- 172.21.15.59:0/3571845117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731802add0 con 0x7f731c0d13e0 2024-04-24T06:35:38.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.581+0000 7f7320a00640 1 -- 172.21.15.59:0/3571845117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318032660 con 0x7f731c0d13e0 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.581+0000 7f7323200640 1 -- 172.21.15.59:0/3571845117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0d13e0 msgr2=0x7f731c0d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.581+0000 7f7323200640 1 --2- 172.21.15.59:0/3571845117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0d13e0 0x7f731c0d0290 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f731c08e2a0 tx=0x7f731802a080 comp rx=0 tx=0).stop 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 -- 172.21.15.59:0/3571845117 shutdown_connections 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 --2- 172.21.15.59:0/3571845117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0d13e0 0x7f731c0d0290 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 --2- 172.21.15.59:0/3571845117 >> v2:172.21.15.66:3300/0 conn(0x7f731c15ed20 0x7f731c0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 -- 172.21.15.59:0/3571845117 >> 172.21.15.59:0/3571845117 conn(0x7f731c08bf90 msgr2=0x7f731c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:38.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 -- 172.21.15.59:0/3571845117 shutdown_connections 2024-04-24T06:35:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.582+0000 7f7323200640 1 -- 172.21.15.59:0/3571845117 wait complete. 2024-04-24T06:35:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.583+0000 7f7323200640 1 Processor -- start 2024-04-24T06:35:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.584+0000 7f7323200640 1 -- start start 2024-04-24T06:35:38.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f731c0d13e0 0x7f731c0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f731c15ed20 0x7f731c1e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f731c0d3490 con 0x7f731c0b3600 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f731c0d31b0 con 0x7f731c15ed20 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7323200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f731c0b4620 con 0x7f731c0d13e0 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35208/0 (socket says 172.21.15.59:35208) 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321e00640 1 -- 172.21.15.59:0/4131673818 learned_addr learned my addr 172.21.15.59:0/4131673818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321400640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f731c0d13e0 0x7f731c0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7322800640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f731c15ed20 0x7f731c1e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321e00640 1 -- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f731c0d13e0 msgr2=0x7f731c0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.585+0000 7f7321e00640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f731c0d13e0 0x7f731c0cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7321e00640 1 -- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f731c15ed20 msgr2=0x7f731c1e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7321e00640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f731c15ed20 0x7f731c1e4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7321e00640 1 -- 172.21.15.59:0/4131673818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c0c8190 con 0x7f731c0b3600 2024-04-24T06:35:38.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7321e00640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7314007e40 tx=0x7f731400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314014620 con 0x7f731c0b3600 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314019070 con 0x7f731c0b3600 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314014a00 con 0x7f731c0b3600 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c0c8470 con 0x7f731c0b3600 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.586+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c0c8a30 con 0x7f731c0b3600 2024-04-24T06:35:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.587+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c0ccf80 con 0x7f731c0b3600 2024-04-24T06:35:38.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.589+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7314014ba0 con 0x7f731c0b3600 2024-04-24T06:35:38.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.590+0000 7f7312000640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:38.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.590+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f731408c500 con 0x7f731c0b3600 2024-04-24T06:35:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.590+0000 7f7321400640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.591+0000 7f7321400640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f730c007fd0 tx=0x7f730c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.594+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f731405a800 con 0x7f731c0b3600 2024-04-24T06:35:38.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.799+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f731c04fd90 con 0x7f731c0b3600 2024-04-24T06:35:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.800+0000 7f7312000640 1 -- 172.21.15.59:0/4131673818 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f731405a100 con 0x7f731c0b3600 2024-04-24T06:35:38.801 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:35:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 msgr2=0x7f730006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f730c007fd0 tx=0x7f730c009040 comp rx=0 tx=0).stop 2024-04-24T06:35:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 msgr2=0x7f731c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7314007e40 tx=0x7f731400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 shutdown_connections 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f731c15ed20 0x7f731c1e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f731c0d13e0 0x7f731c0cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 --2- 172.21.15.59:0/4131673818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0b3600 0x7f731c0c8f50 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 >> 172.21.15.59:0/4131673818 conn(0x7f731c08bf90 msgr2=0x7f731c15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.803+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 shutdown_connections 2024-04-24T06:35:38.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:38.804+0000 7f7323200640 1 -- 172.21.15.59:0/4131673818 wait complete. 2024-04-24T06:35:38.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.037+0000 7f6f7bc00640 1 Processor -- start 2024-04-24T06:35:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.037+0000 7f6f7bc00640 1 -- start start 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f7bc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f740d42c0 0x7f6f7415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f7bc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f7415edb0 con 0x7f6f7415e9d0 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f7bc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6f740d3e30 con 0x7f6f740d42c0 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7415ff20 con 0x7f6f740d1670 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f740d42c0 0x7f6f7415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f740d42c0 0x7f6f7415f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47482/0 (socket says 172.21.15.59:47482) 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 -- 172.21.15.59:0/313048885 learned_addr learned my addr 172.21.15.59:0/313048885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f79200640 1 --1- 172.21.15.59:0/313048885 >> v1:172.21.15.59:6789/0 conn(0x7f6f7415e9d0 0x7f6f740d0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48844/0 (socket says 172.21.15.59:48844) 2024-04-24T06:35:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 -- 172.21.15.59:0/313048885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 msgr2=0x7f6f740d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 --2- 172.21.15.59:0/313048885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 -- 172.21.15.59:0/313048885 >> v1:172.21.15.59:6789/0 conn(0x7f6f7415e9d0 legacy=0x7f6f740d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.039+0000 7f6f73e00640 1 -- 172.21.15.59:0/313048885 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740ba630 con 0x7f6f740d42c0 2024-04-24T06:35:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f73e00640 1 --2- 172.21.15.59:0/313048885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7415f9e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009e50 tx=0x7f6f5c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a30ce27ab9f9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f73400640 1 -- 172.21.15.59:0/313048885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c036070 con 0x7f6f740d42c0 2024-04-24T06:35:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f73400640 1 -- 172.21.15.59:0/313048885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c031600 con 0x7f6f740d42c0 2024-04-24T06:35:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f73400640 1 -- 172.21.15.59:0/313048885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0319d0 con 0x7f6f740d42c0 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f7bc00640 1 -- 172.21.15.59:0/313048885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 msgr2=0x7f6f7415f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.041+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/313048885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7415f9e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009e50 tx=0x7f6f5c02a710 comp rx=0 tx=0).stop 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 -- 172.21.15.59:0/313048885 shutdown_connections 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/313048885 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/313048885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7415f9e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 -- 172.21.15.59:0/313048885 >> 172.21.15.59:0/313048885 conn(0x7f6f7406cbd0 msgr2=0x7f6f7406cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 -- 172.21.15.59:0/313048885 shutdown_connections 2024-04-24T06:35:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.042+0000 7f6f7bc00640 1 -- 172.21.15.59:0/313048885 wait complete. 2024-04-24T06:35:39.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.043+0000 7f6f7bc00640 1 Processor -- start 2024-04-24T06:35:39.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 -- start start 2024-04-24T06:35:39.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f7415e9d0 0x7f6f740cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740cfd70 con 0x7f6f740d42c0 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740cfa90 con 0x7f6f7415e9d0 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.044+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6f740ba0e0 con 0x7f6f740d1670 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47490/0 (socket says 172.21.15.59:47490) 2024-04-24T06:35:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 -- 172.21.15.59:0/2746934535 learned_addr learned my addr 172.21.15.59:0/2746934535 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:39.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f79c00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f7415e9d0 0x7f6f740cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f79200640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 -- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 msgr2=0x7f6f740caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 -- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f7415e9d0 msgr2=0x7f6f740cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f7415e9d0 0x7f6f740cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.045+0000 7f6f73e00640 1 -- 172.21.15.59:0/2746934535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7414dae0 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f73e00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009e50 tx=0x7f6f5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c041020 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c007800 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7414ddc0 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c036040 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.046+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741f4ce0 con 0x7f6f740d42c0 2024-04-24T06:35:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.048+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f74069190 con 0x7f6f740d42c0 2024-04-24T06:35:39.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.048+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f5c044020 con 0x7f6f740d42c0 2024-04-24T06:35:39.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.049+0000 7f6f70c00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.049+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f6f5c0ac9c0 con 0x7f6f740d42c0 2024-04-24T06:35:39.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.050+0000 7f6f79200640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.050+0000 7f6f79200640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 0x7f6f5006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000f60 tx=0x7f6f64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.054+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f5c0aee20 con 0x7f6f740d42c0 2024-04-24T06:35:39.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.299+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f741f4ab0 con 0x7f6f740d42c0 2024-04-24T06:35:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.301+0000 7f6f70c00640 1 -- 172.21.15.59:0/2746934535 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f6f5c04b030 con 0x7f6f740d42c0 2024-04-24T06:35:39.302 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:39.302 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:35:02.002833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":24,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:35:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 52 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 msgr2=0x7f6f5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 0x7f6f5006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000f60 tx=0x7f6f64009040 comp rx=0 tx=0).stop 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 msgr2=0x7f6f7414d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009e50 tx=0x7f6f5c004a40 comp rx=0 tx=0).stop 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 shutdown_connections 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f5006c8f0 0x7f6f5006edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f7415e9d0 0x7f6f740cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f740d42c0 0x7f6f7414d370 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 --2- 172.21.15.59:0/2746934535 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f740d1670 0x7f6f740caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 >> 172.21.15.59:0/2746934535 conn(0x7f6f7406cbd0 msgr2=0x7f6f740b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 shutdown_connections 2024-04-24T06:35:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.304+0000 7f6f7bc00640 1 -- 172.21.15.59:0/2746934535 wait complete. 2024-04-24T06:35:39.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.536+0000 7f2931600640 1 Processor -- start 2024-04-24T06:35:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.536+0000 7f2931600640 1 -- start start 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.537+0000 7f2931600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f292c15e750 0x7f292c0ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f2931600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f292c15eb30 0x7f292c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f2931600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0b8fc0 con 0x7f292c0cfa20 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f2931600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f292c0b92a0 con 0x7f292c15e750 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f2931600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f292c0b9580 con 0x7f292c15eb30 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f292b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f292c0cfa20 0x7f292c0d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48856/0 (socket says 172.21.15.59:48856) 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f292b400640 1 -- 172.21.15.59:0/1902306088 learned_addr learned my addr 172.21.15.59:0/1902306088 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f292aa00640 1 --2- 172.21.15.59:0/1902306088 >> v2:172.21.15.59:3300/0 conn(0x7f292c15e750 0x7f292c0ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.538+0000 7f292a000640 1 --2- 172.21.15.59:0/1902306088 >> v2:172.21.15.66:3300/0 conn(0x7f292c15eb30 0x7f292c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f2929600640 1 -- 172.21.15.59:0/1902306088 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856523323 0 0) 0x7f292c0b8fc0 con 0x7f292c0cfa20 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f2929600640 1 -- 172.21.15.59:0/1902306088 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2910003520 con 0x7f292c0cfa20 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f292aa00640 1 -- 172.21.15.59:0/1902306088 >> v2:172.21.15.66:3300/0 conn(0x7f292c15eb30 msgr2=0x7f292c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f292aa00640 1 --2- 172.21.15.59:0/1902306088 >> v2:172.21.15.66:3300/0 conn(0x7f292c15eb30 0x7f292c0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f292aa00640 1 -- 172.21.15.59:0/1902306088 >> v1:172.21.15.59:6789/0 conn(0x7f292c0cfa20 legacy=0x7f292c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.539+0000 7f292aa00640 1 -- 172.21.15.59:0/1902306088 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c0b9860 con 0x7f292c15e750 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.540+0000 7f292aa00640 1 --2- 172.21.15.59:0/1902306088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c0ba1a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b7710 tx=0x7f291c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca4ee1ccd897042 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.540+0000 7f2929600640 1 -- 172.21.15.59:0/1902306088 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c037070 con 0x7f292c15e750 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.540+0000 7f2929600640 1 -- 172.21.15.59:0/1902306088 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c032440 con 0x7f292c15e750 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.540+0000 7f2929600640 1 -- 172.21.15.59:0/1902306088 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c032810 con 0x7f292c15e750 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 -- 172.21.15.59:0/1902306088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 msgr2=0x7f292c0ba1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 --2- 172.21.15.59:0/1902306088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c0ba1a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b7710 tx=0x7f291c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 -- 172.21.15.59:0/1902306088 shutdown_connections 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 --2- 172.21.15.59:0/1902306088 >> v2:172.21.15.66:3300/0 conn(0x7f292c15eb30 0x7f292c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 --2- 172.21.15.59:0/1902306088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c0ba1a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 -- 172.21.15.59:0/1902306088 >> 172.21.15.59:0/1902306088 conn(0x7f292c067b50 msgr2=0x7f292c067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 -- 172.21.15.59:0/1902306088 shutdown_connections 2024-04-24T06:35:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.541+0000 7f2931600640 1 -- 172.21.15.59:0/1902306088 wait complete. 2024-04-24T06:35:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.543+0000 7f2931600640 1 Processor -- start 2024-04-24T06:35:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.543+0000 7f2931600640 1 -- start start 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f292c0cfa20 0x7f292c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f292c15eb30 0x7f292c161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0b8950 con 0x7f292c15e750 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0ba6e0 con 0x7f292c15eb30 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f2931600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f292c0b8ad0 con 0x7f292c0cfa20 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f292a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f292a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47514/0 (socket says 172.21.15.59:47514) 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.544+0000 7f292a000640 1 -- 172.21.15.59:0/483516020 learned_addr learned my addr 172.21.15.59:0/483516020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 -- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f292c0cfa20 msgr2=0x7f292c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f292c0cfa20 0x7f292c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 -- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f292c15eb30 msgr2=0x7f292c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292b400640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f292c15eb30 0x7f292c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f292c15eb30 0x7f292c161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 -- 172.21.15.59:0/483516020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c161d90 con 0x7f292c15e750 2024-04-24T06:35:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f292a000640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f291400d5c0 tx=0x7f291400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140125f0 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914019070 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140129d0 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.545+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c0c8470 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.546+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c0c8a30 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.547+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c068a80 con 0x7f292c15e750 2024-04-24T06:35:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.548+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2914012b70 con 0x7f292c15e750 2024-04-24T06:35:39.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.549+0000 7f2922a00640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 0x7f290806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:39.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.549+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f291408bf30 con 0x7f292c15e750 2024-04-24T06:35:39.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.549+0000 7f292aa00640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 0x7f290806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:39.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.550+0000 7f292aa00640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 0x7f290806edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f291c034000 tx=0x7f291c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:39.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.553+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f291405a360 con 0x7f292c15e750 2024-04-24T06:35:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.759+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f292c0d1af0 con 0x7f292c15e750 2024-04-24T06:35:39.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.762+0000 7f2922a00640 1 -- 172.21.15.59:0/483516020 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16811 (secure 0 0 0) 0x7f2914059c60 con 0x7f292c15e750 2024-04-24T06:35:39.763 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:39.764 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:28.366221+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.764+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 msgr2=0x7f290806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.764+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 0x7f290806edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f291c034000 tx=0x7f291c02c000 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.764+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 msgr2=0x7f292c152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.764+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f291400d5c0 tx=0x7f291400da80 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 shutdown_connections 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f290806c8f0 0x7f290806edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f292c15eb30 0x7f292c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f292c15e750 0x7f292c152d90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 --2- 172.21.15.59:0/483516020 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f292c0cfa20 0x7f292c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 >> 172.21.15.59:0/483516020 conn(0x7f292c067b50 msgr2=0x7f292c0d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 shutdown_connections 2024-04-24T06:35:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.765+0000 7f2931600640 1 -- 172.21.15.59:0/483516020 wait complete. 2024-04-24T06:35:39.787 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:35:39.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.998+0000 7f8032e00640 1 Processor -- start 2024-04-24T06:35:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:39.998+0000 7f8032e00640 1 -- start start 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8032e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f802c0cd760 0x7f802c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8032e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f802c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8032e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802c160210 con 0x7f802c15c3d0 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8032e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f802c1604f0 con 0x7f802c0cd760 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8032e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f802c1607d0 con 0x7f802c0b4690 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8031a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f802c15c3d0 0x7f802c0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48872/0 (socket says 172.21.15.59:48872) 2024-04-24T06:35:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.000+0000 7f8031a00640 1 -- 172.21.15.59:0/803174740 learned_addr learned my addr 172.21.15.59:0/803174740 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f8032400640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f802c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f8031000640 1 --2- 172.21.15.59:0/803174740 >> v2:172.21.15.66:3300/0 conn(0x7f802c0cd760 0x7f802c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207924825 0 0) 0x7f802c160210 con 0x7f802c15c3d0 2024-04-24T06:35:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8014003520 con 0x7f802c15c3d0 2024-04-24T06:35:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2460452020 0 0) 0x7f8014003520 con 0x7f802c15c3d0 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 msgr2=0x7f802c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f802c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 >> v2:172.21.15.66:3300/0 conn(0x7f802c0cd760 msgr2=0x7f802c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 --2- 172.21.15.59:0/803174740 >> v2:172.21.15.66:3300/0 conn(0x7f802c0cd760 0x7f802c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c160ab0 con 0x7f802c15c3d0 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.001+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8020001670 con 0x7f802c15c3d0 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 >> v1:172.21.15.59:6789/0 conn(0x7f802c15c3d0 legacy=0x7f802c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80140039b0 0x7f8014003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f8014004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014004d40 0x7f80140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1604f0 con 0x7f8014004d40 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1607d0 con 0x7f80140039b0 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f801400b9c0 con 0x7f802c0b4690 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014004d40 0x7f80140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8031a00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80140039b0 0x7f8014003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8031000640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f8014004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 -- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 msgr2=0x7f8014004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0b4690 0x7f8014004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 -- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80140039b0 msgr2=0x7f8014003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80140039b0 0x7f8014003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8032400640 1 -- 172.21.15.59:0/803174740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8014003520 con 0x7f8014004d40 2024-04-24T06:35:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.002+0000 7f8031a00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80140039b0 0x7f8014003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.003+0000 7f8032400640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014004d40 0x7f80140051c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8024009fc0 tx=0x7f8014005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8399bc6135d86992 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024009c10 con 0x7f8014004d40 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8024036070 con 0x7f8014004d40 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f802be00640 1 -- 172.21.15.59:0/803174740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024031880 con 0x7f8014004d40 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f8032e00640 1 -- 172.21.15.59:0/803174740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014004d40 msgr2=0x7f80140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f8032e00640 1 --2- 172.21.15.59:0/803174740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014004d40 0x7f80140051c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8024009fc0 tx=0x7f8014005830 comp rx=0 tx=0).stop 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f8032400640 1 -- 172.21.15.59:0/803174740 reap_dead start 2024-04-24T06:35:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f8032e00640 1 -- 172.21.15.59:0/803174740 shutdown_connections 2024-04-24T06:35:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.004+0000 7f8032e00640 1 -- 172.21.15.59:0/803174740 >> 172.21.15.59:0/803174740 conn(0x7f802c0b21a0 msgr2=0x7f802c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.005+0000 7f8032e00640 1 -- 172.21.15.59:0/803174740 shutdown_connections 2024-04-24T06:35:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.005+0000 7f8032e00640 1 -- 172.21.15.59:0/803174740 wait complete. 2024-04-24T06:35:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.006+0000 7f8032e00640 1 Processor -- start 2024-04-24T06:35:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.006+0000 7f8032e00640 1 -- start start 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 0x7f802c0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0cc180 0x7f802c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f802c15c7b0 con 0x7f802c0cc600 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f802c0b4b00 con 0x7f8014004d40 2024-04-24T06:35:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.007+0000 7f8032e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f802c15c930 con 0x7f802c0cc180 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47534/0 (socket says 172.21.15.59:47534) 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 -- 172.21.15.59:0/898905368 learned_addr learned my addr 172.21.15.59:0/898905368 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8031a00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 0x7f802c0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8031000640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0cc180 0x7f802c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 -- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0cc180 msgr2=0x7f802c158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0cc180 0x7f802c158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 -- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 msgr2=0x7f802c0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 0x7f802c0cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8032400640 1 -- 172.21.15.59:0/898905368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c158ed0 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.008+0000 7f8031a00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 0x7f802c0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8032400640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f802400e230 tx=0x7f8024009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024009c10 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8024031dd0 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024003bd0 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c0cee40 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.009+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c0cf400 con 0x7f802c0cc600 2024-04-24T06:35:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.011+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c0cf600 con 0x7f802c0cc600 2024-04-24T06:35:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.012+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8024005d10 con 0x7f802c0cc600 2024-04-24T06:35:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.013+0000 7f8029600640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 0x7f800c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.013+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f80240ac900 con 0x7f802c0cc600 2024-04-24T06:35:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.013+0000 7f8031a00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 0x7f800c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.014+0000 7f8031a00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 0x7f800c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8020001eb0 tx=0x7f8020009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.018+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f802c0cf600 con 0x7f802c0cc600 2024-04-24T06:35:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.267+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802c073b00 con 0x7f802c0cc600 2024-04-24T06:35:40.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.268+0000 7f8029600640 1 -- 172.21.15.59:0/898905368 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f802407ac00 con 0x7f802c0cc600 2024-04-24T06:35:40.269 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:40.269 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:35:02.002833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":24,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:35:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 52 2024-04-24T06:35:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 msgr2=0x7f800c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 0x7f800c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8020001eb0 tx=0x7f8020009040 comp rx=0 tx=0).stop 2024-04-24T06:35:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 msgr2=0x7f802c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f802400e230 tx=0x7f8024009750 comp rx=0 tx=0).stop 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 shutdown_connections 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f800c06c8f0 0x7f800c06edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f802c0cc600 0x7f802c156e70 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f802c0cc180 0x7f802c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 --2- 172.21.15.59:0/898905368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8014004d40 0x7f802c0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.271+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 >> 172.21.15.59:0/898905368 conn(0x7f802c0b21a0 msgr2=0x7f802c0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.272+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 shutdown_connections 2024-04-24T06:35:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.272+0000 7f8032e00640 1 -- 172.21.15.59:0/898905368 wait complete. 2024-04-24T06:35:40.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:40.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.503+0000 7ffbc1400640 1 Processor -- start 2024-04-24T06:35:40.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.503+0000 7ffbc1400640 1 -- start start 2024-04-24T06:35:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ffbbc15b450 0x7ffbbc0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffbbc0cd760 0x7ffbbc0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc1400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbbc0b3870 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc1400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffbbc0b41a0 con 0x7ffbbc0cd760 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc1400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ffbbc0b3b50 con 0x7ffbbc15b450 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbc0a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffbbc0cd760 0x7ffbbc0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbb3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffbbc06f3f0 0x7ffbbc0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48880/0 (socket says 172.21.15.59:48880) 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.505+0000 7ffbb3400640 1 -- 172.21.15.59:0/2133253129 learned_addr learned my addr 172.21.15.59:0/2133253129 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbbbe00640 1 --2- 172.21.15.59:0/2133253129 >> v2:172.21.15.66:3300/0 conn(0x7ffbbc15b450 0x7ffbbc0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbc0a00640 1 -- 172.21.15.59:0/2133253129 >> v2:172.21.15.66:3300/0 conn(0x7ffbbc15b450 msgr2=0x7ffbbc0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbc0a00640 1 --2- 172.21.15.59:0/2133253129 >> v2:172.21.15.66:3300/0 conn(0x7ffbbc15b450 0x7ffbbc0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbc0a00640 1 -- 172.21.15.59:0/2133253129 >> v1:172.21.15.59:6789/0 conn(0x7ffbbc06f3f0 legacy=0x7ffbbc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbbb400640 1 -- 172.21.15.59:0/2133253129 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749922040 0 0) 0x7ffbbc0b3870 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.506+0000 7ffbc0a00640 1 -- 172.21.15.59:0/2133253129 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc0b3e30 con 0x7ffbbc0cd760 2024-04-24T06:35:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.507+0000 7ffbc0a00640 1 --2- 172.21.15.59:0/2133253129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc0cd760 0x7ffbbc0ad060 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc06cb20 tx=0x7ffbb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d845ca0a3df11a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.507+0000 7ffbbb400640 1 -- 172.21.15.59:0/2133253129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4037070 con 0x7ffbbc0cd760 2024-04-24T06:35:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.507+0000 7ffbbb400640 1 -- 172.21.15.59:0/2133253129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4032440 con 0x7ffbbc0cd760 2024-04-24T06:35:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.507+0000 7ffbbb400640 1 -- 172.21.15.59:0/2133253129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4032810 con 0x7ffbbc0cd760 2024-04-24T06:35:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 -- 172.21.15.59:0/2133253129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc0cd760 msgr2=0x7ffbbc0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 --2- 172.21.15.59:0/2133253129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc0cd760 0x7ffbbc0ad060 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc06cb20 tx=0x7ffbb402a230 comp rx=0 tx=0).stop 2024-04-24T06:35:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 -- 172.21.15.59:0/2133253129 shutdown_connections 2024-04-24T06:35:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 --2- 172.21.15.59:0/2133253129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc0cd760 0x7ffbbc0ad060 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 --2- 172.21.15.59:0/2133253129 >> v2:172.21.15.66:3300/0 conn(0x7ffbbc15b450 0x7ffbbc0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 -- 172.21.15.59:0/2133253129 >> 172.21.15.59:0/2133253129 conn(0x7ffbbc0b21a0 msgr2=0x7ffbbc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 -- 172.21.15.59:0/2133253129 shutdown_connections 2024-04-24T06:35:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.508+0000 7ffbc1400640 1 -- 172.21.15.59:0/2133253129 wait complete. 2024-04-24T06:35:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.510+0000 7ffbc1400640 1 Processor -- start 2024-04-24T06:35:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.510+0000 7ffbc1400640 1 -- start start 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbbc0cd760 0x7ffbbc1fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbbc15b450 0x7ffbbc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc0b4ff0 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc15b830 con 0x7ffbbc15b450 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbc1400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffbbc0b3080 con 0x7ffbbc0cd760 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbbbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbbbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47544/0 (socket says 172.21.15.59:47544) 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.511+0000 7ffbbbe00640 1 -- 172.21.15.59:0/704322770 learned_addr learned my addr 172.21.15.59:0/704322770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 -- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbbc0cd760 msgr2=0x7ffbbc1fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbbc0cd760 0x7ffbbc1fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 -- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbbc15b450 msgr2=0x7ffbbc15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbbc15b450 0x7ffbbc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 -- 172.21.15.59:0/704322770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc15e870 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbbbe00640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d620 tx=0x7ffbac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00b620 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac019070 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00ba00 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.512+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc0720d0 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.513+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc072690 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.514+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbbc04fd90 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.515+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbac00bba0 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.516+0000 7ffbb8c00640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 0x7ffb9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.516+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7ffbac08c100 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.516+0000 7ffbb3400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 0x7ffb9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.517+0000 7ffbb3400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 0x7ffb9806edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001560 tx=0x7ffba4007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.520+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbac05a400 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.767+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbbc06c510 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.768+0000 7ffbb8c00640 1 -- 172.21.15.59:0/704322770 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7ffbac059d00 con 0x7ffbbc06f3f0 2024-04-24T06:35:40.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:40.769 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:33:44.703546+0000","modified":"2024-04-24T06:35:02.002833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":24,"addr":"172.21.15.59:6837/209407257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":209407257},{"type":"v1","addr":"172.21.15.59:6837","nonce":209407257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":5}]} 2024-04-24T06:35:40.770 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 52 2024-04-24T06:35:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 msgr2=0x7ffb9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 0x7ffb9806edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001560 tx=0x7ffba4007ca0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 msgr2=0x7ffbbc1f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d620 tx=0x7ffbac00dae0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 shutdown_connections 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb9806c8f0 0x7ffb9806edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbbc15b450 0x7ffbbc15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffbbc0cd760 0x7ffbbc1fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 --2- 172.21.15.59:0/704322770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbbc06f3f0 0x7ffbbc1f8cd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 >> 172.21.15.59:0/704322770 conn(0x7ffbbc0b21a0 msgr2=0x7ffbbc0cb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 shutdown_connections 2024-04-24T06:35:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.771+0000 7ffbc1400640 1 -- 172.21.15.59:0/704322770 wait complete. 2024-04-24T06:35:40.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.993+0000 7f1fd3800640 1 Processor -- start 2024-04-24T06:35:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.993+0000 7f1fd3800640 1 -- start start 2024-04-24T06:35:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.994+0000 7f1fd3800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fcc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.994+0000 7f1fd3800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1fcc0b3e40 0x7f1fcc160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.994+0000 7f1fd3800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fcc0cd520 con 0x7f1fcc1454e0 2024-04-24T06:35:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.994+0000 7f1fd3800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1fcc0cddc0 con 0x7f1fcc0b3e40 2024-04-24T06:35:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.994+0000 7f1fd3800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc1462c0 con 0x7f1fcc0cd020 2024-04-24T06:35:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fd0e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1fcc1454e0 0x7f1fcc145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48886/0 (socket says 172.21.15.59:48886) 2024-04-24T06:35:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fd0e00640 1 -- 172.21.15.59:0/2352296794 learned_addr learned my addr 172.21.15.59:0/2352296794 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fd1800640 1 --2- 172.21.15.59:0/2352296794 >> v2:172.21.15.66:3300/0 conn(0x7f1fcc0b3e40 0x7f1fcc160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fc3e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fcc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271403906 0 0) 0x7f1fcc0cd520 con 0x7f1fcc1454e0 2024-04-24T06:35:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.995+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb0003520 con 0x7f1fcc1454e0 2024-04-24T06:35:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 988316506 0 0) 0x7f1fb0003520 con 0x7f1fcc1454e0 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 msgr2=0x7f1fcc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fcc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 >> v2:172.21.15.66:3300/0 conn(0x7f1fcc0b3e40 msgr2=0x7f1fcc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 --2- 172.21.15.59:0/2352296794 >> v2:172.21.15.66:3300/0 conn(0x7f1fcc0b3e40 0x7f1fcc160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc0cd800 con 0x7f1fcc1454e0 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1fbc001670 con 0x7f1fcc1454e0 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 >> v1:172.21.15.59:6789/0 conn(0x7f1fcc1454e0 legacy=0x7f1fcc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fb00039b0 0x7f1fb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fb0004800 0x7f1fb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0cddc0 con 0x7f1fb00039b0 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1462c0 con 0x7f1fb0004800 2024-04-24T06:35:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb000ba10 con 0x7f1fcc0cd020 2024-04-24T06:35:40.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.996+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fb00039b0 0x7f1fb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fc3e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fb0004800 0x7f1fb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd1800640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd0e00640 1 -- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 msgr2=0x7f1fb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc0cd020 0x7f1fb00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd0e00640 1 -- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fb0004800 msgr2=0x7f1fb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fb0004800 0x7f1fb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.997+0000 7f1fd0e00640 1 -- 172.21.15.59:0/2352296794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb0003520 con 0x7f1fb00039b0 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.998+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fb00039b0 0x7f1fb0003dd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c1c0 tx=0x7f1fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31564e236d1d5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007c80 con 0x7f1fb00039b0 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc02e070 con 0x7f1fb00039b0 2024-04-24T06:35:41.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fcbe00640 1 -- 172.21.15.59:0/2352296794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005d30 con 0x7f1fb00039b0 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 -- 172.21.15.59:0/2352296794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fb00039b0 msgr2=0x7f1fb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 --2- 172.21.15.59:0/2352296794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fb00039b0 0x7f1fb0003dd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c1c0 tx=0x7f1fb0005830 comp rx=0 tx=0).stop 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd1800640 1 -- 172.21.15.59:0/2352296794 reap_dead start 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 -- 172.21.15.59:0/2352296794 shutdown_connections 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 -- 172.21.15.59:0/2352296794 >> 172.21.15.59:0/2352296794 conn(0x7f1fcc067ea0 msgr2=0x7f1fcc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 -- 172.21.15.59:0/2352296794 shutdown_connections 2024-04-24T06:35:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:40.999+0000 7f1fd3800640 1 -- 172.21.15.59:0/2352296794 wait complete. 2024-04-24T06:35:41.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.001+0000 7f1fd3800640 1 Processor -- start 2024-04-24T06:35:41.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.002+0000 7f1fd3800640 1 -- start start 2024-04-24T06:35:41.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc1f84e0 0x7f1fcc15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc143fb0 0x7f1fcc13e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b3900 con 0x7f1fcc143bd0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1603f0 con 0x7f1fcc1f84e0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd3800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b3a80 con 0x7f1fcc143fb0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47550/0 (socket says 172.21.15.59:47550) 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 -- 172.21.15.59:0/3949283736 learned_addr learned my addr 172.21.15.59:0/3949283736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fc3e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc1f84e0 0x7f1fcc15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 -- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc143fb0 msgr2=0x7f1fcc13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc143fb0 0x7f1fcc13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 -- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc1f84e0 msgr2=0x7f1fcc15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc1f84e0 0x7f1fcc15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.003+0000 7f1fd0e00640 1 -- 172.21.15.59:0/3949283736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc15cd20 con 0x7f1fcc143bd0 2024-04-24T06:35:41.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fd0e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c160 tx=0x7f1fbc001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007c80 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc007480 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc001bc0 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc154770 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.004+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc154d30 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.005+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc068a80 con 0x7f1fcc143bd0 2024-04-24T06:35:41.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.006+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fbc033470 con 0x7f1fcc143bd0 2024-04-24T06:35:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.007+0000 7f1fc9600640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 0x7f1fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.008+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f1fbc0aba80 con 0x7f1fcc143bd0 2024-04-24T06:35:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.008+0000 7f1fc3e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 0x7f1fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.009+0000 7f1fc3e00640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 0x7f1fa806edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1f88c0 tx=0x7f1fb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.012+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fbc0ade20 con 0x7f1fcc143bd0 2024-04-24T06:35:41.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.217+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fcc160030 con 0x7f1fcc143bd0 2024-04-24T06:35:41.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.220+0000 7f1fc9600640 1 -- 172.21.15.59:0/3949283736 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16811 (secure 0 0 0) 0x7f1fbc079e40 con 0x7f1fcc143bd0 2024-04-24T06:35:41.222 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:41.222 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:34:28.366221+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:33:42.531956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-24T06:33:43.698690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:41.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.223+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 msgr2=0x7f1fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.223+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 0x7f1fa806edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1f88c0 tx=0x7f1fb4008040 comp rx=0 tx=0).stop 2024-04-24T06:35:41.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.223+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 msgr2=0x7f1fcc1f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.223+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c160 tx=0x7f1fbc001970 comp rx=0 tx=0).stop 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 shutdown_connections 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1fa806c8f0 0x7f1fa806edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1fcc143fb0 0x7f1fcc13e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fcc1f84e0 0x7f1fcc15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 --2- 172.21.15.59:0/3949283736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fcc143bd0 0x7f1fcc1f7fa0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 >> 172.21.15.59:0/3949283736 conn(0x7f1fcc067ea0 msgr2=0x7f1fcc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 shutdown_connections 2024-04-24T06:35:41.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.224+0000 7f1fd3800640 1 -- 172.21.15.59:0/3949283736 wait complete. 2024-04-24T06:35:41.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:35:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.451+0000 7fdb6ee00640 1 Processor -- start 2024-04-24T06:35:41.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.451+0000 7fdb6ee00640 1 -- start start 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb680af890 0x7fdb6815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb68144a20 0x7fdb680d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb680b3720 con 0x7fdb680af4b0 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdb68144ed0 con 0x7fdb68144a20 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdb6815f3f0 con 0x7fdb680af890 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdb680af4b0 0x7fdb680b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48900/0 (socket says 172.21.15.59:48900) 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6da00640 1 -- 172.21.15.59:0/2448889516 learned_addr learned my addr 172.21.15.59:0/2448889516 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6e400640 1 --2- 172.21.15.59:0/2448889516 >> v2:172.21.15.59:3300/0 conn(0x7fdb68144a20 0x7fdb680d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.453+0000 7fdb6d000640 1 --2- 172.21.15.59:0/2448889516 >> v2:172.21.15.66:3300/0 conn(0x7fdb680af890 0x7fdb6815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb67e00640 1 -- 172.21.15.59:0/2448889516 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704713965 0 0) 0x7fdb680b3720 con 0x7fdb680af4b0 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb67e00640 1 -- 172.21.15.59:0/2448889516 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb50003520 con 0x7fdb680af4b0 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb6e400640 1 -- 172.21.15.59:0/2448889516 >> v2:172.21.15.66:3300/0 conn(0x7fdb680af890 msgr2=0x7fdb6815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb6e400640 1 --2- 172.21.15.59:0/2448889516 >> v2:172.21.15.66:3300/0 conn(0x7fdb680af890 0x7fdb6815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb6e400640 1 -- 172.21.15.59:0/2448889516 >> v1:172.21.15.59:6789/0 conn(0x7fdb680af4b0 legacy=0x7fdb680b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.454+0000 7fdb6e400640 1 -- 172.21.15.59:0/2448889516 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6815f6d0 con 0x7fdb68144a20 2024-04-24T06:35:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb6e400640 1 --2- 172.21.15.59:0/2448889516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb680d05c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb680cdc00 tx=0x7fdb6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619414e37b802099 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb67e00640 1 -- 172.21.15.59:0/2448889516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007740 con 0x7fdb68144a20 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb67e00640 1 -- 172.21.15.59:0/2448889516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb600078e0 con 0x7fdb68144a20 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb67e00640 1 -- 172.21.15.59:0/2448889516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007c90 con 0x7fdb68144a20 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb6ee00640 1 -- 172.21.15.59:0/2448889516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 msgr2=0x7fdb680d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.456+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/2448889516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb680d05c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb680cdc00 tx=0x7fdb6002a680 comp rx=0 tx=0).stop 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 -- 172.21.15.59:0/2448889516 shutdown_connections 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/2448889516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb680d05c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/2448889516 >> v2:172.21.15.66:3300/0 conn(0x7fdb680af890 0x7fdb6815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 -- 172.21.15.59:0/2448889516 >> 172.21.15.59:0/2448889516 conn(0x7fdb6806bdf0 msgr2=0x7fdb6806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 -- 172.21.15.59:0/2448889516 shutdown_connections 2024-04-24T06:35:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.457+0000 7fdb6ee00640 1 -- 172.21.15.59:0/2448889516 wait complete. 2024-04-24T06:35:41.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.459+0000 7fdb6ee00640 1 Processor -- start 2024-04-24T06:35:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.459+0000 7fdb6ee00640 1 -- start start 2024-04-24T06:35:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.460+0000 7fdb6ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb680af4b0 0x7fdb68150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.460+0000 7fdb6ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb6809d670 0x7fdb6809db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb680d0b00 con 0x7fdb68144a20 2024-04-24T06:35:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6815fc90 con 0x7fdb680af4b0 2024-04-24T06:35:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdb680d0c80 con 0x7fdb6809d670 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47560/0 (socket says 172.21.15.59:47560) 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6d000640 1 -- 172.21.15.59:0/321330093 learned_addr learned my addr 172.21.15.59:0/321330093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6da00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb680af4b0 0x7fdb68150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.461+0000 7fdb6e400640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb6809d670 0x7fdb6809db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 -- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb6809d670 msgr2=0x7fdb6809db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb6809d670 0x7fdb6809db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 -- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb680af4b0 msgr2=0x7fdb68150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb680af4b0 0x7fdb68150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 -- 172.21.15.59:0/321330093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb68153c70 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6d000640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9b0 tx=0x7fdb5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017070 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5400c9a0 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400cd90 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb681508a0 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.462+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6809a8a0 con 0x7fdb68144a20 2024-04-24T06:35:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.464+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6809b230 con 0x7fdb68144a20 2024-04-24T06:35:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.465+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb5401a070 con 0x7fdb68144a20 2024-04-24T06:35:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.466+0000 7fdb65600640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.466+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fdb5408bd60 con 0x7fdb68144a20 2024-04-24T06:35:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.466+0000 7fdb6da00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.467+0000 7fdb6da00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 0x7fdb4806edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005880 tx=0x7fdb5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.470+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb5405a060 con 0x7fdb68144a20 2024-04-24T06:35:41.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.715+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdb6815f130 con 0x7fdb68144a20 2024-04-24T06:35:41.717 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:35:41.716+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:35:41.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.719+0000 7fdb65600640 1 -- 172.21.15.59:0/321330093 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7fdb54059960 con 0x7fdb68144a20 2024-04-24T06:35:41.720 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:35:41.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.722+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 msgr2=0x7fdb4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.722+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 0x7fdb4806edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005880 tx=0x7fdb5c009040 comp rx=0 tx=0).stop 2024-04-24T06:35:41.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.722+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 msgr2=0x7fdb6806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.722+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400e9b0 tx=0x7fdb5400ee70 comp rx=0 tx=0).stop 2024-04-24T06:35:41.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 shutdown_connections 2024-04-24T06:35:41.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb4806c8f0 0x7fdb4806edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb6809d670 0x7fdb6809db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68144a20 0x7fdb6806ee40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 --2- 172.21.15.59:0/321330093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb680af4b0 0x7fdb68150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 >> 172.21.15.59:0/321330093 conn(0x7fdb6806bdf0 msgr2=0x7fdb681614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:41.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 shutdown_connections 2024-04-24T06:35:41.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.723+0000 7fdb6ee00640 1 -- 172.21.15.59:0/321330093 wait complete. 2024-04-24T06:35:41.736 INFO:tasks.ceph.mds.a.smithi059.stderr: -4842> 2024-04-24T06:34:11.184+0000 7fbc83c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:35:41.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:35:41.807 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:35:41.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.965+0000 7f7fe6e00640 1 Processor -- start 2024-04-24T06:35:41.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.965+0000 7f7fe6e00640 1 -- start start 2024-04-24T06:35:41.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7fe015b220 0x7f7fe00af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe006f8d0 0x7f7fe00adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7fe00d05d0 0x7f7fe00d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe00d0f60 con 0x7f7fe00d05d0 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe006fd30 con 0x7f7fe015b220 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.966+0000 7f7fe6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe00ae3f0 con 0x7f7fe006f8d0 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7fe00d05d0 0x7f7fe00d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7fe00d05d0 0x7f7fe00d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47576/0 (socket says 172.21.15.59:47576) 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 -- 172.21.15.59:0/539342918 learned_addr learned my addr 172.21.15.59:0/539342918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe5a00640 1 --2- 172.21.15.59:0/539342918 >> v2:172.21.15.66:3300/0 conn(0x7f7fe015b220 0x7f7fe00af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe5000640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe006f8d0 0x7f7fe00adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 -- 172.21.15.59:0/539342918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe006f8d0 msgr2=0x7f7fe00adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe006f8d0 0x7f7fe00adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 -- 172.21.15.59:0/539342918 >> v2:172.21.15.66:3300/0 conn(0x7f7fe015b220 msgr2=0x7f7fe00af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:41.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 --2- 172.21.15.59:0/539342918 >> v2:172.21.15.66:3300/0 conn(0x7f7fe015b220 0x7f7fe00af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.967+0000 7f7fe6400640 1 -- 172.21.15.59:0/539342918 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe00ae6b0 con 0x7f7fe00d05d0 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.968+0000 7f7fe6400640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe00d0a20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006e450 tx=0x7f7fd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673ef8731159a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.968+0000 7f7fdfe00640 1 -- 172.21.15.59:0/539342918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8036070 con 0x7f7fe00d05d0 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.968+0000 7f7fdfe00640 1 -- 172.21.15.59:0/539342918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd8031600 con 0x7f7fe00d05d0 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.968+0000 7f7fdfe00640 1 -- 172.21.15.59:0/539342918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd80319d0 con 0x7f7fe00d05d0 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 -- 172.21.15.59:0/539342918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 msgr2=0x7f7fe00d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe00d0a20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7fe006e450 tx=0x7f7fd802a6a0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 -- 172.21.15.59:0/539342918 shutdown_connections 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe00d0a20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/539342918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe006f8d0 0x7f7fe00adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/539342918 >> v2:172.21.15.66:3300/0 conn(0x7f7fe015b220 0x7f7fe00af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 -- 172.21.15.59:0/539342918 >> 172.21.15.59:0/539342918 conn(0x7f7fe006bdf0 msgr2=0x7f7fe006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 -- 172.21.15.59:0/539342918 shutdown_connections 2024-04-24T06:35:41.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.969+0000 7f7fe6e00640 1 -- 172.21.15.59:0/539342918 wait complete. 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.971+0000 7f7fe6e00640 1 Processor -- start 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.971+0000 7f7fe6e00640 1 -- start start 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe006f8d0 0x7f7fe009a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe015b220 0x7f7fe01f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe00aef50 con 0x7f7fe00d05d0 2024-04-24T06:35:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe00aec70 con 0x7f7fe006f8d0 2024-04-24T06:35:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe00af230 con 0x7f7fe015b220 2024-04-24T06:35:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47578/0 (socket says 172.21.15.59:47578) 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe5000640 1 -- 172.21.15.59:0/3330688682 learned_addr learned my addr 172.21.15.59:0/3330688682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe5a00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe006f8d0 0x7f7fe009a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.972+0000 7f7fe6400640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe015b220 0x7f7fe01f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 -- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe015b220 msgr2=0x7f7fe01f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe015b220 0x7f7fe01f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 -- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe006f8d0 msgr2=0x7f7fe009a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe006f8d0 0x7f7fe009a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 -- 172.21.15.59:0/3330688682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe0099c10 con 0x7f7fe00d05d0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe5000640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d800 tx=0x7f7fcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00b6c0 con 0x7f7fe00d05d0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fcc019070 con 0x7f7fe00d05d0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00ba80 con 0x7f7fe00d05d0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe00b4690 con 0x7f7fe00d05d0 2024-04-24T06:35:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.973+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe00998c0 con 0x7f7fe00d05d0 2024-04-24T06:35:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.974+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe0070610 con 0x7f7fe00d05d0 2024-04-24T06:35:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.975+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fcc00bc20 con 0x7f7fe00d05d0 2024-04-24T06:35:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.976+0000 7f7fdd600640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 0x7f7fc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.976+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5939+0+0 (secure 0 0 0) 0x7f7fcc08c140 con 0x7f7fe00d05d0 2024-04-24T06:35:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.976+0000 7f7fe5a00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 0x7f7fc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.977+0000 7f7fe5a00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 0x7f7fc006edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:41.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:41.980+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fe0070610 con 0x7f7fe00d05d0 2024-04-24T06:35:42.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.221+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7fe0144e00 con 0x7f7fe00d05d0 2024-04-24T06:35:42.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.723+0000 7f7fdd600640 1 -- 172.21.15.59:0/3330688682 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f7fe0144e00 con 0x7f7fe00d05d0 2024-04-24T06:35:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.727+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 msgr2=0x7f7fc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.727+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 0x7f7fc006edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4008040 comp rx=0 tx=0).stop 2024-04-24T06:35:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 msgr2=0x7f7fe009aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d800 tx=0x7f7fcc00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:35:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 shutdown_connections 2024-04-24T06:35:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7fc006c8f0 0x7f7fc006edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7fe015b220 0x7f7fe01f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fe00d05d0 0x7f7fe009aa30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 --2- 172.21.15.59:0/3330688682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fe006f8d0 0x7f7fe009a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 >> 172.21.15.59:0/3330688682 conn(0x7f7fe006bdf0 msgr2=0x7f7fe0150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 shutdown_connections 2024-04-24T06:35:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:42.728+0000 7f7fe6e00640 1 -- 172.21.15.59:0/3330688682 wait complete. 2024-04-24T06:35:42.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:35:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.014+0000 7f52bf200640 1 Processor -- start 2024-04-24T06:35:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.014+0000 7f52bf200640 1 -- start start 2024-04-24T06:35:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52c00cd110 0x7f52c0072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c006f050 0x7f52c00ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bf200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c006f7c0 con 0x7f52c0161bc0 2024-04-24T06:35:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f52c006faa0 con 0x7f52c00cd110 2024-04-24T06:35:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f52c006fd80 con 0x7f52c006f050 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52c00cd110 0x7f52c0072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52c00cd110 0x7f52c0072fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47580/0 (socket says 172.21.15.59:47580) 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bd400640 1 -- 172.21.15.59:0/2687470677 learned_addr learned my addr 172.21.15.59:0/2687470677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52be800640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c006f050 0x7f52c00ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bd400640 1 -- 172.21.15.59:0/2687470677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c006f050 msgr2=0x7f52c00ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.016+0000 7f52bd400640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c006f050 0x7f52c00ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.017+0000 7f52bd400640 1 -- 172.21.15.59:0/2687470677 >> v1:172.21.15.66:6789/0 conn(0x7f52c0161bc0 legacy=0x7f52c00cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.017+0000 7f52bd400640 1 -- 172.21.15.59:0/2687470677 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c00ce320 con 0x7f52c00cd110 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.018+0000 7f52bd400640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c0072fd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009870 tx=0x7f52ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cac8151900bbd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.018+0000 7f52bca00640 1 -- 172.21.15.59:0/2687470677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac037070 con 0x7f52c00cd110 2024-04-24T06:35:43.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.018+0000 7f52bca00640 1 -- 172.21.15.59:0/2687470677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac032440 con 0x7f52c00cd110 2024-04-24T06:35:43.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.018+0000 7f52bca00640 1 -- 172.21.15.59:0/2687470677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac032810 con 0x7f52c00cd110 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 -- 172.21.15.59:0/2687470677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 msgr2=0x7f52c0072fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c0072fd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009870 tx=0x7f52ac02a210 comp rx=0 tx=0).stop 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 -- 172.21.15.59:0/2687470677 shutdown_connections 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c006f050 0x7f52c00ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 --2- 172.21.15.59:0/2687470677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c0072fd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 -- 172.21.15.59:0/2687470677 >> 172.21.15.59:0/2687470677 conn(0x7f52c00b1f90 msgr2=0x7f52c00b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 -- 172.21.15.59:0/2687470677 shutdown_connections 2024-04-24T06:35:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.019+0000 7f52bf200640 1 -- 172.21.15.59:0/2687470677 wait complete. 2024-04-24T06:35:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.021+0000 7f52bf200640 1 Processor -- start 2024-04-24T06:35:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.021+0000 7f52bf200640 1 -- start start 2024-04-24T06:35:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 0x7f52c014c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c0161bc0 0x7f52c015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52c00ced80 con 0x7f52c00cd110 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0161fa0 con 0x7f52c006f050 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f52c00cef00 con 0x7f52c0161bc0 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47594/0 (socket says 172.21.15.59:47594) 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.022+0000 7f52bd400640 1 -- 172.21.15.59:0/3162449989 learned_addr learned my addr 172.21.15.59:0/3162449989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52be800640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c0161bc0 0x7f52c015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 -- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c0161bc0 msgr2=0x7f52c015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bde00640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 0x7f52c014c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c0161bc0 0x7f52c015b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 -- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 msgr2=0x7f52c014c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 0x7f52c014c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 -- 172.21.15.59:0/3162449989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c0157840 con 0x7f52c00cd110 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bde00640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 0x7f52c014c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.023+0000 7f52bd400640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.024+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac037040 con 0x7f52c00cd110 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.024+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac042070 con 0x7f52c00cd110 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.024+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c0157b20 con 0x7f52c00cd110 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.024+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac030bb0 con 0x7f52c00cd110 2024-04-24T06:35:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.024+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c00c99e0 con 0x7f52c00cd110 2024-04-24T06:35:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.025+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c004fd90 con 0x7f52c00cd110 2024-04-24T06:35:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.026+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52ac047460 con 0x7f52c00cd110 2024-04-24T06:35:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.027+0000 7f52b6000640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.027+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5939+0+0 (secure 0 0 0) 0x7f52ac0ac8d0 con 0x7f52c00cd110 2024-04-24T06:35:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.028+0000 7f52bde00640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.028+0000 7f52bde00640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 0x7f52a006edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f52b000a830 tx=0x7f52b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.032+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52ac07ac00 con 0x7f52c00cd110 2024-04-24T06:35:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.236+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f52c0161070 con 0x7f52c00cd110 2024-04-24T06:35:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.734+0000 7f52b6000640 1 -- 172.21.15.59:0/3162449989 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7f52ac07a500 con 0x7f52c00cd110 2024-04-24T06:35:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 msgr2=0x7f52a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 0x7f52a006edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f52b000a830 tx=0x7f52b0008040 comp rx=0 tx=0).stop 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 msgr2=0x7f52c01596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac0046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 shutdown_connections 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f52c0161bc0 0x7f52c015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52c00cd110 0x7f52c01596c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 --2- 172.21.15.59:0/3162449989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52c006f050 0x7f52c014c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.739+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 >> 172.21.15.59:0/3162449989 conn(0x7f52c00b1f90 msgr2=0x7f52c00735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.740+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 shutdown_connections 2024-04-24T06:35:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.740+0000 7f52bf200640 1 -- 172.21.15.59:0/3162449989 wait complete. 2024-04-24T06:35:43.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:35:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.969+0000 7f4aabe00640 1 Processor -- start 2024-04-24T06:35:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.970+0000 7f4aabe00640 1 -- start start 2024-04-24T06:35:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.971+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.971+0000 7f4aabe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aac06efa0 0x7f4aac0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.971+0000 7f4aabe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aac0cd9c0 con 0x7f4aac0cf290 2024-04-24T06:35:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.971+0000 7f4aabe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4aac0cfc30 con 0x7f4aac06efa0 2024-04-24T06:35:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.971+0000 7f4aabe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4aac06ddb0 con 0x7f4aac15eb20 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aab400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aac06efa0 0x7f4aac0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aaa000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4aac0cf290 0x7f4aac06e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48908/0 (socket says 172.21.15.59:48908) 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aaa000640 1 -- 172.21.15.59:0/3805096089 learned_addr learned my addr 172.21.15.59:0/3805096089 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aaaa00640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aab400640 1 -- 172.21.15.59:0/3805096089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 msgr2=0x7f4aac0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aab400640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aab400640 1 -- 172.21.15.59:0/3805096089 >> v1:172.21.15.59:6789/0 conn(0x7f4aac0cf290 legacy=0x7f4aac06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aab400640 1 -- 172.21.15.59:0/3805096089 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac06e090 con 0x7f4aac06efa0 2024-04-24T06:35:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.972+0000 7f4aa9600640 1 -- 172.21.15.59:0/3805096089 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039808007 0 0) 0x7f4aac0cd9c0 con 0x7f4aac0cf290 2024-04-24T06:35:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.973+0000 7f4aab400640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac0cf6f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313a81d9ab5c8b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aa9600640 1 -- 172.21.15.59:0/3805096089 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007740 con 0x7f4aac06efa0 2024-04-24T06:35:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aa9600640 1 -- 172.21.15.59:0/3805096089 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa40078e0 con 0x7f4aac06efa0 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aa9600640 1 -- 172.21.15.59:0/3805096089 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007cb0 con 0x7f4aac06efa0 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 -- 172.21.15.59:0/3805096089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 msgr2=0x7f4aac0cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac0cf6f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009d90 tx=0x7f4aa402a660 comp rx=0 tx=0).stop 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 -- 172.21.15.59:0/3805096089 shutdown_connections 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac0cf6f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 --2- 172.21.15.59:0/3805096089 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.974+0000 7f4aabe00640 1 -- 172.21.15.59:0/3805096089 >> 172.21.15.59:0/3805096089 conn(0x7f4aac0b1f90 msgr2=0x7f4aac0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.975+0000 7f4aabe00640 1 -- 172.21.15.59:0/3805096089 shutdown_connections 2024-04-24T06:35:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.975+0000 7f4aabe00640 1 -- 172.21.15.59:0/3805096089 wait complete. 2024-04-24T06:35:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.976+0000 7f4aabe00640 1 Processor -- start 2024-04-24T06:35:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.976+0000 7f4aabe00640 1 -- start start 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aac0cf290 0x7f4aac15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac0ad7b0 con 0x7f4aac06efa0 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac0cdca0 con 0x7f4aac0cf290 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.977+0000 7f4aabe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4aac0ad930 con 0x7f4aac15eb20 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47614/0 (socket says 172.21.15.59:47614) 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 -- 172.21.15.59:0/4247348997 learned_addr learned my addr 172.21.15.59:0/4247348997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aab400640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 -- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 msgr2=0x7f4aac0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 -- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aac0cf290 msgr2=0x7f4aac15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaa000640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aac0cf290 0x7f4aac15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aac0cf290 0x7f4aac15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.978+0000 7f4aaaa00640 1 -- 172.21.15.59:0/4247348997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac14bc10 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aaaa00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00d5f0 tx=0x7f4a9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0125f0 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c019070 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0129d0 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac0c9720 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.979+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac0c9ce0 con 0x7f4aac06efa0 2024-04-24T06:35:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.981+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac04fd90 con 0x7f4aac06efa0 2024-04-24T06:35:43.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.981+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a9c012b70 con 0x7f4aac06efa0 2024-04-24T06:35:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.982+0000 7f4aa2a00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 0x7f4a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.983+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5501+0+0 (secure 0 0 0) 0x7f4a9c08be40 con 0x7f4aac06efa0 2024-04-24T06:35:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.983+0000 7f4aaa000640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 0x7f4a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.983+0000 7f4aaa000640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 0x7f4a8806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007fd0 tx=0x7f4a94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:43.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:43.987+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4aac14bc10 con 0x7f4aac06efa0 2024-04-24T06:35:44.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.193+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4aac0af100 con 0x7f4aac06efa0 2024-04-24T06:35:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.738+0000 7f4aa2a00640 1 -- 172.21.15.59:0/4247348997 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f4a9c05a3d0 con 0x7f4aac06efa0 2024-04-24T06:35:44.741 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:35:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.742+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 msgr2=0x7f4a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.742+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 0x7f4a8806edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007fd0 tx=0x7f4a94009040 comp rx=0 tx=0).stop 2024-04-24T06:35:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 msgr2=0x7f4aac14b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00d5f0 tx=0x7f4a9c00dab0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 shutdown_connections 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8806c8f0 0x7f4a8806edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aac15eb20 0x7f4aac0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aac0cf290 0x7f4aac15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 --2- 172.21.15.59:0/4247348997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac06efa0 0x7f4aac14b6d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.743+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 >> 172.21.15.59:0/4247348997 conn(0x7f4aac0b1f90 msgr2=0x7f4aac0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.745+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 shutdown_connections 2024-04-24T06:35:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.745+0000 7f4aabe00640 1 -- 172.21.15.59:0/4247348997 wait complete. 2024-04-24T06:35:44.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:35:44.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.977+0000 7f2f7dc00640 1 Processor -- start 2024-04-24T06:35:44.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.977+0000 7f2f7dc00640 1 -- start start 2024-04-24T06:35:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.978+0000 7f2f7dc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 0x7f2f780af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.978+0000 7f2f7dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f780d05d0 0x7f2f780d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.978+0000 7f2f7dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7806fd60 con 0x7f2f7806f8d0 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.978+0000 7f2f7dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2f780d0ef0 con 0x7f2f780d05d0 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.978+0000 7f2f7dc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2f780ae6b0 con 0x7f2f7815b220 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f6fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f7806f8d0 0x7f2f780adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48924/0 (socket says 172.21.15.59:48924) 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f6fe00640 1 -- 172.21.15.59:0/867510341 learned_addr learned my addr 172.21.15.59:0/867510341 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:44.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f7d200640 1 --2- 172.21.15.59:0/867510341 >> v2:172.21.15.59:3300/0 conn(0x7f2f780d05d0 0x7f2f780d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f77e00640 1 --2- 172.21.15.59:0/867510341 >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 0x7f2f780af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f77400640 1 -- 172.21.15.59:0/867510341 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414726721 0 0) 0x7f2f7806fd60 con 0x7f2f7806f8d0 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f77400640 1 -- 172.21.15.59:0/867510341 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f5c003520 con 0x7f2f7806f8d0 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f7d200640 1 -- 172.21.15.59:0/867510341 >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 msgr2=0x7f2f780af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.979+0000 7f2f7d200640 1 --2- 172.21.15.59:0/867510341 >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 0x7f2f780af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.980+0000 7f2f7d200640 1 -- 172.21.15.59:0/867510341 >> v1:172.21.15.59:6789/0 conn(0x7f2f7806f8d0 legacy=0x7f2f780adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.980+0000 7f2f7d200640 1 -- 172.21.15.59:0/867510341 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f780ae990 con 0x7f2f780d05d0 2024-04-24T06:35:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.980+0000 7f2f77e00640 1 --2- 172.21.15.59:0/867510341 >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 0x7f2f780af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f7d200640 1 --2- 172.21.15.59:0/867510341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f780d05d0 0x7f2f780d09b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2f7806e450 tx=0x7f2f7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bf88fa9a767d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f77400640 1 -- 172.21.15.59:0/867510341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007740 con 0x7f2f780d05d0 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f77400640 1 -- 172.21.15.59:0/867510341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f700078e0 con 0x7f2f780d05d0 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f77400640 1 -- 172.21.15.59:0/867510341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007cb0 con 0x7f2f780d05d0 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f7dc00640 1 -- 172.21.15.59:0/867510341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f780d05d0 msgr2=0x7f2f780d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.981+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/867510341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f780d05d0 0x7f2f780d09b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2f7806e450 tx=0x7f2f7002a680 comp rx=0 tx=0).stop 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 -- 172.21.15.59:0/867510341 shutdown_connections 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/867510341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f780d05d0 0x7f2f780d09b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/867510341 >> v2:172.21.15.66:3300/0 conn(0x7f2f7815b220 0x7f2f780af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 -- 172.21.15.59:0/867510341 >> 172.21.15.59:0/867510341 conn(0x7f2f7806bdf0 msgr2=0x7f2f7806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 -- 172.21.15.59:0/867510341 shutdown_connections 2024-04-24T06:35:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.982+0000 7f2f7dc00640 1 -- 172.21.15.59:0/867510341 wait complete. 2024-04-24T06:35:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.984+0000 7f2f7dc00640 1 Processor -- start 2024-04-24T06:35:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.984+0000 7f2f7dc00640 1 -- start start 2024-04-24T06:35:44.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f7806f8d0 0x7f2f780c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f780d05d0 0x7f2f7815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780af230 con 0x7f2f7815b220 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780aef50 con 0x7f2f7806f8d0 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7dc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7815b600 con 0x7f2f780d05d0 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47634/0 (socket says 172.21.15.59:47634) 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f7d200640 1 -- 172.21.15.59:0/3070023058 learned_addr learned my addr 172.21.15.59:0/3070023058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f6fe00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f780d05d0 0x7f2f7815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.985+0000 7f2f77e00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f7806f8d0 0x7f2f780c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 -- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f780d05d0 msgr2=0x7f2f7815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f780d05d0 0x7f2f7815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 -- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f7806f8d0 msgr2=0x7f2f780c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f7806f8d0 0x7f2f780c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 -- 172.21.15.59:0/3070023058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78150e10 con 0x7f2f7815b220 2024-04-24T06:35:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7d200640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009e50 tx=0x7f2f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70040070 con 0x7f2f7815b220 2024-04-24T06:35:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70031bf0 con 0x7f2f7815b220 2024-04-24T06:35:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f781510f0 con 0x7f2f7815b220 2024-04-24T06:35:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.986+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70009450 con 0x7f2f7815b220 2024-04-24T06:35:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.987+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f780c9ba0 con 0x7f2f7815b220 2024-04-24T06:35:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.988+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7804fd90 con 0x7f2f7815b220 2024-04-24T06:35:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.989+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f70039040 con 0x7f2f7815b220 2024-04-24T06:35:44.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.990+0000 7f2f74c00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 0x7f2f5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:44.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.990+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f2f700ad1d0 con 0x7f2f7815b220 2024-04-24T06:35:44.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.990+0000 7f2f77e00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 0x7f2f5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:44.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.991+0000 7f2f77e00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 0x7f2f5406edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2f68006fd0 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:44.995+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f70043360 con 0x7f2f7815b220 2024-04-24T06:35:45.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.200+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2f780d0ae0 con 0x7f2f7815b220 2024-04-24T06:35:45.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.208+0000 7f2f74c00640 1 -- 172.21.15.59:0/3070023058 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f2f7007b800 con 0x7f2f7815b220 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 msgr2=0x7f2f5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 0x7f2f5406edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2f68006fd0 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 msgr2=0x7f2f781f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009e50 tx=0x7f2f70004a40 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 shutdown_connections 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2f5406c8f0 0x7f2f5406edb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7815b220 0x7f2f781f40d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2f780d05d0 0x7f2f7815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 --2- 172.21.15.59:0/3070023058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f7806f8d0 0x7f2f780c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.210+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 >> 172.21.15.59:0/3070023058 conn(0x7f2f7806bdf0 msgr2=0x7f2f780af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.211+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 shutdown_connections 2024-04-24T06:35:45.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.211+0000 7f2f7dc00640 1 -- 172.21.15.59:0/3070023058 wait complete. 2024-04-24T06:35:45.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-24T06:35:45.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.438+0000 7f1ad7200640 1 Processor -- start 2024-04-24T06:35:45.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.438+0000 7f1ad7200640 1 -- start start 2024-04-24T06:35:45.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1ad80b5ff0 0x7f1ad80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ad80d0d90 0x7f1ad80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad80cf030 con 0x7f1ad815e7c0 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad80b7000 con 0x7f1ad80d0d90 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad80d1720 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ad80d0d90 0x7f1ad80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ad80d0d90 0x7f1ad80d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47636/0 (socket says 172.21.15.59:47636) 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad6800640 1 -- 172.21.15.59:0/3554611300 learned_addr learned my addr 172.21.15.59:0/3554611300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.440+0000 7f1ad5e00640 1 --1- 172.21.15.59:0/3554611300 >> v1:172.21.15.59:6789/0 conn(0x7f1ad815e7c0 0x7f1ad80d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48936/0 (socket says 172.21.15.59:48936) 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.441+0000 7f1acd400640 1 --2- 172.21.15.59:0/3554611300 >> v2:172.21.15.66:3300/0 conn(0x7f1ad80b5ff0 0x7f1ad80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.441+0000 7f1ad6800640 1 -- 172.21.15.59:0/3554611300 >> v2:172.21.15.66:3300/0 conn(0x7f1ad80b5ff0 msgr2=0x7f1ad80b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.441+0000 7f1ad6800640 1 --2- 172.21.15.59:0/3554611300 >> v2:172.21.15.66:3300/0 conn(0x7f1ad80b5ff0 0x7f1ad80b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.441+0000 7f1ad6800640 1 -- 172.21.15.59:0/3554611300 >> v1:172.21.15.59:6789/0 conn(0x7f1ad815e7c0 legacy=0x7f1ad80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.441+0000 7f1ad6800640 1 -- 172.21.15.59:0/3554611300 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad80b6a40 con 0x7f1ad80d0d90 2024-04-24T06:35:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.442+0000 7f1ad6800640 1 --2- 172.21.15.59:0/3554611300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80d0d90 0x7f1ad80d11e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80d55c0 tx=0x7f1ad002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18032f2892114ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.442+0000 7f1ad5400640 1 -- 172.21.15.59:0/3554611300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0036070 con 0x7f1ad80d0d90 2024-04-24T06:35:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.442+0000 7f1ad5400640 1 -- 172.21.15.59:0/3554611300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad0031600 con 0x7f1ad80d0d90 2024-04-24T06:35:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.442+0000 7f1ad5400640 1 -- 172.21.15.59:0/3554611300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00319d0 con 0x7f1ad80d0d90 2024-04-24T06:35:45.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 -- 172.21.15.59:0/3554611300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80d0d90 msgr2=0x7f1ad80d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3554611300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80d0d90 0x7f1ad80d11e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80d55c0 tx=0x7f1ad002a730 comp rx=0 tx=0).stop 2024-04-24T06:35:45.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 -- 172.21.15.59:0/3554611300 shutdown_connections 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3554611300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80d0d90 0x7f1ad80d11e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3554611300 >> v2:172.21.15.66:3300/0 conn(0x7f1ad80b5ff0 0x7f1ad80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 -- 172.21.15.59:0/3554611300 >> 172.21.15.59:0/3554611300 conn(0x7f1ad809d920 msgr2=0x7f1ad808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 -- 172.21.15.59:0/3554611300 shutdown_connections 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.443+0000 7f1ad7200640 1 -- 172.21.15.59:0/3554611300 wait complete. 2024-04-24T06:35:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.445+0000 7f1ad7200640 1 Processor -- start 2024-04-24T06:35:45.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.445+0000 7f1ad7200640 1 -- start start 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ad80d0d90 0x7f1ad80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad815e7c0 0x7f1ad80cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad80cf2b0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad80cf620 con 0x7f1ad815e7c0 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad80d2800 con 0x7f1ad80d0d90 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47642/0 (socket says 172.21.15.59:47642) 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.446+0000 7f1ad5e00640 1 -- 172.21.15.59:0/3419190450 learned_addr learned my addr 172.21.15.59:0/3419190450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad6800640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad815e7c0 0x7f1ad80cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 -- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ad80d0d90 msgr2=0x7f1ad80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ad80d0d90 0x7f1ad80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 -- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad815e7c0 msgr2=0x7f1ad80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad815e7c0 0x7f1ad80cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 -- 172.21.15.59:0/3419190450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad80b99e0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad5e00640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800c3c0 tx=0x7f1ac800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:45.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80074e0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac8007680 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad80b9cc0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.447+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004a00 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.448+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad80cb5f0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.449+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad804fd90 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.450+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ac801a020 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.451+0000 7f1acea00640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 0x7f1ab406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.451+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f1ac808b120 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.451+0000 7f1acd400640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 0x7f1ab406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.452+0000 7f1acd400640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 0x7f1ab406ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ad815ddf0 tx=0x7f1ac0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:45.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.455+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ad80cf030 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.734+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f1ad806dcd0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.755+0000 7f1acea00640 1 -- 172.21.15.59:0/3419190450 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v350) ==== 166+0+0 (secure 0 0 0) 0x7f1ac80596c0 con 0x7f1ad80b5ff0 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 msgr2=0x7f1ab406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 0x7f1ab406ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ad815ddf0 tx=0x7f1ac0008040 comp rx=0 tx=0).stop 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 msgr2=0x7f1ad80b9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800c3c0 tx=0x7f1ac800c880 comp rx=0 tx=0).stop 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 shutdown_connections 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1ab406c920 0x7f1ab406ede0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ad815e7c0 0x7f1ad80cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1ad80d0d90 0x7f1ad80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 --2- 172.21.15.59:0/3419190450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad80b5ff0 0x7f1ad80b9400 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 >> 172.21.15.59:0/3419190450 conn(0x7f1ad809d920 msgr2=0x7f1ad80b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 shutdown_connections 2024-04-24T06:35:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.758+0000 7f1ad7200640 1 -- 172.21.15.59:0/3419190450 wait complete. 2024-04-24T06:35:45.777 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-24T06:35:45.777 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-24T06:35:45.777 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:35:45.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.988+0000 7fbc50e00640 1 Processor -- start 2024-04-24T06:35:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.989+0000 7fbc50e00640 1 -- start start 2024-04-24T06:35:45.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc50e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c15e750 0x7fbc4c0ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc50e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc50e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc4c0b8fc0 con 0x7fbc4c15eb30 2024-04-24T06:35:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc50e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbc4c0b92a0 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc50e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4c0b9580 con 0x7fbc4c15e750 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc4a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc4a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47646/0 (socket says 172.21.15.59:47646) 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc49400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc4c15eb30 0x7fbc4c0cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48946/0 (socket says 172.21.15.59:48946) 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.990+0000 7fbc4a800640 1 -- 172.21.15.59:0/2161680294 learned_addr learned my addr 172.21.15.59:0/2161680294 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc49e00640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c15e750 0x7fbc4c0ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc4a800640 1 -- 172.21.15.59:0/2161680294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c15e750 msgr2=0x7fbc4c0ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc4a800640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c15e750 0x7fbc4c0ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc4a800640 1 -- 172.21.15.59:0/2161680294 >> v1:172.21.15.59:6789/0 conn(0x7fbc4c15eb30 legacy=0x7fbc4c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc4a800640 1 -- 172.21.15.59:0/2161680294 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c0b9860 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.991+0000 7fbc48a00640 1 -- 172.21.15.59:0/2161680294 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2032202941 0 0) 0x7fbc4c0b8fc0 con 0x7fbc4c15eb30 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.992+0000 7fbc4a800640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e50 tx=0x7fbc4402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103d6b2b1f984dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.992+0000 7fbc48a00640 1 -- 172.21.15.59:0/2161680294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44036070 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.992+0000 7fbc48a00640 1 -- 172.21.15.59:0/2161680294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc44031600 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.992+0000 7fbc48a00640 1 -- 172.21.15.59:0/2161680294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc440319d0 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 -- 172.21.15.59:0/2161680294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0cfbf0 msgr2=0x7fbc4c0b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e50 tx=0x7fbc4402a710 comp rx=0 tx=0).stop 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 -- 172.21.15.59:0/2161680294 shutdown_connections 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0cfbf0 0x7fbc4c0b7940 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 --2- 172.21.15.59:0/2161680294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c15e750 0x7fbc4c0ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 -- 172.21.15.59:0/2161680294 >> 172.21.15.59:0/2161680294 conn(0x7fbc4c067b50 msgr2=0x7fbc4c067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 -- 172.21.15.59:0/2161680294 shutdown_connections 2024-04-24T06:35:45.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.993+0000 7fbc50e00640 1 -- 172.21.15.59:0/2161680294 wait complete. 2024-04-24T06:35:45.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.995+0000 7fbc50e00640 1 Processor -- start 2024-04-24T06:35:45.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.995+0000 7fbc50e00640 1 -- start start 2024-04-24T06:35:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c0cfbf0 0x7fbc4c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc4c15eb30 0x7fbc4c15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c0d3c80 con 0x7fbc4c15e750 2024-04-24T06:35:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c0ba6e0 con 0x7fbc4c15eb30 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc50e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4c0d3e00 con 0x7fbc4c0cfbf0 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47650/0 (socket says 172.21.15.59:47650) 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 -- 172.21.15.59:0/3166402213 learned_addr learned my addr 172.21.15.59:0/3166402213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c0cfbf0 0x7fbc4c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 -- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c0cfbf0 msgr2=0x7fbc4c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c0cfbf0 0x7fbc4c155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 -- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc4c15eb30 msgr2=0x7fbc4c15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc4a800640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc4c15eb30 0x7fbc4c15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc4c15eb30 0x7fbc4c15fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.996+0000 7fbc49400640 1 -- 172.21.15.59:0/3166402213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c161110 con 0x7fbc4c15e750 2024-04-24T06:35:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc49400640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c3f0 tx=0x7fbc3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc380074e0 con 0x7fbc4c15e750 2024-04-24T06:35:46.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc380047e0 con 0x7fbc4c15e750 2024-04-24T06:35:46.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38004c10 con 0x7fbc4c15e750 2024-04-24T06:35:46.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c160500 con 0x7fbc4c15e750 2024-04-24T06:35:46.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.997+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c160ac0 con 0x7fbc4c15e750 2024-04-24T06:35:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:45.999+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4c0cec20 con 0x7fbc4c15e750 2024-04-24T06:35:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.000+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc38011070 con 0x7fbc4c15e750 2024-04-24T06:35:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.001+0000 7fbc42000640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 0x7fbc2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.001+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7fbc3808b4c0 con 0x7fbc4c15e750 2024-04-24T06:35:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.002+0000 7fbc49e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 0x7fbc2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.003+0000 7fbc49e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 0x7fbc2c06ede0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006fd0 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.008+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc4c0cec20 con 0x7fbc4c15e750 2024-04-24T06:35:46.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.204+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc4c068a80 con 0x7fbc4c15e750 2024-04-24T06:35:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.206+0000 7fbc42000640 1 -- 172.21.15.59:0/3166402213 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13065 (secure 0 0 0) 0x7fbc3801a2b0 con 0x7fbc4c15e750 2024-04-24T06:35:46.208 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:46.208 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:44.735932+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/209407257":"2024-04-25T06:35:41.716888+0000","172.21.15.59:6836/209407257":"2024-04-25T06:35:41.716888+0000","172.21.15.66:6835/2419732942":"2024-04-25T06:34:06.952885+0000","172.21.15.66:6833/2419732942":"2024-04-25T06:34:06.952885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 msgr2=0x7fbc2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 0x7fbc2c06ede0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c006fd0 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 msgr2=0x7fbc4c15f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c3f0 tx=0x7fbc3800c8b0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 shutdown_connections 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2c06c920 0x7fbc2c06ede0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.208+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc4c15eb30 0x7fbc4c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.209+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c15e750 0x7fbc4c15f940 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.209+0000 7fbc50e00640 1 --2- 172.21.15.59:0/3166402213 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c0cfbf0 0x7fbc4c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.209+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 >> 172.21.15.59:0/3166402213 conn(0x7fbc4c067b50 msgr2=0x7fbc4c15d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.209+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 shutdown_connections 2024-04-24T06:35:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.209+0000 7fbc50e00640 1 -- 172.21.15.59:0/3166402213 wait complete. 2024-04-24T06:35:46.230 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:35:46.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.438+0000 7efd78e00640 1 Processor -- start 2024-04-24T06:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.438+0000 7efd78e00640 1 -- start start 2024-04-24T06:35:46.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd78e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efd740d0a90 0x7efd740ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd78e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7415ef20 con 0x7efd740d0600 2024-04-24T06:35:46.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd78e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd740d0f00 con 0x7efd74155a80 2024-04-24T06:35:46.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd78e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efd7415f1c0 con 0x7efd740d0a90 2024-04-24T06:35:46.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd72a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efd740d0600 0x7efd740ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48950/0 (socket says 172.21.15.59:48950) 2024-04-24T06:35:46.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd72a00640 1 -- 172.21.15.59:0/29040598 learned_addr learned my addr 172.21.15.59:0/29040598 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:46.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.440+0000 7efd73e00640 1 --2- 172.21.15.59:0/29040598 >> v2:172.21.15.66:3300/0 conn(0x7efd740d0a90 0x7efd740ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301684051 0 0) 0x7efd7415ef20 con 0x7efd740d0600 2024-04-24T06:35:46.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd58003520 con 0x7efd740d0600 2024-04-24T06:35:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2656391611 0 0) 0x7efd58003520 con 0x7efd740d0600 2024-04-24T06:35:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 >> v2:172.21.15.66:3300/0 conn(0x7efd740d0a90 msgr2=0x7efd740ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 --2- 172.21.15.59:0/29040598 >> v2:172.21.15.66:3300/0 conn(0x7efd740d0a90 0x7efd740ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 >> v1:172.21.15.66:6789/0 conn(0x7efd74155a80 legacy=0x7efd7406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7415f4a0 con 0x7efd740d0600 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7efd5c001670 con 0x7efd740d0600 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 >> v1:172.21.15.59:6789/0 conn(0x7efd740d0600 legacy=0x7efd740ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58003a00 0x7efd58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 0x7efd5800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd7415f1c0 con 0x7efd580051c0 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd5800e2d0 con 0x7efd58003a00 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.441+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efd5800e4d0 con 0x7efd58004800 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 0x7efd5800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73400640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58003a00 0x7efd58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd72a00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 -- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd58004800 msgr2=0x7efd58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 -- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58003a00 msgr2=0x7efd58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58003a00 0x7efd58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73e00640 1 -- 172.21.15.59:0/29040598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd58003520 con 0x7efd580051c0 2024-04-24T06:35:46.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.442+0000 7efd73400640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58003a00 0x7efd58003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.443+0000 7efd73e00640 1 -- 172.21.15.59:0/29040598 reap_dead start 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.443+0000 7efd73e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 0x7efd5800dbd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009ac0 tx=0x7efd58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c032217f35a50158 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c009690 con 0x7efd580051c0 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c004020 con 0x7efd580051c0 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd72000640 1 -- 172.21.15.59:0/29040598 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c004410 con 0x7efd580051c0 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 -- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 msgr2=0x7efd5800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 0x7efd5800dbd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009ac0 tx=0x7efd58005830 comp rx=0 tx=0).stop 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 -- 172.21.15.59:0/29040598 shutdown_connections 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 --2- 172.21.15.59:0/29040598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd580051c0 0x7efd5800dbd0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 -- 172.21.15.59:0/29040598 >> 172.21.15.59:0/29040598 conn(0x7efd7406bdf0 msgr2=0x7efd7406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.444+0000 7efd78e00640 1 -- 172.21.15.59:0/29040598 shutdown_connections 2024-04-24T06:35:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.445+0000 7efd78e00640 1 -- 172.21.15.59:0/29040598 wait complete. 2024-04-24T06:35:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.446+0000 7efd78e00640 1 Processor -- start 2024-04-24T06:35:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.446+0000 7efd78e00640 1 -- start start 2024-04-24T06:35:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd580051c0 0x7efd740cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd740b08f0 0x7efd7415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd7415fd40 con 0x7efd74142dc0 2024-04-24T06:35:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd7415fa60 con 0x7efd740b08f0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.447+0000 7efd78e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efd7406ef80 con 0x7efd580051c0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47668/0 (socket says 172.21.15.59:47668) 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 -- 172.21.15.59:0/4074454654 learned_addr learned my addr 172.21.15.59:0/4074454654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73400640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd580051c0 0x7efd740cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd72a00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd740b08f0 0x7efd7415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 -- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd580051c0 msgr2=0x7efd740cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd580051c0 0x7efd740cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 -- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd740b08f0 msgr2=0x7efd7415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd740b08f0 0x7efd7415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.448+0000 7efd73e00640 1 -- 172.21.15.59:0/4074454654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd740cf160 con 0x7efd74142dc0 2024-04-24T06:35:46.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd73e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7efd6c02abe0 tx=0x7efd6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c036040 con 0x7efd74142dc0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c040070 con 0x7efd74142dc0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c009690 con 0x7efd74142dc0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd740b0e20 con 0x7efd74142dc0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.449+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd740b13e0 con 0x7efd74142dc0 2024-04-24T06:35:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.450+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7404fd90 con 0x7efd74142dc0 2024-04-24T06:35:46.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.451+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd6c044410 con 0x7efd74142dc0 2024-04-24T06:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.452+0000 7efd6b400640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 0x7efd5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.452+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7efd6c0ab360 con 0x7efd74142dc0 2024-04-24T06:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.453+0000 7efd73400640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 0x7efd5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.453+0000 7efd73400640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 0x7efd5006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efd64001e40 tx=0x7efd64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.457+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd6c049020 con 0x7efd74142dc0 2024-04-24T06:35:46.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.663+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd7406eba0 con 0x7efd74142dc0 2024-04-24T06:35:46.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.664+0000 7efd6b400640 1 -- 172.21.15.59:0/4074454654 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7efd6c079990 con 0x7efd74142dc0 2024-04-24T06:35:46.666 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.667+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 msgr2=0x7efd5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.667+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 0x7efd5006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efd64001e40 tx=0x7efd64009040 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.667+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 msgr2=0x7efd740b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.667+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7efd6c02abe0 tx=0x7efd6c004730 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 shutdown_connections 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efd5006c8f0 0x7efd5006edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd74142dc0 0x7efd740b1ec0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd740b08f0 0x7efd7415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 --2- 172.21.15.59:0/4074454654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efd580051c0 0x7efd740cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 >> 172.21.15.59:0/4074454654 conn(0x7efd7406bdf0 msgr2=0x7efd7406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 shutdown_connections 2024-04-24T06:35:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.668+0000 7efd78e00640 1 -- 172.21.15.59:0/4074454654 wait complete. 2024-04-24T06:35:46.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-24T06:35:46.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.902+0000 7f2813200640 1 Processor -- start 2024-04-24T06:35:46.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.902+0000 7f2813200640 1 -- start start 2024-04-24T06:35:46.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.903+0000 7f2813200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.903+0000 7f2813200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0b3870 con 0x7f280c0b3080 2024-04-24T06:35:46.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.903+0000 7f2813200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0b3b50 con 0x7f280c15b450 2024-04-24T06:35:46.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.903+0000 7f2813200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f280c0b41a0 con 0x7f280c06f3f0 2024-04-24T06:35:46.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.904+0000 7f2812800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f280c0b3080 0x7f280c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48958/0 (socket says 172.21.15.59:48958) 2024-04-24T06:35:46.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.904+0000 7f2812800640 1 -- 172.21.15.59:0/3092190642 learned_addr learned my addr 172.21.15.59:0/3092190642 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:46.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.904+0000 7f2811400640 1 --2- 172.21.15.59:0/3092190642 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.904+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824878161 0 0) 0x7f280c0b3870 con 0x7f280c0b3080 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.904+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f4005730 con 0x7f280c0b3080 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3296970057 0 0) 0x7f27f4005730 con 0x7f280c0b3080 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 msgr2=0x7f280c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 --2- 172.21.15.59:0/3092190642 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 >> v1:172.21.15.66:6789/0 conn(0x7f280c15b450 legacy=0x7f280c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0b3e30 con 0x7f280c0b3080 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f28080019f0 con 0x7f280c0b3080 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 >> v1:172.21.15.59:6789/0 conn(0x7f280c0b3080 legacy=0x7f280c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f27f4006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f4006a10 0x7f27f4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f40073d0 0x7f27f400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b41a0 con 0x7f27f4005c10 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27f400e2e0 con 0x7f27f4006a10 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27f400e4e0 con 0x7f27f40073d0 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.905+0000 7f2811e00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f27f4006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811400640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f4006a10 0x7f27f4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811e00640 1 -- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f40073d0 msgr2=0x7f27f400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811e00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f40073d0 0x7f27f400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811e00640 1 -- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f4006a10 msgr2=0x7f27f4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811e00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f4006a10 0x7f27f4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811e00640 1 -- 172.21.15.59:0/3092190642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f4005730 con 0x7f27f4005c10 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2811400640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f4006a10 0x7f27f4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:46.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.906+0000 7f2812800640 1 -- 172.21.15.59:0/3092190642 reap_dead start 2024-04-24T06:35:46.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.907+0000 7f2811e00640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f27f4006050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2800001750 tx=0x7f27f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5d9133525aab65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.907+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800001370 con 0x7f27f4005c10 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.907+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28000052d0 con 0x7f27f4005c10 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.907+0000 7f2810a00640 1 -- 172.21.15.59:0/3092190642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280002abe0 con 0x7f27f4005c10 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 -- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 msgr2=0x7f27f4006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f27f4006050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2800001750 tx=0x7f27f4007a40 comp rx=0 tx=0).stop 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 -- 172.21.15.59:0/3092190642 shutdown_connections 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 --2- 172.21.15.59:0/3092190642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f27f4006050 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 -- 172.21.15.59:0/3092190642 >> 172.21.15.59:0/3092190642 conn(0x7f280c0b21a0 msgr2=0x7f280c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 -- 172.21.15.59:0/3092190642 shutdown_connections 2024-04-24T06:35:46.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.908+0000 7f2813200640 1 -- 172.21.15.59:0/3092190642 wait complete. 2024-04-24T06:35:46.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.910+0000 7f2813200640 1 Processor -- start 2024-04-24T06:35:46.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.910+0000 7f2813200640 1 -- start start 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1e3a60 0x7f280c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c1e4520 0x7f280c1e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b4630 con 0x7f27f4005c10 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b4ff0 con 0x7f280c1e3a60 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2813200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0b47b0 con 0x7f280c1e4520 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2811e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2811e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47690/0 (socket says 172.21.15.59:47690) 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.911+0000 7f2811e00640 1 -- 172.21.15.59:0/2201639111 learned_addr learned my addr 172.21.15.59:0/2201639111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2812800640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c1e4520 0x7f280c1e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 -- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c1e4520 msgr2=0x7f280c1e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811400640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1e3a60 0x7f280c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c1e4520 0x7f280c1e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 -- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1e3a60 msgr2=0x7f280c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1e3a60 0x7f280c15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 -- 172.21.15.59:0/2201639111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c15d0d0 con 0x7f27f4005c10 2024-04-24T06:35:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2811e00640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280003f030 con 0x7f27f4005c10 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2800005480 con 0x7f27f4005c10 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.912+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800037040 con 0x7f27f4005c10 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.913+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c15d7b0 con 0x7f27f4005c10 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.913+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c094f90 con 0x7f27f4005c10 2024-04-24T06:35:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.914+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c04fd90 con 0x7f27f4005c10 2024-04-24T06:35:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.915+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2800042070 con 0x7f27f4005c10 2024-04-24T06:35:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.916+0000 7f2806000640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.917+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f28000ab2d0 con 0x7f27f4005c10 2024-04-24T06:35:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.917+0000 7f2811400640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.917+0000 7f2811400640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ce040 tx=0x7f27f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:46.921+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2800079990 con 0x7f27f4005c10 2024-04-24T06:35:47.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.204+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f280c0ad300 con 0x7f27f4005c10 2024-04-24T06:35:47.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.768+0000 7f2806000640 1 -- 172.21.15.59:0/2201639111 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v352) ==== 170+0+0 (secure 0 0 0) 0x7f28000a9e20 con 0x7f27f4005c10 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 msgr2=0x7f27ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ce040 tx=0x7f27f8008040 comp rx=0 tx=0).stop 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 msgr2=0x7f280c158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2800000c00 tx=0x7f2800001120 comp rx=0 tx=0).stop 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 shutdown_connections 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c1e4520 0x7f280c1e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1e3a60 0x7f280c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 --2- 172.21.15.59:0/2201639111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f4005c10 0x7f280c158290 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 >> 172.21.15.59:0/2201639111 conn(0x7f280c0b21a0 msgr2=0x7f280c06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 shutdown_connections 2024-04-24T06:35:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:47.772+0000 7f2813200640 1 -- 172.21.15.59:0/2201639111 wait complete. 2024-04-24T06:35:47.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:35:48.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.009+0000 7f53ac600640 1 Processor -- start 2024-04-24T06:35:48.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.009+0000 7f53ac600640 1 -- start start 2024-04-24T06:35:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.010+0000 7f53ac600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a40b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.010+0000 7f53ac600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53a40d12b0 0x7f53a415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.010+0000 7f53ac600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a415ef80 con 0x7f53a415eaf0 2024-04-24T06:35:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.010+0000 7f53ac600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f53a40b2dc0 con 0x7f53a40d12b0 2024-04-24T06:35:48.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.010+0000 7f53ac600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f53a4163320 con 0x7f53a40d0ed0 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53a40d12b0 0x7f53a415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f53a40d12b0 0x7f53a415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47698/0 (socket says 172.21.15.59:47698) 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 -- 172.21.15.59:0/396286137 learned_addr learned my addr 172.21.15.59:0/396286137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9c00640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a40b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 -- 172.21.15.59:0/396286137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 msgr2=0x7f53a40b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a40b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 -- 172.21.15.59:0/396286137 >> v1:172.21.15.66:6789/0 conn(0x7f53a415eaf0 legacy=0x7f53a4162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.011+0000 7f53a9200640 1 -- 172.21.15.59:0/396286137 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40b2770 con 0x7f53a40d12b0 2024-04-24T06:35:48.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.012+0000 7f53a9200640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a40d12b0 0x7f53a415e5b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f53a40b2f40 tx=0x7f539002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4facc40870425c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53a3e00640 1 -- 172.21.15.59:0/396286137 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390037070 con 0x7f53a40d12b0 2024-04-24T06:35:48.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53a3e00640 1 -- 172.21.15.59:0/396286137 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5390032440 con 0x7f53a40d12b0 2024-04-24T06:35:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53a3e00640 1 -- 172.21.15.59:0/396286137 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390032810 con 0x7f53a40d12b0 2024-04-24T06:35:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 -- 172.21.15.59:0/396286137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a40d12b0 msgr2=0x7f53a415e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a40d12b0 0x7f53a415e5b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f53a40b2f40 tx=0x7f539002a290 comp rx=0 tx=0).stop 2024-04-24T06:35:48.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 -- 172.21.15.59:0/396286137 shutdown_connections 2024-04-24T06:35:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a40d12b0 0x7f53a415e5b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 --2- 172.21.15.59:0/396286137 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a40b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 -- 172.21.15.59:0/396286137 >> 172.21.15.59:0/396286137 conn(0x7f53a40d2680 msgr2=0x7f53a40d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 -- 172.21.15.59:0/396286137 shutdown_connections 2024-04-24T06:35:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.013+0000 7f53ac600640 1 -- 172.21.15.59:0/396286137 wait complete. 2024-04-24T06:35:48.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.015+0000 7f53ac600640 1 Processor -- start 2024-04-24T06:35:48.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.015+0000 7f53ac600640 1 -- start start 2024-04-24T06:35:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a4158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 0x7f53a40af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4162ef0 con 0x7f53a415eaf0 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4162d70 con 0x7f53a40d12b0 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53ac600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f53a4163070 con 0x7f53a40d0ed0 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53aa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53aa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47714/0 (socket says 172.21.15.59:47714) 2024-04-24T06:35:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.016+0000 7f53aa600640 1 -- 172.21.15.59:0/2128292006 learned_addr learned my addr 172.21.15.59:0/2128292006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a9200640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 0x7f53a40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a9c00640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a4158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 -- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 msgr2=0x7f53a4158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a4158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 -- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 msgr2=0x7f53a40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 0x7f53a40af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 -- 172.21.15.59:0/2128292006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40b1040 con 0x7f53a415eaf0 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a9200640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 0x7f53a40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53aa600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f539c00b970 tx=0x7f539c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c019070 con 0x7f53a415eaf0 2024-04-24T06:35:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539c00cb40 con 0x7f53a415eaf0 2024-04-24T06:35:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c014440 con 0x7f53a415eaf0 2024-04-24T06:35:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.017+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a40b1320 con 0x7f53a415eaf0 2024-04-24T06:35:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.018+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a4158fc0 con 0x7f53a415eaf0 2024-04-24T06:35:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.019+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a4069190 con 0x7f53a415eaf0 2024-04-24T06:35:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.019+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f539c01c070 con 0x7f53a415eaf0 2024-04-24T06:35:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.020+0000 7f53a1600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 0x7f538406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.020+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f539c08b4c0 con 0x7f53a415eaf0 2024-04-24T06:35:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.021+0000 7f53a9c00640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 0x7f538406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.021+0000 7f53a9c00640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 0x7f538406ede0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.025+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f539c059b80 con 0x7f53a415eaf0 2024-04-24T06:35:48.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.227+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f53a40cead0 con 0x7f53a415eaf0 2024-04-24T06:35:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.236+0000 7f53a1600640 1 -- 172.21.15.59:0/2128292006 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f539c059480 con 0x7f53a415eaf0 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.238+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 msgr2=0x7f538406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.238+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 0x7f538406ede0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394008040 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.238+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 msgr2=0x7f53a40af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.238+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f539c00b970 tx=0x7f539c00be30 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 shutdown_connections 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f538406c920 0x7f538406ede0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53a415eaf0 0x7f53a40af8a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a40d12b0 0x7f53a40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 --2- 172.21.15.59:0/2128292006 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f53a40d0ed0 0x7f53a4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 >> 172.21.15.59:0/2128292006 conn(0x7f53a40d2680 msgr2=0x7f53a40701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 shutdown_connections 2024-04-24T06:35:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.239+0000 7f53ac600640 1 -- 172.21.15.59:0/2128292006 wait complete. 2024-04-24T06:35:48.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:35:48.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:35:48.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:35:48.491 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:35:48.491 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:35:48.491 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:48.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:35:48.553 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:35:48.553 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:48.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:35:48.613 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:35:48.613 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:48.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:35:48.672 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:35:48.672 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:35:48.734 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:35:48.734 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:35:48.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.952+0000 7fd189400640 1 Processor -- start 2024-04-24T06:35:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.952+0000 7fd189400640 1 -- start start 2024-04-24T06:35:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.954+0000 7fd189400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd1841454e0 0x7fd184145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.954+0000 7fd189400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd1840b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.954+0000 7fd189400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1840cd520 con 0x7fd1840b3e40 2024-04-24T06:35:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.954+0000 7fd189400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd1840cddc0 con 0x7fd1841454e0 2024-04-24T06:35:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.954+0000 7fd189400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1840cd800 con 0x7fd1840cd020 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd183200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd1840b3e40 0x7fd184160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36218/0 (socket says 172.21.15.59:36218) 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd183200640 1 -- 172.21.15.59:0/3344607171 learned_addr learned my addr 172.21.15.59:0/3344607171 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd17be00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd1840b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd182800640 1 --2- 172.21.15.59:0/3344607171 >> v2:172.21.15.66:3300/0 conn(0x7fd1841454e0 0x7fd184145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318633236 0 0) 0x7fd1840cd520 con 0x7fd1840b3e40 2024-04-24T06:35:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.955+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd168003520 con 0x7fd1840b3e40 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3869633559 0 0) 0x7fd168003520 con 0x7fd1840b3e40 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 msgr2=0x7fd1840b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd1840b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 >> v2:172.21.15.66:3300/0 conn(0x7fd1841454e0 msgr2=0x7fd184145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 --2- 172.21.15.59:0/3344607171 >> v2:172.21.15.66:3300/0 conn(0x7fd1841454e0 0x7fd184145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1840cdae0 con 0x7fd1840b3e40 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd17c0019f0 con 0x7fd1840b3e40 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 >> v1:172.21.15.59:6789/0 conn(0x7fd1840b3e40 legacy=0x7fd184160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd168004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd168004d40 0x7fd1680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840cddc0 con 0x7fd1680039b0 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840cd800 con 0x7fd168004d40 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd16800b9f0 con 0x7fd1840cd020 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.956+0000 7fd182800640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd182800640 1 -- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 msgr2=0x7fd168004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd17be00640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd168004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd182800640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1840cd020 0x7fd168004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd182800640 1 -- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd168004d40 msgr2=0x7fd1680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd182800640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd168004d40 0x7fd1680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.957+0000 7fd182800640 1 -- 172.21.15.59:0/3344607171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd168003520 con 0x7fd1680039b0 2024-04-24T06:35:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.958+0000 7fd182800640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd168003dd0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd174009ac0 tx=0x7fd168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3f89ca63e1d83b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.958+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174036070 con 0x7fd1680039b0 2024-04-24T06:35:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.958+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd174031400 con 0x7fd1680039b0 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.958+0000 7fd181e00640 1 -- 172.21.15.59:0/3344607171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174009690 con 0x7fd1680039b0 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 -- 172.21.15.59:0/3344607171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 msgr2=0x7fd168003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 --2- 172.21.15.59:0/3344607171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd168003dd0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd174009ac0 tx=0x7fd168005830 comp rx=0 tx=0).stop 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd183200640 1 -- 172.21.15.59:0/3344607171 reap_dead start 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 -- 172.21.15.59:0/3344607171 shutdown_connections 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 -- 172.21.15.59:0/3344607171 >> 172.21.15.59:0/3344607171 conn(0x7fd184067ea0 msgr2=0x7fd18406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 -- 172.21.15.59:0/3344607171 shutdown_connections 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.959+0000 7fd189400640 1 -- 172.21.15.59:0/3344607171 wait complete. 2024-04-24T06:35:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.961+0000 7fd189400640 1 Processor -- start 2024-04-24T06:35:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.961+0000 7fd189400640 1 -- start start 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1841596d0 0x7fd184143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1841444a0 0x7fd1841e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd184146270 con 0x7fd1680039b0 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841460f0 con 0x7fd1841444a0 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd189400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd184161020 con 0x7fd1841596d0 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46496/0 (socket says 172.21.15.59:46496) 2024-04-24T06:35:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 -- 172.21.15.59:0/2414413611 learned_addr learned my addr 172.21.15.59:0/2414413611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 -- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1841596d0 msgr2=0x7fd184143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1841596d0 0x7fd184143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 -- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1841444a0 msgr2=0x7fd1841e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1841444a0 0x7fd1841e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.962+0000 7fd182800640 1 -- 172.21.15.59:0/2414413611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1841e6d60 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd182800640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd174034040 tx=0x7fd174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174040020 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd174003950 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174036040 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1841e7040 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.963+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841e7600 con 0x7fd1680039b0 2024-04-24T06:35:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.964+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1841448c0 con 0x7fd1680039b0 2024-04-24T06:35:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.965+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd174043020 con 0x7fd1680039b0 2024-04-24T06:35:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.966+0000 7fd17aa00640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.966+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7fd1740aba20 con 0x7fd1680039b0 2024-04-24T06:35:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.967+0000 7fd17be00640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.968+0000 7fd17be00640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 0x7fd16006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd184144c30 tx=0x7fd16c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:48.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:48.971+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd17404a070 con 0x7fd1680039b0 2024-04-24T06:35:49.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.214+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1840b1e30 con 0x7fd1680039b0 2024-04-24T06:35:49.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.215+0000 7fd17aa00640 1 -- 172.21.15.59:0/2414413611 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+3863 (secure 0 0 0) 0x7fd17407a050 con 0x7fd1680039b0 2024-04-24T06:35:49.217 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:49.217 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5180,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3429469149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3429469149},{"type":"v1","addr":"172.21.15.66:6834","nonce":3429469149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3923270635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3923270635},{"type":"v1","addr":"172.21.15.66:6835","nonce":3923270635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6756,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/62520890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":62520890},{"type":"v1","addr":"172.21.15.59:6835","nonce":62520890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":7464,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1802672000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1802672000},{"type":"v1","addr":"172.21.15.59:6837","nonce":1802672000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-04-24T06:35:49.217 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 54 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.217+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 msgr2=0x7fd16006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.217+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 0x7fd16006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd184144c30 tx=0x7fd16c008040 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.217+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 msgr2=0x7fd1841e7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.217+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd174034040 tx=0x7fd174004730 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 shutdown_connections 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd16006c8f0 0x7fd16006edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1841444a0 0x7fd1841e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1841596d0 0x7fd184143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 --2- 172.21.15.59:0/2414413611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1680039b0 0x7fd1841e7e00 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 >> 172.21.15.59:0/2414413611 conn(0x7fd184067ea0 msgr2=0x7fd184141570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:49.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 shutdown_connections 2024-04-24T06:35:49.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.218+0000 7fd189400640 1 -- 172.21.15.59:0/2414413611 wait complete. 2024-04-24T06:35:49.239 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:35:49.239 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:35:49.239 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:35:49.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:35:49.239 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:35:49.239 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:35:49.240 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:35:49.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:35:49.240 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:35:49.240 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:35:49.240 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:35:49.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:35:49.240 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:35:49.240 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:35:49.241 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:35:49.241 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:35:49.260 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:35:49.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:35:49.266 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:35:49.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:35:49.274 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:35:49.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:35:49.275 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:35:49.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:35:49.312 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:35:49.312 DEBUG:teuthology.parallel:result is None 2024-04-24T06:35:49.317 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:35:49.317 DEBUG:teuthology.parallel:result is None 2024-04-24T06:35:49.323 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:35:49.323 DEBUG:teuthology.parallel:result is None 2024-04-24T06:35:49.328 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:35:49.328 DEBUG:teuthology.parallel:result is None 2024-04-24T06:35:49.329 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:35:49.469 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:35:49.473 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:35:49.484 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:35:49.493 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:35:49.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.510+0000 7f7f05800640 1 Processor -- start 2024-04-24T06:35:49.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.510+0000 7f7f05800640 1 -- start start 2024-04-24T06:35:49.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7f05800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7f05800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f00162de0 0x7f7f000b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7f05800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f001632e0 con 0x7f7f000afa80 2024-04-24T06:35:49.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7f05800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7f000aefd0 con 0x7f7f00162de0 2024-04-24T06:35:49.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7f05800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000adcd0 con 0x7f7f00161e60 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7efea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7ef7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7f00162de0 0x7f7f000b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7efea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:54086/0 (socket says 172.21.15.59:54086) 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7efea00640 1 -- 172.21.15.59:0/3757319421 learned_addr learned my addr 172.21.15.59:0/3757319421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.512+0000 7f7eff400640 1 --1- 172.21.15.59:0/3757319421 >> v1:172.21.15.66:6789/0 conn(0x7f7f000afa80 0x7f7f0006ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:44348/0 (socket says 172.21.15.59:44348) 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.513+0000 7f7efea00640 1 -- 172.21.15.59:0/3757319421 >> v2:172.21.15.66:3300/0 conn(0x7f7f00162de0 msgr2=0x7f7f000b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.513+0000 7f7efea00640 1 --2- 172.21.15.59:0/3757319421 >> v2:172.21.15.66:3300/0 conn(0x7f7f00162de0 0x7f7f000b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.513+0000 7f7efea00640 1 -- 172.21.15.59:0/3757319421 >> v1:172.21.15.66:6789/0 conn(0x7f7f000afa80 legacy=0x7f7f0006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.513+0000 7f7efea00640 1 -- 172.21.15.59:0/3757319421 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f000adfb0 con 0x7f7f00161e60 2024-04-24T06:35:49.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.514+0000 7f7efea00640 1 --2- 172.21.15.59:0/3757319421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009870 tx=0x7f7ef002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9cd2b09926e1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:49.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.514+0000 7f7efe000640 1 -- 172.21.15.59:0/3757319421 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0037070 con 0x7f7f00161e60 2024-04-24T06:35:49.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.514+0000 7f7efe000640 1 -- 172.21.15.59:0/3757319421 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0032440 con 0x7f7f00161e60 2024-04-24T06:35:49.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.514+0000 7f7efe000640 1 -- 172.21.15.59:0/3757319421 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0032810 con 0x7f7f00161e60 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 -- 172.21.15.59:0/3757319421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 msgr2=0x7f7f000aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 --2- 172.21.15.59:0/3757319421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009870 tx=0x7f7ef002a1c0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 -- 172.21.15.59:0/3757319421 shutdown_connections 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 --2- 172.21.15.59:0/3757319421 >> v2:172.21.15.66:3300/0 conn(0x7f7f00162de0 0x7f7f000b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 --2- 172.21.15.59:0/3757319421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f00161e60 0x7f7f000aea90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 -- 172.21.15.59:0/3757319421 >> 172.21.15.59:0/3757319421 conn(0x7f7f000b2130 msgr2=0x7f7f000b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 -- 172.21.15.59:0/3757319421 shutdown_connections 2024-04-24T06:35:49.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.515+0000 7f7f05800640 1 -- 172.21.15.59:0/3757319421 wait complete. 2024-04-24T06:35:49.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.517+0000 7f7f05800640 1 Processor -- start 2024-04-24T06:35:49.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.517+0000 7f7f05800640 1 -- start start 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 0x7f7f0015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f00162de0 0x7f7f00160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f000af480 con 0x7f7f00161e60 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f000ae570 con 0x7f7f00162de0 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7f05800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000af600 con 0x7f7f000afa80 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7ef7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7ef7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46550/0 (socket says 172.21.15.59:46550) 2024-04-24T06:35:49.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7ef7e00640 1 -- 172.21.15.59:0/918145324 learned_addr learned my addr 172.21.15.59:0/918145324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:49.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7efea00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 0x7f7f0015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.518+0000 7f7eff400640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f00162de0 0x7f7f00160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 -- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 msgr2=0x7f7f0015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 0x7f7f0015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 -- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f00162de0 msgr2=0x7f7f00160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f00162de0 0x7f7f00160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 -- 172.21.15.59:0/918145324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00161050 con 0x7f7f00161e60 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7efea00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 0x7f7f0015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef7e00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9b0 tx=0x7f7ee800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8017070 con 0x7f7f00161e60 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee800c9a0 con 0x7f7f00161e60 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0015a990 con 0x7f7f00161e60 2024-04-24T06:35:49.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800cd90 con 0x7f7f00161e60 2024-04-24T06:35:49.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.519+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0015af50 con 0x7f7f00161e60 2024-04-24T06:35:49.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.520+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f00068a80 con 0x7f7f00161e60 2024-04-24T06:35:49.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.521+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ee801a070 con 0x7f7f00161e60 2024-04-24T06:35:49.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.522+0000 7f7ef6a00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 0x7f7edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:49.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.522+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f7ee808b9a0 con 0x7f7f00161e60 2024-04-24T06:35:49.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.523+0000 7f7efea00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 0x7f7edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:49.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.523+0000 7f7efea00640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 0x7f7edc06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0034000 tx=0x7f7ef002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:49.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.527+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ee8059fd0 con 0x7f7f00161e60 2024-04-24T06:35:49.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.743+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7f0015c000 con 0x7f7f00161e60 2024-04-24T06:35:49.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.793+0000 7f7ef6a00640 1 -- 172.21.15.59:0/918145324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f7ee80598d0 con 0x7f7f00161e60 2024-04-24T06:35:49.795 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:35:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 msgr2=0x7f7edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 0x7f7edc06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0034000 tx=0x7f7ef002e000 comp rx=0 tx=0).stop 2024-04-24T06:35:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 msgr2=0x7f7f00149e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e9b0 tx=0x7f7ee800ee70 comp rx=0 tx=0).stop 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 shutdown_connections 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7edc06c8f0 0x7f7edc06edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f00162de0 0x7f7f00160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f00161e60 0x7f7f00149e50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 --2- 172.21.15.59:0/918145324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7f000afa80 0x7f7f0015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.797+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 >> 172.21.15.59:0/918145324 conn(0x7f7f000b2130 msgr2=0x7f7f000b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.798+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 shutdown_connections 2024-04-24T06:35:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:49.798+0000 7f7f05800640 1 -- 172.21.15.59:0/918145324 wait complete. 2024-04-24T06:35:49.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:35:50.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.023+0000 7f1552a00640 1 Processor -- start 2024-04-24T06:35:50.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.023+0000 7f1552a00640 1 -- start start 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1552a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 0x7f154c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1552a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f154c0b4690 0x7f154c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1552a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154c160210 con 0x7f154c15c3d0 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1552a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f154c1604f0 con 0x7f154c0b4690 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1552a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f154c1607d0 con 0x7f154c0cd760 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1551600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f154c15c3d0 0x7f154c0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36234/0 (socket says 172.21.15.59:36234) 2024-04-24T06:35:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.025+0000 7f1551600640 1 -- 172.21.15.59:0/3306071783 learned_addr learned my addr 172.21.15.59:0/3306071783 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f1552000640 1 --2- 172.21.15.59:0/3306071783 >> v2:172.21.15.66:3300/0 conn(0x7f154c0b4690 0x7f154c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f1550c00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 0x7f154c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3948806134 0 0) 0x7f154c160210 con 0x7f154c15c3d0 2024-04-24T06:35:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1534003520 con 0x7f154c15c3d0 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1734999210 0 0) 0x7f1534003520 con 0x7f154c15c3d0 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 msgr2=0x7f154c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 0x7f154c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 >> v2:172.21.15.66:3300/0 conn(0x7f154c0b4690 msgr2=0x7f154c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 --2- 172.21.15.59:0/3306071783 >> v2:172.21.15.66:3300/0 conn(0x7f154c0b4690 0x7f154c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c160ab0 con 0x7f154c15c3d0 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1540001670 con 0x7f154c15c3d0 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 >> v1:172.21.15.59:6789/0 conn(0x7f154c15c3d0 legacy=0x7f154c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 0x7f15340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f154c1604f0 con 0x7f15340039b0 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f154c1607d0 con 0x7f1534004800 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.026+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f153400ba40 con 0x7f154c0cd760 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1550c00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 -- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 msgr2=0x7f15340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cd760 0x7f15340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 -- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1534004800 msgr2=0x7f1534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1551600640 1 -- 172.21.15.59:0/3306071783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1534003520 con 0x7f15340039b0 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.027+0000 7f1550c00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.028+0000 7f1551600640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f154000c290 tx=0x7f1534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8f07a6fc436db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.028+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540007d50 con 0x7f15340039b0 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.028+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540005ad0 con 0x7f15340039b0 2024-04-24T06:35:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.028+0000 7f154be00640 1 -- 172.21.15.59:0/3306071783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540007070 con 0x7f15340039b0 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 -- 172.21.15.59:0/3306071783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 msgr2=0x7f1534003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 --2- 172.21.15.59:0/3306071783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f154000c290 tx=0x7f1534005830 comp rx=0 tx=0).stop 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552000640 1 -- 172.21.15.59:0/3306071783 reap_dead start 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 -- 172.21.15.59:0/3306071783 shutdown_connections 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 -- 172.21.15.59:0/3306071783 >> 172.21.15.59:0/3306071783 conn(0x7f154c0b21a0 msgr2=0x7f154c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 -- 172.21.15.59:0/3306071783 shutdown_connections 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.029+0000 7f1552a00640 1 -- 172.21.15.59:0/3306071783 wait complete. 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.031+0000 7f1552a00640 1 Processor -- start 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.031+0000 7f1552a00640 1 -- start start 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cfa60 0x7f154c15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f154c158ad0 0x7f154c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f154c15c7b0 con 0x7f15340039b0 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f154c0b4ac0 con 0x7f154c158ad0 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1552a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f154c15c930 con 0x7f154c0cfa60 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1551600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1551600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.032+0000 7f1551600640 1 -- 172.21.15.59:0/945524032 learned_addr learned my addr 172.21.15.59:0/945524032 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1550c00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cfa60 0x7f154c15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 -- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cfa60 msgr2=0x7f154c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cfa60 0x7f154c15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1552000640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f154c158ad0 0x7f154c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 -- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f154c158ad0 msgr2=0x7f154c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f154c158ad0 0x7f154c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 -- 172.21.15.59:0/945524032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c0cebe0 con 0x7f15340039b0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1551600640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f154000c2c0 tx=0x7f154c1603e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400015b0 con 0x7f15340039b0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540038040 con 0x7f15340039b0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154c0ceec0 con 0x7f15340039b0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.033+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540007290 con 0x7f15340039b0 2024-04-24T06:35:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.034+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154c0aef40 con 0x7f15340039b0 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.035+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154c04fd90 con 0x7f15340039b0 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.036+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1540007430 con 0x7f15340039b0 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.036+0000 7f1549600640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 0x7f152c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.037+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f15400ab7c0 con 0x7f15340039b0 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.037+0000 7f1550c00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 0x7f152c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.037+0000 7f1550c00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 0x7f152c06edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f154c099da0 tx=0x7f1538008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.041+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f154004b020 con 0x7f15340039b0 2024-04-24T06:35:50.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.326+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f154c15dbe0 con 0x7f15340039b0 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.327+0000 7f1549600640 1 -- 172.21.15.59:0/945524032 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f1540079f20 con 0x7f15340039b0 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.329 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:35:50.330 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.331 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.332 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:35:50.333 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.334 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:35:50.335 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:35:50.337 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:35:50.338 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:35:50.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:35:50.341 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:35:50.342 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.329+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 msgr2=0x7f152c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.329+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 0x7f152c06edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f154c099da0 tx=0x7f1538008040 comp rx=0 tx=0).stop 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.329+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 msgr2=0x7f154c0af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.329+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f154000c2c0 tx=0x7f154c1603e0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 shutdown_connections 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f152c06c8f0 0x7f152c06edb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f154c158ad0 0x7f154c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f154c0cfa60 0x7f154c15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 --2- 172.21.15.59:0/945524032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15340039b0 0x7f154c0af460 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 >> 172.21.15.59:0/945524032 conn(0x7f154c0b21a0 msgr2=0x7f154c06ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:50.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 shutdown_connections 2024-04-24T06:35:50.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.330+0000 7f1552a00640 1 -- 172.21.15.59:0/945524032 wait complete. 2024-04-24T06:35:50.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.565+0000 7f280da00640 1 Processor -- start 2024-04-24T06:35:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.566+0000 7f280da00640 1 -- start start 2024-04-24T06:35:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.567+0000 7f280da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f28081454e0 0x7f2808145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.567+0000 7f280da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 0x7f2808160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f280da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28080cd520 con 0x7f28080cd020 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f280da00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f28080cddc0 con 0x7f28081454e0 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f280da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f28080cd800 con 0x7f2808160910 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f2806000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f28080cd020 0x7f28080b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36244/0 (socket says 172.21.15.59:36244) 2024-04-24T06:35:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f2806000640 1 -- 172.21.15.59:0/2490576424 learned_addr learned my addr 172.21.15.59:0/2490576424 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f2806a00640 1 --2- 172.21.15.59:0/2490576424 >> v2:172.21.15.66:3300/0 conn(0x7f28081454e0 0x7f2808145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f2807400640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 0x7f2808160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.568+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743069610 0 0) 0x7f28080cd520 con 0x7f28080cd020 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27ec003520 con 0x7f28080cd020 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3371379577 0 0) 0x7f27ec003520 con 0x7f28080cd020 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 msgr2=0x7f2808160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 0x7f2808160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 >> v2:172.21.15.66:3300/0 conn(0x7f28081454e0 msgr2=0x7f2808145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 --2- 172.21.15.59:0/2490576424 >> v2:172.21.15.66:3300/0 conn(0x7f28081454e0 0x7f2808145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28080cdae0 con 0x7f28080cd020 2024-04-24T06:35:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f27f0001670 con 0x7f28080cd020 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 >> v1:172.21.15.59:6789/0 conn(0x7f28080cd020 legacy=0x7f28080b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 0x7f27ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec003f30 0x7f27ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27ec004d40 0x7f27ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28080cddc0 con 0x7f27ec004d40 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28080cd800 con 0x7f27ec003f30 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec00ba10 con 0x7f2808160910 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.569+0000 7f2807400640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27ec004d40 0x7f27ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2806000640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec003f30 0x7f27ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2807400640 1 -- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 msgr2=0x7f27ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2807400640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2808160910 0x7f27ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2807400640 1 -- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec003f30 msgr2=0x7f27ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2807400640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec003f30 0x7f27ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.570+0000 7f2807400640 1 -- 172.21.15.59:0/2490576424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec003520 con 0x7f27ec004d40 2024-04-24T06:35:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.571+0000 7f2807400640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27ec004d40 0x7f27ec0051c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f280000e170 tx=0x7f27ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1382d1ab951bb3bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.571+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800034020 con 0x7f27ec004d40 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.571+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2800030430 con 0x7f27ec004d40 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.571+0000 7f2805600640 1 -- 172.21.15.59:0/2490576424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800009c10 con 0x7f27ec004d40 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 -- 172.21.15.59:0/2490576424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27ec004d40 msgr2=0x7f27ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 --2- 172.21.15.59:0/2490576424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27ec004d40 0x7f27ec0051c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f280000e170 tx=0x7f27ec005830 comp rx=0 tx=0).stop 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f2807400640 1 -- 172.21.15.59:0/2490576424 reap_dead start 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 -- 172.21.15.59:0/2490576424 shutdown_connections 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 -- 172.21.15.59:0/2490576424 >> 172.21.15.59:0/2490576424 conn(0x7f2808067ea0 msgr2=0x7f280806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 -- 172.21.15.59:0/2490576424 shutdown_connections 2024-04-24T06:35:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.572+0000 7f280da00640 1 -- 172.21.15.59:0/2490576424 wait complete. 2024-04-24T06:35:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.574+0000 7f280da00640 1 Processor -- start 2024-04-24T06:35:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.574+0000 7f280da00640 1 -- start start 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec004d40 0x7f2808139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 0x7f28080b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28080b4010 con 0x7f280815bf70 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28081460f0 con 0x7f27ec004d40 2024-04-24T06:35:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f280da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f280813e490 con 0x7f280813be50 2024-04-24T06:35:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.575+0000 7f2806000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec004d40 0x7f2808139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46584/0 (socket says 172.21.15.59:46584) 2024-04-24T06:35:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 -- 172.21.15.59:0/3713856462 learned_addr learned my addr 172.21.15.59:0/3713856462 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2807400640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 0x7f28080b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 -- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 msgr2=0x7f28080b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 0x7f28080b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 -- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec004d40 msgr2=0x7f2808139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec004d40 0x7f2808139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2806000640 1 -- 172.21.15.59:0/3713856462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280813c3b0 con 0x7f280815bf70 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.576+0000 7f2807400640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 0x7f28080b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f2806000640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e8f0 tx=0x7f27f000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000c870 con 0x7f280815bf70 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f0018070 con 0x7f280815bf70 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000cc50 con 0x7f280815bf70 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280813c690 con 0x7f280815bf70 2024-04-24T06:35:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.577+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081f85f0 con 0x7f280815bf70 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.578+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2808068a80 con 0x7f280815bf70 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.579+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27f0013420 con 0x7f280815bf70 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.580+0000 7f27fea00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 0x7f27e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.580+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f27f008b3b0 con 0x7f280815bf70 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.580+0000 7f2806a00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 0x7f27e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:50.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.581+0000 7f2806a00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 0x7f27e406ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004770 tx=0x7f27f80094a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.585+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27f0059b20 con 0x7f280815bf70 2024-04-24T06:35:50.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.794+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2808160030 con 0x7f280815bf70 2024-04-24T06:35:50.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.795+0000 7f27fea00640 1 -- 172.21.15.59:0/3713856462 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+12808 (secure 0 0 0) 0x7f27f0059420 con 0x7f280815bf70 2024-04-24T06:35:50.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:50.798 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":79,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:49.808724+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.798+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 msgr2=0x7f27e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.798+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 0x7f27e406ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004770 tx=0x7f27f80094a0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.798+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 msgr2=0x7f280813b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:50.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.798+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e8f0 tx=0x7f27f000edb0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 shutdown_connections 2024-04-24T06:35:50.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27e406c920 0x7f27e406ede0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280813be50 0x7f28080b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f280815bf70 0x7f280813b910 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 --2- 172.21.15.59:0/3713856462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27ec004d40 0x7f2808139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 >> 172.21.15.59:0/3713856462 conn(0x7f2808067ea0 msgr2=0x7f28080b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 shutdown_connections 2024-04-24T06:35:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:50.799+0000 7f280da00640 1 -- 172.21.15.59:0/3713856462 wait complete. 2024-04-24T06:35:50.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:35:50.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:35:51.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.020+0000 7fbc4b200640 1 Processor -- start 2024-04-24T06:35:51.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.020+0000 7fbc4b200640 1 -- start start 2024-04-24T06:35:51.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.021+0000 7fbc4b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbc4c0d0a90 0x7fbc4c0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.021+0000 7fbc4b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc4c15ef20 con 0x7fbc4c0d0600 2024-04-24T06:35:51.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.021+0000 7fbc4b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc4c0d0f00 con 0x7fbc4c155a80 2024-04-24T06:35:51.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.021+0000 7fbc4b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbc4c15f1c0 con 0x7fbc4c0d0a90 2024-04-24T06:35:51.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.021+0000 7fbc41400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc4c0d0600 0x7fbc4c0ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36248/0 (socket says 172.21.15.59:36248) 2024-04-24T06:35:51.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc41400640 1 -- 172.21.15.59:0/603309643 learned_addr learned my addr 172.21.15.59:0/603309643 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:51.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc4a800640 1 --2- 172.21.15.59:0/603309643 >> v2:172.21.15.66:3300/0 conn(0x7fbc4c0d0a90 0x7fbc4c0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268382608 0 0) 0x7fbc4c15ef20 con 0x7fbc4c0d0600 2024-04-24T06:35:51.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc30003520 con 0x7fbc4c0d0600 2024-04-24T06:35:51.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 339142299 0 0) 0x7fbc30003520 con 0x7fbc4c0d0600 2024-04-24T06:35:51.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 >> v2:172.21.15.66:3300/0 conn(0x7fbc4c0d0a90 msgr2=0x7fbc4c0ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 --2- 172.21.15.59:0/603309643 >> v2:172.21.15.66:3300/0 conn(0x7fbc4c0d0a90 0x7fbc4c0ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 >> v1:172.21.15.66:6789/0 conn(0x7fbc4c155a80 legacy=0x7fbc4c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c15f4a0 con 0x7fbc4c0d0600 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.022+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbc34001670 con 0x7fbc4c0d0600 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 >> v1:172.21.15.59:6789/0 conn(0x7fbc4c0d0600 legacy=0x7fbc4c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30003a00 0x7fbc30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 0x7fbc30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc300051c0 0x7fbc3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c15f1c0 con 0x7fbc30004800 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3000c0c0 con 0x7fbc30003a00 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3000c2c0 con 0x7fbc300051c0 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 0x7fbc30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc49e00640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30003a00 0x7fbc30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 -- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc300051c0 msgr2=0x7fbc3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc300051c0 0x7fbc3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 -- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30003a00 msgr2=0x7fbc30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30003a00 0x7fbc30003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.023+0000 7fbc41400640 1 -- 172.21.15.59:0/603309643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc30003520 con 0x7fbc30004800 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.024+0000 7fbc4a800640 1 -- 172.21.15.59:0/603309643 reap_dead start 2024-04-24T06:35:51.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.024+0000 7fbc41400640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 0x7fbc30004c80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c190 tx=0x7fbc30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30abc60177fab628 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.025+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34007c60 con 0x7fbc30004800 2024-04-24T06:35:51.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.025+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34005af0 con 0x7fbc30004800 2024-04-24T06:35:51.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.025+0000 7fbc49400640 1 -- 172.21.15.59:0/603309643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34007070 con 0x7fbc30004800 2024-04-24T06:35:51.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.025+0000 7fbc4b200640 1 -- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 msgr2=0x7fbc30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.025+0000 7fbc4b200640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 0x7fbc30004c80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c190 tx=0x7fbc30005830 comp rx=0 tx=0).stop 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.026+0000 7fbc4b200640 1 -- 172.21.15.59:0/603309643 shutdown_connections 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.026+0000 7fbc4b200640 1 --2- 172.21.15.59:0/603309643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc30004800 0x7fbc30004c80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.026+0000 7fbc4b200640 1 -- 172.21.15.59:0/603309643 >> 172.21.15.59:0/603309643 conn(0x7fbc4c06bdf0 msgr2=0x7fbc4c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.026+0000 7fbc4b200640 1 -- 172.21.15.59:0/603309643 shutdown_connections 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.026+0000 7fbc4b200640 1 -- 172.21.15.59:0/603309643 wait complete. 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.028+0000 7fbc4b200640 1 Processor -- start 2024-04-24T06:35:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.028+0000 7fbc4b200640 1 -- start start 2024-04-24T06:35:51.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30004800 0x7fbc4c14e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c14eb30 0x7fbc4c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c15fd40 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c15fa60 con 0x7fbc30004800 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4c06ef80 con 0x7fbc4c14eb30 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc41400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc41400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46614/0 (socket says 172.21.15.59:46614) 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc41400640 1 -- 172.21.15.59:0/4065504841 learned_addr learned my addr 172.21.15.59:0/4065504841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc49e00640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30004800 0x7fbc4c14e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc4a800640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c14eb30 0x7fbc4c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.029+0000 7fbc41400640 1 -- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c14eb30 msgr2=0x7fbc4c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc41400640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c14eb30 0x7fbc4c1542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc41400640 1 -- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30004800 msgr2=0x7fbc4c14e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc41400640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30004800 0x7fbc4c14e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc41400640 1 -- 172.21.15.59:0/4065504841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c0b0fa0 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc41400640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c190 tx=0x7fbc34001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34001670 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34043070 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34007290 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c0b1280 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.030+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c0b1840 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.032+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4c04fd90 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.033+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc34007430 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.034+0000 7fbc42a00640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 0x7fbc2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.034+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fbc340aba50 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.034+0000 7fbc49e00640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 0x7fbc2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.035+0000 7fbc49e00640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 0x7fbc2806ede0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007fd0 tx=0x7fbc3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.038+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc34030380 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.243+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc4c0b34b0 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.244+0000 7fbc42a00640 1 -- 172.21.15.59:0/4065504841 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fbc3407a140 con 0x7fbc4c0b0b00 2024-04-24T06:35:51.246 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:35:51.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.246+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 msgr2=0x7fbc2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.246+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 0x7fbc2806ede0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007fd0 tx=0x7fbc3c009040 comp rx=0 tx=0).stop 2024-04-24T06:35:51.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 msgr2=0x7fbc4c14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c190 tx=0x7fbc34001970 comp rx=0 tx=0).stop 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 shutdown_connections 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc2806c920 0x7fbc2806ede0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc4c14eb30 0x7fbc4c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc4c0b0b00 0x7fbc4c14dfc0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 --2- 172.21.15.59:0/4065504841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc30004800 0x7fbc4c14e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 >> 172.21.15.59:0/4065504841 conn(0x7fbc4c06bdf0 msgr2=0x7fbc4c0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:51.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 shutdown_connections 2024-04-24T06:35:51.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.247+0000 7fbc4b200640 1 -- 172.21.15.59:0/4065504841 wait complete. 2024-04-24T06:35:51.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:35:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.483+0000 7f085ba00640 1 Processor -- start 2024-04-24T06:35:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.483+0000 7f085ba00640 1 -- start start 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.484+0000 7f085ba00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f085415e9d0 0x7f085415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f085ba00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08540d1670 0x7f08540d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f085ba00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085415edb0 con 0x7f08540d42c0 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f085ba00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f08540d3e30 con 0x7f085415e9d0 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f085ba00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f085415ff20 con 0x7f08540d1670 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f085415e9d0 0x7f085415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f085415e9d0 0x7f085415f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46628/0 (socket says 172.21.15.59:46628) 2024-04-24T06:35:51.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 -- 172.21.15.59:0/2443805015 learned_addr learned my addr 172.21.15.59:0/2443805015 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 -- 172.21.15.59:0/2443805015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08540d1670 msgr2=0x7f08540d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 --2- 172.21.15.59:0/2443805015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08540d1670 0x7f08540d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 -- 172.21.15.59:0/2443805015 >> v1:172.21.15.66:6789/0 conn(0x7f08540d42c0 legacy=0x7f08540d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.485+0000 7f0859000640 1 -- 172.21.15.59:0/2443805015 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540ba630 con 0x7f085415e9d0 2024-04-24T06:35:51.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.486+0000 7f0859000640 1 --2- 172.21.15.59:0/2443805015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085415e9d0 0x7f085415f9e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0844009870 tx=0x7f084402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823755c074e25a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.487+0000 7f0853400640 1 -- 172.21.15.59:0/2443805015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844037070 con 0x7f085415e9d0 2024-04-24T06:35:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.487+0000 7f0853400640 1 -- 172.21.15.59:0/2443805015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844032440 con 0x7f085415e9d0 2024-04-24T06:35:51.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.487+0000 7f0853400640 1 -- 172.21.15.59:0/2443805015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844032810 con 0x7f085415e9d0 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.487+0000 7f085ba00640 1 -- 172.21.15.59:0/2443805015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085415e9d0 msgr2=0x7f085415f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.487+0000 7f085ba00640 1 --2- 172.21.15.59:0/2443805015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085415e9d0 0x7f085415f9e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0844009870 tx=0x7f084402a210 comp rx=0 tx=0).stop 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 -- 172.21.15.59:0/2443805015 shutdown_connections 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 --2- 172.21.15.59:0/2443805015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f08540d1670 0x7f08540d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 --2- 172.21.15.59:0/2443805015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f085415e9d0 0x7f085415f9e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 -- 172.21.15.59:0/2443805015 >> 172.21.15.59:0/2443805015 conn(0x7f085406cbd0 msgr2=0x7f085406cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 -- 172.21.15.59:0/2443805015 shutdown_connections 2024-04-24T06:35:51.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.488+0000 7f085ba00640 1 -- 172.21.15.59:0/2443805015 wait complete. 2024-04-24T06:35:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.489+0000 7f085ba00640 1 Processor -- start 2024-04-24T06:35:51.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.490+0000 7f085ba00640 1 -- start start 2024-04-24T06:35:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08540d1670 0x7f085414c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f085415e9d0 0x7f08540cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08540cfd70 con 0x7f08540d42c0 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08540cfa90 con 0x7f08540d1670 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f085ba00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f08540ba0e0 con 0x7f085415e9d0 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46644/0 (socket says 172.21.15.59:46644) 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 -- 172.21.15.59:0/3117449957 learned_addr learned my addr 172.21.15.59:0/3117449957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0859000640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08540d1670 0x7f085414c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0859a00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f085415e9d0 0x7f08540cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 -- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f085415e9d0 msgr2=0x7f08540cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f085415e9d0 0x7f08540cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 -- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08540d1670 msgr2=0x7f085414c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.491+0000 7f0853e00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08540d1670 0x7f085414c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f0853e00640 1 -- 172.21.15.59:0/3117449957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541f46f0 con 0x7f08540d42c0 2024-04-24T06:35:51.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f0853e00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f084000c7a0 tx=0x7f084000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840017070 con 0x7f08540d42c0 2024-04-24T06:35:51.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0840003da0 con 0x7f08540d42c0 2024-04-24T06:35:51.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08400052b0 con 0x7f08540d42c0 2024-04-24T06:35:51.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08540c9dd0 con 0x7f08540d42c0 2024-04-24T06:35:51.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.492+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08540ca390 con 0x7f08540d42c0 2024-04-24T06:35:51.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.493+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540b5530 con 0x7f08540d42c0 2024-04-24T06:35:51.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.494+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084001a070 con 0x7f08540d42c0 2024-04-24T06:35:51.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.495+0000 7f0850c00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 0x7f083406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.496+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f084008afc0 con 0x7f08540d42c0 2024-04-24T06:35:51.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.496+0000 7f0859000640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 0x7f083406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.496+0000 7f0859000640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 0x7f083406edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0844034000 tx=0x7f084402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.500+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08400596a0 con 0x7f08540d42c0 2024-04-24T06:35:51.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.705+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f08540c9240 con 0x7f08540d42c0 2024-04-24T06:35:51.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.706+0000 7f0850c00640 1 -- 172.21.15.59:0/3117449957 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f0840058fa0 con 0x7f08540d42c0 2024-04-24T06:35:51.708 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:51.708 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:35:51.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:35:51.708 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:35:51.709 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:35:51.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:35:51.709 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 msgr2=0x7f083406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 0x7f083406edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0844034000 tx=0x7f084402e000 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 msgr2=0x7f08540c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f084000c7a0 tx=0x7f084000cc60 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 shutdown_connections 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f083406c8f0 0x7f083406edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f085415e9d0 0x7f08540cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08540d42c0 0x7f08540c8d00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.709+0000 7f085ba00640 1 --2- 172.21.15.59:0/3117449957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08540d1670 0x7f085414c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.710+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 >> 172.21.15.59:0/3117449957 conn(0x7f085406cbd0 msgr2=0x7f08540b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.710+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 shutdown_connections 2024-04-24T06:35:51.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.710+0000 7f085ba00640 1 -- 172.21.15.59:0/3117449957 wait complete. 2024-04-24T06:35:51.731 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:35:51.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:35:51.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.951+0000 7f83fd200640 1 Processor -- start 2024-04-24T06:35:51.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.951+0000 7f83fd200640 1 -- start start 2024-04-24T06:35:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83f8144f40 0x7f83f80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f80d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 0x7f83f80b4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80ce9c0 con 0x7f83f8144f40 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80ceca0 con 0x7f83f80d0650 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.953+0000 7f83fd200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80cef80 con 0x7f83f80ce230 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83f8144f40 0x7f83f80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83f8144f40 0x7f83f80b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46660/0 (socket says 172.21.15.59:46660) 2024-04-24T06:35:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 -- 172.21.15.59:0/3703207814 learned_addr learned my addr 172.21.15.59:0/3703207814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f6800640 1 --2- 172.21.15.59:0/3703207814 >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 0x7f83f80b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5400640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f80d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 -- 172.21.15.59:0/3703207814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 msgr2=0x7f83f80d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f80d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 -- 172.21.15.59:0/3703207814 >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 msgr2=0x7f83f80b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 --2- 172.21.15.59:0/3703207814 >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 0x7f83f80b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f5e00640 1 -- 172.21.15.59:0/3703207814 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80cf260 con 0x7f83f8144f40 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.954+0000 7f83f6800640 1 --2- 172.21.15.59:0/3703207814 >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 0x7f83f80b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.955+0000 7f83f5e00640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f80b3f60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ccc70 tx=0x7f83e802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6d8200552e3682 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83f4a00640 1 -- 172.21.15.59:0/3703207814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8032790 con 0x7f83f8144f40 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83f4a00640 1 -- 172.21.15.59:0/3703207814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8038070 con 0x7f83f8144f40 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83f4a00640 1 -- 172.21.15.59:0/3703207814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8032b50 con 0x7f83f8144f40 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 -- 172.21.15.59:0/3703207814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 msgr2=0x7f83f80b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f80b3f60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ccc70 tx=0x7f83e802c890 comp rx=0 tx=0).stop 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 -- 172.21.15.59:0/3703207814 shutdown_connections 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 --2- 172.21.15.59:0/3703207814 >> v2:172.21.15.66:3300/0 conn(0x7f83f80d0650 0x7f83f80b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f80d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 --2- 172.21.15.59:0/3703207814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f80b3f60 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.956+0000 7f83fd200640 1 -- 172.21.15.59:0/3703207814 >> 172.21.15.59:0/3703207814 conn(0x7f83f806b9d0 msgr2=0x7f83f806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.957+0000 7f83fd200640 1 -- 172.21.15.59:0/3703207814 shutdown_connections 2024-04-24T06:35:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.957+0000 7f83fd200640 1 -- 172.21.15.59:0/3703207814 wait complete. 2024-04-24T06:35:51.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.958+0000 7f83fd200640 1 Processor -- start 2024-04-24T06:35:51.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.958+0000 7f83fd200640 1 -- start start 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.959+0000 7f83fd200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.959+0000 7f83fd200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83f80d0650 0x7f83f815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83fd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83fd200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8145320 con 0x7f83f8144f40 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83fd200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80ce690 con 0x7f83f80d0650 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83fd200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80b4d90 con 0x7f83f80ce230 2024-04-24T06:35:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46664/0 (socket says 172.21.15.59:46664) 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 -- 172.21.15.59:0/2926911821 learned_addr learned my addr 172.21.15.59:0/2926911821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f5e00640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f5400640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83f80d0650 0x7f83f815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 -- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 msgr2=0x7f83f815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 -- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83f80d0650 msgr2=0x7f83f815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.960+0000 7f83f6800640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83f80d0650 0x7f83f815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83f6800640 1 -- 172.21.15.59:0/2926911821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f81e6d50 con 0x7f83f8144f40 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83f6800640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f83f000cef0 tx=0x7f83f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f001a070 con 0x7f83f8144f40 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f000d860 con 0x7f83f8144f40 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f000dc50 con 0x7f83f8144f40 2024-04-24T06:35:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f81e7030 con 0x7f83f8144f40 2024-04-24T06:35:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.961+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81e75f0 con 0x7f83f8144f40 2024-04-24T06:35:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.963+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f806f140 con 0x7f83f8144f40 2024-04-24T06:35:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.964+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83f001d070 con 0x7f83f8144f40 2024-04-24T06:35:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.965+0000 7f83ee000640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 0x7f83d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.965+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f83f008c090 con 0x7f83f8144f40 2024-04-24T06:35:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.965+0000 7f83f5e00640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 0x7f83d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:51.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.966+0000 7f83f5e00640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 0x7f83d806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83e8037000 tx=0x7f83e8032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:51.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:51.969+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83f005a8d0 con 0x7f83f8144f40 2024-04-24T06:35:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.173+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f83f8068a80 con 0x7f83f8144f40 2024-04-24T06:35:52.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.805+0000 7f83ee000640 1 -- 172.21.15.59:0/2926911821 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f83f005a1d0 con 0x7f83f8144f40 2024-04-24T06:35:52.807 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:35:52.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.809+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 msgr2=0x7f83d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:52.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.809+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 0x7f83d806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83e8037000 tx=0x7f83e8032000 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.809+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 msgr2=0x7f83f815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.809+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f83f000cef0 tx=0x7f83f000a710 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 shutdown_connections 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f83d806c8f0 0x7f83d806edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83f8144f40 0x7f83f815b240 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83f80d0650 0x7f83f815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 --2- 172.21.15.59:0/2926911821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f83f80ce230 0x7f83f815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 >> 172.21.15.59:0/2926911821 conn(0x7f83f806b9d0 msgr2=0x7f83f806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:52.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 shutdown_connections 2024-04-24T06:35:52.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:52.810+0000 7f83fd200640 1 -- 172.21.15.59:0/2926911821 wait complete. 2024-04-24T06:35:52.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:35:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.040+0000 7efcc9200640 1 Processor -- start 2024-04-24T06:35:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.040+0000 7efcc9200640 1 -- start start 2024-04-24T06:35:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.042+0000 7efcc9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcc40afab0 0x7efcc40aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.042+0000 7efcc9200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efcc40b5280 0x7efcc40b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.042+0000 7efcc9200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc406efc0 con 0x7efcc415f900 2024-04-24T06:35:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.042+0000 7efcc9200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efcc415fce0 con 0x7efcc40afab0 2024-04-24T06:35:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.042+0000 7efcc9200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efcc406fbd0 con 0x7efcc40b5280 2024-04-24T06:35:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcc40afab0 0x7efcc40aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcc40afab0 0x7efcc40aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46668/0 (socket says 172.21.15.59:46668) 2024-04-24T06:35:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc2a00640 1 -- 172.21.15.59:0/2286673380 learned_addr learned my addr 172.21.15.59:0/2286673380 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc3e00640 1 --2- 172.21.15.59:0/2286673380 >> v2:172.21.15.66:3300/0 conn(0x7efcc40b5280 0x7efcc40b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc2a00640 1 -- 172.21.15.59:0/2286673380 >> v2:172.21.15.66:3300/0 conn(0x7efcc40b5280 msgr2=0x7efcc40b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.043+0000 7efcc2a00640 1 --2- 172.21.15.59:0/2286673380 >> v2:172.21.15.66:3300/0 conn(0x7efcc40b5280 0x7efcc40b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.044+0000 7efcc2a00640 1 -- 172.21.15.59:0/2286673380 >> v1:172.21.15.66:6789/0 conn(0x7efcc415f900 legacy=0x7efcc406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.044+0000 7efcc2a00640 1 -- 172.21.15.59:0/2286673380 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc406f2a0 con 0x7efcc40afab0 2024-04-24T06:35:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.045+0000 7efcc2a00640 1 --2- 172.21.15.59:0/2286673380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc40aff60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7efcc40b2940 tx=0x7efcb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807155dd9fb579d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.045+0000 7efcc2000640 1 -- 172.21.15.59:0/2286673380 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0037070 con 0x7efcc40afab0 2024-04-24T06:35:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.045+0000 7efcc2000640 1 -- 172.21.15.59:0/2286673380 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb0032440 con 0x7efcc40afab0 2024-04-24T06:35:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.045+0000 7efcc2000640 1 -- 172.21.15.59:0/2286673380 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0032810 con 0x7efcc40afab0 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 -- 172.21.15.59:0/2286673380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 msgr2=0x7efcc40aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 --2- 172.21.15.59:0/2286673380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc40aff60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7efcc40b2940 tx=0x7efcb002a1e0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 -- 172.21.15.59:0/2286673380 shutdown_connections 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 --2- 172.21.15.59:0/2286673380 >> v2:172.21.15.66:3300/0 conn(0x7efcc40b5280 0x7efcc40b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 --2- 172.21.15.59:0/2286673380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc40aff60 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 -- 172.21.15.59:0/2286673380 >> 172.21.15.59:0/2286673380 conn(0x7efcc4073340 msgr2=0x7efcc406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 -- 172.21.15.59:0/2286673380 shutdown_connections 2024-04-24T06:35:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.046+0000 7efcc9200640 1 -- 172.21.15.59:0/2286673380 wait complete. 2024-04-24T06:35:53.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.048+0000 7efcc9200640 1 Processor -- start 2024-04-24T06:35:53.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.048+0000 7efcc9200640 1 -- start start 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efcc40b5280 0x7efcc413ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcc415f900 0x7efcc41404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efcc406ec80 con 0x7efcc40afab0 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efcc406f860 con 0x7efcc415f900 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc9200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efcc415ef10 con 0x7efcc40b5280 2024-04-24T06:35:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46676/0 (socket says 172.21.15.59:46676) 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3400640 1 -- 172.21.15.59:0/58592723 learned_addr learned my addr 172.21.15.59:0/58592723 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc2a00640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efcc40b5280 0x7efcc413ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3400640 1 -- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efcc40b5280 msgr2=0x7efcc413ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3400640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efcc40b5280 0x7efcc413ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.049+0000 7efcc3e00640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcc415f900 0x7efcc41404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc3400640 1 -- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcc415f900 msgr2=0x7efcc41404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc3400640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcc415f900 0x7efcc41404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc3400640 1 -- 172.21.15.59:0/58592723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc415b520 con 0x7efcc40afab0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc3400640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c430 tx=0x7efcb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40052d0 con 0x7efcc40afab0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb4005490 con 0x7efcc40afab0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007260 con 0x7efcc40afab0 2024-04-24T06:35:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc415b7a0 con 0x7efcc40afab0 2024-04-24T06:35:53.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.050+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc415ba60 con 0x7efcc40afab0 2024-04-24T06:35:53.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.052+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc404fd90 con 0x7efcc40afab0 2024-04-24T06:35:53.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.053+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcb4007400 con 0x7efcc40afab0 2024-04-24T06:35:53.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.054+0000 7efcbb400640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:53.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.054+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7efcb408b000 con 0x7efcc40afab0 2024-04-24T06:35:53.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.054+0000 7efcc2a00640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:53.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.055+0000 7efcc2a00640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 0x7efca406ede0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:53.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.059+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcc415b520 con 0x7efcc40afab0 2024-04-24T06:35:53.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.266+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efcc40b69b0 con 0x7efcc40afab0 2024-04-24T06:35:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.824+0000 7efcbb400640 1 -- 172.21.15.59:0/58592723 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7efcb40594e0 con 0x7efcc40afab0 2024-04-24T06:35:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 msgr2=0x7efca406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 0x7efca406ede0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb002e000 comp rx=0 tx=0).stop 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 msgr2=0x7efcc4159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c430 tx=0x7efcb400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 shutdown_connections 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efca406c920 0x7efca406ede0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcc415f900 0x7efcc41404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efcc40b5280 0x7efcc413ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 --2- 172.21.15.59:0/58592723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcc40afab0 0x7efcc4159a90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:53.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.828+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 >> 172.21.15.59:0/58592723 conn(0x7efcc4073340 msgr2=0x7efcc415d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:53.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.829+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 shutdown_connections 2024-04-24T06:35:53.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:53.829+0000 7efcc9200640 1 -- 172.21.15.59:0/58592723 wait complete. 2024-04-24T06:35:53.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:35:54.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.050+0000 7f5fdfe00640 1 Processor -- start 2024-04-24T06:35:54.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.051+0000 7f5fdfe00640 1 -- start start 2024-04-24T06:35:54.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.052+0000 7f5fdfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fe01567e0 0x7f5fe0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.052+0000 7f5fdfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5fe00d0d10 0x7f5fe0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.052+0000 7f5fdfe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe015cab0 con 0x7f5fe015d810 2024-04-24T06:35:54.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.052+0000 7f5fdfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe0160e40 con 0x7f5fe01567e0 2024-04-24T06:35:54.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.052+0000 7f5fdfe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe015cd30 con 0x7f5fe00d0d10 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fe01567e0 0x7f5fe0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fe01567e0 0x7f5fe0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46684/0 (socket says 172.21.15.59:46684) 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 -- 172.21.15.59:0/2825824437 learned_addr learned my addr 172.21.15.59:0/2825824437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdf400640 1 --2- 172.21.15.59:0/2825824437 >> v2:172.21.15.66:3300/0 conn(0x7f5fe00d0d10 0x7f5fe0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 -- 172.21.15.59:0/2825824437 >> v2:172.21.15.66:3300/0 conn(0x7f5fe00d0d10 msgr2=0x7f5fe0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 --2- 172.21.15.59:0/2825824437 >> v2:172.21.15.66:3300/0 conn(0x7f5fe00d0d10 0x7f5fe0160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 -- 172.21.15.59:0/2825824437 >> v1:172.21.15.66:6789/0 conn(0x7f5fe015d810 legacy=0x7f5fe00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:54.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.053+0000 7f5fdea00640 1 -- 172.21.15.59:0/2825824437 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe015d010 con 0x7f5fe01567e0 2024-04-24T06:35:54.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.054+0000 7f5fdea00640 1 --2- 172.21.15.59:0/2825824437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe01567e0 0x7f5fe0156bc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00b4cb0 tx=0x7f5fd002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584f24d786bc4df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:54.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.055+0000 7f5fde000640 1 -- 172.21.15.59:0/2825824437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0037070 con 0x7f5fe01567e0 2024-04-24T06:35:54.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.055+0000 7f5fde000640 1 -- 172.21.15.59:0/2825824437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd0032440 con 0x7f5fe01567e0 2024-04-24T06:35:54.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.055+0000 7f5fde000640 1 -- 172.21.15.59:0/2825824437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0032810 con 0x7f5fe01567e0 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.055+0000 7f5fdfe00640 1 -- 172.21.15.59:0/2825824437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe01567e0 msgr2=0x7f5fe0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.055+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/2825824437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe01567e0 0x7f5fe0156bc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00b4cb0 tx=0x7f5fd002a230 comp rx=0 tx=0).stop 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 -- 172.21.15.59:0/2825824437 shutdown_connections 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/2825824437 >> v2:172.21.15.66:3300/0 conn(0x7f5fe00d0d10 0x7f5fe0160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/2825824437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe01567e0 0x7f5fe0156bc0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 -- 172.21.15.59:0/2825824437 >> 172.21.15.59:0/2825824437 conn(0x7f5fe006bdf0 msgr2=0x7f5fe006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 -- 172.21.15.59:0/2825824437 shutdown_connections 2024-04-24T06:35:54.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.056+0000 7f5fdfe00640 1 -- 172.21.15.59:0/2825824437 wait complete. 2024-04-24T06:35:54.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.058+0000 7f5fdfe00640 1 Processor -- start 2024-04-24T06:35:54.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.058+0000 7f5fdfe00640 1 -- start start 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fe01567e0 0x7f5fe0143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe015d810 0x7f5fe0143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01607d0 con 0x7f5fe00d0d10 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0160650 con 0x7f5fe015d810 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe0160950 con 0x7f5fe01567e0 2024-04-24T06:35:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46688/0 (socket says 172.21.15.59:46688) 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 -- 172.21.15.59:0/3976932577 learned_addr learned my addr 172.21.15.59:0/3976932577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fd7e00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fe01567e0 0x7f5fe0143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 -- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fe01567e0 msgr2=0x7f5fe0143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdf400640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe015d810 0x7f5fe0143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fe01567e0 0x7f5fe0143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 -- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe015d810 msgr2=0x7f5fe0143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe015d810 0x7f5fe0143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.059+0000 7f5fdea00640 1 -- 172.21.15.59:0/3976932577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe01e6db0 con 0x7f5fe00d0d10 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fdea00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009960 tx=0x7f5fd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0041020 con 0x7f5fe00d0d10 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd00307a0 con 0x7f5fe00d0d10 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0037040 con 0x7f5fe00d0d10 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe009a280 con 0x7f5fe00d0d10 2024-04-24T06:35:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.060+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe009a840 con 0x7f5fe00d0d10 2024-04-24T06:35:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.062+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe004fd90 con 0x7f5fe00d0d10 2024-04-24T06:35:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.063+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fd0044020 con 0x7f5fe00d0d10 2024-04-24T06:35:54.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.064+0000 7f5fd6a00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 0x7f5fbc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:54.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.064+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7f5fd00ac270 con 0x7f5fe00d0d10 2024-04-24T06:35:54.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.064+0000 7f5fd7e00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 0x7f5fbc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:54.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.065+0000 7f5fd7e00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 0x7f5fbc06ee00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe01559e0 tx=0x7f5fc8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:54.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.069+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fd004b030 con 0x7f5fe00d0d10 2024-04-24T06:35:54.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.317+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5fe00b5150 con 0x7f5fe00d0d10 2024-04-24T06:35:54.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.871+0000 7f5fd6a00640 1 -- 172.21.15.59:0/3976932577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v57) ==== 174+0+0 (secure 0 0 0) 0x7f5fe01e6db0 con 0x7f5fe00d0d10 2024-04-24T06:35:54.877 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 12 and data pool 13 2024-04-24T06:35:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.883+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 msgr2=0x7f5fbc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.883+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 0x7f5fbc06ee00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5fe01559e0 tx=0x7f5fc8009040 comp rx=0 tx=0).stop 2024-04-24T06:35:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.883+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 msgr2=0x7f5fe01e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.883+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009960 tx=0x7f5fd00046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 shutdown_connections 2024-04-24T06:35:54.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5fbc06c940 0x7f5fbc06ee00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fe015d810 0x7f5fe0143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5fe01567e0 0x7f5fe0143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 --2- 172.21.15.59:0/3976932577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe00d0d10 0x7f5fe01e6870 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:54.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.886+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 >> 172.21.15.59:0/3976932577 conn(0x7f5fe006bdf0 msgr2=0x7f5fe0072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:54.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.888+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 shutdown_connections 2024-04-24T06:35:54.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:54.888+0000 7f5fdfe00640 1 -- 172.21.15.59:0/3976932577 wait complete. 2024-04-24T06:35:54.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.101+0000 7f8f6ae00640 1 Processor -- start 2024-04-24T06:35:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.102+0000 7f8f6ae00640 1 -- start start 2024-04-24T06:35:55.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f6ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f6406d0f0 0x7f8f640b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f6ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6409ec50 0x7f8f6406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f6ae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f64073210 con 0x7f8f6415c980 2024-04-24T06:35:55.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f6ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6415cd60 con 0x7f8f6406d0f0 2024-04-24T06:35:55.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f6ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f64160fb0 con 0x7f8f6409ec50 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f63400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f6406d0f0 0x7f8f640b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f63400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f6406d0f0 0x7f8f640b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46690/0 (socket says 172.21.15.59:46690) 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.103+0000 7f8f63400640 1 -- 172.21.15.59:0/3581985770 learned_addr learned my addr 172.21.15.59:0/3581985770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.104+0000 7f8f68e00640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6409ec50 0x7f8f6406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.104+0000 7f8f63400640 1 -- 172.21.15.59:0/3581985770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6409ec50 msgr2=0x7f8f6406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.104+0000 7f8f63400640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6409ec50 0x7f8f6406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.104+0000 7f8f63400640 1 -- 172.21.15.59:0/3581985770 >> v1:172.21.15.66:6789/0 conn(0x7f8f6415c980 legacy=0x7f8f6406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.104+0000 7f8f63400640 1 -- 172.21.15.59:0/3581985770 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640734b0 con 0x7f8f6406d0f0 2024-04-24T06:35:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.105+0000 7f8f63400640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406d0f0 0x7f8f640b00b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009870 tx=0x7f8f4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c80666211826f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.105+0000 7f8f62a00640 1 -- 172.21.15.59:0/3581985770 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c037070 con 0x7f8f6406d0f0 2024-04-24T06:35:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.105+0000 7f8f62a00640 1 -- 172.21.15.59:0/3581985770 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c032440 con 0x7f8f6406d0f0 2024-04-24T06:35:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.105+0000 7f8f62a00640 1 -- 172.21.15.59:0/3581985770 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c032810 con 0x7f8f6406d0f0 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 -- 172.21.15.59:0/3581985770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406d0f0 msgr2=0x7f8f640b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406d0f0 0x7f8f640b00b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009870 tx=0x7f8f4c02a210 comp rx=0 tx=0).stop 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 -- 172.21.15.59:0/3581985770 shutdown_connections 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6409ec50 0x7f8f6406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/3581985770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406d0f0 0x7f8f640b00b0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 -- 172.21.15.59:0/3581985770 >> 172.21.15.59:0/3581985770 conn(0x7f8f64072650 msgr2=0x7f8f64072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 -- 172.21.15.59:0/3581985770 shutdown_connections 2024-04-24T06:35:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.106+0000 7f8f6ae00640 1 -- 172.21.15.59:0/3581985770 wait complete. 2024-04-24T06:35:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.108+0000 7f8f6ae00640 1 Processor -- start 2024-04-24T06:35:55.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.108+0000 7f8f6ae00640 1 -- start start 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6406d0f0 0x7f8f641598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6415c980 0x7f8f640cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6406ef80 con 0x7f8f6409ec50 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64073950 con 0x7f8f6415c980 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f6ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6406f100 con 0x7f8f6406d0f0 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f63400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f63400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46704/0 (socket says 172.21.15.59:46704) 2024-04-24T06:35:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.109+0000 7f8f63400640 1 -- 172.21.15.59:0/703652108 learned_addr learned my addr 172.21.15.59:0/703652108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63e00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6406d0f0 0x7f8f641598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f68e00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6415c980 0x7f8f640cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 -- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6406d0f0 msgr2=0x7f8f641598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6406d0f0 0x7f8f641598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 -- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6415c980 msgr2=0x7f8f640cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6415c980 0x7f8f640cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 -- 172.21.15.59:0/703652108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f64145170 con 0x7f8f6409ec50 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.110+0000 7f8f63400640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c000c00 tx=0x7f8f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.111+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c041020 con 0x7f8f6409ec50 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.111+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0306c0 con 0x7f8f6409ec50 2024-04-24T06:35:55.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.111+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6415bd00 con 0x7f8f6409ec50 2024-04-24T06:35:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.111+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c037040 con 0x7f8f6409ec50 2024-04-24T06:35:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.111+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641f6370 con 0x7f8f6409ec50 2024-04-24T06:35:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.112+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f641f6e30 con 0x7f8f6409ec50 2024-04-24T06:35:55.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.113+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f4c044020 con 0x7f8f6409ec50 2024-04-24T06:35:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.114+0000 7f8f5be00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 0x7f8f4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.115+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8f4c0ac360 con 0x7f8f6409ec50 2024-04-24T06:35:55.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.115+0000 7f8f63e00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 0x7f8f4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.115+0000 7f8f63e00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 0x7f8f4006ede0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009f20 tx=0x7f8f540094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.119+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f4c04b030 con 0x7f8f6409ec50 2024-04-24T06:35:55.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.325+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f64068a80 con 0x7f8f6409ec50 2024-04-24T06:35:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.327+0000 7f8f5be00640 1 -- 172.21.15.59:0/703652108 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16589 (secure 0 0 0) 0x7f8f4c07a780 con 0x7f8f6409ec50 2024-04-24T06:35:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:55.329 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:54.320228+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 msgr2=0x7f8f4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 0x7f8f4006ede0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009f20 tx=0x7f8f540094d0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 msgr2=0x7f8f64159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c000c00 tx=0x7f8f4c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 shutdown_connections 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f4006c920 0x7f8f4006ede0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f6415c980 0x7f8f640cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6409ec50 0x7f8f64159df0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 --2- 172.21.15.59:0/703652108 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f6406d0f0 0x7f8f641598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 >> 172.21.15.59:0/703652108 conn(0x7f8f64072650 msgr2=0x7f8f6406c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 shutdown_connections 2024-04-24T06:35:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.330+0000 7f8f6ae00640 1 -- 172.21.15.59:0/703652108 wait complete. 2024-04-24T06:35:55.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.561+0000 7f4d71c00640 1 Processor -- start 2024-04-24T06:35:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.561+0000 7f4d71c00640 1 -- start start 2024-04-24T06:35:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.563+0000 7f4d71c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4d6c1454e0 0x7f4d6c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.563+0000 7f4d71c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d6c160910 0x7f4d6c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.563+0000 7f4d71c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d6c0cd520 con 0x7f4d6c0cd020 2024-04-24T06:35:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.563+0000 7f4d71c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c0cddc0 con 0x7f4d6c160910 2024-04-24T06:35:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.563+0000 7f4d71c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c0cd800 con 0x7f4d6c1454e0 2024-04-24T06:35:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4d6c0cd020 0x7f4d6c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36250/0 (socket says 172.21.15.59:36250) 2024-04-24T06:35:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6a000640 1 -- 172.21.15.59:0/2594871174 learned_addr learned my addr 172.21.15.59:0/2594871174 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6b400640 1 --2- 172.21.15.59:0/2594871174 >> v2:172.21.15.59:3300/0 conn(0x7f4d6c160910 0x7f4d6c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6aa00640 1 --2- 172.21.15.59:0/2594871174 >> v2:172.21.15.66:3300/0 conn(0x7f4d6c1454e0 0x7f4d6c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d69600640 1 -- 172.21.15.59:0/2594871174 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948135292 0 0) 0x7f4d6c0cd520 con 0x7f4d6c0cd020 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d69600640 1 -- 172.21.15.59:0/2594871174 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d50003520 con 0x7f4d6c0cd020 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6b400640 1 -- 172.21.15.59:0/2594871174 >> v2:172.21.15.66:3300/0 conn(0x7f4d6c1454e0 msgr2=0x7f4d6c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6b400640 1 --2- 172.21.15.59:0/2594871174 >> v2:172.21.15.66:3300/0 conn(0x7f4d6c1454e0 0x7f4d6c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.564+0000 7f4d6b400640 1 -- 172.21.15.59:0/2594871174 >> v1:172.21.15.59:6789/0 conn(0x7f4d6c0cd020 legacy=0x7f4d6c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.565+0000 7f4d6b400640 1 -- 172.21.15.59:0/2594871174 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c0cdae0 con 0x7f4d6c160910 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.566+0000 7f4d6b400640 1 --2- 172.21.15.59:0/2594871174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c160910 0x7f4d6c160d80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c073280 tx=0x7f4d6402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6247ec2536a2dec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.566+0000 7f4d69600640 1 -- 172.21.15.59:0/2594871174 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64007740 con 0x7f4d6c160910 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.566+0000 7f4d69600640 1 -- 172.21.15.59:0/2594871174 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d640078e0 con 0x7f4d6c160910 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.566+0000 7f4d69600640 1 -- 172.21.15.59:0/2594871174 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64007c90 con 0x7f4d6c160910 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 -- 172.21.15.59:0/2594871174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c160910 msgr2=0x7f4d6c160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 --2- 172.21.15.59:0/2594871174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c160910 0x7f4d6c160d80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c073280 tx=0x7f4d6402a5f0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 -- 172.21.15.59:0/2594871174 shutdown_connections 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 --2- 172.21.15.59:0/2594871174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c160910 0x7f4d6c160d80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 --2- 172.21.15.59:0/2594871174 >> v2:172.21.15.66:3300/0 conn(0x7f4d6c1454e0 0x7f4d6c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 -- 172.21.15.59:0/2594871174 >> 172.21.15.59:0/2594871174 conn(0x7f4d6c067ea0 msgr2=0x7f4d6c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 -- 172.21.15.59:0/2594871174 shutdown_connections 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.567+0000 7f4d71c00640 1 -- 172.21.15.59:0/2594871174 wait complete. 2024-04-24T06:35:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.569+0000 7f4d71c00640 1 Processor -- start 2024-04-24T06:35:55.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.569+0000 7f4d71c00640 1 -- start start 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c1454e0 0x7f4d6c15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d6c160910 0x7f4d6c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c0b4010 con 0x7f4d6c0cd020 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1460f0 con 0x7f4d6c1454e0 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d71c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6c06dde0 con 0x7f4d6c160910 2024-04-24T06:35:55.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46712/0 (socket says 172.21.15.59:46712) 2024-04-24T06:35:55.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.570+0000 7f4d6aa00640 1 -- 172.21.15.59:0/4093962004 learned_addr learned my addr 172.21.15.59:0/4093962004 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:55.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 -- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d6c160910 msgr2=0x7f4d6c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6a000640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c1454e0 0x7f4d6c15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6b400640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d6c160910 0x7f4d6c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d6c160910 0x7f4d6c15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 -- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c1454e0 msgr2=0x7f4d6c15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c1454e0 0x7f4d6c15ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 -- 172.21.15.59:0/4093962004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c06e030 con 0x7f4d6c0cd020 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d6aa00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00e9e0 tx=0x7f4d5c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c017070 con 0x7f4d6c0cd020 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00c9d0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c06e310 con 0x7f4d6c0cd020 2024-04-24T06:35:55.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.571+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00cdc0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.572+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c06e8d0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.573+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c068a80 con 0x7f4d6c0cd020 2024-04-24T06:35:55.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.574+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d5c01a070 con 0x7f4d6c0cd020 2024-04-24T06:35:55.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.575+0000 7f4d62a00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 0x7f4d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.575+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f4d5c08bfe0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.575+0000 7f4d6a000640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 0x7f4d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:55.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.576+0000 7f4d6a000640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 0x7f4d4806edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c144c60 tx=0x7f4d54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:55.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.580+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d5c05a3b0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.783+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d6c160030 con 0x7f4d6c0cd020 2024-04-24T06:35:55.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.786+0000 7f4d62a00640 1 -- 172.21.15.59:0/4093962004 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16589 (secure 0 0 0) 0x7f4d5c059cb0 con 0x7f4d6c0cd020 2024-04-24T06:35:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:55.789 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:54.320228+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:55.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 msgr2=0x7f4d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 0x7f4d4806edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c144c60 tx=0x7f4d54009040 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 msgr2=0x7f4d6c0aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00e9e0 tx=0x7f4d5c00eea0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 shutdown_connections 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d4806c8f0 0x7f4d4806edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d6c160910 0x7f4d6c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d6c1454e0 0x7f4d6c15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 --2- 172.21.15.59:0/4093962004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c0cd020 0x7f4d6c0aeb30 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 >> 172.21.15.59:0/4093962004 conn(0x7f4d6c067ea0 msgr2=0x7f4d6c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 shutdown_connections 2024-04-24T06:35:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:55.789+0000 7f4d71c00640 1 -- 172.21.15.59:0/4093962004 wait complete. 2024-04-24T06:35:55.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:35:56.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.022+0000 7f5a6ec00640 1 Processor -- start 2024-04-24T06:35:56.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.022+0000 7f5a6ec00640 1 -- start start 2024-04-24T06:35:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.023+0000 7f5a6ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5a681567e0 0x7f5a68156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.023+0000 7f5a6ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a680d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.023+0000 7f5a6ec00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6815cab0 con 0x7f5a680d0b40 2024-04-24T06:35:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.023+0000 7f5a6ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5a680d0f70 con 0x7f5a681567e0 2024-04-24T06:35:56.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.023+0000 7f5a6ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a68160e40 con 0x7f5a6815d810 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a6e200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5a680d0b40 0x7f5a68160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36266/0 (socket says 172.21.15.59:36266) 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a6e200640 1 -- 172.21.15.59:0/3999653896 learned_addr learned my addr 172.21.15.59:0/3999653896 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3999653896 >> v2:172.21.15.66:3300/0 conn(0x7f5a681567e0 0x7f5a68156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a680d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690946124 0 0) 0x7f5a60001110 con 0x7f5a680d0b40 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.024+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a50003520 con 0x7f5a680d0b40 2024-04-24T06:35:56.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169006015 0 0) 0x7f5a50003520 con 0x7f5a680d0b40 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 msgr2=0x7f5a680d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a680d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 >> v2:172.21.15.66:3300/0 conn(0x7f5a681567e0 msgr2=0x7f5a68156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 --2- 172.21.15.59:0/3999653896 >> v2:172.21.15.66:3300/0 conn(0x7f5a681567e0 0x7f5a68156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6815cd30 con 0x7f5a680d0b40 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5a60001840 con 0x7f5a680d0b40 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 >> v1:172.21.15.59:6789/0 conn(0x7f5a680d0b40 legacy=0x7f5a68160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a50004800 0x7f5a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a680d0f70 con 0x7f5a500039b0 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68160e40 con 0x7f5a50004800 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5000b9c0 con 0x7f5a6815d810 2024-04-24T06:35:56.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.025+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6e200640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a50004800 0x7f5a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6d800640 1 -- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 msgr2=0x7f5a500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:56.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6d800640 1 -- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a50004800 msgr2=0x7f5a50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a50004800 0x7f5a50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6d800640 1 -- 172.21.15.59:0/3999653896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a50003520 con 0x7f5a500039b0 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6e200640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6815d810 0x7f5a500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.026+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a50004800 0x7f5a50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.027+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a50003dd0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009fc0 tx=0x7f5a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06ccb9a8ba59be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c009c10 con 0x7f5a500039b0 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c036070 con 0x7f5a500039b0 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a67e00640 1 -- 172.21.15.59:0/3999653896 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c02d070 con 0x7f5a500039b0 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3999653896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 msgr2=0x7f5a50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3999653896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a50003dd0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009fc0 tx=0x7f5a50005830 comp rx=0 tx=0).stop 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6e200640 1 -- 172.21.15.59:0/3999653896 reap_dead start 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3999653896 shutdown_connections 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3999653896 >> 172.21.15.59:0/3999653896 conn(0x7f5a6806bdf0 msgr2=0x7f5a6806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3999653896 shutdown_connections 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.028+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3999653896 wait complete. 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.030+0000 7f5a6ec00640 1 Processor -- start 2024-04-24T06:35:56.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.030+0000 7f5a6ec00640 1 -- start start 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6814db20 0x7f5a681f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 0x7f5a68153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a680d0600 con 0x7f5a500039b0 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6815d2f0 con 0x7f5a681f4500 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a680d0780 con 0x7f5a6814db20 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46734/0 (socket says 172.21.15.59:46734) 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.031+0000 7f5a6d800640 1 -- 172.21.15.59:0/3252582346 learned_addr learned my addr 172.21.15.59:0/3252582346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6e200640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 0x7f5a68153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6814db20 0x7f5a681f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 -- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6814db20 msgr2=0x7f5a681f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6814db20 0x7f5a681f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 -- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 msgr2=0x7f5a68153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 0x7f5a68153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 -- 172.21.15.59:0/3252582346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a68154050 con 0x7f5a500039b0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6e200640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 0x7f5a68153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.032+0000 7f5a6d800640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c02af10 tx=0x7f5a5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.033+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c009c10 con 0x7f5a500039b0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.033+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c003bb0 con 0x7f5a500039b0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.033+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c004020 con 0x7f5a500039b0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.033+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a68154330 con 0x7f5a500039b0 2024-04-24T06:35:56.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.033+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a68151b50 con 0x7f5a500039b0 2024-04-24T06:35:56.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.034+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a680b6410 con 0x7f5a500039b0 2024-04-24T06:35:56.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.035+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a5c005d10 con 0x7f5a500039b0 2024-04-24T06:35:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.036+0000 7f5a65600640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 0x7f5a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.036+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f5a5c0acfb0 con 0x7f5a500039b0 2024-04-24T06:35:56.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.036+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 0x7f5a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:56.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.037+0000 7f5a6ce00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 0x7f5a4806ede0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5a6814e4c0 tx=0x7f5a54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.041+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a680b6410 con 0x7f5a500039b0 2024-04-24T06:35:56.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.290+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5a680b5150 con 0x7f5a500039b0 2024-04-24T06:35:56.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.886+0000 7f5a65600640 1 -- 172.21.15.59:0/3252582346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f5a5c07b3d0 con 0x7f5a500039b0 2024-04-24T06:35:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 msgr2=0x7f5a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 0x7f5a4806ede0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5a6814e4c0 tx=0x7f5a54008040 comp rx=0 tx=0).stop 2024-04-24T06:35:56.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 msgr2=0x7f5a68151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c02af10 tx=0x7f5a5c009750 comp rx=0 tx=0).stop 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 shutdown_connections 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a4806c920 0x7f5a4806ede0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a681f4500 0x7f5a68153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a6814db20 0x7f5a681f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 --2- 172.21.15.59:0/3252582346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500039b0 0x7f5a68151610 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.891+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 >> 172.21.15.59:0/3252582346 conn(0x7f5a6806bdf0 msgr2=0x7f5a6815dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.892+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 shutdown_connections 2024-04-24T06:35:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:56.892+0000 7f5a6ec00640 1 -- 172.21.15.59:0/3252582346 wait complete. 2024-04-24T06:35:56.917 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:35:56.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:35:57.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.136+0000 7f21d1400640 1 Processor -- start 2024-04-24T06:35:57.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.136+0000 7f21d1400640 1 -- start start 2024-04-24T06:35:57.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.137+0000 7f21d1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21cc15e9d0 0x7f21cc15f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.137+0000 7f21d1400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21cc0d42c0 0x7f21cc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.137+0000 7f21d1400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc15edb0 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21d1400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc0d3e30 con 0x7f21cc15e9d0 2024-04-24T06:35:57.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21d1400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc15ff20 con 0x7f21cc0d42c0 2024-04-24T06:35:57.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21ca800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21cc15e9d0 0x7f21cc15f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21ca800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21cc15e9d0 0x7f21cc15f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46746/0 (socket says 172.21.15.59:46746) 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21ca800640 1 -- 172.21.15.59:0/3035082674 learned_addr learned my addr 172.21.15.59:0/3035082674 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.138+0000 7f21c9e00640 1 --2- 172.21.15.59:0/3035082674 >> v2:172.21.15.66:3300/0 conn(0x7f21cc0d42c0 0x7f21cc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.139+0000 7f21ca800640 1 -- 172.21.15.59:0/3035082674 >> v2:172.21.15.66:3300/0 conn(0x7f21cc0d42c0 msgr2=0x7f21cc0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.139+0000 7f21ca800640 1 --2- 172.21.15.59:0/3035082674 >> v2:172.21.15.66:3300/0 conn(0x7f21cc0d42c0 0x7f21cc0d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.139+0000 7f21ca800640 1 -- 172.21.15.59:0/3035082674 >> v1:172.21.15.66:6789/0 conn(0x7f21cc0ba0e0 legacy=0x7f21cc0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.139+0000 7f21ca800640 1 -- 172.21.15.59:0/3035082674 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0ba630 con 0x7f21cc15e9d0 2024-04-24T06:35:57.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.140+0000 7f21ca800640 1 --2- 172.21.15.59:0/3035082674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc15e9d0 0x7f21cc15f9e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0d3fb0 tx=0x7f21bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88f8fd7701e3902 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:57.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.140+0000 7f21c9400640 1 -- 172.21.15.59:0/3035082674 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037070 con 0x7f21cc15e9d0 2024-04-24T06:35:57.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.140+0000 7f21c9400640 1 -- 172.21.15.59:0/3035082674 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc032440 con 0x7f21cc15e9d0 2024-04-24T06:35:57.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.140+0000 7f21c9400640 1 -- 172.21.15.59:0/3035082674 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc032810 con 0x7f21cc15e9d0 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 -- 172.21.15.59:0/3035082674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc15e9d0 msgr2=0x7f21cc15f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 --2- 172.21.15.59:0/3035082674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc15e9d0 0x7f21cc15f9e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0d3fb0 tx=0x7f21bc02a180 comp rx=0 tx=0).stop 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 -- 172.21.15.59:0/3035082674 shutdown_connections 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 --2- 172.21.15.59:0/3035082674 >> v2:172.21.15.66:3300/0 conn(0x7f21cc0d42c0 0x7f21cc0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 --2- 172.21.15.59:0/3035082674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc15e9d0 0x7f21cc15f9e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 -- 172.21.15.59:0/3035082674 >> 172.21.15.59:0/3035082674 conn(0x7f21cc06cbd0 msgr2=0x7f21cc06cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 -- 172.21.15.59:0/3035082674 shutdown_connections 2024-04-24T06:35:57.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.141+0000 7f21d1400640 1 -- 172.21.15.59:0/3035082674 wait complete. 2024-04-24T06:35:57.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.143+0000 7f21d1400640 1 Processor -- start 2024-04-24T06:35:57.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.143+0000 7f21d1400640 1 -- start start 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21cc0d42c0 0x7f21cc14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21cc15e9d0 0x7f21cc0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0cfd70 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0cfa90 con 0x7f21cc0d42c0 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21d1400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0d14a0 con 0x7f21cc15e9d0 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21ca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21ca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46762/0 (socket says 172.21.15.59:46762) 2024-04-24T06:35:57.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.145+0000 7f21ca800640 1 -- 172.21.15.59:0/521196566 learned_addr learned my addr 172.21.15.59:0/521196566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:57.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 -- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21cc15e9d0 msgr2=0x7f21cc0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:57.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21cc15e9d0 0x7f21cc0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 -- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21cc0d42c0 msgr2=0x7f21cc14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:57.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21cc0d42c0 0x7f21cc14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 -- 172.21.15.59:0/521196566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0b5930 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21ca800640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f21bc02a1c0 tx=0x7f21bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037040 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc042070 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc0b5c10 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.146+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc030bd0 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.147+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc0b61d0 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.148+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc069190 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.149+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21bc047460 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.150+0000 7f21c2a00640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 0x7f21a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.150+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f21bc0ac890 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.151+0000 7f21c9e00640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 0x7f21a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:57.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.151+0000 7f21c9e00640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 0x7f21a806ede0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:57.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.155+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21bc07ac20 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.403+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f21cc06bc00 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.897+0000 7f21c2a00640 1 -- 172.21.15.59:0/521196566 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f21bc07a520 con 0x7f21cc0ba0e0 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 msgr2=0x7f21a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 0x7f21a806ede0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006fd0 tx=0x7f21b4008040 comp rx=0 tx=0).stop 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 msgr2=0x7f21cc0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f21bc02a1c0 tx=0x7f21bc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 shutdown_connections 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f21a806c920 0x7f21a806ede0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21cc15e9d0 0x7f21cc0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21cc0d42c0 0x7f21cc14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 --2- 172.21.15.59:0/521196566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21cc0ba0e0 0x7f21cc0c8ec0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 >> 172.21.15.59:0/521196566 conn(0x7f21cc06cbd0 msgr2=0x7f21cc0d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 shutdown_connections 2024-04-24T06:35:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:57.901+0000 7f21d1400640 1 -- 172.21.15.59:0/521196566 wait complete. 2024-04-24T06:35:57.925 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:35:57.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:35:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.138+0000 7fcf72e00640 1 Processor -- start 2024-04-24T06:35:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.138+0000 7fcf72e00640 1 -- start start 2024-04-24T06:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.139+0000 7fcf72e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf6c15e7c0 0x7fcf6c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.139+0000 7fcf72e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf6c0d2630 0x7fcf6c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.139+0000 7fcf72e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6c0cf030 con 0x7fcf6c0b6530 2024-04-24T06:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.139+0000 7fcf72e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6c0b7000 con 0x7fcf6c0d2630 2024-04-24T06:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.139+0000 7fcf72e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6c0cf620 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.140+0000 7fcf71000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf6c0d2630 0x7fcf6c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.140+0000 7fcf72400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcf6c0b6530 0x7fcf6c0d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36280/0 (socket says 172.21.15.59:36280) 2024-04-24T06:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.140+0000 7fcf72400640 1 -- 172.21.15.59:0/3217195213 learned_addr learned my addr 172.21.15.59:0/3217195213 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.140+0000 7fcf71a00640 1 --2- 172.21.15.59:0/3217195213 >> v2:172.21.15.66:3300/0 conn(0x7fcf6c15e7c0 0x7fcf6c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf6be00640 1 -- 172.21.15.59:0/3217195213 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129624202 0 0) 0x7fcf64001490 con 0x7fcf6c0b6530 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf6be00640 1 -- 172.21.15.59:0/3217195213 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf54003520 con 0x7fcf6c0b6530 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf71000640 1 -- 172.21.15.59:0/3217195213 >> v2:172.21.15.66:3300/0 conn(0x7fcf6c15e7c0 msgr2=0x7fcf6c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf71000640 1 --2- 172.21.15.59:0/3217195213 >> v2:172.21.15.66:3300/0 conn(0x7fcf6c15e7c0 0x7fcf6c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf71000640 1 -- 172.21.15.59:0/3217195213 >> v1:172.21.15.59:6789/0 conn(0x7fcf6c0b6530 legacy=0x7fcf6c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.141+0000 7fcf71000640 1 -- 172.21.15.59:0/3217195213 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c0b6a40 con 0x7fcf6c0d2630 2024-04-24T06:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.142+0000 7fcf71000640 1 --2- 172.21.15.59:0/3217195213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0b5ff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c08c6d0 tx=0x7fcf5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cce8a9d64a3ff69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.143+0000 7fcf6be00640 1 -- 172.21.15.59:0/3217195213 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58037070 con 0x7fcf6c0d2630 2024-04-24T06:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.143+0000 7fcf6be00640 1 -- 172.21.15.59:0/3217195213 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf58032440 con 0x7fcf6c0d2630 2024-04-24T06:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.143+0000 7fcf6be00640 1 -- 172.21.15.59:0/3217195213 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58032810 con 0x7fcf6c0d2630 2024-04-24T06:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.143+0000 7fcf72e00640 1 -- 172.21.15.59:0/3217195213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c0d2630 msgr2=0x7fcf6c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.143+0000 7fcf72e00640 1 --2- 172.21.15.59:0/3217195213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0b5ff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c08c6d0 tx=0x7fcf5802a1a0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 -- 172.21.15.59:0/3217195213 shutdown_connections 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 --2- 172.21.15.59:0/3217195213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0b5ff0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 --2- 172.21.15.59:0/3217195213 >> v2:172.21.15.66:3300/0 conn(0x7fcf6c15e7c0 0x7fcf6c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 -- 172.21.15.59:0/3217195213 >> 172.21.15.59:0/3217195213 conn(0x7fcf6c09d920 msgr2=0x7fcf6c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 -- 172.21.15.59:0/3217195213 shutdown_connections 2024-04-24T06:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.144+0000 7fcf72e00640 1 -- 172.21.15.59:0/3217195213 wait complete. 2024-04-24T06:35:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.145+0000 7fcf72e00640 1 Processor -- start 2024-04-24T06:35:58.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.146+0000 7fcf72e00640 1 -- start start 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf6c0b6530 0x7fcf6c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c0d1a80 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c0cf2b0 con 0x7fcf6c0d2630 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c0d1c00 con 0x7fcf6c0b6530 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46786/0 (socket says 172.21.15.59:46786) 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 -- 172.21.15.59:0/1990997222 learned_addr learned my addr 172.21.15.59:0/1990997222 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf71a00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf6c0b6530 0x7fcf6c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf71000640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 -- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf6c0b6530 msgr2=0x7fcf6c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf6c0b6530 0x7fcf6c15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 -- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf6c0d2630 msgr2=0x7fcf6c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.147+0000 7fcf72400640 1 -- 172.21.15.59:0/1990997222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c15a820 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf72400640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400cb10 tx=0x7fcf6400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6400ac70 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64012640 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c15aac0 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.148+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c15b080 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.149+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6c04fd90 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.150+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf6401a070 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.151+0000 7fcf69600640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 0x7fcf4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.151+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fcf6408b9e0 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.152+0000 7fcf71a00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 0x7fcf4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.152+0000 7fcf71a00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 0x7fcf4c06edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009a30 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.156+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf6c0cf030 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.405+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf6c0d1eb0 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.406+0000 7fcf69600640 1 -- 172.21.15.59:0/1990997222 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5177 (secure 0 0 0) 0x7fcf6400ae20 con 0x7fcf6c15e7c0 2024-04-24T06:35:58.408 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:58.408 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7548,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:35:57.406140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6833/301885470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":301885470},{"type":"v1","addr":"172.21.15.66:6833","nonce":301885470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7563,"qdb_cluster":[7563]},"id":6}]} 2024-04-24T06:35:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 61 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 msgr2=0x7fcf4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 0x7fcf4c06edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009a30 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 msgr2=0x7fcf6c0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400cb10 tx=0x7fcf6400a060 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 shutdown_connections 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf4c06c8f0 0x7fcf4c06edb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf6c15e7c0 0x7fcf6c0cbb30 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf6c0d2630 0x7fcf6c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 --2- 172.21.15.59:0/1990997222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf6c0b6530 0x7fcf6c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 >> 172.21.15.59:0/1990997222 conn(0x7fcf6c09d920 msgr2=0x7fcf6c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 shutdown_connections 2024-04-24T06:35:58.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.409+0000 7fcf72e00640 1 -- 172.21.15.59:0/1990997222 wait complete. 2024-04-24T06:35:58.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:35:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.642+0000 7f162d800640 1 Processor -- start 2024-04-24T06:35:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.642+0000 7f162d800640 1 -- start start 2024-04-24T06:35:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f162d800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f16280b5ff0 0x7f16280b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f162d800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f16280d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f162d800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16280cf030 con 0x7f162815e7c0 2024-04-24T06:35:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f162d800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f16280b7000 con 0x7f16280b5ff0 2024-04-24T06:35:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f162d800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f16280cf620 con 0x7f16280d0d90 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f1627e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f162815e7c0 0x7f16280d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36290/0 (socket says 172.21.15.59:36290) 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.643+0000 7f1627e00640 1 -- 172.21.15.59:0/2748307590 learned_addr learned my addr 172.21.15.59:0/2748307590 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f162ce00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f16280d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1627400640 1 --2- 172.21.15.59:0/2748307590 >> v2:172.21.15.66:3300/0 conn(0x7f16280b5ff0 0x7f16280b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248829989 0 0) 0x7f1618001110 con 0x7f162815e7c0 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f160c003520 con 0x7f162815e7c0 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3326875979 0 0) 0x7f160c003520 con 0x7f162815e7c0 2024-04-24T06:35:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 msgr2=0x7f16280d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f16280d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 >> v2:172.21.15.66:3300/0 conn(0x7f16280b5ff0 msgr2=0x7f16280b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 --2- 172.21.15.59:0/2748307590 >> v2:172.21.15.66:3300/0 conn(0x7f16280b5ff0 0x7f16280b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16280b6a40 con 0x7f162815e7c0 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1618001840 con 0x7f162815e7c0 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 >> v1:172.21.15.59:6789/0 conn(0x7f162815e7c0 legacy=0x7f16280d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f160c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c003f30 0x7f160c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f160c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16280b7000 con 0x7f160c004d40 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16280cf620 con 0x7f160c003f30 2024-04-24T06:35:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f160c00ba10 con 0x7f16280d0d90 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.644+0000 7f1627400640 1 --2- 172.21.15.59:0/2748307590 >> v2:172.21.15.66:3300/0 conn(0x7f16280b5ff0 0x7f16280b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f160c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f1627e00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f160c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 -- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 msgr2=0x7f160c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f16280d0d90 0x7f160c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 -- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c003f30 msgr2=0x7f160c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c003f30 0x7f160c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.645+0000 7f162ce00640 1 -- 172.21.15.59:0/2748307590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c003520 con 0x7f160c004d40 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.646+0000 7f162ce00640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f160c0051c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1620009ac0 tx=0x7f160c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e652e4337ec00fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.646+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620036070 con 0x7f160c004d40 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620031400 con 0x7f160c004d40 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f1626a00640 1 -- 172.21.15.59:0/2748307590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620009690 con 0x7f160c004d40 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 -- 172.21.15.59:0/2748307590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 msgr2=0x7f160c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 --2- 172.21.15.59:0/2748307590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f160c0051c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1620009ac0 tx=0x7f160c005830 comp rx=0 tx=0).stop 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162ce00640 1 -- 172.21.15.59:0/2748307590 reap_dead start 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 -- 172.21.15.59:0/2748307590 shutdown_connections 2024-04-24T06:35:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 -- 172.21.15.59:0/2748307590 >> 172.21.15.59:0/2748307590 conn(0x7f162809d920 msgr2=0x7f162808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:58.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 -- 172.21.15.59:0/2748307590 shutdown_connections 2024-04-24T06:35:58.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.647+0000 7f162d800640 1 -- 172.21.15.59:0/2748307590 wait complete. 2024-04-24T06:35:58.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.649+0000 7f162d800640 1 Processor -- start 2024-04-24T06:35:58.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.649+0000 7f162d800640 1 -- start start 2024-04-24T06:35:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1628153a60 0x7f1628153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1628155580 0x7f1628155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16280d2800 con 0x7f160c004d40 2024-04-24T06:35:58.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16280cf2b0 con 0x7f1628153a60 2024-04-24T06:35:58.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.650+0000 7f162d800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f16280d1a80 con 0x7f1628155580 2024-04-24T06:35:58.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46796/0 (socket says 172.21.15.59:46796) 2024-04-24T06:35:58.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 -- 172.21.15.59:0/2333441149 learned_addr learned my addr 172.21.15.59:0/2333441149 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 -- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1628155580 msgr2=0x7f1628155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1628155580 0x7f1628155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 -- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1628153a60 msgr2=0x7f1628153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1628153a60 0x7f1628153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 -- 172.21.15.59:0/2333441149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16281544e0 con 0x7f160c004d40 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.651+0000 7f1627e00640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f161800d8c0 tx=0x7f161800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.652+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618019070 con 0x7f160c004d40 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.652+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16180074e0 con 0x7f160c004d40 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.652+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004a20 con 0x7f160c004d40 2024-04-24T06:35:58.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.652+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16281547c0 con 0x7f160c004d40 2024-04-24T06:35:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.652+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1628154d80 con 0x7f160c004d40 2024-04-24T06:35:58.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.653+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162804fd90 con 0x7f160c004d40 2024-04-24T06:35:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.654+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f161801c070 con 0x7f160c004d40 2024-04-24T06:35:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.655+0000 7f161fe00640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 0x7f160806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.655+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f161808b640 con 0x7f160c004d40 2024-04-24T06:35:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.656+0000 7f1627400640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 0x7f160806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:58.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.656+0000 7f1627400640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 0x7f160806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1614009540 tx=0x7f1614009470 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:58.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.660+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16280cf030 con 0x7f160c004d40 2024-04-24T06:35:58.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.865+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16280b57b0 con 0x7f160c004d40 2024-04-24T06:35:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.868+0000 7f161fe00640 1 -- 172.21.15.59:0/2333441149 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16576 (secure 0 0 0) 0x7f1618059b00 con 0x7f160c004d40 2024-04-24T06:35:58.870 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:58.871 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:55.872230+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:35:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 msgr2=0x7f160806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 0x7f160806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1614009540 tx=0x7f1614009470 comp rx=0 tx=0).stop 2024-04-24T06:35:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 msgr2=0x7f1628152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f161800d8c0 tx=0x7f161800dd80 comp rx=0 tx=0).stop 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 shutdown_connections 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f160806c8f0 0x7f160806edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1628155580 0x7f1628155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1628153a60 0x7f1628153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 --2- 172.21.15.59:0/2333441149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c004d40 0x7f1628152a30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 >> 172.21.15.59:0/2333441149 conn(0x7f162809d920 msgr2=0x7f16280b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 shutdown_connections 2024-04-24T06:35:58.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:58.871+0000 7f162d800640 1 -- 172.21.15.59:0/2333441149 wait complete. 2024-04-24T06:35:58.893 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:35:58.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:35:59.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.098+0000 7f0e4da00640 1 Processor -- start 2024-04-24T06:35:59.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.098+0000 7f0e4da00640 1 -- start start 2024-04-24T06:35:59.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.100+0000 7f0e4da00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e4806c0e0 0x7f0e480cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.100+0000 7f0e4da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e4815b270 con 0x7f0e4806f350 2024-04-24T06:35:59.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.100+0000 7f0e4da00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e480b4930 con 0x7f0e4815ae90 2024-04-24T06:35:59.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.100+0000 7f0e4da00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0e480b3080 con 0x7f0e4806c0e0 2024-04-24T06:35:59.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e3fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0e4806f350 0x7f0e4806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44384/0 (socket says 172.21.15.59:44384) 2024-04-24T06:35:59.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e3fe00640 1 -- 172.21.15.59:0/2509898447 learned_addr learned my addr 172.21.15.59:0/2509898447 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:35:59.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e4d000640 1 --2- 172.21.15.59:0/2509898447 >> v2:172.21.15.66:3300/0 conn(0x7f0e4806c0e0 0x7f0e480cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3220114775 0 0) 0x7f0e4815b270 con 0x7f0e4806f350 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e2c003520 con 0x7f0e4806f350 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 861324541 0 0) 0x7f0e2c003520 con 0x7f0e4806f350 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 >> v2:172.21.15.66:3300/0 conn(0x7f0e4806c0e0 msgr2=0x7f0e480cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 --2- 172.21.15.59:0/2509898447 >> v2:172.21.15.66:3300/0 conn(0x7f0e4806c0e0 0x7f0e480cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 >> v1:172.21.15.66:6789/0 conn(0x7f0e4815ae90 legacy=0x7f0e48071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480b4c10 con 0x7f0e4806f350 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.101+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0e30001670 con 0x7f0e4806f350 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 >> v1:172.21.15.59:6789/0 conn(0x7f0e4806f350 legacy=0x7f0e4806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e2c004800 0x7f0e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480b3080 con 0x7f0e2c0051c0 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c00c060 con 0x7f0e2c0039b0 2024-04-24T06:35:59.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2c00c260 con 0x7f0e2c004800 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47e00640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 -- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e2c004800 msgr2=0x7f0e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e2c004800 0x7f0e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 -- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e2c0039b0 msgr2=0x7f0e2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e4d000640 1 -- 172.21.15.59:0/2509898447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c003520 con 0x7f0e2c0051c0 2024-04-24T06:35:59.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.102+0000 7f0e47e00640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:59.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.103+0000 7f0e4d000640 1 -- 172.21.15.59:0/2509898447 reap_dead start 2024-04-24T06:35:59.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.103+0000 7f0e4d000640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e2c00b9c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009f90 tx=0x7f0e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe29dad6e332813 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40035020 con 0x7f0e2c0051c0 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e40003b90 con 0x7f0e2c0051c0 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e47400640 1 -- 172.21.15.59:0/2509898447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40009c10 con 0x7f0e2c0051c0 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e4da00640 1 -- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 msgr2=0x7f0e2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e4da00640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e2c00b9c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009f90 tx=0x7f0e2c005830 comp rx=0 tx=0).stop 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e4da00640 1 -- 172.21.15.59:0/2509898447 shutdown_connections 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e4da00640 1 --2- 172.21.15.59:0/2509898447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e2c00b9c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.104+0000 7f0e4da00640 1 -- 172.21.15.59:0/2509898447 >> 172.21.15.59:0/2509898447 conn(0x7f0e480b21a0 msgr2=0x7f0e480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.105+0000 7f0e4da00640 1 -- 172.21.15.59:0/2509898447 shutdown_connections 2024-04-24T06:35:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.105+0000 7f0e4da00640 1 -- 172.21.15.59:0/2509898447 wait complete. 2024-04-24T06:35:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.106+0000 7f0e4da00640 1 Processor -- start 2024-04-24T06:35:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.107+0000 7f0e4da00640 1 -- start start 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.107+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.107+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4815dc00 0x7f0e4815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e480d0330 0x7f0e480d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e4da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480b45c0 con 0x7f0e2c0051c0 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e4da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480b32d0 con 0x7f0e4815dc00 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e4da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e480cdd80 con 0x7f0e480d0330 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34976/0 (socket says 172.21.15.59:34976) 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 -- 172.21.15.59:0/978033991 learned_addr learned my addr 172.21.15.59:0/978033991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 -- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e480d0330 msgr2=0x7f0e480d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e480d0330 0x7f0e480d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 -- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4815dc00 msgr2=0x7f0e4815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4815dc00 0x7f0e4815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 -- 172.21.15.59:0/978033991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e481f7b90 con 0x7f0e2c0051c0 2024-04-24T06:35:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.108+0000 7f0e47e00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d7d0 tx=0x7f0e3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.109+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005270 con 0x7f0e2c0051c0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.109+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e38017070 con 0x7f0e2c0051c0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.109+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800b6b0 con 0x7f0e2c0051c0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.109+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e481f7e70 con 0x7f0e2c0051c0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.109+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e481f8430 con 0x7f0e2c0051c0 2024-04-24T06:35:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.110+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4804fd90 con 0x7f0e2c0051c0 2024-04-24T06:35:59.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.111+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e3800b850 con 0x7f0e2c0051c0 2024-04-24T06:35:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.112+0000 7f0e44c00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 0x7f0e2406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.112+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0e3808ba00 con 0x7f0e2c0051c0 2024-04-24T06:35:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.112+0000 7f0e3fe00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 0x7f0e2406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.113+0000 7f0e3fe00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 0x7f0e2406ed80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e30000fd0 tx=0x7f0e30001490 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.116+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e38019310 con 0x7f0e2c0051c0 2024-04-24T06:35:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.321+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e480af760 con 0x7f0e2406c8c0 2024-04-24T06:35:59.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.329+0000 7f0e44c00640 1 -- 172.21.15.59:0/978033991 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240186 (secure 0 0 0) 0x7f0e480af760 con 0x7f0e2406c8c0 2024-04-24T06:35:59.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:59.335 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.334+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 msgr2=0x7f0e2406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.334+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 0x7f0e2406ed80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e30000fd0 tx=0x7f0e30001490 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.334+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 msgr2=0x7f0e481f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.334+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d7d0 tx=0x7f0e3800dc90 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 shutdown_connections 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0e2406c8c0 0x7f0e2406ed80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e480d0330 0x7f0e480d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e4815dc00 0x7f0e4815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 --2- 172.21.15.59:0/978033991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e2c0051c0 0x7f0e481f7590 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 >> 172.21.15.59:0/978033991 conn(0x7f0e480b21a0 msgr2=0x7f0e48072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 shutdown_connections 2024-04-24T06:35:59.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.335+0000 7f0e4da00640 1 -- 172.21.15.59:0/978033991 wait complete. 2024-04-24T06:35:59.356 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-04-24T06:35:57.335957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258232,"kb_used_data":8768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716488,"statfs":{"total":773094113280,"available":772829683712,"internally_reserved":0,"allocated":8978432,"data_stored":4476554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.527169"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.827847+0000","last_change":"2024-04-24T06:35:52.827847+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.827847+0000","last_undegraded":"2024-04-24T06:35:52.827847+0000","last_fullsized":"2024-04-24T06:35:52.827847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.814155+0000","last_change":"2024-04-24T06:35:52.814155+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.814155+0000","last_undegraded":"2024-04-24T06:35:52.814155+0000","last_fullsized":"2024-04-24T06:35:52.814155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.822911+0000","last_change":"2024-04-24T06:35:52.822911+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.822911+0000","last_undegraded":"2024-04-24T06:35:52.822911+0000","last_fullsized":"2024-04-24T06:35:52.822911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.823832+0000","last_change":"2024-04-24T06:35:52.823832+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.823832+0000","last_undegraded":"2024-04-24T06:35:52.823832+0000","last_fullsized":"2024-04-24T06:35:52.823832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.813909+0000","last_change":"2024-04-24T06:35:52.813909+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.813909+0000","last_undegraded":"2024-04-24T06:35:52.813909+0000","last_fullsized":"2024-04-24T06:35:52.813909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.815663+0000","last_change":"2024-04-24T06:35:52.815663+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.815663+0000","last_undegraded":"2024-04-24T06:35:52.815663+0000","last_fullsized":"2024-04-24T06:35:52.815663+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.821645+0000","last_change":"2024-04-24T06:35:52.821645+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.821645+0000","last_undegraded":"2024-04-24T06:35:52.821645+0000","last_fullsized":"2024-04-24T06:35:52.821645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.816602+0000","last_change":"2024-04-24T06:35:52.816602+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.816602+0000","last_undegraded":"2024-04-24T06:35:52.816602+0000","last_fullsized":"2024-04-24T06:35:52.816602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.825296+0000","last_change":"2024-04-24T06:35:52.825296+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.825296+0000","last_undegraded":"2024-04-24T06:35:52.825296+0000","last_fullsized":"2024-04-24T06:35:52.825296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.822927+0000","last_change":"2024-04-24T06:35:52.822927+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.822927+0000","last_undegraded":"2024-04-24T06:35:52.822927+0000","last_fullsized":"2024-04-24T06:35:52.822927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.822639+0000","last_change":"2024-04-24T06:35:52.822639+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.822639+0000","last_undegraded":"2024-04-24T06:35:52.822639+0000","last_fullsized":"2024-04-24T06:35:52.822639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.831879+0000","last_change":"2024-04-24T06:35:52.831879+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.831879+0000","last_undegraded":"2024-04-24T06:35:52.831879+0000","last_fullsized":"2024-04-24T06:35:52.831879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.816327+0000","last_change":"2024-04-24T06:35:52.816327+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.816327+0000","last_undegraded":"2024-04-24T06:35:52.816327+0000","last_fullsized":"2024-04-24T06:35:52.816327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.819716+0000","last_change":"2024-04-24T06:35:52.819716+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.819716+0000","last_undegraded":"2024-04-24T06:35:52.819716+0000","last_fullsized":"2024-04-24T06:35:52.819716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.832191+0000","last_change":"2024-04-24T06:35:52.832191+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.832191+0000","last_undegraded":"2024-04-24T06:35:52.832191+0000","last_fullsized":"2024-04-24T06:35:52.832191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.820845+0000","last_change":"2024-04-24T06:35:52.820845+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.820845+0000","last_undegraded":"2024-04-24T06:35:52.820845+0000","last_fullsized":"2024-04-24T06:35:52.820845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.824124+0000","last_change":"2024-04-24T06:35:52.824124+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.824124+0000","last_undegraded":"2024-04-24T06:35:52.824124+0000","last_fullsized":"2024-04-24T06:35:52.824124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.823268+0000","last_change":"2024-04-24T06:35:52.823268+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.823268+0000","last_undegraded":"2024-04-24T06:35:52.823268+0000","last_fullsized":"2024-04-24T06:35:52.823268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.822590+0000","last_change":"2024-04-24T06:35:52.822590+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.822590+0000","last_undegraded":"2024-04-24T06:35:52.822590+0000","last_fullsized":"2024-04-24T06:35:52.822590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.819496+0000","last_change":"2024-04-24T06:35:52.819496+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.819496+0000","last_undegraded":"2024-04-24T06:35:52.819496+0000","last_fullsized":"2024-04-24T06:35:52.819496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.830132+0000","last_change":"2024-04-24T06:35:52.830132+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.830132+0000","last_undegraded":"2024-04-24T06:35:52.830132+0000","last_fullsized":"2024-04-24T06:35:52.830132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.830285+0000","last_change":"2024-04-24T06:35:52.830285+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.830285+0000","last_undegraded":"2024-04-24T06:35:52.830285+0000","last_fullsized":"2024-04-24T06:35:52.830285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.819344+0000","last_change":"2024-04-24T06:35:52.819344+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.819344+0000","last_undegraded":"2024-04-24T06:35:52.819344+0000","last_fullsized":"2024-04-24T06:35:52.819344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.816201+0000","last_change":"2024-04-24T06:35:52.816201+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.816201+0000","last_undegraded":"2024-04-24T06:35:52.816201+0000","last_fullsized":"2024-04-24T06:35:52.816201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.815978+0000","last_change":"2024-04-24T06:35:52.815978+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.815978+0000","last_undegraded":"2024-04-24T06:35:52.815978+0000","last_fullsized":"2024-04-24T06:35:52.815978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.820867+0000","last_change":"2024-04-24T06:35:52.820867+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.820867+0000","last_undegraded":"2024-04-24T06:35:52.820867+0000","last_fullsized":"2024-04-24T06:35:52.820867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.821691+0000","last_change":"2024-04-24T06:35:52.821691+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.821691+0000","last_undegraded":"2024-04-24T06:35:52.821691+0000","last_fullsized":"2024-04-24T06:35:52.821691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.825115+0000","last_change":"2024-04-24T06:35:52.825115+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.825115+0000","last_undegraded":"2024-04-24T06:35:52.825115+0000","last_fullsized":"2024-04-24T06:35:52.825115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.813986+0000","last_change":"2024-04-24T06:35:52.813986+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.813986+0000","last_undegraded":"2024-04-24T06:35:52.813986+0000","last_fullsized":"2024-04-24T06:35:52.813986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.821023+0000","last_change":"2024-04-24T06:35:52.821023+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.821023+0000","last_undegraded":"2024-04-24T06:35:52.821023+0000","last_fullsized":"2024-04-24T06:35:52.821023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.814018+0000","last_change":"2024-04-24T06:35:52.814018+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.814018+0000","last_undegraded":"2024-04-24T06:35:52.814018+0000","last_fullsized":"2024-04-24T06:35:52.814018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-24T06:35:53.817900+0000","last_change":"2024-04-24T06:35:53.817900+0000","last_active":"2024-04-24T06:35:53.817900+0000","last_peered":"2024-04-24T06:35:53.817900+0000","last_clean":"2024-04-24T06:35:53.817900+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:53.817900+0000","last_undegraded":"2024-04-24T06:35:53.817900+0000","last_fullsized":"2024-04-24T06:35:53.817900+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-24T06:35:52.823691+0000","last_change":"2024-04-24T06:35:52.823691+0000","last_active":"2024-04-24T06:35:52.800823+0000","last_peered":"2024-04-24T06:35:52.800823+0000","last_clean":"2024-04-24T06:35:52.800823+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-24T06:35:52.823691+0000","last_undegraded":"2024-04-24T06:35:52.823691+0000","last_fullsized":"2024-04-24T06:35:52.823691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T06:35:52.823610+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:35:52.823610+0000","last_peered":"2024-04-24T06:35:52.823610+0000","last_clean":"2024-04-24T06:35:52.823610+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:35:52.823610+0000","last_undegraded":"2024-04-24T06:35:52.823610+0000","last_fullsized":"2024-04-24T06:35:52.823610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1000891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":410523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":983040,"data_stored":416340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":970752,"data_stored":410523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":410523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":410523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":983040,"data_stored":416340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":1000891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:35:59.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:35:59.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.608+0000 7f61f3e00640 1 Processor -- start 2024-04-24T06:35:59.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.608+0000 7f61f3e00640 1 -- start start 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61f415e7c0 0x7f61f40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61f40d2630 0x7f61f40b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f40b6530 0x7f61f40d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f61f40b7000 con 0x7f61f40d2630 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f61f40cf620 con 0x7f61f415e7c0 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f61f40cf030 con 0x7f61f40b6530 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61f40d2630 0x7f61f40b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61f40d2630 0x7f61f40b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34988/0 (socket says 172.21.15.59:34988) 2024-04-24T06:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.610+0000 7f61f2000640 1 -- 172.21.15.59:0/463017057 learned_addr learned my addr 172.21.15.59:0/463017057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2000640 1 -- 172.21.15.59:0/463017057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f40b6530 msgr2=0x7f61f40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2000640 1 --2- 172.21.15.59:0/463017057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f40b6530 0x7f61f40d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2000640 1 -- 172.21.15.59:0/463017057 >> v2:172.21.15.66:3300/0 conn(0x7f61f415e7c0 msgr2=0x7f61f40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2a00640 1 --2- 172.21.15.59:0/463017057 >> v2:172.21.15.66:3300/0 conn(0x7f61f415e7c0 0x7f61f40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2000640 1 --2- 172.21.15.59:0/463017057 >> v2:172.21.15.66:3300/0 conn(0x7f61f415e7c0 0x7f61f40d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.611+0000 7f61f2000640 1 -- 172.21.15.59:0/463017057 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f40b6a40 con 0x7f61f40d2630 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.612+0000 7f61f2000640 1 --2- 172.21.15.59:0/463017057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b5ff0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f61f408c6d0 tx=0x7f61dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165e8696d632853f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.612+0000 7f61f1600640 1 -- 172.21.15.59:0/463017057 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc037070 con 0x7f61f40d2630 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.612+0000 7f61f1600640 1 -- 172.21.15.59:0/463017057 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc032440 con 0x7f61f40d2630 2024-04-24T06:35:59.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f1600640 1 -- 172.21.15.59:0/463017057 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc032810 con 0x7f61f40d2630 2024-04-24T06:35:59.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 -- 172.21.15.59:0/463017057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 msgr2=0x7f61f40b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 --2- 172.21.15.59:0/463017057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b5ff0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f61f408c6d0 tx=0x7f61dc02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 -- 172.21.15.59:0/463017057 shutdown_connections 2024-04-24T06:35:59.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 --2- 172.21.15.59:0/463017057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f40b6530 0x7f61f40d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 --2- 172.21.15.59:0/463017057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b5ff0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 --2- 172.21.15.59:0/463017057 >> v2:172.21.15.66:3300/0 conn(0x7f61f415e7c0 0x7f61f40d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 -- 172.21.15.59:0/463017057 >> 172.21.15.59:0/463017057 conn(0x7f61f409d920 msgr2=0x7f61f408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:59.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 -- 172.21.15.59:0/463017057 shutdown_connections 2024-04-24T06:35:59.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.613+0000 7f61f3e00640 1 -- 172.21.15.59:0/463017057 wait complete. 2024-04-24T06:35:59.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.615+0000 7f61f3e00640 1 Processor -- start 2024-04-24T06:35:59.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.615+0000 7f61f3e00640 1 -- start start 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 0x7f61f414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 0x7f61f40b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40d1a80 con 0x7f61f40d2630 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61f40cf2b0 con 0x7f61f40b6530 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f61f40d1c00 con 0x7f61f415e7c0 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.616+0000 7f61f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34994/0 (socket says 172.21.15.59:34994) 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 -- 172.21.15.59:0/2721008704 learned_addr learned my addr 172.21.15.59:0/2721008704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2a00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 0x7f61f414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 -- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 msgr2=0x7f61f40b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f3400640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 0x7f61f40b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 0x7f61f40b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 -- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 msgr2=0x7f61f414fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 0x7f61f414fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 -- 172.21.15.59:0/2721008704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f40b11a0 con 0x7f61f40d2630 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2a00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 0x7f61f414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f3400640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 0x7f61f40b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.617+0000 7f61f2000640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009960 tx=0x7f61dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.618+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc041020 con 0x7f61f40d2630 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.618+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc030800 con 0x7f61f40d2630 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.618+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f40b1440 con 0x7f61f40d2630 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.618+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc037040 con 0x7f61f40d2630 2024-04-24T06:35:59.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.618+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f40b1a00 con 0x7f61f40d2630 2024-04-24T06:35:59.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.619+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f404fd90 con 0x7f61f40d2630 2024-04-24T06:35:59.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.620+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61dc044020 con 0x7f61f40d2630 2024-04-24T06:35:59.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.621+0000 7f61eaa00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:59.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.621+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f61dc0ac420 con 0x7f61f40d2630 2024-04-24T06:35:59.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.621+0000 7f61f2a00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:59.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.622+0000 7f61f2a00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 0x7f61d006edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61e400b820 tx=0x7f61e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:59.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.626+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61dc04b030 con 0x7f61f40d2630 2024-04-24T06:35:59.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.829+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61f4163090 con 0x7f61d006c8f0 2024-04-24T06:35:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.840+0000 7f61eaa00640 1 -- 172.21.15.59:0/2721008704 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252084 (secure 0 0 0) 0x7f61f4163090 con 0x7f61d006c8f0 2024-04-24T06:35:59.846 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:35:59.847 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 msgr2=0x7f61d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 0x7f61d006edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61e400b820 tx=0x7f61e4008040 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 msgr2=0x7f61f40b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009960 tx=0x7f61dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 shutdown_connections 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61f415e7c0 0x7f61f40b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61f40d2630 0x7f61f40b71d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 --2- 172.21.15.59:0/2721008704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f40b6530 0x7f61f414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:59.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 >> 172.21.15.59:0/2721008704 conn(0x7f61f409d920 msgr2=0x7f61f40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:59.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 shutdown_connections 2024-04-24T06:35:59.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:35:59.847+0000 7f61f3e00640 1 -- 172.21.15.59:0/2721008704 wait complete. 2024-04-24T06:35:59.868 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-24T06:35:59.336537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258648,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716072,"statfs":{"total":773094113280,"available":772829257728,"internally_reserved":0,"allocated":9412608,"data_stored":4671076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001892"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411257+0000","last_change":"2024-04-24T06:35:54.899238+0000","last_active":"2024-04-24T06:35:56.411257+0000","last_peered":"2024-04-24T06:35:56.411257+0000","last_clean":"2024-04-24T06:35:56.411257+0000","last_became_active":"2024-04-24T06:35:54.898518+0000","last_became_peered":"2024-04-24T06:35:54.898518+0000","last_unstale":"2024-04-24T06:35:56.411257+0000","last_undegraded":"2024-04-24T06:35:56.411257+0000","last_fullsized":"2024-04-24T06:35:56.411257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:54.609914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893638+0000","last_change":"2024-04-24T06:35:54.910199+0000","last_active":"2024-04-24T06:35:55.893638+0000","last_peered":"2024-04-24T06:35:55.893638+0000","last_clean":"2024-04-24T06:35:55.893638+0000","last_became_active":"2024-04-24T06:35:54.909860+0000","last_became_peered":"2024-04-24T06:35:54.909860+0000","last_unstale":"2024-04-24T06:35:55.893638+0000","last_undegraded":"2024-04-24T06:35:55.893638+0000","last_fullsized":"2024-04-24T06:35:55.893638+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:38:24.135716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890255+0000","last_change":"2024-04-24T06:35:54.898034+0000","last_active":"2024-04-24T06:35:55.890255+0000","last_peered":"2024-04-24T06:35:55.890255+0000","last_clean":"2024-04-24T06:35:55.890255+0000","last_became_active":"2024-04-24T06:35:54.897812+0000","last_became_peered":"2024-04-24T06:35:54.897812+0000","last_unstale":"2024-04-24T06:35:55.890255+0000","last_undegraded":"2024-04-24T06:35:55.890255+0000","last_fullsized":"2024-04-24T06:35:55.890255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:54.609914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407993+0000","last_change":"2024-04-24T06:35:54.898114+0000","last_active":"2024-04-24T06:35:56.407993+0000","last_peered":"2024-04-24T06:35:56.407993+0000","last_clean":"2024-04-24T06:35:56.407993+0000","last_became_active":"2024-04-24T06:35:54.897783+0000","last_became_peered":"2024-04-24T06:35:54.897783+0000","last_unstale":"2024-04-24T06:35:56.407993+0000","last_undegraded":"2024-04-24T06:35:56.407993+0000","last_fullsized":"2024-04-24T06:35:56.407993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:31.440770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408726+0000","last_change":"2024-04-24T06:35:54.899965+0000","last_active":"2024-04-24T06:35:56.408726+0000","last_peered":"2024-04-24T06:35:56.408726+0000","last_clean":"2024-04-24T06:35:56.408726+0000","last_became_active":"2024-04-24T06:35:54.899678+0000","last_became_peered":"2024-04-24T06:35:54.899678+0000","last_unstale":"2024-04-24T06:35:56.408726+0000","last_undegraded":"2024-04-24T06:35:56.408726+0000","last_fullsized":"2024-04-24T06:35:56.408726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:52:28.608835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.199102+0000","last_change":"2024-04-24T06:35:54.901982+0000","last_active":"2024-04-24T06:35:56.199102+0000","last_peered":"2024-04-24T06:35:56.199102+0000","last_clean":"2024-04-24T06:35:56.199102+0000","last_became_active":"2024-04-24T06:35:54.901543+0000","last_became_peered":"2024-04-24T06:35:54.901543+0000","last_unstale":"2024-04-24T06:35:56.199102+0000","last_undegraded":"2024-04-24T06:35:56.199102+0000","last_fullsized":"2024-04-24T06:35:56.199102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:33:50.316414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407347+0000","last_change":"2024-04-24T06:35:54.898995+0000","last_active":"2024-04-24T06:35:56.407347+0000","last_peered":"2024-04-24T06:35:56.407347+0000","last_clean":"2024-04-24T06:35:56.407347+0000","last_became_active":"2024-04-24T06:35:54.898205+0000","last_became_peered":"2024-04-24T06:35:54.898205+0000","last_unstale":"2024-04-24T06:35:56.407347+0000","last_undegraded":"2024-04-24T06:35:56.407347+0000","last_fullsized":"2024-04-24T06:35:56.407347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:51.819986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.198625+0000","last_change":"2024-04-24T06:35:54.896275+0000","last_active":"2024-04-24T06:35:56.198625+0000","last_peered":"2024-04-24T06:35:56.198625+0000","last_clean":"2024-04-24T06:35:56.198625+0000","last_became_active":"2024-04-24T06:35:54.895956+0000","last_became_peered":"2024-04-24T06:35:54.895956+0000","last_unstale":"2024-04-24T06:35:56.198625+0000","last_undegraded":"2024-04-24T06:35:56.198625+0000","last_fullsized":"2024-04-24T06:35:56.198625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:22:11.171257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.199269+0000","last_change":"2024-04-24T06:35:54.893003+0000","last_active":"2024-04-24T06:35:56.199269+0000","last_peered":"2024-04-24T06:35:56.199269+0000","last_clean":"2024-04-24T06:35:56.199269+0000","last_became_active":"2024-04-24T06:35:54.892666+0000","last_became_peered":"2024-04-24T06:35:54.892666+0000","last_unstale":"2024-04-24T06:35:56.199269+0000","last_undegraded":"2024-04-24T06:35:56.199269+0000","last_fullsized":"2024-04-24T06:35:56.199269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:46.413234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411330+0000","last_change":"2024-04-24T06:35:54.895443+0000","last_active":"2024-04-24T06:35:56.411330+0000","last_peered":"2024-04-24T06:35:56.411330+0000","last_clean":"2024-04-24T06:35:56.411330+0000","last_became_active":"2024-04-24T06:35:54.895136+0000","last_became_peered":"2024-04-24T06:35:54.895136+0000","last_unstale":"2024-04-24T06:35:56.411330+0000","last_undegraded":"2024-04-24T06:35:56.411330+0000","last_fullsized":"2024-04-24T06:35:56.411330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:15.212844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893862+0000","last_change":"2024-04-24T06:35:54.894863+0000","last_active":"2024-04-24T06:35:55.893862+0000","last_peered":"2024-04-24T06:35:55.893862+0000","last_clean":"2024-04-24T06:35:55.893862+0000","last_became_active":"2024-04-24T06:35:54.893745+0000","last_became_peered":"2024-04-24T06:35:54.893745+0000","last_unstale":"2024-04-24T06:35:55.893862+0000","last_undegraded":"2024-04-24T06:35:55.893862+0000","last_fullsized":"2024-04-24T06:35:55.893862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:52.682204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410551+0000","last_change":"2024-04-24T06:35:54.888705+0000","last_active":"2024-04-24T06:35:56.410551+0000","last_peered":"2024-04-24T06:35:56.410551+0000","last_clean":"2024-04-24T06:35:56.410551+0000","last_became_active":"2024-04-24T06:35:54.885474+0000","last_became_peered":"2024-04-24T06:35:54.885474+0000","last_unstale":"2024-04-24T06:35:56.410551+0000","last_undegraded":"2024-04-24T06:35:56.410551+0000","last_fullsized":"2024-04-24T06:35:56.410551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:52:28.608835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890066+0000","last_change":"2024-04-24T06:35:54.884638+0000","last_active":"2024-04-24T06:35:55.890066+0000","last_peered":"2024-04-24T06:35:55.890066+0000","last_clean":"2024-04-24T06:35:55.890066+0000","last_became_active":"2024-04-24T06:35:54.884277+0000","last_became_peered":"2024-04-24T06:35:54.884277+0000","last_unstale":"2024-04-24T06:35:55.890066+0000","last_undegraded":"2024-04-24T06:35:55.890066+0000","last_fullsized":"2024-04-24T06:35:55.890066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:52:28.608835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408512+0000","last_change":"2024-04-24T06:35:54.912023+0000","last_active":"2024-04-24T06:35:56.408512+0000","last_peered":"2024-04-24T06:35:56.408512+0000","last_clean":"2024-04-24T06:35:56.408512+0000","last_became_active":"2024-04-24T06:35:54.911617+0000","last_became_peered":"2024-04-24T06:35:54.911617+0000","last_unstale":"2024-04-24T06:35:56.408512+0000","last_undegraded":"2024-04-24T06:35:56.408512+0000","last_fullsized":"2024-04-24T06:35:56.408512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:00:32.045424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.888822+0000","last_change":"2024-04-24T06:35:54.894923+0000","last_active":"2024-04-24T06:35:55.888822+0000","last_peered":"2024-04-24T06:35:55.888822+0000","last_clean":"2024-04-24T06:35:55.888822+0000","last_became_active":"2024-04-24T06:35:54.894609+0000","last_became_peered":"2024-04-24T06:35:54.894609+0000","last_unstale":"2024-04-24T06:35:55.888822+0000","last_undegraded":"2024-04-24T06:35:55.888822+0000","last_fullsized":"2024-04-24T06:35:55.888822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:19:21.922993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408058+0000","last_change":"2024-04-24T06:35:54.888350+0000","last_active":"2024-04-24T06:35:56.408058+0000","last_peered":"2024-04-24T06:35:56.408058+0000","last_clean":"2024-04-24T06:35:56.408058+0000","last_became_active":"2024-04-24T06:35:54.887968+0000","last_became_peered":"2024-04-24T06:35:54.887968+0000","last_unstale":"2024-04-24T06:35:56.408058+0000","last_undegraded":"2024-04-24T06:35:56.408058+0000","last_fullsized":"2024-04-24T06:35:56.408058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:46.413234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408979+0000","last_change":"2024-04-24T06:35:54.890524+0000","last_active":"2024-04-24T06:35:56.408979+0000","last_peered":"2024-04-24T06:35:56.408979+0000","last_clean":"2024-04-24T06:35:56.408979+0000","last_became_active":"2024-04-24T06:35:54.890212+0000","last_became_peered":"2024-04-24T06:35:54.890212+0000","last_unstale":"2024-04-24T06:35:56.408979+0000","last_undegraded":"2024-04-24T06:35:56.408979+0000","last_fullsized":"2024-04-24T06:35:56.408979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:22:40.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408225+0000","last_change":"2024-04-24T06:35:54.904583+0000","last_active":"2024-04-24T06:35:56.408225+0000","last_peered":"2024-04-24T06:35:56.408225+0000","last_clean":"2024-04-24T06:35:56.408225+0000","last_became_active":"2024-04-24T06:35:54.904290+0000","last_became_peered":"2024-04-24T06:35:54.904290+0000","last_unstale":"2024-04-24T06:35:56.408225+0000","last_undegraded":"2024-04-24T06:35:56.408225+0000","last_fullsized":"2024-04-24T06:35:56.408225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:47:32.963077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898761+0000","last_change":"2024-04-24T06:35:54.884107+0000","last_active":"2024-04-24T06:35:55.898761+0000","last_peered":"2024-04-24T06:35:55.898761+0000","last_clean":"2024-04-24T06:35:55.898761+0000","last_became_active":"2024-04-24T06:35:54.883805+0000","last_became_peered":"2024-04-24T06:35:54.883805+0000","last_unstale":"2024-04-24T06:35:55.898761+0000","last_undegraded":"2024-04-24T06:35:55.898761+0000","last_fullsized":"2024-04-24T06:35:55.898761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:41.249252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407916+0000","last_change":"2024-04-24T06:35:54.889801+0000","last_active":"2024-04-24T06:35:56.407916+0000","last_peered":"2024-04-24T06:35:56.407916+0000","last_clean":"2024-04-24T06:35:56.407916+0000","last_became_active":"2024-04-24T06:35:54.889459+0000","last_became_peered":"2024-04-24T06:35:54.889459+0000","last_unstale":"2024-04-24T06:35:56.407916+0000","last_undegraded":"2024-04-24T06:35:56.407916+0000","last_fullsized":"2024-04-24T06:35:56.407916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:33:07.022051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408027+0000","last_change":"2024-04-24T06:35:54.903338+0000","last_active":"2024-04-24T06:35:56.408027+0000","last_peered":"2024-04-24T06:35:56.408027+0000","last_clean":"2024-04-24T06:35:56.408027+0000","last_became_active":"2024-04-24T06:35:54.902997+0000","last_became_peered":"2024-04-24T06:35:54.902997+0000","last_unstale":"2024-04-24T06:35:56.408027+0000","last_undegraded":"2024-04-24T06:35:56.408027+0000","last_fullsized":"2024-04-24T06:35:56.408027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:16:12.554384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892651+0000","last_change":"2024-04-24T06:35:54.894160+0000","last_active":"2024-04-24T06:35:55.892651+0000","last_peered":"2024-04-24T06:35:55.892651+0000","last_clean":"2024-04-24T06:35:55.892651+0000","last_became_active":"2024-04-24T06:35:54.891600+0000","last_became_peered":"2024-04-24T06:35:54.891600+0000","last_unstale":"2024-04-24T06:35:55.892651+0000","last_undegraded":"2024-04-24T06:35:55.892651+0000","last_fullsized":"2024-04-24T06:35:55.892651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:38.082192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409837+0000","last_change":"2024-04-24T06:35:54.897704+0000","last_active":"2024-04-24T06:35:56.409837+0000","last_peered":"2024-04-24T06:35:56.409837+0000","last_clean":"2024-04-24T06:35:56.409837+0000","last_became_active":"2024-04-24T06:35:54.897343+0000","last_became_peered":"2024-04-24T06:35:54.897343+0000","last_unstale":"2024-04-24T06:35:56.409837+0000","last_undegraded":"2024-04-24T06:35:56.409837+0000","last_fullsized":"2024-04-24T06:35:56.409837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:12.120165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407258+0000","last_change":"2024-04-24T06:35:54.899515+0000","last_active":"2024-04-24T06:35:56.407258+0000","last_peered":"2024-04-24T06:35:56.407258+0000","last_clean":"2024-04-24T06:35:56.407258+0000","last_became_active":"2024-04-24T06:35:54.899004+0000","last_became_peered":"2024-04-24T06:35:54.899004+0000","last_unstale":"2024-04-24T06:35:56.407258+0000","last_undegraded":"2024-04-24T06:35:56.407258+0000","last_fullsized":"2024-04-24T06:35:56.407258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:32:34.309252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893776+0000","last_change":"2024-04-24T06:35:54.905683+0000","last_active":"2024-04-24T06:35:55.893776+0000","last_peered":"2024-04-24T06:35:55.893776+0000","last_clean":"2024-04-24T06:35:55.893776+0000","last_became_active":"2024-04-24T06:35:54.905393+0000","last_became_peered":"2024-04-24T06:35:54.905393+0000","last_unstale":"2024-04-24T06:35:55.893776+0000","last_undegraded":"2024-04-24T06:35:55.893776+0000","last_fullsized":"2024-04-24T06:35:55.893776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:49:33.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.889973+0000","last_change":"2024-04-24T06:35:54.898027+0000","last_active":"2024-04-24T06:35:55.889973+0000","last_peered":"2024-04-24T06:35:55.889973+0000","last_clean":"2024-04-24T06:35:55.889973+0000","last_became_active":"2024-04-24T06:35:54.897657+0000","last_became_peered":"2024-04-24T06:35:54.897657+0000","last_unstale":"2024-04-24T06:35:55.889973+0000","last_undegraded":"2024-04-24T06:35:55.889973+0000","last_fullsized":"2024-04-24T06:35:55.889973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:25:47.855861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407973+0000","last_change":"2024-04-24T06:35:54.884996+0000","last_active":"2024-04-24T06:35:56.407973+0000","last_peered":"2024-04-24T06:35:56.407973+0000","last_clean":"2024-04-24T06:35:56.407973+0000","last_became_active":"2024-04-24T06:35:54.884639+0000","last_became_peered":"2024-04-24T06:35:54.884639+0000","last_unstale":"2024-04-24T06:35:56.407973+0000","last_undegraded":"2024-04-24T06:35:56.407973+0000","last_fullsized":"2024-04-24T06:35:56.407973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:12.120165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410069+0000","last_change":"2024-04-24T06:35:54.894938+0000","last_active":"2024-04-24T06:35:56.410069+0000","last_peered":"2024-04-24T06:35:56.410069+0000","last_clean":"2024-04-24T06:35:56.410069+0000","last_became_active":"2024-04-24T06:35:54.894615+0000","last_became_peered":"2024-04-24T06:35:54.894615+0000","last_unstale":"2024-04-24T06:35:56.410069+0000","last_undegraded":"2024-04-24T06:35:56.410069+0000","last_fullsized":"2024-04-24T06:35:56.410069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:46.413234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893115+0000","last_change":"2024-04-24T06:35:54.888415+0000","last_active":"2024-04-24T06:35:55.893115+0000","last_peered":"2024-04-24T06:35:55.893115+0000","last_clean":"2024-04-24T06:35:55.893115+0000","last_became_active":"2024-04-24T06:35:54.888141+0000","last_became_peered":"2024-04-24T06:35:54.888141+0000","last_unstale":"2024-04-24T06:35:55.893115+0000","last_undegraded":"2024-04-24T06:35:55.893115+0000","last_fullsized":"2024-04-24T06:35:55.893115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:06:35.546320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892816+0000","last_change":"2024-04-24T06:35:54.892885+0000","last_active":"2024-04-24T06:35:55.892816+0000","last_peered":"2024-04-24T06:35:55.892816+0000","last_clean":"2024-04-24T06:35:55.892816+0000","last_became_active":"2024-04-24T06:35:54.892608+0000","last_became_peered":"2024-04-24T06:35:54.892608+0000","last_unstale":"2024-04-24T06:35:55.892816+0000","last_undegraded":"2024-04-24T06:35:55.892816+0000","last_fullsized":"2024-04-24T06:35:55.892816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:11:12.207646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407689+0000","last_change":"2024-04-24T06:35:54.881685+0000","last_active":"2024-04-24T06:35:56.407689+0000","last_peered":"2024-04-24T06:35:56.407689+0000","last_clean":"2024-04-24T06:35:56.407689+0000","last_became_active":"2024-04-24T06:35:54.881381+0000","last_became_peered":"2024-04-24T06:35:54.881381+0000","last_unstale":"2024-04-24T06:35:56.407689+0000","last_undegraded":"2024-04-24T06:35:56.407689+0000","last_fullsized":"2024-04-24T06:35:56.407689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:41:49.646192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408333+0000","last_change":"2024-04-24T06:35:54.897914+0000","last_active":"2024-04-24T06:35:56.408333+0000","last_peered":"2024-04-24T06:35:56.408333+0000","last_clean":"2024-04-24T06:35:56.408333+0000","last_became_active":"2024-04-24T06:35:54.897574+0000","last_became_peered":"2024-04-24T06:35:54.897574+0000","last_unstale":"2024-04-24T06:35:56.408333+0000","last_undegraded":"2024-04-24T06:35:56.408333+0000","last_fullsized":"2024-04-24T06:35:56.408333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:33:50.316414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409260+0000","last_change":"2024-04-24T06:35:54.891712+0000","last_active":"2024-04-24T06:35:56.409260+0000","last_peered":"2024-04-24T06:35:56.409260+0000","last_clean":"2024-04-24T06:35:56.409260+0000","last_became_active":"2024-04-24T06:35:54.891386+0000","last_became_peered":"2024-04-24T06:35:54.891386+0000","last_unstale":"2024-04-24T06:35:56.409260+0000","last_undegraded":"2024-04-24T06:35:56.409260+0000","last_fullsized":"2024-04-24T06:35:56.409260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:22:20.940853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411517+0000","last_change":"2024-04-24T06:35:53.867966+0000","last_active":"2024-04-24T06:35:56.411517+0000","last_peered":"2024-04-24T06:35:56.411517+0000","last_clean":"2024-04-24T06:35:56.411517+0000","last_became_active":"2024-04-24T06:35:53.865502+0000","last_became_peered":"2024-04-24T06:35:53.865502+0000","last_unstale":"2024-04-24T06:35:56.411517+0000","last_undegraded":"2024-04-24T06:35:56.411517+0000","last_fullsized":"2024-04-24T06:35:56.411517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:09.388126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411342+0000","last_change":"2024-04-24T06:35:54.898957+0000","last_active":"2024-04-24T06:35:56.411342+0000","last_peered":"2024-04-24T06:35:56.411342+0000","last_clean":"2024-04-24T06:35:56.411342+0000","last_became_active":"2024-04-24T06:35:54.898189+0000","last_became_peered":"2024-04-24T06:35:54.898189+0000","last_unstale":"2024-04-24T06:35:56.411342+0000","last_undegraded":"2024-04-24T06:35:56.411342+0000","last_fullsized":"2024-04-24T06:35:56.411342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:22:40.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409458+0000","last_change":"2024-04-24T06:35:53.858571+0000","last_active":"2024-04-24T06:35:56.409458+0000","last_peered":"2024-04-24T06:35:56.409458+0000","last_clean":"2024-04-24T06:35:56.409458+0000","last_became_active":"2024-04-24T06:35:53.857516+0000","last_became_peered":"2024-04-24T06:35:53.857516+0000","last_unstale":"2024-04-24T06:35:56.409458+0000","last_undegraded":"2024-04-24T06:35:56.409458+0000","last_fullsized":"2024-04-24T06:35:56.409458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:37.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410151+0000","last_change":"2024-04-24T06:35:54.898545+0000","last_active":"2024-04-24T06:35:56.410151+0000","last_peered":"2024-04-24T06:35:56.410151+0000","last_clean":"2024-04-24T06:35:56.410151+0000","last_became_active":"2024-04-24T06:35:54.896430+0000","last_became_peered":"2024-04-24T06:35:54.896430+0000","last_unstale":"2024-04-24T06:35:56.410151+0000","last_undegraded":"2024-04-24T06:35:56.410151+0000","last_fullsized":"2024-04-24T06:35:56.410151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:41.249252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898685+0000","last_change":"2024-04-24T06:35:53.859359+0000","last_active":"2024-04-24T06:35:55.898685+0000","last_peered":"2024-04-24T06:35:55.898685+0000","last_clean":"2024-04-24T06:35:55.898685+0000","last_became_active":"2024-04-24T06:35:53.859065+0000","last_became_peered":"2024-04-24T06:35:53.859065+0000","last_unstale":"2024-04-24T06:35:55.898685+0000","last_undegraded":"2024-04-24T06:35:55.898685+0000","last_fullsized":"2024-04-24T06:35:55.898685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:51.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890597+0000","last_change":"2024-04-24T06:35:54.897294+0000","last_active":"2024-04-24T06:35:55.890597+0000","last_peered":"2024-04-24T06:35:55.890597+0000","last_clean":"2024-04-24T06:35:55.890597+0000","last_became_active":"2024-04-24T06:35:54.897031+0000","last_became_peered":"2024-04-24T06:35:54.897031+0000","last_unstale":"2024-04-24T06:35:55.890597+0000","last_undegraded":"2024-04-24T06:35:55.890597+0000","last_fullsized":"2024-04-24T06:35:55.890597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:11.875714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898423+0000","last_change":"2024-04-24T06:35:53.856641+0000","last_active":"2024-04-24T06:35:55.898423+0000","last_peered":"2024-04-24T06:35:55.898423+0000","last_clean":"2024-04-24T06:35:55.898423+0000","last_became_active":"2024-04-24T06:35:53.856343+0000","last_became_peered":"2024-04-24T06:35:53.856343+0000","last_unstale":"2024-04-24T06:35:55.898423+0000","last_undegraded":"2024-04-24T06:35:55.898423+0000","last_fullsized":"2024-04-24T06:35:55.898423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:13.104346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892874+0000","last_change":"2024-04-24T06:35:54.897654+0000","last_active":"2024-04-24T06:35:55.892874+0000","last_peered":"2024-04-24T06:35:55.892874+0000","last_clean":"2024-04-24T06:35:55.892874+0000","last_became_active":"2024-04-24T06:35:54.897328+0000","last_became_peered":"2024-04-24T06:35:54.897328+0000","last_unstale":"2024-04-24T06:35:55.892874+0000","last_undegraded":"2024-04-24T06:35:55.892874+0000","last_fullsized":"2024-04-24T06:35:55.892874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:41:43.644364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409112+0000","last_change":"2024-04-24T06:35:53.862769+0000","last_active":"2024-04-24T06:35:56.409112+0000","last_peered":"2024-04-24T06:35:56.409112+0000","last_clean":"2024-04-24T06:35:56.409112+0000","last_became_active":"2024-04-24T06:35:53.862363+0000","last_became_peered":"2024-04-24T06:35:53.862363+0000","last_unstale":"2024-04-24T06:35:56.409112+0000","last_undegraded":"2024-04-24T06:35:56.409112+0000","last_fullsized":"2024-04-24T06:35:56.409112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:28.715327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408970+0000","last_change":"2024-04-24T06:35:54.890019+0000","last_active":"2024-04-24T06:35:56.408970+0000","last_peered":"2024-04-24T06:35:56.408970+0000","last_clean":"2024-04-24T06:35:56.408970+0000","last_became_active":"2024-04-24T06:35:54.889682+0000","last_became_peered":"2024-04-24T06:35:54.889682+0000","last_unstale":"2024-04-24T06:35:56.408970+0000","last_undegraded":"2024-04-24T06:35:56.408970+0000","last_fullsized":"2024-04-24T06:35:56.408970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:15.212844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407827+0000","last_change":"2024-04-24T06:35:53.858762+0000","last_active":"2024-04-24T06:35:56.407827+0000","last_peered":"2024-04-24T06:35:56.407827+0000","last_clean":"2024-04-24T06:35:56.407827+0000","last_became_active":"2024-04-24T06:35:53.856987+0000","last_became_peered":"2024-04-24T06:35:53.856987+0000","last_unstale":"2024-04-24T06:35:56.407827+0000","last_undegraded":"2024-04-24T06:35:56.407827+0000","last_fullsized":"2024-04-24T06:35:56.407827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:31:35.094858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.889674+0000","last_change":"2024-04-24T06:35:54.890479+0000","last_active":"2024-04-24T06:35:55.889674+0000","last_peered":"2024-04-24T06:35:55.889674+0000","last_clean":"2024-04-24T06:35:55.889674+0000","last_became_active":"2024-04-24T06:35:54.890187+0000","last_became_peered":"2024-04-24T06:35:54.890187+0000","last_unstale":"2024-04-24T06:35:55.889674+0000","last_undegraded":"2024-04-24T06:35:55.889674+0000","last_fullsized":"2024-04-24T06:35:55.889674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:41.249252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898069+0000","last_change":"2024-04-24T06:35:53.862169+0000","last_active":"2024-04-24T06:35:55.898069+0000","last_peered":"2024-04-24T06:35:55.898069+0000","last_clean":"2024-04-24T06:35:55.898069+0000","last_became_active":"2024-04-24T06:35:53.861710+0000","last_became_peered":"2024-04-24T06:35:53.861710+0000","last_unstale":"2024-04-24T06:35:55.898069+0000","last_undegraded":"2024-04-24T06:35:55.898069+0000","last_fullsized":"2024-04-24T06:35:55.898069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:41:42.627287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.888990+0000","last_change":"2024-04-24T06:35:54.892990+0000","last_active":"2024-04-24T06:35:55.888990+0000","last_peered":"2024-04-24T06:35:55.888990+0000","last_clean":"2024-04-24T06:35:55.888990+0000","last_became_active":"2024-04-24T06:35:54.892415+0000","last_became_peered":"2024-04-24T06:35:54.892415+0000","last_unstale":"2024-04-24T06:35:55.888990+0000","last_undegraded":"2024-04-24T06:35:55.888990+0000","last_fullsized":"2024-04-24T06:35:55.888990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:31:02.706969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892331+0000","last_change":"2024-04-24T06:35:53.862289+0000","last_active":"2024-04-24T06:35:55.892331+0000","last_peered":"2024-04-24T06:35:55.892331+0000","last_clean":"2024-04-24T06:35:55.892331+0000","last_became_active":"2024-04-24T06:35:53.862013+0000","last_became_peered":"2024-04-24T06:35:53.862013+0000","last_unstale":"2024-04-24T06:35:55.892331+0000","last_undegraded":"2024-04-24T06:35:55.892331+0000","last_fullsized":"2024-04-24T06:35:55.892331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:22:48.897664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.889537+0000","last_change":"2024-04-24T06:35:54.896745+0000","last_active":"2024-04-24T06:35:55.889537+0000","last_peered":"2024-04-24T06:35:55.889537+0000","last_clean":"2024-04-24T06:35:55.889537+0000","last_became_active":"2024-04-24T06:35:54.896406+0000","last_became_peered":"2024-04-24T06:35:54.896406+0000","last_unstale":"2024-04-24T06:35:55.889537+0000","last_undegraded":"2024-04-24T06:35:55.889537+0000","last_fullsized":"2024-04-24T06:35:55.889537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:48:04.031972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411815+0000","last_change":"2024-04-24T06:35:53.865237+0000","last_active":"2024-04-24T06:35:56.411815+0000","last_peered":"2024-04-24T06:35:56.411815+0000","last_clean":"2024-04-24T06:35:56.411815+0000","last_became_active":"2024-04-24T06:35:53.864873+0000","last_became_peered":"2024-04-24T06:35:53.864873+0000","last_unstale":"2024-04-24T06:35:56.411815+0000","last_undegraded":"2024-04-24T06:35:56.411815+0000","last_fullsized":"2024-04-24T06:35:56.411815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:48:26.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.199234+0000","last_change":"2024-04-24T06:35:54.889236+0000","last_active":"2024-04-24T06:35:56.199234+0000","last_peered":"2024-04-24T06:35:56.199234+0000","last_clean":"2024-04-24T06:35:56.199234+0000","last_became_active":"2024-04-24T06:35:54.888906+0000","last_became_peered":"2024-04-24T06:35:54.888906+0000","last_unstale":"2024-04-24T06:35:56.199234+0000","last_undegraded":"2024-04-24T06:35:56.199234+0000","last_fullsized":"2024-04-24T06:35:56.199234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:12.120165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411144+0000","last_change":"2024-04-24T06:35:53.866878+0000","last_active":"2024-04-24T06:35:56.411144+0000","last_peered":"2024-04-24T06:35:56.411144+0000","last_clean":"2024-04-24T06:35:56.411144+0000","last_became_active":"2024-04-24T06:35:53.866574+0000","last_became_peered":"2024-04-24T06:35:53.866574+0000","last_unstale":"2024-04-24T06:35:56.411144+0000","last_undegraded":"2024-04-24T06:35:56.411144+0000","last_fullsized":"2024-04-24T06:35:56.411144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:28.715327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898444+0000","last_change":"2024-04-24T06:35:54.890362+0000","last_active":"2024-04-24T06:35:55.898444+0000","last_peered":"2024-04-24T06:35:55.898444+0000","last_clean":"2024-04-24T06:35:55.898444+0000","last_became_active":"2024-04-24T06:35:54.889035+0000","last_became_peered":"2024-04-24T06:35:54.889035+0000","last_unstale":"2024-04-24T06:35:55.898444+0000","last_undegraded":"2024-04-24T06:35:55.898444+0000","last_fullsized":"2024-04-24T06:35:55.898444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:32:12.120165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409657+0000","last_change":"2024-04-24T06:35:53.867855+0000","last_active":"2024-04-24T06:35:56.409657+0000","last_peered":"2024-04-24T06:35:56.409657+0000","last_clean":"2024-04-24T06:35:56.409657+0000","last_became_active":"2024-04-24T06:35:53.866369+0000","last_became_peered":"2024-04-24T06:35:53.866369+0000","last_unstale":"2024-04-24T06:35:56.409657+0000","last_undegraded":"2024-04-24T06:35:56.409657+0000","last_fullsized":"2024-04-24T06:35:56.409657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:27:48.355489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408449+0000","last_change":"2024-04-24T06:35:54.882719+0000","last_active":"2024-04-24T06:35:56.408449+0000","last_peered":"2024-04-24T06:35:56.408449+0000","last_clean":"2024-04-24T06:35:56.408449+0000","last_became_active":"2024-04-24T06:35:54.882360+0000","last_became_peered":"2024-04-24T06:35:54.882360+0000","last_unstale":"2024-04-24T06:35:56.408449+0000","last_undegraded":"2024-04-24T06:35:56.408449+0000","last_fullsized":"2024-04-24T06:35:56.408449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:41.249252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.198658+0000","last_change":"2024-04-24T06:35:53.854134+0000","last_active":"2024-04-24T06:35:56.198658+0000","last_peered":"2024-04-24T06:35:56.198658+0000","last_clean":"2024-04-24T06:35:56.198658+0000","last_became_active":"2024-04-24T06:35:53.853811+0000","last_became_peered":"2024-04-24T06:35:53.853811+0000","last_unstale":"2024-04-24T06:35:56.198658+0000","last_undegraded":"2024-04-24T06:35:56.198658+0000","last_fullsized":"2024-04-24T06:35:56.198658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:37.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898243+0000","last_change":"2024-04-24T06:35:54.880546+0000","last_active":"2024-04-24T06:35:55.898243+0000","last_peered":"2024-04-24T06:35:55.898243+0000","last_clean":"2024-04-24T06:35:55.898243+0000","last_became_active":"2024-04-24T06:35:54.879830+0000","last_became_peered":"2024-04-24T06:35:54.879830+0000","last_unstale":"2024-04-24T06:35:55.898243+0000","last_undegraded":"2024-04-24T06:35:55.898243+0000","last_fullsized":"2024-04-24T06:35:55.898243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:41:49.646192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893102+0000","last_change":"2024-04-24T06:35:53.859178+0000","last_active":"2024-04-24T06:35:55.893102+0000","last_peered":"2024-04-24T06:35:55.893102+0000","last_clean":"2024-04-24T06:35:55.893102+0000","last_became_active":"2024-04-24T06:35:53.858885+0000","last_became_peered":"2024-04-24T06:35:53.858885+0000","last_unstale":"2024-04-24T06:35:55.893102+0000","last_undegraded":"2024-04-24T06:35:55.893102+0000","last_fullsized":"2024-04-24T06:35:55.893102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:28.536519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890564+0000","last_change":"2024-04-24T06:35:54.893760+0000","last_active":"2024-04-24T06:35:55.890564+0000","last_peered":"2024-04-24T06:35:55.890564+0000","last_clean":"2024-04-24T06:35:55.890564+0000","last_became_active":"2024-04-24T06:35:54.893429+0000","last_became_peered":"2024-04-24T06:35:54.893429+0000","last_unstale":"2024-04-24T06:35:55.890564+0000","last_undegraded":"2024-04-24T06:35:55.890564+0000","last_fullsized":"2024-04-24T06:35:55.890564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:22:20.940853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.888622+0000","last_change":"2024-04-24T06:35:53.856051+0000","last_active":"2024-04-24T06:35:55.888622+0000","last_peered":"2024-04-24T06:35:55.888622+0000","last_clean":"2024-04-24T06:35:55.888622+0000","last_became_active":"2024-04-24T06:35:53.855704+0000","last_became_peered":"2024-04-24T06:35:53.855704+0000","last_unstale":"2024-04-24T06:35:55.888622+0000","last_undegraded":"2024-04-24T06:35:55.888622+0000","last_fullsized":"2024-04-24T06:35:55.888622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:55.772731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892667+0000","last_change":"2024-04-24T06:35:54.901519+0000","last_active":"2024-04-24T06:35:55.892667+0000","last_peered":"2024-04-24T06:35:55.892667+0000","last_clean":"2024-04-24T06:35:55.892667+0000","last_became_active":"2024-04-24T06:35:54.901031+0000","last_became_peered":"2024-04-24T06:35:54.901031+0000","last_unstale":"2024-04-24T06:35:55.892667+0000","last_undegraded":"2024-04-24T06:35:55.892667+0000","last_fullsized":"2024-04-24T06:35:55.892667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:44.472782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.198491+0000","last_change":"2024-04-24T06:35:53.868438+0000","last_active":"2024-04-24T06:35:56.198491+0000","last_peered":"2024-04-24T06:35:56.198491+0000","last_clean":"2024-04-24T06:35:56.198491+0000","last_became_active":"2024-04-24T06:35:53.866461+0000","last_became_peered":"2024-04-24T06:35:53.866461+0000","last_unstale":"2024-04-24T06:35:56.198491+0000","last_undegraded":"2024-04-24T06:35:56.198491+0000","last_fullsized":"2024-04-24T06:35:56.198491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:43.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.892288+0000","last_change":"2024-04-24T06:35:54.904741+0000","last_active":"2024-04-24T06:35:55.892288+0000","last_peered":"2024-04-24T06:35:55.892288+0000","last_clean":"2024-04-24T06:35:55.892288+0000","last_became_active":"2024-04-24T06:35:54.904412+0000","last_became_peered":"2024-04-24T06:35:54.904412+0000","last_unstale":"2024-04-24T06:35:55.892288+0000","last_undegraded":"2024-04-24T06:35:55.892288+0000","last_fullsized":"2024-04-24T06:35:55.892288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:38:29.732403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.897904+0000","last_change":"2024-04-24T06:35:53.865133+0000","last_active":"2024-04-24T06:35:55.897904+0000","last_peered":"2024-04-24T06:35:55.897904+0000","last_clean":"2024-04-24T06:35:55.897904+0000","last_became_active":"2024-04-24T06:35:53.864848+0000","last_became_peered":"2024-04-24T06:35:53.864848+0000","last_unstale":"2024-04-24T06:35:55.897904+0000","last_undegraded":"2024-04-24T06:35:55.897904+0000","last_fullsized":"2024-04-24T06:35:55.897904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:49:32.531310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890304+0000","last_change":"2024-04-24T06:35:54.891233+0000","last_active":"2024-04-24T06:35:55.890304+0000","last_peered":"2024-04-24T06:35:55.890304+0000","last_clean":"2024-04-24T06:35:55.890304+0000","last_became_active":"2024-04-24T06:35:54.890879+0000","last_became_peered":"2024-04-24T06:35:54.890879+0000","last_unstale":"2024-04-24T06:35:55.890304+0000","last_undegraded":"2024-04-24T06:35:55.890304+0000","last_fullsized":"2024-04-24T06:35:55.890304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:15.212844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.412057+0000","last_change":"2024-04-24T06:35:53.864248+0000","last_active":"2024-04-24T06:35:56.412057+0000","last_peered":"2024-04-24T06:35:56.412057+0000","last_clean":"2024-04-24T06:35:56.412057+0000","last_became_active":"2024-04-24T06:35:53.862648+0000","last_became_peered":"2024-04-24T06:35:53.862648+0000","last_unstale":"2024-04-24T06:35:56.412057+0000","last_undegraded":"2024-04-24T06:35:56.412057+0000","last_fullsized":"2024-04-24T06:35:56.412057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:31:35.094858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.411706+0000","last_change":"2024-04-24T06:35:54.904758+0000","last_active":"2024-04-24T06:35:56.411706+0000","last_peered":"2024-04-24T06:35:56.411706+0000","last_clean":"2024-04-24T06:35:56.411706+0000","last_became_active":"2024-04-24T06:35:54.904478+0000","last_became_peered":"2024-04-24T06:35:54.904478+0000","last_unstale":"2024-04-24T06:35:56.411706+0000","last_undegraded":"2024-04-24T06:35:56.411706+0000","last_fullsized":"2024-04-24T06:35:56.411706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:22:20.940853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898642+0000","last_change":"2024-04-24T06:35:53.862200+0000","last_active":"2024-04-24T06:35:55.898642+0000","last_peered":"2024-04-24T06:35:55.898642+0000","last_clean":"2024-04-24T06:35:55.898642+0000","last_became_active":"2024-04-24T06:35:53.861362+0000","last_became_peered":"2024-04-24T06:35:53.861362+0000","last_unstale":"2024-04-24T06:35:55.898642+0000","last_undegraded":"2024-04-24T06:35:55.898642+0000","last_fullsized":"2024-04-24T06:35:55.898642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:38:23.118640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408639+0000","last_change":"2024-04-24T06:35:54.894832+0000","last_active":"2024-04-24T06:35:56.408639+0000","last_peered":"2024-04-24T06:35:56.408639+0000","last_clean":"2024-04-24T06:35:56.408639+0000","last_became_active":"2024-04-24T06:35:54.894531+0000","last_became_peered":"2024-04-24T06:35:54.894531+0000","last_unstale":"2024-04-24T06:35:56.408639+0000","last_undegraded":"2024-04-24T06:35:56.408639+0000","last_fullsized":"2024-04-24T06:35:56.408639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:25:47.855861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898533+0000","last_change":"2024-04-24T06:35:53.862034+0000","last_active":"2024-04-24T06:35:55.898533+0000","last_peered":"2024-04-24T06:35:55.898533+0000","last_clean":"2024-04-24T06:35:55.898533+0000","last_became_active":"2024-04-24T06:35:53.861731+0000","last_became_peered":"2024-04-24T06:35:53.861731+0000","last_unstale":"2024-04-24T06:35:55.898533+0000","last_undegraded":"2024-04-24T06:35:55.898533+0000","last_fullsized":"2024-04-24T06:35:55.898533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:36.322811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407814+0000","last_change":"2024-04-24T06:35:54.911000+0000","last_active":"2024-04-24T06:35:56.407814+0000","last_peered":"2024-04-24T06:35:56.407814+0000","last_clean":"2024-04-24T06:35:56.407814+0000","last_became_active":"2024-04-24T06:35:54.910675+0000","last_became_peered":"2024-04-24T06:35:54.910675+0000","last_unstale":"2024-04-24T06:35:56.407814+0000","last_undegraded":"2024-04-24T06:35:56.407814+0000","last_fullsized":"2024-04-24T06:35:56.407814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:37:32.378681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.888706+0000","last_change":"2024-04-24T06:35:53.861051+0000","last_active":"2024-04-24T06:35:55.888706+0000","last_peered":"2024-04-24T06:35:55.888706+0000","last_clean":"2024-04-24T06:35:55.888706+0000","last_became_active":"2024-04-24T06:35:53.859883+0000","last_became_peered":"2024-04-24T06:35:53.859883+0000","last_unstale":"2024-04-24T06:35:55.888706+0000","last_undegraded":"2024-04-24T06:35:55.888706+0000","last_fullsized":"2024-04-24T06:35:55.888706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:14:35.131586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.198627+0000","last_change":"2024-04-24T06:35:54.889025+0000","last_active":"2024-04-24T06:35:56.198627+0000","last_peered":"2024-04-24T06:35:56.198627+0000","last_clean":"2024-04-24T06:35:56.198627+0000","last_became_active":"2024-04-24T06:35:54.888595+0000","last_became_peered":"2024-04-24T06:35:54.888595+0000","last_unstale":"2024-04-24T06:35:56.198627+0000","last_undegraded":"2024-04-24T06:35:56.198627+0000","last_fullsized":"2024-04-24T06:35:56.198627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:11.418688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407973+0000","last_change":"2024-04-24T06:35:53.861248+0000","last_active":"2024-04-24T06:35:56.407973+0000","last_peered":"2024-04-24T06:35:56.407973+0000","last_clean":"2024-04-24T06:35:56.407973+0000","last_became_active":"2024-04-24T06:35:53.859574+0000","last_became_peered":"2024-04-24T06:35:53.859574+0000","last_unstale":"2024-04-24T06:35:56.407973+0000","last_undegraded":"2024-04-24T06:35:56.407973+0000","last_fullsized":"2024-04-24T06:35:56.407973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:02.318840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408163+0000","last_change":"2024-04-24T06:35:54.887804+0000","last_active":"2024-04-24T06:35:56.408163+0000","last_peered":"2024-04-24T06:35:56.408163+0000","last_clean":"2024-04-24T06:35:56.408163+0000","last_became_active":"2024-04-24T06:35:54.887453+0000","last_became_peered":"2024-04-24T06:35:54.887453+0000","last_unstale":"2024-04-24T06:35:56.408163+0000","last_undegraded":"2024-04-24T06:35:56.408163+0000","last_fullsized":"2024-04-24T06:35:56.408163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:23:02.435317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407169+0000","last_change":"2024-04-24T06:35:53.866230+0000","last_active":"2024-04-24T06:35:56.407169+0000","last_peered":"2024-04-24T06:35:56.407169+0000","last_clean":"2024-04-24T06:35:56.407169+0000","last_became_active":"2024-04-24T06:35:53.865952+0000","last_became_peered":"2024-04-24T06:35:53.865952+0000","last_unstale":"2024-04-24T06:35:56.407169+0000","last_undegraded":"2024-04-24T06:35:56.407169+0000","last_fullsized":"2024-04-24T06:35:56.407169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:09:30.270749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.199084+0000","last_change":"2024-04-24T06:35:54.891989+0000","last_active":"2024-04-24T06:35:56.199084+0000","last_peered":"2024-04-24T06:35:56.199084+0000","last_clean":"2024-04-24T06:35:56.199084+0000","last_became_active":"2024-04-24T06:35:54.891588+0000","last_became_peered":"2024-04-24T06:35:54.891588+0000","last_unstale":"2024-04-24T06:35:56.199084+0000","last_undegraded":"2024-04-24T06:35:56.199084+0000","last_fullsized":"2024-04-24T06:35:56.199084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:41:49.646192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.888346+0000","last_change":"2024-04-24T06:35:53.851462+0000","last_active":"2024-04-24T06:35:55.888346+0000","last_peered":"2024-04-24T06:35:55.888346+0000","last_clean":"2024-04-24T06:35:55.888346+0000","last_became_active":"2024-04-24T06:35:53.851142+0000","last_became_peered":"2024-04-24T06:35:53.851142+0000","last_unstale":"2024-04-24T06:35:55.888346+0000","last_undegraded":"2024-04-24T06:35:55.888346+0000","last_fullsized":"2024-04-24T06:35:55.888346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:10.858638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409505+0000","last_change":"2024-04-24T06:35:54.880332+0000","last_active":"2024-04-24T06:35:56.409505+0000","last_peered":"2024-04-24T06:35:56.409505+0000","last_clean":"2024-04-24T06:35:56.409505+0000","last_became_active":"2024-04-24T06:35:54.880017+0000","last_became_peered":"2024-04-24T06:35:54.880017+0000","last_unstale":"2024-04-24T06:35:56.409505+0000","last_undegraded":"2024-04-24T06:35:56.409505+0000","last_fullsized":"2024-04-24T06:35:56.409505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:11.875714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893752+0000","last_change":"2024-04-24T06:35:53.864453+0000","last_active":"2024-04-24T06:35:55.893752+0000","last_peered":"2024-04-24T06:35:55.893752+0000","last_clean":"2024-04-24T06:35:55.893752+0000","last_became_active":"2024-04-24T06:35:53.864134+0000","last_became_peered":"2024-04-24T06:35:53.864134+0000","last_unstale":"2024-04-24T06:35:55.893752+0000","last_undegraded":"2024-04-24T06:35:55.893752+0000","last_fullsized":"2024-04-24T06:35:55.893752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:34.996404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890471+0000","last_change":"2024-04-24T06:35:54.905353+0000","last_active":"2024-04-24T06:35:55.890471+0000","last_peered":"2024-04-24T06:35:55.890471+0000","last_clean":"2024-04-24T06:35:55.890471+0000","last_became_active":"2024-04-24T06:35:54.905093+0000","last_became_peered":"2024-04-24T06:35:54.905093+0000","last_unstale":"2024-04-24T06:35:55.890471+0000","last_undegraded":"2024-04-24T06:35:55.890471+0000","last_fullsized":"2024-04-24T06:35:55.890471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:22:40.593997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893738+0000","last_change":"2024-04-24T06:35:53.854561+0000","last_active":"2024-04-24T06:35:55.893738+0000","last_peered":"2024-04-24T06:35:55.893738+0000","last_clean":"2024-04-24T06:35:55.893738+0000","last_became_active":"2024-04-24T06:35:53.854281+0000","last_became_peered":"2024-04-24T06:35:53.854281+0000","last_unstale":"2024-04-24T06:35:55.893738+0000","last_undegraded":"2024-04-24T06:35:55.893738+0000","last_fullsized":"2024-04-24T06:35:55.893738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:44.403863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410494+0000","last_change":"2024-04-24T06:35:54.902591+0000","last_active":"2024-04-24T06:35:56.410494+0000","last_peered":"2024-04-24T06:35:56.410494+0000","last_clean":"2024-04-24T06:35:56.410494+0000","last_became_active":"2024-04-24T06:35:54.896300+0000","last_became_peered":"2024-04-24T06:35:54.896300+0000","last_unstale":"2024-04-24T06:35:56.410494+0000","last_undegraded":"2024-04-24T06:35:56.410494+0000","last_fullsized":"2024-04-24T06:35:56.410494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:25:47.855861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410620+0000","last_change":"2024-04-24T06:35:53.866001+0000","last_active":"2024-04-24T06:35:56.410620+0000","last_peered":"2024-04-24T06:35:56.410620+0000","last_clean":"2024-04-24T06:35:56.410620+0000","last_became_active":"2024-04-24T06:35:53.865148+0000","last_became_peered":"2024-04-24T06:35:53.865148+0000","last_unstale":"2024-04-24T06:35:56.410620+0000","last_undegraded":"2024-04-24T06:35:56.410620+0000","last_fullsized":"2024-04-24T06:35:56.410620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:28.649778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408900+0000","last_change":"2024-04-24T06:35:54.881977+0000","last_active":"2024-04-24T06:35:56.408900+0000","last_peered":"2024-04-24T06:35:56.408900+0000","last_clean":"2024-04-24T06:35:56.408900+0000","last_became_active":"2024-04-24T06:35:54.881656+0000","last_became_peered":"2024-04-24T06:35:54.881656+0000","last_unstale":"2024-04-24T06:35:56.408900+0000","last_undegraded":"2024-04-24T06:35:56.408900+0000","last_fullsized":"2024-04-24T06:35:56.408900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:54.609914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.410630+0000","last_change":"2024-04-24T06:35:53.863435+0000","last_active":"2024-04-24T06:35:56.410630+0000","last_peered":"2024-04-24T06:35:56.410630+0000","last_clean":"2024-04-24T06:35:56.410630+0000","last_became_active":"2024-04-24T06:35:53.862957+0000","last_became_peered":"2024-04-24T06:35:53.862957+0000","last_unstale":"2024-04-24T06:35:56.410630+0000","last_undegraded":"2024-04-24T06:35:56.410630+0000","last_fullsized":"2024-04-24T06:35:56.410630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:43.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.198903+0000","last_change":"2024-04-24T06:35:54.887096+0000","last_active":"2024-04-24T06:35:56.198903+0000","last_peered":"2024-04-24T06:35:56.198903+0000","last_clean":"2024-04-24T06:35:56.198903+0000","last_became_active":"2024-04-24T06:35:54.886724+0000","last_became_peered":"2024-04-24T06:35:54.886724+0000","last_unstale":"2024-04-24T06:35:56.198903+0000","last_undegraded":"2024-04-24T06:35:56.198903+0000","last_fullsized":"2024-04-24T06:35:56.198903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:22:45.957285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898354+0000","last_change":"2024-04-24T06:35:53.866392+0000","last_active":"2024-04-24T06:35:55.898354+0000","last_peered":"2024-04-24T06:35:55.898354+0000","last_clean":"2024-04-24T06:35:55.898354+0000","last_became_active":"2024-04-24T06:35:53.864763+0000","last_became_peered":"2024-04-24T06:35:53.864763+0000","last_unstale":"2024-04-24T06:35:55.898354+0000","last_undegraded":"2024-04-24T06:35:55.898354+0000","last_fullsized":"2024-04-24T06:35:55.898354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:06:34.529244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.898805+0000","last_change":"2024-04-24T06:35:54.885735+0000","last_active":"2024-04-24T06:35:55.898805+0000","last_peered":"2024-04-24T06:35:55.898805+0000","last_clean":"2024-04-24T06:35:55.898805+0000","last_became_active":"2024-04-24T06:35:54.879416+0000","last_became_peered":"2024-04-24T06:35:54.879416+0000","last_unstale":"2024-04-24T06:35:55.898805+0000","last_undegraded":"2024-04-24T06:35:55.898805+0000","last_fullsized":"2024-04-24T06:35:55.898805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:46.413234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.408136+0000","last_change":"2024-04-24T06:35:53.857052+0000","last_active":"2024-04-24T06:35:56.408136+0000","last_peered":"2024-04-24T06:35:56.408136+0000","last_clean":"2024-04-24T06:35:56.408136+0000","last_became_active":"2024-04-24T06:35:53.856745+0000","last_became_peered":"2024-04-24T06:35:53.856745+0000","last_unstale":"2024-04-24T06:35:56.408136+0000","last_undegraded":"2024-04-24T06:35:56.408136+0000","last_fullsized":"2024-04-24T06:35:56.408136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:09.388126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890953+0000","last_change":"2024-04-24T06:35:54.901164+0000","last_active":"2024-04-24T06:35:55.890953+0000","last_peered":"2024-04-24T06:35:55.890953+0000","last_clean":"2024-04-24T06:35:55.890953+0000","last_became_active":"2024-04-24T06:35:54.900841+0000","last_became_peered":"2024-04-24T06:35:54.900841+0000","last_unstale":"2024-04-24T06:35:55.890953+0000","last_undegraded":"2024-04-24T06:35:55.890953+0000","last_fullsized":"2024-04-24T06:35:55.890953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:14:36.148662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409791+0000","last_change":"2024-04-24T06:35:53.859118+0000","last_active":"2024-04-24T06:35:56.409791+0000","last_peered":"2024-04-24T06:35:56.409791+0000","last_clean":"2024-04-24T06:35:56.409791+0000","last_became_active":"2024-04-24T06:35:53.858803+0000","last_became_peered":"2024-04-24T06:35:53.858803+0000","last_unstale":"2024-04-24T06:35:56.409791+0000","last_undegraded":"2024-04-24T06:35:56.409791+0000","last_fullsized":"2024-04-24T06:35:56.409791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:37.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.409626+0000","last_change":"2024-04-24T06:35:54.897013+0000","last_active":"2024-04-24T06:35:56.409626+0000","last_peered":"2024-04-24T06:35:56.409626+0000","last_clean":"2024-04-24T06:35:56.409626+0000","last_became_active":"2024-04-24T06:35:54.896640+0000","last_became_peered":"2024-04-24T06:35:54.896640+0000","last_unstale":"2024-04-24T06:35:56.409626+0000","last_undegraded":"2024-04-24T06:35:56.409626+0000","last_fullsized":"2024-04-24T06:35:56.409626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:41:49.646192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:56.407449+0000","last_change":"2024-04-24T06:35:53.861767+0000","last_active":"2024-04-24T06:35:56.407449+0000","last_peered":"2024-04-24T06:35:56.407449+0000","last_clean":"2024-04-24T06:35:56.407449+0000","last_became_active":"2024-04-24T06:35:53.861215+0000","last_became_peered":"2024-04-24T06:35:53.861215+0000","last_unstale":"2024-04-24T06:35:56.407449+0000","last_undegraded":"2024-04-24T06:35:56.407449+0000","last_fullsized":"2024-04-24T06:35:56.407449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:43.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.893352+0000","last_change":"2024-04-24T06:35:54.896973+0000","last_active":"2024-04-24T06:35:55.893352+0000","last_peered":"2024-04-24T06:35:55.893352+0000","last_clean":"2024-04-24T06:35:55.893352+0000","last_became_active":"2024-04-24T06:35:54.896674+0000","last_became_peered":"2024-04-24T06:35:54.896674+0000","last_unstale":"2024-04-24T06:35:55.893352+0000","last_undegraded":"2024-04-24T06:35:55.893352+0000","last_fullsized":"2024-04-24T06:35:55.893352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:53.817900+0000","last_clean_scrub_stamp":"2024-04-24T06:35:53.817900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:14.121423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.890308+0000","last_change":"2024-04-24T06:35:53.868333+0000","last_active":"2024-04-24T06:35:55.890308+0000","last_peered":"2024-04-24T06:35:55.890308+0000","last_clean":"2024-04-24T06:35:55.890308+0000","last_became_active":"2024-04-24T06:35:53.867557+0000","last_became_peered":"2024-04-24T06:35:53.867557+0000","last_unstale":"2024-04-24T06:35:55.890308+0000","last_undegraded":"2024-04-24T06:35:55.890308+0000","last_fullsized":"2024-04-24T06:35:55.890308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:35:52.800823+0000","last_clean_scrub_stamp":"2024-04-24T06:35:52.800823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:49:32.531310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-24T06:35:55.889878+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:35:55.889878+0000","last_peered":"2024-04-24T06:35:55.889878+0000","last_clean":"2024-04-24T06:35:55.889878+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:35:55.889878+0000","last_undegraded":"2024-04-24T06:35:55.889878+0000","last_fullsized":"2024-04-24T06:35:55.889878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1026128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":436384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":437324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":435690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":437302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":436272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":435884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1026092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:35:59.872 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:35:59.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:36:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.135+0000 7f63b8a00640 1 Processor -- start 2024-04-24T06:36:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.135+0000 7f63b8a00640 1 -- start start 2024-04-24T06:36:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.137+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f63b406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.137+0000 7f63b8a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b40cfc20 con 0x7f63b4071d70 2024-04-24T06:36:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.137+0000 7f63b8a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b40cff00 con 0x7f63b4145e10 2024-04-24T06:36:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.137+0000 7f63b8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40d01e0 con 0x7f63b40d0cc0 2024-04-24T06:36:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.138+0000 7f63b2800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f63b4071d70 0x7f63b415cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44386/0 (socket says 172.21.15.59:44386) 2024-04-24T06:36:00.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.138+0000 7f63b2800640 1 -- 172.21.15.59:0/2159588049 learned_addr learned my addr 172.21.15.59:0/2159588049 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.138+0000 7f63b1400640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f63b406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.138+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891109101 0 0) 0x7f63b40cfc20 con 0x7f63b4071d70 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.138+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6398003520 con 0x7f63b4071d70 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3506460286 0 0) 0x7f6398003520 con 0x7f63b4071d70 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 msgr2=0x7f63b406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f63b406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 >> v1:172.21.15.66:6789/0 conn(0x7f63b4145e10 legacy=0x7f63b40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b40d04c0 con 0x7f63b4071d70 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f63ac0019f0 con 0x7f63b4071d70 2024-04-24T06:36:00.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 >> v1:172.21.15.59:6789/0 conn(0x7f63b4071d70 legacy=0x7f63b415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f6398003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f6398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6398004d40 0x7f63980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40d01e0 con 0x7f63980039e0 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f639800ffe0 con 0x7f6398004d40 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f63980101e0 con 0x7f63b40d0cc0 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f6398003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.139+0000 7f63b1400640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f6398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b2800640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6398004d40 0x7f63980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b1e00640 1 -- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 msgr2=0x7f6398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b40d0cc0 0x7f6398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b1e00640 1 -- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6398004d40 msgr2=0x7f63980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6398004d40 0x7f63980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b1e00640 1 -- 172.21.15.59:0/2159588049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6398003520 con 0x7f63980039e0 2024-04-24T06:36:00.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.140+0000 7f63b2800640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6398004d40 0x7f63980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:00.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.141+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f6398003e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f63a4001750 tx=0x7f6398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc4ec1633bf9a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.141+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4001370 con 0x7f63980039e0 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.141+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a40052d0 con 0x7f63980039e0 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.141+0000 7f63b0a00640 1 -- 172.21.15.59:0/2159588049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a402abe0 con 0x7f63980039e0 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 -- 172.21.15.59:0/2159588049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 msgr2=0x7f6398003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2159588049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f6398003e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f63a4001750 tx=0x7f6398005830 comp rx=0 tx=0).stop 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b2800640 1 -- 172.21.15.59:0/2159588049 reap_dead start 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 -- 172.21.15.59:0/2159588049 shutdown_connections 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 -- 172.21.15.59:0/2159588049 >> 172.21.15.59:0/2159588049 conn(0x7f63b4067d90 msgr2=0x7f63b406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 -- 172.21.15.59:0/2159588049 shutdown_connections 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.142+0000 7f63b8a00640 1 -- 172.21.15.59:0/2159588049 wait complete. 2024-04-24T06:36:00.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.144+0000 7f63b8a00640 1 Processor -- start 2024-04-24T06:36:00.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.144+0000 7f63b8a00640 1 -- start start 2024-04-24T06:36:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415ea50 0x7f63b4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b41408c0 0x7f63b415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41461f0 con 0x7f63980039e0 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63b406f130 con 0x7f63b415ea50 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f63b413bff0 con 0x7f63b41408c0 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35008/0 (socket says 172.21.15.59:35008) 2024-04-24T06:36:00.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 -- 172.21.15.59:0/2232430099 learned_addr learned my addr 172.21.15.59:0/2232430099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1400640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415ea50 0x7f63b4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b2800640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b41408c0 0x7f63b415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 -- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b41408c0 msgr2=0x7f63b415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b41408c0 0x7f63b415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 -- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415ea50 msgr2=0x7f63b4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415ea50 0x7f63b4140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.145+0000 7f63b1e00640 1 -- 172.21.15.59:0/2232430099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b413c2d0 con 0x7f63980039e0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63b1e00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4042070 con 0x7f63980039e0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4005480 con 0x7f63980039e0 2024-04-24T06:36:00.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4037040 con 0x7f63980039e0 2024-04-24T06:36:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b413c5b0 con 0x7f63980039e0 2024-04-24T06:36:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.146+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b413cb70 con 0x7f63980039e0 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.148+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b4140cd0 con 0x7f63980039e0 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.148+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63a40069d0 con 0x7f63980039e0 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.149+0000 7f63aa000640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.150+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f63a40ab450 con 0x7f63980039e0 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.150+0000 7f63b1400640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.150+0000 7f63b1400640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 0x7f639406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f63b41f3c50 tx=0x7f63a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.154+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63b4140cd0 con 0x7f63980039e0 2024-04-24T06:36:00.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.430+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f63b4068a80 con 0x7f63980039e0 2024-04-24T06:36:00.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.436+0000 7f63aa000640 1 -- 172.21.15.59:0/2232430099 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f63a4079750 con 0x7f63980039e0 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:36:00.438 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:36:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 msgr2=0x7f639406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 0x7f639406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f63b41f3c50 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2024-04-24T06:36:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 msgr2=0x7f63b4137e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4001120 comp rx=0 tx=0).stop 2024-04-24T06:36:00.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 shutdown_connections 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f639406c8f0 0x7f639406edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f63b41408c0 0x7f63b415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63b415ea50 0x7f63b4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 --2- 172.21.15.59:0/2232430099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63980039e0 0x7f63b4137e50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 >> 172.21.15.59:0/2232430099 conn(0x7f63b4067d90 msgr2=0x7f63b415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 shutdown_connections 2024-04-24T06:36:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.439+0000 7f63b8a00640 1 -- 172.21.15.59:0/2232430099 wait complete. 2024-04-24T06:36:00.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:00.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.670+0000 7f5326800640 1 Processor -- start 2024-04-24T06:36:00.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.670+0000 7f5326800640 1 -- start start 2024-04-24T06:36:00.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.671+0000 7f5326800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f53200d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.671+0000 7f5326800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5320160180 0x7f53200d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.671+0000 7f5326800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53200cff50 con 0x7f532015cb20 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.671+0000 7f5326800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f532015cf00 con 0x7f5320160180 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.671+0000 7f5326800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f53200d01c0 con 0x7f5320163770 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f532015cb20 0x7f53200cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44396/0 (socket says 172.21.15.59:44396) 2024-04-24T06:36:00.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531f400640 1 -- 172.21.15.59:0/2620429551 learned_addr learned my addr 172.21.15.59:0/2620429551 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:00.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531fe00640 1 --2- 172.21.15.59:0/2620429551 >> v2:172.21.15.66:3300/0 conn(0x7f5320160180 0x7f53200d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531ea00640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f53200d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077673413 0 0) 0x7f53200cff50 con 0x7f532015cb20 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.672+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5304003520 con 0x7f532015cb20 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3225438144 0 0) 0x7f5304003520 con 0x7f532015cb20 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 msgr2=0x7f53200d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f53200d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 >> v2:172.21.15.66:3300/0 conn(0x7f5320160180 msgr2=0x7f53200d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 --2- 172.21.15.59:0/2620429551 >> v2:172.21.15.66:3300/0 conn(0x7f5320160180 0x7f53200d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532015f6a0 con 0x7f532015cb20 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f53100019f0 con 0x7f532015cb20 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 >> v1:172.21.15.59:6789/0 conn(0x7f532015cb20 legacy=0x7f53200cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53040039b0 0x7f5304003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f5304004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5304004d40 0x7f53040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f532015cf00 con 0x7f53040039b0 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53200d01c0 con 0x7f5304004d40 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f530400b9f0 con 0x7f5320163770 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.673+0000 7f531f400640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53040039b0 0x7f5304003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531ea00640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f5304004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531fe00640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5304004d40 0x7f53040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531f400640 1 -- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 msgr2=0x7f5304004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531f400640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f5304004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531f400640 1 -- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5304004d40 msgr2=0x7f53040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531f400640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5304004d40 0x7f53040051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531f400640 1 -- 172.21.15.59:0/2620429551 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5304003520 con 0x7f53040039b0 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.674+0000 7f531ea00640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5320163770 0x7f5304004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:00.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.675+0000 7f531f400640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53040039b0 0x7f5304003dd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f531000a7a0 tx=0x7f5304005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ebb595ea110621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.675+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531000a3f0 con 0x7f53040039b0 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.675+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531002f070 con 0x7f53040039b0 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.675+0000 7f531e000640 1 -- 172.21.15.59:0/2620429551 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531002bca0 con 0x7f53040039b0 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 -- 172.21.15.59:0/2620429551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53040039b0 msgr2=0x7f5304003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 --2- 172.21.15.59:0/2620429551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53040039b0 0x7f5304003dd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f531000a7a0 tx=0x7f5304005830 comp rx=0 tx=0).stop 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f531fe00640 1 -- 172.21.15.59:0/2620429551 reap_dead start 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 -- 172.21.15.59:0/2620429551 shutdown_connections 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 -- 172.21.15.59:0/2620429551 >> 172.21.15.59:0/2620429551 conn(0x7f532006cc70 msgr2=0x7f532006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 -- 172.21.15.59:0/2620429551 shutdown_connections 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.676+0000 7f5326800640 1 -- 172.21.15.59:0/2620429551 wait complete. 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.678+0000 7f5326800640 1 Processor -- start 2024-04-24T06:36:00.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.678+0000 7f5326800640 1 -- start start 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 0x7f53200c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53200caef0 0x7f53201e9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53200d13c0 con 0x7f532014ea60 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f53200d1240 con 0x7f53200caef0 2024-04-24T06:36:00.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f5326800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f53200d1540 con 0x7f532014e2b0 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f531fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f531fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35042/0 (socket says 172.21.15.59:35042) 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.679+0000 7f531fe00640 1 -- 172.21.15.59:0/4044794044 learned_addr learned my addr 172.21.15.59:0/4044794044 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531f400640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 0x7f53200c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 -- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 msgr2=0x7f53200c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531ea00640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53200caef0 0x7f53201e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 0x7f53200c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 -- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53200caef0 msgr2=0x7f53201e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53200caef0 0x7f53201e9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 -- 172.21.15.59:0/4044794044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53200c9e50 con 0x7f532014ea60 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531f400640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 0x7f53200c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f531fe00640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5318010790 tx=0x7f5318010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800f040 con 0x7f532014ea60 2024-04-24T06:36:00.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531801c070 con 0x7f532014ea60 2024-04-24T06:36:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800eb10 con 0x7f532014ea60 2024-04-24T06:36:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.680+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53200ca130 con 0x7f532014ea60 2024-04-24T06:36:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.681+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53200ca6f0 con 0x7f532014ea60 2024-04-24T06:36:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.682+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5320069200 con 0x7f532014ea60 2024-04-24T06:36:00.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.683+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f531800ecb0 con 0x7f532014ea60 2024-04-24T06:36:00.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.684+0000 7f5317400640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 0x7f530006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:00.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.684+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f531808b6b0 con 0x7f532014ea60 2024-04-24T06:36:00.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.684+0000 7f531f400640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 0x7f530006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:00.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.685+0000 7f531f400640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 0x7f530006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5310033000 tx=0x7f5310038040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:00.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.689+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5320069200 con 0x7f532014ea60 2024-04-24T06:36:00.804 INFO:tasks.ceph.mds.d.smithi066.stderr:2024-04-24T06:36:00.803+0000 7f40f6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:36:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.935+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53200d29c0 con 0x7f532014ea60 2024-04-24T06:36:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.936+0000 7f5317400640 1 -- 172.21.15.59:0/4044794044 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5177 (secure 0 0 0) 0x7f5318026020 con 0x7f532014ea60 2024-04-24T06:36:00.938 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:00.938 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7548,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:35:57.406140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6833/301885470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":301885470},{"type":"v1","addr":"172.21.15.66:6833","nonce":301885470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7563,"qdb_cluster":[7563]},"id":6}]} 2024-04-24T06:36:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 61 2024-04-24T06:36:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 msgr2=0x7f530006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 0x7f530006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5310033000 tx=0x7f5310038040 comp rx=0 tx=0).stop 2024-04-24T06:36:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 msgr2=0x7f532014eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5318010790 tx=0x7f5318010b90 comp rx=0 tx=0).stop 2024-04-24T06:36:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 shutdown_connections 2024-04-24T06:36:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f530006c8f0 0x7f530006edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f532014ea60 0x7f532014eee0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53200caef0 0x7f53201e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 --2- 172.21.15.59:0/4044794044 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f532014e2b0 0x7f53200c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 >> 172.21.15.59:0/4044794044 conn(0x7f532006cc70 msgr2=0x7f532015e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 shutdown_connections 2024-04-24T06:36:00.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:00.939+0000 7f5326800640 1 -- 172.21.15.59:0/4044794044 wait complete. 2024-04-24T06:36:01.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:35:57.406140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7563': {'gid': 7563, 'name': 'd', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6833/301885470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 301885470}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 301885470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7563, 'qdb_cluster': [7563]} 2024-04-24T06:36:01.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:36:01.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T06:36:01.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.585+0000 7f6e56200640 1 Processor -- start 2024-04-24T06:36:01.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.586+0000 7f6e56200640 1 -- start start 2024-04-24T06:36:01.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.587+0000 7f6e56200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6e500d0f80 0x7f6e500cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.587+0000 7f6e56200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e5015fd30 con 0x7f6e5015ea40 2024-04-24T06:36:01.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.587+0000 7f6e56200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e500cf930 con 0x7f6e5015f880 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.587+0000 7f6e56200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6e500cfc10 con 0x7f6e500d0f80 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.588+0000 7f6e4fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6e5015ea40 0x7f6e500d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44410/0 (socket says 172.21.15.59:44410) 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.588+0000 7f6e4fe00640 1 -- 172.21.15.59:0/4055582659 learned_addr learned my addr 172.21.15.59:0/4055582659 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.588+0000 7f6e54e00640 1 --2- 172.21.15.59:0/4055582659 >> v2:172.21.15.66:3300/0 conn(0x7f6e500d0f80 0x7f6e500cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.588+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311083461 0 0) 0x7f6e5015fd30 con 0x7f6e5015ea40 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.588+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e34003520 con 0x7f6e5015ea40 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1707348820 0 0) 0x7f6e34003520 con 0x7f6e5015ea40 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 >> v2:172.21.15.66:3300/0 conn(0x7f6e500d0f80 msgr2=0x7f6e500cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 --2- 172.21.15.59:0/4055582659 >> v2:172.21.15.66:3300/0 conn(0x7f6e500d0f80 0x7f6e500cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 >> v1:172.21.15.66:6789/0 conn(0x7f6e5015f880 legacy=0x7f6e500d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e500cfef0 con 0x7f6e5015ea40 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6e3c001670 con 0x7f6e5015ea40 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 >> v1:172.21.15.59:6789/0 conn(0x7f6e5015ea40 legacy=0x7f6e500d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 0x7f6e34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34005210 0x7f6e3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500cfc10 con 0x7f6e34003a00 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3400c160 con 0x7f6e34004800 2024-04-24T06:36:01.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3400c360 con 0x7f6e34005210 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e54e00640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 0x7f6e34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.589+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e54e00640 1 -- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34005210 msgr2=0x7f6e3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e54e00640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34005210 0x7f6e3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e54e00640 1 -- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e34004800 msgr2=0x7f6e34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e54e00640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e34004800 0x7f6e34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e54e00640 1 -- 172.21.15.59:0/4055582659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34003520 con 0x7f6e34003a00 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.590+0000 7f6e55800640 1 -- 172.21.15.59:0/4055582659 reap_dead start 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e54e00640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 0x7f6e34003e20 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009f90 tx=0x7f6e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc8f8cabf68de5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40009c10 con 0x7f6e34003a00 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40036070 con 0x7f6e34003a00 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e4f400640 1 -- 172.21.15.59:0/4055582659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40031850 con 0x7f6e34003a00 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e56200640 1 -- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 msgr2=0x7f6e34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.591+0000 7f6e56200640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 0x7f6e34003e20 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009f90 tx=0x7f6e34005830 comp rx=0 tx=0).stop 2024-04-24T06:36:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.592+0000 7f6e56200640 1 -- 172.21.15.59:0/4055582659 shutdown_connections 2024-04-24T06:36:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.592+0000 7f6e56200640 1 --2- 172.21.15.59:0/4055582659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e34003a00 0x7f6e34003e20 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.592+0000 7f6e56200640 1 -- 172.21.15.59:0/4055582659 >> 172.21.15.59:0/4055582659 conn(0x7f6e5008bf90 msgr2=0x7f6e500d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.592+0000 7f6e56200640 1 -- 172.21.15.59:0/4055582659 shutdown_connections 2024-04-24T06:36:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.592+0000 7f6e56200640 1 -- 172.21.15.59:0/4055582659 wait complete. 2024-04-24T06:36:01.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.593+0000 7f6e56200640 1 Processor -- start 2024-04-24T06:36:01.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.594+0000 7f6e56200640 1 -- start start 2024-04-24T06:36:01.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.594+0000 7f6e56200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34003a00 0x7f6e5014c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.594+0000 7f6e56200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e500cad00 0x7f6e500c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e56200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e56200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500cf250 con 0x7f6e500cb180 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e56200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500d04b0 con 0x7f6e500cad00 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e56200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6e500cf3d0 con 0x7f6e34003a00 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35058/0 (socket says 172.21.15.59:35058) 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 -- 172.21.15.59:0/351779438 learned_addr learned my addr 172.21.15.59:0/351779438 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e54e00640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34003a00 0x7f6e5014c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e500cad00 0x7f6e500c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 -- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34003a00 msgr2=0x7f6e5014c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34003a00 0x7f6e5014c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 -- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e500cad00 msgr2=0x7f6e500c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e500cad00 0x7f6e500c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.595+0000 7f6e55800640 1 -- 172.21.15.59:0/351779438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e500c88f0 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e55800640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6e500d1bf0 tx=0x7f6e4800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e4cc00640 1 -- 172.21.15.59:0/351779438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48004220 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e4cc00640 1 -- 172.21.15.59:0/351779438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e480043c0 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e4cc00640 1 -- 172.21.15.59:0/351779438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48007310 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e500c8bd0 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.596+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e500c9190 con 0x7f6e500cb180 2024-04-24T06:36:01.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.599+0000 7f6e4cc00640 1 -- 172.21.15.59:0/351779438 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e480057c0 con 0x7f6e500cb180 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.600+0000 7f6e4cc00640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 0x7f6e3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.600+0000 7f6e4cc00640 1 -- 172.21.15.59:0/351779438 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f6e4808aed0 con 0x7f6e500cb180 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.600+0000 7f6e54e00640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 0x7f6e3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.600+0000 7f6e5a674740 1 Processor -- start 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.601+0000 7f6e5a674740 1 -- start start 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.601+0000 7f6e54e00640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 0x7f6e3006ede0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e40035000 tx=0x7f6e4002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.601+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba76aaa0 0x55beba769e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76a360 0x55beba73f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e5a674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55beba6a4b10 con 0x55beba787980 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55beba64b2b0 con 0x55beba76aaa0 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55beba6a0ad0 con 0x55beba76a360 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35064/0 (socket says 172.21.15.59:35064) 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 -- 172.21.15.59:0/2987837057 learned_addr learned my addr 172.21.15.59:0/2987837057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba76aaa0 0x55beba769e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e55800640 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76a360 0x55beba73f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 -- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76a360 msgr2=0x55beba73f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76a360 0x55beba73f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 -- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba76aaa0 msgr2=0x55beba769e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba76aaa0 0x55beba769e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.602+0000 7f6e54e00640 1 -- 172.21.15.59:0/2987837057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beba3b72d0 con 0x55beba787980 2024-04-24T06:36:01.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.603+0000 7f6e54e00640 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000e200 tx=0x7f6e4002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77872e04b9a19f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.603+0000 7f6e45600640 1 -- 172.21.15.59:0/2987837057 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4003ce60 con 0x55beba787980 2024-04-24T06:36:01.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.603+0000 7f6e45600640 1 -- 172.21.15.59:0/2987837057 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40036070 con 0x55beba787980 2024-04-24T06:36:01.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.603+0000 7f6e45600640 1 -- 172.21.15.59:0/2987837057 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40003db0 con 0x55beba787980 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 -- 172.21.15.59:0/2987837057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 msgr2=0x55beba73a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000e200 tx=0x7f6e4002d000 comp rx=0 tx=0).stop 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 -- 172.21.15.59:0/2987837057 shutdown_connections 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76a360 0x55beba73f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba76aaa0 0x55beba769e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 --2- 172.21.15.59:0/2987837057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba73a310 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 -- 172.21.15.59:0/2987837057 >> 172.21.15.59:0/2987837057 conn(0x55beba73c580 msgr2=0x55beba784e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 -- 172.21.15.59:0/2987837057 shutdown_connections 2024-04-24T06:36:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.604+0000 7f6e5a674740 1 -- 172.21.15.59:0/2987837057 wait complete. 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.605+0000 7f6e5a674740 1 -- start start 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.605+0000 7f6e5a674740 1 Processor -- start 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba75a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba76b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55beba6a4b10 con 0x55beba77c8a0 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55beba65e590 con 0x55beba787980 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55beba73e730 con 0x55beba76aaa0 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e5a674740 10 client.0 _ll_register_callbacks cb 0x55beba6413a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35078/0 (socket says 172.21.15.59:35078) 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.606+0000 7f6e55800640 1 -- 172.21.15.59:0/3456448409 learned_addr learned my addr 172.21.15.59:0/3456448409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 -- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 msgr2=0x55beba75a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba75a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 -- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 msgr2=0x55beba76b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba76b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 -- 172.21.15.59:0/3456448409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beba4816b0 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e55800640 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6e48089300 tx=0x7f6e480593a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acd72eac1328a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e3a000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48057fb0 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f6e48058150 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e40036070 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6e48089050 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e5a674740 10 client.7689 fetch_fsmap learned FSMap version 61 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.607+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55beba77c0a0 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40036070 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f6e4805f110 con 0x55beba77c8a0 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 10 client.7689 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 10 client.7689 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:36:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 20 client.7689 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba795300 0x55beba76d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 4 client.7689 mds_command: new command op to 7563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.608+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 --> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beba6b2530 con 0x55beba795300 2024-04-24T06:36:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.609+0000 7f6e54e00640 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba795300 0x55beba76d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.610+0000 7f6e54e00640 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba795300 0x55beba76d1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f630758676299b0b server_cookie=b400de0b79e2d7a7 in_seq=0 out_seq=0 2024-04-24T06:36:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.610+0000 7f6e3a000640 10 client.7689 ms_handle_connect on v2:172.21.15.66:6832/301885470 2024-04-24T06:36:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.612+0000 7f6e3a000640 1 -- 172.21.15.59:0/3456448409 <== mds.0 v2:172.21.15.66:6832/301885470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55beba6b2530 con 0x55beba795300 2024-04-24T06:36:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.612+0000 7f6e3a000640 10 client.7689 handle_command_reply: tid=0 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.612+0000 7f6e5a674740 1 client.7689 shutdown 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 msgr2=0x55beba784e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6e48089300 tx=0x7f6e480593a0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 shutdown_connections 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba795300 0x55beba76d1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba784e20 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba76b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 --2- 172.21.15.59:0/3456448409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba75a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e3a000640 0 client.7689 ms_handle_reset on v2:172.21.15.66:6832/301885470 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 >> 172.21.15.59:0/3456448409 conn(0x55beba76a360 msgr2=0x55beba767140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 shutdown_connections 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 1 -- 172.21.15.59:0/3456448409 wait complete. 2024-04-24T06:36:01.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.613+0000 7f6e5a674740 20 client.7689 trim_cache size 0 max 16384 2024-04-24T06:36:01.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.633+0000 7f6e5a674740 1 Processor -- start 2024-04-24T06:36:01.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.633+0000 7f6e5a674740 1 -- start start 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba73a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba7390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50200500 con 0x55beba77c8a0 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501ff860 con 0x55beba787980 2024-04-24T06:36:01.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.634+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6e501ffeb0 con 0x55beba76aaa0 2024-04-24T06:36:01.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35080/0 (socket says 172.21.15.59:35080) 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 -- 172.21.15.59:0/952876583 learned_addr learned my addr 172.21.15.59:0/952876583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e54e00640 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba73a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e55800640 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba7390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 -- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 msgr2=0x55beba73a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba73a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 -- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 msgr2=0x55beba7390e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba7390e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 -- 172.21.15.59:0/952876583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beba64a8b0 con 0x55beba77c8a0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.635+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001700 tx=0x7f6e3c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e980cda0689a2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e44c00640 1 -- 172.21.15.59:0/952876583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c010040 con 0x55beba77c8a0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e44c00640 1 -- 172.21.15.59:0/952876583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c01a070 con 0x55beba77c8a0 2024-04-24T06:36:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e44c00640 1 -- 172.21.15.59:0/952876583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011e20 con 0x55beba77c8a0 2024-04-24T06:36:01.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 -- 172.21.15.59:0/952876583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 msgr2=0x55beba77e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001700 tx=0x7f6e3c005bc0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 -- 172.21.15.59:0/952876583 shutdown_connections 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba787980 0x55beba7390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba77c8a0 0x55beba77e080 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 --2- 172.21.15.59:0/952876583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba73a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 -- 172.21.15.59:0/952876583 >> 172.21.15.59:0/952876583 conn(0x55beba76a360 msgr2=0x55beba75a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.636+0000 7f6e5a674740 1 -- 172.21.15.59:0/952876583 shutdown_connections 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.637+0000 7f6e5a674740 1 -- 172.21.15.59:0/952876583 wait complete. 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.637+0000 7f6e5a674740 1 -- start start 2024-04-24T06:36:01.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.637+0000 7f6e5a674740 1 Processor -- start 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba7e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba77c8a0 0x55beba7847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55beba795180 con 0x55beba787980 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55beba794c80 con 0x55beba77c8a0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.638+0000 7f6e5a674740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55beba7957d0 con 0x55beba76aaa0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e5a674740 10 client.0 _ll_register_callbacks cb 0x7f6e48054e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35088/0 (socket says 172.21.15.59:35088) 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 -- 172.21.15.59:0/1475209896 learned_addr learned my addr 172.21.15.59:0/1475209896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e4fe00640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba77c8a0 0x55beba7847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e54e00640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba7e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 -- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 msgr2=0x55beba7e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba7e47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 -- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba77c8a0 msgr2=0x55beba7847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba77c8a0 0x55beba7847e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.639+0000 7f6e55800640 1 -- 172.21.15.59:0/1475209896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beba77d9d0 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e55800640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6e48058860 tx=0x7f6e4801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444b2061af4a8d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4805cf10 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f6e480548b0 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c01a070 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6e48054ab0 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 10 client.7695 fetch_fsmap learned FSMap version 61 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55beba802ba0 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c01a070 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f6e48005ab0 con 0x55beba787980 2024-04-24T06:36:01.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 10 client.7695 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 10 client.7695 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 20 client.7695 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba77ae90 0x55beba7390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 4 client.7695 mds_command: new command op to 7563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.640+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 --> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] -- command(tid 0: {"prefix": "status"}) -- 0x55beba6ac3a0 con 0x55beba77ae90 2024-04-24T06:36:01.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.641+0000 7f6e54e00640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba77ae90 0x55beba7390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.642+0000 7f6e54e00640 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba77ae90 0x55beba7390e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf27518b601ac617 server_cookie=7f0617fc560aef07 in_seq=0 out_seq=0 2024-04-24T06:36:01.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.642+0000 7f6e3a000640 10 client.7695 ms_handle_connect on v2:172.21.15.66:6832/301885470 2024-04-24T06:36:01.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e3a000640 1 -- 172.21.15.59:0/1475209896 <== mds.0 v2:172.21.15.66:6832/301885470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55beba6ac3a0 con 0x55beba77ae90 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e3a000640 10 client.7695 handle_command_reply: tid=0 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 client.7695 shutdown 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 msgr2=0x55beba788240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6e48058860 tx=0x7f6e4801d040 comp rx=0 tx=0).stop 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 shutdown_connections 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:6832/301885470,v1:172.21.15.66:6833/301885470] conn(0x55beba77ae90 0x55beba7390e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55beba787980 0x55beba788240 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55beba77c8a0 0x55beba7847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e3a000640 0 client.7695 ms_handle_reset on v2:172.21.15.66:6832/301885470 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 --2- 172.21.15.59:0/1475209896 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55beba76aaa0 0x55beba7e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 >> 172.21.15.59:0/1475209896 conn(0x55beba76a360 msgr2=0x55beba767140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 shutdown_connections 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 1 -- 172.21.15.59:0/1475209896 wait complete. 2024-04-24T06:36:01.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.643+0000 7f6e5a674740 20 client.7695 trim_cache size 0 max 16384 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7563, 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7694346149999998, 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 61, 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 83, 2024-04-24T06:36:01.648 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 83, 2024-04-24T06:36:01.649 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.155432614 2024-04-24T06:36:01.649 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:01.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 msgr2=0x7f6e3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 0x7f6e3006ede0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e40035000 tx=0x7f6e4002f000 comp rx=0 tx=0).stop 2024-04-24T06:36:01.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 msgr2=0x7f6e500c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6e500d1bf0 tx=0x7f6e4800c820 comp rx=0 tx=0).stop 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 shutdown_connections 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6e3006c920 0x7f6e3006ede0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e500cb180 0x7f6e500c9ed0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e500cad00 0x7f6e500c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 --2- 172.21.15.59:0/351779438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6e34003a00 0x7f6e5014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 >> 172.21.15.59:0/351779438 conn(0x7f6e5008bf90 msgr2=0x7f6e50160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 shutdown_connections 2024-04-24T06:36:01.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.644+0000 7f6e56200640 1 -- 172.21.15.59:0/351779438 wait complete. 2024-04-24T06:36:01.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:36:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.863+0000 7ff298c00640 1 Processor -- start 2024-04-24T06:36:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.863+0000 7ff298c00640 1 -- start start 2024-04-24T06:36:01.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.865+0000 7ff298c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.865+0000 7ff298c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2940cd020 0x7ff2940b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.865+0000 7ff298c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2940cd520 con 0x7ff2940b3e40 2024-04-24T06:36:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.865+0000 7ff298c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2940cddc0 con 0x7ff2940cd020 2024-04-24T06:36:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.865+0000 7ff298c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff2940cd800 con 0x7ff2941454e0 2024-04-24T06:36:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2940cd020 0x7ff2940b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2940cd020 0x7ff2940b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35096/0 (socket says 172.21.15.59:35096) 2024-04-24T06:36:01.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 -- 172.21.15.59:0/2951856317 learned_addr learned my addr 172.21.15.59:0/2951856317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291e00640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 -- 172.21.15.59:0/2951856317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 msgr2=0x7ff294145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 -- 172.21.15.59:0/2951856317 >> v1:172.21.15.66:6789/0 conn(0x7ff2940b3e40 legacy=0x7ff294160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291400640 1 -- 172.21.15.59:0/2951856317 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2940cdae0 con 0x7ff2940cd020 2024-04-24T06:36:01.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.866+0000 7ff291e00640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:01.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.867+0000 7ff291400640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940cd020 0x7ff2940b3900 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009870 tx=0x7ff27c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3db7cbaafd7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff290a00640 1 -- 172.21.15.59:0/2951856317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c037070 con 0x7ff2940cd020 2024-04-24T06:36:01.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff290a00640 1 -- 172.21.15.59:0/2951856317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c032440 con 0x7ff2940cd020 2024-04-24T06:36:01.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff290a00640 1 -- 172.21.15.59:0/2951856317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c032810 con 0x7ff2940cd020 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff298c00640 1 -- 172.21.15.59:0/2951856317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940cd020 msgr2=0x7ff2940b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff298c00640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940cd020 0x7ff2940b3900 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009870 tx=0x7ff27c02a180 comp rx=0 tx=0).stop 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff298c00640 1 -- 172.21.15.59:0/2951856317 shutdown_connections 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff298c00640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940cd020 0x7ff2940b3900 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.868+0000 7ff298c00640 1 --2- 172.21.15.59:0/2951856317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.869+0000 7ff298c00640 1 -- 172.21.15.59:0/2951856317 >> 172.21.15.59:0/2951856317 conn(0x7ff294067ea0 msgr2=0x7ff29406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.869+0000 7ff298c00640 1 -- 172.21.15.59:0/2951856317 shutdown_connections 2024-04-24T06:36:01.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.869+0000 7ff298c00640 1 -- 172.21.15.59:0/2951856317 wait complete. 2024-04-24T06:36:01.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.870+0000 7ff298c00640 1 Processor -- start 2024-04-24T06:36:01.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.870+0000 7ff298c00640 1 -- start start 2024-04-24T06:36:01.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 0x7ff29415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff294146270 con 0x7ff2940b3e40 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2941460f0 con 0x7ff2940cd020 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.871+0000 7ff298c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff294161020 con 0x7ff2941454e0 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35104/0 (socket says 172.21.15.59:35104) 2024-04-24T06:36:01.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 -- 172.21.15.59:0/2032802027 learned_addr learned my addr 172.21.15.59:0/2032802027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:01.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291400640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 0x7ff29415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 -- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 msgr2=0x7ff294140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff292800640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294140400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 -- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 msgr2=0x7ff29415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 0x7ff29415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 -- 172.21.15.59:0/2032802027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29413f120 con 0x7ff2940b3e40 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291400640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 0x7ff29415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.872+0000 7ff291e00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c7f0 tx=0x7ff28400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.873+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284015070 con 0x7ff2940b3e40 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.873+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff284007820 con 0x7ff2940b3e40 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.873+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284007bd0 con 0x7ff2940b3e40 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.873+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29406e1a0 con 0x7ff2940b3e40 2024-04-24T06:36:01.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.873+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29406e760 con 0x7ff2940b3e40 2024-04-24T06:36:01.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.874+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff294145950 con 0x7ff2940b3e40 2024-04-24T06:36:01.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.875+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff284018070 con 0x7ff2940b3e40 2024-04-24T06:36:01.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.877+0000 7ff28a000640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 0x7ff27006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:01.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.877+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff28408bc70 con 0x7ff2940b3e40 2024-04-24T06:36:01.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.877+0000 7ff291400640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 0x7ff27006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:01.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.878+0000 7ff291400640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 0x7ff27006edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff27c034000 tx=0x7ff27c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:01.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:01.881+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff294145950 con 0x7ff2940b3e40 2024-04-24T06:36:02.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.130+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff294160030 con 0x7ff2940b3e40 2024-04-24T06:36:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.130+0000 7ff28a000640 1 -- 172.21.15.59:0/2032802027 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7ff28405a090 con 0x7ff2940b3e40 2024-04-24T06:36:02.132 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 msgr2=0x7ff27006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 0x7ff27006edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff27c034000 tx=0x7ff27c02e000 comp rx=0 tx=0).stop 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 msgr2=0x7ff29413ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c7f0 tx=0x7ff28400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 shutdown_connections 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff27006c8f0 0x7ff27006edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff2941454e0 0x7ff294140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2940cd020 0x7ff29415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 --2- 172.21.15.59:0/2032802027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2940b3e40 0x7ff29413ebe0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 >> 172.21.15.59:0/2032802027 conn(0x7ff294067ea0 msgr2=0x7ff2940b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 shutdown_connections 2024-04-24T06:36:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02.133+0000 7ff298c00640 1 -- 172.21.15.59:0/2032802027 wait complete. 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:36:02.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:36:02.155 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:02.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:02.155 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:36:02.208 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:36:02.208 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:02.209 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:36:02.210 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:36:02.211 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:02.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:02.212 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:36:02.295 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:36:02.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:02.300 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:36:02.358 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:36:02.358 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:36:02.358 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:36:02.358 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:36:02.358 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:36:02.359 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:36:02.359 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:36:02.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:36:02.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:36:02.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:36:02.360 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:36:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:36:02.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:36:02.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:02.585 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:36:02.637 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:36:02.638 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:36:02.639 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:02.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:02.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:36:02.726 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:36:02.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:36:02.726 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:02.784 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:02.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:36:02.785 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:02.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:36:02.938 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:36:03.002 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:36:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:36:03.004 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:03.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.006 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:36:03.057 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:36:03.057 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:36:03.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:36:03.137 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:36:03.181 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:36:03.181 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:36:03.182 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:36:03.182 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:36:03.182 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:36:03.183 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:36:03.183 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:36:03.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:36:03.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:36:03.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:36:03.184 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:36:03.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:36:03.184 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:36:03.184 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:36:03.185 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:36:03.185 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:36:03.186 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:03.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.188 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:36:03.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:36:03.235+0000 7fefd79da480 -1 init, newargv = 0x5563073f53e0 newargc=15 2024-04-24T06:36:03.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63224]: starting ceph client 2024-04-24T06:36:03.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63224]: starting fuse 2024-04-24T06:36:03.264 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:36:03.265 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:36:03.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:36:03.265 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:36:03.265 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:36:03.265 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:36:03.266 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:36:03.267 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:36:03.268 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:03.268 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:36:03.268 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:36:03.460 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.63224.asok 2024-04-24T06:36:03.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:36:03.478 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63224.asok 2024-04-24T06:36:03.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63224.asok status 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:36:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "63224", 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7710, 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7710 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:36:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3373032095 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3373032095 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7710 192.168.0.1:0/3373032095", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3373032095", 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 61, 2024-04-24T06:36:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 83, 2024-04-24T06:36:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:36:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:36:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:36:03.716 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:03.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.728 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:36:03.728 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:03.780 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:36:03.781 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:03.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:03.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.870 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-24T06:36:03.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:03.948 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-24T06:36:04.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:04.015 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-24T06:36:04.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:04.079 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-24T06:36:04.158 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:36:04.159 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:36:04.159 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:36:04.159 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:04.159 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-24T06:36:04.159 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:36:04.313 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:36:04.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:36:04.328 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:36:04.328 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:36:04.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:36:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.643+0000 7fde7e400640 1 Processor -- start 2024-04-24T06:36:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.643+0000 7fde7e400640 1 -- start start 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde780cd050 0x7fde7806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde780b3080 0x7fde780cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7e400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7815f160 con 0x7fde7815ece0 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fde780ced20 con 0x7fde780b3080 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fde7815fcd0 con 0x7fde780cd050 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde77e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fde7815ece0 0x7fde780cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44424/0 (socket says 172.21.15.59:44424) 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde77e00640 1 -- 172.21.15.59:0/1561795944 learned_addr learned my addr 172.21.15.59:0/1561795944 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:04.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7da00640 1 --2- 172.21.15.59:0/1561795944 >> v2:172.21.15.59:3300/0 conn(0x7fde780b3080 0x7fde780cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.645+0000 7fde7d000640 1 --2- 172.21.15.59:0/1561795944 >> v2:172.21.15.66:3300/0 conn(0x7fde780cd050 0x7fde7806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde77400640 1 -- 172.21.15.59:0/1561795944 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367135368 0 0) 0x7fde7815f160 con 0x7fde7815ece0 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde77400640 1 -- 172.21.15.59:0/1561795944 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde5c003520 con 0x7fde7815ece0 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde7da00640 1 -- 172.21.15.59:0/1561795944 >> v2:172.21.15.66:3300/0 conn(0x7fde780cd050 msgr2=0x7fde7806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde7da00640 1 --2- 172.21.15.59:0/1561795944 >> v2:172.21.15.66:3300/0 conn(0x7fde780cd050 0x7fde7806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde7da00640 1 -- 172.21.15.59:0/1561795944 >> v1:172.21.15.59:6789/0 conn(0x7fde7815ece0 legacy=0x7fde780cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.646+0000 7fde7da00640 1 -- 172.21.15.59:0/1561795944 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7815f430 con 0x7fde780b3080 2024-04-24T06:36:04.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.647+0000 7fde7da00640 1 --2- 172.21.15.59:0/1561795944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780b3080 0x7fde780cf470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fde7806e080 tx=0x7fde7002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b7056d10b1c123 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:04.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.647+0000 7fde77400640 1 -- 172.21.15.59:0/1561795944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70037070 con 0x7fde780b3080 2024-04-24T06:36:04.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.647+0000 7fde77400640 1 -- 172.21.15.59:0/1561795944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde70032440 con 0x7fde780b3080 2024-04-24T06:36:04.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.647+0000 7fde77400640 1 -- 172.21.15.59:0/1561795944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7002ae60 con 0x7fde780b3080 2024-04-24T06:36:04.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 -- 172.21.15.59:0/1561795944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780b3080 msgr2=0x7fde780cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 --2- 172.21.15.59:0/1561795944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780b3080 0x7fde780cf470 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fde7806e080 tx=0x7fde7002a110 comp rx=0 tx=0).stop 2024-04-24T06:36:04.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 -- 172.21.15.59:0/1561795944 shutdown_connections 2024-04-24T06:36:04.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 --2- 172.21.15.59:0/1561795944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780b3080 0x7fde780cf470 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 --2- 172.21.15.59:0/1561795944 >> v2:172.21.15.66:3300/0 conn(0x7fde780cd050 0x7fde7806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 -- 172.21.15.59:0/1561795944 >> 172.21.15.59:0/1561795944 conn(0x7fde780b21a0 msgr2=0x7fde780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:04.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 -- 172.21.15.59:0/1561795944 shutdown_connections 2024-04-24T06:36:04.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.648+0000 7fde7e400640 1 -- 172.21.15.59:0/1561795944 wait complete. 2024-04-24T06:36:04.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.650+0000 7fde7e400640 1 Processor -- start 2024-04-24T06:36:04.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.650+0000 7fde7e400640 1 -- start start 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde780b3080 0x7fde78157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde7815ece0 0x7fde7809d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde780cef50 con 0x7fde780cd050 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde7815f9f0 con 0x7fde780b3080 2024-04-24T06:36:04.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde7e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fde78159110 con 0x7fde7815ece0 2024-04-24T06:36:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35120/0 (socket says 172.21.15.59:35120) 2024-04-24T06:36:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.651+0000 7fde77e00640 1 -- 172.21.15.59:0/2437257019 learned_addr learned my addr 172.21.15.59:0/2437257019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 -- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde7815ece0 msgr2=0x7fde7809d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde7815ece0 0x7fde7809d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 -- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde780b3080 msgr2=0x7fde78157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde780b3080 0x7fde78157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 -- 172.21.15.59:0/2437257019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde780c9f90 con 0x7fde780cd050 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde77e00640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fde6000caa0 tx=0x7fde6000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60017070 con 0x7fde780cd050 2024-04-24T06:36:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.652+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6000ac00 con 0x7fde780cd050 2024-04-24T06:36:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.653+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60012400 con 0x7fde780cd050 2024-04-24T06:36:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.653+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde781f7cf0 con 0x7fde780cd050 2024-04-24T06:36:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.653+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781f82b0 con 0x7fde780cd050 2024-04-24T06:36:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.654+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7804fd90 con 0x7fde780cd050 2024-04-24T06:36:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.655+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde6001a070 con 0x7fde780cd050 2024-04-24T06:36:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.656+0000 7fde74c00640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.656+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fde6008b990 con 0x7fde780cd050 2024-04-24T06:36:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.656+0000 7fde7d000640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.657+0000 7fde7d000640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fde780c7af0 tx=0x7fde68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.660+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde60059db0 con 0x7fde780cd050 2024-04-24T06:36:04.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.867+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde7815d420 con 0x7fde780cd050 2024-04-24T06:36:04.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.871+0000 7fde74c00640 1 -- 172.21.15.59:0/2437257019 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16576 (secure 0 0 0) 0x7fde600596b0 con 0x7fde780cd050 2024-04-24T06:36:04.873 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:04.873 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:55.872230+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:36:04.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 msgr2=0x7fde5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fde780c7af0 tx=0x7fde68008040 comp rx=0 tx=0).stop 2024-04-24T06:36:04.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 msgr2=0x7fde780c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fde6000caa0 tx=0x7fde6000cf60 comp rx=0 tx=0).stop 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 shutdown_connections 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde7815ece0 0x7fde7809d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde780cd050 0x7fde780c9a50 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 --2- 172.21.15.59:0/2437257019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde780b3080 0x7fde78157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 >> 172.21.15.59:0/2437257019 conn(0x7fde780b21a0 msgr2=0x7fde7815c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 shutdown_connections 2024-04-24T06:36:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:04.873+0000 7fde7e400640 1 -- 172.21.15.59:0/2437257019 wait complete. 2024-04-24T06:36:04.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:36:04.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:36:05.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.110+0000 7fe92be00640 1 Processor -- start 2024-04-24T06:36:05.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.110+0000 7fe92be00640 1 -- start start 2024-04-24T06:36:05.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.112+0000 7fe92be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe92c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.112+0000 7fe92be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c0b51c0 con 0x7fe92c0af2a0 2024-04-24T06:36:05.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.112+0000 7fe92be00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c15ef20 con 0x7fe92c0d0600 2024-04-24T06:36:05.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.112+0000 7fe92be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c15f1c0 con 0x7fe92c155a80 2024-04-24T06:36:05.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe92b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe92c0af2a0 0x7fe92c0b4ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44434/0 (socket says 172.21.15.59:44434) 2024-04-24T06:36:05.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe92b400640 1 -- 172.21.15.59:0/283999929 learned_addr learned my addr 172.21.15.59:0/283999929 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:05.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe92aa00640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe92c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310988081 0 0) 0x7fe92c0b51c0 con 0x7fe92c0af2a0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe910003520 con 0x7fe92c0af2a0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1292553397 0 0) 0x7fe910003520 con 0x7fe92c0af2a0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 msgr2=0x7fe92c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe92c06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 >> v1:172.21.15.66:6789/0 conn(0x7fe92c0d0600 legacy=0x7fe92c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c15f4a0 con 0x7fe92c0af2a0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.113+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe9240019f0 con 0x7fe92c0af2a0 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 >> v1:172.21.15.59:6789/0 conn(0x7fe92c0af2a0 legacy=0x7fe92c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9100039e0 0x7fe910003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe910004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe9100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c15f1c0 con 0x7fe910004800 2024-04-24T06:36:05.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe91000dcb0 con 0x7fe9100039e0 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe91000deb0 con 0x7fe92c155a80 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe92a000640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe910004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe92aa00640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9100039e0 0x7fe910003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.114+0000 7fe92b400640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe9100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.115+0000 7fe92a000640 1 -- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 msgr2=0x7fe9100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.115+0000 7fe92a000640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c155a80 0x7fe9100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.115+0000 7fe92a000640 1 -- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9100039e0 msgr2=0x7fe910003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.115+0000 7fe92a000640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9100039e0 0x7fe910003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.115+0000 7fe92a000640 1 -- 172.21.15.59:0/283999929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe910003520 con 0x7fe910004800 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.116+0000 7fe92a000640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe910004c80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe918001720 tx=0x7fe910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6684574d5aef54a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.116+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918001370 con 0x7fe910004800 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.116+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9180052d0 con 0x7fe910004800 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.116+0000 7fe929600640 1 -- 172.21.15.59:0/283999929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91802ab90 con 0x7fe910004800 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 -- 172.21.15.59:0/283999929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 msgr2=0x7fe910004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 --2- 172.21.15.59:0/283999929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe910004c80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe918001720 tx=0x7fe910005830 comp rx=0 tx=0).stop 2024-04-24T06:36:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92b400640 1 -- 172.21.15.59:0/283999929 reap_dead start 2024-04-24T06:36:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 -- 172.21.15.59:0/283999929 shutdown_connections 2024-04-24T06:36:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 -- 172.21.15.59:0/283999929 >> 172.21.15.59:0/283999929 conn(0x7fe92c06bdf0 msgr2=0x7fe92c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 -- 172.21.15.59:0/283999929 shutdown_connections 2024-04-24T06:36:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.117+0000 7fe92be00640 1 -- 172.21.15.59:0/283999929 wait complete. 2024-04-24T06:36:05.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.119+0000 7fe92be00640 1 Processor -- start 2024-04-24T06:36:05.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.119+0000 7fe92be00640 1 -- start start 2024-04-24T06:36:05.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 0x7fe92c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 0x7fe92c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c15fd40 con 0x7fe910004800 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c15fa60 con 0x7fe92c1f0d80 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c06ef80 con 0x7fe92c15c840 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35138/0 (socket says 172.21.15.59:35138) 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.120+0000 7fe92aa00640 1 -- 172.21.15.59:0/166182917 learned_addr learned my addr 172.21.15.59:0/166182917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92a000640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 0x7fe92c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92b400640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 0x7fe92c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 -- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 msgr2=0x7fe92c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 0x7fe92c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 -- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 msgr2=0x7fe92c150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 0x7fe92c150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 -- 172.21.15.59:0/166182917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c15ccc0 con 0x7fe910004800 2024-04-24T06:36:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92a000640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 0x7fe92c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92b400640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 0x7fe92c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe92aa00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe91c011600 tx=0x7fe91c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f920 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.121+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c00fac0 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.122+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00fe70 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.122+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c15cfa0 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.122+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c15d560 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.123+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c0aff30 con 0x7fe910004800 2024-04-24T06:36:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.124+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe91c015460 con 0x7fe910004800 2024-04-24T06:36:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.125+0000 7fe923400640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 0x7fe90c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.125+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe91c08c590 con 0x7fe910004800 2024-04-24T06:36:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.125+0000 7fe92a000640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 0x7fe90c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.126+0000 7fe92a000640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 0x7fe90c06edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe918032000 tx=0x7fe91802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.130+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe92c0aff30 con 0x7fe910004800 2024-04-24T06:36:05.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.329+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe92c04fd90 con 0x7fe910004800 2024-04-24T06:36:05.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.330+0000 7fe923400640 1 -- 172.21.15.59:0/166182917 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fe91c019030 con 0x7fe910004800 2024-04-24T06:36:05.332 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 msgr2=0x7fe90c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 0x7fe90c06edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe918032000 tx=0x7fe91802e000 comp rx=0 tx=0).stop 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 msgr2=0x7fe92c15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe91c011600 tx=0x7fe91c011ac0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 shutdown_connections 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe90c06c8f0 0x7fe90c06edb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c1f0d80 0x7fe92c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe92c15c840 0x7fe92c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 --2- 172.21.15.59:0/166182917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe910004800 0x7fe92c15dd60 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 >> 172.21.15.59:0/166182917 conn(0x7fe92c06bdf0 msgr2=0x7fe92c0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:05.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 shutdown_connections 2024-04-24T06:36:05.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.333+0000 7fe92be00640 1 -- 172.21.15.59:0/166182917 wait complete. 2024-04-24T06:36:05.355 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:36:05.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.571+0000 7f7d2fe00640 1 Processor -- start 2024-04-24T06:36:05.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.571+0000 7f7d2fe00640 1 -- start start 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d300d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3015ef20 con 0x7f7d300ae290 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3015f1c0 con 0x7f7d30155a80 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300d0fa0 con 0x7f7d300d0600 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7d300ae290 0x7f7d300ae720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44448/0 (socket says 172.21.15.59:44448) 2024-04-24T06:36:05.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.573+0000 7f7d2f400640 1 -- 172.21.15.59:0/2762827245 learned_addr learned my addr 172.21.15.59:0/2762827245 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:05.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2e000640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d300d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168147483 0 0) 0x7f7d3015ef20 con 0x7f7d300ae290 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d14005730 con 0x7f7d300ae290 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4013836166 0 0) 0x7f7d14005730 con 0x7f7d300ae290 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 msgr2=0x7f7d300d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d300d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 >> v1:172.21.15.66:6789/0 conn(0x7f7d30155a80 legacy=0x7f7d3006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3015f4a0 con 0x7f7d300ae290 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7d28001670 con 0x7f7d300ae290 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 >> v1:172.21.15.59:6789/0 conn(0x7f7d300ae290 legacy=0x7f7d300ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d14005bf0 0x7f7d14006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d14006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14006f50 0x7f7d140073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1400dc80 con 0x7f7d14005bf0 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1400de30 con 0x7f7d14006f50 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.574+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1400e030 con 0x7f7d300d0600 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d14005bf0 0x7f7d14006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2f400640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14006f50 0x7f7d140073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2e000640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d14006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 -- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 msgr2=0x7f7d14006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d300d0600 0x7f7d14006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 -- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14006f50 msgr2=0x7f7d140073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14006f50 0x7f7d140073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2ea00640 1 -- 172.21.15.59:0/2762827245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d14005730 con 0x7f7d14005bf0 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.575+0000 7f7d2f400640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14006f50 0x7f7d140073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:05.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.576+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d14005bf0 0x7f7d14006010 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7d20001e20 tx=0x7f7d14007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d88d7398feb7af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20001a70 con 0x7f7d14005bf0 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2002ae40 con 0x7f7d14005bf0 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2d600640 1 -- 172.21.15.59:0/2762827245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003bb0 con 0x7f7d14005bf0 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 -- 172.21.15.59:0/2762827245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d14005bf0 msgr2=0x7f7d14006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/2762827245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d14005bf0 0x7f7d14006010 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7d20001e20 tx=0x7f7d14007a40 comp rx=0 tx=0).stop 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2f400640 1 -- 172.21.15.59:0/2762827245 reap_dead start 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 -- 172.21.15.59:0/2762827245 shutdown_connections 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 -- 172.21.15.59:0/2762827245 >> 172.21.15.59:0/2762827245 conn(0x7f7d3006bdf0 msgr2=0x7f7d3006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 -- 172.21.15.59:0/2762827245 shutdown_connections 2024-04-24T06:36:05.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.577+0000 7f7d2fe00640 1 -- 172.21.15.59:0/2762827245 wait complete. 2024-04-24T06:36:05.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.579+0000 7f7d2fe00640 1 Processor -- start 2024-04-24T06:36:05.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.579+0000 7f7d2fe00640 1 -- start start 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 0x7f7d30154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d301527c0 0x7f7d30152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3015fd40 con 0x7f7d300cc9f0 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3015fa60 con 0x7f7d14005bf0 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.580+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3006ef80 con 0x7f7d301527c0 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35144/0 (socket says 172.21.15.59:35144) 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 -- 172.21.15.59:0/3398604244 learned_addr learned my addr 172.21.15.59:0/3398604244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 0x7f7d30154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 -- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d301527c0 msgr2=0x7f7d30152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d301527c0 0x7f7d30152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 -- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 msgr2=0x7f7d30154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 0x7f7d30154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 -- 172.21.15.59:0/3398604244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d300cd160 con 0x7f7d300cc9f0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 0x7f7d30154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.581+0000 7f7d2e000640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7d18010860 tx=0x7f7d18010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.582+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1800f040 con 0x7f7d300cc9f0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.582+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1800eb60 con 0x7f7d300cc9f0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.582+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d300cd440 con 0x7f7d300cc9f0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.582+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1801c400 con 0x7f7d300cc9f0 2024-04-24T06:36:05.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.582+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d300cda00 con 0x7f7d300cc9f0 2024-04-24T06:36:05.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.583+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3004fd90 con 0x7f7d300cc9f0 2024-04-24T06:36:05.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.584+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d1801c5a0 con 0x7f7d300cc9f0 2024-04-24T06:36:05.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.585+0000 7f7d27400640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 0x7f7d0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:05.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.585+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7d1808c150 con 0x7f7d300cc9f0 2024-04-24T06:36:05.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.585+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 0x7f7d0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:05.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.586+0000 7f7d2ea00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 0x7f7d0c06ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7d20032000 tx=0x7f7d2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:05.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.590+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d18018030 con 0x7f7d300cc9f0 2024-04-24T06:36:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.788+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d30151c90 con 0x7f7d300cc9f0 2024-04-24T06:36:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.788+0000 7f7d27400640 1 -- 172.21.15.59:0/3398604244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f7d1805a4e0 con 0x7f7d300cc9f0 2024-04-24T06:36:05.790 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 msgr2=0x7f7d0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 0x7f7d0c06ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7d20032000 tx=0x7f7d2002e000 comp rx=0 tx=0).stop 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 msgr2=0x7f7d301f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7d18010860 tx=0x7f7d18010d20 comp rx=0 tx=0).stop 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 shutdown_connections 2024-04-24T06:36:05.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d0c06c920 0x7f7d0c06ede0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d301527c0 0x7f7d30152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d300cc9f0 0x7f7d301f71d0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 --2- 172.21.15.59:0/3398604244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d14005bf0 0x7f7d30154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.791+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 >> 172.21.15.59:0/3398604244 conn(0x7f7d3006bdf0 msgr2=0x7f7d300b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.792+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 shutdown_connections 2024-04-24T06:36:05.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:05.792+0000 7f7d2fe00640 1 -- 172.21.15.59:0/3398604244 wait complete. 2024-04-24T06:36:05.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:36:06.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.016+0000 7fd765c00640 1 Processor -- start 2024-04-24T06:36:06.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.016+0000 7fd765c00640 1 -- start start 2024-04-24T06:36:06.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd7600d15b0 0x7fd7600cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7600b4f20 0x7fd7600cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7600b5da0 con 0x7fd76015ed20 2024-04-24T06:36:06.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd76015f100 con 0x7fd7600b4f20 2024-04-24T06:36:06.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600d0a20 con 0x7fd7600d15b0 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7600b4f20 0x7fd7600cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7600b4f20 0x7fd7600cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35156/0 (socket says 172.21.15.59:35156) 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.018+0000 7fd765200640 1 -- 172.21.15.59:0/2674674269 learned_addr learned my addr 172.21.15.59:0/2674674269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.019+0000 7fd75f400640 1 --2- 172.21.15.59:0/2674674269 >> v2:172.21.15.66:3300/0 conn(0x7fd7600d15b0 0x7fd7600cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.019+0000 7fd765200640 1 -- 172.21.15.59:0/2674674269 >> v2:172.21.15.66:3300/0 conn(0x7fd7600d15b0 msgr2=0x7fd7600cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.019+0000 7fd765200640 1 --2- 172.21.15.59:0/2674674269 >> v2:172.21.15.66:3300/0 conn(0x7fd7600d15b0 0x7fd7600cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.019+0000 7fd765200640 1 -- 172.21.15.59:0/2674674269 >> v1:172.21.15.66:6789/0 conn(0x7fd76015ed20 legacy=0x7fd7600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:06.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.019+0000 7fd765200640 1 -- 172.21.15.59:0/2674674269 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600d2ed0 con 0x7fd7600b4f20 2024-04-24T06:36:06.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.020+0000 7fd765200640 1 --2- 172.21.15.59:0/2674674269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600cf7c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd7600d4520 tx=0x7fd75802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e22d26507e8863 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.021+0000 7fd75ea00640 1 -- 172.21.15.59:0/2674674269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758036070 con 0x7fd7600b4f20 2024-04-24T06:36:06.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.021+0000 7fd75ea00640 1 -- 172.21.15.59:0/2674674269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd758031600 con 0x7fd7600b4f20 2024-04-24T06:36:06.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.021+0000 7fd75ea00640 1 -- 172.21.15.59:0/2674674269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7580319d0 con 0x7fd7600b4f20 2024-04-24T06:36:06.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.021+0000 7fd765c00640 1 -- 172.21.15.59:0/2674674269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 msgr2=0x7fd7600cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.021+0000 7fd765c00640 1 --2- 172.21.15.59:0/2674674269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600cf7c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd7600d4520 tx=0x7fd75802a5d0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 -- 172.21.15.59:0/2674674269 shutdown_connections 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 --2- 172.21.15.59:0/2674674269 >> v2:172.21.15.66:3300/0 conn(0x7fd7600d15b0 0x7fd7600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 --2- 172.21.15.59:0/2674674269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600cf7c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 -- 172.21.15.59:0/2674674269 >> 172.21.15.59:0/2674674269 conn(0x7fd76008bf90 msgr2=0x7fd7600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 -- 172.21.15.59:0/2674674269 shutdown_connections 2024-04-24T06:36:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.022+0000 7fd765c00640 1 -- 172.21.15.59:0/2674674269 wait complete. 2024-04-24T06:36:06.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.023+0000 7fd765c00640 1 Processor -- start 2024-04-24T06:36:06.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 -- start start 2024-04-24T06:36:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7600d15b0 0x7fd760159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd76015ed20 0x7fd7601f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b3600 con 0x7fd7600b4f20 2024-04-24T06:36:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600d3490 con 0x7fd7600d15b0 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.024+0000 7fd765c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7600b3780 con 0x7fd76015ed20 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35166/0 (socket says 172.21.15.59:35166) 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 -- 172.21.15.59:0/444547784 learned_addr learned my addr 172.21.15.59:0/444547784 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd765200640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd76015ed20 0x7fd7601f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75f400640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7600d15b0 0x7fd760159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 -- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd76015ed20 msgr2=0x7fd7601f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd76015ed20 0x7fd7601f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 -- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7600d15b0 msgr2=0x7fd760159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7600d15b0 0x7fd760159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.025+0000 7fd75fe00640 1 -- 172.21.15.59:0/444547784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7601f9b60 con 0x7fd7600b4f20 2024-04-24T06:36:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd75fe00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c430 tx=0x7fd75000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750011790 con 0x7fd7600b4f20 2024-04-24T06:36:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd750017070 con 0x7fd7600b4f20 2024-04-24T06:36:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7601f95a0 con 0x7fd7600b4f20 2024-04-24T06:36:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750011bb0 con 0x7fd7600b4f20 2024-04-24T06:36:06.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.026+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd760159780 con 0x7fd7600b4f20 2024-04-24T06:36:06.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.027+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7601595a0 con 0x7fd7600b4f20 2024-04-24T06:36:06.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.028+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd75001c460 con 0x7fd7600b4f20 2024-04-24T06:36:06.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.029+0000 7fd757e00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.030+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fd75008ba90 con 0x7fd7600b4f20 2024-04-24T06:36:06.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.030+0000 7fd75f400640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.030+0000 7fd75f400640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 0x7fd74006ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006fd0 tx=0x7fd74c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.034+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7601595a0 con 0x7fd7600b4f20 2024-04-24T06:36:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.236+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd76004fd90 con 0x7fd7600b4f20 2024-04-24T06:36:06.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.237+0000 7fd757e00640 1 -- 172.21.15.59:0/444547784 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fd750059e20 con 0x7fd7600b4f20 2024-04-24T06:36:06.239 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 msgr2=0x7fd74006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 0x7fd74006ede0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd74c006fd0 tx=0x7fd74c008040 comp rx=0 tx=0).stop 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 msgr2=0x7fd7600c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd75000c430 tx=0x7fd75000c8f0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 shutdown_connections 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd74006c920 0x7fd74006ede0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd76015ed20 0x7fd7601f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7600d15b0 0x7fd760159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 --2- 172.21.15.59:0/444547784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7600b4f20 0x7fd7600c9d60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 >> 172.21.15.59:0/444547784 conn(0x7fd76008bf90 msgr2=0x7fd76006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:06.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 shutdown_connections 2024-04-24T06:36:06.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.240+0000 7fd765c00640 1 -- 172.21.15.59:0/444547784 wait complete. 2024-04-24T06:36:06.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:36:06.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.460+0000 7fce24c00640 1 Processor -- start 2024-04-24T06:36:06.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.460+0000 7fce24c00640 1 -- start start 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce24c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce200af4b0 0x7fce2015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce24c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fce20144a20 0x7fce200d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce24c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce200b3720 con 0x7fce200af890 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce24c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fce20144ed0 con 0x7fce200af4b0 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce24c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fce2015f3f0 con 0x7fce20144a20 2024-04-24T06:36:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce1f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce200af4b0 0x7fce2015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce1f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce200af4b0 0x7fce2015ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35184/0 (socket says 172.21.15.59:35184) 2024-04-24T06:36:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.462+0000 7fce1f400640 1 -- 172.21.15.59:0/1115819994 learned_addr learned my addr 172.21.15.59:0/1115819994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1fe00640 1 --2- 172.21.15.59:0/1115819994 >> v2:172.21.15.66:3300/0 conn(0x7fce20144a20 0x7fce200d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1f400640 1 -- 172.21.15.59:0/1115819994 >> v2:172.21.15.66:3300/0 conn(0x7fce20144a20 msgr2=0x7fce200d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1f400640 1 --2- 172.21.15.59:0/1115819994 >> v2:172.21.15.66:3300/0 conn(0x7fce20144a20 0x7fce200d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1f400640 1 -- 172.21.15.59:0/1115819994 >> v1:172.21.15.59:6789/0 conn(0x7fce200af890 legacy=0x7fce200b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1f400640 1 -- 172.21.15.59:0/1115819994 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2015f6d0 con 0x7fce200af4b0 2024-04-24T06:36:06.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.463+0000 7fce1ea00640 1 -- 172.21.15.59:0/1115819994 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230125351 0 0) 0x7fce200b3720 con 0x7fce200af890 2024-04-24T06:36:06.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce1f400640 1 --2- 172.21.15.59:0/1115819994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce200af4b0 0x7fce2015ed50 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fce201623e0 tx=0x7fce1002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17376e75138c2649 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce1ea00640 1 -- 172.21.15.59:0/1115819994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10036070 con 0x7fce200af4b0 2024-04-24T06:36:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce1ea00640 1 -- 172.21.15.59:0/1115819994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce10031600 con 0x7fce200af4b0 2024-04-24T06:36:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce1ea00640 1 -- 172.21.15.59:0/1115819994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100319d0 con 0x7fce200af4b0 2024-04-24T06:36:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce24c00640 1 -- 172.21.15.59:0/1115819994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce200af4b0 msgr2=0x7fce2015ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.465+0000 7fce24c00640 1 --2- 172.21.15.59:0/1115819994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce200af4b0 0x7fce2015ed50 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fce201623e0 tx=0x7fce1002a730 comp rx=0 tx=0).stop 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 -- 172.21.15.59:0/1115819994 shutdown_connections 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 --2- 172.21.15.59:0/1115819994 >> v2:172.21.15.66:3300/0 conn(0x7fce20144a20 0x7fce200d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 --2- 172.21.15.59:0/1115819994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce200af4b0 0x7fce2015ed50 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 -- 172.21.15.59:0/1115819994 >> 172.21.15.59:0/1115819994 conn(0x7fce2006bdf0 msgr2=0x7fce2006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 -- 172.21.15.59:0/1115819994 shutdown_connections 2024-04-24T06:36:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.466+0000 7fce24c00640 1 -- 172.21.15.59:0/1115819994 wait complete. 2024-04-24T06:36:06.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.468+0000 7fce24c00640 1 Processor -- start 2024-04-24T06:36:06.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.468+0000 7fce24c00640 1 -- start start 2024-04-24T06:36:06.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce200af4b0 0x7fce200adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fce200acff0 0x7fce20151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce200d0b00 con 0x7fce20144a20 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce2015fc90 con 0x7fce200af4b0 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce24c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fce200d0c80 con 0x7fce200acff0 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.469+0000 7fce16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35194/0 (socket says 172.21.15.59:35194) 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 -- 172.21.15.59:0/1169086047 learned_addr learned my addr 172.21.15.59:0/1169086047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce1f400640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce200af4b0 0x7fce200adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce1fe00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fce200acff0 0x7fce20151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 -- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fce200acff0 msgr2=0x7fce20151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fce200acff0 0x7fce20151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 -- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce200af4b0 msgr2=0x7fce200adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce200af4b0 0x7fce200adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 -- 172.21.15.59:0/1169086047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce200ae760 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.470+0000 7fce16a00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fce0800c510 tx=0x7fce0800c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.471+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08017070 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.471+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce08005480 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.471+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08003830 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.471+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce200aea40 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.471+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2015b3a0 con 0x7fce20144a20 2024-04-24T06:36:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.472+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2004fd90 con 0x7fce20144a20 2024-04-24T06:36:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.473+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x7fce20144a20 2024-04-24T06:36:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.475+0000 7fce17e00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 0x7fcdfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.475+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fce0808af10 con 0x7fce20144a20 2024-04-24T06:36:06.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.475+0000 7fce1f400640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 0x7fcdfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.476+0000 7fce1f400640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 0x7fcdfc06ede0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce10033000 tx=0x7fce10009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.479+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce08059330 con 0x7fce20144a20 2024-04-24T06:36:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.693+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce200b0270 con 0x7fce20144a20 2024-04-24T06:36:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.694+0000 7fce17e00640 1 -- 172.21.15.59:0/1169086047 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7fce08058c30 con 0x7fce20144a20 2024-04-24T06:36:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:36:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 msgr2=0x7fcdfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 0x7fcdfc06ede0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce10033000 tx=0x7fce10009000 comp rx=0 tx=0).stop 2024-04-24T06:36:06.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 msgr2=0x7fce20151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fce0800c510 tx=0x7fce0800c9d0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 shutdown_connections 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcdfc06c920 0x7fcdfc06ede0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fce200acff0 0x7fce20151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce20144a20 0x7fce20151650 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 --2- 172.21.15.59:0/1169086047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce200af4b0 0x7fce200adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 >> 172.21.15.59:0/1169086047 conn(0x7fce2006bdf0 msgr2=0x7fce200b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 shutdown_connections 2024-04-24T06:36:06.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.696+0000 7fce24c00640 1 -- 172.21.15.59:0/1169086047 wait complete. 2024-04-24T06:36:06.716 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:36:06.716 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:36:06.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:36:06.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.972+0000 7f2641800640 1 Processor -- start 2024-04-24T06:36:06.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.972+0000 7f2641800640 1 -- start start 2024-04-24T06:36:06.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2641800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f263c15c1d0 0x7f263c15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2641800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f263c06fed0 0x7f263c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2641800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c06f120 con 0x7f263c15dd90 2024-04-24T06:36:06.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2641800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f263c06f400 con 0x7f263c06fed0 2024-04-24T06:36:06.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2641800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f263c06f6e0 con 0x7f263c15c1d0 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f2632000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f263c06fed0 0x7f263c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f263b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f263c15dd90 0x7f263c071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44478/0 (socket says 172.21.15.59:44478) 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.974+0000 7f263b400640 1 -- 172.21.15.59:0/574575062 learned_addr learned my addr 172.21.15.59:0/574575062 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f263aa00640 1 --2- 172.21.15.59:0/574575062 >> v2:172.21.15.66:3300/0 conn(0x7f263c15c1d0 0x7f263c15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f263a000640 1 -- 172.21.15.59:0/574575062 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081028164 0 0) 0x7f263c06f120 con 0x7f263c15dd90 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f2632000640 1 -- 172.21.15.59:0/574575062 >> v2:172.21.15.66:3300/0 conn(0x7f263c15c1d0 msgr2=0x7f263c15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:06.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f2632000640 1 --2- 172.21.15.59:0/574575062 >> v2:172.21.15.66:3300/0 conn(0x7f263c15c1d0 0x7f263c15afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f2632000640 1 -- 172.21.15.59:0/574575062 >> v1:172.21.15.59:6789/0 conn(0x7f263c15dd90 legacy=0x7f263c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.975+0000 7f2632000640 1 -- 172.21.15.59:0/574575062 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c0b3850 con 0x7f263c06fed0 2024-04-24T06:36:06.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.977+0000 7f2632000640 1 --2- 172.21.15.59:0/574575062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c06fed0 0x7f263c06e8d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f263c072e60 tx=0x7f262402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb08e1121bd7eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f263a000640 1 -- 172.21.15.59:0/574575062 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624007740 con 0x7f263c06fed0 2024-04-24T06:36:06.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f263a000640 1 -- 172.21.15.59:0/574575062 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26240078e0 con 0x7f263c06fed0 2024-04-24T06:36:06.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f263a000640 1 -- 172.21.15.59:0/574575062 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624007cb0 con 0x7f263c06fed0 2024-04-24T06:36:06.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 -- 172.21.15.59:0/574575062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c06fed0 msgr2=0x7f263c06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 --2- 172.21.15.59:0/574575062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c06fed0 0x7f263c06e8d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f263c072e60 tx=0x7f262402a680 comp rx=0 tx=0).stop 2024-04-24T06:36:06.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 -- 172.21.15.59:0/574575062 shutdown_connections 2024-04-24T06:36:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 --2- 172.21.15.59:0/574575062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c06fed0 0x7f263c06e8d0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 --2- 172.21.15.59:0/574575062 >> v2:172.21.15.66:3300/0 conn(0x7f263c15c1d0 0x7f263c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 -- 172.21.15.59:0/574575062 >> 172.21.15.59:0/574575062 conn(0x7f263c067ea0 msgr2=0x7f263c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.978+0000 7f2641800640 1 -- 172.21.15.59:0/574575062 shutdown_connections 2024-04-24T06:36:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.979+0000 7f2641800640 1 -- 172.21.15.59:0/574575062 wait complete. 2024-04-24T06:36:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.980+0000 7f2641800640 1 Processor -- start 2024-04-24T06:36:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.980+0000 7f2641800640 1 -- start start 2024-04-24T06:36:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f263c06fed0 0x7f263c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f263c15dd90 0x7f263c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f263c15e380 con 0x7f263c15c1d0 2024-04-24T06:36:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f263c15e200 con 0x7f263c15dd90 2024-04-24T06:36:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.981+0000 7f2641800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f263c0724f0 con 0x7f263c06fed0 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f263aa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f263c06fed0 0x7f263c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f263b400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f263c15dd90 0x7f263c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35218/0 (socket says 172.21.15.59:35218) 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 -- 172.21.15.59:0/626395157 learned_addr learned my addr 172.21.15.59:0/626395157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 -- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f263c06fed0 msgr2=0x7f263c1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f263c06fed0 0x7f263c1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 -- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f263c15dd90 msgr2=0x7f263c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f263c15dd90 0x7f263c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.982+0000 7f2632000640 1 -- 172.21.15.59:0/626395157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c1f5600 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2632000640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2624009e80 tx=0x7f2624004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262403f070 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2624031c40 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624043410 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1577f0 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.983+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c1f5870 con 0x7f263c15c1d0 2024-04-24T06:36:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.984+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c068a80 con 0x7f263c15c1d0 2024-04-24T06:36:06.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.985+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26240435b0 con 0x7f263c15c1d0 2024-04-24T06:36:06.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.986+0000 7f2633400640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:06.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.987+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f26240acea0 con 0x7f263c15c1d0 2024-04-24T06:36:06.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.987+0000 7f263aa00640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.987+0000 7f263aa00640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 0x7f261806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f262c006fd0 tx=0x7f262c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:06.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:06.991+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f262407ac70 con 0x7f263c15c1d0 2024-04-24T06:36:07.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:36:07.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:36:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.199+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f263c06e2d0 con 0x7f263c15c1d0 2024-04-24T06:36:07.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.202+0000 7f2633400640 1 -- 172.21.15.59:0/626395157 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16576 (secure 0 0 0) 0x7f262407ef70 con 0x7f263c15c1d0 2024-04-24T06:36:07.204 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:07.204 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:35:55.872230+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.204+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 msgr2=0x7f261806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.204+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 0x7f261806edb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f262c006fd0 tx=0x7f262c008040 comp rx=0 tx=0).stop 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 msgr2=0x7f263c0c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2624009e80 tx=0x7f2624004a40 comp rx=0 tx=0).stop 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 shutdown_connections 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f263c15dd90 0x7f263c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263c15c1d0 0x7f263c0c6240 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 --2- 172.21.15.59:0/626395157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f263c06fed0 0x7f263c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 >> 172.21.15.59:0/626395157 conn(0x7f263c067ea0 msgr2=0x7f263c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 shutdown_connections 2024-04-24T06:36:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.205+0000 7f2641800640 1 -- 172.21.15.59:0/626395157 wait complete. 2024-04-24T06:36:07.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:36:07.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:36:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.482+0000 7f7833e00640 1 Processor -- start 2024-04-24T06:36:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.482+0000 7f7833e00640 1 -- start start 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7833e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7834160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7833e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783415cab0 con 0x7f78341567e0 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7833e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7834160e40 con 0x7f78340d0d10 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7833e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f783415cd30 con 0x7f783415d810 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7832a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f78341567e0 0x7f78340d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44494/0 (socket says 172.21.15.59:44494) 2024-04-24T06:36:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7832a00640 1 -- 172.21.15.59:0/2552394965 learned_addr learned my addr 172.21.15.59:0/2552394965 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.484+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243520310 0 0) 0x7f7824001490 con 0x7f78341567e0 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7818003520 con 0x7f78341567e0 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7833400640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7834160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3691598854 0 0) 0x7f7818003520 con 0x7f78341567e0 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 msgr2=0x7f7834160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7834160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 >> v1:172.21.15.66:6789/0 conn(0x7f78340d0d10 legacy=0x7f7834160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783415d010 con 0x7f78341567e0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7824001bc0 con 0x7f78341567e0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 >> v1:172.21.15.59:6789/0 conn(0x7f78341567e0 legacy=0x7f78340d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78180039e0 0x7f7818003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7818004800 0x7f7818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7818005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f783415cd30 con 0x7f7818004800 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f781800baf0 con 0x7f78180039e0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f781800bcf0 con 0x7f783415d810 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.485+0000 7f7832000640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7818004800 0x7f7818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832a00640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78180039e0 0x7f7818003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7833400640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7818005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832000640 1 -- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 msgr2=0x7f7818005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832000640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f783415d810 0x7f7818005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832000640 1 -- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78180039e0 msgr2=0x7f7818003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832000640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78180039e0 0x7f7818003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.486+0000 7f7832000640 1 -- 172.21.15.59:0/2552394965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7818003520 con 0x7f7818004800 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.487+0000 7f7832000640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7818004800 0x7f7818004c80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f78340afdb0 tx=0x7f7818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41f8b1f1552d018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.487+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c001370 con 0x7f7818004800 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.487+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c0052d0 con 0x7f7818004800 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.487+0000 7f7831600640 1 -- 172.21.15.59:0/2552394965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c02ab90 con 0x7f7818004800 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 -- 172.21.15.59:0/2552394965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7818004800 msgr2=0x7f7818004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 --2- 172.21.15.59:0/2552394965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7818004800 0x7f7818004c80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f78340afdb0 tx=0x7f7818005830 comp rx=0 tx=0).stop 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833400640 1 -- 172.21.15.59:0/2552394965 reap_dead start 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 -- 172.21.15.59:0/2552394965 shutdown_connections 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 -- 172.21.15.59:0/2552394965 >> 172.21.15.59:0/2552394965 conn(0x7f783406bdf0 msgr2=0x7f783406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 -- 172.21.15.59:0/2552394965 shutdown_connections 2024-04-24T06:36:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.488+0000 7f7833e00640 1 -- 172.21.15.59:0/2552394965 wait complete. 2024-04-24T06:36:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.490+0000 7f7833e00640 1 Processor -- start 2024-04-24T06:36:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.490+0000 7f7833e00640 1 -- start start 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7818004800 0x7f78340999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f78341eaca0 0x7f7834156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f78341609a0 con 0x7f7834157af0 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7834160820 con 0x7f7818004800 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7834160b20 con 0x7f78341eaca0 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35244/0 (socket says 172.21.15.59:35244) 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.491+0000 7f7833400640 1 -- 172.21.15.59:0/1210591776 learned_addr learned my addr 172.21.15.59:0/1210591776 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7832000640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f78341eaca0 0x7f7834156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7832a00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7818004800 0x7f78340999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 -- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f78341eaca0 msgr2=0x7f7834156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f78341eaca0 0x7f7834156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 -- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7818004800 msgr2=0x7f78340999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7818004800 0x7f78340999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 -- 172.21.15.59:0/1210591776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78341e7d60 con 0x7f7834157af0 2024-04-24T06:36:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.492+0000 7f7833400640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f782c006fd0 tx=0x7f782c00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.493+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c017020 con 0x7f7834157af0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.493+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c004440 con 0x7f7834157af0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.493+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c003850 con 0x7f7834157af0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.493+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78341ea1c0 con 0x7f7834157af0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.493+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7834157190 con 0x7f7834157af0 2024-04-24T06:36:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.494+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783404fd90 con 0x7f7834157af0 2024-04-24T06:36:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.495+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f782c01a020 con 0x7f7834157af0 2024-04-24T06:36:07.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.496+0000 7f782aa00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 0x7f781006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.496+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f782c08b960 con 0x7f7834157af0 2024-04-24T06:36:07.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.497+0000 7f7832a00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 0x7f781006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.497+0000 7f7832a00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 0x7f781006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78240013d0 tx=0x7f7824009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.501+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f782c021020 con 0x7f7834157af0 2024-04-24T06:36:07.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.703+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78340b5150 con 0x7f7834157af0 2024-04-24T06:36:07.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.704+0000 7f782aa00640 1 -- 172.21.15.59:0/1210591776 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f782c059e20 con 0x7f7834157af0 2024-04-24T06:36:07.706 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:36:07.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 msgr2=0x7f781006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 0x7f781006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78240013d0 tx=0x7f7824009040 comp rx=0 tx=0).stop 2024-04-24T06:36:07.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 msgr2=0x7f7834157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f782c006fd0 tx=0x7f782c00f920 comp rx=0 tx=0).stop 2024-04-24T06:36:07.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 shutdown_connections 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f781006c8f0 0x7f781006edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7834157af0 0x7f7834157f70 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f78341eaca0 0x7f7834156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 --2- 172.21.15.59:0/1210591776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7818004800 0x7f78340999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 >> 172.21.15.59:0/1210591776 conn(0x7f783406bdf0 msgr2=0x7f78340d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 shutdown_connections 2024-04-24T06:36:07.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.707+0000 7f7833e00640 1 -- 172.21.15.59:0/1210591776 wait complete. 2024-04-24T06:36:07.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.942+0000 7ff956000640 1 Processor -- start 2024-04-24T06:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.942+0000 7ff956000640 1 -- start start 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff956000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9500d0ed0 0x7ff9500b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff956000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff9500d12b0 0x7ff95015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff956000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95015ef80 con 0x7ff95015eaf0 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff956000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff9500b2dc0 con 0x7ff9500d0ed0 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff956000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff950163320 con 0x7ff9500d12b0 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff94f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9500d0ed0 0x7ff9500b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff94f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9500d0ed0 0x7ff9500b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35258/0 (socket says 172.21.15.59:35258) 2024-04-24T06:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.944+0000 7ff94f400640 1 -- 172.21.15.59:0/4023994475 learned_addr learned my addr 172.21.15.59:0/4023994475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.945+0000 7ff946a00640 1 --2- 172.21.15.59:0/4023994475 >> v2:172.21.15.66:3300/0 conn(0x7ff9500d12b0 0x7ff95015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.945+0000 7ff94f400640 1 -- 172.21.15.59:0/4023994475 >> v2:172.21.15.66:3300/0 conn(0x7ff9500d12b0 msgr2=0x7ff95015e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.945+0000 7ff94f400640 1 --2- 172.21.15.59:0/4023994475 >> v2:172.21.15.66:3300/0 conn(0x7ff9500d12b0 0x7ff95015e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.945+0000 7ff94f400640 1 -- 172.21.15.59:0/4023994475 >> v1:172.21.15.66:6789/0 conn(0x7ff95015eaf0 legacy=0x7ff950162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.945+0000 7ff94f400640 1 -- 172.21.15.59:0/4023994475 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500b2770 con 0x7ff9500d0ed0 2024-04-24T06:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.946+0000 7ff94f400640 1 --2- 172.21.15.59:0/4023994475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9500b3250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff9500a06b0 tx=0x7ff94002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134064fe044b80d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.947+0000 7ff94ea00640 1 -- 172.21.15.59:0/4023994475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940037070 con 0x7ff9500d0ed0 2024-04-24T06:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.947+0000 7ff94ea00640 1 -- 172.21.15.59:0/4023994475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff940032440 con 0x7ff9500d0ed0 2024-04-24T06:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.947+0000 7ff94ea00640 1 -- 172.21.15.59:0/4023994475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940032810 con 0x7ff9500d0ed0 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 -- 172.21.15.59:0/4023994475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 msgr2=0x7ff9500b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 --2- 172.21.15.59:0/4023994475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9500b3250 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff9500a06b0 tx=0x7ff94002a2c0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 -- 172.21.15.59:0/4023994475 shutdown_connections 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 --2- 172.21.15.59:0/4023994475 >> v2:172.21.15.66:3300/0 conn(0x7ff9500d12b0 0x7ff95015e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 --2- 172.21.15.59:0/4023994475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9500b3250 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 -- 172.21.15.59:0/4023994475 >> 172.21.15.59:0/4023994475 conn(0x7ff9500d2680 msgr2=0x7ff9500d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 -- 172.21.15.59:0/4023994475 shutdown_connections 2024-04-24T06:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.948+0000 7ff956000640 1 -- 172.21.15.59:0/4023994475 wait complete. 2024-04-24T06:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.950+0000 7ff956000640 1 Processor -- start 2024-04-24T06:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.950+0000 7ff956000640 1 -- start start 2024-04-24T06:36:07.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9500d12b0 0x7ff9500b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff95015eaf0 0x7ff95015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff950162ef0 con 0x7ff9500d0ed0 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff950162d70 con 0x7ff95015eaf0 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff956000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff950163070 con 0x7ff9500d12b0 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35264/0 (socket says 172.21.15.59:35264) 2024-04-24T06:36:07.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 -- 172.21.15.59:0/2609051045 learned_addr learned my addr 172.21.15.59:0/2609051045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:07.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff946a00640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9500d12b0 0x7ff9500b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 -- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9500d12b0 msgr2=0x7ff9500b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94fe00640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff95015eaf0 0x7ff95015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9500d12b0 0x7ff9500b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 -- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff95015eaf0 msgr2=0x7ff95015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff95015eaf0 0x7ff95015aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.951+0000 7ff94f400640 1 -- 172.21.15.59:0/2609051045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500b7100 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff94f400640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff940000c00 tx=0x7ff9400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940041020 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9400307a0 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940037040 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9500b73e0 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.952+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95015af60 con 0x7ff9500d0ed0 2024-04-24T06:36:07.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.954+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950069190 con 0x7ff9500d0ed0 2024-04-24T06:36:07.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.955+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff940044020 con 0x7ff9500d0ed0 2024-04-24T06:36:07.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.956+0000 7ff947e00640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 0x7ff92c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:07.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.957+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff9400ac2f0 con 0x7ff9500d0ed0 2024-04-24T06:36:07.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.957+0000 7ff946a00640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 0x7ff92c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:07.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.957+0000 7ff946a00640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 0x7ff92c06ee30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff95015dc40 tx=0x7ff938008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:07.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:07.962+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff94004b030 con 0x7ff9500d0ed0 2024-04-24T06:36:08.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.163+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9500b20e0 con 0x7ff9500d0ed0 2024-04-24T06:36:08.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.164+0000 7ff947e00640 1 -- 172.21.15.59:0/2609051045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7ff95015ef80 con 0x7ff9500d0ed0 2024-04-24T06:36:08.166 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 msgr2=0x7ff92c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 0x7ff92c06ee30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff95015dc40 tx=0x7ff938008040 comp rx=0 tx=0).stop 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 msgr2=0x7ff9501534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff940000c00 tx=0x7ff9400046c0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 shutdown_connections 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff92c06c970 0x7ff92c06ee30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff95015eaf0 0x7ff95015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9500d12b0 0x7ff9500b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 --2- 172.21.15.59:0/2609051045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9500d0ed0 0x7ff9501534b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 >> 172.21.15.59:0/2609051045 conn(0x7ff9500d2680 msgr2=0x7ff950070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 shutdown_connections 2024-04-24T06:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.167+0000 7ff956000640 1 -- 172.21.15.59:0/2609051045 wait complete. 2024-04-24T06:36:08.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:36:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.399+0000 7f1349000640 1 Processor -- start 2024-04-24T06:36:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.399+0000 7f1349000640 1 -- start start 2024-04-24T06:36:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.401+0000 7f1349000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 0x7f13440d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.401+0000 7f1349000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.401+0000 7f1349000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13440b5da0 con 0x7f13440b3600 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.401+0000 7f1349000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f134415f100 con 0x7f134415ed20 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.401+0000 7f1349000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f13440d07d0 con 0x7f13440d13e0 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.402+0000 7f1343400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 0x7f13440d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.402+0000 7f1343e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.402+0000 7f1343400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 0x7f13440d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:34826/0 (socket says 172.21.15.59:34826) 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.402+0000 7f1343e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:44184/0 (socket says 172.21.15.59:44184) 2024-04-24T06:36:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.402+0000 7f1343400640 1 -- 172.21.15.59:0/3068855954 learned_addr learned my addr 172.21.15.59:0/3068855954 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.403+0000 7f1343e00640 1 -- 172.21.15.59:0/3068855954 >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 msgr2=0x7f13440d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.403+0000 7f1343e00640 1 --2- 172.21.15.59:0/3068855954 >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 0x7f13440d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.403+0000 7f1343e00640 1 -- 172.21.15.59:0/3068855954 >> v1:172.21.15.66:6789/0 conn(0x7f13440b3600 legacy=0x7f13440b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.403+0000 7f1343e00640 1 -- 172.21.15.59:0/3068855954 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13440d0a20 con 0x7f13440d13e0 2024-04-24T06:36:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.405+0000 7f1343e00640 1 --2- 172.21.15.59:0/3068855954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f133c009d90 tx=0x7f133c02a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b177d9c2a1163cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.405+0000 7f1342a00640 1 -- 172.21.15.59:0/3068855954 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c007740 con 0x7f13440d13e0 2024-04-24T06:36:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.406+0000 7f1342a00640 1 -- 172.21.15.59:0/3068855954 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c0078e0 con 0x7f13440d13e0 2024-04-24T06:36:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.406+0000 7f1342a00640 1 -- 172.21.15.59:0/3068855954 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c007cb0 con 0x7f13440d13e0 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.406+0000 7f1349000640 1 -- 172.21.15.59:0/3068855954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 msgr2=0x7f13440d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.406+0000 7f1349000640 1 --2- 172.21.15.59:0/3068855954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f133c009d90 tx=0x7f133c02a530 comp rx=0 tx=0).stop 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 -- 172.21.15.59:0/3068855954 shutdown_connections 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 --2- 172.21.15.59:0/3068855954 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440d0290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 --2- 172.21.15.59:0/3068855954 >> v2:172.21.15.66:3300/0 conn(0x7f134415ed20 0x7f13440d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 -- 172.21.15.59:0/3068855954 >> 172.21.15.59:0/3068855954 conn(0x7f134408bf90 msgr2=0x7f13440d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 -- 172.21.15.59:0/3068855954 shutdown_connections 2024-04-24T06:36:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.407+0000 7f1349000640 1 -- 172.21.15.59:0/3068855954 wait complete. 2024-04-24T06:36:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.408+0000 7f1349000640 1 Processor -- start 2024-04-24T06:36:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 -- start start 2024-04-24T06:36:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134415ed20 0x7f13440cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13440d3490 con 0x7f13440b3600 2024-04-24T06:36:08.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f13440d31b0 con 0x7f134415ed20 2024-04-24T06:36:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.409+0000 7f1349000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f13440b4620 con 0x7f13440d13e0 2024-04-24T06:36:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35268/0 (socket says 172.21.15.59:35268) 2024-04-24T06:36:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 -- 172.21.15.59:0/505217481 learned_addr learned my addr 172.21.15.59:0/505217481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f133be00640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343e00640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134415ed20 0x7f13440cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 -- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 msgr2=0x7f13440a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 -- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134415ed20 msgr2=0x7f13440cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134415ed20 0x7f13440cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.410+0000 7f1343400640 1 -- 172.21.15.59:0/505217481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13440cd140 con 0x7f13440b3600 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.411+0000 7f1343400640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f133400e840 tx=0x7f133400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.411+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334017070 con 0x7f13440b3600 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.411+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133400c8e0 con 0x7f13440b3600 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.411+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133400ccd0 con 0x7f13440b3600 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.411+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13440cbb20 con 0x7f13440b3600 2024-04-24T06:36:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.412+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13440cc0e0 con 0x7f13440b3600 2024-04-24T06:36:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.413+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134404fd90 con 0x7f13440b3600 2024-04-24T06:36:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.414+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133401a070 con 0x7f13440b3600 2024-04-24T06:36:08.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.415+0000 7f133b400640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 0x7f132006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.415+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f133408baf0 con 0x7f13440b3600 2024-04-24T06:36:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.415+0000 7f133be00640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 0x7f132006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.416+0000 7f133be00640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 0x7f132006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f13441f7dc0 tx=0x7f132c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.419+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1334059e80 con 0x7f13440b3600 2024-04-24T06:36:08.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.622+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13440d54f0 con 0x7f13440b3600 2024-04-24T06:36:08.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.623+0000 7f133b400640 1 -- 172.21.15.59:0/505217481 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f1334059780 con 0x7f13440b3600 2024-04-24T06:36:08.625 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.625+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 msgr2=0x7f132006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.625+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 0x7f132006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f13441f7dc0 tx=0x7f132c009040 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.625+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 msgr2=0x7f13440ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.625+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f133400e840 tx=0x7f133400ed00 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 shutdown_connections 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f132006c8f0 0x7f132006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f134415ed20 0x7f13440cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f13440d13e0 0x7f13440a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 --2- 172.21.15.59:0/505217481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13440b3600 0x7f13440ca990 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 >> 172.21.15.59:0/505217481 conn(0x7f134408bf90 msgr2=0x7f13440d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 shutdown_connections 2024-04-24T06:36:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.626+0000 7f1349000640 1 -- 172.21.15.59:0/505217481 wait complete. 2024-04-24T06:36:08.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:36:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.863+0000 7f7bdaa00640 1 Processor -- start 2024-04-24T06:36:08.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.863+0000 7f7bdaa00640 1 -- start start 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bdaa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bd415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bdaa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd4160140 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bdaa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd415c180 con 0x7f7bd406ef00 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bdaa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd40b2f50 con 0x7f7bd415bc00 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bd8a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7bd40ce7a0 0x7f7bd40cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44502/0 (socket says 172.21.15.59:44502) 2024-04-24T06:36:08.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.865+0000 7f7bd8a00640 1 -- 172.21.15.59:0/3497864101 learned_addr learned my addr 172.21.15.59:0/3497864101 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bcbe00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bd415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893652576 0 0) 0x7f7bd4160140 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bb8005730 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 328073489 0 0) 0x7f7bb8005730 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 msgr2=0x7f7bd415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bd415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 >> v1:172.21.15.66:6789/0 conn(0x7f7bd406ef00 legacy=0x7f7bd406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd415c460 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f7bcc0019f0 con 0x7f7bd40ce7a0 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 >> v1:172.21.15.59:6789/0 conn(0x7f7bd40ce7a0 legacy=0x7f7bd40cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bb8005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bb8006130 0x7f7bb8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bb80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800ffd0 con 0x7f7bb8006f50 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8010150 con 0x7f7bb8006130 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.866+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb80102d0 con 0x7f7bd415bc00 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bb80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bcbe00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bb8006130 0x7f7bb8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd3e00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bb8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 -- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 msgr2=0x7f7bb8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bb8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 -- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bb8006130 msgr2=0x7f7bb8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bb8006130 0x7f7bb8006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd8a00640 1 -- 172.21.15.59:0/3497864101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb8005730 con 0x7f7bb8006f50 2024-04-24T06:36:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.867+0000 7f7bd3e00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd415bc00 0x7f7bb8005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.868+0000 7f7bd8a00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bb80073d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00a850 tx=0x7f7bb8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8d428a93e52cce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.868+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00a4a0 con 0x7f7bb8006f50 2024-04-24T06:36:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc02f070 con 0x7f7bb8006f50 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bd3400640 1 -- 172.21.15.59:0/3497864101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc02bd20 con 0x7f7bb8006f50 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 -- 172.21.15.59:0/3497864101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 msgr2=0x7f7bb80073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/3497864101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bb80073d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00a850 tx=0x7f7bb8007a40 comp rx=0 tx=0).stop 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bd8a00640 1 -- 172.21.15.59:0/3497864101 reap_dead start 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 -- 172.21.15.59:0/3497864101 shutdown_connections 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 -- 172.21.15.59:0/3497864101 >> 172.21.15.59:0/3497864101 conn(0x7f7bd40b2130 msgr2=0x7f7bd40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 -- 172.21.15.59:0/3497864101 shutdown_connections 2024-04-24T06:36:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.869+0000 7f7bdaa00640 1 -- 172.21.15.59:0/3497864101 wait complete. 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.871+0000 7f7bdaa00640 1 Processor -- start 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.871+0000 7f7bdaa00640 1 -- start start 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd414fc40 0x7f7bd4072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd4148280 0x7f7bd4148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd40ce210 con 0x7f7bb8006f50 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd415ca20 con 0x7f7bd414fc40 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bdaa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd4155ff0 con 0x7f7bd4148280 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bd3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32848/0 (socket says 172.21.15.59:32848) 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.872+0000 7f7bd3e00640 1 -- 172.21.15.59:0/1543797449 learned_addr learned my addr 172.21.15.59:0/1543797449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bcbe00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd414fc40 0x7f7bd4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 -- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd4148280 msgr2=0x7f7bd4148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd4148280 0x7f7bd4148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 -- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd414fc40 msgr2=0x7f7bd4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd414fc40 0x7f7bd4072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 -- 172.21.15.59:0/1543797449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd4148c50 con 0x7f7bb8006f50 2024-04-24T06:36:08.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd3e00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d7a0 tx=0x7f7bc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4005070 con 0x7f7bb8006f50 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4005210 con 0x7f7bb8006f50 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.873+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4007030 con 0x7f7bb8006f50 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.874+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd415d7d0 con 0x7f7bb8006f50 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.874+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd415dd90 con 0x7f7bb8006f50 2024-04-24T06:36:08.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.875+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd4068a80 con 0x7f7bb8006f50 2024-04-24T06:36:08.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.876+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bc401a020 con 0x7f7bb8006f50 2024-04-24T06:36:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.877+0000 7f7bd0c00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.877+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f7bc408b360 con 0x7f7bb8006f50 2024-04-24T06:36:08.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.878+0000 7f7bcbe00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:08.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.878+0000 7f7bcbe00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 0x7f7bb006ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7bd415b850 tx=0x7f7bbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:08.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:08.882+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bc4059790 con 0x7f7bb8006f50 2024-04-24T06:36:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.099+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7bd40afbb0 con 0x7f7bb8006f50 2024-04-24T06:36:09.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.100+0000 7f7bd0c00640 1 -- 172.21.15.59:0/1543797449 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7f7bc4059090 con 0x7f7bb8006f50 2024-04-24T06:36:09.102 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:36:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 msgr2=0x7f7bb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 0x7f7bb006ede0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7bd415b850 tx=0x7f7bbc008040 comp rx=0 tx=0).stop 2024-04-24T06:36:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 msgr2=0x7f7bd415ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d7a0 tx=0x7f7bc400dc60 comp rx=0 tx=0).stop 2024-04-24T06:36:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 shutdown_connections 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7bb006c920 0x7f7bb006ede0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7bd4148280 0x7f7bd4148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7bd414fc40 0x7f7bd4072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 --2- 172.21.15.59:0/1543797449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb8006f50 0x7f7bd415ce80 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 >> 172.21.15.59:0/1543797449 conn(0x7f7bd40b2130 msgr2=0x7f7bd406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 shutdown_connections 2024-04-24T06:36:09.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:09.102+0000 7f7bdaa00640 1 -- 172.21.15.59:0/1543797449 wait complete. 2024-04-24T06:36:09.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:36:09.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:09.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:09.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63224]: fuse finished with error 0 and tester_r 0 2024-04-24T06:36:09.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:36:15.311 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:36:15.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:15.312 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:36:15.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:15.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.590+0000 7fc70ea00640 1 Processor -- start 2024-04-24T06:36:15.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.591+0000 7fc70ea00640 1 -- start start 2024-04-24T06:36:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.592+0000 7fc70ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc70815c3d0 0x7fc7080cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.592+0000 7fc70ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080b4860 0x7fc7080cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.592+0000 7fc70ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc708160210 con 0x7fc7080cd590 2024-04-24T06:36:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.592+0000 7fc70ea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc7081604f0 con 0x7fc70815c3d0 2024-04-24T06:36:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.592+0000 7fc70ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7081607d0 con 0x7fc7080b4860 2024-04-24T06:36:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc70815c3d0 0x7fc7080cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc70815c3d0 0x7fc7080cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32864/0 (socket says 172.21.15.59:32864) 2024-04-24T06:36:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 -- 172.21.15.59:0/3731617926 learned_addr learned my addr 172.21.15.59:0/3731617926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70e000640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080b4860 0x7fc7080cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 -- 172.21.15.59:0/3731617926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080b4860 msgr2=0x7fc7080cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080b4860 0x7fc7080cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 -- 172.21.15.59:0/3731617926 >> v1:172.21.15.66:6789/0 conn(0x7fc7080cd590 legacy=0x7fc7080b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.593+0000 7fc70d600640 1 -- 172.21.15.59:0/3731617926 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc708160ab0 con 0x7fc70815c3d0 2024-04-24T06:36:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.594+0000 7fc70d600640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc70815c3d0 0x7fc7080cd050 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009e50 tx=0x7fc6fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d65813f307ca3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.595+0000 7fc707e00640 1 -- 172.21.15.59:0/3731617926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc036070 con 0x7fc70815c3d0 2024-04-24T06:36:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.595+0000 7fc707e00640 1 -- 172.21.15.59:0/3731617926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc031600 con 0x7fc70815c3d0 2024-04-24T06:36:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.595+0000 7fc707e00640 1 -- 172.21.15.59:0/3731617926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0319d0 con 0x7fc70815c3d0 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.595+0000 7fc70ea00640 1 -- 172.21.15.59:0/3731617926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc70815c3d0 msgr2=0x7fc7080cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.595+0000 7fc70ea00640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc70815c3d0 0x7fc7080cd050 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009e50 tx=0x7fc6fc02a710 comp rx=0 tx=0).stop 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 -- 172.21.15.59:0/3731617926 shutdown_connections 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080b4860 0x7fc7080cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 --2- 172.21.15.59:0/3731617926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc70815c3d0 0x7fc7080cd050 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 -- 172.21.15.59:0/3731617926 >> 172.21.15.59:0/3731617926 conn(0x7fc7080b21a0 msgr2=0x7fc7080b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 -- 172.21.15.59:0/3731617926 shutdown_connections 2024-04-24T06:36:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.596+0000 7fc70ea00640 1 -- 172.21.15.59:0/3731617926 wait complete. 2024-04-24T06:36:15.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.597+0000 7fc70ea00640 1 Processor -- start 2024-04-24T06:36:15.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 -- start start 2024-04-24T06:36:15.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080cd590 0x7fc7080ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 0x7fc7080ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc70815c930 con 0x7fc7080b4860 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc70815c7b0 con 0x7fc70815c3d0 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.598+0000 7fc70ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7080cbd10 con 0x7fc7080cd590 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32880/0 (socket says 172.21.15.59:32880) 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 -- 172.21.15.59:0/4187403762 learned_addr learned my addr 172.21.15.59:0/4187403762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70cc00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080cd590 0x7fc7080ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70e000640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 0x7fc7080ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 -- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080cd590 msgr2=0x7fc7080ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080cd590 0x7fc7080ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 -- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 msgr2=0x7fc7080ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 0x7fc7080ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70d600640 1 -- 172.21.15.59:0/4187403762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7080ca350 con 0x7fc7080b4860 2024-04-24T06:36:15.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.599+0000 7fc70e000640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 0x7fc7080ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:15.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc70d600640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009f70 tx=0x7fc6fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:15.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc041020 con 0x7fc7080b4860 2024-04-24T06:36:15.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc008790 con 0x7fc7080b4860 2024-04-24T06:36:15.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc036040 con 0x7fc7080b4860 2024-04-24T06:36:15.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70815f670 con 0x7fc7080b4860 2024-04-24T06:36:15.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.600+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc708150d20 con 0x7fc7080b4860 2024-04-24T06:36:15.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.602+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70804fd90 con 0x7fc7080b4860 2024-04-24T06:36:15.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.603+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6fc044020 con 0x7fc7080b4860 2024-04-24T06:36:15.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.603+0000 7fc705600640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:15.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.604+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fc6fc0ac8e0 con 0x7fc7080b4860 2024-04-24T06:36:15.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.604+0000 7fc70cc00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:15.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.605+0000 7fc70cc00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 0x7fc6e806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc7080d0f30 tx=0x7fc6f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:15.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.609+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6fc04b070 con 0x7fc7080b4860 2024-04-24T06:36:15.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.859+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc70806e400 con 0x7fc7080b4860 2024-04-24T06:36:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.860+0000 7fc705600640 1 -- 172.21.15.59:0/4187403762 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5177 (secure 0 0 0) 0x7fc6fc07ac70 con 0x7fc7080b4860 2024-04-24T06:36:15.862 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:15.862 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7548,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:35:57.406140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7563":{"gid":7563,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6833/301885470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":301885470},{"type":"v1","addr":"172.21.15.66:6833","nonce":301885470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7563,"qdb_cluster":[7563]},"id":6}]} 2024-04-24T06:36:15.862 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 61 2024-04-24T06:36:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 msgr2=0x7fc6e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:15.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 0x7fc6e806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc7080d0f30 tx=0x7fc6f4009040 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 msgr2=0x7fc7080c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009f70 tx=0x7fc6fc004a40 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 shutdown_connections 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc70815c3d0 0x7fc7080ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7080cd590 0x7fc7080ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 --2- 172.21.15.59:0/4187403762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7080b4860 0x7fc7080c9e10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 >> 172.21.15.59:0/4187403762 conn(0x7fc7080b21a0 msgr2=0x7fc7080d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 shutdown_connections 2024-04-24T06:36:15.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:15.863+0000 7fc70ea00640 1 -- 172.21.15.59:0/4187403762 wait complete. 2024-04-24T06:36:15.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-24T06:36:16.153 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:36:16.154 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:36:16.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:36:16.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.370+0000 7fa0eb200640 1 Processor -- start 2024-04-24T06:36:16.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.370+0000 7fa0eb200640 1 -- start start 2024-04-24T06:36:16.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0eb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0e415c670 0x7fa0e40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0eb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e40b4cf0 con 0x7fa0e40b4230 2024-04-24T06:36:16.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0eb200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e40ccdd0 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0eb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e415ca50 con 0x7fa0e415c670 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0e9400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa0e40b4230 0x7fa0e40d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46802/0 (socket says 172.21.15.59:46802) 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0e9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0e415c670 0x7fa0e40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.372+0000 7fa0e9400640 1 -- 172.21.15.59:0/1017608790 learned_addr learned my addr 172.21.15.59:0/1017608790 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.373+0000 7fa0e9e00640 1 -- 172.21.15.59:0/1017608790 >> v1:172.21.15.66:6789/0 conn(0x7fa0e40d0cd0 legacy=0x7fa0e406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.373+0000 7fa0e9e00640 1 -- 172.21.15.59:0/1017608790 >> v1:172.21.15.59:6789/0 conn(0x7fa0e40b4230 legacy=0x7fa0e40d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.373+0000 7fa0e9e00640 1 -- 172.21.15.59:0/1017608790 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e40cd0b0 con 0x7fa0e415c670 2024-04-24T06:36:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0e9e00640 1 --2- 172.21.15.59:0/1017608790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e415c670 0x7fa0e40cd8b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80097a0 tx=0x7fa0d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6437fcee1e9e95d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0e8a00640 1 -- 172.21.15.59:0/1017608790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8037070 con 0x7fa0e415c670 2024-04-24T06:36:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0e8a00640 1 -- 172.21.15.59:0/1017608790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d8032440 con 0x7fa0e415c670 2024-04-24T06:36:16.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0e8a00640 1 -- 172.21.15.59:0/1017608790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8032810 con 0x7fa0e415c670 2024-04-24T06:36:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0eb200640 1 -- 172.21.15.59:0/1017608790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e415c670 msgr2=0x7fa0e40cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.374+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1017608790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e415c670 0x7fa0e40cd8b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80097a0 tx=0x7fa0d802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.375+0000 7fa0eb200640 1 -- 172.21.15.59:0/1017608790 shutdown_connections 2024-04-24T06:36:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.375+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1017608790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e415c670 0x7fa0e40cd8b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.375+0000 7fa0eb200640 1 -- 172.21.15.59:0/1017608790 >> 172.21.15.59:0/1017608790 conn(0x7fa0e406bdf0 msgr2=0x7fa0e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.375+0000 7fa0eb200640 1 -- 172.21.15.59:0/1017608790 shutdown_connections 2024-04-24T06:36:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.375+0000 7fa0eb200640 1 -- 172.21.15.59:0/1017608790 wait complete. 2024-04-24T06:36:16.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.377+0000 7fa0eb200640 1 Processor -- start 2024-04-24T06:36:16.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.377+0000 7fa0eb200640 1 -- start start 2024-04-24T06:36:16.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e40b4230 0x7fa0e41f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0e415c670 0x7fa0e4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40b3e60 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40ccb80 con 0x7fa0e40b4230 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0eb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40b3fe0 con 0x7fa0e415c670 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0e9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0e9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32898/0 (socket says 172.21.15.59:32898) 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0e9400640 1 -- 172.21.15.59:0/1327223981 learned_addr learned my addr 172.21.15.59:0/1327223981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.378+0000 7fa0e9e00640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e40b4230 0x7fa0e41f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 -- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0e415c670 msgr2=0x7fa0e4157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0e415c670 0x7fa0e4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 -- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e40b4230 msgr2=0x7fa0e41f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e40b4230 0x7fa0e41f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 -- 172.21.15.59:0/1327223981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e4158720 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0e9400640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c470 tx=0x7fa0d4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4005ce0 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40072f0 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e4158a00 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.379+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e4158fc0 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.381+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e404fd90 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.382+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0d401a070 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.383+0000 7fa0de000640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.383+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa0d408ae50 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.383+0000 7fa0e9e00640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.384+0000 7fa0e9e00640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 0x7fa0c806edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034000 tx=0x7fa0d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.388+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0d4059310 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.637+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa0e40ceab0 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.641 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:36:16.638+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:36:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.642+0000 7fa0de000640 1 -- 172.21.15.59:0/1327223981 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7fa0d4058c10 con 0x7fa0e40d0cd0 2024-04-24T06:36:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:36:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 msgr2=0x7fa0c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 0x7fa0c806edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034000 tx=0x7fa0d802e000 comp rx=0 tx=0).stop 2024-04-24T06:36:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 msgr2=0x7fa0e414b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c470 tx=0x7fa0d4001e70 comp rx=0 tx=0).stop 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 shutdown_connections 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0e415c670 0x7fa0e4157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40d0cd0 0x7fa0e414b580 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 --2- 172.21.15.59:0/1327223981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0e40b4230 0x7fa0e41f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 >> 172.21.15.59:0/1327223981 conn(0x7fa0e406bdf0 msgr2=0x7fa0e4073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 shutdown_connections 2024-04-24T06:36:16.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.645+0000 7fa0eb200640 1 -- 172.21.15.59:0/1327223981 wait complete. 2024-04-24T06:36:16.666 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-24T06:36:16.675 INFO:tasks.ceph.mds.d.smithi066.stderr: -4048> 2024-04-24T06:36:00.803+0000 7f40f6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.759+0000 7f793933ef40 1 Processor -- start 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.759+0000 7f793933ef40 1 -- start start 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daeb6c1d0 0x562daec48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daec2bac0 0x562daec46f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562daeb6e9c0 con 0x562daec46ad0 2024-04-24T06:36:16.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562daeb15cc0 con 0x562daeb6c1d0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f793933ef40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562daeb6cb20 con 0x562daec2bac0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.760+0000 7f7937000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7936600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daeb6c1d0 0x562daec48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7935c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daec2bac0 0x562daec46f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32908/0 (socket says 172.21.15.59:32908) 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 -- 172.21.15.59:0/2051297495 learned_addr learned my addr 172.21.15.59:0/2051297495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 -- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daec2bac0 msgr2=0x562daec46f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daec2bac0 0x562daec46f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 -- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daeb6c1d0 msgr2=0x562daec48950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daeb6c1d0 0x562daec48950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 -- 172.21.15.59:0/2051297495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562daeb58ce0 con 0x562daec46ad0 2024-04-24T06:36:16.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.761+0000 7f7937000640 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f793000a770 tx=0x7f793000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6978afdd3f0efe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.765 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f792fe00640 1 -- 172.21.15.59:0/2051297495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930018070 con 0x562daec46ad0 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f792fe00640 1 -- 172.21.15.59:0/2051297495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930007600 con 0x562daec46ad0 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f792fe00640 1 -- 172.21.15.59:0/2051297495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300079d0 con 0x562daec46ad0 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 -- 172.21.15.59:0/2051297495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 msgr2=0x562daec474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f793000a770 tx=0x7f793000a860 comp rx=0 tx=0).stop 2024-04-24T06:36:16.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 -- 172.21.15.59:0/2051297495 shutdown_connections 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daec474b0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daec2bac0 0x562daec46f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 --2- 172.21.15.59:0/2051297495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daeb6c1d0 0x562daec48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 -- 172.21.15.59:0/2051297495 >> 172.21.15.59:0/2051297495 conn(0x562daebb8cc0 msgr2=0x562daebb7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 -- 172.21.15.59:0/2051297495 shutdown_connections 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.762+0000 7f793933ef40 1 -- 172.21.15.59:0/2051297495 wait complete. 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.763+0000 7f793933ef40 1 Processor -- start 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.763+0000 7f793933ef40 1 -- start start 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daeb6c1d0 0x562daec412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daec2bac0 0x562daeb9c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daeb9c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562daec43340 con 0x562daec46ad0 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562daec49ef0 con 0x562daec2bac0 2024-04-24T06:36:16.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f793933ef40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562daec43060 con 0x562daeb6c1d0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f7937000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daeb9c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f7937000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daeb9c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32920/0 (socket says 172.21.15.59:32920) 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.764+0000 7f7937000640 1 -- 172.21.15.59:0/2845522317 learned_addr learned my addr 172.21.15.59:0/2845522317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7935c00640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daec2bac0 0x562daeb9c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7936600640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daeb6c1d0 0x562daec412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 -- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daeb6c1d0 msgr2=0x562daec412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daeb6c1d0 0x562daec412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 -- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daec2bac0 msgr2=0x562daeb9c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562daec2bac0 0x562daeb9c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 -- 172.21.15.59:0/2845522317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562daecdbfe0 con 0x562daec46ad0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7936600640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562daeb6c1d0 0x562daec412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.765+0000 7f7937000640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daeb9c5b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x562daeba2230 tx=0x7f7930003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.766+0000 7f792ea00640 1 -- 172.21.15.59:0/2845522317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930021020 con 0x562daec46ad0 2024-04-24T06:36:16.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.766+0000 7f792ea00640 1 -- 172.21.15.59:0/2845522317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300077b0 con 0x562daec46ad0 2024-04-24T06:36:16.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.766+0000 7f792ea00640 1 -- 172.21.15.59:0/2845522317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930018040 con 0x562daec46ad0 2024-04-24T06:36:16.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.766+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562daec2c8d0 con 0x562daec46ad0 2024-04-24T06:36:16.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.766+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562daec47110 con 0x562daec46ad0 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.768+0000 7f792ea00640 1 -- 172.21.15.59:0/2845522317 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7930024020 con 0x562daec46ad0 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.769+0000 7f792ea00640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f791806b2d0 0x7f791806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.769+0000 7f792ea00640 1 -- 172.21.15.59:0/2845522317 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f793008b060 con 0x562daec46ad0 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.769+0000 7f7936600640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f791806b2d0 0x7f791806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.769+0000 7f793933ef40 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x562daec48d40 0x562daec45480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.769+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e84) -- 0x562daec473c0 con 0x562daec48d40 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.770+0000 7f7935c00640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x562daec48d40 0x562daec45480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.770+0000 7f7936600640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f791806b2d0 0x7f791806d790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7928006fd0 tx=0x7f7928008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.771+0000 7f7935c00640 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x562daec48d40 0x562daec45480 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f7935c00640 1 -- 172.21.15.59:0/2845522317 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v84'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f792405c040 con 0x562daec48d40 2024-04-24T06:36:16.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x562daec48d40 msgr2=0x562daec45480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x562daec48d40 0x562daec45480 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f791806b2d0 msgr2=0x7f791806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f791806b2d0 0x7f791806d790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7928006fd0 tx=0x7f7928008040 comp rx=0 tx=0).stop 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 msgr2=0x562daeb9c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 --2- 172.21.15.59:0/2845522317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562daec46ad0 0x562daeb9c5b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x562daeba2230 tx=0x7f7930003020 comp rx=0 tx=0).stop 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f7937000640 1 -- 172.21.15.59:0/2845522317 reap_dead start 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 shutdown_connections 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 >> 172.21.15.59:0/2845522317 conn(0x562daebb8cc0 msgr2=0x562daebb90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 shutdown_connections 2024-04-24T06:36:16.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.773+0000 7f793933ef40 1 -- 172.21.15.59:0/2845522317 wait complete. 2024-04-24T06:36:16.780 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-24T06:36:16.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.864+0000 7ff430b98280 1 -- start start 2024-04-24T06:36:16.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.864+0000 7ff430b98280 1 Processor -- start 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614efe86c00 0x5614efffa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614efe87800 0x5614efffa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf2600 con 0x5614efe87c00 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf2480 con 0x5614efe87800 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5614f0bf2300 con 0x5614efe86c00 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32924/0 (socket says 172.21.15.59:32924) 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 -- 172.21.15.59:0/218099693 learned_addr learned my addr 172.21.15.59:0/218099693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 -- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614efe86c00 msgr2=0x5614efffa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614efe86c00 0x5614efffa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 -- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614efe87800 msgr2=0x5614efffa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614efe87800 0x5614efffa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.865+0000 7ff42f600640 1 -- 172.21.15.59:0/218099693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efeba780 con 0x5614efe87c00 2024-04-24T06:36:16.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42f600640 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x5614f0ca9380 tx=0x5614f0c20ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29806e4ab7c540e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42ba00640 1 -- 172.21.15.59:0/218099693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac9c0 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42ba00640 1 -- 172.21.15.59:0/218099693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5614efeb2c40 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42ba00640 1 -- 172.21.15.59:0/218099693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac680 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efeba3c0 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5614efebb680 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff430b98280 4 waiting for MDS map... 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42ba00640 1 -- 172.21.15.59:0/218099693 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5614f0d54200 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.866+0000 7ff42ba00640 1 -- 172.21.15.59:0/218099693 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x5614efee1180 con 0x5614efe87c00 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.867+0000 7ff430b98280 4 Got MDS map 62 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.867+0000 7ff430b98280 10 main: JournalTool::main 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.867+0000 7ff430b98280 4 main: JournalTool: connecting to RADOS... 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.867+0000 7ff430b98280 1 Processor -- start 2024-04-24T06:36:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 -- start start 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf2180 con 0x5614f0db0000 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614efee8480 con 0x5614f0db0400 2024-04-24T06:36:16.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.868+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5614efee8900 con 0x5614f0db0800 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32926/0 (socket says 172.21.15.59:32926) 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 -- 172.21.15.59:0/1096645979 learned_addr learned my addr 172.21.15.59:0/1096645979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 -- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 msgr2=0x5614efffbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 -- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0400 msgr2=0x5614efffb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 -- 172.21.15.59:0/1096645979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efebb4a0 con 0x5614f0db0000 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.869+0000 7ff42e200640 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5eb10 tx=0x5614f0d68d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e129f5cb8f00efa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff429200640 1 -- 172.21.15.59:0/1096645979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac9c0 con 0x5614f0db0000 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff429200640 1 -- 172.21.15.59:0/1096645979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5614efeb2700 con 0x5614f0db0000 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff429200640 1 -- 172.21.15.59:0/1096645979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac680 con 0x5614f0db0000 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 -- 172.21.15.59:0/1096645979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 msgr2=0x5614efffb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5eb10 tx=0x5614f0d68d20 comp rx=0 tx=0).stop 2024-04-24T06:36:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 -- 172.21.15.59:0/1096645979 shutdown_connections 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 --2- 172.21.15.59:0/1096645979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 -- 172.21.15.59:0/1096645979 >> 172.21.15.59:0/1096645979 conn(0x5614f0c1fc00 msgr2=0x5614f0c0a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 -- 172.21.15.59:0/1096645979 shutdown_connections 2024-04-24T06:36:16.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.870+0000 7ff430b98280 1 -- 172.21.15.59:0/1096645979 wait complete. 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.871+0000 7ff430b98280 1 Processor -- start 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.871+0000 7ff430b98280 1 -- start start 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf3380 con 0x5614f0db0400 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf2180 con 0x5614f0db0000 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5614f0bf3500 con 0x5614f0db0800 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff42ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff42ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32928/0 (socket says 172.21.15.59:32928) 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.872+0000 7ff42ec00640 1 -- 172.21.15.59:0/2386123156 learned_addr learned my addr 172.21.15.59:0/2386123156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 -- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 msgr2=0x5614efffbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 -- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0000 msgr2=0x5614efffb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 -- 172.21.15.59:0/2386123156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efeba000 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff42ec00640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5f7d0 tx=0x5614f0d68f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff427400640 1 -- 172.21.15.59:0/2386123156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac9c0 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff427400640 1 -- 172.21.15.59:0/2386123156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5614efeb2a80 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614efeba960 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff427400640 1 -- 172.21.15.59:0/2386123156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614efeac680 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efebb860 con 0x5614f0db0400 2024-04-24T06:36:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.873+0000 7ff430b98280 4 main: JournalTool: resolving pool 12 2024-04-24T06:36:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.875+0000 7ff427400640 1 -- 172.21.15.59:0/2386123156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5614f0dc0c00 con 0x5614f0db0400 2024-04-24T06:36:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.876+0000 7ff427400640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 0x5614efffc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.876+0000 7ff427400640 1 -- 172.21.15.59:0/2386123156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5614f0dbf000 con 0x5614f0db0400 2024-04-24T06:36:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.876+0000 7ff430b98280 4 main: JournalTool: creating IoCtx.. 2024-04-24T06:36:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.876+0000 7ff430b98280 4 main: Executing for rank 0 2024-04-24T06:36:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.876+0000 7ff42e200640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 0x5614efffc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.877+0000 7ff42e200640 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 0x5614efffc100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5eb10 tx=0x5614f0d68c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.877+0000 7ff430b98280 1 -- start start 2024-04-24T06:36:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.877+0000 7ff430b98280 1 Processor -- start 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db1000 0x5614efffc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 0x5614efffcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0dcd400 0x5614efffd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf3800 con 0x5614f0dcd000 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614f0bf3980 con 0x5614f0db1000 2024-04-24T06:36:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff430b98280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5614efee8900 con 0x5614f0dcd400 2024-04-24T06:36:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.878+0000 7ff42ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 0x5614efffcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 0x5614efffcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32942/0 (socket says 172.21.15.59:32942) 2024-04-24T06:36:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 -- 172.21.15.59:0/4136783538 learned_addr learned my addr 172.21.15.59:0/4136783538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db1000 0x5614efffc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0dcd400 msgr2=0x5614efffd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0dcd400 0x5614efffd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db1000 msgr2=0x5614efffc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db1000 0x5614efffc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efebba40 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db1000 0x5614efffc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff42ec00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 0x5614efffcc00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x5614f01a74a0 tx=0x5614f0d698c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e682b09d8235234 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.879+0000 7ff423800640 1 -- 172.21.15.59:0/4136783538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614f018b860 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff423800640 1 -- 172.21.15.59:0/4136783538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5614efeb28c0 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff423800640 1 -- 172.21.15.59:0/4136783538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5614f018ba00 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efeba000 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5614efeba960 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 4 waiting for MDS map... 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff423800640 1 -- 172.21.15.59:0/4136783538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5614f0262200 con 0x5614f0dcd000 2024-04-24T06:36:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff423800640 1 -- 172.21.15.59:0/4136783538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x5614efee1500 con 0x5614f0dcd000 2024-04-24T06:36:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 4 Got MDS map 62 2024-04-24T06:36:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5614f0dcd800 0x5614efffd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.880+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84) -- 0x5614f0dcdc00 con 0x5614f0dcd800 2024-04-24T06:36:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.881+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5614f0dcd800 0x5614efffd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.882+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5614f0dcd800 0x5614efffd700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=c78d74d291aed987 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.883+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5614f0c10000 con 0x5614f0dcd800 2024-04-24T06:36:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.883+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5614f0274400 0x5614f0282000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.883+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84) -- 0x5614f0274800 con 0x5614f0274400 2024-04-24T06:36:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.884+0000 7ff42e200640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5614f0274400 0x5614f0282000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.885+0000 7ff42e200640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5614f0274400 0x5614f0282000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=c4af45e18b6ff1ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.891+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5614f0c10500 con 0x5614f0274400 2024-04-24T06:36:16.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.892+0000 7ff421a00640 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84) -- 0x5614f028a000 con 0x5614f0274400 2024-04-24T06:36:16.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.892+0000 7ff421a00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5614f028a400 0x5614f0283b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.892+0000 7ff421a00640 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84) -- 0x5614f028a800 con 0x5614f028a400 2024-04-24T06:36:16.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.893+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5614f0c10500 con 0x5614f0274400 2024-04-24T06:36:16.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.893+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5614f028a400 0x5614f0283b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.894+0000 7ff42f600640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5614f028a400 0x5614f0283b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=de3dc5fcda2f2696 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.902+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f028a400 2024-04-24T06:36:16.903 INFO:teuthology.orchestra.run.smithi059.stdout:old journal was 4209834~892 2024-04-24T06:36:16.903 INFO:teuthology.orchestra.run.smithi059.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-24T06:36:16.904 INFO:teuthology.orchestra.run.smithi059.stdout:writing journal head 2024-04-24T06:36:16.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.902+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f0274c00 con 0x5614f0274400 2024-04-24T06:36:16.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v84'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10500 con 0x5614f0274400 2024-04-24T06:36:16.909 INFO:teuthology.orchestra.run.smithi059.stdout:writing EResetJournal entry 2024-04-24T06:36:16.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f0275000 con 0x5614f028a400 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5614f0275400 0x5614f0282580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f0275800 con 0x5614f0275400 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f0275c00 con 0x5614f0dcd800 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f028a000 con 0x5614f028a400 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f028ac00 con 0x5614f028a400 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5614f028b000 0x5614f0282b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f028b400 con 0x5614f028b000 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.907+0000 7ff42ec00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5614f0275400 0x5614f0282580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.908+0000 7ff42e200640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5614f028b000 0x5614f0282b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.908+0000 7ff42ec00640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5614f0275400 0x5614f0282580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=773c8dcb7411e1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.908+0000 7ff42e200640 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5614f028b000 0x5614f0282b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=77ec4efc16bb8c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:16.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.910+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(6 200.00000002 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f028a400 2024-04-24T06:36:16.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.911+0000 7ff42ec00640 1 -- 172.21.15.59:0/4136783538 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(7 200.00000003 [delete] v84'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10780 con 0x5614f0275400 2024-04-24T06:36:16.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.911+0000 7ff421a00640 1 -- 172.21.15.59:0/4136783538 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5614f0275000 con 0x5614f028a400 2024-04-24T06:36:16.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.911+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(8 200.00000004 [delete] v84'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f0dcd800 2024-04-24T06:36:16.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.913+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(11 200.00000007 [delete] v84'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10500 con 0x5614f028b000 2024-04-24T06:36:16.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.913+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(10 200.00000006 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f028a400 2024-04-24T06:36:16.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.913+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(9 200.00000005 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f028a400 2024-04-24T06:36:16.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff42f600640 1 -- 172.21.15.59:0/4136783538 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v84'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5614f0c10000 con 0x5614f028a400 2024-04-24T06:36:16.917 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-04-24T06:36:16.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5614f028b000 msgr2=0x5614f0282b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5614f028b000 0x5614f0282b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5614f0275400 msgr2=0x5614f0282580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5614f0275400 0x5614f0282580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5614f028a400 msgr2=0x5614f0283b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5614f028a400 0x5614f0283b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5614f0274400 msgr2=0x5614f0282000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5614f0274400 0x5614f0282000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5614f0dcd800 msgr2=0x5614efffd700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5614f0dcd800 0x5614efffd700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 msgr2=0x5614efffcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 --2- 172.21.15.59:0/4136783538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0dcd000 0x5614efffcc00 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x5614f01a74a0 tx=0x5614f0d698c0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 reap_dead start 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 reap_dead start 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 shutdown_connections 2024-04-24T06:36:16.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 >> 172.21.15.59:0/4136783538 conn(0x5614f0db0c00 msgr2=0x5614f0c0a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.915+0000 7ff42e200640 1 -- 172.21.15.59:0/4136783538 reap_dead start 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 shutdown_connections 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/4136783538 wait complete. 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 msgr2=0x5614efffc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 0x5614efffc100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5eb10 tx=0x5614f0d68c00 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 msgr2=0x5614efffb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x5614f0d5f7d0 tx=0x5614f0d68f60 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 shutdown_connections 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5614f0dccc00 0x5614efffc100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614f0db0800 0x5614efffbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614f0db0400 0x5614efffb600 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 --2- 172.21.15.59:0/2386123156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614f0db0000 0x5614efffb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 >> 172.21.15.59:0/2386123156 conn(0x5614f0c1fc00 msgr2=0x5614f0c0a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 shutdown_connections 2024-04-24T06:36:16.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.916+0000 7ff430b98280 1 -- 172.21.15.59:0/2386123156 wait complete. 2024-04-24T06:36:16.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 msgr2=0x5614efffab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:16.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x5614f0ca9380 tx=0x5614f0c20ae0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 shutdown_connections 2024-04-24T06:36:16.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5614efe87c00 0x5614efffab00 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614efe87800 0x5614efffa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 --2- 172.21.15.59:0/218099693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5614efe86c00 0x5614efffa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:16.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 >> 172.21.15.59:0/218099693 conn(0x5614efe87000 msgr2=0x5614f0c0b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:16.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 shutdown_connections 2024-04-24T06:36:16.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:16.917+0000 7ff430b98280 1 -- 172.21.15.59:0/218099693 wait complete. 2024-04-24T06:36:16.926 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.146192, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-24T06:36:16.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:36:17.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.128+0000 7fdd6f000640 1 Processor -- start 2024-04-24T06:36:17.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.128+0000 7fdd6f000640 1 -- start start 2024-04-24T06:36:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdd680d0d10 0x7fdd68156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6f000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdd6815d810 0x7fdd68160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6f000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6815cab0 con 0x7fdd681567e0 2024-04-24T06:36:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdd68160e40 con 0x7fdd680d0d10 2024-04-24T06:36:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6f000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6815cd30 con 0x7fdd6815d810 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdd680d0d10 0x7fdd68156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6dc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdd681567e0 0x7fdd680d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46836/0 (socket says 172.21.15.59:46836) 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.130+0000 7fdd6dc00640 1 -- 172.21.15.59:0/363943456 learned_addr learned my addr 172.21.15.59:0/363943456 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.131+0000 7fdd6e600640 1 --2- 172.21.15.59:0/363943456 >> v2:172.21.15.66:3300/0 conn(0x7fdd6815d810 0x7fdd68160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.131+0000 7fdd6d200640 1 -- 172.21.15.59:0/363943456 >> v2:172.21.15.66:3300/0 conn(0x7fdd6815d810 msgr2=0x7fdd68160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.131+0000 7fdd6d200640 1 --2- 172.21.15.59:0/363943456 >> v2:172.21.15.66:3300/0 conn(0x7fdd6815d810 0x7fdd68160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.131+0000 7fdd6d200640 1 -- 172.21.15.59:0/363943456 >> v1:172.21.15.59:6789/0 conn(0x7fdd681567e0 legacy=0x7fdd680d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.131+0000 7fdd6d200640 1 -- 172.21.15.59:0/363943456 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6815d010 con 0x7fdd680d0d10 2024-04-24T06:36:17.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.132+0000 7fdd6d200640 1 --2- 172.21.15.59:0/363943456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd680d0d10 0x7fdd68156bc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdd680b4cb0 tx=0x7fdd5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14afd2a8739f7514 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd67e00640 1 -- 172.21.15.59:0/363943456 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54037070 con 0x7fdd680d0d10 2024-04-24T06:36:17.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd67e00640 1 -- 172.21.15.59:0/363943456 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd54032440 con 0x7fdd680d0d10 2024-04-24T06:36:17.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd67e00640 1 -- 172.21.15.59:0/363943456 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54032810 con 0x7fdd680d0d10 2024-04-24T06:36:17.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 -- 172.21.15.59:0/363943456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd680d0d10 msgr2=0x7fdd68156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 --2- 172.21.15.59:0/363943456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd680d0d10 0x7fdd68156bc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdd680b4cb0 tx=0x7fdd5402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 -- 172.21.15.59:0/363943456 shutdown_connections 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 --2- 172.21.15.59:0/363943456 >> v2:172.21.15.66:3300/0 conn(0x7fdd6815d810 0x7fdd68160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 --2- 172.21.15.59:0/363943456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd680d0d10 0x7fdd68156bc0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.133+0000 7fdd6f000640 1 -- 172.21.15.59:0/363943456 >> 172.21.15.59:0/363943456 conn(0x7fdd6806bdf0 msgr2=0x7fdd6806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.134+0000 7fdd6f000640 1 -- 172.21.15.59:0/363943456 shutdown_connections 2024-04-24T06:36:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.134+0000 7fdd6f000640 1 -- 172.21.15.59:0/363943456 wait complete. 2024-04-24T06:36:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.135+0000 7fdd6f000640 1 Processor -- start 2024-04-24T06:36:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.136+0000 7fdd6f000640 1 -- start start 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.136+0000 7fdd6f000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 0x7fdd6814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd681567e0 0x7fdd68154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd681607d0 con 0x7fdd6815d810 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdd68160650 con 0x7fdd681567e0 2024-04-24T06:36:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6f000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdd68160950 con 0x7fdd680d0d10 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32964/0 (socket says 172.21.15.59:32964) 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6e600640 1 -- 172.21.15.59:0/2509583839 learned_addr learned my addr 172.21.15.59:0/2509583839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6d200640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd681567e0 0x7fdd68154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6dc00640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 0x7fdd6814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6e600640 1 -- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 msgr2=0x7fdd6814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.137+0000 7fdd6e600640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 0x7fdd6814c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6e600640 1 -- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd681567e0 msgr2=0x7fdd68154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6e600640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd681567e0 0x7fdd68154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6e600640 1 -- 172.21.15.59:0/2509583839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6813bd50 con 0x7fdd6815d810 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6dc00640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 0x7fdd6814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6e600640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdd60007e60 tx=0x7fdd6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60014620 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd60019070 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60014a00 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6814f070 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.138+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6813c2f0 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.140+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6804fd90 con 0x7fdd6815d810 2024-04-24T06:36:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.141+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd60014ba0 con 0x7fdd6815d810 2024-04-24T06:36:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.142+0000 7fdd65600640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 0x7fdd4806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.142+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fdd6008c510 con 0x7fdd6815d810 2024-04-24T06:36:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.142+0000 7fdd6dc00640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 0x7fdd4806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.143+0000 7fdd6dc00640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 0x7fdd4806ee00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c000f60 tx=0x7fdd5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.147+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd6005a8c0 con 0x7fdd6815d810 2024-04-24T06:36:17.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.393+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fdd681602e0 con 0x7fdd6815d810 2024-04-24T06:36:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.646+0000 7fdd65600640 1 -- 172.21.15.59:0/2509583839 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7fdd6005a1c0 con 0x7fdd6815d810 2024-04-24T06:36:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:36:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 msgr2=0x7fdd4806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 0x7fdd4806ee00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c000f60 tx=0x7fdd5c009040 comp rx=0 tx=0).stop 2024-04-24T06:36:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 msgr2=0x7fdd681550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdd60007e60 tx=0x7fdd6000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 shutdown_connections 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd4806c940 0x7fdd4806ee00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd6815d810 0x7fdd681550e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd681567e0 0x7fdd68154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 --2- 172.21.15.59:0/2509583839 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdd680d0d10 0x7fdd6814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.650+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 >> 172.21.15.59:0/2509583839 conn(0x7fdd6806bdf0 msgr2=0x7fdd68154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.651+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 shutdown_connections 2024-04-24T06:36:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.651+0000 7fdd6f000640 1 -- 172.21.15.59:0/2509583839 wait complete. 2024-04-24T06:36:17.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.865+0000 7fab9e000640 1 Processor -- start 2024-04-24T06:36:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.866+0000 7fab9e000640 1 -- start start 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab9e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab980d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab9e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab980d2630 0x7fab980b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab9e000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab980cf030 con 0x7fab980b6530 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab9e000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fab980b7000 con 0x7fab980d2630 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab9e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab980cf620 con 0x7fab9815e7c0 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab97e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab980d2630 0x7fab980b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab97e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab980d2630 0x7fab980b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32966/0 (socket says 172.21.15.59:32966) 2024-04-24T06:36:17.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.867+0000 7fab97e00640 1 -- 172.21.15.59:0/1640611804 learned_addr learned my addr 172.21.15.59:0/1640611804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:17.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.868+0000 7fab97e00640 1 -- 172.21.15.59:0/1640611804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 msgr2=0x7fab980d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:17.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.868+0000 7fab97e00640 1 --2- 172.21.15.59:0/1640611804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab980d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.868+0000 7fab97e00640 1 -- 172.21.15.59:0/1640611804 >> v1:172.21.15.66:6789/0 conn(0x7fab980b6530 legacy=0x7fab980d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:17.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.868+0000 7fab97e00640 1 -- 172.21.15.59:0/1640611804 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab980b6a40 con 0x7fab980d2630 2024-04-24T06:36:17.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.869+0000 7fab97e00640 1 --2- 172.21.15.59:0/1640611804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980d2630 0x7fab980b5ff0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fab88009870 tx=0x7fab8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90a9391e6962fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.869+0000 7fab97400640 1 -- 172.21.15.59:0/1640611804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88037070 con 0x7fab980d2630 2024-04-24T06:36:17.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.869+0000 7fab97400640 1 -- 172.21.15.59:0/1640611804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab88032440 con 0x7fab980d2630 2024-04-24T06:36:17.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.869+0000 7fab97400640 1 -- 172.21.15.59:0/1640611804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88032810 con 0x7fab980d2630 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 -- 172.21.15.59:0/1640611804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980d2630 msgr2=0x7fab980b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 --2- 172.21.15.59:0/1640611804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980d2630 0x7fab980b5ff0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fab88009870 tx=0x7fab8802a210 comp rx=0 tx=0).stop 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 -- 172.21.15.59:0/1640611804 shutdown_connections 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 --2- 172.21.15.59:0/1640611804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980d2630 0x7fab980b5ff0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 --2- 172.21.15.59:0/1640611804 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab980d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 -- 172.21.15.59:0/1640611804 >> 172.21.15.59:0/1640611804 conn(0x7fab9809d920 msgr2=0x7fab9808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 -- 172.21.15.59:0/1640611804 shutdown_connections 2024-04-24T06:36:17.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.870+0000 7fab9e000640 1 -- 172.21.15.59:0/1640611804 wait complete. 2024-04-24T06:36:17.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.872+0000 7fab9e000640 1 Processor -- start 2024-04-24T06:36:17.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.872+0000 7fab9e000640 1 -- start start 2024-04-24T06:36:17.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 0x7fab9814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab98160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab980d1a80 con 0x7fab980b6530 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab980cf2b0 con 0x7fab980d2630 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab980d1c00 con 0x7fab9815e7c0 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32974/0 (socket says 172.21.15.59:32974) 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab9cc00640 1 -- 172.21.15.59:0/911209245 learned_addr learned my addr 172.21.15.59:0/911209245 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.873+0000 7fab97e00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 0x7fab9814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9d600640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab98160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 -- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 msgr2=0x7fab98160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab98160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 -- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 msgr2=0x7fab9814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:17.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 0x7fab9814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:17.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 -- 172.21.15.59:0/911209245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9815b2f0 con 0x7fab980b6530 2024-04-24T06:36:17.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab97e00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 0x7fab9814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:17.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9cc00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b440 tx=0x7fab8c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c015020 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c011890 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c011c80 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9815b5d0 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.874+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9815bb90 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.876+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9804fd90 con 0x7fab980b6530 2024-04-24T06:36:17.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.876+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab8c018020 con 0x7fab980b6530 2024-04-24T06:36:17.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.877+0000 7fab94c00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 0x7fab7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:17.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.877+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fab8c08bdd0 con 0x7fab980b6530 2024-04-24T06:36:17.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.878+0000 7fab97e00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 0x7fab7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:17.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.878+0000 7fab97e00640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 0x7fab7c06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab88034000 tx=0x7fab8802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:17.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:17.882+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab8c01f020 con 0x7fab980b6530 2024-04-24T06:36:18.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.128+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab980d2d70 con 0x7fab980b6530 2024-04-24T06:36:18.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.129+0000 7fab94c00640 1 -- 172.21.15.59:0/911209245 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5174 (secure 0 0 0) 0x7fab8c00d040 con 0x7fab980b6530 2024-04-24T06:36:18.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:18.132 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:17.651740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-24T06:36:18.133 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 64 2024-04-24T06:36:18.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.131+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 msgr2=0x7fab7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:18.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.131+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 0x7fab7c06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab88034000 tx=0x7fab8802e000 comp rx=0 tx=0).stop 2024-04-24T06:36:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.131+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 msgr2=0x7fab9815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.131+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b440 tx=0x7fab8c00b900 comp rx=0 tx=0).stop 2024-04-24T06:36:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 shutdown_connections 2024-04-24T06:36:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab7c06c8f0 0x7fab7c06edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:18.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fab9815e7c0 0x7fab98160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:18.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab980d2630 0x7fab9814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:18.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 --2- 172.21.15.59:0/911209245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab980b6530 0x7fab9815c390 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:18.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 >> 172.21.15.59:0/911209245 conn(0x7fab9809d920 msgr2=0x7fab9815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:18.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 shutdown_connections 2024-04-24T06:36:18.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:18.132+0000 7fab9e000640 1 -- 172.21.15.59:0/911209245 wait complete. 2024-04-24T06:36:18.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:36:17.651740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7548': {'gid': 7548, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/2792490076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2792490076}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2792490076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:36:18.556 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7548:up:replay 2024-04-24T06:36:19.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:19.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.777+0000 7f2643200640 1 Processor -- start 2024-04-24T06:36:19.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.778+0000 7f2643200640 1 -- start start 2024-04-24T06:36:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.779+0000 7f2643200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2644162df0 0x7f264406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.779+0000 7f2643200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.779+0000 7f2643200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264406ed40 con 0x7f26440cfb90 2024-04-24T06:36:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.779+0000 7f2643200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f264406eaf0 con 0x7f2644162df0 2024-04-24T06:36:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.779+0000 7f2643200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26440b3590 con 0x7f26440b4130 2024-04-24T06:36:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.780+0000 7f2642800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.780+0000 7f2642800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:57038/0 (socket says 172.21.15.59:57038) 2024-04-24T06:36:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.780+0000 7f2642800640 1 -- 172.21.15.59:0/59628450 learned_addr learned my addr 172.21.15.59:0/59628450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.780+0000 7f2641e00640 1 --2- 172.21.15.59:0/59628450 >> v2:172.21.15.66:3300/0 conn(0x7f2644162df0 0x7f264406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.781+0000 7f2642800640 1 -- 172.21.15.59:0/59628450 >> v2:172.21.15.66:3300/0 conn(0x7f2644162df0 msgr2=0x7f264406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.781+0000 7f2642800640 1 --2- 172.21.15.59:0/59628450 >> v2:172.21.15.66:3300/0 conn(0x7f2644162df0 0x7f264406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.781+0000 7f2642800640 1 -- 172.21.15.59:0/59628450 >> v1:172.21.15.66:6789/0 conn(0x7f26440cfb90 legacy=0x7f26440b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.781+0000 7f2642800640 1 -- 172.21.15.59:0/59628450 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440b3220 con 0x7f26440b4130 2024-04-24T06:36:19.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.782+0000 7f2642800640 1 --2- 172.21.15.59:0/59628450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9fff43d9e14f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.783+0000 7f2641400640 1 -- 172.21.15.59:0/59628450 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c036070 con 0x7f26440b4130 2024-04-24T06:36:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.783+0000 7f2641400640 1 -- 172.21.15.59:0/59628450 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c031600 con 0x7f26440b4130 2024-04-24T06:36:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.783+0000 7f2641400640 1 -- 172.21.15.59:0/59628450 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c0319d0 con 0x7f26440b4130 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 -- 172.21.15.59:0/59628450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 msgr2=0x7f26440c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 --2- 172.21.15.59:0/59628450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f263c009e50 tx=0x7f263c02a710 comp rx=0 tx=0).stop 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 -- 172.21.15.59:0/59628450 shutdown_connections 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 --2- 172.21.15.59:0/59628450 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26440b4130 0x7f26440c31a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 --2- 172.21.15.59:0/59628450 >> v2:172.21.15.66:3300/0 conn(0x7f2644162df0 0x7f264406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 -- 172.21.15.59:0/59628450 >> 172.21.15.59:0/59628450 conn(0x7f26440b21a0 msgr2=0x7f26440b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 -- 172.21.15.59:0/59628450 shutdown_connections 2024-04-24T06:36:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.784+0000 7f2643200640 1 -- 172.21.15.59:0/59628450 wait complete. 2024-04-24T06:36:19.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.786+0000 7f2643200640 1 Processor -- start 2024-04-24T06:36:19.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.786+0000 7f2643200640 1 -- start start 2024-04-24T06:36:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 0x7f26440724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2644162df0 0x7f2644160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26440c3860 con 0x7f26440b4130 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26440c36e0 con 0x7f26440cfb90 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2643200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26440c39e0 con 0x7f2644162df0 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.787+0000 7f2641e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35148/0 (socket says 172.21.15.59:35148) 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 -- 172.21.15.59:0/3571329275 learned_addr learned my addr 172.21.15.59:0/3571329275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2639400640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 0x7f26440724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2642800640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2644162df0 0x7f2644160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 -- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2644162df0 msgr2=0x7f2644160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2644162df0 0x7f2644160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 -- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 msgr2=0x7f26440724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 0x7f26440724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2641e00640 1 -- 172.21.15.59:0/3571329275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440c76a0 con 0x7f26440b4130 2024-04-24T06:36:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.788+0000 7f2639400640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 0x7f26440724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f2641e00640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f263400e8a0 tx=0x7f263400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634017070 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263400c940 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400cd30 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26440c7980 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.789+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26440c7f40 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.791+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264404fd90 con 0x7f26440b4130 2024-04-24T06:36:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.792+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f263401a070 con 0x7f26440b4130 2024-04-24T06:36:19.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.793+0000 7f263aa00640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 0x7f262006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.793+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f263408ba10 con 0x7f26440b4130 2024-04-24T06:36:19.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.793+0000 7f2639400640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 0x7f262006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.794+0000 7f2639400640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 0x7f262006ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:19.799+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2634059dd0 con 0x7f26440b4130 2024-04-24T06:36:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.052+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26440cc7a0 con 0x7f26440b4130 2024-04-24T06:36:20.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.052+0000 7f263aa00640 1 -- 172.21.15.59:0/3571329275 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5175 (secure 0 0 0) 0x7f26340596d0 con 0x7f26440b4130 2024-04-24T06:36:20.055 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:20.055 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:18.665539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-24T06:36:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 66 2024-04-24T06:36:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 msgr2=0x7f262006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 0x7f262006ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).stop 2024-04-24T06:36:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 msgr2=0x7f264415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f263400e8a0 tx=0x7f263400ed60 comp rx=0 tx=0).stop 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 shutdown_connections 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f262006c920 0x7f262006ede0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2644162df0 0x7f2644160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26440cfb90 0x7f26440724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 --2- 172.21.15.59:0/3571329275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26440b4130 0x7f264415f310 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 >> 172.21.15.59:0/3571329275 conn(0x7f26440b21a0 msgr2=0x7f26440b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.055+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 shutdown_connections 2024-04-24T06:36:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:20.056+0000 7f2643200640 1 -- 172.21.15.59:0/3571329275 wait complete. 2024-04-24T06:36:20.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:36:18.665539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7548': {'gid': 7548, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.59:6835/2792490076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2792490076}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2792490076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:36:20.481 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7548:up:rejoin 2024-04-24T06:36:21.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.703+0000 7f9f22600640 1 Processor -- start 2024-04-24T06:36:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.703+0000 7f9f22600640 1 -- start start 2024-04-24T06:36:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f22600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f22600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f1c0cd760 0x7f9f1c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f22600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1c0b3870 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f22600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9f1c0b41a0 con 0x7f9f1c0cd760 2024-04-24T06:36:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f22600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1c0b3b50 con 0x7f9f1c15b450 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f21c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f1c0cd760 0x7f9f1c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f21c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f1c0cd760 0x7f9f1c0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35152/0 (socket says 172.21.15.59:35152) 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.705+0000 7f9f21c00640 1 -- 172.21.15.59:0/3813131891 learned_addr learned my addr 172.21.15.59:0/3813131891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.706+0000 7f9f21200640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.706+0000 7f9f21c00640 1 -- 172.21.15.59:0/3813131891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 msgr2=0x7f9f1c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.706+0000 7f9f21c00640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.706+0000 7f9f21c00640 1 -- 172.21.15.59:0/3813131891 >> v1:172.21.15.66:6789/0 conn(0x7f9f1c06f3f0 legacy=0x7f9f1c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.706+0000 7f9f21c00640 1 -- 172.21.15.59:0/3813131891 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c0b3e30 con 0x7f9f1c0cd760 2024-04-24T06:36:21.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.707+0000 7f9f21c00640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c0ad060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1a592a11400aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:21.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.708+0000 7f9f1be00640 1 -- 172.21.15.59:0/3813131891 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14037070 con 0x7f9f1c0cd760 2024-04-24T06:36:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.708+0000 7f9f1be00640 1 -- 172.21.15.59:0/3813131891 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f14032440 con 0x7f9f1c0cd760 2024-04-24T06:36:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.708+0000 7f9f1be00640 1 -- 172.21.15.59:0/3813131891 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14032810 con 0x7f9f1c0cd760 2024-04-24T06:36:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.708+0000 7f9f22600640 1 -- 172.21.15.59:0/3813131891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c0cd760 msgr2=0x7f9f1c0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.708+0000 7f9f22600640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c0ad060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a250 comp rx=0 tx=0).stop 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 -- 172.21.15.59:0/3813131891 shutdown_connections 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c0ad060 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 --2- 172.21.15.59:0/3813131891 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 -- 172.21.15.59:0/3813131891 >> 172.21.15.59:0/3813131891 conn(0x7f9f1c0b21a0 msgr2=0x7f9f1c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 -- 172.21.15.59:0/3813131891 shutdown_connections 2024-04-24T06:36:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.709+0000 7f9f22600640 1 -- 172.21.15.59:0/3813131891 wait complete. 2024-04-24T06:36:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.711+0000 7f9f22600640 1 Processor -- start 2024-04-24T06:36:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.711+0000 7f9f22600640 1 -- start start 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c1e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c1e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c0b4ff0 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c15b830 con 0x7f9f1c0cd760 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f22600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1c0b3080 con 0x7f9f1c15b450 2024-04-24T06:36:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35158/0 (socket says 172.21.15.59:35158) 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 -- 172.21.15.59:0/3914384564 learned_addr learned my addr 172.21.15.59:0/3914384564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f13e00640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c1e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 -- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 msgr2=0x7f9f1c1e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c1e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 -- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f1c0cd760 msgr2=0x7f9f1c1e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c1e80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.712+0000 7f9f21200640 1 -- 172.21.15.59:0/3914384564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c15bf40 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f21200640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c007e70 tx=0x7f9f0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c014620 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f0c019070 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c014a00 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c0c65c0 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.713+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c0c6b80 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.714+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c04fd90 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.715+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f0c014ba0 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.716+0000 7f9f19600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 0x7f9ef806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.716+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f9f0c08c460 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.717+0000 7f9f13e00640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 0x7f9ef806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:21.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.717+0000 7f9f13e00640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 0x7f9ef806edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c1e8da0 tx=0x7f9f04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:21.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.721+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f0c05a820 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.966+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1c0d0c20 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.968+0000 7f9f19600640 1 -- 172.21.15.59:0/3914384564 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5182 (secure 0 0 0) 0x7f9f0c05a120 con 0x7f9f1c06f3f0 2024-04-24T06:36:21.970 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:21.970 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:20.665016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":11,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7548,"qdb_cluster":[7548]},"id":6}]} 2024-04-24T06:36:21.970 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 67 2024-04-24T06:36:21.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 msgr2=0x7f9ef806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 0x7f9ef806edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c1e8da0 tx=0x7f9f04009040 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 msgr2=0x7f9f1c0c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c007e70 tx=0x7f9f0c00e4b0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 shutdown_connections 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ef806c8f0 0x7f9ef806edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f1c15b450 0x7f9f1c1e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f1c0cd760 0x7f9f1c1e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 --2- 172.21.15.59:0/3914384564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f1c06f3f0 0x7f9f1c0c3ce0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.970+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 >> 172.21.15.59:0/3914384564 conn(0x7f9f1c0b21a0 msgr2=0x7f9f1c0cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.971+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 shutdown_connections 2024-04-24T06:36:21.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:21.971+0000 7f9f22600640 1 -- 172.21.15.59:0/3914384564 wait complete. 2024-04-24T06:36:22.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:36:20.665016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7548': {'gid': 7548, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.59:6835/2792490076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2792490076}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2792490076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7548, 'qdb_cluster': [7548]} 2024-04-24T06:36:22.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:36:22.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T06:36:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.616+0000 7f09e6e00640 1 Processor -- start 2024-04-24T06:36:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.616+0000 7f09e6e00640 1 -- start start 2024-04-24T06:36:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 0x7f09e00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f09e00b4230 0x7f09e00b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e6e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e00b4cf0 con 0x7f09e00d05c0 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f09e015ca50 con 0x7f09e00b4230 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f09e00ccb80 con 0x7f09e015c670 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f09e00b4230 0x7f09e00b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f09e00b4230 0x7f09e00b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35170/0 (socket says 172.21.15.59:35170) 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.618+0000 7f09e5000640 1 -- 172.21.15.59:0/1039743964 learned_addr learned my addr 172.21.15.59:0/1039743964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5a00640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 0x7f09e00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5000640 1 -- 172.21.15.59:0/1039743964 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 msgr2=0x7f09e00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5000640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 0x7f09e00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5000640 1 -- 172.21.15.59:0/1039743964 >> v1:172.21.15.66:6789/0 conn(0x7f09e00d05c0 legacy=0x7f09e0160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5000640 1 -- 172.21.15.59:0/1039743964 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e00ccdd0 con 0x7f09e00b4230 2024-04-24T06:36:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.619+0000 7f09e5a00640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 0x7f09e00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.620+0000 7f09e5000640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e00b4230 0x7f09e00b4690 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009870 tx=0x7f09cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab57092710fdbac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.620+0000 7f09dfe00640 1 -- 172.21.15.59:0/1039743964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc037070 con 0x7f09e00b4230 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.620+0000 7f09dfe00640 1 -- 172.21.15.59:0/1039743964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc032440 con 0x7f09e00b4230 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.620+0000 7f09dfe00640 1 -- 172.21.15.59:0/1039743964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc032810 con 0x7f09e00b4230 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 -- 172.21.15.59:0/1039743964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e00b4230 msgr2=0x7f09e00b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e00b4230 0x7f09e00b4690 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009870 tx=0x7f09cc02a210 comp rx=0 tx=0).stop 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 -- 172.21.15.59:0/1039743964 shutdown_connections 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e00b4230 0x7f09e00b4690 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 --2- 172.21.15.59:0/1039743964 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e015c670 0x7f09e00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 -- 172.21.15.59:0/1039743964 >> 172.21.15.59:0/1039743964 conn(0x7f09e006bdf0 msgr2=0x7f09e006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 -- 172.21.15.59:0/1039743964 shutdown_connections 2024-04-24T06:36:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.621+0000 7f09e6e00640 1 -- 172.21.15.59:0/1039743964 wait complete. 2024-04-24T06:36:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.623+0000 7f09e6e00640 1 Processor -- start 2024-04-24T06:36:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.623+0000 7f09e6e00640 1 -- start start 2024-04-24T06:36:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e00b4230 0x7f09e009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e00d05c0 0x7f09e0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00b3e60 con 0x7f09e015c670 2024-04-24T06:36:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09e00cd390 con 0x7f09e00d05c0 2024-04-24T06:36:22.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f09e00b3fe0 con 0x7f09e00b4230 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35172/0 (socket says 172.21.15.59:35172) 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e6400640 1 -- 172.21.15.59:0/826628940 learned_addr learned my addr 172.21.15.59:0/826628940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e5a00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e00b4230 0x7f09e009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.624+0000 7f09e5000640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e00d05c0 0x7f09e0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 -- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e00b4230 msgr2=0x7f09e009dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e00b4230 0x7f09e009dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 -- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e00d05c0 msgr2=0x7f09e0159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e00d05c0 0x7f09e0159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 -- 172.21.15.59:0/826628940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e015b370 con 0x7f09e015c670 2024-04-24T06:36:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09e6400640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f09d800c700 tx=0x7f09d800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.625+0000 7f09dd600640 1 -- 172.21.15.59:0/826628940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8016020 con 0x7f09e015c670 2024-04-24T06:36:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.626+0000 7f09dd600640 1 -- 172.21.15.59:0/826628940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d800aa50 con 0x7f09e015c670 2024-04-24T06:36:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.626+0000 7f09dd600640 1 -- 172.21.15.59:0/826628940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800ce10 con 0x7f09e015c670 2024-04-24T06:36:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.626+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e015a2d0 con 0x7f09e015c670 2024-04-24T06:36:22.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.626+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e015a890 con 0x7f09e015c670 2024-04-24T06:36:22.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.629+0000 7f09dd600640 1 -- 172.21.15.59:0/826628940 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09d8019020 con 0x7f09e015c670 2024-04-24T06:36:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.629+0000 7f09eb170740 1 Processor -- start 2024-04-24T06:36:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09eb170740 1 -- start start 2024-04-24T06:36:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09dd600640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 0x7f09c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09dd600640 1 -- 172.21.15.59:0/826628940 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f09d808be30 con 0x7f09e015c670 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09e5000640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 0x7f09c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e8aa980 0x56347e85d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09eb170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.630+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88d360 0x56347e862170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09eb170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56347e7c7b10 con 0x56347e88daa0 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56347e76e2b0 con 0x56347e8aa980 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56347e7c3ad0 con 0x56347e88d360 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35192/0 (socket says 172.21.15.59:35192) 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 -- 172.21.15.59:0/392480567 learned_addr learned my addr 172.21.15.59:0/392480567 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5a00640 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e8aa980 0x56347e85d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e6400640 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88d360 0x56347e862170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 0x7f09c006ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f09cc034000 tx=0x7f09cc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 -- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88d360 msgr2=0x56347e862170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88d360 0x56347e862170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 -- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e8aa980 msgr2=0x56347e85d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e8aa980 0x56347e85d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.631+0000 7f09e5000640 1 -- 172.21.15.59:0/392480567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347e4da2d0 con 0x56347e88daa0 2024-04-24T06:36:22.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.632+0000 7f09e5000640 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f09cc02a800 tx=0x7f09cc03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95315e1963a8990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.632+0000 7f09d2000640 1 -- 172.21.15.59:0/392480567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc043070 con 0x56347e88daa0 2024-04-24T06:36:22.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.632+0000 7f09d2000640 1 -- 172.21.15.59:0/392480567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc04c070 con 0x56347e88daa0 2024-04-24T06:36:22.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.632+0000 7f09d2000640 1 -- 172.21.15.59:0/392480567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc037040 con 0x56347e88daa0 2024-04-24T06:36:22.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 -- 172.21.15.59:0/392480567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 msgr2=0x56347e88ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f09cc02a800 tx=0x7f09cc03c000 comp rx=0 tx=0).stop 2024-04-24T06:36:22.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 -- 172.21.15.59:0/392480567 shutdown_connections 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88d360 0x56347e862170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e88ce20 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 --2- 172.21.15.59:0/392480567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e8aa980 0x56347e85d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 -- 172.21.15.59:0/392480567 >> 172.21.15.59:0/392480567 conn(0x56347e85f580 msgr2=0x56347e8a7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 -- 172.21.15.59:0/392480567 shutdown_connections 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.633+0000 7f09eb170740 1 -- 172.21.15.59:0/392480567 wait complete. 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.634+0000 7f09eb170740 1 -- start start 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.634+0000 7f09eb170740 1 Processor -- start 2024-04-24T06:36:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 0x56347e89fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e862100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56347e861730 con 0x56347e8aa980 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56347e781590 con 0x56347e87d8f0 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56347e860b20 con 0x56347e88daa0 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09eb170740 10 client.0 _ll_register_callbacks cb 0x56347e7643a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35200/0 (socket says 172.21.15.59:35200) 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.635+0000 7f09e5000640 1 -- 172.21.15.59:0/195162057 learned_addr learned my addr 172.21.15.59:0/195162057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5a00640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 0x56347e89fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e6400640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e862100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 -- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 msgr2=0x56347e89fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 0x56347e89fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 -- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 msgr2=0x56347e862100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e862100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 -- 172.21.15.59:0/195162057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347e5a46b0 con 0x56347e8aa980 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5a00640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 0x56347e89fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09e5000640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f09cc035040 tx=0x7f09cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4791594c90b7459f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc056070 con 0x56347e8aa980 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f09cc003a20 con 0x56347e8aa980 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.636+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc04c070 con 0x56347e8aa980 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f09cc043050 con 0x56347e8aa980 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 10 client.7866 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x56347e88f830 con 0x56347e8aa980 2024-04-24T06:36:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc04c070 con 0x56347e8aa980 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f09cc003c20 con 0x56347e8aa980 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 10 client.7866 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 10 client.7866 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 20 client.7866 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e88e800 0x56347e8c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 4 client.7866 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.637+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347e7d5530 con 0x56347e88e800 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.638+0000 7f09e5a00640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e88e800 0x56347e8c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.638+0000 7f09e5a00640 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e88e800 0x56347e8c6370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe720986e7daad28 server_cookie=d9c14e255ce13731 in_seq=0 out_seq=0 2024-04-24T06:36:22.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.638+0000 7f09c6a00640 10 client.7866 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.639+0000 7f09c6a00640 1 -- 172.21.15.59:0/195162057 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56347e7d5530 con 0x56347e88e800 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.639+0000 7f09c6a00640 10 client.7866 handle_command_reply: tid=0 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.639+0000 7f09eb170740 1 client.7866 shutdown 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.639+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 msgr2=0x56347e861bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.639+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f09cc035040 tx=0x7f09cc004730 comp rx=0 tx=0).stop 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 shutdown_connections 2024-04-24T06:36:22.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e88e800 0x56347e8c6370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e862100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e8aa980 0x56347e861bc0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 --2- 172.21.15.59:0/195162057 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e88daa0 0x56347e89fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09c6a00640 0 client.7866 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 >> 172.21.15.59:0/195162057 conn(0x56347e88d360 msgr2=0x56347e8ab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 shutdown_connections 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 1 -- 172.21.15.59:0/195162057 wait complete. 2024-04-24T06:36:22.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.640+0000 7f09eb170740 20 client.7866 trim_cache size 0 max 16384 2024-04-24T06:36:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.661+0000 7f09eb170740 1 Processor -- start 2024-04-24T06:36:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.661+0000 7f09eb170740 1 -- start start 2024-04-24T06:36:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e88cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e85c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0200e10 con 0x56347e88daa0 2024-04-24T06:36:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ffe80 con 0x56347e87d8f0 2024-04-24T06:36:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f09e015e400 con 0x56347e8aa980 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35210/0 (socket says 172.21.15.59:35210) 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09e5000640 1 -- 172.21.15.59:0/3055437735 learned_addr learned my addr 172.21.15.59:0/3055437735 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09e5a00640 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e88cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.662+0000 7f09e6400640 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e85c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 -- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 msgr2=0x56347e85c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e85c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 -- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 msgr2=0x56347e88cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e88cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 -- 172.21.15.59:0/3055437735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347e8abaf0 con 0x56347e88daa0 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09e5000640 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004580 tx=0x7f09cc050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1d3efd81b1a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09d1600640 1 -- 172.21.15.59:0/3055437735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc056070 con 0x56347e88daa0 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09d1600640 1 -- 172.21.15.59:0/3055437735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc04c040 con 0x56347e88daa0 2024-04-24T06:36:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.663+0000 7f09d1600640 1 -- 172.21.15.59:0/3055437735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc037040 con 0x56347e88daa0 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 -- 172.21.15.59:0/3055437735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 msgr2=0x56347e8cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004580 tx=0x7f09cc050000 comp rx=0 tx=0).stop 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 -- 172.21.15.59:0/3055437735 shutdown_connections 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e85c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e88daa0 0x56347e8cb5c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 --2- 172.21.15.59:0/3055437735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e87d8f0 0x56347e88cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 -- 172.21.15.59:0/3055437735 >> 172.21.15.59:0/3055437735 conn(0x56347e88d360 msgr2=0x56347e85d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 -- 172.21.15.59:0/3055437735 shutdown_connections 2024-04-24T06:36:22.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.664+0000 7f09eb170740 1 -- 172.21.15.59:0/3055437735 wait complete. 2024-04-24T06:36:22.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.665+0000 7f09eb170740 1 -- start start 2024-04-24T06:36:22.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.665+0000 7f09eb170740 1 Processor -- start 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e88daa0 0x56347e8cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e8ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56347e88fdf0 con 0x56347e87d8f0 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ff830 con 0x56347e88daa0 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56347e8b88a0 con 0x56347e8aa980 2024-04-24T06:36:22.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.666+0000 7f09eb170740 10 client.0 _ll_register_callbacks cb 0x7f09cc004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35212/0 (socket says 172.21.15.59:35212) 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 -- 172.21.15.59:0/2792600572 learned_addr learned my addr 172.21.15.59:0/2792600572 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e6400640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e8ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5000640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e88daa0 0x56347e8cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 -- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 msgr2=0x56347e8ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e8ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 -- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e88daa0 msgr2=0x56347e8cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e88daa0 0x56347e8cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 -- 172.21.15.59:0/2792600572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347e916f60 con 0x56347e87d8f0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09e5a00640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x56347e88ffc0 tx=0x7f09d40157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43aad141f8b106f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.667+0000 7f09c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4004240 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f09d40043e0 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc04c040 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f09d402a050 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09eb170740 10 client.7872 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x56347e922870 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc04c040 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f09d402d020 con 0x56347e87d8f0 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.668+0000 7f09eb170740 10 client.7872 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:22.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09eb170740 10 client.7872 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09eb170740 20 client.7872 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e8b7400 0x56347e917480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09eb170740 4 client.7872 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "status"}) -- 0x56347e7cf3a0 con 0x56347e8b7400 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09e5000640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e8b7400 0x56347e917480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09e5000640 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e8b7400 0x56347e917480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f90e965075432f0 server_cookie=ea4ac30afb6ec4a4 in_seq=0 out_seq=0 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.669+0000 7f09c6a00640 10 client.7872 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09c6a00640 1 -- 172.21.15.59:0/2792600572 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56347e7cf3a0 con 0x56347e8b7400 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09c6a00640 10 client.7872 handle_command_reply: tid=0 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 client.7872 shutdown 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 msgr2=0x56347e9166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x56347e88ffc0 tx=0x7f09d40157f0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 shutdown_connections 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x56347e8b7400 0x56347e917480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56347e8aa980 0x56347e8ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56347e88daa0 0x56347e8cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 --2- 172.21.15.59:0/2792600572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56347e87d8f0 0x56347e9166d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09c6a00640 0 client.7872 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 >> 172.21.15.59:0/2792600572 conn(0x56347e88d360 msgr2=0x56347e85c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 shutdown_connections 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 1 -- 172.21.15.59:0/2792600572 wait complete. 2024-04-24T06:36:22.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.670+0000 7f09eb170740 20 client.7872 trim_cache size 0 max 16384 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7548, 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0128321329999999, 2024-04-24T06:36:22.677 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 67, 2024-04-24T06:36:22.678 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 84, 2024-04-24T06:36:22.678 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 84, 2024-04-24T06:36:22.678 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 33.196102019999998 2024-04-24T06:36:22.678 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 msgr2=0x7f09c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 0x7f09c006ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f09cc034000 tx=0x7f09cc02c000 comp rx=0 tx=0).stop 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 msgr2=0x7f09e00b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f09d800c700 tx=0x7f09d800cbc0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 shutdown_connections 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f09c006c920 0x7f09c006ede0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09e015c670 0x7f09e00b2450 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09e00d05c0 0x7f09e0159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 --2- 172.21.15.59:0/826628940 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f09e00b4230 0x7f09e009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 >> 172.21.15.59:0/826628940 conn(0x7f09e006bdf0 msgr2=0x7f09e0157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 shutdown_connections 2024-04-24T06:36:22.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.671+0000 7f09e6e00640 1 -- 172.21.15.59:0/826628940 wait complete. 2024-04-24T06:36:22.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:36:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.892+0000 7f5ed3e00640 1 Processor -- start 2024-04-24T06:36:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.892+0000 7f5ed3e00640 1 -- start start 2024-04-24T06:36:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.893+0000 7f5ed3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ed40cce70 0x7f5ed4162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.893+0000 7f5ed3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5ed40b1f60 0x7f5ed40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.893+0000 7f5ed3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed41634e0 con 0x7f5ed40d07c0 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed40ce190 con 0x7f5ed40cce70 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed40d0ca0 con 0x7f5ed40b1f60 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ed40cce70 0x7f5ed4162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ed40d07c0 0x7f5ed40ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58374/0 (socket says 172.21.15.59:58374) 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2000640 1 -- 172.21.15.59:0/3473585378 learned_addr learned my addr 172.21.15.59:0/3473585378 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed3400640 1 --2- 172.21.15.59:0/3473585378 >> v2:172.21.15.66:3300/0 conn(0x7f5ed40b1f60 0x7f5ed40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed1600640 1 -- 172.21.15.59:0/3473585378 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106459518 0 0) 0x7f5ed41634e0 con 0x7f5ed40d07c0 2024-04-24T06:36:22.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed1600640 1 -- 172.21.15.59:0/3473585378 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eb8003520 con 0x7f5ed40d07c0 2024-04-24T06:36:22.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2a00640 1 -- 172.21.15.59:0/3473585378 >> v2:172.21.15.66:3300/0 conn(0x7f5ed40b1f60 msgr2=0x7f5ed40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:22.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2a00640 1 --2- 172.21.15.59:0/3473585378 >> v2:172.21.15.66:3300/0 conn(0x7f5ed40b1f60 0x7f5ed40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2a00640 1 -- 172.21.15.59:0/3473585378 >> v1:172.21.15.59:6789/0 conn(0x7f5ed40d07c0 legacy=0x7f5ed40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.894+0000 7f5ed2a00640 1 -- 172.21.15.59:0/3473585378 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed40cdb40 con 0x7f5ed40cce70 2024-04-24T06:36:22.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.896+0000 7f5ed2a00640 1 --2- 172.21.15.59:0/3473585378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed4162fa0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5ed41636b0 tx=0x7f5ebc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ebd49fd80f1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.896+0000 7f5ed1600640 1 -- 172.21.15.59:0/3473585378 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc037070 con 0x7f5ed40cce70 2024-04-24T06:36:22.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.896+0000 7f5ed1600640 1 -- 172.21.15.59:0/3473585378 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc032440 con 0x7f5ed40cce70 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.896+0000 7f5ed1600640 1 -- 172.21.15.59:0/3473585378 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc032810 con 0x7f5ed40cce70 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3473585378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 msgr2=0x7f5ed4162fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3473585378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed4162fa0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5ed41636b0 tx=0x7f5ebc02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3473585378 shutdown_connections 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3473585378 >> v2:172.21.15.66:3300/0 conn(0x7f5ed40b1f60 0x7f5ed40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3473585378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed4162fa0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3473585378 >> 172.21.15.59:0/3473585378 conn(0x7f5ed409aa00 msgr2=0x7f5ed40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3473585378 shutdown_connections 2024-04-24T06:36:22.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.897+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3473585378 wait complete. 2024-04-24T06:36:22.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.899+0000 7f5ed3e00640 1 Processor -- start 2024-04-24T06:36:22.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.899+0000 7f5ed3e00640 1 -- start start 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ed40b1f60 0x7f5ed415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed40d07c0 0x7f5ed40ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40cd250 con 0x7f5ed40cce70 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40d0f80 con 0x7f5ed40d07c0 2024-04-24T06:36:22.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed40d1db0 con 0x7f5ed40b1f60 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35240/0 (socket says 172.21.15.59:35240) 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.900+0000 7f5ed2000640 1 -- 172.21.15.59:0/3015040718 learned_addr learned my addr 172.21.15.59:0/3015040718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed3400640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed40d07c0 0x7f5ed40ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2a00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ed40b1f60 0x7f5ed415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 -- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ed40b1f60 msgr2=0x7f5ed415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ed40b1f60 0x7f5ed415fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 -- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed40d07c0 msgr2=0x7f5ed40ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed40d07c0 0x7f5ed40ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 -- 172.21.15.59:0/3015040718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41605d0 con 0x7f5ed40cce70 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed2000640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cb40 tx=0x7f5ec400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4017070 con 0x7f5ed40cce70 2024-04-24T06:36:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec400aca0 con 0x7f5ed40cce70 2024-04-24T06:36:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4012620 con 0x7f5ed40cce70 2024-04-24T06:36:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.901+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41608b0 con 0x7f5ed40cce70 2024-04-24T06:36:22.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.902+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed4160e70 con 0x7f5ed40cce70 2024-04-24T06:36:22.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.903+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed41483c0 con 0x7f5ed40cce70 2024-04-24T06:36:22.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.904+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ec401a070 con 0x7f5ed40cce70 2024-04-24T06:36:22.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.905+0000 7f5ecaa00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 0x7f5eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:22.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.905+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f5ec408ba20 con 0x7f5ed40cce70 2024-04-24T06:36:22.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.905+0000 7f5ed2a00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 0x7f5eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:22.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.906+0000 7f5ed2a00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 0x7f5eb006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc034000 tx=0x7f5ebc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:22.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:22.910+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ed41483c0 con 0x7f5ed40cce70 2024-04-24T06:36:23.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.160+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ed404fd90 con 0x7f5ed40cce70 2024-04-24T06:36:23.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.161+0000 7f5ecaa00640 1 -- 172.21.15.59:0/3015040718 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f5ec4059de0 con 0x7f5ed40cce70 2024-04-24T06:36:23.163 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.163+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 msgr2=0x7f5eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.163+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 0x7f5eb006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc034000 tx=0x7f5ebc02e000 comp rx=0 tx=0).stop 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.163+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 msgr2=0x7f5ed40ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.163+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cb40 tx=0x7f5ec400a060 comp rx=0 tx=0).stop 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 shutdown_connections 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5eb006c8f0 0x7f5eb006edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed40d07c0 0x7f5ed40ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:23.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ed40cce70 0x7f5ed40ae070 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 --2- 172.21.15.59:0/3015040718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5ed40b1f60 0x7f5ed415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 >> 172.21.15.59:0/3015040718 conn(0x7f5ed409aa00 msgr2=0x7f5ed40b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 shutdown_connections 2024-04-24T06:36:23.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23.164+0000 7f5ed3e00640 1 -- 172.21.15.59:0/3015040718 wait complete. 2024-04-24T06:36:23.185 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:36:23.185 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:36:23.186 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:23.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:23.186 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:36:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:36:23.244 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:36:23.244 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:36:23.244 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.244 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42161sec preferred_lft 42161sec 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:36:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:36:23.247 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:23.247 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:36:23.247 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:36:23.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:23.247 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:36:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:36:23.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:23.343 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:36:23.408 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:36:23.408 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:36:23.408 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:36:23.408 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:36:23.408 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:36:23.409 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:36:23.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:36:23.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:36:23.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:23.640 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:36:23.692 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:36:23.692 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:36:23.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:23.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:23.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:36:23.827 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:36:23.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:36:23.827 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:23.881 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:36:23.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:36:23.881 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:23.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:36:24.029 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:36:24.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:36:24.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:24.094 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:24.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:24.095 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:36:24.150 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:36:24.150 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:36:24.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:36:24.224 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:36:24.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:36:24.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:36:24.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:24.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:36:24.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:24.273 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:36:24.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:36:24.329+0000 7f70fff1f480 -1 init, newargv = 0x5654ab5125d0 newargc=15 2024-04-24T06:36:24.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64651]: starting ceph client 2024-04-24T06:36:24.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64651]: starting fuse 2024-04-24T06:36:24.366 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:36:24.366 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:36:24.367 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:36:24.368 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:36:24.583 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.64651.asok 2024-04-24T06:36:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:36:24.601 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64651.asok 2024-04-24T06:36:24.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:24.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64651.asok status 2024-04-24T06:36:24.802 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:36:24.799+0000 7f14d0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:36:24.841 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:24.841 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:36:24.841 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:36:24.841 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:36:24.841 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "64651", 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7887, 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7887 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:24.842 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4061313240 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4061313240 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7887 192.168.0.1:0/4061313240", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/4061313240", 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:36:24.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 67, 2024-04-24T06:36:24.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 84, 2024-04-24T06:36:24.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:36:24.844 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:36:24.844 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:36:24.844 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:24.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:24.856 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:36:24.857 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:24.911 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:36:24.912 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:24.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:24.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:36:25.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:25.002 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-24T06:36:25.066 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:36:25.064+0000 7f14cb200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-24T06:36:25.068 INFO:teuthology.orchestra.run.smithi059.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-24T06:36:25.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:36:25.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:25.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-24T06:36:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.283+0000 7f4048200640 1 Processor -- start 2024-04-24T06:36:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.283+0000 7f4048200640 1 -- start start 2024-04-24T06:36:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4048200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f40400b3930 0x7f404006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4048200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b110 0x7f404015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4048200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40400d0ac0 con 0x7f404009b820 2024-04-24T06:36:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4048200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f40400d16d0 con 0x7f40400b3930 2024-04-24T06:36:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4048200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40400b2f00 con 0x7f404009b110 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f40400b3930 0x7f404006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f40400b3930 0x7f404006ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35248/0 (socket says 172.21.15.59:35248) 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 -- 172.21.15.59:0/82544673 learned_addr learned my addr 172.21.15.59:0/82544673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4046200640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b110 0x7f404015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 -- 172.21.15.59:0/82544673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b110 msgr2=0x7f404015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b110 0x7f404015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 -- 172.21.15.59:0/82544673 >> v1:172.21.15.66:6789/0 conn(0x7f404009b820 legacy=0x7f404009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.286+0000 7f4045800640 1 -- 172.21.15.59:0/82544673 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400d0da0 con 0x7f40400b3930 2024-04-24T06:36:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.287+0000 7f4045800640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40400b3930 0x7f404006ccf0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4034009870 tx=0x7f403402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2765477236d6a8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f403fe00640 1 -- 172.21.15.59:0/82544673 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034037070 con 0x7f40400b3930 2024-04-24T06:36:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f403fe00640 1 -- 172.21.15.59:0/82544673 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403402ae00 con 0x7f40400b3930 2024-04-24T06:36:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f403fe00640 1 -- 172.21.15.59:0/82544673 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034032660 con 0x7f40400b3930 2024-04-24T06:36:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 -- 172.21.15.59:0/82544673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40400b3930 msgr2=0x7f404006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40400b3930 0x7f404006ccf0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4034009870 tx=0x7f403402a0f0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 -- 172.21.15.59:0/82544673 shutdown_connections 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b110 0x7f404015e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 --2- 172.21.15.59:0/82544673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40400b3930 0x7f404006ccf0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.288+0000 7f4048200640 1 -- 172.21.15.59:0/82544673 >> 172.21.15.59:0/82544673 conn(0x7f40400b4080 msgr2=0x7f40400b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.289+0000 7f4048200640 1 -- 172.21.15.59:0/82544673 shutdown_connections 2024-04-24T06:36:25.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.289+0000 7f4048200640 1 -- 172.21.15.59:0/82544673 wait complete. 2024-04-24T06:36:25.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.290+0000 7f4048200640 1 Processor -- start 2024-04-24T06:36:25.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.290+0000 7f4048200640 1 -- start start 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b820 0x7f40400b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 0x7f40400c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f404006d230 con 0x7f404009b110 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40400d1360 con 0x7f40400b3930 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.291+0000 7f4048200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40400b3d10 con 0x7f404009b820 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35252/0 (socket says 172.21.15.59:35252) 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 -- 172.21.15.59:0/3182663838 learned_addr learned my addr 172.21.15.59:0/3182663838 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4044e00640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b820 0x7f40400b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4046200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 0x7f40400c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 -- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b820 msgr2=0x7f40400b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b820 0x7f40400b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 -- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 msgr2=0x7f40400c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 0x7f40400c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4045800640 1 -- 172.21.15.59:0/3182663838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400b65f0 con 0x7f404009b110 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.292+0000 7f4046200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 0x7f40400c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f4045800640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4034009870 tx=0x7f40340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f403d600640 1 -- 172.21.15.59:0/3182663838 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034041070 con 0x7f404009b110 2024-04-24T06:36:25.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f403d600640 1 -- 172.21.15.59:0/3182663838 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403403c670 con 0x7f404009b110 2024-04-24T06:36:25.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f403d600640 1 -- 172.21.15.59:0/3182663838 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034037040 con 0x7f404009b110 2024-04-24T06:36:25.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40400b68d0 con 0x7f404009b110 2024-04-24T06:36:25.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.293+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40400b6e90 con 0x7f404009b110 2024-04-24T06:36:25.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f403d600640 1 -- 172.21.15.59:0/3182663838 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f403403a070 con 0x7f404009b110 2024-04-24T06:36:25.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f403d600640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 0x7f402006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f403d600640 1 -- 172.21.15.59:0/3182663838 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f40340abd90 con 0x7f404009b110 2024-04-24T06:36:25.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f4044e00640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 0x7f402006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f404afd8740 1 Processor -- start 2024-04-24T06:36:25.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.298+0000 7f404afd8740 1 -- start start 2024-04-24T06:36:25.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f4044e00640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 0x7f402006edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f402c006fd0 tx=0x7f402c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbe78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe7df0 0x55960fbbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55960fb20f50 con 0x55960fc05410 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55960fac72c0 con 0x55960fbe8530 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55960fb1cf10 con 0x55960fbe7df0 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.299+0000 7f4045800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35256/0 (socket says 172.21.15.59:35256) 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 -- 172.21.15.59:0/2463660397 learned_addr learned my addr 172.21.15.59:0/2463660397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4044e00640 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbe78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 -- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe7df0 msgr2=0x55960fbbcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe7df0 0x55960fbbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 -- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 msgr2=0x55960fbe78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbe78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 -- 172.21.15.59:0/2463660397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960f8ff7b0 con 0x55960fc05410 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4044e00640 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbe78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.300+0000 7f4045800640 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f40340aa100 tx=0x7f40340094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c6dfddce28d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f4032000640 1 -- 172.21.15.59:0/2463660397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403404a050 con 0x55960fc05410 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f4032000640 1 -- 172.21.15.59:0/2463660397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403407dd70 con 0x55960fc05410 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f4032000640 1 -- 172.21.15.59:0/2463660397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340825a0 con 0x55960fc05410 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 -- 172.21.15.59:0/2463660397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 msgr2=0x55960fbb7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f40340aa100 tx=0x7f40340094d0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 -- 172.21.15.59:0/2463660397 shutdown_connections 2024-04-24T06:36:25.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe7df0 0x55960fbbcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbe78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 --2- 172.21.15.59:0/2463660397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbb7da0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 -- 172.21.15.59:0/2463660397 >> 172.21.15.59:0/2463660397 conn(0x55960fbba010 msgr2=0x55960fc028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.301+0000 7f404afd8740 1 -- 172.21.15.59:0/2463660397 shutdown_connections 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.302+0000 7f404afd8740 1 -- 172.21.15.59:0/2463660397 wait complete. 2024-04-24T06:36:25.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.302+0000 7f404afd8740 1 -- start start 2024-04-24T06:36:25.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.302+0000 7f404afd8740 1 Processor -- start 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.303+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe8530 0x55960fbbd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.303+0000 7f404afd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.303+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.303+0000 7f404afd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55960fb20f50 con 0x55960fc05410 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.303+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55960fada680 con 0x55960fbd4c90 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55960fbbc1c0 con 0x55960fbe8530 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f404afd8740 10 client.0 _ll_register_callbacks cb 0x55960fb174f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35272/0 (socket says 172.21.15.59:35272) 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 -- 172.21.15.59:0/1546722177 learned_addr learned my addr 172.21.15.59:0/1546722177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4045800640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe8530 0x55960fbbd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4046200640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 -- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe8530 msgr2=0x55960fbbd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe8530 0x55960fbbd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 -- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 msgr2=0x55960fbd5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbd5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4044e00640 1 -- 172.21.15.59:0/1546722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960f95ff40 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.304+0000 7f4046200640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbd5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4044e00640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f402c00ded0 tx=0x7f402c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4e248b354dc8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4026a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c014e40 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f402c0125f0 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403407dd70 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f402c004020 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f404afd8740 10 client.7899 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55960f7f30a0 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.305+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403407dd70 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f402c013070 con 0x55960fc05410 2024-04-24T06:36:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 10 client.7899 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 10 client.7899 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 20 client.7899 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fbf9c40 0x55960fbe8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 4 client.7899 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55960fb2e970 con 0x55960fbf9c40 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f4045800640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fbf9c40 0x55960fbe8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f4045800640 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fbf9c40 0x55960fbe8a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d691fddcbb43d2e server_cookie=ed456b6b1c0c9372 in_seq=0 out_seq=0 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.306+0000 7f4026a00640 10 client.7899 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.307+0000 7f4026a00640 1 -- 172.21.15.59:0/1546722177 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55960fb2e970 con 0x55960fbf9c40 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.307+0000 7f4026a00640 10 client.7899 handle_command_reply: tid=0 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.307+0000 7f404afd8740 1 client.7899 shutdown 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.307+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 msgr2=0x55960fbfab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.307+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f402c00ded0 tx=0x7f402c004680 comp rx=0 tx=0).stop 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 shutdown_connections 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fbf9c40 0x55960fbe8a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbd5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fbfab70 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 --2- 172.21.15.59:0/1546722177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbe8530 0x55960fbbd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f4026a00640 0 client.7899 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 >> 172.21.15.59:0/1546722177 conn(0x55960fbe7df0 msgr2=0x55960fbb74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 shutdown_connections 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 1 -- 172.21.15.59:0/1546722177 wait complete. 2024-04-24T06:36:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.308+0000 7f404afd8740 20 client.7899 trim_cache size 0 max 16384 2024-04-24T06:36:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.333+0000 7f404afd8740 1 Processor -- start 2024-04-24T06:36:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.333+0000 7f404afd8740 1 -- start start 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbd4c90 0x55960fc266e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4040201290 con 0x55960fc05410 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4040200300 con 0x55960fbe8530 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40401e9c90 con 0x55960fbd4c90 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f4046200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f4046200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35280/0 (socket says 172.21.15.59:35280) 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f4046200640 1 -- 172.21.15.59:0/1356076333 learned_addr learned my addr 172.21.15.59:0/1356076333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.334+0000 7f4045800640 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbd4c90 0x55960fc266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4044e00640 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 -- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbd4c90 msgr2=0x55960fc266e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbd4c90 0x55960fc266e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 -- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 msgr2=0x55960fbbb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbbb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 -- 172.21.15.59:0/1356076333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960fbfb710 con 0x55960fc05410 2024-04-24T06:36:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.335+0000 7f4046200640 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4038007c90 tx=0x7f403800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d65b5cc8b979a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f4031600640 1 -- 172.21.15.59:0/1356076333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038019480 con 0x55960fc05410 2024-04-24T06:36:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f4031600640 1 -- 172.21.15.59:0/1356076333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038019620 con 0x55960fc05410 2024-04-24T06:36:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f4031600640 1 -- 172.21.15.59:0/1356076333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380199f0 con 0x55960fc05410 2024-04-24T06:36:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 -- 172.21.15.59:0/1356076333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 msgr2=0x55960fc026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4038007c90 tx=0x7f403800e720 comp rx=0 tx=0).stop 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 -- 172.21.15.59:0/1356076333 shutdown_connections 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fc05410 0x55960fc026b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbe8530 0x55960fbbb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 --2- 172.21.15.59:0/1356076333 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fbd4c90 0x55960fc266e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.336+0000 7f404afd8740 1 -- 172.21.15.59:0/1356076333 >> 172.21.15.59:0/1356076333 conn(0x55960fbe7df0 msgr2=0x55960fbfa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.337+0000 7f404afd8740 1 -- 172.21.15.59:0/1356076333 shutdown_connections 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.337+0000 7f404afd8740 1 -- 172.21.15.59:0/1356076333 wait complete. 2024-04-24T06:36:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.337+0000 7f404afd8740 1 -- start start 2024-04-24T06:36:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.338+0000 7f404afd8740 1 Processor -- start 2024-04-24T06:36:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.338+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbbb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fc05410 0x55960fbeb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40400b5a00 con 0x55960fbe8530 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55960fbfb450 con 0x55960fbd4c90 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55960fc7d9c0 con 0x55960fc05410 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f404afd8740 10 client.0 _ll_register_callbacks cb 0x7f402c00db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35282/0 (socket says 172.21.15.59:35282) 2024-04-24T06:36:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 -- 172.21.15.59:0/1491760215 learned_addr learned my addr 172.21.15.59:0/1491760215 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4045800640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbbb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4046200640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fc05410 0x55960fbeb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 -- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fc05410 msgr2=0x55960fbeb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fc05410 0x55960fbeb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 -- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 msgr2=0x55960fbbb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbbb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.339+0000 7f4044e00640 1 -- 172.21.15.59:0/1491760215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960fbd39a0 con 0x55960fbe8530 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f4044e00640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f402c008010 tx=0x7f402c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0cb3e5f9bb13fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f4026a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c006450 con 0x55960fbe8530 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4038019620 con 0x55960fbe8530 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f402c0065f0 con 0x55960fbe8530 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f402c0067f0 con 0x55960fbe8530 2024-04-24T06:36:25.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f404afd8740 10 client.7905 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.340+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55960fc7d7c0 con 0x55960fbe8530 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038019620 con 0x55960fbe8530 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f402c0125f0 con 0x55960fbe8530 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 10 client.7905 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 10 client.7905 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 20 client.7905 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fc14d30 0x55960fc15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 4 client.7905 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55960fbfc2b0 con 0x55960fc14d30 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.341+0000 7f4045800640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fc14d30 0x55960fc15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f4045800640 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fc14d30 0x55960fc15140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb075c1974450c9e server_cookie=74bbb295caa62158 in_seq=0 out_seq=0 2024-04-24T06:36:25.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f4026a00640 10 client.7905 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f4026a00640 1 -- 172.21.15.59:0/1491760215 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55960fbfc2b0 con 0x55960fc14d30 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f4026a00640 10 client.7905 handle_command_reply: tid=0 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f404afd8740 1 client.7905 shutdown 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 msgr2=0x55960fbeb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.342+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f402c008010 tx=0x7f402c012000 comp rx=0 tx=0).stop 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 shutdown_connections 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55960fc14d30 0x55960fc15140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55960fc05410 0x55960fbeb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960fbe8530 0x55960fbeb160 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 --2- 172.21.15.59:0/1491760215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55960fbd4c90 0x55960fbbb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.349 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f14cb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0) n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x56308c7d6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-24T06:36:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f4026a00640 0 client.7905 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 >> 172.21.15.59:0/1491760215 conn(0x55960fbe7df0 msgr2=0x55960fc02d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 shutdown_connections 2024-04-24T06:36:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 1 -- 172.21.15.59:0/1491760215 wait complete. 2024-04-24T06:36:25.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.343+0000 7f404afd8740 20 client.7905 trim_cache size 0 max 16384 2024-04-24T06:36:25.351 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:25.351 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:36:25.351 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "37b729cc-8b3d-4059-9b5f-2e37d33bb297", 2024-04-24T06:36:25.351 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:36:25.351 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 msgr2=0x7f402006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 0x7f402006edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f402c006fd0 tx=0x7f402c008040 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 msgr2=0x7f404015b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4034009870 tx=0x7f40340046c0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 shutdown_connections 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402006c8f0 0x7f402006edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40400b3930 0x7f40400c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f404009b820 0x7f40400b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 --2- 172.21.15.59:0/3182663838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f404009b110 0x7f404015b790 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 >> 172.21.15.59:0/3182663838 conn(0x7f40400b4080 msgr2=0x7f4040161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 shutdown_connections 2024-04-24T06:36:25.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.344+0000 7f4048200640 1 -- 172.21.15.59:0/3182663838 wait complete. 2024-04-24T06:36:25.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:25.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:36:25.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.577+0000 7f127b200640 1 Processor -- start 2024-04-24T06:36:25.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.577+0000 7f127b200640 1 -- start start 2024-04-24T06:36:25.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f127406fb50 0x7f12740b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12740728b0 con 0x7f12740b3ed0 2024-04-24T06:36:25.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1274162a70 con 0x7f12740724d0 2024-04-24T06:36:25.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1274162d50 con 0x7f127406fb50 2024-04-24T06:36:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f12740b3ed0 0x7f12740d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58378/0 (socket says 172.21.15.59:58378) 2024-04-24T06:36:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f127a800640 1 -- 172.21.15.59:0/2339910912 learned_addr learned my addr 172.21.15.59:0/2339910912 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f1279400640 1 --2- 172.21.15.59:0/2339910912 >> v2:172.21.15.66:3300/0 conn(0x7f127406fb50 0x7f12740b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174334553 0 0) 0x7f12740728b0 con 0x7f12740b3ed0 2024-04-24T06:36:25.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.579+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f125c003520 con 0x7f12740b3ed0 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801666930 0 0) 0x7f1274162a70 con 0x7f12740724d0 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12740728b0 con 0x7f12740724d0 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133444400 0 0) 0x7f125c003520 con 0x7f12740b3ed0 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 >> v2:172.21.15.66:3300/0 conn(0x7f127406fb50 msgr2=0x7f12740b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 --2- 172.21.15.59:0/2339910912 >> v2:172.21.15.66:3300/0 conn(0x7f127406fb50 0x7f12740b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 >> v1:172.21.15.66:6789/0 conn(0x7f12740724d0 legacy=0x7f1274162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274163030 con 0x7f12740b3ed0 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f12700019f0 con 0x7f12740b3ed0 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 >> v1:172.21.15.59:6789/0 conn(0x7f12740b3ed0 legacy=0x7f12740d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c003bc0 0x7f125c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 0x7f125c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f125c0051c0 0x7f125c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1274162d50 con 0x7f125c004800 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f125c00c0c0 con 0x7f125c003bc0 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f125c00c2c0 con 0x7f125c0051c0 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.580+0000 7f1279400640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 0x7f125c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f1279400640 1 -- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f125c0051c0 msgr2=0x7f125c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f1279400640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f125c0051c0 0x7f125c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f1279400640 1 -- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c003bc0 msgr2=0x7f125c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f1279400640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c003bc0 0x7f125c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f1279400640 1 -- 172.21.15.59:0/2339910912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c003520 con 0x7f125c004800 2024-04-24T06:36:25.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.581+0000 7f127a800640 1 -- 172.21.15.59:0/2339910912 reap_dead start 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.582+0000 7f1279400640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 0x7f125c004c80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ac0 tx=0x7f125c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fc17b91102dda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.582+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036070 con 0x7f125c004800 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.582+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264031400 con 0x7f125c004800 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.582+0000 7f1278a00640 1 -- 172.21.15.59:0/2339910912 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264009690 con 0x7f125c004800 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 -- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 msgr2=0x7f125c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 0x7f125c004c80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ac0 tx=0x7f125c005830 comp rx=0 tx=0).stop 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 -- 172.21.15.59:0/2339910912 shutdown_connections 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 --2- 172.21.15.59:0/2339910912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f125c004800 0x7f125c004c80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 -- 172.21.15.59:0/2339910912 >> 172.21.15.59:0/2339910912 conn(0x7f127406bdf0 msgr2=0x7f127406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 -- 172.21.15.59:0/2339910912 shutdown_connections 2024-04-24T06:36:25.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.583+0000 7f127b200640 1 -- 172.21.15.59:0/2339910912 wait complete. 2024-04-24T06:36:25.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.585+0000 7f127b200640 1 Processor -- start 2024-04-24T06:36:25.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.585+0000 7f127b200640 1 -- start start 2024-04-24T06:36:25.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c004800 0x7f1274154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f12741575c0 0x7f1274140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12740b3990 con 0x7f1274157a20 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f12740b3470 con 0x7f125c004800 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f12740b3b10 con 0x7f12741575c0 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35300/0 (socket says 172.21.15.59:35300) 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f127a800640 1 -- 172.21.15.59:0/2652902973 learned_addr learned my addr 172.21.15.59:0/2652902973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f1279e00640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c004800 0x7f1274154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.586+0000 7f1279400640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f12741575c0 0x7f1274140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 -- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f12741575c0 msgr2=0x7f1274140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f12741575c0 0x7f1274140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 -- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c004800 msgr2=0x7f1274154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c004800 0x7f1274154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 -- 172.21.15.59:0/2652902973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274154e70 con 0x7f1274157a20 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127a800640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f12740b5160 tx=0x7f127000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f126e000640 1 -- 172.21.15.59:0/2652902973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127000a640 con 0x7f1274157a20 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f126e000640 1 -- 172.21.15.59:0/2652902973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127000a7e0 con 0x7f1274157a20 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1274155150 con 0x7f1274157a20 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.587+0000 7f126e000640 1 -- 172.21.15.59:0/2652902973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127000abb0 con 0x7f1274157a20 2024-04-24T06:36:25.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.588+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1274155710 con 0x7f1274157a20 2024-04-24T06:36:25.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f126e000640 1 -- 172.21.15.59:0/2652902973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f127000ad50 con 0x7f1274157a20 2024-04-24T06:36:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f126e000640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 0x7f125806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f126e000640 1 -- 172.21.15.59:0/2652902973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f127008b5b0 con 0x7f1274157a20 2024-04-24T06:36:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f1279e00640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 0x7f125806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f127f77a740 1 Processor -- start 2024-04-24T06:36:25.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.591+0000 7f127f77a740 1 -- start start 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f1279e00640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 0x7f125806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1268000f60 tx=0x7f1268009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaee2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf127e0 0x55ffcaee75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ffcae4c1f0 con 0x55ffcaf12f20 2024-04-24T06:36:25.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcadf2350 con 0x55ffcaf2fe00 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.592+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ffcae481b0 con 0x55ffcaf127e0 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35304/0 (socket says 172.21.15.59:35304) 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 -- 172.21.15.59:0/1288802069 learned_addr learned my addr 172.21.15.59:0/1288802069 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279e00640 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaee2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f127a800640 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf127e0 0x55ffcaee75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 -- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf127e0 msgr2=0x55ffcaee75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf127e0 0x55ffcaee75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 -- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 msgr2=0x55ffcaee2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaee2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 -- 172.21.15.59:0/1288802069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcae46f50 con 0x55ffcaf12f20 2024-04-24T06:36:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.593+0000 7f1279400640 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1264009bb0 tx=0x7f126402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87237c1f38c8141f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.594+0000 7f1262600640 1 -- 172.21.15.59:0/1288802069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036040 con 0x55ffcaf12f20 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.594+0000 7f1262600640 1 -- 172.21.15.59:0/1288802069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264041070 con 0x55ffcaf12f20 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.594+0000 7f1262600640 1 -- 172.21.15.59:0/1288802069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264004460 con 0x55ffcaf12f20 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.594+0000 7f127f77a740 1 -- 172.21.15.59:0/1288802069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 msgr2=0x55ffcaf122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.594+0000 7f127f77a740 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1264009bb0 tx=0x7f126402e000 comp rx=0 tx=0).stop 2024-04-24T06:36:25.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 -- 172.21.15.59:0/1288802069 shutdown_connections 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf127e0 0x55ffcaee75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf12f20 0x55ffcaf122a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 --2- 172.21.15.59:0/1288802069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaee2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 -- 172.21.15.59:0/1288802069 >> 172.21.15.59:0/1288802069 conn(0x55ffcaee4a00 msgr2=0x55ffcaf2d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 -- 172.21.15.59:0/1288802069 shutdown_connections 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 -- 172.21.15.59:0/1288802069 wait complete. 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 -- start start 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.596+0000 7f127f77a740 1 Processor -- start 2024-04-24T06:36:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ffcaee6bb0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcae05690 con 0x55ffcaf3d960 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ffcaee5fa0 con 0x55ffcaf12f20 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127f77a740 10 client.0 _ll_register_callbacks cb 0x55ffcae41650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35314/0 (socket says 172.21.15.59:35314) 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 -- 172.21.15.59:0/2225474303 learned_addr learned my addr 172.21.15.59:0/2225474303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279e00640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f127a800640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 -- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 msgr2=0x55ffcaf23f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf23f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 -- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 msgr2=0x55ffcaf41460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf41460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279400640 1 -- 172.21.15.59:0/2225474303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcae04920 con 0x55ffcaf2fe00 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.597+0000 7f1279e00640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.598+0000 7f1279400640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ac0 tx=0x7f12640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d45bba8e566149 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.598+0000 7f1256a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.598+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036040 con 0x55ffcaf2fe00 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.598+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f1264003810 con 0x55ffcaf2fe00 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.598+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1264041070 con 0x55ffcaf2fe00 2024-04-24T06:36:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f1264003a10 con 0x55ffcaf2fe00 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 10 client.7917 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ffcac287d0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264041070 con 0x55ffcaf2fe00 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f1264003bb0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 10 client.7917 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 10 client.7917 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 20 client.7917 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:25.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf14cd0 0x55ffcaf2d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 4 client.7917 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:25.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcae59c10 con 0x55ffcaf14cd0 2024-04-24T06:36:25.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.599+0000 7f1279e00640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf14cd0 0x55ffcaf2d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.600+0000 7f1279e00640 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf14cd0 0x55ffcaf2d2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dce7acc66bfc4e server_cookie=4250a130edcbb423 in_seq=0 out_seq=0 2024-04-24T06:36:25.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.600+0000 7f1256a00640 10 client.7917 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f1256a00640 1 -- 172.21.15.59:0/2225474303 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffcae59c10 con 0x55ffcaf14cd0 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f1256a00640 10 client.7917 handle_command_reply: tid=0 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 client.7917 shutdown 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 msgr2=0x55ffcaf40f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ac0 tx=0x7f12640046c0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 shutdown_connections 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf14cd0 0x55ffcaf2d2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf40f20 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 --2- 172.21.15.59:0/2225474303 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f1256a00640 0 client.7917 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 >> 172.21.15.59:0/2225474303 conn(0x55ffcaf127e0 msgr2=0x55ffcaf306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 shutdown_connections 2024-04-24T06:36:25.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.601+0000 7f127f77a740 1 -- 172.21.15.59:0/2225474303 wait complete. 2024-04-24T06:36:25.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.602+0000 7f127f77a740 20 client.7917 trim_cache size 0 max 16384 2024-04-24T06:36:25.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.621+0000 7f127f77a740 1 Processor -- start 2024-04-24T06:36:25.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.621+0000 7f127f77a740 1 -- start start 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12742013a0 con 0x55ffcaf3d960 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1274142e80 con 0x55ffcaf2fe00 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f127415a540 con 0x55ffcaf12f20 2024-04-24T06:36:25.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35320/0 (socket says 172.21.15.59:35320) 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.622+0000 7f127a800640 1 -- 172.21.15.59:0/379261700 learned_addr learned my addr 172.21.15.59:0/379261700 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1279e00640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 -- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 msgr2=0x55ffcaf152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1279400640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 -- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 msgr2=0x55ffcaf402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 -- 172.21.15.59:0/379261700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcaee5b40 con 0x55ffcaf3d960 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1279e00640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1279400640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f127a800640 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1270089950 tx=0x7f1270059050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd256d5973f18ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1261c00640 1 -- 172.21.15.59:0/379261700 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270061dc0 con 0x55ffcaf3d960 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1261c00640 1 -- 172.21.15.59:0/379261700 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1270057450 con 0x55ffcaf3d960 2024-04-24T06:36:25.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.623+0000 7f1261c00640 1 -- 172.21.15.59:0/379261700 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127005d150 con 0x55ffcaf3d960 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 -- 172.21.15.59:0/379261700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 msgr2=0x55ffcaf40810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1270089950 tx=0x7f1270059050 comp rx=0 tx=0).stop 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 -- 172.21.15.59:0/379261700 shutdown_connections 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf3d960 0x55ffcaf40810 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 --2- 172.21.15.59:0/379261700 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 -- 172.21.15.59:0/379261700 >> 172.21.15.59:0/379261700 conn(0x55ffcaf127e0 msgr2=0x55ffcaeddf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 -- 172.21.15.59:0/379261700 shutdown_connections 2024-04-24T06:36:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.624+0000 7f127f77a740 1 -- 172.21.15.59:0/379261700 wait complete. 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.625+0000 7f127f77a740 1 -- start start 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.625+0000 7f127f77a740 1 Processor -- start 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf50910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1274142830 con 0x55ffcaf2fe00 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcaee9bf0 con 0x55ffcaf3d960 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ffcafb2120 con 0x55ffcaf12f20 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f127f77a740 10 client.0 _ll_register_callbacks cb 0x7f125c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f1279400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35330/0 (socket says 172.21.15.59:35330) 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.626+0000 7f1279400640 1 -- 172.21.15.59:0/3250271033 learned_addr learned my addr 172.21.15.59:0/3250271033 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:25.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279e00640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 -- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 msgr2=0x55ffcaf406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 -- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 msgr2=0x55ffcaf50910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf50910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 -- 172.21.15.59:0/3250271033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcab5e3f0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1279400640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f126403c660 tx=0x7f1264008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc7c38800cdddee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1256a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036040 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f1264005da0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.627+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1270057450 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f1264009710 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 10 client.7923 fetch_fsmap learned FSMap version 67 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ffcab1e0f0 con 0x55ffcaf2fe00 2024-04-24T06:36:25.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1270057450 con 0x55ffcaf2fe00 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f1264063020 con 0x55ffcaf2fe00 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 10 client.7923 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 10 client.7923 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 20 client.7923 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf101d0 0x55ffcaf50e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 4 client.7923 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffcae53a80 con 0x55ffcaf101d0 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.628+0000 7f1279e00640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf101d0 0x55ffcaf50e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f1279e00640 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf101d0 0x55ffcaf50e50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9af338e5423019 server_cookie=bfdbb38262607d36 in_seq=0 out_seq=0 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f1256a00640 10 client.7923 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f1256a00640 1 -- 172.21.15.59:0/3250271033 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55ffcae53a80 con 0x55ffcaf101d0 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f1256a00640 10 client.7923 handle_command_reply: tid=0 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 client.7923 shutdown 2024-04-24T06:36:25.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 msgr2=0x55ffcaf503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f126403c660 tx=0x7f1264008030 comp rx=0 tx=0).stop 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 shutdown_connections 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ffcaf101d0 0x55ffcaf50e50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcaf3d960 0x55ffcaf50910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ffcaf2fe00 0x55ffcaf503d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f1256a00640 0 client.7923 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 --2- 172.21.15.59:0/3250271033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ffcaf12f20 0x55ffcaf406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.629+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 >> 172.21.15.59:0/3250271033 conn(0x55ffcaf127e0 msgr2=0x55ffcaedde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 shutdown_connections 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127f77a740 1 -- 172.21.15.59:0/3250271033 wait complete. 2024-04-24T06:36:25.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127f77a740 20 client.7923 trim_cache size 0 max 16384 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "37b729cc-8b3d-4059-9b5f-2e37d33bb297": { 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "37b729cc-8b3d-4059-9b5f-2e37d33bb297", 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:25.636 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 msgr2=0x7f125806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 0x7f125806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1268000f60 tx=0x7f1268009040 comp rx=0 tx=0).stop 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 msgr2=0x7f1274140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.630+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f12740b5160 tx=0x7f127000ccd0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 shutdown_connections 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f125806c920 0x7f125806ede0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1274157a20 0x7f1274140b70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f12741575c0 0x7f1274140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 --2- 172.21.15.59:0/2652902973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f125c004800 0x7f1274154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:25.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 >> 172.21.15.59:0/2652902973 conn(0x7f127406bdf0 msgr2=0x7f127415cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:25.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 shutdown_connections 2024-04-24T06:36:25.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:25.631+0000 7f127b200640 1 -- 172.21.15.59:0/2652902973 wait complete. 2024-04-24T06:36:25.649 INFO:tasks.cephfs.filesystem:scrub status for tag:37b729cc-8b3d-4059-9b5f-2e37d33bb297 - {'path': '/dir', 'tag': '37b729cc-8b3d-4059-9b5f-2e37d33bb297', 'options': 'recursive'} 2024-04-24T06:36:31.847 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:36:31.846+0000 7f6490400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T06:36:37.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:36:37.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:36:55.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:55.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:36:55.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.872+0000 7f73be600640 1 Processor -- start 2024-04-24T06:36:55.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.873+0000 7f73be600640 1 -- start start 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.874+0000 7f73be600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f73b80724d0 0x7f73b8162240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.874+0000 7f73be600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 0x7f73b80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.874+0000 7f73be600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80728b0 con 0x7f73b80b3ed0 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.874+0000 7f73be600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f73b8162a70 con 0x7f73b80724d0 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.874+0000 7f73be600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8162d50 con 0x7f73b806fb50 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bd200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f73b80724d0 0x7f73b8162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bdc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f73b80b3ed0 0x7f73b80d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40344/0 (socket says 172.21.15.59:40344) 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bdc00640 1 -- 172.21.15.59:0/3589383197 learned_addr learned my addr 172.21.15.59:0/3589383197 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 0x7f73b80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bd200640 1 -- 172.21.15.59:0/3589383197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 msgr2=0x7f73b80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bd200640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 0x7f73b80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bd200640 1 -- 172.21.15.59:0/3589383197 >> v1:172.21.15.59:6789/0 conn(0x7f73b80b3ed0 legacy=0x7f73b80d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73bd200640 1 -- 172.21.15.59:0/3589383197 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8163030 con 0x7f73b80724d0 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73b7400640 1 -- 172.21.15.59:0/3589383197 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884937677 0 0) 0x7f73b80728b0 con 0x7f73b80b3ed0 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.875+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 0x7f73b80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73bd200640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b80724d0 0x7f73b8162240 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f73a80097a0 tx=0x7f73a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d044d5ab6d563ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73b7400640 1 -- 172.21.15.59:0/3589383197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8037070 con 0x7f73b80724d0 2024-04-24T06:36:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73b7400640 1 -- 172.21.15.59:0/3589383197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a8032440 con 0x7f73b80724d0 2024-04-24T06:36:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73b7400640 1 -- 172.21.15.59:0/3589383197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8032810 con 0x7f73b80724d0 2024-04-24T06:36:55.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73be600640 1 -- 172.21.15.59:0/3589383197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b80724d0 msgr2=0x7f73b8162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.877+0000 7f73be600640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b80724d0 0x7f73b8162240 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f73a80097a0 tx=0x7f73a802a180 comp rx=0 tx=0).stop 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 -- 172.21.15.59:0/3589383197 shutdown_connections 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b806fb50 0x7f73b80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 --2- 172.21.15.59:0/3589383197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b80724d0 0x7f73b8162240 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 -- 172.21.15.59:0/3589383197 >> 172.21.15.59:0/3589383197 conn(0x7f73b806bdf0 msgr2=0x7f73b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 -- 172.21.15.59:0/3589383197 shutdown_connections 2024-04-24T06:36:55.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.878+0000 7f73be600640 1 -- 172.21.15.59:0/3589383197 wait complete. 2024-04-24T06:36:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.879+0000 7f73be600640 1 Processor -- start 2024-04-24T06:36:55.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.880+0000 7f73be600640 1 -- start start 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 0x7f73b809cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 0x7f73b809d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8162780 con 0x7f73b8146290 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80b3470 con 0x7f73b806fb50 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73be600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80d0ce0 con 0x7f73b80724d0 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47316/0 (socket says 172.21.15.59:47316) 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 -- 172.21.15.59:0/2111748568 learned_addr learned my addr 172.21.15.59:0/2111748568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bd200640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 0x7f73b809cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73b7e00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 0x7f73b809d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 -- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 msgr2=0x7f73b809d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 0x7f73b809d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 -- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 msgr2=0x7f73b809cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 0x7f73b809cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bdc00640 1 -- 172.21.15.59:0/2111748568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8161860 con 0x7f73b8146290 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73bd200640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 0x7f73b809cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.881+0000 7f73b7e00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 0x7f73b809d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c440 tx=0x7f73b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73b4c00640 1 -- 172.21.15.59:0/2111748568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b00074e0 con 0x7f73b8146290 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73b4c00640 1 -- 172.21.15.59:0/2111748568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b00047e0 con 0x7f73b8146290 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b81443e0 con 0x7f73b8146290 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73b4c00640 1 -- 172.21.15.59:0/2111748568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0004c10 con 0x7f73b8146290 2024-04-24T06:36:55.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.882+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81449a0 con 0x7f73b8146290 2024-04-24T06:36:55.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.885+0000 7f73b4c00640 1 -- 172.21.15.59:0/2111748568 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73b0011070 con 0x7f73b8146290 2024-04-24T06:36:55.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.885+0000 7f73b4c00640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 0x7f739806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.886+0000 7f73b4c00640 1 -- 172.21.15.59:0/2111748568 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f73b008b810 con 0x7f73b8146290 2024-04-24T06:36:55.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.886+0000 7f73bd200640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 0x7f739806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.886+0000 7f73c2a00740 1 Processor -- start 2024-04-24T06:36:55.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.886+0000 7f73c2a00740 1 -- start start 2024-04-24T06:36:55.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73bd200640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 0x7f739806ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73a8034000 tx=0x7f73a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc7fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc82d7e0 0x5578bc8025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5578bc7671f0 con 0x5578bc82df20 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5578bc70d350 con 0x5578bc84ae00 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.887+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5578bc7631b0 con 0x5578bc82d7e0 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47332/0 (socket says 172.21.15.59:47332) 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 -- 172.21.15.59:0/3301419854 learned_addr learned my addr 172.21.15.59:0/3301419854 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73bdc00640 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc82d7e0 0x5578bc8025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73bd200640 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc7fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 -- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc82d7e0 msgr2=0x5578bc8025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc82d7e0 0x5578bc8025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 -- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 msgr2=0x5578bc7fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc7fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 -- 172.21.15.59:0/3301419854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578bc761f50 con 0x5578bc82df20 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.888+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fc0 tx=0x7f73a400d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9daadb48038012 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.889+0000 7f73ad600640 1 -- 172.21.15.59:0/3301419854 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400f920 con 0x5578bc82df20 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.889+0000 7f73ad600640 1 -- 172.21.15.59:0/3301419854 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a400fac0 con 0x5578bc82df20 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.889+0000 7f73ad600640 1 -- 172.21.15.59:0/3301419854 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4018480 con 0x5578bc82df20 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.889+0000 7f73c2a00740 1 -- 172.21.15.59:0/3301419854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 msgr2=0x5578bc82d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.889+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f73a4007fc0 tx=0x7f73a400d800 comp rx=0 tx=0).stop 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 -- 172.21.15.59:0/3301419854 shutdown_connections 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc82d7e0 0x5578bc8025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc82d2a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3301419854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc7fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 -- 172.21.15.59:0/3301419854 >> 172.21.15.59:0/3301419854 conn(0x5578bc7ffa00 msgr2=0x5578bc8482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 -- 172.21.15.59:0/3301419854 shutdown_connections 2024-04-24T06:36:55.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 -- 172.21.15.59:0/3301419854 wait complete. 2024-04-24T06:36:55.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.890+0000 7f73c2a00740 1 -- start start 2024-04-24T06:36:55.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.891+0000 7f73c2a00740 1 Processor -- start 2024-04-24T06:36:55.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc85c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc859500 0x5578bc803090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5578bc801bb0 con 0x5578bc82df20 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5578bc720690 con 0x5578bc84ae00 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5578bc800fa0 con 0x5578bc859500 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73c2a00740 10 client.0 _ll_register_callbacks cb 0x5578bc75c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47340/0 (socket says 172.21.15.59:47340) 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 -- 172.21.15.59:0/3136031085 learned_addr learned my addr 172.21.15.59:0/3136031085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 -- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc859500 msgr2=0x5578bc803090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc859500 0x5578bc803090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc85c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 -- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 msgr2=0x5578bc85c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc85c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.892+0000 7f73bd200640 1 -- 172.21.15.59:0/3136031085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578bc71f920 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73bd200640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009890 tx=0x7f73a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94a6a1683cfef70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73a2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a804e060 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f73a803e400 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a400fac0 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f73a8054050 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73c2a00740 10 client.7938 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.893+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5578bc5437d0 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a400fac0 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f73a8057020 con 0x5578bc82df20 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 10 client.7938 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 10 client.7938 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:55.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 20 client.7938 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc82e8e0 0x5578bc845e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 4 client.7938 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578bc774c10 con 0x5578bc82e8e0 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.894+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc82e8e0 0x5578bc845e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.895+0000 7f73b7e00640 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc82e8e0 0x5578bc845e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d0eeb3e28e36cf server_cookie=37dc0c553fec0dd0 in_seq=0 out_seq=0 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.895+0000 7f73a2000640 10 client.7938 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73a2000640 1 -- 172.21.15.59:0/3136031085 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578bc774c10 con 0x5578bc82e8e0 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73a2000640 10 client.7938 handle_command_reply: tid=0 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 client.7938 shutdown 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 msgr2=0x5578bc828b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009890 tx=0x7f73a8004730 comp rx=0 tx=0).stop 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 shutdown_connections 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc82e8e0 0x5578bc845e50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc859500 0x5578bc803090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc84ae00 0x5578bc85c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 --2- 172.21.15.59:0/3136031085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc828b60 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73a2000640 0 client.7938 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:55.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 >> 172.21.15.59:0/3136031085 conn(0x5578bc82d7e0 msgr2=0x5578bc801280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 shutdown_connections 2024-04-24T06:36:55.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.896+0000 7f73c2a00740 1 -- 172.21.15.59:0/3136031085 wait complete. 2024-04-24T06:36:55.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.897+0000 7f73c2a00740 20 client.7938 trim_cache size 0 max 16384 2024-04-24T06:36:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.916+0000 7f73c2a00740 1 Processor -- start 2024-04-24T06:36:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.916+0000 7f73c2a00740 1 -- start start 2024-04-24T06:36:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc82df20 0x5578bc8400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc86c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8200e00 con 0x5578bc859500 2024-04-24T06:36:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ffe90 con 0x5578bc82df20 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80ae070 con 0x5578bc84ae00 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47344/0 (socket says 172.21.15.59:47344) 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.917+0000 7f73bdc00640 1 -- 172.21.15.59:0/2326596866 learned_addr learned my addr 172.21.15.59:0/2326596866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 -- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 msgr2=0x5578bc86c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc86c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 -- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc82df20 msgr2=0x5578bc8400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc82df20 0x5578bc8400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 -- 172.21.15.59:0/2326596866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578bc4793f0 con 0x5578bc859500 2024-04-24T06:36:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73bdc00640 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f73b00558b0 tx=0x7f73b001a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea78423150dd1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73acc00640 1 -- 172.21.15.59:0/2326596866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0088070 con 0x5578bc859500 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73acc00640 1 -- 172.21.15.59:0/2326596866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b0059450 con 0x5578bc859500 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.918+0000 7f73acc00640 1 -- 172.21.15.59:0/2326596866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b006de60 con 0x5578bc859500 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 -- 172.21.15.59:0/2326596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 msgr2=0x5578bc7fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f73b00558b0 tx=0x7f73b001a2b0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 -- 172.21.15.59:0/2326596866 shutdown_connections 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc859500 0x5578bc7fd800 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc86c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 --2- 172.21.15.59:0/2326596866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc82df20 0x5578bc8400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 -- 172.21.15.59:0/2326596866 >> 172.21.15.59:0/2326596866 conn(0x5578bc82d7e0 msgr2=0x5578bc7f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 -- 172.21.15.59:0/2326596866 shutdown_connections 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.919+0000 7f73c2a00740 1 -- 172.21.15.59:0/2326596866 wait complete. 2024-04-24T06:36:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.920+0000 7f73c2a00740 1 -- start start 2024-04-24T06:36:55.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.920+0000 7f73c2a00740 1 Processor -- start 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc8bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc859500 0x5578bc8bc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5578bc8493d0 con 0x5578bc82df20 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80ae6c0 con 0x5578bc859500 2024-04-24T06:36:55.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5578bc828a00 con 0x5578bc84ae00 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73c2a00740 10 client.0 _ll_register_callbacks cb 0x7f73a8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73bd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73bd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47348/0 (socket says 172.21.15.59:47348) 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.921+0000 7f73bd200640 1 -- 172.21.15.59:0/564030774 learned_addr learned my addr 172.21.15.59:0/564030774 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 -- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 msgr2=0x5578bc8bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc8bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 -- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc859500 msgr2=0x5578bc8bc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc859500 0x5578bc8bc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 -- 172.21.15.59:0/564030774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578bc4390f0 con 0x5578bc82df20 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73bd200640 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f73a80096a0 tx=0x7f73a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03b834dc5674e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73a2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a804e050 con 0x5578bc82df20 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f73a803e6d0 con 0x5578bc82df20 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.922+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73b0059450 con 0x5578bc82df20 2024-04-24T06:36:55.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f73a8037040 con 0x5578bc82df20 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 10 client.7944 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5578bc72c810 con 0x5578bc82df20 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b0059450 con 0x5578bc82df20 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f73a8057040 con 0x5578bc82df20 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 10 client.7944 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 10 client.7944 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 20 client.7944 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc86c710 0x5578bc829a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 4 client.7944 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578bc76ea80 con 0x5578bc86c710 2024-04-24T06:36:55.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.923+0000 7f73b7e00640 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc86c710 0x5578bc829a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:55.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73b7e00640 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc86c710 0x5578bc829a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39bce9865c52b21 server_cookie=dddb4494d57d8621 in_seq=0 out_seq=0 2024-04-24T06:36:55.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73a2000640 10 client.7944 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:55.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73a2000640 1 -- 172.21.15.59:0/564030774 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578bc76ea80 con 0x5578bc86c710 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73a2000640 10 client.7944 handle_command_reply: tid=0 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 client.7944 shutdown 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 msgr2=0x5578bc8b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f73a80096a0 tx=0x7f73a80046c0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 shutdown_connections 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5578bc86c710 0x5578bc829a00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578bc859500 0x5578bc8bc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5578bc84ae00 0x5578bc8bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 --2- 172.21.15.59:0/564030774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578bc82df20 0x5578bc8b6190 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 >> 172.21.15.59:0/564030774 conn(0x5578bc82d7e0 msgr2=0x5578bc83fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.924+0000 7f73a2000640 0 client.7944 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 shutdown_connections 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73c2a00740 1 -- 172.21.15.59:0/564030774 wait complete. 2024-04-24T06:36:55.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73c2a00740 20 client.7944 trim_cache size 0 max 16384 2024-04-24T06:36:55.929 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:36:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:36:55.929 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 msgr2=0x7f739806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 0x7f739806ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f73a8034000 tx=0x7f73a802e000 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 msgr2=0x7f73b8146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.925+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f73b000c440 tx=0x7f73b000c900 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 shutdown_connections 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f739806c920 0x7f739806ede0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73b8146290 0x7f73b8146710 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f73b80724d0 0x7f73b809d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 --2- 172.21.15.59:0/2111748568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73b806fb50 0x7f73b809cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 >> 172.21.15.59:0/2111748568 conn(0x7f73b806bdf0 msgr2=0x7f73b80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 shutdown_connections 2024-04-24T06:36:55.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:55.926+0000 7f73be600640 1 -- 172.21.15.59:0/2111748568 wait complete. 2024-04-24T06:36:55.944 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:36:55.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:36:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.147+0000 7f914b200640 1 Processor -- start 2024-04-24T06:36:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.148+0000 7f914b200640 1 -- start start 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.149+0000 7f914b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f914c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.149+0000 7f914b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c0cf030 con 0x7f914c0d0d90 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.149+0000 7f914b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914c0b6a40 con 0x7f914c0d2630 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.149+0000 7f914b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f914c0b7000 con 0x7f914c15e7c0 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f914a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f914c0d0d90 0x7f914c0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40354/0 (socket says 172.21.15.59:40354) 2024-04-24T06:36:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f914a800640 1 -- 172.21.15.59:0/4122208838 learned_addr learned my addr 172.21.15.59:0/4122208838 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9149e00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f914c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737055706 0 0) 0x7f9144001110 con 0x7f914c0d0d90 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c003520 con 0x7f914c0d0d90 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723720961 0 0) 0x7f912c003520 con 0x7f914c0d0d90 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 msgr2=0x7f914c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f914c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.150+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 >> v1:172.21.15.66:6789/0 conn(0x7f914c0d2630 legacy=0x7f914c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0b6d20 con 0x7f914c0d0d90 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9144001840 con 0x7f914c0d0d90 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 >> v1:172.21.15.59:6789/0 conn(0x7f914c0d0d90 legacy=0x7f914c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0039e0 0x7f912c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f912c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f912c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0b7000 con 0x7f912c004800 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f912c00ba70 con 0x7f912c0039e0 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f912c00bc70 con 0x7f914c15e7c0 2024-04-24T06:36:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9149400640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f912c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f9149e00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0039e0 0x7f912c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.151+0000 7f914a800640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f912c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149400640 1 -- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 msgr2=0x7f912c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149400640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c15e7c0 0x7f912c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149400640 1 -- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0039e0 msgr2=0x7f912c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149400640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0039e0 0x7f912c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149400640 1 -- 172.21.15.59:0/4122208838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c003520 con 0x7f912c004800 2024-04-24T06:36:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.152+0000 7f9149e00640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f912c0039e0 0x7f912c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:36:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.153+0000 7f9149400640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f912c004c80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f914c0d2a30 tx=0x7f912c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b677d7b6120d5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.153+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134001a70 con 0x7f912c004800 2024-04-24T06:36:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.153+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913402adf0 con 0x7f912c004800 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.153+0000 7f9148a00640 1 -- 172.21.15.59:0/4122208838 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134003bb0 con 0x7f912c004800 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 -- 172.21.15.59:0/4122208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 msgr2=0x7f912c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 --2- 172.21.15.59:0/4122208838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f912c004c80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f914c0d2a30 tx=0x7f912c005830 comp rx=0 tx=0).stop 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914a800640 1 -- 172.21.15.59:0/4122208838 reap_dead start 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 -- 172.21.15.59:0/4122208838 shutdown_connections 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 -- 172.21.15.59:0/4122208838 >> 172.21.15.59:0/4122208838 conn(0x7f914c09d920 msgr2=0x7f914c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 -- 172.21.15.59:0/4122208838 shutdown_connections 2024-04-24T06:36:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.154+0000 7f914b200640 1 -- 172.21.15.59:0/4122208838 wait complete. 2024-04-24T06:36:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.156+0000 7f914b200640 1 Processor -- start 2024-04-24T06:36:56.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.156+0000 7f914b200640 1 -- start start 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c0b2950 0x7f914c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f914c15d630 0x7f914c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0b6700 con 0x7f912c004800 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0cf620 con 0x7f914c15d630 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f914c0d1a80 con 0x7f914c0b2950 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f9149e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f9149e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47366/0 (socket says 172.21.15.59:47366) 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f9149e00640 1 -- 172.21.15.59:0/1711219336 learned_addr learned my addr 172.21.15.59:0/1711219336 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f9149400640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c0b2950 0x7f914c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.157+0000 7f914a800640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f914c15d630 0x7f914c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 -- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c0b2950 msgr2=0x7f914c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c0b2950 0x7f914c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 -- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f914c15d630 msgr2=0x7f914c15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f914c15d630 0x7f914c15dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 -- 172.21.15.59:0/1711219336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0cd050 con 0x7f912c004800 2024-04-24T06:36:56.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9149e00640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f913c011730 tx=0x7f913c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00f9c0 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c00fb60 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.158+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c015460 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.159+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c0cd330 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.159+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c0cd8f0 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.160+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c04fd90 con 0x7f912c004800 2024-04-24T06:36:56.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.161+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f913c015600 con 0x7f912c004800 2024-04-24T06:36:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.162+0000 7f9142000640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.162+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f913c08c6b0 con 0x7f912c004800 2024-04-24T06:36:56.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.162+0000 7f9149400640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.163+0000 7f9149400640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9134032000 tx=0x7f913402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.167+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f913c019030 con 0x7f912c004800 2024-04-24T06:36:56.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.411+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914c0b8000 con 0x7f912c004800 2024-04-24T06:36:56.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.413+0000 7f9142000640 1 -- 172.21.15.59:0/1711219336 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5182 (secure 0 0 0) 0x7f913c05a960 con 0x7f912c004800 2024-04-24T06:36:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:56.414 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:31.666787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7548,"qdb_cluster":[7548]},"id":6}]} 2024-04-24T06:36:56.414 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 68 2024-04-24T06:36:56.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 msgr2=0x7f912806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9134032000 tx=0x7f913402e000 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 msgr2=0x7f914c0c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f913c011730 tx=0x7f913c011bf0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 shutdown_connections 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f914c15d630 0x7f914c15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f914c0b2950 0x7f914c0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 --2- 172.21.15.59:0/1711219336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f912c004800 0x7f914c0c9f80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 >> 172.21.15.59:0/1711219336 conn(0x7f914c09d920 msgr2=0x7f914c1613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.416+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 shutdown_connections 2024-04-24T06:36:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.417+0000 7f914b200640 1 -- 172.21.15.59:0/1711219336 wait complete. 2024-04-24T06:36:56.438 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-24T06:36:56.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.652+0000 7fad9f200640 1 Processor -- start 2024-04-24T06:36:56.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.653+0000 7fad9f200640 1 -- start start 2024-04-24T06:36:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.654+0000 7fad9f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.654+0000 7fad9f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fada00b4f20 0x7fada00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.654+0000 7fad9f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada00b5da0 con 0x7fada015ed20 2024-04-24T06:36:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.654+0000 7fad9f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fada015f100 con 0x7fada00b4f20 2024-04-24T06:36:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.654+0000 7fad9f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fada00d0a20 con 0x7fada00d15b0 2024-04-24T06:36:56.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fada015ed20 0x7fada00d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40362/0 (socket says 172.21.15.59:40362) 2024-04-24T06:36:56.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9de00640 1 -- 172.21.15.59:0/4127594894 learned_addr learned my addr 172.21.15.59:0/4127594894 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:56.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9e800640 1 --2- 172.21.15.59:0/4127594894 >> v2:172.21.15.59:3300/0 conn(0x7fada00b4f20 0x7fada00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9d400640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9e800640 1 -- 172.21.15.59:0/4127594894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 msgr2=0x7fada00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9e800640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9e800640 1 -- 172.21.15.59:0/4127594894 >> v1:172.21.15.59:6789/0 conn(0x7fada015ed20 legacy=0x7fada00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9ca00640 1 -- 172.21.15.59:0/4127594894 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131832656 0 0) 0x7fada00b5da0 con 0x7fada015ed20 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.655+0000 7fad9e800640 1 -- 172.21.15.59:0/4127594894 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00d2ed0 con 0x7fada00b4f20 2024-04-24T06:36:56.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.656+0000 7fad9e800640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00cf7c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad980097a0 tx=0x7fad9802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e343377e16d73fac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.657+0000 7fad9ca00640 1 -- 172.21.15.59:0/4127594894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98037070 con 0x7fada00b4f20 2024-04-24T06:36:56.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.657+0000 7fad9ca00640 1 -- 172.21.15.59:0/4127594894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9802adb0 con 0x7fada00b4f20 2024-04-24T06:36:56.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.657+0000 7fad9ca00640 1 -- 172.21.15.59:0/4127594894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98032660 con 0x7fada00b4f20 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.657+0000 7fad9f200640 1 -- 172.21.15.59:0/4127594894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 msgr2=0x7fada00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.657+0000 7fad9f200640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00cf7c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad980097a0 tx=0x7fad9802a060 comp rx=0 tx=0).stop 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 -- 172.21.15.59:0/4127594894 shutdown_connections 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00cf7c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 --2- 172.21.15.59:0/4127594894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 -- 172.21.15.59:0/4127594894 >> 172.21.15.59:0/4127594894 conn(0x7fada008bf90 msgr2=0x7fada00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 -- 172.21.15.59:0/4127594894 shutdown_connections 2024-04-24T06:36:56.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.658+0000 7fad9f200640 1 -- 172.21.15.59:0/4127594894 wait complete. 2024-04-24T06:36:56.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.659+0000 7fad9f200640 1 Processor -- start 2024-04-24T06:36:56.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 -- start start 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada015ed20 0x7fada00ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fada00b3600 con 0x7fada00b4f20 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fada00d3490 con 0x7fada015ed20 2024-04-24T06:36:56.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.660+0000 7fad9f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fada00b3780 con 0x7fada00d15b0 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47386/0 (socket says 172.21.15.59:47386) 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 -- 172.21.15.59:0/3707848379 learned_addr learned my addr 172.21.15.59:0/3707848379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9d400640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9e800640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada015ed20 0x7fada00ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 -- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 msgr2=0x7fada015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada015bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 -- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada015ed20 msgr2=0x7fada00ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada015ed20 0x7fada00ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.661+0000 7fad9de00640 1 -- 172.21.15.59:0/3707848379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0159530 con 0x7fada00b4f20 2024-04-24T06:36:56.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad9de00640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fad9000c460 tx=0x7fad9000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad96000640 1 -- 172.21.15.59:0/3707848379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90018070 con 0x7fada00b4f20 2024-04-24T06:36:56.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad96000640 1 -- 172.21.15.59:0/3707848379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90007500 con 0x7fada00b4f20 2024-04-24T06:36:56.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad96000640 1 -- 172.21.15.59:0/3707848379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90004a20 con 0x7fada00b4f20 2024-04-24T06:36:56.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada0159810 con 0x7fada00b4f20 2024-04-24T06:36:56.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.662+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada0157910 con 0x7fada00b4f20 2024-04-24T06:36:56.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.664+0000 7fad96000640 1 -- 172.21.15.59:0/3707848379 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad90004bc0 con 0x7fada00b4f20 2024-04-24T06:36:56.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.665+0000 7fad96000640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.665+0000 7fad96000640 1 -- 172.21.15.59:0/3707848379 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fad9008ae70 con 0x7fada00b4f20 2024-04-24T06:36:56.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.666+0000 7fad9d400640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.666+0000 7fada7806740 1 Processor -- start 2024-04-24T06:36:56.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.666+0000 7fada7806740 1 -- start start 2024-04-24T06:36:56.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.666+0000 7fad9d400640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 0x7fad7c06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fada00c97c0 tx=0x7fad8800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56409569bf00 0x5640956e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564095600190 con 0x5640956c6220 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5640955a6030 con 0x5640956e3ac0 2024-04-24T06:36:56.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5640955fc150 con 0x56409569bf00 2024-04-24T06:36:56.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.667+0000 7fad9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47396/0 (socket says 172.21.15.59:47396) 2024-04-24T06:36:56.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 -- 172.21.15.59:0/2234712642 learned_addr learned my addr 172.21.15.59:0/2234712642 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9d400640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9e800640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56409569bf00 0x5640956e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 -- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56409569bf00 msgr2=0x5640956e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56409569bf00 0x5640956e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 -- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 msgr2=0x5640956e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 -- 172.21.15.59:0/2234712642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409558eb60 con 0x5640956c6220 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9d400640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad9de00640 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fad90054fa0 tx=0x7fad90058a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25067034ff493e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.668+0000 7fad8e600640 1 -- 172.21.15.59:0/2234712642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90061720 con 0x5640956c6220 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fad8e600640 1 -- 172.21.15.59:0/2234712642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90056e40 con 0x5640956c6220 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fad8e600640 1 -- 172.21.15.59:0/2234712642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90057f90 con 0x5640956c6220 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 -- 172.21.15.59:0/2234712642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 msgr2=0x5640956c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fad90054fa0 tx=0x7fad90058a30 comp rx=0 tx=0).stop 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 -- 172.21.15.59:0/2234712642 shutdown_connections 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56409569bf00 0x5640956e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 --2- 172.21.15.59:0/2234712642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956c6220 0x5640956c6600 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.669+0000 7fada7806740 1 -- 172.21.15.59:0/2234712642 >> 172.21.15.59:0/2234712642 conn(0x564095698830 msgr2=0x5640956e1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.670+0000 7fada7806740 1 -- 172.21.15.59:0/2234712642 shutdown_connections 2024-04-24T06:36:56.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.670+0000 7fada7806740 1 -- 172.21.15.59:0/2234712642 wait complete. 2024-04-24T06:36:56.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.670+0000 7fada7806740 1 -- start start 2024-04-24T06:36:56.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.671+0000 7fada7806740 1 Processor -- start 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x5640956da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56409569cca0 con 0x5640956e17c0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5640955b9500 con 0x5640956e3ac0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564095696760 con 0x5640956c6220 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fada7806740 10 client.0 _ll_register_callbacks cb 0x5640955f6270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fad9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fad9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47408/0 (socket says 172.21.15.59:47408) 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fad9e800640 1 -- 172.21.15.59:0/1202731294 learned_addr learned my addr 172.21.15.59:0/1202731294 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fad9d400640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.672+0000 7fad9de00640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x5640956da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 -- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 msgr2=0x5640956da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x5640956da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 -- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 msgr2=0x5640956de7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956de7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 -- 172.21.15.59:0/1202731294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640953de7c0 con 0x5640956e17c0 2024-04-24T06:36:56.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad9e800640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fad98009890 tx=0x7fad98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3f73ec0ca9f7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad86a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98041030 con 0x5640956e17c0 2024-04-24T06:36:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fad980043f0 con 0x5640956e17c0 2024-04-24T06:36:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad90056e40 con 0x5640956e17c0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.673+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fad9804d050 con 0x5640956e17c0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 10 client.7965 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56409543ef50 con 0x5640956e17c0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90056e40 con 0x5640956e17c0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fad98050020 con 0x5640956e17c0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 10 client.7965 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 10 client.7965 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 20 client.7965 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956f4e20 0x5640956f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 4 client.7965 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56409560dbb0 con 0x5640956f4e20 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.674+0000 7fad9de00640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956f4e20 0x5640956f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.675+0000 7fad9de00640 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956f4e20 0x5640956f5200 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5eed18f0c4a398 server_cookie=92ad6d0d24f18ec1 in_seq=0 out_seq=0 2024-04-24T06:36:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.675+0000 7fad86a00640 10 client.7965 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.675+0000 7fad86a00640 1 -- 172.21.15.59:0/1202731294 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56409560dbb0 con 0x5640956f4e20 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.675+0000 7fad86a00640 10 client.7965 handle_command_reply: tid=0 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 client.7965 shutdown 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 msgr2=0x5640956dba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fad98009890 tx=0x7fad98004730 comp rx=0 tx=0).stop 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 shutdown_connections 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956f4e20 0x5640956f5200 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x5640956dba50 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x5640956de7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 --2- 172.21.15.59:0/1202731294 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x5640956da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 >> 172.21.15.59:0/1202731294 conn(0x56409569bf00 msgr2=0x5640956e4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fad86a00640 0 client.7965 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 shutdown_connections 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 1 -- 172.21.15.59:0/1202731294 wait complete. 2024-04-24T06:36:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.676+0000 7fada7806740 20 client.7965 trim_cache size 0 max 16384 2024-04-24T06:36:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.691+0000 7fada7806740 1 Processor -- start 2024-04-24T06:36:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.691+0000 7fada7806740 1 -- start start 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956c6220 0x5640956c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 0x564095696b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fada01ffe30 con 0x5640956e3ac0 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fada0159140 con 0x5640956c6220 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.692+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fada01ff7e0 con 0x5640956e17c0 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47416/0 (socket says 172.21.15.59:47416) 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 -- 172.21.15.59:0/2922437517 learned_addr learned my addr 172.21.15.59:0/2922437517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9de00640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956c6220 0x5640956c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9d400640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 0x564095696b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 -- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 msgr2=0x564095696b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 0x564095696b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 -- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956c6220 msgr2=0x5640956c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956c6220 0x5640956c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 -- 172.21.15.59:0/2922437517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640952d20b0 con 0x5640956e3ac0 2024-04-24T06:36:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9d400640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 0x564095696b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.693+0000 7fad9e800640 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad980094c0 tx=0x7fad9803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33048ae012617c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.694+0000 7fad8dc00640 1 -- 172.21.15.59:0/2922437517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98041050 con 0x5640956e3ac0 2024-04-24T06:36:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.694+0000 7fad8dc00640 1 -- 172.21.15.59:0/2922437517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad980038c0 con 0x5640956e3ac0 2024-04-24T06:36:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.694+0000 7fad8dc00640 1 -- 172.21.15.59:0/2922437517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98037040 con 0x5640956e3ac0 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.694+0000 7fada7806740 1 -- 172.21.15.59:0/2922437517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 msgr2=0x56409569b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.694+0000 7fada7806740 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad980094c0 tx=0x7fad9803d000 comp rx=0 tx=0).stop 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 -- 172.21.15.59:0/2922437517 shutdown_connections 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e3ac0 0x56409569b130 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956e17c0 0x564095696b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 --2- 172.21.15.59:0/2922437517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956c6220 0x5640956c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 -- 172.21.15.59:0/2922437517 >> 172.21.15.59:0/2922437517 conn(0x56409569bf00 msgr2=0x5640956de5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 -- 172.21.15.59:0/2922437517 shutdown_connections 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 -- 172.21.15.59:0/2922437517 wait complete. 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.695+0000 7fada7806740 1 -- start start 2024-04-24T06:36:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 Processor -- start 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x564095740f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x564095703c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640956d8d10 con 0x5640956e17c0 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5640956d9b50 con 0x5640956e3ac0 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.696+0000 7fada7806740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5640956babd0 con 0x5640956c6220 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fada7806740 10 client.0 _ll_register_callbacks cb 0x7fad98035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47430/0 (socket says 172.21.15.59:47430) 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 -- 172.21.15.59:0/2826044140 learned_addr learned my addr 172.21.15.59:0/2826044140 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9de00640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x564095740f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9e800640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x564095703c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 -- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 msgr2=0x564095740f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x564095740f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 -- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 msgr2=0x564095703c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x564095703c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.697+0000 7fad9d400640 1 -- 172.21.15.59:0/2826044140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640956da2a0 con 0x5640956e17c0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fad9d400640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fada00a24e0 tx=0x7fad88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae7ddfc8da01664 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fad86a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88027020 con 0x5640956e17c0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fad880147e0 con 0x5640956e17c0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad980038c0 con 0x5640956e17c0 2024-04-24T06:36:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fad88031050 con 0x5640956e17c0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fada7806740 10 client.7971 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.698+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56409569a210 con 0x5640956e17c0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad980038c0 con 0x5640956e17c0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fad88034020 con 0x5640956e17c0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 10 client.7971 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 10 client.7971 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 20 client.7971 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956d69c0 0x5640956d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 4 client.7971 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564095607a20 con 0x5640956d69c0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fad9de00640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956d69c0 0x5640956d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fad86a00640 10 client.7971 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.699+0000 7fad9de00640 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956d69c0 0x5640956d6da0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b418fe30594f1172 server_cookie=efe894c5dd601b75 in_seq=0 out_seq=0 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fad86a00640 1 -- 172.21.15.59:0/2826044140 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x564095607a20 con 0x5640956d69c0 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fad86a00640 10 client.7971 handle_command_reply: tid=0 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 client.7971 shutdown 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 msgr2=0x564095699c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fada00a24e0 tx=0x7fad88004ab0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 shutdown_connections 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5640956d69c0 0x5640956d6da0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5640956e3ac0 0x564095703c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fad86a00640 0 client.7971 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640956e17c0 0x564095699c40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 --2- 172.21.15.59:0/2826044140 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5640956c6220 0x564095740f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 >> 172.21.15.59:0/2826044140 conn(0x56409569bf00 msgr2=0x56409569b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 shutdown_connections 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 1 -- 172.21.15.59:0/2826044140 wait complete. 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.700+0000 7fada7806740 20 client.7971 trim_cache size 0 max 16384 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "dir_frag", 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 1939526196, 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: "frag": "*", 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir" 2024-04-24T06:36:56.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3403960017, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir" 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4293049403, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/testdir/hardlink" 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:56.706 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 msgr2=0x7fad7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 0x7fad7c06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fada00c97c0 tx=0x7fad8800b040 comp rx=0 tx=0).stop 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 msgr2=0x7fada00c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fad9000c460 tx=0x7fad9000c920 comp rx=0 tx=0).stop 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 shutdown_connections 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fada015ed20 0x7fada00ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fada00d15b0 0x7fada015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 --2- 172.21.15.59:0/3707848379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00b4f20 0x7fada00c9280 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 >> 172.21.15.59:0/3707848379 conn(0x7fada008bf90 msgr2=0x7fada006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 shutdown_connections 2024-04-24T06:36:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.701+0000 7fad9f200640 1 -- 172.21.15.59:0/3707848379 wait complete. 2024-04-24T06:36:56.719 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:36:56.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.908+0000 7f4d43e00640 1 Processor -- start 2024-04-24T06:36:56.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.908+0000 7f4d43e00640 1 -- start start 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d43e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d4415e7c0 0x7f4d440d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440d2630 0x7f4d440b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d43e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d440cf030 con 0x7f4d440b6530 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d43e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4d440b7000 con 0x7f4d4415e7c0 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d43e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4d440cf620 con 0x7f4d440d2630 2024-04-24T06:36:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d42a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d4415e7c0 0x7f4d440d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d42a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d4415e7c0 0x7f4d440d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47438/0 (socket says 172.21.15.59:47438) 2024-04-24T06:36:56.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.910+0000 7f4d42a00640 1 -- 172.21.15.59:0/3912088758 learned_addr learned my addr 172.21.15.59:0/3912088758 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.911+0000 7f4d3a000640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440d2630 0x7f4d440b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.911+0000 7f4d42a00640 1 -- 172.21.15.59:0/3912088758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440d2630 msgr2=0x7f4d440b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.911+0000 7f4d42a00640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440d2630 0x7f4d440b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.911+0000 7f4d42a00640 1 -- 172.21.15.59:0/3912088758 >> v1:172.21.15.66:6789/0 conn(0x7f4d440b6530 legacy=0x7f4d440d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:56.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.911+0000 7f4d42a00640 1 -- 172.21.15.59:0/3912088758 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d440b6a40 con 0x7f4d4415e7c0 2024-04-24T06:36:56.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.913+0000 7f4d42a00640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4415e7c0 0x7f4d440d20f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009e50 tx=0x7f4d3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a509dabc3fca24d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.913+0000 7f4d42000640 1 -- 172.21.15.59:0/3912088758 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34036070 con 0x7f4d4415e7c0 2024-04-24T06:36:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.913+0000 7f4d42000640 1 -- 172.21.15.59:0/3912088758 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34031600 con 0x7f4d4415e7c0 2024-04-24T06:36:56.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.913+0000 7f4d42000640 1 -- 172.21.15.59:0/3912088758 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340319d0 con 0x7f4d4415e7c0 2024-04-24T06:36:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 -- 172.21.15.59:0/3912088758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4415e7c0 msgr2=0x7f4d440d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4415e7c0 0x7f4d440d20f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009e50 tx=0x7f4d3402a6c0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 -- 172.21.15.59:0/3912088758 shutdown_connections 2024-04-24T06:36:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440d2630 0x7f4d440b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 --2- 172.21.15.59:0/3912088758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4415e7c0 0x7f4d440d20f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 -- 172.21.15.59:0/3912088758 >> 172.21.15.59:0/3912088758 conn(0x7f4d4409d920 msgr2=0x7f4d4408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.914+0000 7f4d43e00640 1 -- 172.21.15.59:0/3912088758 shutdown_connections 2024-04-24T06:36:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.915+0000 7f4d43e00640 1 -- 172.21.15.59:0/3912088758 wait complete. 2024-04-24T06:36:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.916+0000 7f4d43e00640 1 Processor -- start 2024-04-24T06:36:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.916+0000 7f4d43e00640 1 -- start start 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440b6530 0x7f4d440cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d4415e7c0 0x7f4d440b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d440d1a80 con 0x7f4d440d2630 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4d440cf2b0 con 0x7f4d4415e7c0 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.917+0000 7f4d43e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4d440d1c00 con 0x7f4d440b6530 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47442/0 (socket says 172.21.15.59:47442) 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 -- 172.21.15.59:0/2956856300 learned_addr learned my addr 172.21.15.59:0/2956856300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d43400640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d4415e7c0 0x7f4d440b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d42a00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440b6530 0x7f4d440cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 -- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440b6530 msgr2=0x7f4d440cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440b6530 0x7f4d440cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 -- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d4415e7c0 msgr2=0x7f4d440b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d4415e7c0 0x7f4d440b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 -- 172.21.15.59:0/2956856300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d440b1c20 con 0x7f4d440d2630 2024-04-24T06:36:56.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.918+0000 7f4d3a000640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d7d0 tx=0x7f4d2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.919+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00b6c0 con 0x7f4d440d2630 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.919+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c019070 con 0x7f4d440d2630 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.919+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c00bac0 con 0x7f4d440d2630 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.919+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d440b1f00 con 0x7f4d440d2630 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.919+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d440b24c0 con 0x7f4d440d2630 2024-04-24T06:36:56.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.920+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4404fd90 con 0x7f4d440d2630 2024-04-24T06:36:56.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.921+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2c00bc60 con 0x7f4d440d2630 2024-04-24T06:36:56.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.923+0000 7f4d3b400640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 0x7f4d2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.923+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f4d2c08c150 con 0x7f4d440d2630 2024-04-24T06:36:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.923+0000 7f4d42a00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 0x7f4d2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.924+0000 7f4d42a00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 0x7f4d2006ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d34033000 tx=0x7f4d34009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:56.927+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d2c05a400 con 0x7f4d440d2630 2024-04-24T06:36:57.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.175+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4d440b42d0 con 0x7f4d440d2630 2024-04-24T06:36:57.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.175+0000 7f4d3b400640 1 -- 172.21.15.59:0/2956856300 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f4d2c059d00 con 0x7f4d440d2630 2024-04-24T06:36:57.176 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:36:57.177 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 msgr2=0x7f4d2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 0x7f4d2006ede0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d34033000 tx=0x7f4d34009000 comp rx=0 tx=0).stop 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 msgr2=0x7f4d440b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d7d0 tx=0x7f4d2c00dc90 comp rx=0 tx=0).stop 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 shutdown_connections 2024-04-24T06:36:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4d2006c920 0x7f4d2006ede0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d4415e7c0 0x7f4d440b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d440d2630 0x7f4d440b14b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 --2- 172.21.15.59:0/2956856300 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4d440b6530 0x7f4d440cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.178+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 >> 172.21.15.59:0/2956856300 conn(0x7f4d4409d920 msgr2=0x7f4d4415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.179+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 shutdown_connections 2024-04-24T06:36:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.179+0000 7f4d43e00640 1 -- 172.21.15.59:0/2956856300 wait complete. 2024-04-24T06:36:57.198 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:36:57.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:57.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-24T06:36:57.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.418+0000 7fa4df200640 1 Processor -- start 2024-04-24T06:36:57.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.418+0000 7fa4df200640 1 -- start start 2024-04-24T06:36:57.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4df200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa4e0158c10 0x7fa4e0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4df200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4e0158ff0 0x7fa4e00b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4df200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e015f1d0 con 0x7fa4e006bdf0 2024-04-24T06:36:57.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4df200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e015fd50 con 0x7fa4e0158ff0 2024-04-24T06:36:57.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4df200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e0142350 con 0x7fa4e0158c10 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4dd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4e0158ff0 0x7fa4e00b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4de800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa4e006bdf0 0x7fa4e00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40376/0 (socket says 172.21.15.59:40376) 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4de800640 1 -- 172.21.15.59:0/943519202 learned_addr learned my addr 172.21.15.59:0/943519202 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.420+0000 7fa4dde00640 1 --2- 172.21.15.59:0/943519202 >> v2:172.21.15.66:3300/0 conn(0x7fa4e0158c10 0x7fa4e0141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dca00640 1 -- 172.21.15.59:0/943519202 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222820098 0 0) 0x7fa4e015f1d0 con 0x7fa4e006bdf0 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dca00640 1 -- 172.21.15.59:0/943519202 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4c4003520 con 0x7fa4e006bdf0 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dd400640 1 -- 172.21.15.59:0/943519202 >> v2:172.21.15.66:3300/0 conn(0x7fa4e0158c10 msgr2=0x7fa4e0141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dd400640 1 --2- 172.21.15.59:0/943519202 >> v2:172.21.15.66:3300/0 conn(0x7fa4e0158c10 0x7fa4e0141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dd400640 1 -- 172.21.15.59:0/943519202 >> v1:172.21.15.59:6789/0 conn(0x7fa4e006bdf0 legacy=0x7fa4e00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.421+0000 7fa4dd400640 1 -- 172.21.15.59:0/943519202 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e015f4b0 con 0x7fa4e0158ff0 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.422+0000 7fa4dd400640 1 --2- 172.21.15.59:0/943519202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b3ac0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00d12e0 tx=0x7fa4cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249974029ab97910 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4dca00640 1 -- 172.21.15.59:0/943519202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc037070 con 0x7fa4e0158ff0 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4dca00640 1 -- 172.21.15.59:0/943519202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc032440 con 0x7fa4e0158ff0 2024-04-24T06:36:57.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4dca00640 1 -- 172.21.15.59:0/943519202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc02ae60 con 0x7fa4e0158ff0 2024-04-24T06:36:57.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 -- 172.21.15.59:0/943519202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158ff0 msgr2=0x7fa4e00b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 --2- 172.21.15.59:0/943519202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b3ac0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00d12e0 tx=0x7fa4cc02a110 comp rx=0 tx=0).stop 2024-04-24T06:36:57.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 -- 172.21.15.59:0/943519202 shutdown_connections 2024-04-24T06:36:57.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 --2- 172.21.15.59:0/943519202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b3ac0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 --2- 172.21.15.59:0/943519202 >> v2:172.21.15.66:3300/0 conn(0x7fa4e0158c10 0x7fa4e0141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 -- 172.21.15.59:0/943519202 >> 172.21.15.59:0/943519202 conn(0x7fa4e00b2200 msgr2=0x7fa4e00b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 -- 172.21.15.59:0/943519202 shutdown_connections 2024-04-24T06:36:57.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.423+0000 7fa4df200640 1 -- 172.21.15.59:0/943519202 wait complete. 2024-04-24T06:36:57.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.425+0000 7fa4df200640 1 Processor -- start 2024-04-24T06:36:57.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.425+0000 7fa4df200640 1 -- start start 2024-04-24T06:36:57.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4e006bdf0 0x7fa4e009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e015fa70 con 0x7fa4e0158c10 2024-04-24T06:36:57.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b4000 con 0x7fa4e0158ff0 2024-04-24T06:36:57.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.426+0000 7fa4df200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e0071e10 con 0x7fa4e006bdf0 2024-04-24T06:36:57.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47450/0 (socket says 172.21.15.59:47450) 2024-04-24T06:36:57.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 -- 172.21.15.59:0/1721297159 learned_addr learned my addr 172.21.15.59:0/1721297159 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dde00640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4e006bdf0 0x7fa4e009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4de800640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 -- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4e006bdf0 msgr2=0x7fa4e009dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4e006bdf0 0x7fa4e009dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 -- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0158ff0 msgr2=0x7fa4e00b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.427+0000 7fa4dd400640 1 -- 172.21.15.59:0/1721297159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e009ac70 con 0x7fa4e0158c10 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4dd400640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc000c00 tx=0x7fa4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4d6000640 1 -- 172.21.15.59:0/1721297159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc041020 con 0x7fa4e0158c10 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4d6000640 1 -- 172.21.15.59:0/1721297159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc03d700 con 0x7fa4e0158c10 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4d6000640 1 -- 172.21.15.59:0/1721297159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc037040 con 0x7fa4e0158c10 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e009e1e0 con 0x7fa4e0158c10 2024-04-24T06:36:57.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.428+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e009b1f0 con 0x7fa4e0158c10 2024-04-24T06:36:57.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.430+0000 7fa4d6000640 1 -- 172.21.15.59:0/1721297159 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4cc044020 con 0x7fa4e0158c10 2024-04-24T06:36:57.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.431+0000 7fa4d6000640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.432+0000 7fa4d6000640 1 -- 172.21.15.59:0/1721297159 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa4cc0ac350 con 0x7fa4e0158c10 2024-04-24T06:36:57.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.432+0000 7fa4dde00640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.432+0000 7fa4e7a08740 1 Processor -- start 2024-04-24T06:36:57.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.432+0000 7fa4e7a08740 1 -- start start 2024-04-24T06:36:57.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.432+0000 7fa4dde00640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 0x7fa4c006edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009a10 tx=0x7fa4d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0020 0x5651b72a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5651b7209e70 con 0x5651b72ed640 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5651b71b0380 con 0x5651b72d0760 2024-04-24T06:36:57.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.433+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5651b7205e30 con 0x5651b72d0020 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47460/0 (socket says 172.21.15.59:47460) 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 -- 172.21.15.59:0/86321177 learned_addr learned my addr 172.21.15.59:0/86321177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dd400640 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4de800640 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0020 0x5651b72a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 -- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0020 msgr2=0x5651b72a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0020 0x5651b72a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 -- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 msgr2=0x5651b72cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.434+0000 7fa4dde00640 1 -- 172.21.15.59:0/86321177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b6fe47e0 con 0x5651b72ed640 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4dde00640 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0010be0 tx=0x7fa4d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339106e51e73e563 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4ca600640 1 -- 172.21.15.59:0/86321177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d001c050 con 0x5651b72ed640 2024-04-24T06:36:57.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4ca600640 1 -- 172.21.15.59:0/86321177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d0026070 con 0x5651b72ed640 2024-04-24T06:36:57.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4ca600640 1 -- 172.21.15.59:0/86321177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000f040 con 0x5651b72ed640 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4e7a08740 1 -- 172.21.15.59:0/86321177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 msgr2=0x5651b729ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.435+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0010be0 tx=0x7fa4d0013000 comp rx=0 tx=0).stop 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 -- 172.21.15.59:0/86321177 shutdown_connections 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0020 0x5651b72a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/86321177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b729ffd0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 -- 172.21.15.59:0/86321177 >> 172.21.15.59:0/86321177 conn(0x5651b72a2240 msgr2=0x5651b72eaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 -- 172.21.15.59:0/86321177 shutdown_connections 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 -- 172.21.15.59:0/86321177 wait complete. 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.436+0000 7fa4e7a08740 1 -- start start 2024-04-24T06:36:57.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.437+0000 7fa4e7a08740 1 Processor -- start 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b72ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5651b72a43f0 con 0x5651b72ed640 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5651b71c3660 con 0x5651b72d0760 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5651b72a37e0 con 0x5651b72cb4a0 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4e7a08740 10 client.0 _ll_register_callbacks cb 0x5651b71b1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47468/0 (socket says 172.21.15.59:47468) 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 -- 172.21.15.59:0/2011054663 learned_addr learned my addr 172.21.15.59:0/2011054663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dde00640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4de800640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b72ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 -- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 msgr2=0x5651b72ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b72ebc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 -- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 msgr2=0x5651b72b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72b9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.438+0000 7fa4dd400640 1 -- 172.21.15.59:0/2011054663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b7044f70 con 0x5651b72ed640 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4dd400640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc02a150 tx=0x7fa4cc07a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2e1c899df23fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4bea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc097060 con 0x5651b72ed640 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa4cc0826f0 con 0x5651b72ed640 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d0026070 con 0x5651b72ed640 2024-04-24T06:36:57.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fa4cc0aa050 con 0x5651b72ed640 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4e7a08740 10 client.7992 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5651b6edc170 con 0x5651b72ed640 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.439+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc0a5030 con 0x5651b72ed640 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fa4cc03a030 con 0x5651b72ed640 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 10 client.7992 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 10 client.7992 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 20 client.7992 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72d2720 0x5651b72d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 4 client.7992 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651b7217890 con 0x5651b72d2720 2024-04-24T06:36:57.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.440+0000 7fa4dde00640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72d2720 0x5651b72d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.441+0000 7fa4dde00640 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72d2720 0x5651b72d2ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf225193cfc8d41 server_cookie=2de0c94f4a026aa in_seq=0 out_seq=0 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.441+0000 7fa4bea00640 10 client.7992 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.441+0000 7fa4bea00640 1 -- 172.21.15.59:0/2011054663 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651b7217890 con 0x5651b72d2720 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.441+0000 7fa4bea00640 10 client.7992 handle_command_reply: tid=0 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.441+0000 7fa4e7a08740 1 client.7992 shutdown 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 msgr2=0x5651b72e2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc02a150 tx=0x7fa4cc07a6a0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 shutdown_connections 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72d2720 0x5651b72d2ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b72ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72e2c50 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/2011054663 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 >> 172.21.15.59:0/2011054663 conn(0x5651b72d0020 msgr2=0x5651b72ecb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4bea00640 0 client.7992 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 shutdown_connections 2024-04-24T06:36:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 1 -- 172.21.15.59:0/2011054663 wait complete. 2024-04-24T06:36:57.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.442+0000 7fa4e7a08740 20 client.7992 trim_cache size 0 max 16384 2024-04-24T06:36:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.462+0000 7fa4e7a08740 1 Processor -- start 2024-04-24T06:36:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.462+0000 7fa4e7a08740 1 -- start start 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72cb4a0 0x5651b729fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0760 0x5651b72a5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0200c80 con 0x5651b72ed640 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e01ffcd0 con 0x5651b72cb4a0 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e01564d0 con 0x5651b72d0760 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47472/0 (socket says 172.21.15.59:47472) 2024-04-24T06:36:57.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.463+0000 7fa4de800640 1 -- 172.21.15.59:0/1787422897 learned_addr learned my addr 172.21.15.59:0/1787422897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4dde00640 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72cb4a0 0x5651b729fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4dd400640 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0760 0x5651b72a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 -- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0760 msgr2=0x5651b72a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0760 0x5651b72a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 -- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72cb4a0 msgr2=0x5651b729fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72cb4a0 0x5651b729fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 -- 172.21.15.59:0/1787422897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b7215450 con 0x5651b72ed640 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4de800640 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001560 tx=0x7fa4d8001f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d40dee8670d260 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.464+0000 7fa4c9c00640 1 -- 172.21.15.59:0/1787422897 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80075f0 con 0x5651b72ed640 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4c9c00640 1 -- 172.21.15.59:0/1787422897 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d8019070 con 0x5651b72ed640 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4c9c00640 1 -- 172.21.15.59:0/1787422897 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80079d0 con 0x5651b72ed640 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 -- 172.21.15.59:0/1787422897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 msgr2=0x5651b72a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001560 tx=0x7fa4d8001f20 comp rx=0 tx=0).stop 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 -- 172.21.15.59:0/1787422897 shutdown_connections 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72a5e50 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72d0760 0x5651b72a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/1787422897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72cb4a0 0x5651b729fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 -- 172.21.15.59:0/1787422897 >> 172.21.15.59:0/1787422897 conn(0x5651b72d0020 msgr2=0x5651b72ecc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 -- 172.21.15.59:0/1787422897 shutdown_connections 2024-04-24T06:36:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.465+0000 7fa4e7a08740 1 -- 172.21.15.59:0/1787422897 wait complete. 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.466+0000 7fa4e7a08740 1 -- start start 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.466+0000 7fa4e7a08740 1 Processor -- start 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b7364c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5651b72cd3b0 con 0x5651b72ed640 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0156b20 con 0x5651b72d0760 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5651b72d37c0 con 0x5651b72cb4a0 2024-04-24T06:36:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.467+0000 7fa4e7a08740 10 client.0 _ll_register_callbacks cb 0x7fa4cc009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47488/0 (socket says 172.21.15.59:47488) 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 -- 172.21.15.59:0/972450955 learned_addr learned my addr 172.21.15.59:0/972450955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4dd400640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4dde00640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b7364c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 -- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 msgr2=0x5651b7364c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b7364c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 -- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 msgr2=0x5651b72e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4de800640 1 -- 172.21.15.59:0/972450955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b72cbff0 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.468+0000 7fa4dde00640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b7364c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4de800640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001f60 tx=0x7fa4d8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28077bdc668c4f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4bea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8005dd0 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa4d80147b0 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d8019070 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fa4d80149b0 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4e7a08740 10 client.7998 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5651b72cc200 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.469+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d8019070 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fa4d8014b50 con 0x5651b72ed640 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 10 client.7998 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 10 client.7998 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 20 client.7998 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72e1f70 0x5651b72e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 4 client.7998 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-24T06:36:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x5651b7211700 con 0x5651b72e1f70 2024-04-24T06:36:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.470+0000 7fa4dde00640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72e1f70 0x5651b72e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.471+0000 7fa4dde00640 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72e1f70 0x5651b72e2380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5dbbc1d62c9eeb3 server_cookie=afb643798f396f0c in_seq=0 out_seq=0 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.471+0000 7fa4bea00640 10 client.7998 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.471+0000 7fa4bea00640 1 -- 172.21.15.59:0/972450955 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651b7211700 con 0x5651b72e1f70 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.471+0000 7fa4bea00640 10 client.7998 handle_command_reply: tid=0 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.471+0000 7fa4e7a08740 1 client.7998 shutdown 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 msgr2=0x5651b72cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001f60 tx=0x7fa4d8001250 comp rx=0 tx=0).stop 2024-04-24T06:36:57.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 shutdown_connections 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x5651b72e1f70 0x5651b72e2380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5651b72ed640 0x5651b72cbab0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651b72d0760 0x5651b72e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 --2- 172.21.15.59:0/972450955 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5651b72cb4a0 0x5651b7364c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 >> 172.21.15.59:0/972450955 conn(0x5651b72d0020 msgr2=0x5651b72e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4bea00640 0 client.7998 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 shutdown_connections 2024-04-24T06:36:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 1 -- 172.21.15.59:0/972450955 wait complete. 2024-04-24T06:36:57.475 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:36:57.472+0000 7f14cb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)->f(v1 m2024-04-24T06:36:04.008555+0000) n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)->n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56308c7d6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.472+0000 7fa4e7a08740 20 client.7998 trim_cache size 0 max 16384 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "ee0e53e1-0fc9-4d72-a92f-36fc52f1af64", 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:36:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 msgr2=0x7fa4c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 0x7fa4c006edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009a10 tx=0x7fa4d0008040 comp rx=0 tx=0).stop 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 msgr2=0x7fa4e01576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc000c00 tx=0x7fa4cc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 shutdown_connections 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4e0158ff0 0x7fa4e00b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4e0158c10 0x7fa4e01576e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 --2- 172.21.15.59:0/1721297159 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa4e006bdf0 0x7fa4e009dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 >> 172.21.15.59:0/1721297159 conn(0x7fa4e00b2200 msgr2=0x7fa4e00b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 shutdown_connections 2024-04-24T06:36:57.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.473+0000 7fa4df200640 1 -- 172.21.15.59:0/1721297159 wait complete. 2024-04-24T06:36:57.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:36:57.491 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:36:57.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.696+0000 7f6abe400640 1 Processor -- start 2024-04-24T06:36:57.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.697+0000 7f6abe400640 1 -- start start 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 0x7f6ab80d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 0x7f6ab80af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ab80af5c0 0x7f6ab80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab80d0f70 con 0x7f6ab80af5c0 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab8160120 con 0x7f6ab815c410 2024-04-24T06:36:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.698+0000 7f6abe400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab8160400 con 0x7f6ab80d0b10 2024-04-24T06:36:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ab80af5c0 0x7f6ab80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ab80af5c0 0x7f6ab80b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47496/0 (socket says 172.21.15.59:47496) 2024-04-24T06:36:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 -- 172.21.15.59:0/3686784282 learned_addr learned my addr 172.21.15.59:0/3686784282 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abd000640 1 --2- 172.21.15.59:0/3686784282 >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 0x7f6ab80d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 0x7f6ab80af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 -- 172.21.15.59:0/3686784282 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 msgr2=0x7f6ab80af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 0x7f6ab80af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 -- 172.21.15.59:0/3686784282 >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 msgr2=0x7f6ab80d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 --2- 172.21.15.59:0/3686784282 >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 0x7f6ab80d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abda00640 1 -- 172.21.15.59:0/3686784282 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab81606e0 con 0x7f6ab80af5c0 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6abd000640 1 --2- 172.21.15.59:0/3686784282 >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 0x7f6ab80d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.699+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 0x7f6ab80af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.700+0000 7f6abda00640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80b3190 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80741a0 tx=0x7f6ab002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17053f7257363c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6ab7400640 1 -- 172.21.15.59:0/3686784282 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0036070 con 0x7f6ab80af5c0 2024-04-24T06:36:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6ab7400640 1 -- 172.21.15.59:0/3686784282 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab0031600 con 0x7f6ab80af5c0 2024-04-24T06:36:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6ab7400640 1 -- 172.21.15.59:0/3686784282 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00319d0 con 0x7f6ab80af5c0 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 -- 172.21.15.59:0/3686784282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 msgr2=0x7f6ab80b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80b3190 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80741a0 tx=0x7f6ab002a690 comp rx=0 tx=0).stop 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 -- 172.21.15.59:0/3686784282 shutdown_connections 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80b3190 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 --2- 172.21.15.59:0/3686784282 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab80d0b10 0x7f6ab80af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 --2- 172.21.15.59:0/3686784282 >> v2:172.21.15.66:3300/0 conn(0x7f6ab815c410 0x7f6ab80d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.701+0000 7f6abe400640 1 -- 172.21.15.59:0/3686784282 >> 172.21.15.59:0/3686784282 conn(0x7f6ab806bdf0 msgr2=0x7f6ab806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.702+0000 7f6abe400640 1 -- 172.21.15.59:0/3686784282 shutdown_connections 2024-04-24T06:36:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.702+0000 7f6abe400640 1 -- 172.21.15.59:0/3686784282 wait complete. 2024-04-24T06:36:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.703+0000 7f6abe400640 1 Processor -- start 2024-04-24T06:36:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.703+0000 7f6abe400640 1 -- start start 2024-04-24T06:36:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 0x7f6ab80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab815c410 0x7f6ab80b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab815c7f0 con 0x7f6ab80af5c0 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab80afa50 con 0x7f6ab80d0b10 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.704+0000 7f6abe400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab80ce2b0 con 0x7f6ab815c410 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47498/0 (socket says 172.21.15.59:47498) 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 -- 172.21.15.59:0/4144889134 learned_addr learned my addr 172.21.15.59:0/4144889134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 0x7f6ab80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 -- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab815c410 msgr2=0x7f6ab80b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab815c410 0x7f6ab80b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 -- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 msgr2=0x7f6ab80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 0x7f6ab80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 -- 172.21.15.59:0/4144889134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab80ce590 con 0x7f6ab80af5c0 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 0x7f6ab80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.705+0000 7f6abd000640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c7f0 tx=0x7f6aac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.706+0000 7f6ab4c00640 1 -- 172.21.15.59:0/4144889134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac015070 con 0x7f6ab80af5c0 2024-04-24T06:36:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.706+0000 7f6ab4c00640 1 -- 172.21.15.59:0/4144889134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac007820 con 0x7f6ab80af5c0 2024-04-24T06:36:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.706+0000 7f6ab4c00640 1 -- 172.21.15.59:0/4144889134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac007bd0 con 0x7f6ab80af5c0 2024-04-24T06:36:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.706+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab80ce870 con 0x7f6ab80af5c0 2024-04-24T06:36:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.706+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab80cdf60 con 0x7f6ab80af5c0 2024-04-24T06:36:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.708+0000 7f6ab4c00640 1 -- 172.21.15.59:0/4144889134 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6aac018070 con 0x7f6ab80af5c0 2024-04-24T06:36:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.709+0000 7f6ab4c00640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 0x7f6a9806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.710+0000 7f6ab4c00640 1 -- 172.21.15.59:0/4144889134 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f6aac08bc90 con 0x7f6ab80af5c0 2024-04-24T06:36:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.710+0000 7f6ac27fb740 1 Processor -- start 2024-04-24T06:36:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.710+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 0x7f6a9806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.710+0000 7f6ac27fb740 1 -- start start 2024-04-24T06:36:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 0x7f6a9806ee00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80cec10 tx=0x7f6aa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd44790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 0x564edbd495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564edbcae1f0 con 0x564edbd74f20 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564edbc54350 con 0x564edbd91e00 2024-04-24T06:36:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564edbcaa1b0 con 0x564edbd747e0 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47508/0 (socket says 172.21.15.59:47508) 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.711+0000 7f6ab7e00640 1 -- 172.21.15.59:0/735861594 learned_addr learned my addr 172.21.15.59:0/735861594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6abda00640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 0x564edbd495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6abd000640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd44790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 -- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 msgr2=0x564edbd495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 0x564edbd495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 -- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 msgr2=0x564edbd44790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd44790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 -- 172.21.15.59:0/735861594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564edbca8f50 con 0x564edbd74f20 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6abda00640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 0x564edbd495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400dbc0 tx=0x7f6aa4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98436e2484dcebc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6aa9600640 1 -- 172.21.15.59:0/735861594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4019050 con 0x564edbd74f20 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6aa9600640 1 -- 172.21.15.59:0/735861594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4023070 con 0x564edbd74f20 2024-04-24T06:36:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.712+0000 7f6aa9600640 1 -- 172.21.15.59:0/735861594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4012cb0 con 0x564edbd74f20 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 -- 172.21.15.59:0/735861594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 msgr2=0x564edbd742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400dbc0 tx=0x7f6aa4010000 comp rx=0 tx=0).stop 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 -- 172.21.15.59:0/735861594 shutdown_connections 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd747e0 0x564edbd495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd742a0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/735861594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 -- 172.21.15.59:0/735861594 >> 172.21.15.59:0/735861594 conn(0x564edbd46a00 msgr2=0x564edbd8f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 -- 172.21.15.59:0/735861594 shutdown_connections 2024-04-24T06:36:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.713+0000 7f6ac27fb740 1 -- 172.21.15.59:0/735861594 wait complete. 2024-04-24T06:36:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.714+0000 7f6ac27fb740 1 -- start start 2024-04-24T06:36:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.714+0000 7f6ac27fb740 1 Processor -- start 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd76fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd9fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564edbd48bb0 con 0x564edbd74f20 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564edbc67690 con 0x564edbd91e00 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.715+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564edbd47fa0 con 0x564edbda2bc0 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6ac27fb740 10 client.0 _ll_register_callbacks cb 0x564edbca3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47516/0 (socket says 172.21.15.59:47516) 2024-04-24T06:36:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 -- 172.21.15.59:0/2677581879 learned_addr learned my addr 172.21.15.59:0/2677581879 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd76fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abda00640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd9fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 -- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 msgr2=0x564edbd9fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd9fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 -- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 msgr2=0x564edbd76fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd76fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.716+0000 7f6abd000640 1 -- 172.21.15.59:0/2677581879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564edbc66920 con 0x564edbd74f20 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6abd000640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6aac08a0f0 tx=0x7f6aac05a190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cbdd2b0d1dc27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6a9e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac01f030 con 0x564edbd74f20 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f6aac071b30 con 0x564edbd74f20 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa4023070 con 0x564edbd74f20 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6aac071d50 con 0x564edbd74f20 2024-04-24T06:36:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6ac27fb740 10 client.8010 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x564edba8a7d0 con 0x564edbd74f20 2024-04-24T06:36:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.717+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4023070 con 0x564edbd74f20 2024-04-24T06:36:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f6aac058da0 con 0x564edbd74f20 2024-04-24T06:36:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 10 client.8010 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 10 client.8010 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 20 client.8010 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd4a560 0x564edbd75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 4 client.8010 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564edbcbbc10 con 0x564edbd4a560 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.718+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd4a560 0x564edbd75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.719+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd4a560 0x564edbd75410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d0508a756e7a6f server_cookie=7a52ddd8437bafcd in_seq=0 out_seq=0 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.719+0000 7f6a9e000640 10 client.8010 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.719+0000 7f6a9e000640 1 -- 172.21.15.59:0/2677581879 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564edbcbbc10 con 0x564edbd4a560 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.719+0000 7f6a9e000640 10 client.8010 handle_command_reply: tid=0 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 client.8010 shutdown 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 msgr2=0x564edbd85f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6aac08a0f0 tx=0x7f6aac05a190 comp rx=0 tx=0).stop 2024-04-24T06:36:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 shutdown_connections 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd4a560 0x564edbd75410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd9fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd91e00 0x564edbd76fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/2677581879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd74f20 0x564edbd85f60 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 >> 172.21.15.59:0/2677581879 conn(0x564edbd747e0 msgr2=0x564edbd926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6a9e000640 0 client.8010 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 shutdown_connections 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 1 -- 172.21.15.59:0/2677581879 wait complete. 2024-04-24T06:36:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.720+0000 7f6ac27fb740 20 client.8010 trim_cache size 0 max 16384 2024-04-24T06:36:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.740+0000 7f6ac27fb740 1 Processor -- start 2024-04-24T06:36:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.740+0000 7f6ac27fb740 1 -- start start 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd74f20 0x564edbdb3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81feb60 con 0x564edbd91e00 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81fdc90 con 0x564edbda2bc0 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab815ea50 con 0x564edbd74f20 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47522/0 (socket says 172.21.15.59:47522) 2024-04-24T06:36:57.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.741+0000 7f6ab7e00640 1 -- 172.21.15.59:0/3880294579 learned_addr learned my addr 172.21.15.59:0/3880294579 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 -- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd74f20 msgr2=0x564edbdb3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd74f20 0x564edbdb3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 -- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbda2bc0 msgr2=0x564edbd926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 -- 172.21.15.59:0/3880294579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564edb9c03f0 con 0x564edbd91e00 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400e010 tx=0x7f6aa4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6245c2b41ace1114 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6aa8c00640 1 -- 172.21.15.59:0/3880294579 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4019070 con 0x564edbd91e00 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6aa8c00640 1 -- 172.21.15.59:0/3880294579 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4023040 con 0x564edbd91e00 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.742+0000 7f6aa8c00640 1 -- 172.21.15.59:0/3880294579 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40352a0 con 0x564edbd91e00 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 -- 172.21.15.59:0/3880294579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 msgr2=0x564edbd483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400e010 tx=0x7f6aa4012000 comp rx=0 tx=0).stop 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 -- 172.21.15.59:0/3880294579 shutdown_connections 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd483d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/3880294579 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbd74f20 0x564edbdb3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 -- 172.21.15.59:0/3880294579 >> 172.21.15.59:0/3880294579 conn(0x564edbd747e0 msgr2=0x564edbd6f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 -- 172.21.15.59:0/3880294579 shutdown_connections 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.743+0000 7f6ac27fb740 1 -- 172.21.15.59:0/3880294579 wait complete. 2024-04-24T06:36:57.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.744+0000 7f6ac27fb740 1 -- start start 2024-04-24T06:36:57.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.744+0000 7f6ac27fb740 1 Processor -- start 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd74f20 0x564edbe0bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564edbd48cd0 con 0x564edbd91e00 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab815f0a0 con 0x564edbd74f20 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564edbd9ff60 con 0x564edbda2bc0 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.745+0000 7f6ac27fb740 10 client.0 _ll_register_callbacks cb 0x7f6aac00a450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47536/0 (socket says 172.21.15.59:47536) 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 -- 172.21.15.59:0/1046362832 learned_addr learned my addr 172.21.15.59:0/1046362832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:36:57.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 -- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 msgr2=0x564edbd926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 -- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd74f20 msgr2=0x564edbe0bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd74f20 0x564edbe0bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 -- 172.21.15.59:0/1046362832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564edb9800f0 con 0x564edbd91e00 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6ab7e00640 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400db90 tx=0x7f6aa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44b3292d8a141be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:57.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.746+0000 7f6a9e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4019040 con 0x564edbd91e00 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f6aa40352b0 con 0x564edbd91e00 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa4023040 con 0x564edbd91e00 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6aa400d730 con 0x564edbd91e00 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 10 client.8016 fetch_fsmap learned FSMap version 68 2024-04-24T06:36:57.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x564edbc73810 con 0x564edbd91e00 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa4023040 con 0x564edbd91e00 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f6aa40354b0 con 0x564edbd91e00 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 10 client.8016 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 10 client.8016 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 20 client.8016 populate_metadata read hostname 'smithi059' 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd92c00 0x564edbd708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 4 client.8016 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.747+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564edbcb5a80 con 0x564edbd92c00 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.748+0000 7f6abd000640 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd92c00 0x564edbd708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.748+0000 7f6abd000640 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd92c00 0x564edbd708b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533b09e5c522ea75 server_cookie=b147e0aa86c0f3d2 in_seq=0 out_seq=0 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.748+0000 7f6a9e000640 10 client.8016 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.748+0000 7f6a9e000640 1 -- 172.21.15.59:0/1046362832 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x564edbcb5a80 con 0x564edbd92c00 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.748+0000 7f6a9e000640 10 client.8016 handle_command_reply: tid=0 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 client.8016 shutdown 2024-04-24T06:36:57.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 msgr2=0x564edbd90b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400db90 tx=0x7f6aa4004730 comp rx=0 tx=0).stop 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 shutdown_connections 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x564edbd92c00 0x564edbd708b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564edbda2bc0 0x564edbd926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564edbd91e00 0x564edbd90b20 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 --2- 172.21.15.59:0/1046362832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564edbd74f20 0x564edbe0bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 >> 172.21.15.59:0/1046362832 conn(0x564edbd747e0 msgr2=0x564edbd6f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6a9e000640 0 client.8016 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 shutdown_connections 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 1 -- 172.21.15.59:0/1046362832 wait complete. 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.749+0000 7f6ac27fb740 20 client.8016 trim_cache size 0 max 16384 2024-04-24T06:36:57.752 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "ee0e53e1-0fc9-4d72-a92f-36fc52f1af64": { 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "ee0e53e1-0fc9-4d72-a92f-36fc52f1af64", 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:36:57.753 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 msgr2=0x7f6a9806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 0x7f6a9806ee00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80cec10 tx=0x7f6aa4008040 comp rx=0 tx=0).stop 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 msgr2=0x7f6ab80cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00c7f0 tx=0x7f6aac00ccb0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 shutdown_connections 2024-04-24T06:36:57.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6a9806c940 0x7f6a9806ee00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6ab815c410 0x7f6ab80b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ab80d0b10 0x7f6ab80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 --2- 172.21.15.59:0/4144889134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ab80af5c0 0x7f6ab80cd4e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 >> 172.21.15.59:0/4144889134 conn(0x7f6ab806bdf0 msgr2=0x7f6ab8156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 shutdown_connections 2024-04-24T06:36:57.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:36:57.750+0000 7f6abe400640 1 -- 172.21.15.59:0/4144889134 wait complete. 2024-04-24T06:36:57.769 INFO:tasks.cephfs.filesystem:scrub status for tag:ee0e53e1-0fc9-4d72-a92f-36fc52f1af64 - {'path': '/dir', 'tag': 'ee0e53e1-0fc9-4d72-a92f-36fc52f1af64', 'options': 'recursive,repair'} 2024-04-24T06:37:07.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:37:07.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:37:27.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:27.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:37:27.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.991+0000 7f307d000640 1 Processor -- start 2024-04-24T06:37:27.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.991+0000 7f307d000640 1 -- start start 2024-04-24T06:37:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.992+0000 7f307d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3078145e10 0x7f307806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f307d000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780cfc20 con 0x7f30780d0cc0 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f307d000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780cff00 con 0x7f3078071d70 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f307d000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f30780d01e0 con 0x7f3078145e10 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f3075e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3078145e10 0x7f307806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f306d400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f30780d0cc0 0x7f30780cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36984/0 (socket says 172.21.15.59:36984) 2024-04-24T06:37:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.993+0000 7f306d400640 1 -- 172.21.15.59:0/2795544703 learned_addr learned my addr 172.21.15.59:0/2795544703 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.994+0000 7f3075400640 1 -- 172.21.15.59:0/2795544703 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2197174752 0 0) 0x7f30780cfc20 con 0x7f30780d0cc0 2024-04-24T06:37:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.994+0000 7f3075400640 1 -- 172.21.15.59:0/2795544703 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f305c003520 con 0x7f30780d0cc0 2024-04-24T06:37:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.994+0000 7f3075e00640 1 -- 172.21.15.59:0/2795544703 >> v1:172.21.15.66:6789/0 conn(0x7f3078071d70 legacy=0x7f307815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.994+0000 7f3075e00640 1 -- 172.21.15.59:0/2795544703 >> v1:172.21.15.59:6789/0 conn(0x7f30780d0cc0 legacy=0x7f30780cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.994+0000 7f3075e00640 1 -- 172.21.15.59:0/2795544703 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780d04c0 con 0x7f3078145e10 2024-04-24T06:37:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.995+0000 7f3075e00640 1 --2- 172.21.15.59:0/2795544703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078145e10 0x7f307806f130 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f30680097d0 tx=0x7f306802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abe3a7112e721ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.995+0000 7f3075400640 1 -- 172.21.15.59:0/2795544703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068037070 con 0x7f3078145e10 2024-04-24T06:37:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.995+0000 7f3075400640 1 -- 172.21.15.59:0/2795544703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068032440 con 0x7f3078145e10 2024-04-24T06:37:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.995+0000 7f3075400640 1 -- 172.21.15.59:0/2795544703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068032810 con 0x7f3078145e10 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 -- 172.21.15.59:0/2795544703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078145e10 msgr2=0x7f307806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 --2- 172.21.15.59:0/2795544703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078145e10 0x7f307806f130 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f30680097d0 tx=0x7f306802a210 comp rx=0 tx=0).stop 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 -- 172.21.15.59:0/2795544703 shutdown_connections 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 --2- 172.21.15.59:0/2795544703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078145e10 0x7f307806f130 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 -- 172.21.15.59:0/2795544703 >> 172.21.15.59:0/2795544703 conn(0x7f3078067d90 msgr2=0x7f307806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 -- 172.21.15.59:0/2795544703 shutdown_connections 2024-04-24T06:37:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.996+0000 7f307d000640 1 -- 172.21.15.59:0/2795544703 wait complete. 2024-04-24T06:37:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.998+0000 7f307d000640 1 Processor -- start 2024-04-24T06:37:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.998+0000 7f307d000640 1 -- start start 2024-04-24T06:37:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30780d0cc0 0x7f3078134be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3078145e10 0x7f30781e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30781461f0 con 0x7f3078071d70 2024-04-24T06:37:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3078072200 con 0x7f30780d0cc0 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f307d000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f30781f53d0 con 0x7f3078145e10 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49312/0 (socket says 172.21.15.59:49312) 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:27.999+0000 7f3075e00640 1 -- 172.21.15.59:0/2096139243 learned_addr learned my addr 172.21.15.59:0/2096139243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 -- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3078145e10 msgr2=0x7f30781e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3078145e10 0x7f30781e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 -- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30780d0cc0 msgr2=0x7f3078134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30780d0cc0 0x7f3078134be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 -- 172.21.15.59:0/2096139243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781e5190 con 0x7f3078071d70 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f3075e00640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f30680098f0 tx=0x7f30680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f306ea00640 1 -- 172.21.15.59:0/2096139243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068041020 con 0x7f3078071d70 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f306ea00640 1 -- 172.21.15.59:0/2096139243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306803d6e0 con 0x7f3078071d70 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f306ea00640 1 -- 172.21.15.59:0/2096139243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068037040 con 0x7f3078071d70 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781e5470 con 0x7f3078071d70 2024-04-24T06:37:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.000+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3078135120 con 0x7f3078071d70 2024-04-24T06:37:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.003+0000 7f306ea00640 1 -- 172.21.15.59:0/2096139243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3068044020 con 0x7f3078071d70 2024-04-24T06:37:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.004+0000 7f306ea00640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.004+0000 7f306ea00640 1 -- 172.21.15.59:0/2096139243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f30680ac3f0 con 0x7f3078071d70 2024-04-24T06:37:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.004+0000 7f306d400640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.004+0000 7f307fe35740 1 Processor -- start 2024-04-24T06:37:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.005+0000 7f307fe35740 1 -- start start 2024-04-24T06:37:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.005+0000 7f306d400640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 0x7f305406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f60 tx=0x7f30600078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed581050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58105610 0x55ed580da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed5803f1f0 con 0x55ed58122c30 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ed57fe5350 con 0x55ed58105610 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ed5803b1b0 con 0x55ed58105d50 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49326/0 (socket says 172.21.15.59:49326) 2024-04-24T06:37:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 -- 172.21.15.59:0/3332937905 learned_addr learned my addr 172.21.15.59:0/3332937905 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 -- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 msgr2=0x55ed581050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed581050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 -- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58105610 msgr2=0x55ed580da420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58105610 0x55ed580da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.006+0000 7f3075e00640 1 -- 172.21.15.59:0/3332937905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed58039f50 con 0x55ed58122c30 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.007+0000 7f3075e00640 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f30680826e0 tx=0x7f3068079f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7537fd8708ed1e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.007+0000 7f3066a00640 1 -- 172.21.15.59:0/3332937905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306807e3e0 con 0x55ed58122c30 2024-04-24T06:37:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.007+0000 7f3066a00640 1 -- 172.21.15.59:0/3332937905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068082c10 con 0x55ed58122c30 2024-04-24T06:37:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.007+0000 7f3066a00640 1 -- 172.21.15.59:0/3332937905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306807e0d0 con 0x55ed58122c30 2024-04-24T06:37:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 -- 172.21.15.59:0/3332937905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 msgr2=0x55ed580d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f30680826e0 tx=0x7f3068079f30 comp rx=0 tx=0).stop 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 -- 172.21.15.59:0/3332937905 shutdown_connections 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58105610 0x55ed580da420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed581050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 --2- 172.21.15.59:0/3332937905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d55c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 -- 172.21.15.59:0/3332937905 >> 172.21.15.59:0/3332937905 conn(0x55ed580d7830 msgr2=0x55ed581200d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 -- 172.21.15.59:0/3332937905 shutdown_connections 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.008+0000 7f307fe35740 1 -- 172.21.15.59:0/3332937905 wait complete. 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.009+0000 7f307fe35740 1 -- start start 2024-04-24T06:37:28.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.009+0000 7f307fe35740 1 Processor -- start 2024-04-24T06:37:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58122c30 0x55ed58118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed58116d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed580d99e0 con 0x55ed58105d50 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ed57ff8690 con 0x55ed580f3330 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ed580d8dd0 con 0x55ed58122c30 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f307fe35740 10 client.0 _ll_register_callbacks cb 0x55ed58034650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49338/0 (socket says 172.21.15.59:49338) 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 -- 172.21.15.59:0/900353907 learned_addr learned my addr 172.21.15.59:0/900353907 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 -- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58122c30 msgr2=0x55ed58118ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58122c30 0x55ed58118ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 -- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 msgr2=0x55ed58116d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed58116d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.010+0000 7f3075e00640 1 -- 172.21.15.59:0/900353907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed57ff7920 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f3075e00640 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3068079d30 tx=0x7f3068007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5688b0d9f6040c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f305b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068078150 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f30680766a0 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3068082c10 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f30680aa3b0 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f307fe35740 10 client.8031 fetch_fsmap learned FSMap version 68 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55ed57e1b7d0 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.011+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068082c10 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f306803a030 con 0x55ed58105d50 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 10 client.8031 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 10 client.8031 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 20 client.8031 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed580da0b0 0x55ed580da510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 4 client.8031 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:37:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed5804cc10 con 0x55ed580da0b0 2024-04-24T06:37:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.012+0000 7f306d400640 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed580da0b0 0x55ed580da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.013+0000 7f306d400640 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed580da0b0 0x55ed580da510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a841de5e57e88c server_cookie=e1f6612064ef2d2a in_seq=0 out_seq=0 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.013+0000 7f305b400640 10 client.8031 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.013+0000 7f305b400640 1 -- 172.21.15.59:0/900353907 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed5804cc10 con 0x55ed580da0b0 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.013+0000 7f305b400640 10 client.8031 handle_command_reply: tid=0 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.013+0000 7f307fe35740 1 client.8031 shutdown 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 msgr2=0x55ed580f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3068079d30 tx=0x7f3068007000 comp rx=0 tx=0).stop 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 shutdown_connections 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed580da0b0 0x55ed580da510 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed58116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58122c30 0x55ed58118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 --2- 172.21.15.59:0/900353907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed580f29e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f305b400640 0 client.8031 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 >> 172.21.15.59:0/900353907 conn(0x55ed58105610 msgr2=0x55ed581200d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 shutdown_connections 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 1 -- 172.21.15.59:0/900353907 wait complete. 2024-04-24T06:37:28.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.014+0000 7f307fe35740 20 client.8031 trim_cache size 0 max 16384 2024-04-24T06:37:28.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.034+0000 7f307fe35740 1 Processor -- start 2024-04-24T06:37:28.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.034+0000 7f307fe35740 1 -- start start 2024-04-24T06:37:28.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed580d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed58108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3078201180 con 0x55ed58122c30 2024-04-24T06:37:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30782001c0 con 0x55ed580f3330 2024-04-24T06:37:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f30781f89b0 con 0x55ed58105d50 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f3076800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f3076800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49348/0 (socket says 172.21.15.59:49348) 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f3076800640 1 -- 172.21.15.59:0/3600542905 learned_addr learned my addr 172.21.15.59:0/3600542905 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f306d400640 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed58108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f3075e00640 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed580d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.035+0000 7f3076800640 1 -- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 msgr2=0x55ed58108d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3076800640 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed58108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3076800640 1 -- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 msgr2=0x55ed580d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3076800640 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed580d5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3076800640 1 -- 172.21.15.59:0/3600542905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed57d513f0 con 0x55ed58122c30 2024-04-24T06:37:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3076800640 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3070004330 tx=0x7f30700047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5e74e4f5205c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3066000640 1 -- 172.21.15.59:0/3600542905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000caf0 con 0x55ed58122c30 2024-04-24T06:37:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3066000640 1 -- 172.21.15.59:0/3600542905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3070017070 con 0x55ed58122c30 2024-04-24T06:37:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.036+0000 7f3066000640 1 -- 172.21.15.59:0/3600542905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070012420 con 0x55ed58122c30 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 -- 172.21.15.59:0/3600542905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 msgr2=0x55ed580d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3070004330 tx=0x7f30700047f0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 -- 172.21.15.59:0/3600542905 shutdown_connections 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58122c30 0x55ed580d9140 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed58105d50 0x55ed58108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 --2- 172.21.15.59:0/3600542905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed580f3330 0x55ed580d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 -- 172.21.15.59:0/3600542905 >> 172.21.15.59:0/3600542905 conn(0x55ed58105610 msgr2=0x55ed581005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 -- 172.21.15.59:0/3600542905 shutdown_connections 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.037+0000 7f307fe35740 1 -- 172.21.15.59:0/3600542905 wait complete. 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.038+0000 7f307fe35740 1 -- start start 2024-04-24T06:37:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.038+0000 7f307fe35740 1 Processor -- start 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed580f3330 0x55ed58198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58122c30 0x55ed58132190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed5812ff50 con 0x55ed58105d50 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30780af360 con 0x55ed58122c30 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ed580d00b0 con 0x55ed580f3330 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f307fe35740 10 client.0 _ll_register_callbacks cb 0x7f3068092070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f306d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f306d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49352/0 (socket says 172.21.15.59:49352) 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f306d400640 1 -- 172.21.15.59:0/1021098586 learned_addr learned my addr 172.21.15.59:0/1021098586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.039+0000 7f3075e00640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed580f3330 0x55ed58198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f3076800640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58122c30 0x55ed58132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 -- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed580f3330 msgr2=0x55ed58198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed580f3330 0x55ed58198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 -- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58122c30 msgr2=0x55ed58132190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58122c30 0x55ed58132190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 -- 172.21.15.59:0/1021098586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed57d110f0 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f306d400640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f307815f590 tx=0x7f3060007300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d56e1a6b0977816 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f305b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060026060 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3070017070 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.040+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f30600207c0 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f306002c050 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 10 client.8037 fetch_fsmap learned FSMap version 68 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55ed58004810 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3070017070 con 0x55ed58105d50 2024-04-24T06:37:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f306002f020 con 0x55ed58105d50 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 10 client.8037 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 10 client.8037 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 20 client.8037 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed581326d0 0x55ed5819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 4 client.8037 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:37:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed58046a80 con 0x55ed581326d0 2024-04-24T06:37:28.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.041+0000 7f3075e00640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed581326d0 0x55ed5819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f3075e00640 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed581326d0 0x55ed5819ce40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f586b65ce85a5e2 server_cookie=e321a0f55d19a0ef in_seq=0 out_seq=0 2024-04-24T06:37:28.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f305b400640 10 client.8037 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f305b400640 1 -- 172.21.15.59:0/1021098586 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed58046a80 con 0x55ed581326d0 2024-04-24T06:37:28.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f305b400640 10 client.8037 handle_command_reply: tid=0 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 client.8037 shutdown 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 msgr2=0x55ed58143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f307815f590 tx=0x7f3060007300 comp rx=0 tx=0).stop 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 shutdown_connections 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55ed581326d0 0x55ed5819ce40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.042+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ed58122c30 0x55ed58132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed58105d50 0x55ed58143400 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 1 --2- 172.21.15.59:0/1021098586 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ed580f3330 0x55ed58198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 >> 172.21.15.59:0/1021098586 conn(0x55ed58105610 msgr2=0x55ed580d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f305b400640 0 client.8037 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 shutdown_connections 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 1 -- 172.21.15.59:0/1021098586 wait complete. 2024-04-24T06:37:28.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307fe35740 20 client.8037 trim_cache size 0 max 16384 2024-04-24T06:37:28.048 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:37:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:37:28.048 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:37:28.048 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 msgr2=0x7f305406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 0x7f305406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f60 tx=0x7f30600078f0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 msgr2=0x7f30781e5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.043+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f30680098f0 tx=0x7f30680046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 shutdown_connections 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f305406c8f0 0x7f305406edb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3078145e10 0x7f30781e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30780d0cc0 0x7f3078134be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 --2- 172.21.15.59:0/2096139243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3078071d70 0x7f30781e5c50 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 >> 172.21.15.59:0/2096139243 conn(0x7f3078067d90 msgr2=0x7f3078144400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 shutdown_connections 2024-04-24T06:37:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.044+0000 7f307d000640 1 -- 172.21.15.59:0/2096139243 wait complete. 2024-04-24T06:37:28.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:37:28.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:37:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.270+0000 7fcbdfa00640 1 Processor -- start 2024-04-24T06:37:28.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.270+0000 7fcbdfa00640 1 -- start start 2024-04-24T06:37:28.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbdfa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcbd80b4550 0x7fcbd8163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbdfa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd8160d40 con 0x7fcbd81644b0 2024-04-24T06:37:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbdfa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd8166de0 con 0x7fcbd8160960 2024-04-24T06:37:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbdfa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd8164270 con 0x7fcbd80b4550 2024-04-24T06:37:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbd7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcbd81644b0 0x7fcbd806e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36994/0 (socket says 172.21.15.59:36994) 2024-04-24T06:37:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbd7e00640 1 -- 172.21.15.59:0/42022436 learned_addr learned my addr 172.21.15.59:0/42022436 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbdda00640 1 --2- 172.21.15.59:0/42022436 >> v2:172.21.15.66:3300/0 conn(0x7fcbd80b4550 0x7fcbd8163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.272+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 566324218 0 0) 0x7fcbd8160d40 con 0x7fcbd81644b0 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbbc003520 con 0x7fcbd81644b0 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1394355576 0 0) 0x7fcbbc003520 con 0x7fcbd81644b0 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 >> v2:172.21.15.66:3300/0 conn(0x7fcbd80b4550 msgr2=0x7fcbd8163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 --2- 172.21.15.59:0/42022436 >> v2:172.21.15.66:3300/0 conn(0x7fcbd80b4550 0x7fcbd8163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 >> v1:172.21.15.66:6789/0 conn(0x7fcbd8160960 legacy=0x7fcbd80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd81670c0 con 0x7fcbd81644b0 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fcbc4001690 con 0x7fcbd81644b0 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 >> v1:172.21.15.59:6789/0 conn(0x7fcbd81644b0 legacy=0x7fcbd806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbbc004800 0x7fcbbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbbc005210 0x7fcbbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8164270 con 0x7fcbbc003a00 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc00c160 con 0x7fcbbc004800 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc00c360 con 0x7fcbbc005210 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.273+0000 7fcbdd000640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbd7e00640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbbc004800 0x7fcbbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdda00640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbbc005210 0x7fcbbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdd000640 1 -- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbbc005210 msgr2=0x7fcbbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdd000640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbbc005210 0x7fcbbc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdd000640 1 -- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbbc004800 msgr2=0x7fcbbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdd000640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbbc004800 0x7fcbbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdd000640 1 -- 172.21.15.59:0/42022436 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc003520 con 0x7fcbbc003a00 2024-04-24T06:37:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.274+0000 7fcbdda00640 1 -- 172.21.15.59:0/42022436 reap_dead start 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.275+0000 7fcbdd000640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbbc003e20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001e50 tx=0x7fcbbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8ce7a04f4473a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.275+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8001a70 con 0x7fcbbc003a00 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.275+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc802ae40 con 0x7fcbbc003a00 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.275+0000 7fcbd7400640 1 -- 172.21.15.59:0/42022436 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8003bb0 con 0x7fcbbc003a00 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 -- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 msgr2=0x7fcbbc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbbc003e20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001e50 tx=0x7fcbbc005830 comp rx=0 tx=0).stop 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 -- 172.21.15.59:0/42022436 shutdown_connections 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/42022436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbbc003e20 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 -- 172.21.15.59:0/42022436 >> 172.21.15.59:0/42022436 conn(0x7fcbd8067ea0 msgr2=0x7fcbd806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 -- 172.21.15.59:0/42022436 shutdown_connections 2024-04-24T06:37:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.276+0000 7fcbdfa00640 1 -- 172.21.15.59:0/42022436 wait complete. 2024-04-24T06:37:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.278+0000 7fcbdfa00640 1 Processor -- start 2024-04-24T06:37:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.278+0000 7fcbdfa00640 1 -- start start 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbd8151190 0x7fcbd81515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbd80d0780 0x7fcbd80d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81668e0 con 0x7fcbbc003a00 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8167680 con 0x7fcbd8151190 2024-04-24T06:37:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.279+0000 7fcbdfa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd8166a60 con 0x7fcbd80d0780 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49366/0 (socket says 172.21.15.59:49366) 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 -- 172.21.15.59:0/1154476144 learned_addr learned my addr 172.21.15.59:0/1154476144 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdda00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbd80d0780 0x7fcbd80d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbd7e00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbd8151190 0x7fcbd81515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 -- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbd80d0780 msgr2=0x7fcbd80d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbd80d0780 0x7fcbd80d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 -- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbd8151190 msgr2=0x7fcbd81515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbd8151190 0x7fcbd81515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.280+0000 7fcbdd000640 1 -- 172.21.15.59:0/1154476144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd81f9bc0 con 0x7fcbbc003a00 2024-04-24T06:37:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbdd000640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005040 tx=0x7fcbc80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8037040 con 0x7fcbbc003a00 2024-04-24T06:37:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc8041070 con 0x7fcbbc003a00 2024-04-24T06:37:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd81f9e60 con 0x7fcbbc003a00 2024-04-24T06:37:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8007db0 con 0x7fcbbc003a00 2024-04-24T06:37:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.281+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81fa420 con 0x7fcbbc003a00 2024-04-24T06:37:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.282+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd8068a80 con 0x7fcbbc003a00 2024-04-24T06:37:28.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.283+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbc8006b50 con 0x7fcbbc003a00 2024-04-24T06:37:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.284+0000 7fcbd4c00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 0x7fcbb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.284+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fcbc80abc40 con 0x7fcbbc003a00 2024-04-24T06:37:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.285+0000 7fcbd7e00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 0x7fcbb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.286+0000 7fcbd7e00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 0x7fcbb806ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcbd81fcf80 tx=0x7fcbc4007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.289+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbc80799c0 con 0x7fcbbc003a00 2024-04-24T06:37:28.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.534+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbd8151b30 con 0x7fcbbc003a00 2024-04-24T06:37:28.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.535+0000 7fcbd4c00640 1 -- 172.21.15.59:0/1154476144 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5182 (secure 0 0 0) 0x7fcbc807dcc0 con 0x7fcbbc003a00 2024-04-24T06:37:28.537 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:28.537 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:31.666787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7548,"qdb_cluster":[7548]},"id":6}]} 2024-04-24T06:37:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 68 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 msgr2=0x7fcbb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 0x7fcbb806ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcbd81fcf80 tx=0x7fcbc4007040 comp rx=0 tx=0).stop 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 msgr2=0x7fcbd8162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8005040 tx=0x7fcbc80014a0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 shutdown_connections 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcbb806c920 0x7fcbb806ede0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcbd80d0780 0x7fcbd80d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbd8151190 0x7fcbd81515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 --2- 172.21.15.59:0/1154476144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbbc003a00 0x7fcbd8162000 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.538+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 >> 172.21.15.59:0/1154476144 conn(0x7fcbd8067ea0 msgr2=0x7fcbd815e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.539+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 shutdown_connections 2024-04-24T06:37:28.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.539+0000 7fcbdfa00640 1 -- 172.21.15.59:0/1154476144 wait complete. 2024-04-24T06:37:28.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-24T06:37:28.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.775+0000 7f97aac00640 1 Processor -- start 2024-04-24T06:37:28.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.776+0000 7f97aac00640 1 -- start start 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97aac00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97a40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97aac00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f97a4163770 0x7f97a40d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97aac00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a40d01c0 con 0x7f97a4160180 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97aac00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f97a415cf00 con 0x7f97a4163770 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97aac00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97a415f6a0 con 0x7f97a415cb20 2024-04-24T06:37:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97a8c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f97a4160180 0x7f97a40b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36998/0 (socket says 172.21.15.59:36998) 2024-04-24T06:37:28.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.777+0000 7f97a8c00640 1 -- 172.21.15.59:0/3753462358 learned_addr learned my addr 172.21.15.59:0/3753462358 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a3e00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97a40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a3400640 1 --2- 172.21.15.59:0/3753462358 >> v2:172.21.15.66:3300/0 conn(0x7f97a4163770 0x7f97a40d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331935194 0 0) 0x7f97a40d01c0 con 0x7f97a4160180 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9788003520 con 0x7f97a4160180 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 784252763 0 0) 0x7f9788003520 con 0x7f97a4160180 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 msgr2=0x7f97a40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97a40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 >> v2:172.21.15.66:3300/0 conn(0x7f97a4163770 msgr2=0x7f97a40d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 --2- 172.21.15.59:0/3753462358 >> v2:172.21.15.66:3300/0 conn(0x7f97a4163770 0x7f97a40d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a415f980 con 0x7f97a4160180 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f979c0019f0 con 0x7f97a4160180 2024-04-24T06:37:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 >> v1:172.21.15.59:6789/0 conn(0x7f97a4160180 legacy=0x7f97a40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.778+0000 7f97a2a00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a2a00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9788003f30 0x7f9788004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a2a00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9788004d40 0x7f97880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97a415cf00 con 0x7f9788003f30 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97a415f6a0 con 0x7f9788004d40 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f978800ba10 con 0x7f97a415cb20 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9788003f30 0x7f9788004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3e00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a8c00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9788004d40 0x7f97880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 -- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 msgr2=0x7f97880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97a415cb20 0x7f97880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 -- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9788004d40 msgr2=0x7f97880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9788004d40 0x7f97880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.779+0000 7f97a3400640 1 -- 172.21.15.59:0/3753462358 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9788003520 con 0x7f9788003f30 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.780+0000 7f97a3400640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9788003f30 0x7f9788004800 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9790009af0 tx=0x7f9788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8eaf90231719e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790036070 con 0x7f9788003f30 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790031400 con 0x7f9788003f30 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97a2a00640 1 -- 172.21.15.59:0/3753462358 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790009690 con 0x7f9788003f30 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97aac00640 1 -- 172.21.15.59:0/3753462358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9788003f30 msgr2=0x7f9788004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97aac00640 1 --2- 172.21.15.59:0/3753462358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9788003f30 0x7f9788004800 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9790009af0 tx=0x7f9788005830 comp rx=0 tx=0).stop 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97a8c00640 1 -- 172.21.15.59:0/3753462358 reap_dead start 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97aac00640 1 -- 172.21.15.59:0/3753462358 shutdown_connections 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.781+0000 7f97aac00640 1 -- 172.21.15.59:0/3753462358 >> 172.21.15.59:0/3753462358 conn(0x7f97a406cc70 msgr2=0x7f97a406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.782+0000 7f97aac00640 1 -- 172.21.15.59:0/3753462358 shutdown_connections 2024-04-24T06:37:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.782+0000 7f97aac00640 1 -- 172.21.15.59:0/3753462358 wait complete. 2024-04-24T06:37:28.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.783+0000 7f97aac00640 1 Processor -- start 2024-04-24T06:37:28.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.784+0000 7f97aac00640 1 -- start start 2024-04-24T06:37:28.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.784+0000 7f97aac00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9788003f30 0x7f97a40ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.784+0000 7f97aac00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a40c7710 0x7f97a4149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.784+0000 7f97aac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97aac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40d1b10 con 0x7f97a41499c0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97aac00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40d1990 con 0x7f97a40c7710 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97aac00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97a40cf840 con 0x7f9788003f30 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49378/0 (socket says 172.21.15.59:49378) 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 -- 172.21.15.59:0/63220472 learned_addr learned my addr 172.21.15.59:0/63220472 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a3e00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9788003f30 0x7f97a40ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a3400640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a40c7710 0x7f97a4149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 -- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9788003f30 msgr2=0x7f97a40ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9788003f30 0x7f97a40ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 -- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a40c7710 msgr2=0x7f97a4149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a40c7710 0x7f97a4149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.785+0000 7f97a8c00640 1 -- 172.21.15.59:0/63220472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a40cbd50 con 0x7f97a41499c0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f97a8c00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f979c000f80 tx=0x7f979c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f979be00640 1 -- 172.21.15.59:0/63220472 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c00ec10 con 0x7f97a41499c0 2024-04-24T06:37:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f979be00640 1 -- 172.21.15.59:0/63220472 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c00edb0 con 0x7f97a41499c0 2024-04-24T06:37:28.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f979be00640 1 -- 172.21.15.59:0/63220472 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0146c0 con 0x7f97a41499c0 2024-04-24T06:37:28.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a40c9e60 con 0x7f97a41499c0 2024-04-24T06:37:28.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.786+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a40ca420 con 0x7f97a41499c0 2024-04-24T06:37:28.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.789+0000 7f979be00640 1 -- 172.21.15.59:0/63220472 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f979c01b060 con 0x7f97a41499c0 2024-04-24T06:37:28.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.790+0000 7f979be00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 0x7f978406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.790+0000 7f979be00640 1 -- 172.21.15.59:0/63220472 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f979c08b380 con 0x7f97a41499c0 2024-04-24T06:37:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.791+0000 7f97a3e00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 0x7f978406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.791+0000 7f97adba3740 1 Processor -- start 2024-04-24T06:37:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.791+0000 7f97adba3740 1 -- start start 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.791+0000 7f97a3e00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 0x7f978406edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9794009930 tx=0x7f9794008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e927ff0 0x55a54e9283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 0x55a54e943f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a54e862190 con 0x55a54e945890 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a54e808030 con 0x55a54e8fdcd0 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a54e85e150 con 0x55a54e927ff0 2024-04-24T06:37:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49386/0 (socket says 172.21.15.59:49386) 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3400640 1 -- 172.21.15.59:0/2088991583 learned_addr learned my addr 172.21.15.59:0/2088991583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3e00640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e927ff0 0x55a54e9283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a8c00640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 0x55a54e943f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3400640 1 -- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e927ff0 msgr2=0x55a54e9283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.792+0000 7f97a3400640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e927ff0 0x55a54e9283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f97a3400640 1 -- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 msgr2=0x55a54e943f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f97a3400640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 0x55a54e943f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f97a3400640 1 -- 172.21.15.59:0/2088991583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e7f0b60 con 0x55a54e945890 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f97a8c00640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 0x55a54e943f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f97a3400640 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9790034040 tx=0x7f979002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a9b486c272b3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f978fe00640 1 -- 172.21.15.59:0/2088991583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790040030 con 0x55a54e945890 2024-04-24T06:37:28.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f978fe00640 1 -- 172.21.15.59:0/2088991583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790005990 con 0x55a54e945890 2024-04-24T06:37:28.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.793+0000 7f978fe00640 1 -- 172.21.15.59:0/2088991583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790036040 con 0x55a54e945890 2024-04-24T06:37:28.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 -- 172.21.15.59:0/2088991583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 msgr2=0x55a54e945de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9790034040 tx=0x7f979002e000 comp rx=0 tx=0).stop 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 -- 172.21.15.59:0/2088991583 shutdown_connections 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e8fdcd0 0x55a54e943f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e945890 0x55a54e945de0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 --2- 172.21.15.59:0/2088991583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e927ff0 0x55a54e9283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 -- 172.21.15.59:0/2088991583 >> 172.21.15.59:0/2088991583 conn(0x55a54e8fa600 msgr2=0x55a54e942e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 -- 172.21.15.59:0/2088991583 shutdown_connections 2024-04-24T06:37:28.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.794+0000 7f97adba3740 1 -- 172.21.15.59:0/2088991583 wait complete. 2024-04-24T06:37:28.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.795+0000 7f97adba3740 1 -- start start 2024-04-24T06:37:28.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.795+0000 7f97adba3740 1 Processor -- start 2024-04-24T06:37:28.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e93c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e92b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a54e8fea70 con 0x55a54e927ff0 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a54e81b500 con 0x55a54e945890 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a54e8f8530 con 0x55a54e919c40 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.796+0000 7f97adba3740 10 client.0 _ll_register_callbacks cb 0x55a54e858270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49388/0 (socket says 172.21.15.59:49388) 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 -- 172.21.15.59:0/4129210649 learned_addr learned my addr 172.21.15.59:0/4129210649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a8c00640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e92b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3400640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e93c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 -- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 msgr2=0x55a54e92b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e92b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 -- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 msgr2=0x55a54e93c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e93c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3e00640 1 -- 172.21.15.59:0/4129210649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54e6407c0 con 0x55a54e927ff0 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a3400640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e93c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.797+0000 7f97a8c00640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e92b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9794013f00 tx=0x7f9794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5345aaf8b8a232fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f978cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794028070 con 0x55a54e927ff0 2024-04-24T06:37:28.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f9794024830 con 0x55a54e927ff0 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9790005990 con 0x55a54e927ff0 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9794033050 con 0x55a54e927ff0 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f97adba3740 10 client.8055 fetch_fsmap learned FSMap version 68 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.798+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a54e6a0f50 con 0x55a54e927ff0 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790005990 con 0x55a54e927ff0 2024-04-24T06:37:28.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f9794036020 con 0x55a54e927ff0 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 10 client.8055 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 10 client.8055 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 20 client.8055 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e956c10 0x55a54e957030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 4 client.8055 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a54e86fbb0 con 0x55a54e956c10 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.799+0000 7f97a3400640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e956c10 0x55a54e957030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.800+0000 7f97a3400640 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e956c10 0x55a54e957030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab3b89def6f045f server_cookie=e5a67dd6948aee7 in_seq=0 out_seq=0 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.800+0000 7f978cc00640 10 client.8055 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.800+0000 7f978cc00640 1 -- 172.21.15.59:0/4129210649 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a54e86fbb0 con 0x55a54e956c10 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.800+0000 7f978cc00640 10 client.8055 handle_command_reply: tid=0 2024-04-24T06:37:28.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.800+0000 7f97adba3740 1 client.8055 shutdown 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 msgr2=0x55a54e939eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9794013f00 tx=0x7f9794004730 comp rx=0 tx=0).stop 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 shutdown_connections 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e956c10 0x55a54e957030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e92b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e93c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 --2- 172.21.15.59:0/4129210649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e939eb0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f978cc00640 0 client.8055 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 >> 172.21.15.59:0/4129210649 conn(0x55a54e8fdcd0 msgr2=0x55a54e946380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 shutdown_connections 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 1 -- 172.21.15.59:0/4129210649 wait complete. 2024-04-24T06:37:28.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.801+0000 7f97adba3740 20 client.8055 trim_cache size 0 max 16384 2024-04-24T06:37:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.816+0000 7f97adba3740 1 Processor -- start 2024-04-24T06:37:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.816+0000 7f97adba3740 1 -- start start 2024-04-24T06:37:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e8fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e954e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97a42006e0 con 0x55a54e927ff0 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40ce440 con 0x55a54e945890 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97a40d3680 con 0x55a54e919c40 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49396/0 (socket says 172.21.15.59:49396) 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 -- 172.21.15.59:0/3825835301 learned_addr learned my addr 172.21.15.59:0/3825835301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 -- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 msgr2=0x55a54e8fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e8fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 -- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 msgr2=0x55a54e954e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.817+0000 7f97a3400640 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e954e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.818+0000 7f97a3400640 1 -- 172.21.15.59:0/3825835301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e5340b0 con 0x55a54e927ff0 2024-04-24T06:37:28.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.818+0000 7f97a3400640 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f9790009be0 tx=0x7f979002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f827d5ec24f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.818+0000 7f978f400640 1 -- 172.21.15.59:0/3825835301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790036070 con 0x55a54e927ff0 2024-04-24T06:37:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.818+0000 7f978f400640 1 -- 172.21.15.59:0/3825835301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790004470 con 0x55a54e927ff0 2024-04-24T06:37:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.818+0000 7f978f400640 1 -- 172.21.15.59:0/3825835301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790040040 con 0x55a54e927ff0 2024-04-24T06:37:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- 172.21.15.59:0/3825835301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 msgr2=0x55a54e8fe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f9790009be0 tx=0x7f979002f000 comp rx=0 tx=0).stop 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- 172.21.15.59:0/3825835301 shutdown_connections 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e945890 0x55a54e954e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e927ff0 0x55a54e8fe5f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 --2- 172.21.15.59:0/3825835301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e919c40 0x55a54e8fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- 172.21.15.59:0/3825835301 >> 172.21.15.59:0/3825835301 conn(0x55a54e8fdcd0 msgr2=0x55a54e8f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- 172.21.15.59:0/3825835301 shutdown_connections 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- 172.21.15.59:0/3825835301 wait complete. 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.819+0000 7f97adba3740 1 -- start start 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.820+0000 7f97adba3740 1 Processor -- start 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e927ff0 0x55a54e943fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e945890 0x55a54e92a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a54e929f90 con 0x55a54e919c40 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a54e93b060 con 0x55a54e927ff0 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a54e929840 con 0x55a54e945890 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97adba3740 10 client.0 _ll_register_callbacks cb 0x7f9794006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49410/0 (socket says 172.21.15.59:49410) 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 -- 172.21.15.59:0/4244836322 learned_addr learned my addr 172.21.15.59:0/4244836322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 -- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e945890 msgr2=0x55a54e92a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e945890 0x55a54e92a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 -- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e927ff0 msgr2=0x55a54e943fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3400640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e927ff0 0x55a54e943fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e927ff0 0x55a54e943fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.821+0000 7f97a3e00640 1 -- 172.21.15.59:0/4244836322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54e8ff8d0 con 0x55a54e919c40 2024-04-24T06:37:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.822+0000 7f97a3e00640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9794000c00 tx=0x7f9794009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea250510a04eaa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:28.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.822+0000 7f978cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.822+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979403b030 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.822+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f979402d440 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.822+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9790004470 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9794028050 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 10 client.8061 fetch_fsmap learned FSMap version 68 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a54e926f80 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790004470 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f979404a020 con 0x55a54e919c40 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 10 client.8061 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 10 client.8061 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:37:28.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 20 client.8061 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e92af40 0x55a54e938790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 4 client.8061 mds_command: new command op to 7548 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 --> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55a54e869a20 con 0x55a54e92af40 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.823+0000 7f97a3400640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e92af40 0x55a54e938790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f97a3400640 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e92af40 0x55a54e938790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8888868fd454ab92 server_cookie=8b4c0b152da94d7f in_seq=0 out_seq=0 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f978cc00640 10 client.8061 ms_handle_connect on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f978cc00640 1 -- 172.21.15.59:0/4244836322 <== mds.0 v2:172.21.15.59:6834/2792490076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55a54e869a20 con 0x55a54e92af40 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f978cc00640 10 client.8061 handle_command_reply: tid=0 2024-04-24T06:37:28.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f97adba3740 1 client.8061 shutdown 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 msgr2=0x55a54e9bcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.824+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9794000c00 tx=0x7f9794009190 comp rx=0 tx=0).stop 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 shutdown_connections 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:6834/2792490076,v1:172.21.15.59:6835/2792490076] conn(0x55a54e92af40 0x55a54e938790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a54e945890 0x55a54e92a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a54e927ff0 0x55a54e943fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 --2- 172.21.15.59:0/4244836322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54e919c40 0x55a54e9bcab0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f978cc00640 0 client.8061 ms_handle_reset on v2:172.21.15.59:6834/2792490076 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 >> 172.21.15.59:0/4244836322 conn(0x55a54e8fdcd0 msgr2=0x55a54e941f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 shutdown_connections 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 1 -- 172.21.15.59:0/4244836322 wait complete. 2024-04-24T06:37:28.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.825+0000 7f97adba3740 20 client.8061 trim_cache size 0 max 16384 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4293049403, 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/testdir/hardlink" 2024-04-24T06:37:28.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:37:28.833 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:37:28.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 msgr2=0x7f978406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 0x7f978406edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9794009930 tx=0x7f9794008040 comp rx=0 tx=0).stop 2024-04-24T06:37:28.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 msgr2=0x7f97a40cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f979c000f80 tx=0x7f979c00e530 comp rx=0 tx=0).stop 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 shutdown_connections 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f978406c8f0 0x7f978406edb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a41499c0 0x7f97a40cd570 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a40c7710 0x7f97a4149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 --2- 172.21.15.59:0/63220472 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9788003f30 0x7f97a40ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 >> 172.21.15.59:0/63220472 conn(0x7f97a406cc70 msgr2=0x7f97a415e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 shutdown_connections 2024-04-24T06:37:28.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:28.826+0000 7f97aac00640 1 -- 172.21.15.59:0/63220472 wait complete. 2024-04-24T06:37:28.845 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:37:28.845 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:37:28.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:37:29.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.141+0000 7fa947200640 1 Processor -- start 2024-04-24T06:37:29.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.142+0000 7fa947200640 1 -- start start 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.143+0000 7fa947200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 0x7fa9480b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.143+0000 7fa947200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9480b4690 0x7fa9480cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.143+0000 7fa947200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa948160210 con 0x7fa94815c3d0 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.143+0000 7fa947200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa9481604f0 con 0x7fa9480b4690 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.143+0000 7fa947200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa9481607d0 con 0x7fa9480cd760 2024-04-24T06:37:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.144+0000 7fa945400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 0x7fa9480b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.144+0000 7fa946800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9480b4690 0x7fa9480cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.144+0000 7fa945e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7fa94815c3d0 0x7fa9480cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:59360/0 (socket says 172.21.15.59:59360) 2024-04-24T06:37:29.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.144+0000 7fa945e00640 1 -- 172.21.15.59:0/3367682846 learned_addr learned my addr 172.21.15.59:0/3367682846 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:29.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.144+0000 7fa946800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9480b4690 0x7fa9480cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:49680/0 (socket says 172.21.15.59:49680) 2024-04-24T06:37:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.145+0000 7fa946800640 1 -- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 msgr2=0x7fa9480b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.145+0000 7fa946800640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 0x7fa9480b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.145+0000 7fa946800640 1 -- 172.21.15.59:0/3367682846 >> v1:172.21.15.66:6789/0 conn(0x7fa94815c3d0 legacy=0x7fa9480cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.145+0000 7fa946800640 1 -- 172.21.15.59:0/3367682846 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa948160ab0 con 0x7fa9480b4690 2024-04-24T06:37:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.145+0000 7fa945400640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 0x7fa9480b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:37:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.146+0000 7fa946800640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480b4690 0x7fa9480cb7d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa9400097a0 tx=0x7fa94002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d098346c2ed5d1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.147+0000 7fa944a00640 1 -- 172.21.15.59:0/3367682846 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940037070 con 0x7fa9480b4690 2024-04-24T06:37:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.147+0000 7fa944a00640 1 -- 172.21.15.59:0/3367682846 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940032440 con 0x7fa9480b4690 2024-04-24T06:37:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.147+0000 7fa944a00640 1 -- 172.21.15.59:0/3367682846 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940032810 con 0x7fa9480b4690 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 -- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480b4690 msgr2=0x7fa9480cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480b4690 0x7fa9480cb7d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa9400097a0 tx=0x7fa94002a180 comp rx=0 tx=0).stop 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 -- 172.21.15.59:0/3367682846 shutdown_connections 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480b4690 0x7fa9480cb7d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 --2- 172.21.15.59:0/3367682846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa9480cd760 0x7fa9480b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 -- 172.21.15.59:0/3367682846 >> 172.21.15.59:0/3367682846 conn(0x7fa9480b21a0 msgr2=0x7fa9480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 -- 172.21.15.59:0/3367682846 shutdown_connections 2024-04-24T06:37:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.148+0000 7fa947200640 1 -- 172.21.15.59:0/3367682846 wait complete. 2024-04-24T06:37:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.150+0000 7fa947200640 1 Processor -- start 2024-04-24T06:37:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.150+0000 7fa947200640 1 -- start start 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480cd760 0x7fa94815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa94815c3d0 0x7fa9480ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa94815c7b0 con 0x7fa9480b4690 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480b4ac0 con 0x7fa9480cd760 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa947200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa94815c930 con 0x7fa94815c3d0 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa945e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa945e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33198/0 (socket says 172.21.15.59:33198) 2024-04-24T06:37:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.151+0000 7fa945e00640 1 -- 172.21.15.59:0/695874810 learned_addr learned my addr 172.21.15.59:0/695874810 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 -- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa94815c3d0 msgr2=0x7fa9480ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa94815c3d0 0x7fa9480ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 -- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480cd760 msgr2=0x7fa94815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480cd760 0x7fa94815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 -- 172.21.15.59:0/695874810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94815b7f0 con 0x7fa9480b4690 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa945e00640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c480 tx=0x7fa93800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938017070 con 0x7fa9480b4690 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938005ce0 con 0x7fa9480b4690 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380072d0 con 0x7fa9480b4690 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94815cff0 con 0x7fa9480b4690 2024-04-24T06:37:29.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.152+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94815d570 con 0x7fa9480b4690 2024-04-24T06:37:29.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.154+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94804fd90 con 0x7fa9480b4690 2024-04-24T06:37:29.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.155+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa93801a070 con 0x7fa9480b4690 2024-04-24T06:37:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.156+0000 7fa93e000640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.156+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa93808ae10 con 0x7fa9480b4690 2024-04-24T06:37:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.156+0000 7fa945400640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.157+0000 7fa945400640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 0x7fa92806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa94809dff0 tx=0x7fa93400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.161+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa93808e030 con 0x7fa9480b4690 2024-04-24T06:37:29.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.366+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa94806e400 con 0x7fa9480b4690 2024-04-24T06:37:29.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.368+0000 7fa93e000640 1 -- 172.21.15.59:0/695874810 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16753 (secure 0 0 0) 0x7fa938058b90 con 0x7fa9480b4690 2024-04-24T06:37:29.371 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:29.371 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:36:29.830893+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:37:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 msgr2=0x7fa92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 0x7fa92806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa94809dff0 tx=0x7fa93400b040 comp rx=0 tx=0).stop 2024-04-24T06:37:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 msgr2=0x7fa94815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c480 tx=0x7fa93800c940 comp rx=0 tx=0).stop 2024-04-24T06:37:29.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 shutdown_connections 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa92806c8f0 0x7fa92806edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa94815c3d0 0x7fa9480ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9480cd760 0x7fa94815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 --2- 172.21.15.59:0/695874810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9480b4690 0x7fa94815b2b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 >> 172.21.15.59:0/695874810 conn(0x7fa9480b21a0 msgr2=0x7fa94806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 shutdown_connections 2024-04-24T06:37:29.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.372+0000 7fa947200640 1 -- 172.21.15.59:0/695874810 wait complete. 2024-04-24T06:37:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:37:29.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:37:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.616+0000 7f985ea00640 1 Processor -- start 2024-04-24T06:37:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.617+0000 7f985ea00640 1 -- start start 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f98580d15b0 0x7f98580cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98580b5da0 con 0x7f985815ed20 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98580d0a20 con 0x7f98580b4f20 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f985815f100 con 0x7f98580d15b0 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985d600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f985815ed20 0x7f98580d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54412/0 (socket says 172.21.15.59:54412) 2024-04-24T06:37:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.618+0000 7f985d600640 1 -- 172.21.15.59:0/1785077054 learned_addr learned my addr 172.21.15.59:0/1785077054 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271789997 0 0) 0x7f98580b5da0 con 0x7f985815ed20 2024-04-24T06:37:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9840003520 con 0x7f985815ed20 2024-04-24T06:37:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f985cc00640 1 --2- 172.21.15.59:0/1785077054 >> v2:172.21.15.66:3300/0 conn(0x7f98580d15b0 0x7f98580cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 906987406 0 0) 0x7f9840003520 con 0x7f985815ed20 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 >> v2:172.21.15.66:3300/0 conn(0x7f98580d15b0 msgr2=0x7f98580cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 --2- 172.21.15.59:0/1785077054 >> v2:172.21.15.66:3300/0 conn(0x7f98580d15b0 0x7f98580cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 >> v1:172.21.15.66:6789/0 conn(0x7f98580b4f20 legacy=0x7f98580cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98580d2ed0 con 0x7f985815ed20 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f984c001670 con 0x7f985815ed20 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 >> v1:172.21.15.59:6789/0 conn(0x7f985815ed20 legacy=0x7f98580d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9840004800 0x7f9840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 0x7f984000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f984000c270 con 0x7f98400053c0 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f984000c420 con 0x7f9840003a00 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f984000c620 con 0x7f9840004800 2024-04-24T06:37:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.619+0000 7f985e000640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 0x7f984000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985d600640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985cc00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9840004800 0x7f9840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985e000640 1 -- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9840004800 msgr2=0x7f9840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985e000640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9840004800 0x7f9840004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985e000640 1 -- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9840003a00 msgr2=0x7f9840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985e000640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9840003a00 0x7f9840003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.620+0000 7f985e000640 1 -- 172.21.15.59:0/1785077054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840003520 con 0x7f98400053c0 2024-04-24T06:37:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.621+0000 7f985e000640 1 -- 172.21.15.59:0/1785077054 reap_dead start 2024-04-24T06:37:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.621+0000 7f985e000640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 0x7f984000b9c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9850001e50 tx=0x7f9840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc35b93d255576a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.621+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850001a70 con 0x7f98400053c0 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.621+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9850003970 con 0x7f98400053c0 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.621+0000 7f9857e00640 1 -- 172.21.15.59:0/1785077054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850003d60 con 0x7f98400053c0 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 -- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 msgr2=0x7f984000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 0x7f984000b9c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9850001e50 tx=0x7f9840005830 comp rx=0 tx=0).stop 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 -- 172.21.15.59:0/1785077054 shutdown_connections 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 --2- 172.21.15.59:0/1785077054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98400053c0 0x7f984000b9c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 -- 172.21.15.59:0/1785077054 >> 172.21.15.59:0/1785077054 conn(0x7f985808bf90 msgr2=0x7f98580d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 -- 172.21.15.59:0/1785077054 shutdown_connections 2024-04-24T06:37:29.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.622+0000 7f985ea00640 1 -- 172.21.15.59:0/1785077054 wait complete. 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.624+0000 7f985ea00640 1 Processor -- start 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.624+0000 7f985ea00640 1 -- start start 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98400053c0 0x7f9858150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98580ce3a0 0x7f98581572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98580b3600 con 0x7f9858158db0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98580d3490 con 0x7f98400053c0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.625+0000 7f985ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f98580b3780 con 0x7f98580ce3a0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33210/0 (socket says 172.21.15.59:33210) 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 -- 172.21.15.59:0/422089836 learned_addr learned my addr 172.21.15.59:0/422089836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985d600640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98400053c0 0x7f9858150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985cc00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98580ce3a0 0x7f98581572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 -- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98580ce3a0 msgr2=0x7f98581572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98580ce3a0 0x7f98581572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 -- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98400053c0 msgr2=0x7f9858150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98400053c0 0x7f9858150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 -- 172.21.15.59:0/422089836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985815a650 con 0x7f9858158db0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.626+0000 7f985e000640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9850001f40 tx=0x7f98500014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.627+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850007db0 con 0x7f9858158db0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.627+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985002ac70 con 0x7f9858158db0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.627+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850006d70 con 0x7f9858158db0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.627+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9858159b10 con 0x7f9858158db0 2024-04-24T06:37:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.627+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985815a090 con 0x7f9858158db0 2024-04-24T06:37:29.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.628+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985804fd90 con 0x7f9858158db0 2024-04-24T06:37:29.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.629+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9850007030 con 0x7f9858158db0 2024-04-24T06:37:29.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.630+0000 7f9855600640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 0x7f983806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:29.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.631+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f98500aad40 con 0x7f9858158db0 2024-04-24T06:37:29.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.631+0000 7f985d600640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 0x7f983806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:29.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.632+0000 7f985d600640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 0x7f983806edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f984c001e80 tx=0x7f984c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:29.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.635+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9850079080 con 0x7f9858158db0 2024-04-24T06:37:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.839+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98580d1020 con 0x7f9858158db0 2024-04-24T06:37:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.840+0000 7f9855600640 1 -- 172.21.15.59:0/422089836 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f9850078980 con 0x7f9858158db0 2024-04-24T06:37:29.842 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:37:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.842+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 msgr2=0x7f983806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.842+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 0x7f983806edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f984c001e80 tx=0x7f984c009040 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.842+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 msgr2=0x7f9858159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.842+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9850001f40 tx=0x7f98500014a0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 shutdown_connections 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f983806c8f0 0x7f983806edb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9858158db0 0x7f9858159230 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98580ce3a0 0x7f98581572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 --2- 172.21.15.59:0/422089836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98400053c0 0x7f9858150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 >> 172.21.15.59:0/422089836 conn(0x7f985808bf90 msgr2=0x7f98580d4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 shutdown_connections 2024-04-24T06:37:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:29.843+0000 7f985ea00640 1 -- 172.21.15.59:0/422089836 wait complete. 2024-04-24T06:37:29.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:37:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.087+0000 7f2cb0e00640 1 Processor -- start 2024-04-24T06:37:30.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.087+0000 7f2cb0e00640 1 -- start start 2024-04-24T06:37:30.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.088+0000 7f2cb0e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2cac145e10 0x7f2cac06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.088+0000 7f2cb0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 0x7f2cac0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.088+0000 7f2cb0e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac0cfc20 con 0x7f2cac071d70 2024-04-24T06:37:30.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.088+0000 7f2cb0e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0cff00 con 0x7f2cac145e10 2024-04-24T06:37:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.088+0000 7f2cb0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac0d01e0 con 0x7f2cac0d0cc0 2024-04-24T06:37:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.090+0000 7f2ca9e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2cac145e10 0x7f2cac06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.090+0000 7f2ca1400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 0x7f2cac0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.090+0000 7f2caa800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2cac071d70 0x7f2cac15cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:59386/0 (socket says 172.21.15.59:59386) 2024-04-24T06:37:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.090+0000 7f2caa800640 1 -- 172.21.15.59:0/3975384204 learned_addr learned my addr 172.21.15.59:0/3975384204 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:30.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.091+0000 7f2ca9e00640 1 -- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 msgr2=0x7f2cac0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:30.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.091+0000 7f2ca9e00640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 0x7f2cac0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.091+0000 7f2ca9e00640 1 -- 172.21.15.59:0/3975384204 >> v1:172.21.15.66:6789/0 conn(0x7f2cac071d70 legacy=0x7f2cac15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.091+0000 7f2ca9e00640 1 -- 172.21.15.59:0/3975384204 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac0d04c0 con 0x7f2cac145e10 2024-04-24T06:37:30.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.091+0000 7f2ca1400640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 0x7f2cac0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:37:30.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.092+0000 7f2ca9e00640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac145e10 0x7f2cac06f130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009dc0 tx=0x7f2c9c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19210edbae293b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.092+0000 7f2ca9400640 1 -- 172.21.15.59:0/3975384204 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c02ae60 con 0x7f2cac145e10 2024-04-24T06:37:30.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.092+0000 7f2ca9400640 1 -- 172.21.15.59:0/3975384204 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9c007740 con 0x7f2cac145e10 2024-04-24T06:37:30.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.092+0000 7f2ca9400640 1 -- 172.21.15.59:0/3975384204 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c007b10 con 0x7f2cac145e10 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 -- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac145e10 msgr2=0x7f2cac06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac145e10 0x7f2cac06f130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009dc0 tx=0x7f2c9c02a680 comp rx=0 tx=0).stop 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 -- 172.21.15.59:0/3975384204 shutdown_connections 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac0d0cc0 0x7f2cac0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/3975384204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac145e10 0x7f2cac06f130 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 -- 172.21.15.59:0/3975384204 >> 172.21.15.59:0/3975384204 conn(0x7f2cac067d90 msgr2=0x7f2cac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:30.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 -- 172.21.15.59:0/3975384204 shutdown_connections 2024-04-24T06:37:30.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.093+0000 7f2cb0e00640 1 -- 172.21.15.59:0/3975384204 wait complete. 2024-04-24T06:37:30.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.095+0000 7f2cb0e00640 1 Processor -- start 2024-04-24T06:37:30.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.095+0000 7f2cb0e00640 1 -- start start 2024-04-24T06:37:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac071d70 0x7f2cac13b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac0d0cc0 0x7f2cac13d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0cfa00 con 0x7f2cac145e10 2024-04-24T06:37:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0cf880 con 0x7f2cac0d0cc0 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.096+0000 7f2cb0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac1461f0 con 0x7f2cac071d70 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33216/0 (socket says 172.21.15.59:33216) 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 -- 172.21.15.59:0/4248551301 learned_addr learned my addr 172.21.15.59:0/4248551301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2ca9e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac071d70 0x7f2cac13b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2ca1400640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac0d0cc0 0x7f2cac13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 -- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac071d70 msgr2=0x7f2cac13b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac071d70 0x7f2cac13b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 -- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac0d0cc0 msgr2=0x7f2cac13d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac0d0cc0 0x7f2cac13d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.097+0000 7f2caa800640 1 -- 172.21.15.59:0/4248551301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac13bb40 con 0x7f2cac145e10 2024-04-24T06:37:30.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2caa800640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c450 tx=0x7f2ca400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4017070 con 0x7f2cac145e10 2024-04-24T06:37:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca4005ce0 con 0x7f2cac145e10 2024-04-24T06:37:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4007270 con 0x7f2cac145e10 2024-04-24T06:37:30.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac13be20 con 0x7f2cac145e10 2024-04-24T06:37:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.098+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac13c3e0 con 0x7f2cac145e10 2024-04-24T06:37:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.099+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac068a80 con 0x7f2cac145e10 2024-04-24T06:37:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.100+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ca401a070 con 0x7f2cac145e10 2024-04-24T06:37:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.101+0000 7f2ca2a00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.101+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2ca408ae30 con 0x7f2cac145e10 2024-04-24T06:37:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.102+0000 7f2ca9e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.102+0000 7f2ca9e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.106+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ca4058b20 con 0x7f2cac145e10 2024-04-24T06:37:30.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.315+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2cac13c780 con 0x7f2cac145e10 2024-04-24T06:37:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.316+0000 7f2ca2a00640 1 -- 172.21.15.59:0/4248551301 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f2ca405ce20 con 0x7f2cac145e10 2024-04-24T06:37:30.318 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:37:30.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.319+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 msgr2=0x7f2c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.319+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c033040 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.319+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 msgr2=0x7f2cac1f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.319+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c450 tx=0x7f2ca400c910 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 shutdown_connections 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac145e10 0x7f2cac1f4dc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cac0d0cc0 0x7f2cac13d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 --2- 172.21.15.59:0/4248551301 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2cac071d70 0x7f2cac13b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 >> 172.21.15.59:0/4248551301 conn(0x7f2cac067d90 msgr2=0x7f2cac15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 shutdown_connections 2024-04-24T06:37:30.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.320+0000 7f2cb0e00640 1 -- 172.21.15.59:0/4248551301 wait complete. 2024-04-24T06:37:30.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:37:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.554+0000 7fd006000640 1 Processor -- start 2024-04-24T06:37:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.554+0000 7fd006000640 1 -- start start 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd006000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd00015d300 0x7fd000071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd006000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd000167180 0x7fd000163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd006000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd000163320 con 0x7fd0000d0600 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd006000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd000163650 con 0x7fd00015d300 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd006000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd000072380 con 0x7fd000167180 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd004c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd00015d300 0x7fd000071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd004c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd00015d300 0x7fd000071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33226/0 (socket says 172.21.15.59:33226) 2024-04-24T06:37:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.556+0000 7fd004c00640 1 -- 172.21.15.59:0/803364827 learned_addr learned my addr 172.21.15.59:0/803364827 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:30.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.557+0000 7fcff7e00640 1 --2- 172.21.15.59:0/803364827 >> v2:172.21.15.66:3300/0 conn(0x7fd000167180 0x7fd000163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.557+0000 7fd004c00640 1 -- 172.21.15.59:0/803364827 >> v2:172.21.15.66:3300/0 conn(0x7fd000167180 msgr2=0x7fd000163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:30.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.557+0000 7fd004c00640 1 --2- 172.21.15.59:0/803364827 >> v2:172.21.15.66:3300/0 conn(0x7fd000167180 0x7fd000163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.557+0000 7fd004c00640 1 -- 172.21.15.59:0/803364827 >> v1:172.21.15.66:6789/0 conn(0x7fd0000d0600 legacy=0x7fd00006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:30.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.557+0000 7fd004c00640 1 -- 172.21.15.59:0/803364827 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000072660 con 0x7fd00015d300 2024-04-24T06:37:30.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.558+0000 7fd004c00640 1 --2- 172.21.15.59:0/803364827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd000071c70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd0001637d0 tx=0x7fcff002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c330990a4a8109f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.558+0000 7fcfffe00640 1 -- 172.21.15.59:0/803364827 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0037070 con 0x7fd00015d300 2024-04-24T06:37:30.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.558+0000 7fcfffe00640 1 -- 172.21.15.59:0/803364827 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff0032440 con 0x7fd00015d300 2024-04-24T06:37:30.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.558+0000 7fcfffe00640 1 -- 172.21.15.59:0/803364827 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0032810 con 0x7fd00015d300 2024-04-24T06:37:30.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 -- 172.21.15.59:0/803364827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 msgr2=0x7fd000071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 --2- 172.21.15.59:0/803364827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd000071c70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd0001637d0 tx=0x7fcff002a210 comp rx=0 tx=0).stop 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 -- 172.21.15.59:0/803364827 shutdown_connections 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 --2- 172.21.15.59:0/803364827 >> v2:172.21.15.66:3300/0 conn(0x7fd000167180 0x7fd000163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 --2- 172.21.15.59:0/803364827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd000071c70 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 -- 172.21.15.59:0/803364827 >> 172.21.15.59:0/803364827 conn(0x7fd00006bdf0 msgr2=0x7fd00006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 -- 172.21.15.59:0/803364827 shutdown_connections 2024-04-24T06:37:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.559+0000 7fd006000640 1 -- 172.21.15.59:0/803364827 wait complete. 2024-04-24T06:37:30.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.561+0000 7fd006000640 1 Processor -- start 2024-04-24T06:37:30.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.561+0000 7fd006000640 1 -- start start 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0000d0600 0x7fd0000b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd000167180 0x7fd00009c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd000164020 con 0x7fd00015d300 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001675e0 con 0x7fd000167180 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.562+0000 7fd006000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd00006e8f0 con 0x7fd0000d0600 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33228/0 (socket says 172.21.15.59:33228) 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 -- 172.21.15.59:0/3834384869 learned_addr learned my addr 172.21.15.59:0/3834384869 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fd004c00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0000d0600 0x7fd0000b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 -- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0000d0600 msgr2=0x7fd0000b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0000d0600 0x7fd0000b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 -- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd000167180 msgr2=0x7fd00009c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd000167180 0x7fd00009c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 -- 172.21.15.59:0/3834384869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000157ce0 con 0x7fd00015d300 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.563+0000 7fcff7e00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c5e0 tx=0x7fcfe800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.564+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8015070 con 0x7fd00015d300 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.564+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe8004260 con 0x7fd00015d300 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.564+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003610 con 0x7fd00015d300 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.564+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000157fc0 con 0x7fd00015d300 2024-04-24T06:37:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.564+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001568f0 con 0x7fd00015d300 2024-04-24T06:37:30.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.565+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00004fd90 con 0x7fd00015d300 2024-04-24T06:37:30.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.566+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe80037d0 con 0x7fd00015d300 2024-04-24T06:37:30.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.567+0000 7fcffd600640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:30.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.567+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fcfe808ba50 con 0x7fd00015d300 2024-04-24T06:37:30.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.567+0000 7fd004c00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:30.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.568+0000 7fd004c00640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcff0034000 tx=0x7fcff002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.572+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfe8059d90 con 0x7fd00015d300 2024-04-24T06:37:30.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.775+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd0000cff50 con 0x7fd00015d300 2024-04-24T06:37:30.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.775+0000 7fcffd600640 1 -- 172.21.15.59:0/3834384869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7fcfe8059690 con 0x7fd00015d300 2024-04-24T06:37:30.777 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.778+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 msgr2=0x7fcfdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcff0034000 tx=0x7fcff002e000 comp rx=0 tx=0).stop 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 msgr2=0x7fd0001563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c5e0 tx=0x7fcfe800caa0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 shutdown_connections 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd000167180 0x7fd00009c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd00015d300 0x7fd0001563b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 --2- 172.21.15.59:0/3834384869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0000d0600 0x7fd0000b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 >> 172.21.15.59:0/3834384869 conn(0x7fd00006bdf0 msgr2=0x7fd00006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 shutdown_connections 2024-04-24T06:37:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:30.779+0000 7fd006000640 1 -- 172.21.15.59:0/3834384869 wait complete. 2024-04-24T06:37:30.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:37:31.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.013+0000 7fbd67000640 1 Processor -- start 2024-04-24T06:37:31.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.013+0000 7fbd67000640 1 -- start start 2024-04-24T06:37:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.014+0000 7fbd67000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbd601416e0 0x7fbd6015f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.014+0000 7fbd67000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 0x7fbd601555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.014+0000 7fbd67000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6015ba70 con 0x7fbd600d05c0 2024-04-24T06:37:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.014+0000 7fbd67000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbd6015bdc0 con 0x7fbd601416e0 2024-04-24T06:37:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.014+0000 7fbd67000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6015f5a0 con 0x7fbd6015c250 2024-04-24T06:37:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd65c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbd600d05c0 0x7fbd600d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54420/0 (socket says 172.21.15.59:54420) 2024-04-24T06:37:31.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd65c00640 1 -- 172.21.15.59:0/3060050908 learned_addr learned my addr 172.21.15.59:0/3060050908 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd66600640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 0x7fbd601555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd65200640 1 --2- 172.21.15.59:0/3060050908 >> v2:172.21.15.66:3300/0 conn(0x7fbd601416e0 0x7fbd6015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1640695839 0 0) 0x7fbd6015ba70 con 0x7fbd600d05c0 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.015+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd48005730 con 0x7fbd600d05c0 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3673713931 0 0) 0x7fbd48005730 con 0x7fbd600d05c0 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 msgr2=0x7fbd601555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 0x7fbd601555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 >> v2:172.21.15.66:3300/0 conn(0x7fbd601416e0 msgr2=0x7fbd6015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 --2- 172.21.15.59:0/3060050908 >> v2:172.21.15.66:3300/0 conn(0x7fbd601416e0 0x7fbd6015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6015f800 con 0x7fbd600d05c0 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbd540019f0 con 0x7fbd600d05c0 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 >> v1:172.21.15.59:6789/0 conn(0x7fbd600d05c0 legacy=0x7fbd600d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48005bc0 0x7fbd48005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd48006a10 0x7fbd48006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 0x7fbd480073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6015bdc0 con 0x7fbd48006a10 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6015f5a0 con 0x7fbd48005bc0 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4800dbd0 con 0x7fbd6015c250 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.016+0000 7fbd65200640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd48006a10 0x7fbd48006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65c00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48005bc0 0x7fbd48005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65200640 1 -- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 msgr2=0x7fbd480073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65200640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd6015c250 0x7fbd480073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65200640 1 -- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48005bc0 msgr2=0x7fbd48005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65200640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48005bc0 0x7fbd48005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65200640 1 -- 172.21.15.59:0/3060050908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd48005730 con 0x7fbd48006a10 2024-04-24T06:37:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.017+0000 7fbd65c00640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd48005bc0 0x7fbd48005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.018+0000 7fbd65200640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd48006a10 0x7fbd48006e90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009ac0 tx=0x7fbd48007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac7a637a7c780f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.018+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c036070 con 0x7fbd48006a10 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.018+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4c031400 con 0x7fbd48006a10 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.018+0000 7fbd5fe00640 1 -- 172.21.15.59:0/3060050908 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c009690 con 0x7fbd48006a10 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 -- 172.21.15.59:0/3060050908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd48006a10 msgr2=0x7fbd48006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 --2- 172.21.15.59:0/3060050908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd48006a10 0x7fbd48006e90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009ac0 tx=0x7fbd48007a40 comp rx=0 tx=0).stop 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd66600640 1 -- 172.21.15.59:0/3060050908 reap_dead start 2024-04-24T06:37:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 -- 172.21.15.59:0/3060050908 shutdown_connections 2024-04-24T06:37:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 -- 172.21.15.59:0/3060050908 >> 172.21.15.59:0/3060050908 conn(0x7fbd6006bdf0 msgr2=0x7fbd6006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 -- 172.21.15.59:0/3060050908 shutdown_connections 2024-04-24T06:37:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.019+0000 7fbd67000640 1 -- 172.21.15.59:0/3060050908 wait complete. 2024-04-24T06:37:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.021+0000 7fbd67000640 1 Processor -- start 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.021+0000 7fbd67000640 1 -- start start 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 0x7fbd601f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd6015e050 0x7fbd6015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60141b70 con 0x7fbd60153860 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6015fdc0 con 0x7fbd6015e050 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd67000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6015c6e0 con 0x7fbd48006a10 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd65200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd65200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33254/0 (socket says 172.21.15.59:33254) 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.022+0000 7fbd65200640 1 -- 172.21.15.59:0/1490502837 learned_addr learned my addr 172.21.15.59:0/1490502837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 -- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 msgr2=0x7fbd601f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65c00640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 0x7fbd601f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd66600640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd6015e050 0x7fbd6015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 0x7fbd601f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 -- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd6015e050 msgr2=0x7fbd6015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd6015e050 0x7fbd6015e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 -- 172.21.15.59:0/1490502837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd60154410 con 0x7fbd60153860 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65c00640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 0x7fbd601f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:31.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd65200640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009be0 tx=0x7fbd4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c040020 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4c003950 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd601546f0 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.023+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c036040 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.024+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd60154cb0 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.025+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6004fd90 con 0x7fbd60153860 2024-04-24T06:37:31.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.026+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd4c043020 con 0x7fbd60153860 2024-04-24T06:37:31.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.027+0000 7fbd5d600640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 0x7fbd4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.027+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fbd4c0ac0e0 con 0x7fbd60153860 2024-04-24T06:37:31.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.027+0000 7fbd65c00640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 0x7fbd4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.028+0000 7fbd65c00640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 0x7fbd4006edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007c40 tx=0x7fbd54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.031+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd4c04a030 con 0x7fbd60153860 2024-04-24T06:37:31.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.250+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd600cfec0 con 0x7fbd60153860 2024-04-24T06:37:31.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.251+0000 7fbd5d600640 1 -- 172.21.15.59:0/1490502837 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v86) ==== 80+0+120 (secure 0 0 0) 0x7fbd4c07a4b0 con 0x7fbd60153860 2024-04-24T06:37:31.253 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:37:31.253 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6832/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:37:31.253 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:37:31.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 msgr2=0x7fbd4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 0x7fbd4006edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007c40 tx=0x7fbd54009040 comp rx=0 tx=0).stop 2024-04-24T06:37:31.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 msgr2=0x7fbd6009a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009be0 tx=0x7fbd4c004730 comp rx=0 tx=0).stop 2024-04-24T06:37:31.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 shutdown_connections 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd4006c8f0 0x7fbd4006edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd6015e050 0x7fbd6015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd60153860 0x7fbd6009a170 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 --2- 172.21.15.59:0/1490502837 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd48006a10 0x7fbd601f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 >> 172.21.15.59:0/1490502837 conn(0x7fbd6006bdf0 msgr2=0x7fbd6006db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 shutdown_connections 2024-04-24T06:37:31.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.254+0000 7fbd67000640 1 -- 172.21.15.59:0/1490502837 wait complete. 2024-04-24T06:37:31.276 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:37:31.277 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:37:31.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:37:31.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.544+0000 7f4703200640 1 Processor -- start 2024-04-24T06:37:31.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.545+0000 7f4703200640 1 -- start start 2024-04-24T06:37:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.546+0000 7f4703200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f470415e7c0 0x7f47040d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.546+0000 7f4703200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47040cf030 con 0x7f47040d2630 2024-04-24T06:37:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.546+0000 7f4703200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47040b6a40 con 0x7f47040d0d90 2024-04-24T06:37:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.546+0000 7f4703200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f47040b7000 con 0x7f470415e7c0 2024-04-24T06:37:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f470415e7c0 0x7f47040d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f470415e7c0 0x7f47040d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33264/0 (socket says 172.21.15.59:33264) 2024-04-24T06:37:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f47040d2630 0x7f47040b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54436/0 (socket says 172.21.15.59:54436) 2024-04-24T06:37:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 -- 172.21.15.59:0/2618067992 learned_addr learned my addr 172.21.15.59:0/2618067992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 -- 172.21.15.59:0/2618067992 >> v1:172.21.15.66:6789/0 conn(0x7f47040d0d90 legacy=0x7f47040cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 -- 172.21.15.59:0/2618067992 >> v1:172.21.15.59:6789/0 conn(0x7f47040d2630 legacy=0x7f47040b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.547+0000 7f4701e00640 1 -- 172.21.15.59:0/2618067992 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47040b6d20 con 0x7f470415e7c0 2024-04-24T06:37:31.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4701e00640 1 --2- 172.21.15.59:0/2618067992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f470415e7c0 0x7f47040d20f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e50 tx=0x7f46f402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b64703310744e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4700a00640 1 -- 172.21.15.59:0/2618067992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4036070 con 0x7f470415e7c0 2024-04-24T06:37:31.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4700a00640 1 -- 172.21.15.59:0/2618067992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4031600 con 0x7f470415e7c0 2024-04-24T06:37:31.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4700a00640 1 -- 172.21.15.59:0/2618067992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40319f0 con 0x7f470415e7c0 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4703200640 1 -- 172.21.15.59:0/2618067992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f470415e7c0 msgr2=0x7f47040d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.549+0000 7f4703200640 1 --2- 172.21.15.59:0/2618067992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f470415e7c0 0x7f47040d20f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e50 tx=0x7f46f402a7c0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.550+0000 7f4703200640 1 -- 172.21.15.59:0/2618067992 shutdown_connections 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.550+0000 7f4703200640 1 --2- 172.21.15.59:0/2618067992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f470415e7c0 0x7f47040d20f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.550+0000 7f4703200640 1 -- 172.21.15.59:0/2618067992 >> 172.21.15.59:0/2618067992 conn(0x7f470409d920 msgr2=0x7f470408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.550+0000 7f4703200640 1 -- 172.21.15.59:0/2618067992 shutdown_connections 2024-04-24T06:37:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.550+0000 7f4703200640 1 -- 172.21.15.59:0/2618067992 wait complete. 2024-04-24T06:37:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.551+0000 7f4703200640 1 Processor -- start 2024-04-24T06:37:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.552+0000 7f4703200640 1 -- start start 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 0x7f470415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f470415e7c0 0x7f4704157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47040b6700 con 0x7f47040d0d90 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47040cf620 con 0x7f47040d2630 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4703200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f47040d1a80 con 0x7f470415e7c0 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33272/0 (socket says 172.21.15.59:33272) 2024-04-24T06:37:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 -- 172.21.15.59:0/2817509571 learned_addr learned my addr 172.21.15.59:0/2817509571 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701400640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 0x7f470415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 -- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f470415e7c0 msgr2=0x7f4704157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f470415e7c0 0x7f4704157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 -- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 msgr2=0x7f470415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 0x7f470415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701e00640 1 -- 172.21.15.59:0/2817509571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47040b1510 con 0x7f47040d0d90 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.553+0000 7f4701400640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 0x7f470415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.554+0000 7f4701e00640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009f70 tx=0x7f46f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.554+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4036050 con 0x7f47040d0d90 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.554+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4037040 con 0x7f47040d0d90 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.554+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4009510 con 0x7f47040d0d90 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.554+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47040b17f0 con 0x7f47040d0d90 2024-04-24T06:37:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.555+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47040b1db0 con 0x7f47040d0d90 2024-04-24T06:37:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.556+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47040b20e0 con 0x7f47040d0d90 2024-04-24T06:37:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.557+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46f4044020 con 0x7f47040d0d90 2024-04-24T06:37:31.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.558+0000 7f46fa000640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:31.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.558+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f46f40ac1c0 con 0x7f47040d0d90 2024-04-24T06:37:31.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.558+0000 7f4701400640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:31.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.559+0000 7f4701400640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 0x7f46e006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4704162c30 tx=0x7f46ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:31.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.562+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46f404b020 con 0x7f47040d0d90 2024-04-24T06:37:31.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.768+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f470404fd90 con 0x7f47040d0d90 2024-04-24T06:37:31.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.771+0000 7f46fa000640 1 -- 172.21.15.59:0/2817509571 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16753 (secure 0 0 0) 0x7f46f407a500 con 0x7f47040d0d90 2024-04-24T06:37:31.774 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:31.774 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:36:29.830893+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:37:31.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.774+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 msgr2=0x7f46e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.774+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 0x7f46e006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4704162c30 tx=0x7f46ec009040 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.774+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 msgr2=0x7f47040cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.774+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009f70 tx=0x7f46f4004a40 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 shutdown_connections 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f470415e7c0 0x7f4704157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47040d2630 0x7f470415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 --2- 172.21.15.59:0/2817509571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47040d0d90 0x7f47040cda90 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 >> 172.21.15.59:0/2817509571 conn(0x7f470409d920 msgr2=0x7f470415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 shutdown_connections 2024-04-24T06:37:31.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:31.775+0000 7f4703200640 1 -- 172.21.15.59:0/2817509571 wait complete. 2024-04-24T06:37:31.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:37:31.797 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:37:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.010+0000 7f04bd400640 1 Processor -- start 2024-04-24T06:37:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.010+0000 7f04bd400640 1 -- start start 2024-04-24T06:37:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04b8162c10 0x7f04b80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 0x7f04b8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80cfba0 0x7f04b80722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f04b80d0fd0 con 0x7f04b8162c10 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f04b80b35d0 con 0x7f04b80b4670 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04bd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04b80b38b0 con 0x7f04b80cfba0 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04b8162c10 0x7f04b80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04b8162c10 0x7f04b80d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33278/0 (socket says 172.21.15.59:33278) 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 -- 172.21.15.59:0/141775498 learned_addr learned my addr 172.21.15.59:0/141775498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 -- 172.21.15.59:0/141775498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80cfba0 msgr2=0x7f04b80722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04ade00640 1 --2- 172.21.15.59:0/141775498 >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 0x7f04b8071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 --2- 172.21.15.59:0/141775498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80cfba0 0x7f04b80722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 -- 172.21.15.59:0/141775498 >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 msgr2=0x7f04b8071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 --2- 172.21.15.59:0/141775498 >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 0x7f04b8071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04b6800640 1 -- 172.21.15.59:0/141775498 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b80b3b90 con 0x7f04b8162c10 2024-04-24T06:37:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.012+0000 7f04ade00640 1 --2- 172.21.15.59:0/141775498 >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 0x7f04b8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04b6800640 1 --2- 172.21.15.59:0/141775498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b8162c10 0x7f04b80d0a90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b5b20 tx=0x7f04a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a32de58da900d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04b5e00640 1 -- 172.21.15.59:0/141775498 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8037070 con 0x7f04b8162c10 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04b5e00640 1 -- 172.21.15.59:0/141775498 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8032440 con 0x7f04b8162c10 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04b5e00640 1 -- 172.21.15.59:0/141775498 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032810 con 0x7f04b8162c10 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04bd400640 1 -- 172.21.15.59:0/141775498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b8162c10 msgr2=0x7f04b80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.014+0000 7f04bd400640 1 --2- 172.21.15.59:0/141775498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b8162c10 0x7f04b80d0a90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b5b20 tx=0x7f04a802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 -- 172.21.15.59:0/141775498 shutdown_connections 2024-04-24T06:37:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 --2- 172.21.15.59:0/141775498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80cfba0 0x7f04b80722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 --2- 172.21.15.59:0/141775498 >> v2:172.21.15.66:3300/0 conn(0x7f04b80b4670 0x7f04b8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 --2- 172.21.15.59:0/141775498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b8162c10 0x7f04b80d0a90 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 -- 172.21.15.59:0/141775498 >> 172.21.15.59:0/141775498 conn(0x7f04b8067d90 msgr2=0x7f04b806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 -- 172.21.15.59:0/141775498 shutdown_connections 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.015+0000 7f04bd400640 1 -- 172.21.15.59:0/141775498 wait complete. 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.016+0000 7f04bd400640 1 Processor -- start 2024-04-24T06:37:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.017+0000 7f04bd400640 1 -- start start 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80b4670 0x7f04b814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 0x7f04b814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8162ff0 con 0x7f04b80cfba0 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04b80b4150 con 0x7f04b8162c10 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04bd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04b8163170 con 0x7f04b80b4670 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33286/0 (socket says 172.21.15.59:33286) 2024-04-24T06:37:32.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 -- 172.21.15.59:0/97073553 learned_addr learned my addr 172.21.15.59:0/97073553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04b6800640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80b4670 0x7f04b814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04b7200640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 0x7f04b814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 -- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80b4670 msgr2=0x7f04b814e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80b4670 0x7f04b814e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 -- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 msgr2=0x7f04b814a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 0x7f04b814a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04ade00640 1 -- 172.21.15.59:0/97073553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b81593e0 con 0x7f04b80cfba0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.018+0000 7f04b7200640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 0x7f04b814a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04ade00640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f04a000ce30 tx=0x7f04a000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000f680 con 0x7f04b80cfba0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a000f820 con 0x7f04b80cfba0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b81596c0 con 0x7f04b80cfba0 2024-04-24T06:37:32.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000fc30 con 0x7f04b80cfba0 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.019+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b80b2720 con 0x7f04b80cfba0 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.020+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8068a80 con 0x7f04b80cfba0 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.021+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04a0018400 con 0x7f04b80cfba0 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.023+0000 7f04af400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 0x7f049406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.023+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f04a008cb00 con 0x7f04b80cfba0 2024-04-24T06:37:32.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.023+0000 7f04b6800640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 0x7f049406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.024+0000 7f04b6800640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 0x7f049406edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.027+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04a0014030 con 0x7f04b80cfba0 2024-04-24T06:37:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.231+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04b8159890 con 0x7f04b80cfba0 2024-04-24T06:37:32.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.232+0000 7f04af400640 1 -- 172.21.15.59:0/97073553 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f04a005a7f0 con 0x7f04b80cfba0 2024-04-24T06:37:32.234 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.234+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 msgr2=0x7f049406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.234+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 0x7f049406edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).stop 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.234+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 msgr2=0x7f04b80b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.234+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f04a000ce30 tx=0x7f04a000a650 comp rx=0 tx=0).stop 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 shutdown_connections 2024-04-24T06:37:32.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f049406c8f0 0x7f049406edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04b8162c10 0x7f04b814a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04b80cfba0 0x7f04b80b21e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 --2- 172.21.15.59:0/97073553 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04b80b4670 0x7f04b814e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 >> 172.21.15.59:0/97073553 conn(0x7f04b8067d90 msgr2=0x7f04b8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 shutdown_connections 2024-04-24T06:37:32.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.235+0000 7f04bd400640 1 -- 172.21.15.59:0/97073553 wait complete. 2024-04-24T06:37:32.256 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:37:32.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.471+0000 7f2054a00640 1 Processor -- start 2024-04-24T06:37:32.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.471+0000 7f2054a00640 1 -- start start 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.472+0000 7f2054a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f205009b740 0x7f20500d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f2054a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20500ccfe0 con 0x7f205009bbd0 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f2054a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20500cd280 con 0x7f205009aa00 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f2054a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f20500cde90 con 0x7f205009b740 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f204f200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f205009bbd0 0x7f20500d0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54448/0 (socket says 172.21.15.59:54448) 2024-04-24T06:37:32.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f204f200640 1 -- 172.21.15.59:0/3945150822 learned_addr learned my addr 172.21.15.59:0/3945150822 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131949724 0 0) 0x7f20500ccfe0 con 0x7f205009bbd0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.473+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2034003520 con 0x7f205009bbd0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204de00640 1 --2- 172.21.15.59:0/3945150822 >> v2:172.21.15.66:3300/0 conn(0x7f205009b740 0x7f20500d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030280449 0 0) 0x7f2034003520 con 0x7f205009bbd0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 >> v2:172.21.15.66:3300/0 conn(0x7f205009b740 msgr2=0x7f20500d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 --2- 172.21.15.59:0/3945150822 >> v2:172.21.15.66:3300/0 conn(0x7f205009b740 0x7f20500d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 >> v1:172.21.15.66:6789/0 conn(0x7f205009aa00 legacy=0x7f205009ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20500cd560 con 0x7f205009bbd0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2048001690 con 0x7f205009bbd0 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 >> v1:172.21.15.59:6789/0 conn(0x7f205009bbd0 legacy=0x7f20500d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2034003a00 0x7f2034003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2034004800 0x7f2034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 0x7f203400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20500cde90 con 0x7f20340052e0 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f203400c200 con 0x7f2034004800 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f203400c400 con 0x7f2034003a00 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.474+0000 7f204f200640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 0x7f203400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204de00640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2034004800 0x7f2034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204e800640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2034003a00 0x7f2034003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204f200640 1 -- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2034003a00 msgr2=0x7f2034003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204f200640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2034003a00 0x7f2034003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204f200640 1 -- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2034004800 msgr2=0x7f2034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204f200640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2034004800 0x7f2034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204f200640 1 -- 172.21.15.59:0/3945150822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2034003520 con 0x7f20340052e0 2024-04-24T06:37:32.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.475+0000 7f204de00640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2034004800 0x7f2034004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.476+0000 7f204f200640 1 -- 172.21.15.59:0/3945150822 reap_dead start 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.476+0000 7f204f200640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 0x7f203400b9c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f20500cd1b0 tx=0x7f2034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b43629b5d23bad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.477+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007c80 con 0x7f20340052e0 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.477+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048005ad0 con 0x7f20340052e0 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.477+0000 7f204d400640 1 -- 172.21.15.59:0/3945150822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007070 con 0x7f20340052e0 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.477+0000 7f2054a00640 1 -- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 msgr2=0x7f203400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.477+0000 7f2054a00640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 0x7f203400b9c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f20500cd1b0 tx=0x7f2034005830 comp rx=0 tx=0).stop 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.478+0000 7f2054a00640 1 -- 172.21.15.59:0/3945150822 shutdown_connections 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.478+0000 7f2054a00640 1 --2- 172.21.15.59:0/3945150822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20340052e0 0x7f203400b9c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.478+0000 7f2054a00640 1 -- 172.21.15.59:0/3945150822 >> 172.21.15.59:0/3945150822 conn(0x7f20500b3f30 msgr2=0x7f20500b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.478+0000 7f2054a00640 1 -- 172.21.15.59:0/3945150822 shutdown_connections 2024-04-24T06:37:32.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.478+0000 7f2054a00640 1 -- 172.21.15.59:0/3945150822 wait complete. 2024-04-24T06:37:32.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.480+0000 7f2054a00640 1 Processor -- start 2024-04-24T06:37:32.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.480+0000 7f2054a00640 1 -- start start 2024-04-24T06:37:32.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f20340052e0 0x7f20501534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20501f7e00 0x7f20501f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20500ce320 con 0x7f20500cb160 2024-04-24T06:37:32.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20500cdb20 con 0x7f20501f7e00 2024-04-24T06:37:32.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f2054a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f205015db50 con 0x7f20340052e0 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f204f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f204f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33308/0 (socket says 172.21.15.59:33308) 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f204f200640 1 -- 172.21.15.59:0/3647488555 learned_addr learned my addr 172.21.15.59:0/3647488555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.481+0000 7f204e800640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f20340052e0 0x7f20501534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204de00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20501f7e00 0x7f20501f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 -- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f20340052e0 msgr2=0x7f20501534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f20340052e0 0x7f20501534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 -- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20501f7e00 msgr2=0x7f20501f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20501f7e00 0x7f20501f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 -- 172.21.15.59:0/3647488555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20500ca9e0 con 0x7f20500cb160 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f204f200640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2048001eb0 tx=0x7f2048001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.482+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048037040 con 0x7f20500cb160 2024-04-24T06:37:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.483+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048041070 con 0x7f20500cb160 2024-04-24T06:37:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.483+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007070 con 0x7f20500cb160 2024-04-24T06:37:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.483+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2050159660 con 0x7f20500cb160 2024-04-24T06:37:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.483+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2050159c20 con 0x7f20500cb160 2024-04-24T06:37:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.484+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205004fd90 con 0x7f20500cb160 2024-04-24T06:37:32.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.485+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2048007230 con 0x7f20500cb160 2024-04-24T06:37:32.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.486+0000 7f2046a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 0x7f202c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.486+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f20480ab910 con 0x7f20500cb160 2024-04-24T06:37:32.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.486+0000 7f204e800640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 0x7f202c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.487+0000 7f204e800640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 0x7f202c06ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2040007fd0 tx=0x7f2040009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.490+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2048005aa0 con 0x7f20500cb160 2024-04-24T06:37:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.694+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2050159e60 con 0x7f20500cb160 2024-04-24T06:37:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.695+0000 7f2046a00640 1 -- 172.21.15.59:0/3647488555 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f20480797b0 con 0x7f20500cb160 2024-04-24T06:37:32.697 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.697+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 msgr2=0x7f202c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.697+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 0x7f202c06ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2040007fd0 tx=0x7f2040009040 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 msgr2=0x7f20500cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2048001eb0 tx=0x7f2048001990 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 shutdown_connections 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f202c06c920 0x7f202c06ede0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500cb160 0x7f20500cb5c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20501f7e00 0x7f20501f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 --2- 172.21.15.59:0/3647488555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f20340052e0 0x7f20501534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 >> 172.21.15.59:0/3647488555 conn(0x7f20500b3f30 msgr2=0x7f2050154780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 shutdown_connections 2024-04-24T06:37:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.698+0000 7f2054a00640 1 -- 172.21.15.59:0/3647488555 wait complete. 2024-04-24T06:37:32.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:37:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.922+0000 7f5887a00640 1 Processor -- start 2024-04-24T06:37:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.923+0000 7f5887a00640 1 -- start start 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5887a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800cf360 0x7f58800cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5887a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f58800d1160 0x7f58800d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5887a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58800cf7e0 con 0x7f588015b830 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5887a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f58800d1b30 con 0x7f58800d1160 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5887a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f588015c490 con 0x7f58800cf360 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5885a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f58800d1160 0x7f58800d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5885a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f58800d1160 0x7f58800d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33312/0 (socket says 172.21.15.59:33312) 2024-04-24T06:37:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.924+0000 7f5885a00640 1 -- 172.21.15.59:0/2318423254 learned_addr learned my addr 172.21.15.59:0/2318423254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.925+0000 7f587fe00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800cf360 0x7f58800cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.925+0000 7f5885a00640 1 -- 172.21.15.59:0/2318423254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800cf360 msgr2=0x7f58800cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.925+0000 7f5885a00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800cf360 0x7f58800cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.925+0000 7f5885a00640 1 -- 172.21.15.59:0/2318423254 >> v1:172.21.15.66:6789/0 conn(0x7f588015b830 legacy=0x7f588015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.925+0000 7f5885a00640 1 -- 172.21.15.59:0/2318423254 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588015c770 con 0x7f58800d1160 2024-04-24T06:37:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.926+0000 7f5885a00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58800d1160 0x7f58800d15f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f587802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed874ce46578ed55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.926+0000 7f587f400640 1 -- 172.21.15.59:0/2318423254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878036070 con 0x7f58800d1160 2024-04-24T06:37:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.926+0000 7f587f400640 1 -- 172.21.15.59:0/2318423254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878031600 con 0x7f58800d1160 2024-04-24T06:37:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.926+0000 7f587f400640 1 -- 172.21.15.59:0/2318423254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780319d0 con 0x7f58800d1160 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 -- 172.21.15.59:0/2318423254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58800d1160 msgr2=0x7f58800d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58800d1160 0x7f58800d15f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f587802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 -- 172.21.15.59:0/2318423254 shutdown_connections 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800cf360 0x7f58800cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 --2- 172.21.15.59:0/2318423254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58800d1160 0x7f58800d15f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 -- 172.21.15.59:0/2318423254 >> 172.21.15.59:0/2318423254 conn(0x7f5880073430 msgr2=0x7f588009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 -- 172.21.15.59:0/2318423254 shutdown_connections 2024-04-24T06:37:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.927+0000 7f5887a00640 1 -- 172.21.15.59:0/2318423254 wait complete. 2024-04-24T06:37:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.929+0000 7f5887a00640 1 Processor -- start 2024-04-24T06:37:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.929+0000 7f5887a00640 1 -- start start 2024-04-24T06:37:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58800cf360 0x7f588015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800d1160 0x7f58800c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58800cc500 con 0x7f588015b830 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58800cdd70 con 0x7f58800cf360 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5887a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f58800cc680 con 0x7f58800d1160 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5885a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5885a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33320/0 (socket says 172.21.15.59:33320) 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f5885a00640 1 -- 172.21.15.59:0/621686698 learned_addr learned my addr 172.21.15.59:0/621686698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.930+0000 7f587fe00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800d1160 0x7f58800c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 -- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800d1160 msgr2=0x7f58800c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800d1160 0x7f58800c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 -- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58800cf360 msgr2=0x7f588015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58800cf360 0x7f588015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 -- 172.21.15.59:0/621686698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588015ae40 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5885a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f5878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878036050 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878042070 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878009510 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58800c5050 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.931+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58800c5310 con 0x7f588015b830 2024-04-24T06:37:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.933+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5880069190 con 0x7f588015b830 2024-04-24T06:37:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.934+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5878007bf0 con 0x7f588015b830 2024-04-24T06:37:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.935+0000 7f587cc00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 0x7f586006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.935+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f58780acb20 con 0x7f588015b830 2024-04-24T06:37:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.935+0000 7f5885000640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 0x7f586006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.936+0000 7f5885000640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 0x7f586006edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5874006fd0 tx=0x7f5874006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:32.940+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58800cf7e0 con 0x7f588015b830 2024-04-24T06:37:33.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.144+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f588006f140 con 0x7f588015b830 2024-04-24T06:37:33.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.145+0000 7f587cc00640 1 -- 172.21.15.59:0/621686698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f587807aee0 con 0x7f588015b830 2024-04-24T06:37:33.147 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:37:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.147+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 msgr2=0x7f586006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.147+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 0x7f586006edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5874006fd0 tx=0x7f5874006bc0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.147+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 msgr2=0x7f588015dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.147+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5878009e50 tx=0x7f5878004a40 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 shutdown_connections 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f586006c8f0 0x7f586006edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588015b830 0x7f588015dce0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58800d1160 0x7f58800c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 --2- 172.21.15.59:0/621686698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58800cf360 0x7f588015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 >> 172.21.15.59:0/621686698 conn(0x7f5880073430 msgr2=0x7f588006c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 shutdown_connections 2024-04-24T06:37:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.148+0000 7f5887a00640 1 -- 172.21.15.59:0/621686698 wait complete. 2024-04-24T06:37:33.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:37:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.383+0000 7f480d400640 1 Processor -- start 2024-04-24T06:37:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.383+0000 7f480d400640 1 -- start start 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48080d0830 0x7f48080cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 0x7f4808163bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080af010 0x7f48080af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4808164110 con 0x7f48080d0830 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f48081676a0 con 0x7f48080d0c10 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4808166b20 con 0x7f48080af010 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f4807e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48080d0830 0x7f48080cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f4807e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48080d0830 0x7f48080cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33334/0 (socket says 172.21.15.59:33334) 2024-04-24T06:37:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f4807e00640 1 -- 172.21.15.59:0/1269836927 learned_addr learned my addr 172.21.15.59:0/1269836927 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f4807400640 1 --2- 172.21.15.59:0/1269836927 >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 0x7f4808163bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.385+0000 7f480ca00640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080af010 0x7f48080af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807e00640 1 -- 172.21.15.59:0/1269836927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080af010 msgr2=0x7f48080af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807e00640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080af010 0x7f48080af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807e00640 1 -- 172.21.15.59:0/1269836927 >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 msgr2=0x7f4808163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807e00640 1 --2- 172.21.15.59:0/1269836927 >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 0x7f4808163bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807e00640 1 -- 172.21.15.59:0/1269836927 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4808166e00 con 0x7f48080d0830 2024-04-24T06:37:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.386+0000 7f4807400640 1 --2- 172.21.15.59:0/1269836927 >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 0x7f4808163bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.387+0000 7f4807e00640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080d0830 0x7f48080cefd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f48080d1270 tx=0x7f47f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c17fcb5c541419d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.387+0000 7f4806a00640 1 -- 172.21.15.59:0/1269836927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8037070 con 0x7f48080d0830 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.387+0000 7f4806a00640 1 -- 172.21.15.59:0/1269836927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f8032440 con 0x7f48080d0830 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.387+0000 7f4806a00640 1 -- 172.21.15.59:0/1269836927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8032810 con 0x7f48080d0830 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 -- 172.21.15.59:0/1269836927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080d0830 msgr2=0x7f48080cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080d0830 0x7f48080cefd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f48080d1270 tx=0x7f47f802a140 comp rx=0 tx=0).stop 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 -- 172.21.15.59:0/1269836927 shutdown_connections 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080af010 0x7f48080af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 --2- 172.21.15.59:0/1269836927 >> v2:172.21.15.66:3300/0 conn(0x7f48080d0c10 0x7f4808163bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 --2- 172.21.15.59:0/1269836927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080d0830 0x7f48080cefd0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 -- 172.21.15.59:0/1269836927 >> 172.21.15.59:0/1269836927 conn(0x7f480806bdf0 msgr2=0x7f480806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 -- 172.21.15.59:0/1269836927 shutdown_connections 2024-04-24T06:37:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.388+0000 7f480d400640 1 -- 172.21.15.59:0/1269836927 wait complete. 2024-04-24T06:37:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.390+0000 7f480d400640 1 Processor -- start 2024-04-24T06:37:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.390+0000 7f480d400640 1 -- start start 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48080d0830 0x7f480806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080d0c10 0x7f480815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48080af990 con 0x7f48080af010 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f48081673c0 con 0x7f48080d0830 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f480d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f48080b45f0 con 0x7f48080d0c10 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33336/0 (socket says 172.21.15.59:33336) 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 -- 172.21.15.59:0/57780318 learned_addr learned my addr 172.21.15.59:0/57780318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 -- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080d0c10 msgr2=0x7f480815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080d0c10 0x7f480815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 -- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48080d0830 msgr2=0x7f480806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48080d0830 0x7f480806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.391+0000 7f4807e00640 1 -- 172.21.15.59:0/57780318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480815cac0 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f4807e00640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f47f802a180 tx=0x7f47f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8037040 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f8042070 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8030bb0 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480806f0a0 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.392+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480815d020 con 0x7f48080af010 2024-04-24T06:37:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.394+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080aedf0 con 0x7f48080af010 2024-04-24T06:37:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.394+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47f8047460 con 0x7f48080af010 2024-04-24T06:37:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.395+0000 7f47ffe00640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.395+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f47f80ac7e0 con 0x7f48080af010 2024-04-24T06:37:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.396+0000 7f4807400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.396+0000 7f4807400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 0x7f47e406edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006b10 tx=0x7f47f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.400+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48080aedf0 con 0x7f48080af010 2024-04-24T06:37:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.618+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f480804fd90 con 0x7f48080af010 2024-04-24T06:37:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.619+0000 7f47ffe00640 1 -- 172.21.15.59:0/57780318 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v86) ==== 80+0+120 (secure 0 0 0) 0x7f480815cac0 con 0x7f48080af010 2024-04-24T06:37:33.621 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:37:33.621 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6832/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:37:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:37:33.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 msgr2=0x7f47e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 0x7f47e406edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006b10 tx=0x7f47f0008040 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 msgr2=0x7f4808156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f47f802a180 tx=0x7f47f80046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 shutdown_connections 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f47e406c8f0 0x7f47e406edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f48080d0c10 0x7f480815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48080d0830 0x7f480806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 --2- 172.21.15.59:0/57780318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48080af010 0x7f4808156bf0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.621+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 >> 172.21.15.59:0/57780318 conn(0x7f480806bdf0 msgr2=0x7f48080cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.622+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 shutdown_connections 2024-04-24T06:37:33.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:33.622+0000 7f480d400640 1 -- 172.21.15.59:0/57780318 wait complete. 2024-04-24T06:37:33.643 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:37:33.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:33.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:33.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64651]: fuse finished with error 0 and tester_r 0 2024-04-24T06:37:33.733 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:37:33.832 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:37:33.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:33.833 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:37:33.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:37:34.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.089+0000 7efc48200640 1 Processor -- start 2024-04-24T06:37:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.089+0000 7efc48200640 1 -- start start 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.090+0000 7efc48200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc4015cb20 0x7efc400cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.090+0000 7efc48200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc40163770 0x7efc400d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc48200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc400d01c0 con 0x7efc40160180 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc48200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efc4015cf00 con 0x7efc40163770 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc48200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc4015f6a0 con 0x7efc4015cb20 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc44e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc40163770 0x7efc400d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc46200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efc40160180 0x7efc400b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54450/0 (socket says 172.21.15.59:54450) 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc46200640 1 -- 172.21.15.59:0/55426548 learned_addr learned my addr 172.21.15.59:0/55426548 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc45800640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc4015cb20 0x7efc400cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc3fe00640 1 -- 172.21.15.59:0/55426548 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1161965569 0 0) 0x7efc400d01c0 con 0x7efc40160180 2024-04-24T06:37:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc44e00640 1 -- 172.21.15.59:0/55426548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc4015cb20 msgr2=0x7efc400cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc44e00640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc4015cb20 0x7efc400cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc44e00640 1 -- 172.21.15.59:0/55426548 >> v1:172.21.15.59:6789/0 conn(0x7efc40160180 legacy=0x7efc400b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.091+0000 7efc44e00640 1 -- 172.21.15.59:0/55426548 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4015f980 con 0x7efc40163770 2024-04-24T06:37:34.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc44e00640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc400d1240 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0097a0 tx=0x7efc2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b91532ab7d008 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc3fe00640 1 -- 172.21.15.59:0/55426548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c037070 con 0x7efc40163770 2024-04-24T06:37:34.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc3fe00640 1 -- 172.21.15.59:0/55426548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c02ae50 con 0x7efc40163770 2024-04-24T06:37:34.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc3fe00640 1 -- 172.21.15.59:0/55426548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c032660 con 0x7efc40163770 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc48200640 1 -- 172.21.15.59:0/55426548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 msgr2=0x7efc400d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.093+0000 7efc48200640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc400d1240 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0097a0 tx=0x7efc2c02a100 comp rx=0 tx=0).stop 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 -- 172.21.15.59:0/55426548 shutdown_connections 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc400d1240 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 --2- 172.21.15.59:0/55426548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc4015cb20 0x7efc400cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 -- 172.21.15.59:0/55426548 >> 172.21.15.59:0/55426548 conn(0x7efc4006cc70 msgr2=0x7efc4006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 -- 172.21.15.59:0/55426548 shutdown_connections 2024-04-24T06:37:34.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.094+0000 7efc48200640 1 -- 172.21.15.59:0/55426548 wait complete. 2024-04-24T06:37:34.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.095+0000 7efc48200640 1 Processor -- start 2024-04-24T06:37:34.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.096+0000 7efc48200640 1 -- start start 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4015cb20 0x7efc401e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc40160180 0x7efc401ea670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc400d1780 con 0x7efc40163770 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc400d1900 con 0x7efc4015cb20 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc48200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efc400d1a80 con 0x7efc40160180 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33362/0 (socket says 172.21.15.59:33362) 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 -- 172.21.15.59:0/2466358840 learned_addr learned my addr 172.21.15.59:0/2466358840 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:34.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc44e00640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc40160180 0x7efc401ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc45800640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4015cb20 0x7efc401e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 -- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc40160180 msgr2=0x7efc401ea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc40160180 0x7efc401ea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 -- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4015cb20 msgr2=0x7efc401e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.097+0000 7efc46200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4015cb20 0x7efc401e9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc46200640 1 -- 172.21.15.59:0/2466358840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc401e8c60 con 0x7efc40163770 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc46200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c480 tx=0x7efc3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38018070 con 0x7efc40163770 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc38007500 con 0x7efc40163770 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38004a20 con 0x7efc40163770 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401e8f40 con 0x7efc40163770 2024-04-24T06:37:34.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.098+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401e9500 con 0x7efc40163770 2024-04-24T06:37:34.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.099+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40068a80 con 0x7efc40163770 2024-04-24T06:37:34.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.100+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc38004bc0 con 0x7efc40163770 2024-04-24T06:37:34.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.101+0000 7efc3d600640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.101+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7efc3808ae60 con 0x7efc40163770 2024-04-24T06:37:34.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.102+0000 7efc45800640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.102+0000 7efc45800640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 0x7efc2006edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efc30009f30 tx=0x7efc3000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.106+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc38059230 con 0x7efc40163770 2024-04-24T06:37:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.341+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc4015c2d0 con 0x7efc40163770 2024-04-24T06:37:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.342+0000 7efc3d600640 1 -- 172.21.15.59:0/2466358840 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5182 (secure 0 0 0) 0x7efc38058b30 con 0x7efc40163770 2024-04-24T06:37:34.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:34.344 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:36:31.666787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7548":{"gid":7548,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6835/2792490076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2792490076},{"type":"v1","addr":"172.21.15.59:6835","nonce":2792490076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7548,"qdb_cluster":[7548]},"id":6}]} 2024-04-24T06:37:34.344 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 68 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 msgr2=0x7efc2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 0x7efc2006edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efc30009f30 tx=0x7efc3000b040 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 msgr2=0x7efc40159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c480 tx=0x7efc3800c940 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 shutdown_connections 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc40163770 0x7efc40159220 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efc40160180 0x7efc401ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 --2- 172.21.15.59:0/2466358840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc4015cb20 0x7efc401e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 >> 172.21.15.59:0/2466358840 conn(0x7efc4006cc70 msgr2=0x7efc4015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 shutdown_connections 2024-04-24T06:37:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.345+0000 7efc48200640 1 -- 172.21.15.59:0/2466358840 wait complete. 2024-04-24T06:37:34.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-24T06:37:34.609 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-24T06:37:34.609 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:37:34.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:37:34.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.821+0000 7fcd1ce00640 1 Processor -- start 2024-04-24T06:37:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.821+0000 7fcd1ce00640 1 -- start start 2024-04-24T06:37:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.822+0000 7fcd1ce00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcd1815c670 0x7fcd180cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd1ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd1ce00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd180b4cf0 con 0x7fcd180b4230 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd1ce00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1815ca50 con 0x7fcd1815c670 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd1ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcd180ccb80 con 0x7fcd180d0cd0 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56904/0 (socket says 172.21.15.59:56904) 2024-04-24T06:37:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.823+0000 7fcd17e00640 1 -- 172.21.15.59:0/1566427332 learned_addr learned my addr 172.21.15.59:0/1566427332 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.824+0000 7fcd17400640 1 --2- 172.21.15.59:0/1566427332 >> v2:172.21.15.66:3300/0 conn(0x7fcd1815c670 0x7fcd180cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.824+0000 7fcd17e00640 1 -- 172.21.15.59:0/1566427332 >> v2:172.21.15.66:3300/0 conn(0x7fcd1815c670 msgr2=0x7fcd180cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.824+0000 7fcd17e00640 1 --2- 172.21.15.59:0/1566427332 >> v2:172.21.15.66:3300/0 conn(0x7fcd1815c670 0x7fcd180cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.824+0000 7fcd17e00640 1 -- 172.21.15.59:0/1566427332 >> v1:172.21.15.66:6789/0 conn(0x7fcd180b4230 legacy=0x7fcd180d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.824+0000 7fcd17e00640 1 -- 172.21.15.59:0/1566427332 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd180ccdd0 con 0x7fcd180d0cd0 2024-04-24T06:37:34.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.825+0000 7fcd17e00640 1 --2- 172.21.15.59:0/1566427332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd180ccd00 tx=0x7fcd1002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58bfbbe8c495edfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.826+0000 7fcd16a00640 1 -- 172.21.15.59:0/1566427332 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10036070 con 0x7fcd180d0cd0 2024-04-24T06:37:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.826+0000 7fcd16a00640 1 -- 172.21.15.59:0/1566427332 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd10031600 con 0x7fcd180d0cd0 2024-04-24T06:37:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.826+0000 7fcd16a00640 1 -- 172.21.15.59:0/1566427332 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100319d0 con 0x7fcd180d0cd0 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 -- 172.21.15.59:0/1566427332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 msgr2=0x7fcd180b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/1566427332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd180ccd00 tx=0x7fcd1002a710 comp rx=0 tx=0).stop 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 -- 172.21.15.59:0/1566427332 shutdown_connections 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/1566427332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd180b4690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/1566427332 >> v2:172.21.15.66:3300/0 conn(0x7fcd1815c670 0x7fcd180cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 -- 172.21.15.59:0/1566427332 >> 172.21.15.59:0/1566427332 conn(0x7fcd1806bdf0 msgr2=0x7fcd1806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 -- 172.21.15.59:0/1566427332 shutdown_connections 2024-04-24T06:37:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.827+0000 7fcd1ce00640 1 -- 172.21.15.59:0/1566427332 wait complete. 2024-04-24T06:37:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.829+0000 7fcd1ce00640 1 Processor -- start 2024-04-24T06:37:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.829+0000 7fcd1ce00640 1 -- start start 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd180b4230 0x7fcd18154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd1815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd180b3e60 con 0x7fcd1815c670 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd180cd390 con 0x7fcd180b4230 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd1ce00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcd180b3fe0 con 0x7fcd180d0cd0 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33376/0 (socket says 172.21.15.59:33376) 2024-04-24T06:37:34.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd17e00640 1 -- 172.21.15.59:0/2866703976 learned_addr learned my addr 172.21.15.59:0/2866703976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:34.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.830+0000 7fcd0fe00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd1815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 -- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 msgr2=0x7fcd1815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd1815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 -- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd180b4230 msgr2=0x7fcd18154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd180b4230 0x7fcd18154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 -- 172.21.15.59:0/2866703976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd18154f30 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd17e00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009f40 tx=0x7fcd10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10036050 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd10042070 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10009510 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd180b03d0 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.831+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd180b0990 con 0x7fcd1815c670 2024-04-24T06:37:34.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.833+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd180b4690 con 0x7fcd1815c670 2024-04-24T06:37:34.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.833+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd10007bd0 con 0x7fcd1815c670 2024-04-24T06:37:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.834+0000 7fcd0f400640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 0x7fccf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.834+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fcd100ac950 con 0x7fcd1815c670 2024-04-24T06:37:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.835+0000 7fcd17400640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 0x7fccf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.835+0000 7fcd17400640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 0x7fccf406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcd08006a90 tx=0x7fcd08006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:34.839+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd180b4690 con 0x7fcd1815c670 2024-04-24T06:37:35.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.074+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd180ceab0 con 0x7fcd1815c670 2024-04-24T06:37:35.076 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:37:35.075+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:37:35.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.078+0000 7fcd0f400640 1 -- 172.21.15.59:0/2866703976 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7fcd10045360 con 0x7fcd1815c670 2024-04-24T06:37:35.080 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:37:35.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 msgr2=0x7fccf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 0x7fccf406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcd08006a90 tx=0x7fcd08006a20 comp rx=0 tx=0).stop 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 msgr2=0x7fcd1815ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009f40 tx=0x7fcd10004a40 comp rx=0 tx=0).stop 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 shutdown_connections 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fccf406c8f0 0x7fccf406edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1815c670 0x7fcd1815ac80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcd180d0cd0 0x7fcd1815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 --2- 172.21.15.59:0/2866703976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd180b4230 0x7fcd18154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.081+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 >> 172.21.15.59:0/2866703976 conn(0x7fcd1806bdf0 msgr2=0x7fcd18155840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.082+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 shutdown_connections 2024-04-24T06:37:35.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.082+0000 7fcd1ce00640 1 -- 172.21.15.59:0/2866703976 wait complete. 2024-04-24T06:37:35.101 INFO:tasks.ceph.mds.c.smithi059.stderr: -4570> 2024-04-24T06:36:24.799+0000 7f14d0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:37:35.102 INFO:tasks.ceph.mds.c.smithi059.stderr: -4388> 2024-04-24T06:36:25.064+0000 7f14cb200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-24T06:37:35.102 INFO:tasks.ceph.mds.c.smithi059.stderr: -4207> 2024-04-24T06:36:25.343+0000 7f14cb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0) n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x56308c7d6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-24T06:37:35.103 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-24T06:37:35.110 INFO:tasks.ceph.mds.c.smithi059.stderr: -2412> 2024-04-24T06:36:57.472+0000 7f14cb200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)->f(v1 m2024-04-24T06:36:04.008555+0000) n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)->n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56308c7d6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-24T06:36:04.008555+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T06:36:04.148587+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-24T06:37:35.159 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:37:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.193+0000 7fce8c727740 1 -- start start 2024-04-24T06:37:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.193+0000 7fce8c727740 1 Processor -- start 2024-04-24T06:37:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55798fd32c00 0x557990a9e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 0x557990a9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557990aa2180 con 0x55798fd33c00 2024-04-24T06:37:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557990aa2300 con 0x55798fd33800 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.194+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557990aa2480 con 0x55798fd32c00 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33390/0 (socket says 172.21.15.59:33390) 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 -- 172.21.15.59:0/3236801533 learned_addr learned my addr 172.21.15.59:0/3236801533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce89400640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 0x557990a9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 -- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55798fd32c00 msgr2=0x557990a9e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce89e00640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55798fd32c00 0x557990a9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55798fd32c00 0x557990a9e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 -- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 msgr2=0x557990a9e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 0x557990a9e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 -- 172.21.15.59:0/3236801533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798fd5f680 con 0x55798fd33c00 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce89400640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 0x557990a9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.195+0000 7fce8a800640 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x557990b4fcb0 tx=0x557990ad6180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1301b3eb5d8fd9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce86c00640 1 -- 172.21.15.59:0/3236801533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd58680 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce86c00640 1 -- 172.21.15.59:0/3236801533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55798fd7c540 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce86c00640 1 -- 172.21.15.59:0/3236801533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd58b60 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798fd5f860 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55798fd5f2c0 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 4 waiting for MDS map... 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce86c00640 1 -- 172.21.15.59:0/3236801533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x557990c08600 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce86c00640 1 -- 172.21.15.59:0/3236801533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55798fd8b180 con 0x55798fd33c00 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 4 Got MDS map 69 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 4 datascan.main: Using metadata pool output 2024-04-24T06:37:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.196+0000 7fce8c727740 4 datascan.main: connecting to RADOS... 2024-04-24T06:37:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.197+0000 7fce8c727740 1 Processor -- start 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.197+0000 7fce8c727740 1 -- start start 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b400 0x557990a9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b800 0x557990a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557990aa2600 con 0x557990c0bc00 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557990aa2780 con 0x557990c0b400 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557990aa2900 con 0x557990c0b800 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33392/0 (socket says 172.21.15.59:33392) 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 -- 172.21.15.59:0/2531883014 learned_addr learned my addr 172.21.15.59:0/2531883014 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 -- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b800 msgr2=0x557990a9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b800 0x557990a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 -- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b400 msgr2=0x557990a9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b400 0x557990a9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.198+0000 7fce89400640 1 -- 172.21.15.59:0/2531883014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798fd5f4a0 con 0x557990c0bc00 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce89400640 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x557990b4f1d0 tx=0x557990ad26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb1de3e10314a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce84400640 1 -- 172.21.15.59:0/2531883014 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd589c0 con 0x557990c0bc00 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce84400640 1 -- 172.21.15.59:0/2531883014 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55798fd7ce00 con 0x557990c0bc00 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce84400640 1 -- 172.21.15.59:0/2531883014 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd58680 con 0x557990c0bc00 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 -- 172.21.15.59:0/2531883014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 msgr2=0x557990a9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x557990b4f1d0 tx=0x557990ad26c0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 -- 172.21.15.59:0/2531883014 shutdown_connections 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b800 0x557990a9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 --2- 172.21.15.59:0/2531883014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b400 0x557990a9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.199+0000 7fce8c727740 1 -- 172.21.15.59:0/2531883014 >> 172.21.15.59:0/2531883014 conn(0x557990c0b000 msgr2=0x55798fe01000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.200+0000 7fce8c727740 1 -- 172.21.15.59:0/2531883014 shutdown_connections 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.200+0000 7fce8c727740 1 -- 172.21.15.59:0/2531883014 wait complete. 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.200+0000 7fce8c727740 1 Processor -- start 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.200+0000 7fce8c727740 1 -- start start 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b400 0x557990a9f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b800 0x557990a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557990aa3b00 con 0x557990c0bc00 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557990aa3980 con 0x557990c0b800 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce8c727740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557990aa3800 con 0x557990c0b400 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce89400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce89400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33398/0 (socket says 172.21.15.59:33398) 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.201+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 learned_addr learned my addr 172.21.15.59:0/471419889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b400 msgr2=0x557990a9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557990c0b400 0x557990a9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b800 msgr2=0x557990a9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557990c0b800 0x557990a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798fd5f680 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x557990c1db00 tx=0x557990ad2420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce82600640 1 -- 172.21.15.59:0/471419889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd58b60 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce82600640 1 -- 172.21.15.59:0/471419889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55798fd7cc40 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce82600640 1 -- 172.21.15.59:0/471419889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55798fd584e0 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55798fd5f860 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798fd5f0e0 con 0x557990c0bc00 2024-04-24T06:37:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.202+0000 7fce8c727740 4 datascan.init: resolving metadata pool 12 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.204+0000 7fce82600640 1 -- 172.21.15.59:0/471419889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x557990c6ec00 con 0x557990c0bc00 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce82600640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 0x557990aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce82600640 1 -- 172.21.15.59:0/471419889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x557990b32600 con 0x557990c0bc00 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce8c727740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce8c727740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce8c727740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5579900a6400 0x557990aa0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900a6800 con 0x5579900a6400 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 0x557990aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 0x557990aa0100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x557990ad4ea0 tx=0x557990c20f60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.205+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5579900a6400 0x557990aa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.206+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5579900a6400 0x557990aa0680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.207+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.207+0000 7fce83a00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5579900b2000 0x5579900b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.208+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b2400 con 0x5579900b2000 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.208+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5579900b2000 0x5579900b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.209+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5579900b2000 0x5579900b8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.215+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.216+0000 7fce84400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5579900b2c00 0x5579900b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.216+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3000 con 0x5579900b2c00 2024-04-24T06:37:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.216+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5579900b2c00 0x5579900b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.217+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5579900b2c00 0x5579900b8580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.217+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900b2c00 2024-04-24T06:37:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.217+0000 7fce83a00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5579900b2800 0x5579900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.217+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900b2800 2024-04-24T06:37:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.218+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5579900b2800 0x5579900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.219+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5579900b2800 0x5579900b8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.225+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.225+0000 7fce84400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x5579900b3400 0x5579900b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.225+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6000 con 0x5579900b3400 2024-04-24T06:37:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.226+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x5579900b3400 0x5579900b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.226+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x5579900b3400 0x5579900b9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.229+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.229+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3c00 con 0x5579900b3400 2024-04-24T06:37:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.229+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(6 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.229+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6400 con 0x5579900a6400 2024-04-24T06:37:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.230+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.230+0000 7fce83a00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5579900d6c00 0x5579900b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.230+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7000 con 0x5579900d6c00 2024-04-24T06:37:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.231+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5579900d6c00 0x5579900b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.232+0000 7fce89e00640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5579900d6c00 0x5579900b9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.238+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(8 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.238+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6800 con 0x5579900b2000 2024-04-24T06:37:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.240+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.240+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7400 con 0x5579900b2000 2024-04-24T06:37:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.241+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.241+0000 7fce84400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5579900d6000 0x5579900b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.241+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b2400 con 0x5579900d6000 2024-04-24T06:37:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.241+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5579900d6000 0x5579900b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.242+0000 7fce89400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5579900d6000 0x5579900b9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.244+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(11 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900d6000 2024-04-24T06:37:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.244+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7800 con 0x5579900b2000 2024-04-24T06:37:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.245+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.245+0000 7fce84400640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5579900b3c00 0x5579900ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.245+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990c0a800 con 0x5579900b3c00 2024-04-24T06:37:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.246+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5579900b3c00 0x5579900ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.247+0000 7fce8a800640 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5579900b3c00 0x5579900ba100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.249+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(13 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.249+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6400 con 0x5579900b3400 2024-04-24T06:37:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.249+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 3 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.250+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6800 con 0x5579900b2000 2024-04-24T06:37:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.250+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(15 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.251+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900b3400 2024-04-24T06:37:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.251+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 4 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.251+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7c00 con 0x5579900d6c00 2024-04-24T06:37:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.252+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.252+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b2400 con 0x5579900b3c00 2024-04-24T06:37:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.253+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.253+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900d6000 2024-04-24T06:37:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.253+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900d6000 2024-04-24T06:37:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.253+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900a6800 con 0x5579900b3400 2024-04-24T06:37:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.254+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 5 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.254+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7400 con 0x5579900b3400 2024-04-24T06:37:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.254+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 6 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.254+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990ad1400 con 0x5579900b3c00 2024-04-24T06:37:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.255+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(22 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.255+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7800 con 0x5579900b3c00 2024-04-24T06:37:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.255+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(23 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.255+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec000 con 0x5579900d6c00 2024-04-24T06:37:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.256+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(24 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.256+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6400 con 0x5579900b2800 2024-04-24T06:37:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.257+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.258+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec400 con 0x5579900a6400 2024-04-24T06:37:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.259+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.259+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6800 con 0x5579900b2c00 2024-04-24T06:37:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.260+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(27 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900b2c00 2024-04-24T06:37:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.260+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec800 con 0x5579900b2000 2024-04-24T06:37:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.261+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 6 ==== osd_op_reply(28 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.261+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990c0a800 con 0x5579900a6400 2024-04-24T06:37:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.262+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.262+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ecc00 con 0x5579900b3400 2024-04-24T06:37:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.263+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 7 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.263+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7400 con 0x5579900b2800 2024-04-24T06:37:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.264+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(31 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.264+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ed000 con 0x5579900b3c00 2024-04-24T06:37:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.265+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(32 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.265+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7800 con 0x5579900b2000 2024-04-24T06:37:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.266+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 7 ==== osd_op_reply(33 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.266+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ed400 con 0x5579900b2800 2024-04-24T06:37:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.267+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(34 [pgnls start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.267+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7000 con 0x5579900b3c00 2024-04-24T06:37:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.268+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(35 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.268+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec400 con 0x5579900b2c00 2024-04-24T06:37:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.269+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(36 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900b2c00 2024-04-24T06:37:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.269+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b2400 con 0x5579900d6c00 2024-04-24T06:37:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.270+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(37 [pgnls start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.270+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ed400 con 0x5579900b2000 2024-04-24T06:37:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.271+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 8 ==== osd_op_reply(38 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.271+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3000 con 0x5579900b2c00 2024-04-24T06:37:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.272+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(39 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900b2c00 2024-04-24T06:37:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.272+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d6400 con 0x5579900b3400 2024-04-24T06:37:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.272+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 8 ==== osd_op_reply(40 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.272+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990ad1400 con 0x5579900b2800 2024-04-24T06:37:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.273+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(41 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.274+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7400 con 0x5579900b2800 2024-04-24T06:37:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.275+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 6 ==== osd_op_reply(42 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.275+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900a6800 con 0x5579900b2800 2024-04-24T06:37:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.276+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 7 ==== osd_op_reply(43 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.276+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7c00 con 0x5579900b3c00 2024-04-24T06:37:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.277+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(44 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.277+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec800 con 0x5579900b2800 2024-04-24T06:37:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.278+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 8 ==== osd_op_reply(45 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.278+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990c0a800 con 0x5579900d6000 2024-04-24T06:37:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.279+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(46 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900d6000 2024-04-24T06:37:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.279+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ecc00 con 0x5579900a6400 2024-04-24T06:37:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.280+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 5 ==== osd_op_reply(47 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.280+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900a6400 2024-04-24T06:37:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.281+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 6 ==== osd_op_reply(48 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.281+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec000 con 0x5579900d6000 2024-04-24T06:37:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.282+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(49 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900d6000 2024-04-24T06:37:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.282+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900b3c00 2024-04-24T06:37:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.283+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(50 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.283+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ed800 con 0x5579900a6400 2024-04-24T06:37:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.284+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 7 ==== osd_op_reply(51 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.284+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ec800 con 0x5579900a6400 2024-04-24T06:37:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.285+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 8 ==== osd_op_reply(52 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.285+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900edc00 con 0x5579900d6c00 2024-04-24T06:37:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.286+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(53 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.286+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ecc00 con 0x5579900b3400 2024-04-24T06:37:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.287+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.2 v2:172.21.15.59:6809/1388377934 9 ==== osd_op_reply(54 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3400 2024-04-24T06:37:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.287+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900ed000 con 0x5579900b2800 2024-04-24T06:37:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.288+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 9 ==== osd_op_reply(55 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.288+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990c0a800 con 0x5579900b2800 2024-04-24T06:37:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.290+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 10 ==== osd_op_reply(56 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.290+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3800 con 0x5579900a6400 2024-04-24T06:37:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.291+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 9 ==== osd_op_reply(57 [pgnls start_epoch 87 out=81b] v83'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.291+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5579900a6c00 con 0x5579900a6400 2024-04-24T06:37:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.292+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.292+0000 7fce8c727740 10 datascan.operator(): handling object 1099511627777.0 2024-04-24T06:37:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.292+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5579900a7000 con 0x5579900a6400 2024-04-24T06:37:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.298+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.7 v2:172.21.15.66:6824/607700871 11 ==== osd_op_reply(59 10000000001.00000000 [call] v87'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x557990ac0000 con 0x5579900a6400 2024-04-24T06:37:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.298+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900a7400 con 0x5579900b3c00 2024-04-24T06:37:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.298+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.1 v2:172.21.15.59:6800/2375715406 9 ==== osd_op_reply(60 [pgnls start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b3c00 2024-04-24T06:37:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.298+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b2400 con 0x5579900d6c00 2024-04-24T06:37:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.299+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(61 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.299+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900a6800 con 0x5579900d6c00 2024-04-24T06:37:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.300+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 <== osd.5 v2:172.21.15.66:6816/79236266 7 ==== osd_op_reply(62 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0780 con 0x5579900d6c00 2024-04-24T06:37:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.300+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900b3000 con 0x5579900b2000 2024-04-24T06:37:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.301+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 9 ==== osd_op_reply(63 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.301+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7400 con 0x5579900b2800 2024-04-24T06:37:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.303+0000 7fce89400640 1 -- 172.21.15.59:0/471419889 <== osd.4 v2:172.21.15.66:6800/4254676376 11 ==== osd_op_reply(64 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0000 con 0x5579900b2800 2024-04-24T06:37:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.303+0000 7fce84400640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5579900d7800 con 0x5579900b2000 2024-04-24T06:37:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.304+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 10 ==== osd_op_reply(65 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.304+0000 7fce83a00640 1 -- 172.21.15.59:0/471419889 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x557990ad1400 con 0x5579900b2000 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8a800640 1 -- 172.21.15.59:0/471419889 <== osd.6 v2:172.21.15.66:6808/280071528 11 ==== osd_op_reply(66 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557990ac0500 con 0x5579900b2000 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5579900d6000 msgr2=0x5579900b9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5579900d6000 0x5579900b9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5579900b3c00 msgr2=0x5579900ba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5579900b3c00 0x5579900ba100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x5579900b3400 msgr2=0x5579900b9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x5579900b3400 0x5579900b9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5579900b2c00 msgr2=0x5579900b8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 reap_dead start 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x5579900b2c00 0x5579900b8580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5579900b2800 msgr2=0x5579900b8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5579900b2800 0x5579900b8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5579900d6c00 msgr2=0x5579900b9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x5579900d6c00 0x5579900b9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5579900b2000 msgr2=0x5579900b8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x5579900b2000 0x5579900b8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5579900a6400 msgr2=0x557990aa0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5579900a6400 0x557990aa0680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.305+0000 7fce89e00640 1 -- 172.21.15.59:0/471419889 reap_dead start 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 msgr2=0x557990aa0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 0x557990aa0100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x557990ad4ea0 tx=0x557990c20f60 comp rx=0 tx=0).stop 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 msgr2=0x557990a9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x557990c1db00 tx=0x557990ad2420 comp rx=0 tx=0).stop 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 shutdown_connections 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5579900a6000 0x557990aa0100 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 --2- 172.21.15.59:0/471419889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557990c0bc00 0x557990a9fb80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.306+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 >> 172.21.15.59:0/471419889 conn(0x557990c0b000 msgr2=0x55798fe01000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.307+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 shutdown_connections 2024-04-24T06:37:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.307+0000 7fce8c727740 1 -- 172.21.15.59:0/471419889 wait complete. 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.307+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 msgr2=0x557990a9eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.307+0000 7fce8c727740 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x557990b4fcb0 tx=0x557990ad6180 comp rx=0 tx=0).stop 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 shutdown_connections 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798fd33c00 0x557990a9eb00 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798fd33800 0x557990a9e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 --2- 172.21.15.59:0/3236801533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55798fd32c00 0x557990a9e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 >> 172.21.15.59:0/3236801533 conn(0x55798fd33000 msgr2=0x55798fe00800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 shutdown_connections 2024-04-24T06:37:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.308+0000 7fce8c727740 1 -- 172.21.15.59:0/3236801533 wait complete. 2024-04-24T06:37:35.319 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.216482, result: 2024-04-24T06:37:35.319 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-24T06:37:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.413+0000 7f6a8ef2e740 1 -- start start 2024-04-24T06:37:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.413+0000 7f6a8ef2e740 1 Processor -- start 2024-04-24T06:37:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 0x55e508b2f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508b3e800 0x55e508b2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac6000 con 0x55e508b3e000 2024-04-24T06:37:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac6180 con 0x55e507d53c00 2024-04-24T06:37:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.414+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e508ac6300 con 0x55e508b3e800 2024-04-24T06:37:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33416/0 (socket says 172.21.15.59:33416) 2024-04-24T06:37:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 -- 172.21.15.59:0/1048113157 learned_addr learned my addr 172.21.15.59:0/1048113157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 0x55e508b2f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 -- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508b3e800 msgr2=0x55e508b2eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508b3e800 0x55e508b2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 -- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 msgr2=0x55e508b2f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 0x55e508b2f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 -- 172.21.15.59:0/1048113157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e507d7f680 con 0x55e508b3e000 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 0x55e508b2f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.415+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55e508b7fc80 tx=0x55e508af6120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8ccc7bcd57f560 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a89400640 1 -- 172.21.15.59:0/1048113157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78d00 con 0x55e508b3e000 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a89400640 1 -- 172.21.15.59:0/1048113157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e507d9c1c0 con 0x55e508b3e000 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a89400640 1 -- 172.21.15.59:0/1048113157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78680 con 0x55e508b3e000 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e507d7f2c0 con 0x55e508b3e000 2024-04-24T06:37:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e507d7fa40 con 0x55e508b3e000 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 4 waiting for MDS map... 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a89400640 1 -- 172.21.15.59:0/1048113157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x55e508c2c600 con 0x55e508b3e000 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a89400640 1 -- 172.21.15.59:0/1048113157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55e507dab180 con 0x55e508b3e000 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 4 Got MDS map 69 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 4 datascan.main: Using metadata pool output 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.416+0000 7f6a8ef2e740 4 datascan.main: connecting to RADOS... 2024-04-24T06:37:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.417+0000 7f6a8ef2e740 1 Processor -- start 2024-04-24T06:37:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.417+0000 7f6a8ef2e740 1 -- start start 2024-04-24T06:37:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c8e400 0x55e508b2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac6480 con 0x55e508c8e000 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac6600 con 0x55e508c29c00 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e508ac6780 con 0x55e508c8e400 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33426/0 (socket says 172.21.15.59:33426) 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.418+0000 7f6a8c600640 1 -- 172.21.15.59:0/2010545337 learned_addr learned my addr 172.21.15.59:0/2010545337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 -- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c8e400 msgr2=0x55e508b2fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c8e400 0x55e508b2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 -- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c29c00 msgr2=0x55e508b2e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 -- 172.21.15.59:0/2010545337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e507d7e3c0 con 0x55e508c8e000 2024-04-24T06:37:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a8c600640 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x55e508b7fe30 tx=0x55e508ae3740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5acfeee0aad994 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a86c00640 1 -- 172.21.15.59:0/2010545337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78b60 con 0x55e508c8e000 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a86c00640 1 -- 172.21.15.59:0/2010545337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e507d9ca80 con 0x55e508c8e000 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.419+0000 7f6a86c00640 1 -- 172.21.15.59:0/2010545337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78d00 con 0x55e508c8e000 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/2010545337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 msgr2=0x55e508b2f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x55e508b7fe30 tx=0x55e508ae3740 comp rx=0 tx=0).stop 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/2010545337 shutdown_connections 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c8e400 0x55e508b2fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c8e000 0x55e508b2f600 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/2010545337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/2010545337 >> 172.21.15.59:0/2010545337 conn(0x55e508b3e400 msgr2=0x55e507e1f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/2010545337 shutdown_connections 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.420+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/2010545337 wait complete. 2024-04-24T06:37:35.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.421+0000 7f6a8ef2e740 1 Processor -- start 2024-04-24T06:37:35.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.421+0000 7f6a8ef2e740 1 -- start start 2024-04-24T06:37:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.421+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c9a000 0x55e508b2f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8ef2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac7e00 con 0x55e508c9a400 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e508ac7c80 con 0x55e508c9a000 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8ef2e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e508ac7b00 con 0x55e508c29c00 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33434/0 (socket says 172.21.15.59:33434) 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 learned_addr learned my addr 172.21.15.59:0/1755625506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c9a000 0x55e508b2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c29c00 msgr2=0x55e508b2e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508c29c00 0x55e508b2e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c9a000 msgr2=0x55e508b2f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e508c9a000 0x55e508b2f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.422+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e507d7f680 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x55e508b7f3e0 tx=0x55e508af27e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a84e00640 1 -- 172.21.15.59:0/1755625506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78b60 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a84e00640 1 -- 172.21.15.59:0/1755625506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e507d9cc40 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e507d7f2c0 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a84e00640 1 -- 172.21.15.59:0/1755625506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e507d78d00 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e507d7eb40 con 0x55e508c9a400 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.423+0000 7f6a8ef2e740 4 datascan.init: resolving metadata pool 12 2024-04-24T06:37:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.425+0000 7f6a84e00640 1 -- 172.21.15.59:0/1755625506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55e508c8ec00 con 0x55e508c9a400 2024-04-24T06:37:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.425+0000 7f6a84e00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 0x55e508b30100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.425+0000 7f6a84e00640 1 -- 172.21.15.59:0/1755625506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x55e508ca6800 con 0x55e508c9a400 2024-04-24T06:37:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8ef2e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:37:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8ef2e740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8ef2e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55e508c9a800 0x55e508b30680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9ac00 con 0x55e508c9a800 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 0x55e508b30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 0x55e508b30100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55e507e01470 tx=0x55e508c41260 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.426+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55e508c9a800 0x55e508b30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.427+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55e508c9a800 0x55e508b30680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.428+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.428+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55e508c9b000 0x55e508b30c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.428+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e508c9b000 2024-04-24T06:37:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.429+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55e508c9b000 0x55e508b30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.430+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55e508c9b000 0x55e508b30c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.431+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.431+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55e508c9b800 0x55e508b31180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.431+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9bc00 con 0x55e508c9b800 2024-04-24T06:37:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.431+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55e508c9b800 0x55e508b31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.433+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55e508c9b800 0x55e508b31180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.434+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.434+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6400 con 0x55e508c9b000 2024-04-24T06:37:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.435+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.435+0000 7f6a86c00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55e5080d6800 0x55e5080fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.435+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6c00 con 0x55e5080d6800 2024-04-24T06:37:35.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.435+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55e5080d6800 0x55e5080fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.436+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55e5080d6800 0x55e5080fa000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.436+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.436+0000 7f6a86200640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55e5080d7400 0x55e5080fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.436+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7800 con 0x55e5080d7400 2024-04-24T06:37:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.437+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55e5080d7400 0x55e5080fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.438+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55e5080d7400 0x55e5080fa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.439+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.439+0000 7f6a86c00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e5080d7000 0x55e5080fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.439+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6400 con 0x55e5080d7000 2024-04-24T06:37:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.439+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e5080d7000 0x55e5080fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.440+0000 7f6a8c600640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e5080d7000 0x55e5080fab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.440+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.440+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9ac00 con 0x55e5080d7000 2024-04-24T06:37:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.441+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.441+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7800 con 0x55e508c9b800 2024-04-24T06:37:35.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.442+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.442+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29000 con 0x55e508c9a800 2024-04-24T06:37:35.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.443+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.443+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e508c9b000 2024-04-24T06:37:35.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.444+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.444+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29400 con 0x55e508c9b000 2024-04-24T06:37:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.444+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.445+0000 7f6a86c00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55e5080d6400 0x55e5080fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.445+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7c00 con 0x55e5080d6400 2024-04-24T06:37:35.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.445+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55e5080d6400 0x55e5080fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.446+0000 7f6a8bc00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55e5080d6400 0x55e5080fb080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.446+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.446+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e000 con 0x55e508c9b000 2024-04-24T06:37:35.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.447+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.447+0000 7f6a86c00640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55e508c9bc00 0x55e5080fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.447+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f000 con 0x55e508c9bc00 2024-04-24T06:37:35.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.447+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55e508c9bc00 0x55e5080fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.448+0000 7f6a8d000640 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55e508c9bc00 0x55e5080fb600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.448+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.448+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e400 con 0x55e5080d7000 2024-04-24T06:37:35.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.449+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.449+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e508c9b000 2024-04-24T06:37:35.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.450+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.450+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e5080d7000 2024-04-24T06:37:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.450+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.450+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f800 con 0x55e508c9a800 2024-04-24T06:37:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.451+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.451+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5ec00 con 0x55e508c9bc00 2024-04-24T06:37:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.452+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.452+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5fc00 con 0x55e5080d6400 2024-04-24T06:37:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.452+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.452+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29000 con 0x55e5080d7000 2024-04-24T06:37:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.453+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.453+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e400 con 0x55e5080d7000 2024-04-24T06:37:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.453+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.453+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29400 con 0x55e508c9bc00 2024-04-24T06:37:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.454+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.454+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e508c9bc00 2024-04-24T06:37:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.454+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(25 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.454+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9ac00 con 0x55e508c9a800 2024-04-24T06:37:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.455+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.455+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f800 con 0x55e5080d7400 2024-04-24T06:37:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.456+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.456+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e508c9b800 2024-04-24T06:37:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.457+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.457+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e000 con 0x55e5080d6800 2024-04-24T06:37:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.457+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.457+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7800 con 0x55e508c9b000 2024-04-24T06:37:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.458+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.459+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f000 con 0x55e508c9b800 2024-04-24T06:37:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.460+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.460+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e5080d7000 2024-04-24T06:37:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.460+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.460+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118000 con 0x55e5080d7400 2024-04-24T06:37:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.461+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.462+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5ec00 con 0x55e508c9bc00 2024-04-24T06:37:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.462+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.462+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118400 con 0x55e508c9b000 2024-04-24T06:37:35.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.463+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.463+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e5080d7400 2024-04-24T06:37:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.464+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.464+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118800 con 0x55e508c9bc00 2024-04-24T06:37:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.465+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.465+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f800 con 0x55e5080d6800 2024-04-24T06:37:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.466+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.466+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118c00 con 0x55e508c9a800 2024-04-24T06:37:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.467+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.467+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29400 con 0x55e508c9b000 2024-04-24T06:37:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.468+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.468+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119000 con 0x55e5080d6800 2024-04-24T06:37:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.469+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.469+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118c00 con 0x55e5080d7000 2024-04-24T06:37:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.470+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.470+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119400 con 0x55e5080d7400 2024-04-24T06:37:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.471+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.471+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6c00 con 0x55e5080d7400 2024-04-24T06:37:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.473+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.473+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119800 con 0x55e5080d7400 2024-04-24T06:37:35.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.474+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.474+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7c00 con 0x55e508c9bc00 2024-04-24T06:37:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.475+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.475+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118000 con 0x55e5080d7400 2024-04-24T06:37:35.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.476+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.476+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7800 con 0x55e5080d6400 2024-04-24T06:37:35.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.477+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.477+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118400 con 0x55e508c9b800 2024-04-24T06:37:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.478+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.478+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9ac00 con 0x55e508c9b800 2024-04-24T06:37:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.479+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.479+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118800 con 0x55e5080d6400 2024-04-24T06:37:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.480+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.480+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e508c9bc00 2024-04-24T06:37:35.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.481+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.481+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119c00 con 0x55e508c9b800 2024-04-24T06:37:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.482+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.483+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29000 con 0x55e508c9b800 2024-04-24T06:37:35.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.484+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.484+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e508c9a800 2024-04-24T06:37:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.485+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.485+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e000 con 0x55e5080d7000 2024-04-24T06:37:35.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.486+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.486+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119400 con 0x55e5080d7400 2024-04-24T06:37:35.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.487+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.487+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e400 con 0x55e5080d7400 2024-04-24T06:37:35.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.488+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.489+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119800 con 0x55e508c9b800 2024-04-24T06:37:35.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.490+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 87 out=81b] v87'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.490+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118c00 con 0x55e508c9b800 2024-04-24T06:37:35.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.491+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.491+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118000 con 0x55e508c9b000 2024-04-24T06:37:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.493+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.493+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e5080d6800 2024-04-24T06:37:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.494+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 5 ==== osd_op_reply(62 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.494+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6c00 con 0x55e5080d7400 2024-04-24T06:37:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.495+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.495+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5ec00 con 0x55e5080d7000 2024-04-24T06:37:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.496+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.496+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7c00 con 0x55e5080d7000 2024-04-24T06:37:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.497+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.497+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f000 con 0x55e508c9b800 2024-04-24T06:37:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.498+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.498+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29400 con 0x55e508c9a800 2024-04-24T06:37:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.499+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.499+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e508c9b000 2024-04-24T06:37:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.500+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.501+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122000 con 0x55e508c9b000 2024-04-24T06:37:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.502+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.502+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f800 con 0x55e5080d6400 2024-04-24T06:37:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.503+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.503+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122400 con 0x55e508c9b000 2024-04-24T06:37:35.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.504+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.504+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5fc00 con 0x55e508c9bc00 2024-04-24T06:37:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.505+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 9 ==== osd_op_reply(72 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.505+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122800 con 0x55e5080d7000 2024-04-24T06:37:35.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.506+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.506+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e400 con 0x55e508c9b000 2024-04-24T06:37:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.508+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.508+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122c00 con 0x55e5080d7000 2024-04-24T06:37:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.509+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.509+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e508c9a800 2024-04-24T06:37:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.510+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.510+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123000 con 0x55e508c9bc00 2024-04-24T06:37:35.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.511+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 10 ==== osd_op_reply(77 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.511+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5fc00 con 0x55e5080d6400 2024-04-24T06:37:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.512+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.512+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123400 con 0x55e5080d7000 2024-04-24T06:37:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.513+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.513+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122000 con 0x55e5080d7000 2024-04-24T06:37:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.514+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.514+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123800 con 0x55e508c9bc00 2024-04-24T06:37:35.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.515+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 11 ==== osd_op_reply(81 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.515+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122400 con 0x55e508c9bc00 2024-04-24T06:37:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.516+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 12 ==== osd_op_reply(82 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.516+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123c00 con 0x55e508c9a800 2024-04-24T06:37:35.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.517+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.517+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6c00 con 0x55e5080d7400 2024-04-24T06:37:35.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.518+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.518+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5ec00 con 0x55e508c9b800 2024-04-24T06:37:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.519+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.520+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119400 con 0x55e5080d6800 2024-04-24T06:37:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.521+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 6 ==== osd_op_reply(86 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.521+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f000 con 0x55e508c9b000 2024-04-24T06:37:35.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.522+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.522+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118000 con 0x55e508c9b800 2024-04-24T06:37:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.523+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.523+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118c00 con 0x55e5080d7000 2024-04-24T06:37:35.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.524+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.524+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e5080d7400 2024-04-24T06:37:35.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.525+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.525+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119000 con 0x55e508c9bc00 2024-04-24T06:37:35.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.526+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 13 ==== osd_op_reply(91 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.527+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e508c9b000 2024-04-24T06:37:35.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.528+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.528+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118800 con 0x55e5080d7400 2024-04-24T06:37:35.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.529+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.529+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f800 con 0x55e508c9bc00 2024-04-24T06:37:35.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.530+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.530+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7c00 con 0x55e5080d6800 2024-04-24T06:37:35.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.531+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.531+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5fc00 con 0x55e508c9a800 2024-04-24T06:37:35.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.533+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.533+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d7800 con 0x55e508c9b000 2024-04-24T06:37:35.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.534+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.534+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e400 con 0x55e5080d6800 2024-04-24T06:37:35.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.535+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.3 v2:172.21.15.59:6824/2800980920 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6800 2024-04-24T06:37:35.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.535+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122800 con 0x55e5080d7000 2024-04-24T06:37:35.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.536+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.536+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f000 con 0x55e5080d7400 2024-04-24T06:37:35.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.537+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.538+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122c00 con 0x55e5080d7400 2024-04-24T06:37:35.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.539+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.539+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5f400 con 0x55e5080d7400 2024-04-24T06:37:35.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.540+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.540+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123400 con 0x55e508c9bc00 2024-04-24T06:37:35.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.541+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.541+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e800 con 0x55e5080d7400 2024-04-24T06:37:35.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.542+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.542+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29400 con 0x55e5080d6400 2024-04-24T06:37:35.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.543+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.543+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508119000 con 0x55e508c9b800 2024-04-24T06:37:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.545+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.545+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508128000 con 0x55e508c9b800 2024-04-24T06:37:35.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.546+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.546+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e5080d6c00 con 0x55e5080d6400 2024-04-24T06:37:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.547+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.547+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508128400 con 0x55e508c9bc00 2024-04-24T06:37:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.548+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.548+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123000 con 0x55e508c9b800 2024-04-24T06:37:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.549+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.549+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508128800 con 0x55e508c9b800 2024-04-24T06:37:35.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.551+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.551+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508123800 con 0x55e508c9a800 2024-04-24T06:37:35.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.553+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.553+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508128c00 con 0x55e5080d7000 2024-04-24T06:37:35.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.553+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.2 v2:172.21.15.59:6809/1388377934 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e5080d7000 2024-04-24T06:37:35.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.553+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122400 con 0x55e5080d7400 2024-04-24T06:37:35.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.554+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.554+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508129000 con 0x55e5080d7400 2024-04-24T06:37:35.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.555+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.556+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508128400 con 0x55e508c9b800 2024-04-24T06:37:35.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.557+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 87 out=81b] v87'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.557+0000 7f6a8ef2e740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-24T06:37:35.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.557+0000 7f6a8ef2e740 10 datascan.operator(): handling object 10000000001.0 2024-04-24T06:37:35.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.557+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508119c00 con 0x55e508c9b800 2024-04-24T06:37:35.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.558+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.7 v2:172.21.15.66:6824/607700871 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9b800 2024-04-24T06:37:35.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.559+0000 7f6a8ef2e740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-24T06:37:35.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.559+0000 7f6a8ef2e740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-24T06:37:35.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.559+0000 7f6a8ef2e740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-24T06:37:35.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.559+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508119800 con 0x55e5080d6400 2024-04-24T06:37:35.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.560+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.560+0000 7f6a8ef2e740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-24T06:37:35.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.560+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508b5e000 con 0x55e508c9b000 2024-04-24T06:37:35.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.561+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.561+0000 7f6a8ef2e740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-24T06:37:35.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.561+0000 7f6a8ef2e740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-24T06:37:35.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.561+0000 7f6a8ef2e740 20 datascan.get_frag_of: frag is * 2024-04-24T06:37:35.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.561+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9ac00 con 0x55e5080d6400 2024-04-24T06:37:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.563+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.563+0000 7f6a8ef2e740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-24T06:37:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.563+0000 7f6a8ef2e740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-24T06:37:35.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.563+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x55e508c9b400 con 0x55e5080d6400 2024-04-24T06:37:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.564+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.564+0000 7f6a8ef2e740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-24T06:37:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.564+0000 7f6a8ef2e740 20 datascan.read_dentry: file_head not found in result 2024-04-24T06:37:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.564+0000 7f6a8ef2e740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-24T06:37:35.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.564+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x55e508c29000 con 0x55e5080d6400 2024-04-24T06:37:35.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.569+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 <== osd.0 v2:172.21.15.59:6808/2713443937 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v87'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55e508ae0780 con 0x55e5080d6400 2024-04-24T06:37:35.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.569+0000 7f6a8ef2e740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-24T06:37:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.569+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508122c00 con 0x55e508c9bc00 2024-04-24T06:37:35.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.570+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.1 v2:172.21.15.59:6800/2375715406 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9bc00 2024-04-24T06:37:35.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.570+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118800 con 0x55e508c9a800 2024-04-24T06:37:35.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.571+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.571+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508129400 con 0x55e508c9a800 2024-04-24T06:37:35.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.573+0000 7f6a8c600640 1 -- 172.21.15.59:0/1755625506 <== osd.5 v2:172.21.15.66:6816/79236266 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0000 con 0x55e508c9a800 2024-04-24T06:37:35.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.573+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118000 con 0x55e508c9b000 2024-04-24T06:37:35.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.574+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.574+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508129800 con 0x55e5080d7400 2024-04-24T06:37:35.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.575+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.4 v2:172.21.15.66:6800/4254676376 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e5080d7400 2024-04-24T06:37:35.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.576+0000 7f6a86200640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508118c00 con 0x55e508c9b000 2024-04-24T06:37:35.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.577+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.577+0000 7f6a86c00640 1 -- 172.21.15.59:0/1755625506 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x55e508129c00 con 0x55e508c9b000 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8d000640 1 -- 172.21.15.59:0/1755625506 <== osd.6 v2:172.21.15.66:6808/280071528 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e508ae0500 con 0x55e508c9b000 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55e5080d6400 msgr2=0x55e5080fb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55e5080d6400 0x55e5080fb080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55e508c9bc00 msgr2=0x55e5080fb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55e508c9bc00 0x55e5080fb600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e5080d7000 msgr2=0x55e5080fab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e5080d7000 0x55e5080fab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55e5080d6800 msgr2=0x55e5080fa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55e5080d6800 0x55e5080fa000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55e5080d7400 msgr2=0x55e5080fa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55e5080d7400 0x55e5080fa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55e508c9a800 msgr2=0x55e508b30680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55e508c9a800 0x55e508b30680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55e508c9b000 msgr2=0x55e508b30c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55e508c9b000 0x55e508b30c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55e508c9b800 msgr2=0x55e508b31180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55e508c9b800 0x55e508b31180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.578+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 reap_dead start 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 reap_dead start 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8bc00640 1 -- 172.21.15.59:0/1755625506 reap_dead start 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 msgr2=0x55e508b30100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 0x55e508b30100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55e507e01470 tx=0x55e508c41260 comp rx=0 tx=0).stop 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 msgr2=0x55e508b2fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.579+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x55e508b7f3e0 tx=0x55e508af27e0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 shutdown_connections 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e5080d6000 0x55e508b30100 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1755625506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508c9a400 0x55e508b2fb80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 >> 172.21.15.59:0/1755625506 conn(0x55e508b3e400 msgr2=0x55e507e1f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 shutdown_connections 2024-04-24T06:37:35.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.580+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1755625506 wait complete. 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 msgr2=0x55e508b2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55e508b7fc80 tx=0x55e508af6120 comp rx=0 tx=0).stop 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 shutdown_connections 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e508b3e800 0x55e508b2eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e507d53c00 0x55e508b2f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 --2- 172.21.15.59:0/1048113157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e508b3e000 0x55e508b2e580 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.581+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 >> 172.21.15.59:0/1048113157 conn(0x55e507d53000 msgr2=0x55e507e20000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.582+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 shutdown_connections 2024-04-24T06:37:35.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.582+0000 7f6a8ef2e740 1 -- 172.21.15.59:0/1048113157 wait complete. 2024-04-24T06:37:35.594 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.274357, result: 2024-04-24T06:37:35.594 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-24T06:37:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.707+0000 7fc0e6c0c740 1 -- start start 2024-04-24T06:37:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.707+0000 7fc0e6c0c740 1 Processor -- start 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9b88800 0x555ff9b85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 0x555ff9b84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2e000 con 0x555ff8e19c00 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2e180 con 0x555ff9b88800 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555ff9c2e300 con 0x555ff9b89c00 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33460/0 (socket says 172.21.15.59:33460) 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 -- 172.21.15.59:0/3816829988 learned_addr learned my addr 172.21.15.59:0/3816829988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e4200640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9b88800 0x555ff9b85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 0x555ff9b84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 -- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 msgr2=0x555ff9b84580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 0x555ff9b84580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 -- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9b88800 msgr2=0x555ff9b85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9b88800 0x555ff9b85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e3800640 1 -- 172.21.15.59:0/3816829988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff8e45680 con 0x555ff8e19c00 2024-04-24T06:37:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.708+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 0x555ff9b84580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e3800640 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x555ff9c4dc50 tx=0x555ff9bbd020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ebc63fe69ab9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e1000640 1 -- 172.21.15.59:0/3816829988 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3ed00 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e1000640 1 -- 172.21.15.59:0/3816829988 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555ff8e62a80 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e1000640 1 -- 172.21.15.59:0/3816829988 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3e9c0 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff8e452c0 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555ff8e454a0 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e6c0c740 4 waiting for MDS map... 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.709+0000 7fc0e1000640 1 -- 172.21.15.59:0/3816829988 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x555ff9cf6000 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.710+0000 7fc0e1000640 1 -- 172.21.15.59:0/3816829988 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x555ff8e71180 con 0x555ff8e19c00 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.710+0000 7fc0e6c0c740 4 Got MDS map 69 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.710+0000 7fc0e6c0c740 4 datascan.main: Using metadata pool output 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.710+0000 7fc0e6c0c740 4 datascan.main: connecting to RADOS... 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.711+0000 7fc0e6c0c740 1 Processor -- start 2024-04-24T06:37:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.711+0000 7fc0e6c0c740 1 -- start start 2024-04-24T06:37:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2e480 con 0x555ff9b88000 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2e600 con 0x555ff9c35800 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555ff9c2e780 con 0x555ff9c35c00 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33466/0 (socket says 172.21.15.59:33466) 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1244657556 learned_addr learned my addr 172.21.15.59:0/1244657556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 msgr2=0x555ff9b86100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 msgr2=0x555ff9b85b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.712+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1244657556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff8e45a40 con 0x555ff9b88000 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.713+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x555ff9bbad20 tx=0x555ff9bbce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6053171c5fdcca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.713+0000 7fc0de800640 1 -- 172.21.15.59:0/1244657556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3eb60 con 0x555ff9b88000 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.713+0000 7fc0de800640 1 -- 172.21.15.59:0/1244657556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555ff8e62c40 con 0x555ff9b88000 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.713+0000 7fc0de800640 1 -- 172.21.15.59:0/1244657556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3e820 con 0x555ff9b88000 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1244657556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 msgr2=0x555ff9b85600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x555ff9bbad20 tx=0x555ff9bbce40 comp rx=0 tx=0).stop 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1244657556 shutdown_connections 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1244657556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1244657556 >> 172.21.15.59:0/1244657556 conn(0x555ff9b88400 msgr2=0x555ff8f04800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1244657556 shutdown_connections 2024-04-24T06:37:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1244657556 wait complete. 2024-04-24T06:37:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.714+0000 7fc0e6c0c740 1 Processor -- start 2024-04-24T06:37:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 -- start start 2024-04-24T06:37:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2fc80 con 0x555ff9b88000 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555ff9c2fb00 con 0x555ff9c35800 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.715+0000 7fc0e6c0c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555ff9c2f980 con 0x555ff9c35c00 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33470/0 (socket says 172.21.15.59:33470) 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 learned_addr learned my addr 172.21.15.59:0/1864568332 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 msgr2=0x555ff9b86100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9c35c00 0x555ff9b86100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 msgr2=0x555ff9b85b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff8e45680 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.716+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9c35800 0x555ff9b85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x555ff9d07980 tx=0x555ff9bbc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0dca00640 1 -- 172.21.15.59:0/1864568332 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3eb60 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0dca00640 1 -- 172.21.15.59:0/1864568332 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555ff8e63340 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0dca00640 1 -- 172.21.15.59:0/1864568332 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ff8e3e820 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555ff8e452c0 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff8e443c0 con 0x555ff9b88000 2024-04-24T06:37:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.717+0000 7fc0e6c0c740 4 datascan.init: resolving metadata pool 12 2024-04-24T06:37:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.719+0000 7fc0dca00640 1 -- 172.21.15.59:0/1864568332 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x555ff9d56c00 con 0x555ff9b88000 2024-04-24T06:37:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.719+0000 7fc0dca00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 0x555ff9b86680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0dca00640 1 -- 172.21.15.59:0/1864568332 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x555ff9d5a600 con 0x555ff9b88000 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0e6c0c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0e6c0c740 4 datascan.main: resolving metadata pool 12 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 0x555ff9b86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x555ff9194400 0x555ff9b86c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.720+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff9194800 con 0x555ff9194400 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.721+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 0x555ff9b86680 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x555ff9bbaed0 tx=0x555ff9d0cf60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.721+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x555ff9194400 0x555ff9b86c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.722+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x555ff9194400 0x555ff9b86c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.723+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.723+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x555ff9194800 0x555ff91ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.723+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff9c34c00 con 0x555ff9194800 2024-04-24T06:37:35.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.723+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x555ff9194800 0x555ff91ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.724+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x555ff9194800 0x555ff91ae000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.724+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=177b] v87'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.724+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x555ff9c34c00 0x555ff9b84000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.724+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4800 con 0x555ff9c34c00 2024-04-24T06:37:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.724+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x555ff9c34c00 0x555ff9b84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.725+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x555ff9c34c00 0x555ff9b84000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.725+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.725+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x555ff91a4800 0x555ff91ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.725+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4400 con 0x555ff91a4800 2024-04-24T06:37:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.726+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x555ff91a4800 0x555ff91ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.726+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x555ff91a4800 0x555ff91ae580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.727+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.727+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x555ff91a4400 0x555ff91aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.727+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff9c35000 con 0x555ff91a4400 2024-04-24T06:37:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.728+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x555ff91a4400 0x555ff91aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.729+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x555ff91a4400 0x555ff91aeb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.729+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.729+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x555ff9c35000 0x555ff91af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.729+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4c00 con 0x555ff9c35000 2024-04-24T06:37:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.730+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x555ff9c35000 0x555ff91af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.731+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x555ff9c35000 0x555ff91af600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.732+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(6 [pgnls start_epoch 87 out=149b] v86'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.732+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4c00 con 0x555ff9194800 2024-04-24T06:37:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.733+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.733+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4c00 con 0x555ff9194800 2024-04-24T06:37:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.733+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(8 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.733+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4c00 con 0x555ff9194400 2024-04-24T06:37:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.734+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.734+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a4c00 con 0x555ff9c34c00 2024-04-24T06:37:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.734+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.734+0000 7fc0e4200640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x555ff91a4c00 0x555ff91b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.734+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4c00 2024-04-24T06:37:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.735+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x555ff91a4c00 0x555ff91b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.735+0000 7fc0e3800640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x555ff91a4c00 0x555ff91b0100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.736+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(11 [pgnls start_epoch 87 out=81b] v86'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.736+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff9194400 2024-04-24T06:37:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.737+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.737+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4800 2024-04-24T06:37:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.737+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(13 [pgnls start_epoch 87 out=77b] v86'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.737+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4400 2024-04-24T06:37:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.738+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.738+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff9194800 2024-04-24T06:37:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.739+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(15 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.739+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff9194400 2024-04-24T06:37:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.740+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.740+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4400 2024-04-24T06:37:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.741+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.741+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff9c35000 2024-04-24T06:37:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.742+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.742+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4c00 2024-04-24T06:37:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.742+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.742+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff9194800 2024-04-24T06:37:35.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.743+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 5 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.743+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4400 2024-04-24T06:37:35.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.744+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.744+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5c00 con 0x555ff91a4400 2024-04-24T06:37:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.745+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(22 [pgnls start_epoch 87 out=97b] v86'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.745+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x555ff91a5c00 0x555ff91afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.745+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a5c00 2024-04-24T06:37:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.746+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x555ff91a5c00 0x555ff91afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.747+0000 7fc0e4c00640 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x555ff91a5c00 0x555ff91afb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.748+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(23 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a5c00 2024-04-24T06:37:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.748+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff9194800 2024-04-24T06:37:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.748+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 6 ==== osd_op_reply(24 [pgnls start_epoch 87 out=73b] v86'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.748+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff9194800 2024-04-24T06:37:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.749+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 7 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.749+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4800 2024-04-24T06:37:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.750+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=98b] v86'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.750+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff9c34c00 2024-04-24T06:37:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.750+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(27 [pgnls start_epoch 87 out=76b] v86'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.750+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff9194400 2024-04-24T06:37:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.751+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(28 [pgnls start_epoch 87 out=73b] v86'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.751+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4800 2024-04-24T06:37:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.752+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.752+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4400 2024-04-24T06:37:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.753+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 6 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.753+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4400 2024-04-24T06:37:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.754+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 7 ==== osd_op_reply(31 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.754+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4400 2024-04-24T06:37:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.755+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 8 ==== osd_op_reply(32 [pgnls start_epoch 87 out=101b] v86'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.755+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-24T06:37:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.755+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9194c00 con 0x555ff9194800 2024-04-24T06:37:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.756+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.756+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-24T06:37:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.756+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195000 con 0x555ff9194800 2024-04-24T06:37:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.757+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.757+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-24T06:37:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.757+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195400 con 0x555ff9194800 2024-04-24T06:37:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e6c0c740 20 datascan.scan_links: injected ino detected 2024-04-24T06:37:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-24T06:37:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195800 con 0x555ff9194800 2024-04-24T06:37:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-24T06:37:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.758+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9194c00 con 0x555ff9194800 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-24T06:37:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-24T06:37:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-24T06:37:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.759+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195000 con 0x555ff9c35000 2024-04-24T06:37:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.760+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.760+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-24T06:37:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.760+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195400 con 0x555ff9194800 2024-04-24T06:37:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.761+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.761+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-24T06:37:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.761+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195800 con 0x555ff91a4c00 2024-04-24T06:37:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.762+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.762+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-24T06:37:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.762+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-24T06:37:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.762+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-24T06:37:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.762+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5800 con 0x555ff9194800 2024-04-24T06:37:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.763+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.763+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-24T06:37:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.763+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5400 con 0x555ff91a4400 2024-04-24T06:37:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.764+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff9194800 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.765+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.765+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-24T06:37:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.765+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-24T06:37:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.765+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-24T06:37:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.765+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4000 con 0x555ff91a4800 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.766+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-24T06:37:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4400 con 0x555ff91a4800 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.767+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-24T06:37:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.768+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4800 con 0x555ff91a4400 2024-04-24T06:37:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.769+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.769+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194400 2024-04-24T06:37:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.770+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 6 ==== osd_op_reply(47 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.770+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.771+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 16 ==== osd_op_reply(48 [pgnls start_epoch 87 out=177b] v87'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.771+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c34c00 2024-04-24T06:37:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.772+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(49 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.772+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4800 2024-04-24T06:37:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.773+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(50 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.773+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.774+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 11 ==== osd_op_reply(51 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.774+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c35000 2024-04-24T06:37:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.775+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(52 [pgnls start_epoch 87 out=149b] v86'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.775+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.776+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 17 ==== osd_op_reply(53 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.776+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.777+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 18 ==== osd_op_reply(54 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.777+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194400 2024-04-24T06:37:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.778+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 7 ==== osd_op_reply(55 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.778+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c34c00 2024-04-24T06:37:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.779+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 5 ==== osd_op_reply(56 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.779+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4c00 2024-04-24T06:37:35.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.780+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 4 ==== osd_op_reply(57 [pgnls start_epoch 87 out=81b] v86'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.780+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194400 2024-04-24T06:37:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.781+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 8 ==== osd_op_reply(58 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.781+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4800 2024-04-24T06:37:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.782+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(59 [pgnls start_epoch 87 out=77b] v86'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.782+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.783+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 12 ==== osd_op_reply(60 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.783+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.784+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 19 ==== osd_op_reply(61 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.784+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194400 2024-04-24T06:37:35.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.785+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 9 ==== osd_op_reply(62 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.785+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.786+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 13 ==== osd_op_reply(63 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.786+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c35000 2024-04-24T06:37:35.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.788+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(64 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.788+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4c00 2024-04-24T06:37:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.789+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 5 ==== osd_op_reply(65 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.789+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.789+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 20 ==== osd_op_reply(66 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.790+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.790+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 14 ==== osd_op_reply(67 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.791+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.792+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 15 ==== osd_op_reply(68 [pgnls start_epoch 87 out=97b] v86'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.792+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a5c00 2024-04-24T06:37:35.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.793+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(69 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a5c00 2024-04-24T06:37:35.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.793+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.794+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 21 ==== osd_op_reply(70 [pgnls start_epoch 87 out=73b] v86'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.794+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194800 2024-04-24T06:37:35.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.795+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 22 ==== osd_op_reply(71 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.795+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4800 2024-04-24T06:37:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.796+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 9 ==== osd_op_reply(72 [pgnls start_epoch 87 out=98b] v86'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.796+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c34c00 2024-04-24T06:37:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.797+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.3 v2:172.21.15.59:6824/2800980920 6 ==== osd_op_reply(73 [pgnls start_epoch 87 out=76b] v86'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c34c00 2024-04-24T06:37:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.797+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9194400 2024-04-24T06:37:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.798+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.4 v2:172.21.15.66:6800/4254676376 10 ==== osd_op_reply(74 [pgnls start_epoch 87 out=73b] v86'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff9194400 2024-04-24T06:37:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.798+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4800 2024-04-24T06:37:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.799+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 10 ==== osd_op_reply(75 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.799+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.800+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 16 ==== osd_op_reply(76 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.800+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.801+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 17 ==== osd_op_reply(77 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.801+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff91a4400 2024-04-24T06:37:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.802+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 18 ==== osd_op_reply(78 [pgnls start_epoch 87 out=101b] v86'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.802+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-24T06:37:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.802+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d5000 con 0x555ff9194800 2024-04-24T06:37:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.803+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.803+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-24T06:37:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.803+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d5400 con 0x555ff9194800 2024-04-24T06:37:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.804+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.805+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-24T06:37:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.805+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d5800 con 0x555ff9194800 2024-04-24T06:37:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e6c0c740 20 datascan.scan_links: injected ino detected 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e6c0c740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e6c0c740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.806+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d5c00 con 0x555ff9194800 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.807+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.807+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-24T06:37:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.807+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4800 con 0x555ff9194800 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-24T06:37:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-24T06:37:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.808+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4c00 con 0x555ff9c35000 2024-04-24T06:37:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.809+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6780 con 0x555ff9c35000 2024-04-24T06:37:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.809+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-24T06:37:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.809+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d5000 con 0x555ff9194800 2024-04-24T06:37:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.810+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.810+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-24T06:37:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.810+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5800 con 0x555ff91a4c00 2024-04-24T06:37:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.811+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.2 v2:172.21.15.59:6809/1388377934 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4c00 2024-04-24T06:37:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.812+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.812+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.812+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.812+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5400 con 0x555ff9194800 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.812+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.813+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-24T06:37:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.813+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a4400 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.814+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9194c00 con 0x555ff9194800 2024-04-24T06:37:35.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.815+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.815+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-24T06:37:35.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.815+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-24T06:37:35.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.815+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-24T06:37:35.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.815+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195400 con 0x555ff91a4800 2024-04-24T06:37:35.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-24T06:37:35.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-24T06:37:35.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-24T06:37:35.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.816+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195800 con 0x555ff91a4800 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.817+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-24T06:37:35.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-24T06:37:35.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-24T06:37:35.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-24T06:37:35.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.818+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195400 con 0x555ff91a4400 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.6 v2:172.21.15.66:6808/280071528 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff91a4400 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e6c0c740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e6c0c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e6c0c740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e6c0c740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-24T06:37:35.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.819+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5800 con 0x555ff9194800 2024-04-24T06:37:35.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.820+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.820+0000 7fc0e6c0c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-24T06:37:35.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.820+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4000 con 0x555ff9194800 2024-04-24T06:37:35.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v87'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e6c0c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-24T06:37:35.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e6c0c740 10 datascan.scan_links: processing 1 injected_inos 2024-04-24T06:37:35.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e6c0c740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-24T06:37:35.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e6c0c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-24T06:37:35.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.824+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91d4400 con 0x555ff9194800 2024-04-24T06:37:35.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.825+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.825+0000 7fc0e6c0c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-24T06:37:35.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.825+0000 7fc0e6c0c740 20 datascan.scan_links: first is now 2 2024-04-24T06:37:35.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.825+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x555ff9194c00 con 0x555ff9194800 2024-04-24T06:37:35.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.828+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.0 v2:172.21.15.59:6808/2713443937 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v87'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x555ff9ba6500 con 0x555ff9194800 2024-04-24T06:37:35.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.828+0000 7fc0e6c0c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-24T06:37:35.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.828+0000 7fc0e6c0c740 10 datascan.scan_links: updating inotable 2024-04-24T06:37:35.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.828+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff9195400 con 0x555ff91a4800 2024-04-24T06:37:35.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.829+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.830+0000 7fc0e6c0c740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-24T06:37:35.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.830+0000 7fc0e6c0c740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-24T06:37:35.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.830+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5400 con 0x555ff91a4800 2024-04-24T06:37:35.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.834+0000 7fc0e3800640 1 -- 172.21.15.59:0/1864568332 <== osd.1 v2:172.21.15.59:6800/2375715406 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v87'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x555ff9ba6000 con 0x555ff91a4800 2024-04-24T06:37:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.834+0000 7fc0e6c0c740 10 datascan.scan_links: updating snaptable 2024-04-24T06:37:35.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.834+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x555ff91a5000 con 0x555ff91a5c00 2024-04-24T06:37:35.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e4c00640 1 -- 172.21.15.59:0/1864568332 <== osd.7 v2:172.21.15.66:6824/607700871 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x555ff9ba6280 con 0x555ff91a5c00 2024-04-24T06:37:35.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x555ff9194800 msgr2=0x555ff91ae000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x555ff9194800 0x555ff91ae000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x555ff91a4800 msgr2=0x555ff91ae580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x555ff91a4800 0x555ff91ae580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x555ff91a4c00 msgr2=0x555ff91b0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x555ff91a4c00 0x555ff91b0100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x555ff9c34c00 msgr2=0x555ff9b84000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.835+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x555ff9c34c00 0x555ff9b84000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x555ff9194400 msgr2=0x555ff9b86c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x555ff9194400 0x555ff9b86c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x555ff9c35000 msgr2=0x555ff91af600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x555ff9c35000 0x555ff91af600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x555ff91a4400 msgr2=0x555ff91aeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x555ff91a4400 0x555ff91aeb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 reap_dead start 2024-04-24T06:37:35.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 reap_dead start 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x555ff91a5c00 msgr2=0x555ff91afb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x555ff91a5c00 0x555ff91afb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 reap_dead start 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e4200640 1 -- 172.21.15.59:0/1864568332 reap_dead start 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 msgr2=0x555ff9b86680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 0x555ff9b86680 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x555ff9bbaed0 tx=0x555ff9d0cf60 comp rx=0 tx=0).stop 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 msgr2=0x555ff9b85600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.836+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x555ff9d07980 tx=0x555ff9bbc0c0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 shutdown_connections 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x555ff9194000 0x555ff9b86680 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/1864568332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff9b88000 0x555ff9b85600 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 >> 172.21.15.59:0/1864568332 conn(0x555ff9b88400 msgr2=0x555ff8f04800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 shutdown_connections 2024-04-24T06:37:35.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.837+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/1864568332 wait complete. 2024-04-24T06:37:35.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 msgr2=0x555ff9b84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:35.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x555ff9c4dc50 tx=0x555ff9bbd020 comp rx=0 tx=0).stop 2024-04-24T06:37:35.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 shutdown_connections 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555ff9b89c00 0x555ff9b84580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555ff8e19c00 0x555ff9b84b00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 --2- 172.21.15.59:0/3816829988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555ff9b88800 0x555ff9b85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.838+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 >> 172.21.15.59:0/3816829988 conn(0x555ff8e19000 msgr2=0x555ff8f05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.839+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 shutdown_connections 2024-04-24T06:37:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:35.839+0000 7fc0e6c0c740 1 -- 172.21.15.59:0/3816829988 wait complete. 2024-04-24T06:37:35.853 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.259003, result: 2024-04-24T06:37:35.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:37:36.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.073+0000 7f2dafe00640 1 Processor -- start 2024-04-24T06:37:36.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.073+0000 7f2dafe00640 1 -- start start 2024-04-24T06:37:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2dafe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2db01567e0 0x7f2db0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 0x7f2db00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2dafe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db015cab0 con 0x7f2db00d0b40 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2dafe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2db00d0f70 con 0x7f2db01567e0 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2dafe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2db0160e40 con 0x7f2db015d810 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2daea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2db01567e0 0x7f2db0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 0x7f2db00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2daf400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2db00d0b40 0x7f2db0160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:59468/0 (socket says 172.21.15.59:59468) 2024-04-24T06:37:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.075+0000 7f2daf400640 1 -- 172.21.15.59:0/661098721 learned_addr learned my addr 172.21.15.59:0/661098721 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.077+0000 7f2daea00640 1 -- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 msgr2=0x7f2db00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.077+0000 7f2daea00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 0x7f2db00d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.077+0000 7f2daea00640 1 -- 172.21.15.59:0/661098721 >> v1:172.21.15.66:6789/0 conn(0x7f2db00d0b40 legacy=0x7f2db0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.076+0000 7f2dae000640 1 -- 172.21.15.59:0/661098721 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555464137 0 0) 0x7f2da8001490 con 0x7f2db00d0b40 2024-04-24T06:37:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.077+0000 7f2daea00640 1 -- 172.21.15.59:0/661098721 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db015cd30 con 0x7f2db01567e0 2024-04-24T06:37:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.077+0000 7f2da7e00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 0x7f2db00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:37:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.078+0000 7f2daea00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db01567e0 0x7f2db0156bc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2da00097a0 tx=0x7f2da002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca497f0815562a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.079+0000 7f2dae000640 1 -- 172.21.15.59:0/661098721 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0037070 con 0x7f2db01567e0 2024-04-24T06:37:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.079+0000 7f2dae000640 1 -- 172.21.15.59:0/661098721 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da0032440 con 0x7f2db01567e0 2024-04-24T06:37:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.079+0000 7f2dae000640 1 -- 172.21.15.59:0/661098721 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0032810 con 0x7f2db01567e0 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 -- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db01567e0 msgr2=0x7f2db0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db01567e0 0x7f2db0156bc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2da00097a0 tx=0x7f2da002a180 comp rx=0 tx=0).stop 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 -- 172.21.15.59:0/661098721 shutdown_connections 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db015d810 0x7f2db00d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 --2- 172.21.15.59:0/661098721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db01567e0 0x7f2db0156bc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 -- 172.21.15.59:0/661098721 >> 172.21.15.59:0/661098721 conn(0x7f2db006bdf0 msgr2=0x7f2db006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 -- 172.21.15.59:0/661098721 shutdown_connections 2024-04-24T06:37:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.080+0000 7f2dafe00640 1 -- 172.21.15.59:0/661098721 wait complete. 2024-04-24T06:37:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.082+0000 7f2dafe00640 1 Processor -- start 2024-04-24T06:37:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.082+0000 7f2dafe00640 1 -- start start 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db00d0b40 0x7f2db01f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db015d810 0x7f2db009cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0160820 con 0x7f2db01567e0 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2db015d2f0 con 0x7f2db015d810 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2dafe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2db01609a0 con 0x7f2db00d0b40 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33484/0 (socket says 172.21.15.59:33484) 2024-04-24T06:37:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2da7e00640 1 -- 172.21.15.59:0/1181754779 learned_addr learned my addr 172.21.15.59:0/1181754779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:36.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2daea00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db00d0b40 0x7f2db01f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.084+0000 7f2daf400640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db015d810 0x7f2db009cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 -- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db00d0b40 msgr2=0x7f2db01f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db00d0b40 0x7f2db01f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 -- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db015d810 msgr2=0x7f2db009cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db015d810 0x7f2db009cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 -- 172.21.15.59:0/1181754779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db009c220 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da7e00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800ebe0 tx=0x7f2d9800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98017070 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9800cc30 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db009e1e0 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98013470 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.085+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01596b0 con 0x7f2db01567e0 2024-04-24T06:37:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.087+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db004fd90 con 0x7f2db01567e0 2024-04-24T06:37:36.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.088+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d9801a070 con 0x7f2db01567e0 2024-04-24T06:37:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.089+0000 7f2da6a00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 0x7f2d8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.089+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f2d9808bf70 con 0x7f2db01567e0 2024-04-24T06:37:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.089+0000 7f2daea00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 0x7f2d8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:36.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.090+0000 7f2daea00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 0x7f2d8c06ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2da0034000 tx=0x7f2da002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.094+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d9805a230 con 0x7f2db01567e0 2024-04-24T06:37:36.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:36.349+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2db00b5150 con 0x7f2db01567e0 2024-04-24T06:37:37.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.161+0000 7f2da6a00640 1 -- 172.21.15.59:0/1181754779 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) ==== 162+0+0 (secure 0 0 0) 0x7f2d98059b30 con 0x7f2db01567e0 2024-04-24T06:37:37.164 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:37:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.165+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 msgr2=0x7f2d8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.165+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 0x7f2d8c06ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2da0034000 tx=0x7f2da002e000 comp rx=0 tx=0).stop 2024-04-24T06:37:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.165+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 msgr2=0x7f2db009c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.165+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800ebe0 tx=0x7f2d9800c520 comp rx=0 tx=0).stop 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 shutdown_connections 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d8c06c920 0x7f2d8c06ede0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db015d810 0x7f2db009cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2db01567e0 0x7f2db009c700 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 --2- 172.21.15.59:0/1181754779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2db00d0b40 0x7f2db01f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 >> 172.21.15.59:0/1181754779 conn(0x7f2db006bdf0 msgr2=0x7f2db015ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 shutdown_connections 2024-04-24T06:37:37.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.166+0000 7f2dafe00640 1 -- 172.21.15.59:0/1181754779 wait complete. 2024-04-24T06:37:37.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:37:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.388+0000 7ff7b7e00640 1 Processor -- start 2024-04-24T06:37:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.388+0000 7ff7b7e00640 1 -- start start 2024-04-24T06:37:37.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.389+0000 7ff7b7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7b815c9d0 0x7ff7b809b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.389+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80b2e20 0x7ff7b80af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b80b35f0 con 0x7ff7b80d09e0 2024-04-24T06:37:37.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80cd2d0 con 0x7ff7b815c9d0 2024-04-24T06:37:37.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80b3890 con 0x7ff7b80b2e20 2024-04-24T06:37:37.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7b815c9d0 0x7ff7b809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7b815c9d0 0x7ff7b809b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33488/0 (socket says 172.21.15.59:33488) 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b6a00640 1 -- 172.21.15.59:0/1677613864 learned_addr learned my addr 172.21.15.59:0/1677613864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b6000640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80b2e20 0x7ff7b80af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.390+0000 7ff7b6a00640 1 -- 172.21.15.59:0/1677613864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80b2e20 msgr2=0x7ff7b80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.391+0000 7ff7b6a00640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80b2e20 0x7ff7b80af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.391+0000 7ff7b6a00640 1 -- 172.21.15.59:0/1677613864 >> v1:172.21.15.66:6789/0 conn(0x7ff7b80d09e0 legacy=0x7ff7b80d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.391+0000 7ff7b6a00640 1 -- 172.21.15.59:0/1677613864 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80b3b70 con 0x7ff7b815c9d0 2024-04-24T06:37:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b6a00640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b815c9d0 0x7ff7b809b410 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009870 tx=0x7ff7a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ef9c09be1d98aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b5600640 1 -- 172.21.15.59:0/1677613864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8037070 con 0x7ff7b815c9d0 2024-04-24T06:37:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b5600640 1 -- 172.21.15.59:0/1677613864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8032440 con 0x7ff7b815c9d0 2024-04-24T06:37:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b5600640 1 -- 172.21.15.59:0/1677613864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8032810 con 0x7ff7b815c9d0 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b7e00640 1 -- 172.21.15.59:0/1677613864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b815c9d0 msgr2=0x7ff7b809b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.392+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b815c9d0 0x7ff7b809b410 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009870 tx=0x7ff7a802a230 comp rx=0 tx=0).stop 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 -- 172.21.15.59:0/1677613864 shutdown_connections 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80b2e20 0x7ff7b80af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/1677613864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b815c9d0 0x7ff7b809b410 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 -- 172.21.15.59:0/1677613864 >> 172.21.15.59:0/1677613864 conn(0x7ff7b80b3ef0 msgr2=0x7ff7b80b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 -- 172.21.15.59:0/1677613864 shutdown_connections 2024-04-24T06:37:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.393+0000 7ff7b7e00640 1 -- 172.21.15.59:0/1677613864 wait complete. 2024-04-24T06:37:37.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.395+0000 7ff7b7e00640 1 Processor -- start 2024-04-24T06:37:37.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.395+0000 7ff7b7e00640 1 -- start start 2024-04-24T06:37:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80d09e0 0x7ff7b80b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b815c9d0 0x7ff7b80ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80b3280 con 0x7ff7b80b2e20 2024-04-24T06:37:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80ccef0 con 0x7ff7b815c9d0 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b815cdb0 con 0x7ff7b80d09e0 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33504/0 (socket says 172.21.15.59:33504) 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 -- 172.21.15.59:0/2511123197 learned_addr learned my addr 172.21.15.59:0/2511123197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 -- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80d09e0 msgr2=0x7ff7b80b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80d09e0 0x7ff7b80b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 -- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b815c9d0 msgr2=0x7ff7b80ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b815c9d0 0x7ff7b80ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.396+0000 7ff7b6a00640 1 -- 172.21.15.59:0/2511123197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80cad50 con 0x7ff7b80b2e20 2024-04-24T06:37:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7b6a00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009870 tx=0x7ff7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8037040 con 0x7ff7b80b2e20 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8004230 con 0x7ff7b80b2e20 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8003610 con 0x7ff7b80b2e20 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b80cb030 con 0x7ff7b80b2e20 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.397+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b80cb5f0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.399+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b804fd90 con 0x7ff7b80b2e20 2024-04-24T06:37:37.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.399+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a80037b0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.400+0000 7ff7af400640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.400+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7ff7a80acbf0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.401+0000 7ff7b6000640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.401+0000 7ff7b6000640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 0x7ff79406ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006fd0 tx=0x7ff7a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.405+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7a807aeb0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:37:37.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:37:37.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.649+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b815c1d0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.650+0000 7ff7af400640 1 -- 172.21.15.59:0/2511123197 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5174 (secure 0 0 0) 0x7ff7a807a7b0 con 0x7ff7b80b2e20 2024-04-24T06:37:37.652 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:37.652 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:37.167753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-24T06:37:37.652 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 72 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 msgr2=0x7ff79406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 0x7ff79406ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006fd0 tx=0x7ff7a0008040 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 msgr2=0x7ff7b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009870 tx=0x7ff7a80046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 shutdown_connections 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b815c9d0 0x7ff7b80ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff7b80d09e0 0x7ff7b80b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 --2- 172.21.15.59:0/2511123197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7b80b2e20 0x7ff7b80b4dc0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 >> 172.21.15.59:0/2511123197 conn(0x7ff7b80b3ef0 msgr2=0x7ff7b80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:37.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 shutdown_connections 2024-04-24T06:37:37.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:37.653+0000 7ff7b7e00640 1 -- 172.21.15.59:0/2511123197 wait complete. 2024-04-24T06:37:38.075 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:37:37.167753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/3165421940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 3165421940}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3165421940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:37:38.076 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:replay 2024-04-24T06:37:39.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:37:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.300+0000 7f9057e00640 1 Processor -- start 2024-04-24T06:37:39.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.301+0000 7f9057e00640 1 -- start start 2024-04-24T06:37:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.302+0000 7f9057e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f905815e7c0 0x7f90580d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.302+0000 7f9057e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90580cf030 con 0x7f90580d0d90 2024-04-24T06:37:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.302+0000 7f9057e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90580b6a40 con 0x7f90580d2630 2024-04-24T06:37:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.302+0000 7f9057e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f90580b7000 con 0x7f905815e7c0 2024-04-24T06:37:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.303+0000 7f9057400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f90580d0d90 0x7f90580cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54038/0 (socket says 172.21.15.59:54038) 2024-04-24T06:37:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.303+0000 7f9057400640 1 -- 172.21.15.59:0/1296837596 learned_addr learned my addr 172.21.15.59:0/1296837596 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.303+0000 7f9056a00640 1 --2- 172.21.15.59:0/1296837596 >> v2:172.21.15.66:3300/0 conn(0x7f905815e7c0 0x7f90580d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.303+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225392254 0 0) 0x7f9050001110 con 0x7f90580d0d90 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f903c003520 con 0x7f90580d0d90 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307102483 0 0) 0x7f903c003520 con 0x7f90580d0d90 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 >> v2:172.21.15.66:3300/0 conn(0x7f905815e7c0 msgr2=0x7f90580d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 --2- 172.21.15.59:0/1296837596 >> v2:172.21.15.66:3300/0 conn(0x7f905815e7c0 0x7f90580d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 >> v1:172.21.15.66:6789/0 conn(0x7f90580d2630 legacy=0x7f90580b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90580b6d20 con 0x7f90580d0d90 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9050001840 con 0x7f90580d0d90 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 >> v1:172.21.15.59:6789/0 conn(0x7f90580d0d90 legacy=0x7f90580cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 0x7f903c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f903c004800 0x7f903c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0051c0 0x7f903c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90580b7000 con 0x7f903c0039b0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f903c00c060 con 0x7f903c004800 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f903c00c260 con 0x7f903c0051c0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.304+0000 7f9056a00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 0x7f903c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056000640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f903c004800 0x7f903c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9057400640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0051c0 0x7f903c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056a00640 1 -- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0051c0 msgr2=0x7f903c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056a00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0051c0 0x7f903c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056a00640 1 -- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f903c004800 msgr2=0x7f903c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056a00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f903c004800 0x7f903c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056a00640 1 -- 172.21.15.59:0/1296837596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c003520 con 0x7f903c0039b0 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9056000640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f903c004800 0x7f903c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:37:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.305+0000 7f9057400640 1 -- 172.21.15.59:0/1296837596 reap_dead start 2024-04-24T06:37:39.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.306+0000 7f9056a00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 0x7f903c003dd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fc0 tx=0x7f903c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0b3550f789a0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048034020 con 0x7f903c0039b0 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90480305f0 con 0x7f903c0039b0 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9055600640 1 -- 172.21.15.59:0/1296837596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048009c10 con 0x7f903c0039b0 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 -- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 msgr2=0x7f903c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 0x7f903c003dd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fc0 tx=0x7f903c005830 comp rx=0 tx=0).stop 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 -- 172.21.15.59:0/1296837596 shutdown_connections 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 --2- 172.21.15.59:0/1296837596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f903c0039b0 0x7f903c003dd0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 -- 172.21.15.59:0/1296837596 >> 172.21.15.59:0/1296837596 conn(0x7f905809d920 msgr2=0x7f905808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.307+0000 7f9057e00640 1 -- 172.21.15.59:0/1296837596 shutdown_connections 2024-04-24T06:37:39.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.308+0000 7f9057e00640 1 -- 172.21.15.59:0/1296837596 wait complete. 2024-04-24T06:37:39.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.309+0000 7f9057e00640 1 Processor -- start 2024-04-24T06:37:39.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.309+0000 7f9057e00640 1 -- start start 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0039b0 0x7f90580c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90580ce170 0x7f90581f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90580d1a80 con 0x7f90580cc640 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f90580cf620 con 0x7f90580ce170 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f90580d1c00 con 0x7f903c0039b0 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45316/0 (socket says 172.21.15.59:45316) 2024-04-24T06:37:39.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9057400640 1 -- 172.21.15.59:0/4085045572 learned_addr learned my addr 172.21.15.59:0/4085045572 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.311+0000 7f9056a00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0039b0 0x7f90580c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9056000640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90580ce170 0x7f90581f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 -- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0039b0 msgr2=0x7f90580c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0039b0 0x7f90580c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 -- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90580ce170 msgr2=0x7f90581f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90580ce170 0x7f90581f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 -- 172.21.15.59:0/4085045572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581f50e0 con 0x7f90580cc640 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057400640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f90580d0b30 tx=0x7f905000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500072d0 con 0x7f90580cc640 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9050007470 con 0x7f90580cc640 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050004a20 con 0x7f90580cc640 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.312+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581f53c0 con 0x7f90580cc640 2024-04-24T06:37:39.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.313+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90580c80f0 con 0x7f90580cc640 2024-04-24T06:37:39.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.314+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905804fd90 con 0x7f90580cc640 2024-04-24T06:37:39.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.315+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f905001b020 con 0x7f90580cc640 2024-04-24T06:37:39.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.316+0000 7f904f400640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:39.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.316+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f905008b850 con 0x7f90580cc640 2024-04-24T06:37:39.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.316+0000 7f9056a00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:39.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.317+0000 7f9056a00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 0x7f903406edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9048035000 tx=0x7f904802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:39.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.320+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90580cf030 con 0x7f90580cc640 2024-04-24T06:37:39.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.573+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90580d1eb0 con 0x7f90580cc640 2024-04-24T06:37:39.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.574+0000 7f904f400640 1 -- 172.21.15.59:0/4085045572 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5174 (secure 0 0 0) 0x7f9050059b10 con 0x7f90580cc640 2024-04-24T06:37:39.577 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:39.577 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:38.192370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-24T06:37:39.577 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 msgr2=0x7f903406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 0x7f903406edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9048035000 tx=0x7f904802f000 comp rx=0 tx=0).stop 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 msgr2=0x7f905806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f90580d0b30 tx=0x7f905000dc00 comp rx=0 tx=0).stop 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 shutdown_connections 2024-04-24T06:37:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903406c8f0 0x7f903406edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90580cc640 0x7f905806d290 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90580ce170 0x7f90581f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 --2- 172.21.15.59:0/4085045572 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f903c0039b0 0x7f90580c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 >> 172.21.15.59:0/4085045572 conn(0x7f905809d920 msgr2=0x7f90580cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 shutdown_connections 2024-04-24T06:37:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:39.577+0000 7f9057e00640 1 -- 172.21.15.59:0/4085045572 wait complete. 2024-04-24T06:37:40.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:37:38.192370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6835/3165421940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 3165421940}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3165421940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:37:40.002 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:rejoin 2024-04-24T06:37:41.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:37:41.175 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:37:41.173+0000 7f271bc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:37:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.225+0000 7f9bb0c00640 1 Processor -- start 2024-04-24T06:37:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.225+0000 7f9bb0c00640 1 -- start start 2024-04-24T06:37:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.227+0000 7f9bb0c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.227+0000 7f9bb0c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bac0b4550 0x7f9bac06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.227+0000 7f9bb0c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac160d40 con 0x7f9bac160960 2024-04-24T06:37:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.227+0000 7f9bb0c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9bac164270 con 0x7f9bac0b4550 2024-04-24T06:37:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.227+0000 7f9bb0c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac166de0 con 0x7f9bac1644b0 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9ba9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9bac160960 0x7f9bac0b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54050/0 (socket says 172.21.15.59:54050) 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9baa800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bac0b4550 0x7f9bac06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9ba9e00640 1 -- 172.21.15.59:0/1908929824 learned_addr learned my addr 172.21.15.59:0/1908929824 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9ba1400640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9baa800640 1 -- 172.21.15.59:0/1908929824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 msgr2=0x7f9bac163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9baa800640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9baa800640 1 -- 172.21.15.59:0/1908929824 >> v1:172.21.15.59:6789/0 conn(0x7f9bac160960 legacy=0x7f9bac0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9baa800640 1 -- 172.21.15.59:0/1908929824 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac1670c0 con 0x7f9bac0b4550 2024-04-24T06:37:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.228+0000 7f9ba9400640 1 -- 172.21.15.59:0/1908929824 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459572739 0 0) 0x7f9bac160d40 con 0x7f9bac160960 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.229+0000 7f9baa800640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac0b4550 0x7f9bac06e8d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009d90 tx=0x7f9ba402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcf3029af8cec83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.230+0000 7f9ba9400640 1 -- 172.21.15.59:0/1908929824 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4007740 con 0x7f9bac0b4550 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.230+0000 7f9ba9400640 1 -- 172.21.15.59:0/1908929824 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba40078e0 con 0x7f9bac0b4550 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.230+0000 7f9ba9400640 1 -- 172.21.15.59:0/1908929824 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4007cb0 con 0x7f9bac0b4550 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.230+0000 7f9bb0c00640 1 -- 172.21.15.59:0/1908929824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac0b4550 msgr2=0x7f9bac06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.230+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac0b4550 0x7f9bac06e8d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009d90 tx=0x7f9ba402a660 comp rx=0 tx=0).stop 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 -- 172.21.15.59:0/1908929824 shutdown_connections 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac0b4550 0x7f9bac06e8d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/1908929824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 -- 172.21.15.59:0/1908929824 >> 172.21.15.59:0/1908929824 conn(0x7f9bac067ea0 msgr2=0x7f9bac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:41.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 -- 172.21.15.59:0/1908929824 shutdown_connections 2024-04-24T06:37:41.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.231+0000 7f9bb0c00640 1 -- 172.21.15.59:0/1908929824 wait complete. 2024-04-24T06:37:41.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.233+0000 7f9bb0c00640 1 Processor -- start 2024-04-24T06:37:41.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.233+0000 7f9bb0c00640 1 -- start start 2024-04-24T06:37:41.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bac0b4550 0x7f9bac1fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1668e0 con 0x7f9bac160960 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac167680 con 0x7f9bac0b4550 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9bb0c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac166a60 con 0x7f9bac1644b0 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.234+0000 7f9ba1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45336/0 (socket says 172.21.15.59:45336) 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 -- 172.21.15.59:0/3101851379 learned_addr learned my addr 172.21.15.59:0/3101851379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 -- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 msgr2=0x7f9bac161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 -- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bac0b4550 msgr2=0x7f9bac1fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bac0b4550 0x7f9bac1fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 -- 172.21.15.59:0/3101851379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac0cf5e0 con 0x7f9bac160960 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.235+0000 7f9ba1400640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d5c0 tx=0x7f9b9400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:41.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.236+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940125f0 con 0x7f9bac160960 2024-04-24T06:37:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.236+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94019070 con 0x7f9bac160960 2024-04-24T06:37:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.236+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940129d0 con 0x7f9bac160960 2024-04-24T06:37:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.236+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac0cfb70 con 0x7f9bac160960 2024-04-24T06:37:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.236+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac0d0130 con 0x7f9bac160960 2024-04-24T06:37:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.237+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac068340 con 0x7f9bac160960 2024-04-24T06:37:41.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.238+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b94012b70 con 0x7f9bac160960 2024-04-24T06:37:41.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.240+0000 7f9ba2a00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:41.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.240+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f9b9408c040 con 0x7f9bac160960 2024-04-24T06:37:41.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.240+0000 7f9ba9e00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:41.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.241+0000 7f9ba9e00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 0x7f9b8806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007fd0 tx=0x7f9b9c007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:41.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.246+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bac068340 con 0x7f9bac160960 2024-04-24T06:37:41.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.496+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bac068a80 con 0x7f9bac160960 2024-04-24T06:37:41.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.497+0000 7f9ba2a00640 1 -- 172.21.15.59:0/3101851379 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7f9b9405a300 con 0x7f9bac160960 2024-04-24T06:37:41.499 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:37:41.499 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:37:41.500 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 msgr2=0x7f9b8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 0x7f9b8806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007fd0 tx=0x7f9b9c007920 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 msgr2=0x7f9bac0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d5c0 tx=0x7f9b9400da80 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 shutdown_connections 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bac1644b0 0x7f9bac161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bac160960 0x7f9bac0cec50 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 --2- 172.21.15.59:0/3101851379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bac0b4550 0x7f9bac1fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.500+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 >> 172.21.15.59:0/3101851379 conn(0x7f9bac067ea0 msgr2=0x7f9bac1648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.501+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 shutdown_connections 2024-04-24T06:37:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:41.501+0000 7f9bb0c00640 1 -- 172.21.15.59:0/3101851379 wait complete. 2024-04-24T06:37:41.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:35:54.320156+0000', 'modified': '2024-04-24T06:37:40.197960+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6835/3165421940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 3165421940}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3165421940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8133, 'qdb_cluster': [8133]} 2024-04-24T06:37:41.925 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:37:41.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T06:37:42.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.146+0000 7fe143200640 1 Processor -- start 2024-04-24T06:37:42.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.146+0000 7fe143200640 1 -- start start 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.147+0000 7fe143200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1440cd8b0 0x7fe1440d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.147+0000 7fe143200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440d0b00 0x7fe14406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.147+0000 7fe143200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1440b4cf0 con 0x7fe14415c670 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.147+0000 7fe143200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe14415ca50 con 0x7fe1440cd8b0 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.147+0000 7fe143200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1440ccb80 con 0x7fe1440d0b00 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1440cd8b0 0x7fe1440d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1440cd8b0 0x7fe1440d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45352/0 (socket says 172.21.15.59:45352) 2024-04-24T06:37:42.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 -- 172.21.15.59:0/677491217 learned_addr learned my addr 172.21.15.59:0/677491217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe142800640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440d0b00 0x7fe14406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 -- 172.21.15.59:0/677491217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440d0b00 msgr2=0x7fe14406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440d0b00 0x7fe14406ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 -- 172.21.15.59:0/677491217 >> v1:172.21.15.66:6789/0 conn(0x7fe14415c670 legacy=0x7fe1440b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.148+0000 7fe139400640 1 -- 172.21.15.59:0/677491217 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1440ccdd0 con 0x7fe1440cd8b0 2024-04-24T06:37:42.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.150+0000 7fe139400640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440cd8b0 0x7fe1440d05c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009870 tx=0x7fe12c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb6234360b417e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.150+0000 7fe141400640 1 -- 172.21.15.59:0/677491217 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c037070 con 0x7fe1440cd8b0 2024-04-24T06:37:42.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.150+0000 7fe141400640 1 -- 172.21.15.59:0/677491217 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c032440 con 0x7fe1440cd8b0 2024-04-24T06:37:42.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.150+0000 7fe141400640 1 -- 172.21.15.59:0/677491217 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c032810 con 0x7fe1440cd8b0 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 -- 172.21.15.59:0/677491217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440cd8b0 msgr2=0x7fe1440d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440cd8b0 0x7fe1440d05c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009870 tx=0x7fe12c02a210 comp rx=0 tx=0).stop 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 -- 172.21.15.59:0/677491217 shutdown_connections 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440d0b00 0x7fe14406ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 --2- 172.21.15.59:0/677491217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440cd8b0 0x7fe1440d05c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 -- 172.21.15.59:0/677491217 >> 172.21.15.59:0/677491217 conn(0x7fe14406bdf0 msgr2=0x7fe14406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 -- 172.21.15.59:0/677491217 shutdown_connections 2024-04-24T06:37:42.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.151+0000 7fe143200640 1 -- 172.21.15.59:0/677491217 wait complete. 2024-04-24T06:37:42.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.153+0000 7fe143200640 1 Processor -- start 2024-04-24T06:37:42.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.153+0000 7fe143200640 1 -- start start 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440cd8b0 0x7fe144156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe14415c670 0x7fe144159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440b3e60 con 0x7fe1440d0b00 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440cd390 con 0x7fe14415c670 2024-04-24T06:37:42.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe143200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1440b3fe0 con 0x7fe1440cd8b0 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.154+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45368/0 (socket says 172.21.15.59:45368) 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 -- 172.21.15.59:0/3757102003 learned_addr learned my addr 172.21.15.59:0/3757102003 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe141e00640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440cd8b0 0x7fe144156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe142800640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe14415c670 0x7fe144159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 -- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440cd8b0 msgr2=0x7fe144156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440cd8b0 0x7fe144156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 -- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe14415c670 msgr2=0x7fe144159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe14415c670 0x7fe144159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 -- 172.21.15.59:0/3757102003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14414ab20 con 0x7fe1440d0b00 2024-04-24T06:37:42.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.155+0000 7fe139400640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009870 tx=0x7fe12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.156+0000 7fe13aa00640 1 -- 172.21.15.59:0/3757102003 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c037040 con 0x7fe1440d0b00 2024-04-24T06:37:42.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.156+0000 7fe13aa00640 1 -- 172.21.15.59:0/3757102003 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c042070 con 0x7fe1440d0b00 2024-04-24T06:37:42.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.156+0000 7fe13aa00640 1 -- 172.21.15.59:0/3757102003 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c030bb0 con 0x7fe1440d0b00 2024-04-24T06:37:42.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.156+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe144159e20 con 0x7fe1440d0b00 2024-04-24T06:37:42.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.156+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe144153be0 con 0x7fe1440d0b00 2024-04-24T06:37:42.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.159+0000 7fe13aa00640 1 -- 172.21.15.59:0/3757102003 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe12c047460 con 0x7fe1440d0b00 2024-04-24T06:37:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.160+0000 7fe14b84a740 1 Processor -- start 2024-04-24T06:37:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.160+0000 7fe14b84a740 1 -- start start 2024-04-24T06:37:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.160+0000 7fe13aa00640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 0x7fe12006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.160+0000 7fe13aa00640 1 -- 172.21.15.59:0/3757102003 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fe12c0ac7d0 con 0x7fe1440d0b00 2024-04-24T06:37:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe142800640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 0x7fe12006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 0x557930d0f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 0x557930d3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557930c79b10 con 0x557930d3f360 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557930c202b0 con 0x557930d5c980 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557930c75ad0 con 0x557930d3faa0 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.161+0000 7fe142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45376/0 (socket says 172.21.15.59:45376) 2024-04-24T06:37:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 -- 172.21.15.59:0/1380283533 learned_addr learned my addr 172.21.15.59:0/1380283533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 0x7fe12006ee00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b830 tx=0x7fe13c00bc30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe141e00640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 0x557930d0f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe139400640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 0x557930d3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 -- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 msgr2=0x557930d3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 0x557930d3ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 -- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 msgr2=0x557930d0f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 0x557930d0f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 -- 172.21.15.59:0/1380283533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793098c2d0 con 0x557930d3f360 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe141e00640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 0x557930d0f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe139400640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 0x557930d3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.162+0000 7fe142800640 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00db70 tx=0x7fe13c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa64c52ed21380a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe132600640 1 -- 172.21.15.59:0/1380283533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c01a040 con 0x557930d3f360 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe132600640 1 -- 172.21.15.59:0/1380283533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c023070 con 0x557930d3f360 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe132600640 1 -- 172.21.15.59:0/1380283533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c01e430 con 0x557930d3f360 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 -- 172.21.15.59:0/1380283533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 msgr2=0x557930d14170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00db70 tx=0x7fe13c018000 comp rx=0 tx=0).stop 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 -- 172.21.15.59:0/1380283533 shutdown_connections 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d3f360 0x557930d14170 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d3faa0 0x557930d3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1380283533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d5c980 0x557930d0f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.163+0000 7fe14b84a740 1 -- 172.21.15.59:0/1380283533 >> 172.21.15.59:0/1380283533 conn(0x557930d11580 msgr2=0x557930d59e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.164+0000 7fe14b84a740 1 -- 172.21.15.59:0/1380283533 shutdown_connections 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.164+0000 7fe14b84a740 1 -- 172.21.15.59:0/1380283533 wait complete. 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.164+0000 7fe14b84a740 1 -- start start 2024-04-24T06:37:42.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.165+0000 7fe14b84a740 1 Processor -- start 2024-04-24T06:37:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557930d13730 con 0x557930d41b80 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557930c33590 con 0x557930d3faa0 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557930d12b20 con 0x557930d5c980 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe14b84a740 10 client.0 _ll_register_callbacks cb 0x557930c163a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45378/0 (socket says 172.21.15.59:45378) 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 -- 172.21.15.59:0/3349364143 learned_addr learned my addr 172.21.15.59:0/3349364143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe139400640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 -- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 msgr2=0x557930d51ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe141e00640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d51ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 -- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 msgr2=0x557930d518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe142800640 1 -- 172.21.15.59:0/3349364143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557930a566b0 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.166+0000 7fe139400640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe142800640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01f000 tx=0x7fe13c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3239d3029bb1fd0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe126a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c01a070 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c023070 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fe13c01ec70 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fe13c038050 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe14b84a740 10 client.8208 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.167+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x557930d51de0 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c023070 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fe13c03b020 con 0x557930d41b80 2024-04-24T06:37:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 10 client.8208 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 10 client.8208 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 20 client.8208 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d6a560 0x557930d78640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 4 client.8208 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557930c87530 con 0x557930d6a560 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.168+0000 7fe141e00640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d6a560 0x557930d78640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.169+0000 7fe141e00640 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d6a560 0x557930d78640 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bfc9e067be3b1cc server_cookie=200fa8b847dd915f in_seq=0 out_seq=0 2024-04-24T06:37:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.169+0000 7fe126a00640 10 client.8208 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe126a00640 1 -- 172.21.15.59:0/3349364143 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557930c87530 con 0x557930d6a560 2024-04-24T06:37:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe126a00640 10 client.8208 handle_command_reply: tid=0 2024-04-24T06:37:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 client.8208 shutdown 2024-04-24T06:37:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 msgr2=0x557930d52530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01f000 tx=0x7fe13c004730 comp rx=0 tx=0).stop 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 shutdown_connections 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d6a560 0x557930d78640 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d52530 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3349364143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe126a00640 0 client.8208 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 >> 172.21.15.59:0/3349364143 conn(0x557930d3f360 msgr2=0x557930d5af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 shutdown_connections 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.170+0000 7fe14b84a740 1 -- 172.21.15.59:0/3349364143 wait complete. 2024-04-24T06:37:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.171+0000 7fe14b84a740 20 client.8208 trim_cache size 0 max 16384 2024-04-24T06:37:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.191+0000 7fe14b84a740 1 Processor -- start 2024-04-24T06:37:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.191+0000 7fe14b84a740 1 -- start start 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1441f3510 con 0x557930d41b80 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe144073b30 con 0x557930d3faa0 2024-04-24T06:37:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe144072e90 con 0x557930d5c980 2024-04-24T06:37:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.192+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45386/0 (socket says 172.21.15.59:45386) 2024-04-24T06:37:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 -- 172.21.15.59:0/3001291054 learned_addr learned my addr 172.21.15.59:0/3001291054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe141e00640 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 -- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 msgr2=0x557930d13340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 -- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 msgr2=0x557930d517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 -- 172.21.15.59:0/3001291054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557930d0ed40 con 0x557930d41b80 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.193+0000 7fe139400640 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe12c07a3a0 tx=0x7fe12c07a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627aab693d30438 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe131c00640 1 -- 172.21.15.59:0/3001291054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c092020 con 0x557930d41b80 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe131c00640 1 -- 172.21.15.59:0/3001291054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c07a410 con 0x557930d41b80 2024-04-24T06:37:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe131c00640 1 -- 172.21.15.59:0/3001291054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c07e7c0 con 0x557930d41b80 2024-04-24T06:37:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 -- 172.21.15.59:0/3001291054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 msgr2=0x557930d12e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe12c07a3a0 tx=0x7fe12c07a080 comp rx=0 tx=0).stop 2024-04-24T06:37:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 -- 172.21.15.59:0/3001291054 shutdown_connections 2024-04-24T06:37:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d13340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d12e00 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 --2- 172.21.15.59:0/3001291054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.194+0000 7fe14b84a740 1 -- 172.21.15.59:0/3001291054 >> 172.21.15.59:0/3001291054 conn(0x557930d3f360 msgr2=0x557930d6a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.195+0000 7fe14b84a740 1 -- 172.21.15.59:0/3001291054 shutdown_connections 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.195+0000 7fe14b84a740 1 -- 172.21.15.59:0/3001291054 wait complete. 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.195+0000 7fe14b84a740 1 -- start start 2024-04-24T06:37:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.195+0000 7fe14b84a740 1 Processor -- start 2024-04-24T06:37:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d12f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557930d424d0 con 0x557930d41b80 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440734e0 con 0x557930d3faa0 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557930d40310 con 0x557930d5c980 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.196+0000 7fe14b84a740 10 client.0 _ll_register_callbacks cb 0x7fe13c014f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45392/0 (socket says 172.21.15.59:45392) 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 -- 172.21.15.59:0/1271149053 learned_addr learned my addr 172.21.15.59:0/1271149053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 -- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 msgr2=0x557930d12f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d12f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 -- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 msgr2=0x557930d581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 -- 172.21.15.59:0/1271149053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557930d58bc0 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe139400640 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe12c07a110 tx=0x7fe12c008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0036b0340c7ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.197+0000 7fe126a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c096020 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fe12c07a670 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe12c07a410 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fe12c092040 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 10 client.8214 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x557930d5a2a0 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c07a410 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fe12c03a030 con 0x557930d41b80 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 10 client.8214 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 10 client.8214 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 20 client.8214 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d526f0 0x557930d2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 4 client.8214 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:37:42.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.198+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "status"}) -- 0x557930c813a0 con 0x557930d526f0 2024-04-24T06:37:42.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.199+0000 7fe141e00640 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d526f0 0x557930d2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.199+0000 7fe141e00640 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d526f0 0x557930d2ecb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401e660516ff54d0 server_cookie=871b1105e78b53dd in_seq=0 out_seq=0 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.199+0000 7fe126a00640 10 client.8214 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe126a00640 1 -- 172.21.15.59:0/1271149053 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557930c813a0 con 0x557930d526f0 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe126a00640 10 client.8214 handle_command_reply: tid=0 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 client.8214 shutdown 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 msgr2=0x557930d59d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe12c07a110 tx=0x7fe12c008070 comp rx=0 tx=0).stop 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 shutdown_connections 2024-04-24T06:37:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557930d5c980 0x557930d12f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x557930d526f0 0x557930d2ecb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557930d41b80 0x557930d59d60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 --2- 172.21.15.59:0/1271149053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557930d3faa0 0x557930d581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 >> 172.21.15.59:0/1271149053 conn(0x557930d3f360 msgr2=0x557930d0e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe126a00640 0 client.8214 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 shutdown_connections 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 1 -- 172.21.15.59:0/1271149053 wait complete. 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.200+0000 7fe14b84a740 20 client.8214 trim_cache size 0 max 16384 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:37:42.205 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8133, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0261897219999998, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 75, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 87, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 7.0370952740000003 2024-04-24T06:37:42.206 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 msgr2=0x7fe12006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 0x7fe12006ee00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b830 tx=0x7fe13c00bc30 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 msgr2=0x7fe14414a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009870 tx=0x7fe12c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 shutdown_connections 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe12006c940 0x7fe12006ee00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe14415c670 0x7fe144159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1440d0b00 0x7fe14414a5e0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 --2- 172.21.15.59:0/3757102003 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1440cd8b0 0x7fe144156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 >> 172.21.15.59:0/3757102003 conn(0x7fe14406bdf0 msgr2=0x7fe14415afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 shutdown_connections 2024-04-24T06:37:42.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.201+0000 7fe143200640 1 -- 172.21.15.59:0/3757102003 wait complete. 2024-04-24T06:37:42.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:37:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.424+0000 7faa97800640 1 Processor -- start 2024-04-24T06:37:42.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.425+0000 7faa97800640 1 -- start start 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.426+0000 7faa97800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faa9015c5e0 0x7faa900af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.426+0000 7faa97800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9015f0f0 con 0x7faa900d08b0 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.426+0000 7faa97800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9015f3d0 con 0x7faa900af920 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.426+0000 7faa97800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faa900afd80 con 0x7faa9015c5e0 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.427+0000 7faa95800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faa900d08b0 0x7faa901624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54054/0 (socket says 172.21.15.59:54054) 2024-04-24T06:37:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.427+0000 7faa95800640 1 -- 172.21.15.59:0/3807314552 learned_addr learned my addr 172.21.15.59:0/3807314552 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.427+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905357065 0 0) 0x7faa9015f0f0 con 0x7faa900d08b0 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.427+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa74003520 con 0x7faa900d08b0 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.427+0000 7faa94e00640 1 --2- 172.21.15.59:0/3807314552 >> v2:172.21.15.66:3300/0 conn(0x7faa9015c5e0 0x7faa900af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2922582703 0 0) 0x7faa74003520 con 0x7faa900d08b0 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 >> v2:172.21.15.66:3300/0 conn(0x7faa9015c5e0 msgr2=0x7faa900af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 --2- 172.21.15.59:0/3807314552 >> v2:172.21.15.66:3300/0 conn(0x7faa9015c5e0 0x7faa900af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 >> v1:172.21.15.66:6789/0 conn(0x7faa900af920 legacy=0x7faa900d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9015f6b0 con 0x7faa900d08b0 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7faa88001670 con 0x7faa900d08b0 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 >> v1:172.21.15.59:6789/0 conn(0x7faa900d08b0 legacy=0x7faa901624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa74003a00 0x7faa74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa74004800 0x7faa74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa7400c200 con 0x7faa740052e0 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa7400c3b0 con 0x7faa74003a00 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa7400c5b0 con 0x7faa74004800 2024-04-24T06:37:42.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.428+0000 7faa95800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.429+0000 7faa95800640 1 -- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa74004800 msgr2=0x7faa74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.429+0000 7faa95800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa74004800 0x7faa74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.429+0000 7faa95800640 1 -- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa74003a00 msgr2=0x7faa74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.429+0000 7faa95800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa74003a00 0x7faa74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.429+0000 7faa95800640 1 -- 172.21.15.59:0/3807314552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa74003520 con 0x7faa740052e0 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.430+0000 7faa95800640 1 -- 172.21.15.59:0/3807314552 reap_dead start 2024-04-24T06:37:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.430+0000 7faa95800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa7400b9c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c160 tx=0x7faa74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d1116b765cd7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.430+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88007c60 con 0x7faa740052e0 2024-04-24T06:37:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.430+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88005ad0 con 0x7faa740052e0 2024-04-24T06:37:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.430+0000 7faa8f400640 1 -- 172.21.15.59:0/3807314552 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88007070 con 0x7faa740052e0 2024-04-24T06:37:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 -- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 msgr2=0x7faa7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa7400b9c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c160 tx=0x7faa74005830 comp rx=0 tx=0).stop 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 -- 172.21.15.59:0/3807314552 shutdown_connections 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 --2- 172.21.15.59:0/3807314552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa7400b9c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 -- 172.21.15.59:0/3807314552 >> 172.21.15.59:0/3807314552 conn(0x7faa90067ea0 msgr2=0x7faa9006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 -- 172.21.15.59:0/3807314552 shutdown_connections 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.431+0000 7faa97800640 1 -- 172.21.15.59:0/3807314552 wait complete. 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.433+0000 7faa97800640 1 Processor -- start 2024-04-24T06:37:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.433+0000 7faa97800640 1 -- start start 2024-04-24T06:37:42.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9015ad30 0x7faa90155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa90162460 0x7faa901628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa900af3e0 con 0x7faa740052e0 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa9015fa20 con 0x7faa9015ad30 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa97800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faa900af560 con 0x7faa90162460 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa94e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa94e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45400/0 (socket says 172.21.15.59:45400) 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.434+0000 7faa94e00640 1 -- 172.21.15.59:0/918519045 learned_addr learned my addr 172.21.15.59:0/918519045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 -- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa90162460 msgr2=0x7faa901628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa90162460 0x7faa901628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 -- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9015ad30 msgr2=0x7faa90155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9015ad30 0x7faa90155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 -- 172.21.15.59:0/918519045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9015b560 con 0x7faa740052e0 2024-04-24T06:37:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa94e00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7faa8000cda0 tx=0x7faa8000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80003b90 con 0x7faa740052e0 2024-04-24T06:37:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80003d50 con 0x7faa740052e0 2024-04-24T06:37:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80004270 con 0x7faa740052e0 2024-04-24T06:37:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9015b840 con 0x7faa740052e0 2024-04-24T06:37:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.435+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa90156430 con 0x7faa740052e0 2024-04-24T06:37:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.437+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa900cd670 con 0x7faa740052e0 2024-04-24T06:37:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.438+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa80004410 con 0x7faa740052e0 2024-04-24T06:37:42.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.439+0000 7faa8cc00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 0x7faa6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.439+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7faa8008c670 con 0x7faa740052e0 2024-04-24T06:37:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.439+0000 7faa8fe00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 0x7faa6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.440+0000 7faa8fe00640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 0x7faa6c06edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faa90158190 tx=0x7faa780079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.443+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa8005a9c0 con 0x7faa740052e0 2024-04-24T06:37:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.687+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa90162e20 con 0x7faa740052e0 2024-04-24T06:37:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.688+0000 7faa8cc00640 1 -- 172.21.15.59:0/918519045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7faa8005a2c0 con 0x7faa740052e0 2024-04-24T06:37:42.690 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 msgr2=0x7faa6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 0x7faa6c06edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faa90158190 tx=0x7faa780079b0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 msgr2=0x7faa90158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7faa8000cda0 tx=0x7faa8000a660 comp rx=0 tx=0).stop 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 shutdown_connections 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faa6c06c8f0 0x7faa6c06edb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faa90162460 0x7faa901628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9015ad30 0x7faa90155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 --2- 172.21.15.59:0/918519045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa740052e0 0x7faa90158a50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:42.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 >> 172.21.15.59:0/918519045 conn(0x7faa90067ea0 msgr2=0x7faa900b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:42.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 shutdown_connections 2024-04-24T06:37:42.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:42.691+0000 7faa97800640 1 -- 172.21.15.59:0/918519045 wait complete. 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:37:42.713 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:37:42.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:42.714 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:37:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.767 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:37:42.768 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:37:42.769 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:37:42.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:42.770 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:37:42.854 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:37:42.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:42.858 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:37:42.914 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:37:42.914 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:37:42.914 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:37:42.914 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:37:42.915 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:37:42.915 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:37:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:37:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:37:43.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:43.137 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:37:43.180 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:37:43.180 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:37:43.181 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:37:43.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:43.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:37:43.317 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:37:43.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:37:43.317 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:43.370 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:37:43.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:37:43.371 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:43.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:37:43.517 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:37:43.578 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:37:43.579 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:37:43.580 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:37:43.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:43.581 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:37:43.636 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:37:43.637 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:37:43.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:37:43.720 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:37:43.762 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:37:43.762 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:37:43.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:37:43.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:37:43.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:37:43.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:37:43.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:37:43.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:37:43.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:43.766 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:37:43.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:37:43.823+0000 7f4912c81480 -1 init, newargv = 0x564eeb10e820 newargc=15 2024-04-24T06:37:43.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[66780]: starting ceph client 2024-04-24T06:37:43.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[66780]: starting fuse 2024-04-24T06:37:43.849 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:37:43.850 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:37:43.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:37:43.850 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:37:43.850 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:37:43.851 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:37:43.852 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:37:44.054 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.66780.asok 2024-04-24T06:37:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:37:44.072 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.66780.asok 2024-04-24T06:37:44.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:44.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.66780.asok status 2024-04-24T06:37:44.314 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:37:44.314 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:37:44.314 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "66780", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8226, 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:37:44.315 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8226 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2059181990 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2059181990 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8226 192.168.0.1:0/2059181990", 2024-04-24T06:37:44.316 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2059181990", 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 75, 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 87, 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:37:44.317 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:37:44.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:44.330 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:37:44.330 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:44.381 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:37:44.381 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:44.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:44.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:37:44.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:44.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-24T06:37:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.685+0000 7f87eb200640 1 Processor -- start 2024-04-24T06:37:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.686+0000 7f87eb200640 1 -- start start 2024-04-24T06:37:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.687+0000 7f87eb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f87e40b3990 0x7f87e409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.687+0000 7f87eb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.687+0000 7f87eb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e40b3df0 con 0x7f87e415c8a0 2024-04-24T06:37:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.687+0000 7f87eb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f87e40cd780 con 0x7f87e40b3990 2024-04-24T06:37:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.687+0000 7f87eb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f87e40cda60 con 0x7f87e409af40 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.688+0000 7f87e3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f87e415c8a0 0x7f87e40ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54058/0 (socket says 172.21.15.59:54058) 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.688+0000 7f87e3e00640 1 -- 172.21.15.59:0/1976100781 learned_addr learned my addr 172.21.15.59:0/1976100781 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.688+0000 7f87e3400640 1 --2- 172.21.15.59:0/1976100781 >> v2:172.21.15.66:3300/0 conn(0x7f87e40b3990 0x7f87e409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.688+0000 7f87e9200640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.688+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038760003 0 0) 0x7f87e40b3df0 con 0x7f87e415c8a0 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87c8003520 con 0x7f87e415c8a0 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2276771154 0 0) 0x7f87c8003520 con 0x7f87e415c8a0 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e3400640 1 -- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 msgr2=0x7f87e40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e3400640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e3400640 1 -- 172.21.15.59:0/1976100781 >> v1:172.21.15.59:6789/0 conn(0x7f87e415c8a0 legacy=0x7f87e40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.689+0000 7f87e3400640 1 -- 172.21.15.59:0/1976100781 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e40cdd40 con 0x7f87e40b3990 2024-04-24T06:37:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.690+0000 7f87e3400640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e409aa00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b11f519439da5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.691+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc037070 con 0x7f87e40b3990 2024-04-24T06:37:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.691+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc02aba0 con 0x7f87e40b3990 2024-04-24T06:37:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87e2a00640 1 -- 172.21.15.59:0/1976100781 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40019f0 con 0x7f87e40b3990 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 -- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 msgr2=0x7f87e409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e409aa00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02a110 comp rx=0 tx=0).stop 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 -- 172.21.15.59:0/1976100781 shutdown_connections 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 --2- 172.21.15.59:0/1976100781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e409aa00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 -- 172.21.15.59:0/1976100781 >> 172.21.15.59:0/1976100781 conn(0x7f87e40d2600 msgr2=0x7f87e40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 -- 172.21.15.59:0/1976100781 shutdown_connections 2024-04-24T06:37:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.692+0000 7f87eb200640 1 -- 172.21.15.59:0/1976100781 wait complete. 2024-04-24T06:37:44.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.694+0000 7f87eb200640 1 Processor -- start 2024-04-24T06:37:44.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.694+0000 7f87eb200640 1 -- start start 2024-04-24T06:37:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87e415cc80 con 0x7f87e415c8a0 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40ce300 con 0x7f87e40b3990 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.695+0000 7f87eb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f87e40ced60 con 0x7f87e409af40 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45408/0 (socket says 172.21.15.59:45408) 2024-04-24T06:37:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 -- 172.21.15.59:0/1586748373 learned_addr learned my addr 172.21.15.59:0/1586748373 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e3400640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e3e00640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 -- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 msgr2=0x7f87e40b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 -- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 msgr2=0x7f87e415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.696+0000 7f87e9200640 1 -- 172.21.15.59:0/1586748373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e40c87e0 con 0x7f87e415c8a0 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87e9200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e930 tx=0x7f87dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87dbe00640 1 -- 172.21.15.59:0/1586748373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87e415c8a0 2024-04-24T06:37:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87dbe00640 1 -- 172.21.15.59:0/1586748373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc00c980 con 0x7f87e415c8a0 2024-04-24T06:37:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87dbe00640 1 -- 172.21.15.59:0/1586748373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00cd70 con 0x7f87e415c8a0 2024-04-24T06:37:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e406e6c0 con 0x7f87e415c8a0 2024-04-24T06:37:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.697+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e40cbaf0 con 0x7f87e415c8a0 2024-04-24T06:37:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.699+0000 7f87dbe00640 1 -- 172.21.15.59:0/1586748373 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87dc01a070 con 0x7f87e415c8a0 2024-04-24T06:37:44.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.700+0000 7f87dbe00640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 0x7f87c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.701+0000 7f87dbe00640 1 -- 172.21.15.59:0/1586748373 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f87dc08bc20 con 0x7f87e415c8a0 2024-04-24T06:37:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.701+0000 7f87e3e00640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 0x7f87c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.702+0000 7f87ee01f740 1 Processor -- start 2024-04-24T06:37:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.702+0000 7f87e3e00640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 0x7f87c006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f87d4001560 tx=0x7f87d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.702+0000 7f87ee01f740 1 -- start start 2024-04-24T06:37:44.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c43fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442be40 0x5571c4400c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571c4365e70 con 0x5571c442c580 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5571c430c380 con 0x5571c442be40 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5571c4361e30 con 0x5571c4449460 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45422/0 (socket says 172.21.15.59:45422) 2024-04-24T06:37:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 -- 172.21.15.59:0/3746443408 learned_addr learned my addr 172.21.15.59:0/3746443408 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 -- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 msgr2=0x5571c43fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3e00640 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c43fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c43fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e9200640 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442be40 0x5571c4400c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 -- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442be40 msgr2=0x5571c4400c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442be40 0x5571c4400c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.703+0000 7f87e3400640 1 -- 172.21.15.59:0/3746443408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c41407e0 con 0x5571c442c580 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.704+0000 7f87e3400640 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098c0 tx=0x7f87cc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24e251967963e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.704+0000 7f87d8c00640 1 -- 172.21.15.59:0/3746443408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc042070 con 0x5571c442c580 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.704+0000 7f87d8c00640 1 -- 172.21.15.59:0/3746443408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc038040 con 0x5571c442c580 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.704+0000 7f87d8c00640 1 -- 172.21.15.59:0/3746443408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc037070 con 0x5571c442c580 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 -- 172.21.15.59:0/3746443408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 msgr2=0x5571c442b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098c0 tx=0x7f87cc02c000 comp rx=0 tx=0).stop 2024-04-24T06:37:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 -- 172.21.15.59:0/3746443408 shutdown_connections 2024-04-24T06:37:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442be40 0x5571c4400c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442b900 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3746443408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c43fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 -- 172.21.15.59:0/3746443408 >> 172.21.15.59:0/3746443408 conn(0x5571c43fe060 msgr2=0x5571c4446900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 -- 172.21.15.59:0/3746443408 shutdown_connections 2024-04-24T06:37:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.705+0000 7f87ee01f740 1 -- 172.21.15.59:0/3746443408 wait complete. 2024-04-24T06:37:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.706+0000 7f87ee01f740 1 -- start start 2024-04-24T06:37:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.706+0000 7f87ee01f740 1 Processor -- start 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c442ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c4428760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571c4400210 con 0x5571c4445380 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5571c431f660 con 0x5571c442c580 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5571c43ff600 con 0x5571c4449460 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87ee01f740 10 client.0 _ll_register_callbacks cb 0x5571c430daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45424/0 (socket says 172.21.15.59:45424) 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.707+0000 7f87e9200640 1 -- 172.21.15.59:0/210584556 learned_addr learned my addr 172.21.15.59:0/210584556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 -- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 msgr2=0x5571c4428760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c4428760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 -- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 msgr2=0x5571c442ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e3e00640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c442ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c442ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 -- 172.21.15.59:0/210584556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c41a0f70 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87e9200640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87dc061fe0 tx=0x7f87dc059930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc878a7e7b44b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87d1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc021040 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f87dc058d80 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.708+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87cc038040 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f87dc062510 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 10 client.8238 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5571c4038170 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc038040 con 0x5571c4445380 2024-04-24T06:37:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f87dc024060 con 0x5571c4445380 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 10 client.8238 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 10 client.8238 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 20 client.8238 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c442cf40 0x5571c442d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 4 client.8238 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c4373890 con 0x5571c442cf40 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.709+0000 7f87e3e00640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c442cf40 0x5571c442d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.710+0000 7f87e3e00640 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c442cf40 0x5571c442d370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60d86f78bfd7891 server_cookie=4b0650d46b316e7b in_seq=0 out_seq=0 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.710+0000 7f87d1600640 10 client.8238 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87d1600640 1 -- 172.21.15.59:0/210584556 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571c4373890 con 0x5571c442cf40 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87d1600640 10 client.8238 handle_command_reply: tid=0 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 client.8238 shutdown 2024-04-24T06:37:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 msgr2=0x5571c4428ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f87dc061fe0 tx=0x7f87dc059930 comp rx=0 tx=0).stop 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 shutdown_connections 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c442cf40 0x5571c442d370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4445380 0x5571c4428ca0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c4428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 --2- 172.21.15.59:0/210584556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c442ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 >> 172.21.15.59:0/210584556 conn(0x5571c442be40 msgr2=0x5571c43fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.711+0000 7f87d1600640 0 client.8238 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.712+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 shutdown_connections 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.712+0000 7f87ee01f740 1 -- 172.21.15.59:0/210584556 wait complete. 2024-04-24T06:37:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.712+0000 7f87ee01f740 20 client.8238 trim_cache size 0 max 16384 2024-04-24T06:37:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.731+0000 7f87ee01f740 1 Processor -- start 2024-04-24T06:37:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.731+0000 7f87ee01f740 1 -- start start 2024-04-24T06:37:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.732+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.732+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c4445380 0x5571c4443b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.732+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c44000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87ee01f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4200e80 con 0x5571c442c580 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4200030 con 0x5571c4445380 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f87e406e290 con 0x5571c4449460 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45434/0 (socket says 172.21.15.59:45434) 2024-04-24T06:37:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 -- 172.21.15.59:0/3666191620 learned_addr learned my addr 172.21.15.59:0/3666191620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3400640 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c4445380 0x5571c4443b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 -- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 msgr2=0x5571c44000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c44000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 -- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c4445380 msgr2=0x5571c4443b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c4445380 0x5571c4443b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.733+0000 7f87e3e00640 1 -- 172.21.15.59:0/3666191620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c4448c50 con 0x5571c442c580 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.734+0000 7f87e3e00640 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005c00 tx=0x7f87d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33d6b2bbd9c4a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.734+0000 7f87d3e00640 1 -- 172.21.15.59:0/3666191620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4023330 con 0x5571c442c580 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.734+0000 7f87d3e00640 1 -- 172.21.15.59:0/3666191620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d4028070 con 0x5571c442c580 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.734+0000 7f87d3e00640 1 -- 172.21.15.59:0/3666191620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4023710 con 0x5571c442c580 2024-04-24T06:37:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 -- 172.21.15.59:0/3666191620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 msgr2=0x5571c442e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005c00 tx=0x7f87d4011000 comp rx=0 tx=0).stop 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 -- 172.21.15.59:0/3666191620 shutdown_connections 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4449460 0x5571c44000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c4445380 0x5571c4443b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 --2- 172.21.15.59:0/3666191620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c442c580 0x5571c442e6e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 -- 172.21.15.59:0/3666191620 >> 172.21.15.59:0/3666191620 conn(0x5571c442be40 msgr2=0x5571c4427df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 -- 172.21.15.59:0/3666191620 shutdown_connections 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.735+0000 7f87ee01f740 1 -- 172.21.15.59:0/3666191620 wait complete. 2024-04-24T06:37:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.736+0000 7f87ee01f740 1 -- start start 2024-04-24T06:37:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.736+0000 7f87ee01f740 1 Processor -- start 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c44c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4445380 0x5571c4449f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571c446a260 con 0x5571c4449460 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40cb2f0 con 0x5571c442c580 2024-04-24T06:37:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5571c43fc920 con 0x5571c4445380 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87ee01f740 10 client.0 _ll_register_callbacks cb 0x7f87dc009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87e9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45448/0 (socket says 172.21.15.59:45448) 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87e9200640 1 -- 172.21.15.59:0/864046158 learned_addr learned my addr 172.21.15.59:0/864046158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87e3e00640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c44c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.737+0000 7f87e3400640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4445380 0x5571c4449f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 -- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4445380 msgr2=0x5571c4449f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4445380 0x5571c4449f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 -- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 msgr2=0x5571c44c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c44c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 -- 172.21.15.59:0/864046158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c4447c80 con 0x5571c4449460 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e3e00640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c44c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87e9200640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f87dc05dea0 tx=0x7f87dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c7464b50966e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87d1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc082060 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f87dc0587f0 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d4028070 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f87dc021070 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87ee01f740 10 client.8244 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.738+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5571c442ec20 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d4028070 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f87dc0589f0 con 0x5571c4449460 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 10 client.8244 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 10 client.8244 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 20 client.8244 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c44c1fd0 0x5571c43fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 4 client.8244 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-24T06:37:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x5571c44278d0 con 0x5571c44c1fd0 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.739+0000 7f87e3e00640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c44c1fd0 0x5571c43fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.740+0000 7f87e3e00640 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c44c1fd0 0x5571c43fac80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f908dd09897553 server_cookie=48b088c8d907749d in_seq=0 out_seq=0 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.740+0000 7f87d1600640 10 client.8244 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.740+0000 7f87d1600640 1 -- 172.21.15.59:0/864046158 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571c44278d0 con 0x5571c44c1fd0 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.740+0000 7f87d1600640 10 client.8244 handle_command_reply: tid=0 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.740+0000 7f87ee01f740 1 client.8244 shutdown 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 msgr2=0x5571c442e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f87dc05dea0 tx=0x7f87dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 shutdown_connections 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x5571c44c1fd0 0x5571c43fac80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571c4449460 0x5571c442e6e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5571c4445380 0x5571c4449f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 --2- 172.21.15.59:0/864046158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571c442c580 0x5571c44c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 >> 172.21.15.59:0/864046158 conn(0x5571c442be40 msgr2=0x5571c4427d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87d1600640 0 client.8244 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 shutdown_connections 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 1 -- 172.21.15.59:0/864046158 wait complete. 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.741+0000 7f87ee01f740 20 client.8244 trim_cache size 0 max 16384 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "93257841-a3d3-4d74-ada6-5ab9700e7c31", 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:37:44.747 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:37:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 msgr2=0x7f87c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 0x7f87c006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f87d4001560 tx=0x7f87d4009040 comp rx=0 tx=0).stop 2024-04-24T06:37:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 msgr2=0x7f87e415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e930 tx=0x7f87dc00edf0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 shutdown_connections 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f87c006c8f0 0x7f87c006edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e415c8a0 0x7f87e415f1a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e40b3990 0x7f87e415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 --2- 172.21.15.59:0/1586748373 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f87e409af40 0x7f87e40b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 >> 172.21.15.59:0/1586748373 conn(0x7f87e40d2600 msgr2=0x7f87e40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 shutdown_connections 2024-04-24T06:37:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.742+0000 7f87eb200640 1 -- 172.21.15.59:0/1586748373 wait complete. 2024-04-24T06:37:44.749 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-24T06:37:44.743+0000 7f2716c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x555c9de0a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:37:44.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:37:44.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:37:44.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.977+0000 7fa787000640 1 Processor -- start 2024-04-24T06:37:44.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.977+0000 7fa787000640 1 -- start start 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa787000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 0x7fa7800d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa787000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa780072480 0x7fa78006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa787000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7800d0fd0 con 0x7fa7800b4670 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa787000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa7800b35d0 con 0x7fa780072480 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa787000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa7800b38b0 con 0x7fa780162c10 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa77f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa7800b4670 0x7fa780071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54068/0 (socket says 172.21.15.59:54068) 2024-04-24T06:37:44.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.978+0000 7fa77f400640 1 -- 172.21.15.59:0/105803985 learned_addr learned my addr 172.21.15.59:0/105803985 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77fe00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 0x7fa7800d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa785000640 1 --2- 172.21.15.59:0/105803985 >> v2:172.21.15.66:3300/0 conn(0x7fa780072480 0x7fa78006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165903340 0 0) 0x7fa76c001110 con 0x7fa7800b4670 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa764003520 con 0x7fa7800b4670 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1674332155 0 0) 0x7fa764003520 con 0x7fa7800b4670 2024-04-24T06:37:44.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 msgr2=0x7fa7800d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 0x7fa7800d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 >> v2:172.21.15.66:3300/0 conn(0x7fa780072480 msgr2=0x7fa78006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 --2- 172.21.15.59:0/105803985 >> v2:172.21.15.66:3300/0 conn(0x7fa780072480 0x7fa78006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7800b3b90 con 0x7fa7800b4670 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa76c001840 con 0x7fa7800b4670 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 >> v1:172.21.15.59:6789/0 conn(0x7fa7800b4670 legacy=0x7fa780071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7640039b0 0x7fa764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa764004800 0x7fa764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 0x7fa7640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7800b35d0 con 0x7fa7640039b0 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7800b38b0 con 0x7fa764004800 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.979+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa76400b9c0 con 0x7fa780162c10 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7640039b0 0x7fa764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77f400640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa764004800 0x7fa764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 -- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 msgr2=0x7fa7640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa780162c10 0x7fa7640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 -- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa764004800 msgr2=0x7fa764004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa764004800 0x7fa764004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77fe00640 1 -- 172.21.15.59:0/105803985 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa764003520 con 0x7fa7640039b0 2024-04-24T06:37:44.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.980+0000 7fa77f400640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa764004800 0x7fa764004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.981+0000 7fa77fe00640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7640039b0 0x7fa764003dd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa770009ac0 tx=0x7fa764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50663d978f3dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.981+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770036070 con 0x7fa7640039b0 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.981+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770031400 con 0x7fa7640039b0 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.981+0000 7fa77ea00640 1 -- 172.21.15.59:0/105803985 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770009690 con 0x7fa7640039b0 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 -- 172.21.15.59:0/105803985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7640039b0 msgr2=0x7fa764003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 --2- 172.21.15.59:0/105803985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7640039b0 0x7fa764003dd0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa770009ac0 tx=0x7fa764005830 comp rx=0 tx=0).stop 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa785000640 1 -- 172.21.15.59:0/105803985 reap_dead start 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 -- 172.21.15.59:0/105803985 shutdown_connections 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 -- 172.21.15.59:0/105803985 >> 172.21.15.59:0/105803985 conn(0x7fa780067d90 msgr2=0x7fa78006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 -- 172.21.15.59:0/105803985 shutdown_connections 2024-04-24T06:37:44.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.982+0000 7fa787000640 1 -- 172.21.15.59:0/105803985 wait complete. 2024-04-24T06:37:44.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.984+0000 7fa787000640 1 Processor -- start 2024-04-24T06:37:44.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.984+0000 7fa787000640 1 -- start start 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7640039b0 0x7fa78014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 0x7fa78014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa780162ff0 con 0x7fa78014f2f0 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7800b4150 con 0x7fa7640039b0 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.985+0000 7fa787000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa780163170 con 0x7fa78014e950 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45478/0 (socket says 172.21.15.59:45478) 2024-04-24T06:37:44.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 -- 172.21.15.59:0/3477614075 learned_addr learned my addr 172.21.15.59:0/3477614075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7640039b0 0x7fa78014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa77f400640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 0x7fa78014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 -- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 msgr2=0x7fa78014edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 0x7fa78014edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 -- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7640039b0 msgr2=0x7fa78014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7640039b0 0x7fa78014e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 -- 172.21.15.59:0/3477614075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa780160c10 con 0x7fa78014f2f0 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa77f400640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 0x7fa78014edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.986+0000 7fa785000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa7800d1200 tx=0x7fa77800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.987+0000 7fa777e00640 1 -- 172.21.15.59:0/3477614075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800bae0 con 0x7fa78014f2f0 2024-04-24T06:37:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.987+0000 7fa777e00640 1 -- 172.21.15.59:0/3477614075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77801a070 con 0x7fa78014f2f0 2024-04-24T06:37:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.987+0000 7fa777e00640 1 -- 172.21.15.59:0/3477614075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778014400 con 0x7fa78014f2f0 2024-04-24T06:37:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.987+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa780160ef0 con 0x7fa78014f2f0 2024-04-24T06:37:44.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.987+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7801614b0 con 0x7fa78014f2f0 2024-04-24T06:37:44.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.989+0000 7fa777e00640 1 -- 172.21.15.59:0/3477614075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa778005d10 con 0x7fa78014f2f0 2024-04-24T06:37:44.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.990+0000 7fa777e00640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 0x7fa76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.990+0000 7fa777e00640 1 -- 172.21.15.59:0/3477614075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fa77808be50 con 0x7fa78014f2f0 2024-04-24T06:37:44.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.990+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 0x7fa76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.991+0000 7fa789d94740 1 Processor -- start 2024-04-24T06:37:44.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.991+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 0x7fa76006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa770033000 tx=0x7fa770008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.991+0000 7fa789d94740 1 -- start start 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde4d50 0x561dffde40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4610 0x561dffdb9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561dffd1e1f0 con 0x561dffe01c30 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561dffcc4350 con 0x561dffde4d50 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561dffd1a1b0 con 0x561dffde4610 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:44.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45486/0 (socket says 172.21.15.59:45486) 2024-04-24T06:37:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.992+0000 7fa77fe00640 1 -- 172.21.15.59:0/3703421571 learned_addr learned my addr 172.21.15.59:0/3703421571 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:44.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 -- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4610 msgr2=0x561dffdb9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4610 0x561dffdb9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 -- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde4d50 msgr2=0x561dffde40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde4d50 0x561dffde40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 -- 172.21.15.59:0/3703421571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dffd18f50 con 0x561dffe01c30 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.993+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa7700042b0 tx=0x7fa77002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e633654f2e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa774c00640 1 -- 172.21.15.59:0/3703421571 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770040040 con 0x561dffe01c30 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa774c00640 1 -- 172.21.15.59:0/3703421571 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770003d90 con 0x561dffe01c30 2024-04-24T06:37:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa774c00640 1 -- 172.21.15.59:0/3703421571 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770036040 con 0x561dffe01c30 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 -- 172.21.15.59:0/3703421571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 msgr2=0x561dffdb45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa7700042b0 tx=0x7fa77002e000 comp rx=0 tx=0).stop 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 -- 172.21.15.59:0/3703421571 shutdown_connections 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4610 0x561dffdb9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde4d50 0x561dffde40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 --2- 172.21.15.59:0/3703421571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffe01c30 0x561dffdb45c0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 -- 172.21.15.59:0/3703421571 >> 172.21.15.59:0/3703421571 conn(0x561dffdb6830 msgr2=0x561dffdff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.994+0000 7fa789d94740 1 -- 172.21.15.59:0/3703421571 shutdown_connections 2024-04-24T06:37:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.995+0000 7fa789d94740 1 -- 172.21.15.59:0/3703421571 wait complete. 2024-04-24T06:37:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.995+0000 7fa789d94740 1 -- start start 2024-04-24T06:37:44.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.995+0000 7fa789d94740 1 Processor -- start 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffe01c30 0x561dffde0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 0x561dffdf75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561dffdb89e0 con 0x561dffde4d50 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561dffcd7690 con 0x561dffde6290 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.996+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561dffdb7dd0 con 0x561dffe01c30 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa789d94740 10 client.0 _ll_register_callbacks cb 0x561dffd13650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45498/0 (socket says 172.21.15.59:45498) 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 -- 172.21.15.59:0/1768319432 learned_addr learned my addr 172.21.15.59:0/1768319432 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa785000640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 0x561dffdf75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77f400640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffe01c30 0x561dffde0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 -- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffe01c30 msgr2=0x561dffde0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffe01c30 0x561dffde0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 -- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 msgr2=0x561dffdf75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 0x561dffdf75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa77fe00640 1 -- 172.21.15.59:0/1768319432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dffcd6920 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.997+0000 7fa785000640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 0x561dffdf75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa77fe00640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fa77003f040 tx=0x7fa7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83b4b4438b08fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa769600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770055060 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fa770045410 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa770003d90 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa770040070 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa789d94740 10 client.8256 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.998+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561dffafa7d0 con 0x561dffde4d50 2024-04-24T06:37:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770003d90 con 0x561dffde4d50 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fa77005e020 con 0x561dffde4d50 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 10 client.8256 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 10 client.8256 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 20 client.8256 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffe12890 0x561dffe12ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 4 client.8256 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dffd2bc10 con 0x561dffe12890 2024-04-24T06:37:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:44.999+0000 7fa77f400640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffe12890 0x561dffe12ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.000+0000 7fa77f400640 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffe12890 0x561dffe12ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d626f31bd5bcec8 server_cookie=faeabcc3a676b02c in_seq=0 out_seq=0 2024-04-24T06:37:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.000+0000 7fa769600640 10 client.8256 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.000+0000 7fa769600640 1 -- 172.21.15.59:0/1768319432 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561dffd2bc10 con 0x561dffe12890 2024-04-24T06:37:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa769600640 10 client.8256 handle_command_reply: tid=0 2024-04-24T06:37:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 client.8256 shutdown 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 msgr2=0x561dffe0f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fa77003f040 tx=0x7fa7700046c0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 shutdown_connections 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffde6290 0x561dffdf75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffe01c30 0x561dffde0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffe12890 0x561dffe12ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 --2- 172.21.15.59:0/1768319432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde4d50 0x561dffe0f960 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 >> 172.21.15.59:0/1768319432 conn(0x561dffde4610 msgr2=0x561dffdb3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa769600640 0 client.8256 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 shutdown_connections 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 1 -- 172.21.15.59:0/1768319432 wait complete. 2024-04-24T06:37:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.001+0000 7fa789d94740 20 client.8256 trim_cache size 0 max 16384 2024-04-24T06:37:45.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.021+0000 7fa789d94740 1 Processor -- start 2024-04-24T06:37:45.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.021+0000 7fa789d94740 1 -- start start 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffe00c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffde1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa780205d10 con 0x561dffde6290 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa780204d50 con 0x561dffe01c30 2024-04-24T06:37:45.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.022+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa7800af2e0 con 0x561dffde4d50 2024-04-24T06:37:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa785000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffde1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45512/0 (socket says 172.21.15.59:45512) 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 -- 172.21.15.59:0/3492809617 learned_addr learned my addr 172.21.15.59:0/3492809617 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffe00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 -- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 msgr2=0x561dffe00c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffe00c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 -- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 msgr2=0x561dffde1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffde1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 -- 172.21.15.59:0/3492809617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dffa303f0 con 0x561dffde6290 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.023+0000 7fa77f400640 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0012f0 tx=0x7fa76c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bf68703bc619fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa76be00640 1 -- 172.21.15.59:0/3492809617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c007180 con 0x561dffde6290 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa76be00640 1 -- 172.21.15.59:0/3492809617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c01c070 con 0x561dffde6290 2024-04-24T06:37:45.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa76be00640 1 -- 172.21.15.59:0/3492809617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c007540 con 0x561dffde6290 2024-04-24T06:37:45.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 -- 172.21.15.59:0/3492809617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 msgr2=0x561dffde5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:45.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0012f0 tx=0x7fa76c007f10 comp rx=0 tx=0).stop 2024-04-24T06:37:45.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 -- 172.21.15.59:0/3492809617 shutdown_connections 2024-04-24T06:37:45.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffde1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde5cb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 --2- 172.21.15.59:0/3492809617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffe00c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 -- 172.21.15.59:0/3492809617 >> 172.21.15.59:0/3492809617 conn(0x561dffde4610 msgr2=0x561dffde0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.024+0000 7fa789d94740 1 -- 172.21.15.59:0/3492809617 shutdown_connections 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.025+0000 7fa789d94740 1 -- 172.21.15.59:0/3492809617 wait complete. 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.025+0000 7fa789d94740 1 -- start start 2024-04-24T06:37:45.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.025+0000 7fa789d94740 1 Processor -- start 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffde79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffe78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa780204700 con 0x561dffde6290 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561dffde01e0 con 0x561dffe01c30 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x561dffdfeb80 con 0x561dffde4d50 2024-04-24T06:37:45.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.026+0000 7fa789d94740 10 client.0 _ll_register_callbacks cb 0x7fa7640036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45518/0 (socket says 172.21.15.59:45518) 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 -- 172.21.15.59:0/3144427780 learned_addr learned my addr 172.21.15.59:0/3144427780 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa785000640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffe78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffde79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 -- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 msgr2=0x561dffde79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffde79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 -- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 msgr2=0x561dffe78d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffe78d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 -- 172.21.15.59:0/3144427780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dff9f00f0 con 0x561dffde6290 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa785000640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffe78d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa77f400640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0019b0 tx=0x7fa7800b5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90ff705ea08e8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:45.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.027+0000 7fa769600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:37:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c007590 con 0x561dffde6290 2024-04-24T06:37:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fa76c007730 con 0x561dffde6290 2024-04-24T06:37:45.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76c01c070 con 0x561dffde6290 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa76c030050 con 0x561dffde6290 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 10 client.8262 fetch_fsmap learned FSMap version 75 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561dffce3810 con 0x561dffde6290 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c01c070 con 0x561dffde6290 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fa76c033020 con 0x561dffde6290 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 10 client.8262 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 10 client.8262 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.028+0000 7fa789d94740 20 client.8262 populate_metadata read hostname 'smithi059' 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffde16b0 0x561dffe79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa789d94740 4 client.8262 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561dffdb5c80 con 0x561dffde16b0 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffde16b0 0x561dffe79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa77fe00640 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffde16b0 0x561dffe79270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a813715888a1c6ad server_cookie=e97404cdda024d80 in_seq=0 out_seq=0 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa769600640 10 client.8262 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:45.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.029+0000 7fa769600640 1 -- 172.21.15.59:0/3144427780 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x561dffdb5c80 con 0x561dffde16b0 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa769600640 10 client.8262 handle_command_reply: tid=0 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 client.8262 shutdown 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 msgr2=0x561dffde7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0019b0 tx=0x7fa7800b5340 comp rx=0 tx=0).stop 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 shutdown_connections 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dffe01c30 0x561dffe78d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x561dffde16b0 0x561dffe79270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561dffde6290 0x561dffde7f20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 --2- 172.21.15.59:0/3144427780 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x561dffde4d50 0x561dffde79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa769600640 0 client.8262 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 >> 172.21.15.59:0/3144427780 conn(0x561dffde4610 msgr2=0x561dffdb7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 shutdown_connections 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 1 -- 172.21.15.59:0/3144427780 wait complete. 2024-04-24T06:37:45.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.030+0000 7fa789d94740 20 client.8262 trim_cache size 0 max 16384 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "93257841-a3d3-4d74-ada6-5ab9700e7c31": { 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "93257841-a3d3-4d74-ada6-5ab9700e7c31", 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:37:45.035 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:37:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 msgr2=0x7fa76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 0x7fa76006edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa770033000 tx=0x7fa770008030 comp rx=0 tx=0).stop 2024-04-24T06:37:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 msgr2=0x7fa78014f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:45.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fa7800d1200 tx=0x7fa77800b480 comp rx=0 tx=0).stop 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 shutdown_connections 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa76006c8f0 0x7fa76006edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa78014f2f0 0x7fa78014f9e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa78014e950 0x7fa78014edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 --2- 172.21.15.59:0/3477614075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7640039b0 0x7fa78014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 >> 172.21.15.59:0/3477614075 conn(0x7fa780067d90 msgr2=0x7fa780163520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 shutdown_connections 2024-04-24T06:37:45.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:37:45.031+0000 7fa787000640 1 -- 172.21.15.59:0/3477614075 wait complete. 2024-04-24T06:37:45.049 INFO:tasks.cephfs.filesystem:scrub status for tag:93257841-a3d3-4d74-ada6-5ab9700e7c31 - {'path': '/dir', 'tag': '93257841-a3d3-4d74-ada6-5ab9700e7c31', 'options': 'recursive,repair'} 2024-04-24T06:38:07.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:38:07.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:38:15.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:15.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-24T06:38:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.273+0000 7f8c47000640 1 Processor -- start 2024-04-24T06:38:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.273+0000 7f8c47000640 1 -- start start 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c47000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c400af750 0x7f8c4006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c47000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400ae6b0 con 0x7f8c400d05d0 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c47000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400ae990 con 0x7f8c4015b220 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c47000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8c400afbe0 con 0x7f8c400af750 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c45200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c400af750 0x7f8c4006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c45200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c400af750 0x7f8c4006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35066/0 (socket says 172.21.15.59:35066) 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c45200640 1 -- 172.21.15.59:0/296700924 learned_addr learned my addr 172.21.15.59:0/296700924 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.275+0000 7f8c46600640 1 --1- 172.21.15.59:0/296700924 >> v1:172.21.15.59:6789/0 conn(0x7f8c400d05d0 0x7f8c400d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52770/0 (socket says 172.21.15.59:52770) 2024-04-24T06:38:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.276+0000 7f8c45200640 1 -- 172.21.15.59:0/296700924 >> v1:172.21.15.66:6789/0 conn(0x7f8c4015b220 legacy=0x7f8c400adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.276+0000 7f8c3fe00640 1 -- 172.21.15.59:0/296700924 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897673615 0 0) 0x7f8c400ae6b0 con 0x7f8c400d05d0 2024-04-24T06:38:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.276+0000 7f8c45200640 1 -- 172.21.15.59:0/296700924 >> v1:172.21.15.59:6789/0 conn(0x7f8c400d05d0 legacy=0x7f8c400d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.276+0000 7f8c45200640 1 -- 172.21.15.59:0/296700924 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c400aec70 con 0x7f8c400af750 2024-04-24T06:38:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.277+0000 7f8c45200640 1 --2- 172.21.15.59:0/296700924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4006f8d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009870 tx=0x7f8c2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce1ac71147aaf0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.277+0000 7f8c3fe00640 1 -- 172.21.15.59:0/296700924 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c037070 con 0x7f8c400af750 2024-04-24T06:38:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.277+0000 7f8c3fe00640 1 -- 172.21.15.59:0/296700924 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c02ae50 con 0x7f8c400af750 2024-04-24T06:38:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.277+0000 7f8c3fe00640 1 -- 172.21.15.59:0/296700924 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c032860 con 0x7f8c400af750 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 -- 172.21.15.59:0/296700924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 msgr2=0x7f8c4006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 --2- 172.21.15.59:0/296700924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4006f8d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009870 tx=0x7f8c2c02a300 comp rx=0 tx=0).stop 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 -- 172.21.15.59:0/296700924 shutdown_connections 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 --2- 172.21.15.59:0/296700924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4006f8d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 -- 172.21.15.59:0/296700924 >> 172.21.15.59:0/296700924 conn(0x7f8c4006bdf0 msgr2=0x7f8c4006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 -- 172.21.15.59:0/296700924 shutdown_connections 2024-04-24T06:38:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.278+0000 7f8c47000640 1 -- 172.21.15.59:0/296700924 wait complete. 2024-04-24T06:38:15.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.280+0000 7f8c47000640 1 Processor -- start 2024-04-24T06:38:15.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.280+0000 7f8c47000640 1 -- start start 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8c400d05d0 0x7f8c40145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 0x7f8c4009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015b600 con 0x7f8c400af750 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400af230 con 0x7f8c4015b220 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c47000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4015b780 con 0x7f8c400d05d0 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c45c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c45c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35068/0 (socket says 172.21.15.59:35068) 2024-04-24T06:38:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.281+0000 7f8c45c00640 1 -- 172.21.15.59:0/3642367538 learned_addr learned my addr 172.21.15.59:0/3642367538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45200640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8c400d05d0 0x7f8c40145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 -- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8c400d05d0 msgr2=0x7f8c40145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c46600640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 0x7f8c4009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8c400d05d0 0x7f8c40145c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 -- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 msgr2=0x7f8c4009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 0x7f8c4009da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 -- 172.21.15.59:0/3642367538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4009c960 con 0x7f8c400af750 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c46600640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 0x7f8c4009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c45c00640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c430 tx=0x7f8c3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c3d600640 1 -- 172.21.15.59:0/3642367538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34017070 con 0x7f8c400af750 2024-04-24T06:38:15.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c3d600640 1 -- 172.21.15.59:0/3642367538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c340052d0 con 0x7f8c400af750 2024-04-24T06:38:15.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.282+0000 7f8c3d600640 1 -- 172.21.15.59:0/3642367538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34003830 con 0x7f8c400af750 2024-04-24T06:38:15.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.283+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4009cc40 con 0x7f8c400af750 2024-04-24T06:38:15.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.283+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4009d200 con 0x7f8c400af750 2024-04-24T06:38:15.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.285+0000 7f8c3d600640 1 -- 172.21.15.59:0/3642367538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c3401a070 con 0x7f8c400af750 2024-04-24T06:38:15.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.286+0000 7f8c3d600640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 0x7f8c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.286+0000 7f8c3d600640 1 -- 172.21.15.59:0/3642367538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f8c3408af80 con 0x7f8c400af750 2024-04-24T06:38:15.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.286+0000 7f8c45200640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 0x7f8c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.287+0000 7f8c4b364740 1 Processor -- start 2024-04-24T06:38:15.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.287+0000 7f8c4b364740 1 -- start start 2024-04-24T06:38:15.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.287+0000 7f8c45200640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 0x7f8c2006edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c034000 tx=0x7f8c2c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55639bfe11f0 con 0x55639c0a77e0 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55639bf87350 con 0x55639c0a7f20 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55639bfdd1b0 con 0x55639c0c4e00 2024-04-24T06:38:15.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35082/0 (socket says 172.21.15.59:35082) 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 -- 172.21.15.59:0/2637893185 learned_addr learned my addr 172.21.15.59:0/2637893185 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c45200640 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 -- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 msgr2=0x55639c077790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 -- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 msgr2=0x55639c0a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.288+0000 7f8c46600640 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.289+0000 7f8c46600640 1 -- 172.21.15.59:0/2637893185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55639bfdbf50 con 0x55639c0a77e0 2024-04-24T06:38:15.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.289+0000 7f8c46600640 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8c38001ef0 tx=0x7f8c3800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b52b0c6965f0755 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.289+0000 7f8c32000640 1 -- 172.21.15.59:0/2637893185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38004b60 con 0x55639c0a77e0 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.289+0000 7f8c32000640 1 -- 172.21.15.59:0/2637893185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38004d00 con 0x55639c0a77e0 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.289+0000 7f8c32000640 1 -- 172.21.15.59:0/2637893185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380072d0 con 0x55639c0a77e0 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 -- 172.21.15.59:0/2637893185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 msgr2=0x55639c07c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8c38001ef0 tx=0x7f8c3800d520 comp rx=0 tx=0).stop 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 -- 172.21.15.59:0/2637893185 shutdown_connections 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a77e0 0x55639c07c5f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2637893185 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 -- 172.21.15.59:0/2637893185 >> 172.21.15.59:0/2637893185 conn(0x55639c079a00 msgr2=0x55639c0c22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 -- 172.21.15.59:0/2637893185 shutdown_connections 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.290+0000 7f8c4b364740 1 -- 172.21.15.59:0/2637893185 wait complete. 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.291+0000 7f8c4b364740 1 -- start start 2024-04-24T06:38:15.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.291+0000 7f8c4b364740 1 Processor -- start 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55639c07bbb0 con 0x55639c0c4e00 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55639bf9a690 con 0x55639c0a7f20 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55639c07afa0 con 0x55639c0c32f0 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c4b364740 10 client.0 _ll_register_callbacks cb 0x55639bfd6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c45200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c45200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35092/0 (socket says 172.21.15.59:35092) 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.292+0000 7f8c45200640 1 -- 172.21.15.59:0/2215059722 learned_addr learned my addr 172.21.15.59:0/2215059722 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c46600640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 -- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 msgr2=0x55639c0c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 -- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 msgr2=0x55639c0aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 -- 172.21.15.59:0/2215059722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55639bf99920 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c45200640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009630 tx=0x7f8c2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2bdacfcba60da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c037040 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.293+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f8c2c03ca70 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c38004d00 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c056050 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 10 client.8277 fetch_fsmap learned FSMap version 75 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55639bdbd7d0 con 0x55639c0c4e00 2024-04-24T06:38:15.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38004d00 con 0x55639c0c4e00 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f8c2c059020 con 0x55639c0c4e00 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 10 client.8277 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 10 client.8277 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 20 client.8277 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c096f70 0x55639c0973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 4 client.8277 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.294+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55639bfeec10 con 0x55639c096f70 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.295+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c096f70 0x55639c0973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.295+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c096f70 0x55639c0973b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcaff5491564527c server_cookie=d9d0dd9257c573c3 in_seq=0 out_seq=0 2024-04-24T06:38:15.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.295+0000 7f8c26a00640 10 client.8277 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c26a00640 1 -- 172.21.15.59:0/2215059722 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55639bfeec10 con 0x55639c096f70 2024-04-24T06:38:15.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c26a00640 10 client.8277 handle_command_reply: tid=0 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 client.8277 shutdown 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 msgr2=0x55639c0b8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009630 tx=0x7f8c2c004730 comp rx=0 tx=0).stop 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 shutdown_connections 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c096f70 0x55639c0973b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0b8f60 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2215059722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c0aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 >> 172.21.15.59:0/2215059722 conn(0x55639c0a77e0 msgr2=0x55639c0c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c26a00640 0 client.8277 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.296+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 shutdown_connections 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.297+0000 7f8c4b364740 1 -- 172.21.15.59:0/2215059722 wait complete. 2024-04-24T06:38:15.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.297+0000 7f8c4b364740 20 client.8277 trim_cache size 0 max 16384 2024-04-24T06:38:15.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.316+0000 7f8c4b364740 1 Processor -- start 2024-04-24T06:38:15.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.316+0000 7f8c4b364740 1 -- start start 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0c32f0 0x55639c0773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40200dd0 con 0x55639c0a7f20 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401fff20 con 0x55639c0c32f0 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8c400b1190 con 0x55639c0c4e00 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c45c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c45c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35104/0 (socket says 172.21.15.59:35104) 2024-04-24T06:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.317+0000 7f8c45c00640 1 -- 172.21.15.59:0/884419858 learned_addr learned my addr 172.21.15.59:0/884419858 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 -- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 msgr2=0x55639c077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 -- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0c32f0 msgr2=0x55639c0773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0c32f0 0x55639c0773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 -- 172.21.15.59:0/884419858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55639bcf33f0 con 0x55639c0a7f20 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c45c00640 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8c34055030 tx=0x7f8c3400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda076b6ad2569b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c31600640 1 -- 172.21.15.59:0/884419858 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34087070 con 0x55639c0a7f20 2024-04-24T06:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c31600640 1 -- 172.21.15.59:0/884419858 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c34056ed0 con 0x55639c0a7f20 2024-04-24T06:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.318+0000 7f8c31600640 1 -- 172.21.15.59:0/884419858 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34058020 con 0x55639c0a7f20 2024-04-24T06:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 -- 172.21.15.59:0/884419858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 msgr2=0x55639c0e6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8c34055030 tx=0x7f8c3400cf20 comp rx=0 tx=0).stop 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 -- 172.21.15.59:0/884419858 shutdown_connections 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c4e00 0x55639c077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0c32f0 0x55639c0773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 --2- 172.21.15.59:0/884419858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0a7f20 0x55639c0e6270 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 -- 172.21.15.59:0/884419858 >> 172.21.15.59:0/884419858 conn(0x55639c0a77e0 msgr2=0x55639c0aaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 -- 172.21.15.59:0/884419858 shutdown_connections 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.319+0000 7f8c4b364740 1 -- 172.21.15.59:0/884419858 wait complete. 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.320+0000 7f8c4b364740 1 -- start start 2024-04-24T06:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.320+0000 7f8c4b364740 1 Processor -- start 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c13b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400b17e0 con 0x55639c0c4e00 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55639c0aa420 con 0x55639c0a7f20 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55639c13c5a0 con 0x55639c0c32f0 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c4b364740 10 client.0 _ll_register_callbacks cb 0x7f8c2c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c46600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c46600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35110/0 (socket says 172.21.15.59:35110) 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.321+0000 7f8c46600640 1 -- 172.21.15.59:0/2060140412 learned_addr learned my addr 172.21.15.59:0/2060140412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c45200640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 -- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 msgr2=0x55639c0c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 -- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 msgr2=0x55639c13b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c13b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 -- 172.21.15.59:0/2060140412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55639bcb30f0 con 0x55639c0c4e00 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c46600640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000f80 tx=0x7f8c38001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e63ae7a4b1b8cec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.322+0000 7f8c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38025020 con 0x55639c0c4e00 2024-04-24T06:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f8c38004d60 con 0x55639c0c4e00 2024-04-24T06:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c34056ed0 con 0x55639c0c4e00 2024-04-24T06:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8c380070f0 con 0x55639c0c4e00 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 10 client.8283 fetch_fsmap learned FSMap version 75 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55639bfa6810 con 0x55639c0c4e00 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c34056ed0 con 0x55639c0c4e00 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f8c38007290 con 0x55639c0c4e00 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 10 client.8283 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 10 client.8283 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 20 client.8283 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c0d61a0 0x55639c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 4 client.8283 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.323+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55639bfe8a80 con 0x55639c0d61a0 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.324+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c0d61a0 0x55639c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.324+0000 7f8c45c00640 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c0d61a0 0x55639c0d0aa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a55515d257a0d9 server_cookie=d2cf5456881a0d3b in_seq=0 out_seq=0 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.324+0000 7f8c26a00640 10 client.8283 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c26a00640 1 -- 172.21.15.59:0/2060140412 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55639bfe8a80 con 0x55639c0d61a0 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c26a00640 10 client.8283 handle_command_reply: tid=0 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 client.8283 shutdown 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 msgr2=0x55639c0c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000f80 tx=0x7f8c38001250 comp rx=0 tx=0).stop 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 shutdown_connections 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x55639c0d61a0 0x55639c0d0aa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55639c0c4e00 0x55639c0c5c00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55639c0c32f0 0x55639c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 --2- 172.21.15.59:0/2060140412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55639c0a7f20 0x55639c13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c26a00640 0 client.8283 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 >> 172.21.15.59:0/2060140412 conn(0x55639c0a77e0 msgr2=0x55639c0b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 shutdown_connections 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 1 -- 172.21.15.59:0/2060140412 wait complete. 2024-04-24T06:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.325+0000 7f8c4b364740 20 client.8283 trim_cache size 0 max 16384 2024-04-24T06:38:15.329 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:38:15.329 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:38:15.329 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:38:15.329 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 msgr2=0x7f8c2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 0x7f8c2006edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c034000 tx=0x7f8c2c02e000 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 msgr2=0x7f8c4015e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c430 tx=0x7f8c3400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 shutdown_connections 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8c2006c8f0 0x7f8c2006edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c4015b220 0x7f8c4009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8c400d05d0 0x7f8c40145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 --2- 172.21.15.59:0/3642367538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c400af750 0x7f8c4015e3f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 >> 172.21.15.59:0/3642367538 conn(0x7f8c4006bdf0 msgr2=0x7f8c40158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 shutdown_connections 2024-04-24T06:38:15.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.326+0000 7f8c47000640 1 -- 172.21.15.59:0/3642367538 wait complete. 2024-04-24T06:38:15.345 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:38:15.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.546+0000 7f97b5c00640 1 Processor -- start 2024-04-24T06:38:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.546+0000 7f97b5c00640 1 -- start start 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97b5c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f97b015e7c0 0x7f97b00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97b5c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97b5c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97b00cf030 con 0x7f97b00b6530 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97b5c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f97b00b7000 con 0x7f97b015e7c0 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97b5c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97b00cf620 con 0x7f97b00d2630 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f97b015e7c0 0x7f97b00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f97b015e7c0 0x7f97b00d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35114/0 (socket says 172.21.15.59:35114) 2024-04-24T06:38:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 -- 172.21.15.59:0/4088913256 learned_addr learned my addr 172.21.15.59:0/4088913256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97af400640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 -- 172.21.15.59:0/4088913256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 msgr2=0x7f97b00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.548+0000 7f97afe00640 1 -- 172.21.15.59:0/4088913256 >> v1:172.21.15.66:6789/0 conn(0x7f97b00b6530 legacy=0x7f97b00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.549+0000 7f97afe00640 1 -- 172.21.15.59:0/4088913256 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b00b6a40 con 0x7f97b015e7c0 2024-04-24T06:38:15.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.549+0000 7f97af400640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.550+0000 7f97afe00640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00d20f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009870 tx=0x7f97a002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73610329fed4f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.550+0000 7f97aea00640 1 -- 172.21.15.59:0/4088913256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0037070 con 0x7f97b015e7c0 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.550+0000 7f97aea00640 1 -- 172.21.15.59:0/4088913256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a0032440 con 0x7f97b015e7c0 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.550+0000 7f97aea00640 1 -- 172.21.15.59:0/4088913256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0032810 con 0x7f97b015e7c0 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 -- 172.21.15.59:0/4088913256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 msgr2=0x7f97b00d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00d20f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009870 tx=0x7f97a002a200 comp rx=0 tx=0).stop 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 -- 172.21.15.59:0/4088913256 shutdown_connections 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 --2- 172.21.15.59:0/4088913256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00d20f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 -- 172.21.15.59:0/4088913256 >> 172.21.15.59:0/4088913256 conn(0x7f97b009d920 msgr2=0x7f97b008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 -- 172.21.15.59:0/4088913256 shutdown_connections 2024-04-24T06:38:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.551+0000 7f97b5c00640 1 -- 172.21.15.59:0/4088913256 wait complete. 2024-04-24T06:38:15.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.553+0000 7f97b5c00640 1 Processor -- start 2024-04-24T06:38:15.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.553+0000 7f97b5c00640 1 -- start start 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b00b6530 0x7f97b015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97b00d1a80 con 0x7f97b015e7c0 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f97b00cf2b0 con 0x7f97b00b6530 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f97b00d1c00 con 0x7f97b00d2630 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35128/0 (socket says 172.21.15.59:35128) 2024-04-24T06:38:15.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.554+0000 7f97b5200640 1 -- 172.21.15.59:0/3905877942 learned_addr learned my addr 172.21.15.59:0/3905877942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 -- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 msgr2=0x7f97b00b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 -- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b00b6530 msgr2=0x7f97b015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97afe00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b00b6530 0x7f97b015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b00b6530 0x7f97b015bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 -- 172.21.15.59:0/3905877942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b015a180 con 0x7f97b015e7c0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5200640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c700 tx=0x7f97a800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8016020 con 0x7f97b015e7c0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a800aa50 con 0x7f97b015e7c0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a800ce10 con 0x7f97b015e7c0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.555+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b015c4e0 con 0x7f97b015e7c0 2024-04-24T06:38:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.556+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b015b1e0 con 0x7f97b015e7c0 2024-04-24T06:38:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.557+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b004fd90 con 0x7f97b015e7c0 2024-04-24T06:38:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.558+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97a8019020 con 0x7f97b015e7c0 2024-04-24T06:38:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.559+0000 7f97a7e00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 0x7f979006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.559+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f97a808bfe0 con 0x7f97b015e7c0 2024-04-24T06:38:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.559+0000 7f97afe00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 0x7f979006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.560+0000 7f97afe00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 0x7f979006edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97a0034000 tx=0x7f97a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:15.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.564+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97a8020020 con 0x7f97b015e7c0 2024-04-24T06:38:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.806+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97b00d1eb0 con 0x7f97b015e7c0 2024-04-24T06:38:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.807+0000 7f97a7e00640 1 -- 172.21.15.59:0/3905877942 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7f97a805a2e0 con 0x7f97b015e7c0 2024-04-24T06:38:15.808 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:15.809 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:38:15.809 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:38:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 msgr2=0x7f979006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 0x7f979006edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97a0034000 tx=0x7f97a002e000 comp rx=0 tx=0).stop 2024-04-24T06:38:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 msgr2=0x7f97b00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c700 tx=0x7f97a800cbc0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 shutdown_connections 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f979006c8f0 0x7f979006edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97b015e7c0 0x7f97b00cb5f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f97b00d2630 0x7f97b00b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 --2- 172.21.15.59:0/3905877942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97b00b6530 0x7f97b015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 >> 172.21.15.59:0/3905877942 conn(0x7f97b009d920 msgr2=0x7f97b015c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 shutdown_connections 2024-04-24T06:38:15.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:15.810+0000 7f97b5c00640 1 -- 172.21.15.59:0/3905877942 wait complete. 2024-04-24T06:38:15.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-24T06:38:16.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.043+0000 7f811a400640 1 Processor -- start 2024-04-24T06:38:16.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.043+0000 7f811a400640 1 -- start start 2024-04-24T06:38:16.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.045+0000 7f811a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8114162a30 0x7f81140d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.045+0000 7f811a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 0x7f81140d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.045+0000 7f811a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f811406f340 con 0x7f811406f5f0 2024-04-24T06:38:16.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.045+0000 7f811a400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f81140b5500 con 0x7f8114162a30 2024-04-24T06:38:16.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.045+0000 7f811a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8114166660 con 0x7f811406fb40 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.046+0000 7f8119a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 0x7f81140d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.046+0000 7f8113e00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f811406f5f0 0x7f81140b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:39412/0 (socket says 172.21.15.59:39412) 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.046+0000 7f8113e00640 1 -- 172.21.15.59:0/3622205757 learned_addr learned my addr 172.21.15.59:0/3622205757 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.046+0000 7f8119000640 1 --2- 172.21.15.59:0/3622205757 >> v2:172.21.15.66:3300/0 conn(0x7f8114162a30 0x7f81140d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.047+0000 7f8119a00640 1 -- 172.21.15.59:0/3622205757 >> v2:172.21.15.66:3300/0 conn(0x7f8114162a30 msgr2=0x7f81140d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.047+0000 7f8119a00640 1 --2- 172.21.15.59:0/3622205757 >> v2:172.21.15.66:3300/0 conn(0x7f8114162a30 0x7f81140d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.047+0000 7f8119a00640 1 -- 172.21.15.59:0/3622205757 >> v1:172.21.15.66:6789/0 conn(0x7f811406f5f0 legacy=0x7f81140b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.047+0000 7f8113400640 1 -- 172.21.15.59:0/3622205757 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898334184 0 0) 0x7f811406f340 con 0x7f811406f5f0 2024-04-24T06:38:16.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.047+0000 7f8119a00640 1 -- 172.21.15.59:0/3622205757 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141668b0 con 0x7f811406fb40 2024-04-24T06:38:16.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.048+0000 7f8119a00640 1 --2- 172.21.15.59:0/3622205757 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 0x7f81140d0b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81141667e0 tx=0x7f810c02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245f396e3dea8c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.049+0000 7f8113400640 1 -- 172.21.15.59:0/3622205757 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c007740 con 0x7f811406fb40 2024-04-24T06:38:16.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.049+0000 7f8113400640 1 -- 172.21.15.59:0/3622205757 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c0078e0 con 0x7f811406fb40 2024-04-24T06:38:16.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.049+0000 7f8113400640 1 -- 172.21.15.59:0/3622205757 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c007cb0 con 0x7f811406fb40 2024-04-24T06:38:16.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.049+0000 7f811a400640 1 -- 172.21.15.59:0/3622205757 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 msgr2=0x7f81140d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.049+0000 7f811a400640 1 --2- 172.21.15.59:0/3622205757 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 0x7f81140d0b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81141667e0 tx=0x7f810c02a5b0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 -- 172.21.15.59:0/3622205757 shutdown_connections 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 --2- 172.21.15.59:0/3622205757 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f811406fb40 0x7f81140d0b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 --2- 172.21.15.59:0/3622205757 >> v2:172.21.15.66:3300/0 conn(0x7f8114162a30 0x7f81140d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 -- 172.21.15.59:0/3622205757 >> 172.21.15.59:0/3622205757 conn(0x7f811406bdf0 msgr2=0x7f811406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 -- 172.21.15.59:0/3622205757 shutdown_connections 2024-04-24T06:38:16.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.050+0000 7f811a400640 1 -- 172.21.15.59:0/3622205757 wait complete. 2024-04-24T06:38:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.051+0000 7f811a400640 1 Processor -- start 2024-04-24T06:38:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.052+0000 7f811a400640 1 -- start start 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.052+0000 7f811a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811406f5f0 0x7f8114156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.052+0000 7f811a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f811a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8114162a30 0x7f81141fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f811a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8114167150 con 0x7f811406fb40 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f811a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8114166e70 con 0x7f811406f5f0 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f811a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f81140ad170 con 0x7f8114162a30 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35138/0 (socket says 172.21.15.59:35138) 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 -- 172.21.15.59:0/2115280150 learned_addr learned my addr 172.21.15.59:0/2115280150 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8119a00640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8114162a30 0x7f81141fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8119000640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811406f5f0 0x7f8114156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 -- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8114162a30 msgr2=0x7f81141fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8114162a30 0x7f81141fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 -- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811406f5f0 msgr2=0x7f8114156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.053+0000 7f8113e00640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811406f5f0 0x7f8114156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f8113e00640 1 -- 172.21.15.59:0/2115280150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114155200 con 0x7f811406fb40 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f8113e00640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f810000c4a0 tx=0x7f810000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f8110c00640 1 -- 172.21.15.59:0/2115280150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100017070 con 0x7f811406fb40 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f8110c00640 1 -- 172.21.15.59:0/2115280150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100005ce0 con 0x7f811406fb40 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141554e0 con 0x7f811406fb40 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f8110c00640 1 -- 172.21.15.59:0/2115280150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000072b0 con 0x7f811406fb40 2024-04-24T06:38:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.054+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141fc4d0 con 0x7f811406fb40 2024-04-24T06:38:16.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.057+0000 7f8110c00640 1 -- 172.21.15.59:0/2115280150 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f810001a070 con 0x7f811406fb40 2024-04-24T06:38:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.058+0000 7f8110c00640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.058+0000 7f8110c00640 1 -- 172.21.15.59:0/2115280150 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f810008aff0 con 0x7f811406fb40 2024-04-24T06:38:16.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.058+0000 7f8119000640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.058+0000 7f811e7b4740 1 Processor -- start 2024-04-24T06:38:16.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.058+0000 7f811e7b4740 1 -- start start 2024-04-24T06:38:16.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f8119000640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 0x7f80f406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f811415aba0 tx=0x7f8104008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc9572220 0x555cc9572600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9547f00 0x555cc958e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cc94ac190 con 0x555cc958fac0 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555cc9452030 con 0x555cc9572220 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.059+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555cc94a8150 con 0x555cc9547f00 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35152/0 (socket says 172.21.15.59:35152) 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 -- 172.21.15.59:0/4028943369 learned_addr learned my addr 172.21.15.59:0/4028943369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8119000640 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc9572220 0x555cc9572600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 -- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9547f00 msgr2=0x555cc958e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9547f00 0x555cc958e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 -- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc9572220 msgr2=0x555cc9572600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc9572220 0x555cc9572600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 -- 172.21.15.59:0/4028943369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc943ab60 con 0x555cc958fac0 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.060+0000 7f8113e00640 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8100089380 tx=0x7f8100058c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9b0285e31a9fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f8109600640 1 -- 172.21.15.59:0/4028943369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100056ce0 con 0x555cc958fac0 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f8109600640 1 -- 172.21.15.59:0/4028943369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810005f050 con 0x555cc958fac0 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f8109600640 1 -- 172.21.15.59:0/4028943369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000552b0 con 0x555cc958fac0 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 -- 172.21.15.59:0/4028943369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 msgr2=0x555cc9590010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8100089380 tx=0x7f8100058c40 comp rx=0 tx=0).stop 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 -- 172.21.15.59:0/4028943369 shutdown_connections 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9547f00 0x555cc958e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc9590010 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 --2- 172.21.15.59:0/4028943369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc9572220 0x555cc9572600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 -- 172.21.15.59:0/4028943369 >> 172.21.15.59:0/4028943369 conn(0x555cc9544830 msgr2=0x555cc958d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 -- 172.21.15.59:0/4028943369 shutdown_connections 2024-04-24T06:38:16.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.061+0000 7f811e7b4740 1 -- 172.21.15.59:0/4028943369 wait complete. 2024-04-24T06:38:16.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.062+0000 7f811e7b4740 1 -- start start 2024-04-24T06:38:16.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.062+0000 7f811e7b4740 1 Processor -- start 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc955e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 0x555cc9574370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cc9548ca0 con 0x555cc95748b0 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555cc9465500 con 0x555cc958fac0 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555cc9542760 con 0x555cc9572220 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.063+0000 7f811e7b4740 10 client.0 _ll_register_callbacks cb 0x555cc94a2270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 -- 172.21.15.59:0/2420491315 learned_addr learned my addr 172.21.15.59:0/2420491315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119000640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc955e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8113e00640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 0x555cc9574370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 -- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 msgr2=0x555cc955e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc955e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 -- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 msgr2=0x555cc9574370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 0x555cc9574370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8119a00640 1 -- 172.21.15.59:0/2420491315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc928a7c0 con 0x555cc95748b0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.064+0000 7f8113e00640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 0x555cc9574370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f8119a00640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f810c02aba0 tx=0x7f810c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45de2be7e2b175ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f80fe000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c009450 con 0x555cc95748b0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f810c046480 con 0x555cc95748b0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810005f050 con 0x555cc95748b0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f810c009a30 con 0x555cc95748b0 2024-04-24T06:38:16.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f811e7b4740 10 client.8298 fetch_fsmap learned FSMap version 75 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x555cc92eaf50 con 0x555cc95748b0 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.065+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810005f050 con 0x555cc95748b0 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f810c051080 con 0x555cc95748b0 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 10 client.8298 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 10 client.8298 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 20 client.8298 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc9561070 0x555cc9561450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 4 client.8298 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc94b9bb0 con 0x555cc9561070 2024-04-24T06:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.066+0000 7f8119000640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc9561070 0x555cc9561450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.067+0000 7f8119000640 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc9561070 0x555cc9561450 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71648a0739b33794 server_cookie=3b1e7bea3039913a in_seq=0 out_seq=0 2024-04-24T06:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.067+0000 7f80fe000640 10 client.8298 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.067+0000 7f80fe000640 1 -- 172.21.15.59:0/2420491315 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cc94b9bb0 con 0x555cc9561070 2024-04-24T06:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.067+0000 7f80fe000640 10 client.8298 handle_command_reply: tid=0 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.067+0000 7f811e7b4740 1 client.8298 shutdown 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 msgr2=0x555cc9586840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f810c02aba0 tx=0x7f810c004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 shutdown_connections 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc9561070 0x555cc9561450 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc95748b0 0x555cc9586840 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc958fac0 0x555cc9574370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 --2- 172.21.15.59:0/2420491315 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc955e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f80fe000640 0 client.8298 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 >> 172.21.15.59:0/2420491315 conn(0x555cc9547f00 msgr2=0x555cc95905b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 shutdown_connections 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 1 -- 172.21.15.59:0/2420491315 wait complete. 2024-04-24T06:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.068+0000 7f811e7b4740 20 client.8298 trim_cache size 0 max 16384 2024-04-24T06:38:16.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.083+0000 7f811e7b4740 1 Processor -- start 2024-04-24T06:38:16.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.083+0000 7f811e7b4740 1 -- start start 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9575890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc958fac0 0x555cc95482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81142041f0 con 0x555cc9572220 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81141f8ff0 con 0x555cc95748b0 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8114203e90 con 0x555cc958fac0 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f8119000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f8119000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35176/0 (socket says 172.21.15.59:35176) 2024-04-24T06:38:16.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.084+0000 7f8119000640 1 -- 172.21.15.59:0/922653505 learned_addr learned my addr 172.21.15.59:0/922653505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 -- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc958fac0 msgr2=0x555cc95482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc958fac0 0x555cc95482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 -- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 msgr2=0x555cc9575890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9575890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 -- 172.21.15.59:0/922653505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc9586bb0 con 0x555cc9572220 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8119000640 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8104009370 tx=0x7f8104014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9324ff207741c35f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8108c00640 1 -- 172.21.15.59:0/922653505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104004020 con 0x555cc9572220 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8108c00640 1 -- 172.21.15.59:0/922653505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810401a040 con 0x555cc9572220 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.085+0000 7f8108c00640 1 -- 172.21.15.59:0/922653505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040041c0 con 0x555cc9572220 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 -- 172.21.15.59:0/922653505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 msgr2=0x555cc958d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8104009370 tx=0x7f8104014000 comp rx=0 tx=0).stop 2024-04-24T06:38:16.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 -- 172.21.15.59:0/922653505 shutdown_connections 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc958fac0 0x555cc95482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9575890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 --2- 172.21.15.59:0/922653505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc9572220 0x555cc958d1c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 -- 172.21.15.59:0/922653505 >> 172.21.15.59:0/922653505 conn(0x555cc9547f00 msgr2=0x555cc956e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 -- 172.21.15.59:0/922653505 shutdown_connections 2024-04-24T06:38:16.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.086+0000 7f811e7b4740 1 -- 172.21.15.59:0/922653505 wait complete. 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.087+0000 7f811e7b4740 1 -- start start 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.087+0000 7f811e7b4740 1 Processor -- start 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc95482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9548820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555cc9584210 con 0x555cc958fac0 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555cc955f2b0 con 0x555cc95748b0 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555cc9560330 con 0x555cc9572220 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f811e7b4740 10 client.0 _ll_register_callbacks cb 0x7f8104009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35178/0 (socket says 172.21.15.59:35178) 2024-04-24T06:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 -- 172.21.15.59:0/3025141009 learned_addr learned my addr 172.21.15.59:0/3025141009 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 -- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 msgr2=0x555cc95482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc95482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.088+0000 7f8119a00640 1 -- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 msgr2=0x555cc9548820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f8113e00640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9548820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f8119a00640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9548820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f8119a00640 1 -- 172.21.15.59:0/3025141009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc917e0b0 con 0x555cc958fac0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f8119a00640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f810c008000 tx=0x7f810c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bc7cd77061160a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f80fe000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c003b90 con 0x555cc958fac0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f810c003d30 con 0x555cc958fac0 2024-04-24T06:38:16.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810401a040 con 0x555cc958fac0 2024-04-24T06:38:16.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.089+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f810c046690 con 0x555cc958fac0 2024-04-24T06:38:16.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 10 client.8304 fetch_fsmap learned FSMap version 75 2024-04-24T06:38:16.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x555cc958da60 con 0x555cc958fac0 2024-04-24T06:38:16.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810401a040 con 0x555cc958fac0 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7f810c046830 con 0x555cc958fac0 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 10 client.8304 fetch_fsmap finished waiting for FSMap version 75 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 10 client.8304 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 20 client.8304 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc959fd00 0x555cc958ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 4 client.8304 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 --> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x555cc94b3a20 con 0x555cc959fd00 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.090+0000 7f8119000640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc959fd00 0x555cc958ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f8119000640 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc959fd00 0x555cc958ff70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b35a76c12375c6 server_cookie=ea1d76f3c066bf2a in_seq=0 out_seq=0 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f80fe000640 10 client.8304 ms_handle_connect on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f80fe000640 1 -- 172.21.15.59:0/3025141009 <== mds.0 v2:172.21.15.59:6834/3165421940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cc94b3a20 con 0x555cc959fd00 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f80fe000640 10 client.8304 handle_command_reply: tid=0 2024-04-24T06:38:16.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 client.8304 shutdown 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 msgr2=0x555cc959f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f810c008000 tx=0x7f810c009750 comp rx=0 tx=0).stop 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 shutdown_connections 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:6834/3165421940,v1:172.21.15.59:6835/3165421940] conn(0x555cc959fd00 0x555cc958ff70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555cc958fac0 0x555cc959f5d0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555cc95748b0 0x555cc9548820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 --2- 172.21.15.59:0/3025141009 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555cc9572220 0x555cc95482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f80fe000640 0 client.8304 ms_handle_reset on v2:172.21.15.59:6834/3165421940 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.091+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 >> 172.21.15.59:0/3025141009 conn(0x555cc9547f00 msgr2=0x555cc9587c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 shutdown_connections 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811e7b4740 1 -- 172.21.15.59:0/3025141009 wait complete. 2024-04-24T06:38:16.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811e7b4740 20 client.8304 trim_cache size 0 max 16384 2024-04-24T06:38:16.096 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:16.096 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-24T06:38:16.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 msgr2=0x7f80f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 0x7f80f406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f811415aba0 tx=0x7f8104008040 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 msgr2=0x7f8114154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f810000c4a0 tx=0x7f810000c960 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 shutdown_connections 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8114162a30 0x7f81141fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406fb40 0x7f8114154b00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.092+0000 7f811a400640 1 --2- 172.21.15.59:0/2115280150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811406f5f0 0x7f8114156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.093+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 >> 172.21.15.59:0/2115280150 conn(0x7f811406bdf0 msgr2=0x7f81140b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.093+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 shutdown_connections 2024-04-24T06:38:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.093+0000 7f811a400640 1 -- 172.21.15.59:0/2115280150 wait complete. 2024-04-24T06:38:16.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T06:38:16.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.319+0000 7fe43e000640 1 Processor -- start 2024-04-24T06:38:16.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.319+0000 7fe43e000640 1 -- start start 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4380cc7e0 0x7fe438167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43e000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe438160210 con 0x7fe43815c3d0 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43e000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4381604f0 con 0x7fe4380cd760 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43e000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe4381607d0 con 0x7fe4380cc7e0 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4380cc7e0 0x7fe438167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4380cc7e0 0x7fe438167400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35194/0 (socket says 172.21.15.59:35194) 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.320+0000 7fe43cc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe43815c3d0 0x7fe4380cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52776/0 (socket says 172.21.15.59:52776) 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.321+0000 7fe43d600640 1 -- 172.21.15.59:0/1223510000 learned_addr learned my addr 172.21.15.59:0/1223510000 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.321+0000 7fe43d600640 1 -- 172.21.15.59:0/1223510000 >> v1:172.21.15.66:6789/0 conn(0x7fe4380cd760 legacy=0x7fe4380b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.321+0000 7fe43d600640 1 -- 172.21.15.59:0/1223510000 >> v1:172.21.15.59:6789/0 conn(0x7fe43815c3d0 legacy=0x7fe4380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.321+0000 7fe43d600640 1 -- 172.21.15.59:0/1223510000 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe438160ab0 con 0x7fe4380cc7e0 2024-04-24T06:38:16.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.322+0000 7fe43d600640 1 --2- 172.21.15.59:0/1223510000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4380cc7e0 0x7fe438167400 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322dfb7d08fc5fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.322+0000 7fe437400640 1 -- 172.21.15.59:0/1223510000 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430036070 con 0x7fe4380cc7e0 2024-04-24T06:38:16.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.322+0000 7fe437400640 1 -- 172.21.15.59:0/1223510000 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe430031600 con 0x7fe4380cc7e0 2024-04-24T06:38:16.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.322+0000 7fe437400640 1 -- 172.21.15.59:0/1223510000 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300319f0 con 0x7fe4380cc7e0 2024-04-24T06:38:16.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 -- 172.21.15.59:0/1223510000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4380cc7e0 msgr2=0x7fe438167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 --2- 172.21.15.59:0/1223510000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4380cc7e0 0x7fe438167400 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe430009e50 tx=0x7fe43002a7c0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 -- 172.21.15.59:0/1223510000 shutdown_connections 2024-04-24T06:38:16.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 --2- 172.21.15.59:0/1223510000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4380cc7e0 0x7fe438167400 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 -- 172.21.15.59:0/1223510000 >> 172.21.15.59:0/1223510000 conn(0x7fe4380b21a0 msgr2=0x7fe4380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 -- 172.21.15.59:0/1223510000 shutdown_connections 2024-04-24T06:38:16.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.323+0000 7fe43e000640 1 -- 172.21.15.59:0/1223510000 wait complete. 2024-04-24T06:38:16.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.325+0000 7fe43e000640 1 Processor -- start 2024-04-24T06:38:16.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.325+0000 7fe43e000640 1 -- start start 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 0x7fe4381f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 0x7fe4381f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe43815c930 con 0x7fe43815c3d0 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe43815c7b0 con 0x7fe4380cd760 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43e000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe4380b4860 con 0x7fe4380cc7e0 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.326+0000 7fe43d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35198/0 (socket says 172.21.15.59:35198) 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 -- 172.21.15.59:0/1405860058 learned_addr learned my addr 172.21.15.59:0/1405860058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43cc00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 0x7fe4381f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe437e00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 0x7fe4381f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 -- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 msgr2=0x7fe4381f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 0x7fe4381f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 -- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 msgr2=0x7fe4381f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 0x7fe4381f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 -- 172.21.15.59:0/1405860058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380cdf00 con 0x7fe43815c3d0 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43cc00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 0x7fe4381f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe437e00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 0x7fe4381f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:16.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.327+0000 7fe43d600640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe43002a800 tx=0x7fe430004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.328+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430036050 con 0x7fe43815c3d0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.328+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe430042070 con 0x7fe43815c3d0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.328+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430009510 con 0x7fe43815c3d0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.328+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4380ce1e0 con 0x7fe43815c3d0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.328+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4381f7d50 con 0x7fe43815c3d0 2024-04-24T06:38:16.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.329+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43804fd90 con 0x7fe43815c3d0 2024-04-24T06:38:16.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.330+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe430004470 con 0x7fe43815c3d0 2024-04-24T06:38:16.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.331+0000 7fe434c00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:16.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.331+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fe4300acb50 con 0x7fe43815c3d0 2024-04-24T06:38:16.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.332+0000 7fe43cc00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:16.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.332+0000 7fe43cc00640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe43809e1f0 tx=0x7fe42c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.336+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe43007aeb0 con 0x7fe43815c3d0 2024-04-24T06:38:16.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.583+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe4380b0b60 con 0x7fe43815c3d0 2024-04-24T06:38:16.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.584+0000 7fe434c00640 1 -- 172.21.15.59:0/1405860058 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe43007a7b0 con 0x7fe43815c3d0 2024-04-24T06:38:16.585 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:16.585 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T06:38:16.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 msgr2=0x7fe41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe43809e1f0 tx=0x7fe42c009040 comp rx=0 tx=0).stop 2024-04-24T06:38:16.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 msgr2=0x7fe4380c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:16.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe43002a800 tx=0x7fe430004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 shutdown_connections 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe43815c3d0 0x7fe4380c7280 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4380cd760 0x7fe4381f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 --2- 172.21.15.59:0/1405860058 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe4380cc7e0 0x7fe4381f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 >> 172.21.15.59:0/1405860058 conn(0x7fe4380b21a0 msgr2=0x7fe4380c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:16.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 shutdown_connections 2024-04-24T06:38:16.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:16.587+0000 7fe43e000640 1 -- 172.21.15.59:0/1405860058 wait complete. 2024-04-24T06:38:16.608 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-24T06:38:16.608 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:38:16.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:38:16.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:38:16.836 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:38:16.836 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:38:16.837 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:16.837 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:16.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.111+0000 7fae4aa00640 1 Processor -- start 2024-04-24T06:38:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.111+0000 7fae4aa00640 1 -- start start 2024-04-24T06:38:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.112+0000 7fae4aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae440cd760 0x7fae440b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.112+0000 7fae4aa00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae440b4690 0x7fae440cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.112+0000 7fae4aa00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae44160210 con 0x7fae4415c3d0 2024-04-24T06:38:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.112+0000 7fae4aa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fae441604f0 con 0x7fae440cd760 2024-04-24T06:38:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.112+0000 7fae4aa00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fae441607d0 con 0x7fae440b4690 2024-04-24T06:38:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae440cd760 0x7fae440b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae440cd760 0x7fae440b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35214/0 (socket says 172.21.15.59:35214) 2024-04-24T06:38:17.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 -- 172.21.15.59:0/2744771545 learned_addr learned my addr 172.21.15.59:0/2744771545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 -- 172.21.15.59:0/2744771545 >> v2:172.21.15.66:3300/0 conn(0x7fae440b4690 msgr2=0x7fae440cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae4a000640 1 --2- 172.21.15.59:0/2744771545 >> v2:172.21.15.66:3300/0 conn(0x7fae440b4690 0x7fae440cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 --2- 172.21.15.59:0/2744771545 >> v2:172.21.15.66:3300/0 conn(0x7fae440b4690 0x7fae440cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 -- 172.21.15.59:0/2744771545 >> v1:172.21.15.66:6789/0 conn(0x7fae4415c3d0 legacy=0x7fae440cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.113+0000 7fae48c00640 1 -- 172.21.15.59:0/2744771545 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae44160ab0 con 0x7fae440cd760 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.114+0000 7fae48c00640 1 --2- 172.21.15.59:0/2744771545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440cd760 0x7fae440b4150 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fae440b2de0 tx=0x7fae3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa40be3c5849d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae43e00640 1 -- 172.21.15.59:0/2744771545 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae440cd760 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae43e00640 1 -- 172.21.15.59:0/2744771545 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30032440 con 0x7fae440cd760 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae43e00640 1 -- 172.21.15.59:0/2744771545 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30032810 con 0x7fae440cd760 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 -- 172.21.15.59:0/2744771545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440cd760 msgr2=0x7fae440b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2744771545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440cd760 0x7fae440b4150 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fae440b2de0 tx=0x7fae3002a230 comp rx=0 tx=0).stop 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 -- 172.21.15.59:0/2744771545 shutdown_connections 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2744771545 >> v2:172.21.15.66:3300/0 conn(0x7fae440b4690 0x7fae440cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2744771545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440cd760 0x7fae440b4150 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 -- 172.21.15.59:0/2744771545 >> 172.21.15.59:0/2744771545 conn(0x7fae440b21a0 msgr2=0x7fae440b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.115+0000 7fae4aa00640 1 -- 172.21.15.59:0/2744771545 shutdown_connections 2024-04-24T06:38:17.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.116+0000 7fae4aa00640 1 -- 172.21.15.59:0/2744771545 wait complete. 2024-04-24T06:38:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.117+0000 7fae4aa00640 1 Processor -- start 2024-04-24T06:38:17.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.117+0000 7fae4aa00640 1 -- start start 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cd760 0x7fae44158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae4415c3d0 0x7fae44158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae4415c7b0 con 0x7fae440b4690 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae440b4aa0 con 0x7fae440cd760 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.118+0000 7fae4aa00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fae4415c930 con 0x7fae4415c3d0 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35228/0 (socket says 172.21.15.59:35228) 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 -- 172.21.15.59:0/2839503574 learned_addr learned my addr 172.21.15.59:0/2839503574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae48c00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cd760 0x7fae44158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 -- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae4415c3d0 msgr2=0x7fae44158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae4415c3d0 0x7fae44158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 -- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cd760 msgr2=0x7fae44158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cd760 0x7fae44158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 -- 172.21.15.59:0/2839503574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae440c0bc0 con 0x7fae440b4690 2024-04-24T06:38:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.119+0000 7fae49600640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b3f0 tx=0x7fae3800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.120+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38015020 con 0x7fae440b4690 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.120+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae38011830 con 0x7fae440b4690 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.120+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38011c20 con 0x7fae440b4690 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.120+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae440ae590 con 0x7fae440b4690 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.120+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae440aeb50 con 0x7fae440b4690 2024-04-24T06:38:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.121+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4404fd90 con 0x7fae440b4690 2024-04-24T06:38:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.122+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae38018020 con 0x7fae440b4690 2024-04-24T06:38:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.123+0000 7fae41600640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.123+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fae3808b830 con 0x7fae440b4690 2024-04-24T06:38:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.123+0000 7fae48c00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.124+0000 7fae48c00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 0x7fae2406edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fae30034000 tx=0x7fae3002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.128+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae3801f030 con 0x7fae440b4690 2024-04-24T06:38:17.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.336+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae4406e400 con 0x7fae440b4690 2024-04-24T06:38:17.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.338+0000 7fae41600640 1 -- 172.21.15.59:0/2839503574 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16883 (secure 0 0 0) 0x7fae380172c0 con 0x7fae440b4690 2024-04-24T06:38:17.339 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:17.339 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:37:35.076193+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.59:6834/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:17.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 msgr2=0x7fae2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 0x7fae2406edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fae30034000 tx=0x7fae3002e000 comp rx=0 tx=0).stop 2024-04-24T06:38:17.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 msgr2=0x7fae44157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b3f0 tx=0x7fae3800b8b0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 shutdown_connections 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae4415c3d0 0x7fae44158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae440cd760 0x7fae44158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 --2- 172.21.15.59:0/2839503574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae440b4690 0x7fae44157810 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.341+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 >> 172.21.15.59:0/2839503574 conn(0x7fae440b21a0 msgr2=0x7fae440cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.342+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 shutdown_connections 2024-04-24T06:38:17.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.342+0000 7fae4aa00640 1 -- 172.21.15.59:0/2839503574 wait complete. 2024-04-24T06:38:17.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:17.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.583+0000 7f05e3e00640 1 Processor -- start 2024-04-24T06:38:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.583+0000 7f05e3e00640 1 -- start start 2024-04-24T06:38:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.584+0000 7f05e3e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f05e40d0a90 0x7f05e40ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.584+0000 7f05e3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e415ef20 con 0x7f05e4155a80 2024-04-24T06:38:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.584+0000 7f05e3e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e40d0f00 con 0x7f05e40d0600 2024-04-24T06:38:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.584+0000 7f05e3e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f05e415f1c0 con 0x7f05e40d0a90 2024-04-24T06:38:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.585+0000 7f05e2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f05e4155a80 0x7f05e406f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52788/0 (socket says 172.21.15.59:52788) 2024-04-24T06:38:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.585+0000 7f05e2a00640 1 -- 172.21.15.59:0/2845753972 learned_addr learned my addr 172.21.15.59:0/2845753972 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.585+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471037965 0 0) 0x7f05e415ef20 con 0x7f05e4155a80 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.585+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05c8005730 con 0x7f05e4155a80 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.585+0000 7f05e3400640 1 --2- 172.21.15.59:0/2845753972 >> v2:172.21.15.66:3300/0 conn(0x7f05e40d0a90 0x7f05e40ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3068840439 0 0) 0x7f05c8005730 con 0x7f05e4155a80 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 >> v2:172.21.15.66:3300/0 conn(0x7f05e40d0a90 msgr2=0x7f05e40ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 --2- 172.21.15.59:0/2845753972 >> v2:172.21.15.66:3300/0 conn(0x7f05e40d0a90 0x7f05e40ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 >> v1:172.21.15.66:6789/0 conn(0x7f05e40d0600 legacy=0x7f05e40ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e415f4a0 con 0x7f05e4155a80 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f05d4001670 con 0x7f05e4155a80 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 >> v1:172.21.15.59:6789/0 conn(0x7f05e4155a80 legacy=0x7f05e406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05c8005bc0 0x7f05c8005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c8006a10 0x7f05c8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 0x7f05c800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415f1c0 con 0x7f05c80073d0 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05c800e300 con 0x7f05c8006a10 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05c800e500 con 0x7f05c8005bc0 2024-04-24T06:38:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e3400640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 0x7f05c800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.586+0000 7f05e2a00640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05c8005bc0 0x7f05c8005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e3400640 1 -- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05c8005bc0 msgr2=0x7f05c8005fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e2000640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c8006a10 0x7f05c8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e3400640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05c8005bc0 0x7f05c8005fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e3400640 1 -- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c8006a10 msgr2=0x7f05c8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e3400640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c8006a10 0x7f05c8006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e3400640 1 -- 172.21.15.59:0/2845753972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c8005730 con 0x7f05c80073d0 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.587+0000 7f05e2000640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c8006a10 0x7f05c8006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.588+0000 7f05e3400640 1 -- 172.21.15.59:0/2845753972 reap_dead start 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.588+0000 7f05e3400640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 0x7f05c800dbd0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007330 tx=0x7f05c8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9532366bd25fa58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.588+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc009be0 con 0x7f05c80073d0 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.588+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc003b90 con 0x7f05c80073d0 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.588+0000 7f05e1600640 1 -- 172.21.15.59:0/2845753972 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004020 con 0x7f05c80073d0 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 -- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 msgr2=0x7f05c800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 0x7f05c800dbd0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007330 tx=0x7f05c8007a40 comp rx=0 tx=0).stop 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 -- 172.21.15.59:0/2845753972 shutdown_connections 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 --2- 172.21.15.59:0/2845753972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05c80073d0 0x7f05c800dbd0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 -- 172.21.15.59:0/2845753972 >> 172.21.15.59:0/2845753972 conn(0x7f05e406bdf0 msgr2=0x7f05e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 -- 172.21.15.59:0/2845753972 shutdown_connections 2024-04-24T06:38:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.589+0000 7f05e3e00640 1 -- 172.21.15.59:0/2845753972 wait complete. 2024-04-24T06:38:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.591+0000 7f05e3e00640 1 Processor -- start 2024-04-24T06:38:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.591+0000 7f05e3e00640 1 -- start start 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c80073d0 0x7f05e40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05e40b1a60 0x7f05e40b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415fd40 con 0x7f05e40b15e0 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415fa60 con 0x7f05c80073d0 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e3e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f05e406ef80 con 0x7f05e40b1a60 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35242/0 (socket says 172.21.15.59:35242) 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e2000640 1 -- 172.21.15.59:0/336527772 learned_addr learned my addr 172.21.15.59:0/336527772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e2000640 1 -- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05e40b1a60 msgr2=0x7f05e40b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.592+0000 7f05e2000640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05e40b1a60 0x7f05e40b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e2000640 1 -- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c80073d0 msgr2=0x7f05e40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e2000640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c80073d0 0x7f05e40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e2000640 1 -- 172.21.15.59:0/336527772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40b0170 con 0x7f05e40b15e0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e2000640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f05d000d590 tx=0x7f05d000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0016070 con 0x7f05e40b15e0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d0007240 con 0x7f05e40b15e0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0007630 con 0x7f05e40b15e0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40b0450 con 0x7f05e40b15e0 2024-04-24T06:38:17.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.593+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e40b0a10 con 0x7f05e40b15e0 2024-04-24T06:38:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.595+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e404fd90 con 0x7f05e40b15e0 2024-04-24T06:38:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.596+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05d00067d0 con 0x7f05e40b15e0 2024-04-24T06:38:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.597+0000 7f05db400640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 0x7f05c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.597+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f05d008bab0 con 0x7f05e40b15e0 2024-04-24T06:38:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.597+0000 7f05e2a00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 0x7f05c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.598+0000 7f05e2a00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 0x7f05c406ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f05d4000ff0 tx=0x7f05d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.602+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05d0059d70 con 0x7f05e40b15e0 2024-04-24T06:38:17.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.805+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05e406ff90 con 0x7f05e40b15e0 2024-04-24T06:38:17.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.806+0000 7f05db400640 1 -- 172.21.15.59:0/336527772 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f05d0059670 con 0x7f05e40b15e0 2024-04-24T06:38:17.807 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.808+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 msgr2=0x7f05c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.808+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 0x7f05c406ede0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f05d4000ff0 tx=0x7f05d4009040 comp rx=0 tx=0).stop 2024-04-24T06:38:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.808+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 msgr2=0x7f05e40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:17.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.808+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f05d000d590 tx=0x7f05d000da50 comp rx=0 tx=0).stop 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 shutdown_connections 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f05c406c920 0x7f05c406ede0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05e40b1a60 0x7f05e40b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05e40b15e0 0x7f05e40b3fd0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 --2- 172.21.15.59:0/336527772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c80073d0 0x7f05e40ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 >> 172.21.15.59:0/336527772 conn(0x7f05e406bdf0 msgr2=0x7f05e40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 shutdown_connections 2024-04-24T06:38:17.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:17.809+0000 7f05e3e00640 1 -- 172.21.15.59:0/336527772 wait complete. 2024-04-24T06:38:17.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:38:18.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.046+0000 7fa1ca600640 1 Processor -- start 2024-04-24T06:38:18.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.046+0000 7fa1ca600640 1 -- start start 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.047+0000 7fa1ca600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1c41567e0 0x7fa1c4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.047+0000 7fa1ca600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c415cab0 con 0x7fa1c415d810 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.047+0000 7fa1ca600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c4160e40 con 0x7fa1c40d0d10 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.047+0000 7fa1ca600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c415cd30 con 0x7fa1c41567e0 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.048+0000 7fa1c3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1c415d810 0x7fa1c40d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52790/0 (socket says 172.21.15.59:52790) 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.048+0000 7fa1c3e00640 1 -- 172.21.15.59:0/2950443678 learned_addr learned my addr 172.21.15.59:0/2950443678 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.048+0000 7fa1c9200640 1 --2- 172.21.15.59:0/2950443678 >> v2:172.21.15.59:3300/0 conn(0x7fa1c41567e0 0x7fa1c4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.048+0000 7fa1c3400640 1 -- 172.21.15.59:0/2950443678 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562006258 0 0) 0x7fa1b0001110 con 0x7fa1c415d810 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.048+0000 7fa1c3400640 1 -- 172.21.15.59:0/2950443678 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1ac003520 con 0x7fa1c415d810 2024-04-24T06:38:18.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.049+0000 7fa1c9200640 1 -- 172.21.15.59:0/2950443678 >> v1:172.21.15.66:6789/0 conn(0x7fa1c40d0d10 legacy=0x7fa1c4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.049+0000 7fa1c9200640 1 -- 172.21.15.59:0/2950443678 >> v1:172.21.15.59:6789/0 conn(0x7fa1c415d810 legacy=0x7fa1c40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.049+0000 7fa1c9200640 1 -- 172.21.15.59:0/2950443678 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c415d010 con 0x7fa1c41567e0 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.050+0000 7fa1c9200640 1 --2- 172.21.15.59:0/2950443678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c41567e0 0x7fa1c4156bc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80097a0 tx=0x7fa1b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916824ad52e1c359 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.050+0000 7fa1c3400640 1 -- 172.21.15.59:0/2950443678 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8037070 con 0x7fa1c41567e0 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.050+0000 7fa1c3400640 1 -- 172.21.15.59:0/2950443678 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b8032440 con 0x7fa1c41567e0 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.050+0000 7fa1c3400640 1 -- 172.21.15.59:0/2950443678 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80327f0 con 0x7fa1c41567e0 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 -- 172.21.15.59:0/2950443678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c41567e0 msgr2=0x7fa1c4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2950443678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c41567e0 0x7fa1c4156bc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80097a0 tx=0x7fa1b802a230 comp rx=0 tx=0).stop 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 -- 172.21.15.59:0/2950443678 shutdown_connections 2024-04-24T06:38:18.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2950443678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c41567e0 0x7fa1c4156bc0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 -- 172.21.15.59:0/2950443678 >> 172.21.15.59:0/2950443678 conn(0x7fa1c406bdf0 msgr2=0x7fa1c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:18.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 -- 172.21.15.59:0/2950443678 shutdown_connections 2024-04-24T06:38:18.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.051+0000 7fa1ca600640 1 -- 172.21.15.59:0/2950443678 wait complete. 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.053+0000 7fa1ca600640 1 Processor -- start 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.053+0000 7fa1ca600640 1 -- start start 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c41567e0 0x7fa1c4155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c415d810 0x7fa1c409d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41609a0 con 0x7fa1c40d0d10 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4160820 con 0x7fa1c41567e0 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1ca600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c4160b20 con 0x7fa1c415d810 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1c9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1c9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35264/0 (socket says 172.21.15.59:35264) 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.054+0000 7fa1c9200640 1 -- 172.21.15.59:0/2958020527 learned_addr learned my addr 172.21.15.59:0/2958020527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 -- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c415d810 msgr2=0x7fa1c409d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c415d810 0x7fa1c409d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 -- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c41567e0 msgr2=0x7fa1c4155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c41567e0 0x7fa1c4155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 -- 172.21.15.59:0/2958020527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c40c5470 con 0x7fa1c40d0d10 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c9200640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8035040 tx=0x7fa1b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.055+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8037050 con 0x7fa1c40d0d10 2024-04-24T06:38:18.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.056+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b8043070 con 0x7fa1c40d0d10 2024-04-24T06:38:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.056+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c40c5750 con 0x7fa1c40d0d10 2024-04-24T06:38:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.056+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8032830 con 0x7fa1c40d0d10 2024-04-24T06:38:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.056+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41f4b20 con 0x7fa1c40d0d10 2024-04-24T06:38:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.057+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c404fd90 con 0x7fa1c40d0d10 2024-04-24T06:38:18.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.058+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1b80329d0 con 0x7fa1c40d0d10 2024-04-24T06:38:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.059+0000 7fa1c0c00640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.059+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fa1b80ac780 con 0x7fa1c40d0d10 2024-04-24T06:38:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.060+0000 7fa1c3e00640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.061+0000 7fa1c3e00640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 0x7fa1a406edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0007fd0 tx=0x7fa1b0001150 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.064+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1b807abf0 con 0x7fa1c40d0d10 2024-04-24T06:38:18.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.268+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1c40b5150 con 0x7fa1c40d0d10 2024-04-24T06:38:18.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.269+0000 7fa1c0c00640 1 -- 172.21.15.59:0/2958020527 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fa1b807a4f0 con 0x7fa1c40d0d10 2024-04-24T06:38:18.271 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 msgr2=0x7fa1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 0x7fa1a406edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0007fd0 tx=0x7fa1b0001150 comp rx=0 tx=0).stop 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 msgr2=0x7fa1c4159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8035040 tx=0x7fa1b80046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 shutdown_connections 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1a406c8f0 0x7fa1a406edb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c415d810 0x7fa1c409d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c41567e0 0x7fa1c4155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 --2- 172.21.15.59:0/2958020527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c40d0d10 0x7fa1c4159b90 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 >> 172.21.15.59:0/2958020527 conn(0x7fa1c406bdf0 msgr2=0x7fa1c40733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:18.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 shutdown_connections 2024-04-24T06:38:18.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.273+0000 7fa1ca600640 1 -- 172.21.15.59:0/2958020527 wait complete. 2024-04-24T06:38:18.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:38:18.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.510+0000 7f31f1600640 1 Processor -- start 2024-04-24T06:38:18.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.511+0000 7f31f1600640 1 -- start start 2024-04-24T06:38:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.512+0000 7f31f1600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f31ec0d0d10 0x7f31ec156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.512+0000 7f31f1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec15d810 0x7f31ec160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.512+0000 7f31f1600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31ec15cab0 con 0x7f31ec1567e0 2024-04-24T06:38:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.512+0000 7f31f1600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f31ec160e40 con 0x7f31ec0d0d10 2024-04-24T06:38:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.512+0000 7f31f1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec15cd30 con 0x7f31ec15d810 2024-04-24T06:38:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f31ec0d0d10 0x7f31ec156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f31ec0d0d10 0x7f31ec156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35266/0 (socket says 172.21.15.59:35266) 2024-04-24T06:38:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 -- 172.21.15.59:0/326258952 learned_addr learned my addr 172.21.15.59:0/326258952 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31f0c00640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec15d810 0x7f31ec160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 -- 172.21.15.59:0/326258952 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec15d810 msgr2=0x7f31ec160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec15d810 0x7f31ec160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 -- 172.21.15.59:0/326258952 >> v1:172.21.15.66:6789/0 conn(0x7f31ec1567e0 legacy=0x7f31ec0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.513+0000 7f31e3e00640 1 -- 172.21.15.59:0/326258952 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec15d010 con 0x7f31ec0d0d10 2024-04-24T06:38:18.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.514+0000 7f31e3e00640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec0d0d10 0x7f31ec156bc0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009870 tx=0x7f31d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63df938c475b223 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.515+0000 7f31eb400640 1 -- 172.21.15.59:0/326258952 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4037070 con 0x7f31ec0d0d10 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.515+0000 7f31eb400640 1 -- 172.21.15.59:0/326258952 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4032440 con 0x7f31ec0d0d10 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.515+0000 7f31eb400640 1 -- 172.21.15.59:0/326258952 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4032810 con 0x7f31ec0d0d10 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.515+0000 7f31f1600640 1 -- 172.21.15.59:0/326258952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec0d0d10 msgr2=0x7f31ec156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.515+0000 7f31f1600640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec0d0d10 0x7f31ec156bc0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009870 tx=0x7f31d402a210 comp rx=0 tx=0).stop 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 -- 172.21.15.59:0/326258952 shutdown_connections 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec15d810 0x7f31ec160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 --2- 172.21.15.59:0/326258952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec0d0d10 0x7f31ec156bc0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 -- 172.21.15.59:0/326258952 >> 172.21.15.59:0/326258952 conn(0x7f31ec06bdf0 msgr2=0x7f31ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 -- 172.21.15.59:0/326258952 shutdown_connections 2024-04-24T06:38:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.516+0000 7f31f1600640 1 -- 172.21.15.59:0/326258952 wait complete. 2024-04-24T06:38:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.517+0000 7f31f1600640 1 Processor -- start 2024-04-24T06:38:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.518+0000 7f31f1600640 1 -- start start 2024-04-24T06:38:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 0x7f31ec099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31ec15d810 0x7f31ec14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1607d0 con 0x7f31ec1567e0 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec160650 con 0x7f31ec15d810 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec160950 con 0x7f31ec0d0d10 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35274/0 (socket says 172.21.15.59:35274) 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 -- 172.21.15.59:0/4223025254 learned_addr learned my addr 172.21.15.59:0/4223025254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31ebe00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 0x7f31ec099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 -- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 msgr2=0x7f31ec099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 0x7f31ec099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31f0c00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31ec15d810 0x7f31ec14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 -- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31ec15d810 msgr2=0x7f31ec14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31ec15d810 0x7f31ec14f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31e3e00640 1 -- 172.21.15.59:0/4223025254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec150020 con 0x7f31ec1567e0 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.519+0000 7f31ebe00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 0x7f31ec099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:18.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31e3e00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009990 tx=0x7f31d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4037040 con 0x7f31ec1567e0 2024-04-24T06:38:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4042070 con 0x7f31ec1567e0 2024-04-24T06:38:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4030a40 con 0x7f31ec1567e0 2024-04-24T06:38:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec099e70 con 0x7f31ec1567e0 2024-04-24T06:38:18.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.520+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec157fe0 con 0x7f31ec1567e0 2024-04-24T06:38:18.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.521+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec04fd90 con 0x7f31ec1567e0 2024-04-24T06:38:18.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.522+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31d4030be0 con 0x7f31ec1567e0 2024-04-24T06:38:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.524+0000 7f31e8c00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 0x7f31c806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.524+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f31d40ac5b0 con 0x7f31ec1567e0 2024-04-24T06:38:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.524+0000 7f31ebe00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 0x7f31c806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.525+0000 7f31ebe00640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 0x7f31c806ee30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f31dc003eb0 tx=0x7f31dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.529+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31ec15cab0 con 0x7f31ec1567e0 2024-04-24T06:38:18.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.737+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f31ec0b5150 con 0x7f31ec1567e0 2024-04-24T06:38:18.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.738+0000 7f31e8c00640 1 -- 172.21.15.59:0/4223025254 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f31d407a980 con 0x7f31ec1567e0 2024-04-24T06:38:18.739 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 msgr2=0x7f31c806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 0x7f31c806ee30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f31dc003eb0 tx=0x7f31dc009040 comp rx=0 tx=0).stop 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 msgr2=0x7f31ec14f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009990 tx=0x7f31d40046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 shutdown_connections 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f31c806c970 0x7f31c806ee30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31ec15d810 0x7f31ec14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31ec1567e0 0x7f31ec14f340 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 --2- 172.21.15.59:0/4223025254 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f31ec0d0d10 0x7f31ec099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 >> 172.21.15.59:0/4223025254 conn(0x7f31ec06bdf0 msgr2=0x7f31ec15a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 shutdown_connections 2024-04-24T06:38:18.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.741+0000 7f31f1600640 1 -- 172.21.15.59:0/4223025254 wait complete. 2024-04-24T06:38:18.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:38:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.976+0000 7f0fc1e00640 1 Processor -- start 2024-04-24T06:38:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.976+0000 7f0fc1e00640 1 -- start start 2024-04-24T06:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.977+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fbc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.977+0000 7f0fc1e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0fbc072680 0x7f0fbc072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.977+0000 7f0fc1e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0cd880 con 0x7f0fbc0cf410 2024-04-24T06:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.977+0000 7f0fc1e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc071ec0 con 0x7f0fbc072680 2024-04-24T06:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.977+0000 7f0fc1e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0cdb20 con 0x7f0fbc06ddf0 2024-04-24T06:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fbaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0fbc0cf410 0x7f0fbc15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37190/0 (socket says 172.21.15.59:37190) 2024-04-24T06:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fbaa00640 1 -- 172.21.15.59:0/4044961324 learned_addr learned my addr 172.21.15.59:0/4044961324 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fbb400640 1 --2- 172.21.15.59:0/4044961324 >> v2:172.21.15.66:3300/0 conn(0x7f0fbc072680 0x7f0fbc072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fba000640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fbc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783699309 0 0) 0x7f0fbc0cd880 con 0x7f0fbc0cf410 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.978+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fa0003520 con 0x7f0fbc0cf410 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2288391024 0 0) 0x7f0fa0003520 con 0x7f0fbc0cf410 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 msgr2=0x7f0fbc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fbc06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 >> v2:172.21.15.66:3300/0 conn(0x7f0fbc072680 msgr2=0x7f0fbc072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 --2- 172.21.15.59:0/4044961324 >> v2:172.21.15.66:3300/0 conn(0x7f0fbc072680 0x7f0fbc072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc072160 con 0x7f0fbc0cf410 2024-04-24T06:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0fac001670 con 0x7f0fbc0cf410 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 >> v1:172.21.15.59:6789/0 conn(0x7f0fbc0cf410 legacy=0x7f0fbc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fa00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0003f80 0x7f0fa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa0004d40 0x7f0fa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc071ec0 con 0x7f0fa0004d40 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0cdb20 con 0x7f0fa0003f80 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa000ba70 con 0x7f0fbc06ddf0 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.979+0000 7f0fbb400640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa0004d40 0x7f0fa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbaa00640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fa00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fba000640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0003f80 0x7f0fa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbb400640 1 -- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 msgr2=0x7f0fa00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbb400640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc06ddf0 0x7f0fa00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbb400640 1 -- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0003f80 msgr2=0x7f0fa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbb400640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0003f80 0x7f0fa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.980+0000 7f0fbb400640 1 -- 172.21.15.59:0/4044961324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa0003520 con 0x7f0fa0004d40 2024-04-24T06:38:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.981+0000 7f0fbb400640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa0004d40 0x7f0fa00051c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009fc0 tx=0x7f0fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cd87fea6014380 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.981+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4009c10 con 0x7f0fa0004d40 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.981+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4036070 con 0x7f0fa0004d40 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.981+0000 7f0fb9600640 1 -- 172.21.15.59:0/4044961324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb402d070 con 0x7f0fa0004d40 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 -- 172.21.15.59:0/4044961324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa0004d40 msgr2=0x7f0fa00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/4044961324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa0004d40 0x7f0fa00051c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009fc0 tx=0x7f0fa0005830 comp rx=0 tx=0).stop 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fbb400640 1 -- 172.21.15.59:0/4044961324 reap_dead start 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 -- 172.21.15.59:0/4044961324 shutdown_connections 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 -- 172.21.15.59:0/4044961324 >> 172.21.15.59:0/4044961324 conn(0x7f0fbc0b2130 msgr2=0x7f0fbc0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 -- 172.21.15.59:0/4044961324 shutdown_connections 2024-04-24T06:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.982+0000 7f0fc1e00640 1 -- 172.21.15.59:0/4044961324 wait complete. 2024-04-24T06:38:18.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.984+0000 7f0fc1e00640 1 Processor -- start 2024-04-24T06:38:18.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.984+0000 7f0fc1e00640 1 -- start start 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 0x7f0fbc14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 0x7f0fbc15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15a820 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15a6a0 con 0x7f0fa0004d40 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fc1e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc06e250 con 0x7f0fbc0b4bd0 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57104/0 (socket says 172.21.15.59:57104) 2024-04-24T06:38:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.985+0000 7f0fbb400640 1 -- 172.21.15.59:0/425697701 learned_addr learned my addr 172.21.15.59:0/425697701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbaa00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 0x7f0fbc14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fba000640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 0x7f0fbc15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 -- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 msgr2=0x7f0fbc15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 0x7f0fbc15ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 -- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 msgr2=0x7f0fbc14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 0x7f0fbc14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 -- 172.21.15.59:0/425697701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc15e700 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fba000640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 0x7f0fbc15ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbaa00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 0x7f0fbc14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fbb400640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009f60 tx=0x7f0fb4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4009c10 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4003b90 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4031e30 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.986+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc15e9e0 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.987+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc15efa0 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.988+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc068a80 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.989+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fb4005d10 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.990+0000 7f0fb3400640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 0x7f0f9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.990+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f0fb40ad060 con 0x7f0fbc0b14d0 2024-04-24T06:38:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.990+0000 7f0fbaa00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 0x7f0f9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.991+0000 7f0fbaa00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 0x7f0f9806ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005990 tx=0x7f0fac000f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:18.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:18.995+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fb407b3b0 con 0x7f0fbc0b14d0 2024-04-24T06:38:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.213+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fbc153450 con 0x7f0fbc0b14d0 2024-04-24T06:38:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.213+0000 7f0fb3400640 1 -- 172.21.15.59:0/425697701 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v87) ==== 80+0+242 (secure 0 0 0) 0x7f0fbc153450 con 0x7f0fbc0b14d0 2024-04-24T06:38:19.214 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6835/2792490076 2024-04-25T06:37:35.076161+0000 2024-04-24T06:38:19.214 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6834/2792490076 2024-04-25T06:37:35.076161+0000 2024-04-24T06:38:19.215 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:38:19.215 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6832/301885470 2024-04-25T06:36:16.639793+0000 2024-04-24T06:38:19.215 INFO:teuthology.orchestra.run.smithi059.stderr:listed 4 entries 2024-04-24T06:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 msgr2=0x7f0f9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 0x7f0f9806ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005990 tx=0x7f0fac000f90 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 msgr2=0x7f0fbc0b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009f60 tx=0x7f0fb4009750 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 shutdown_connections 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f9806c920 0x7f0f9806ede0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fbc0b14d0 0x7f0fbc0b1980 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0fbc0b4bd0 0x7f0fbc15ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 --2- 172.21.15.59:0/425697701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fa0004d40 0x7f0fbc14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 >> 172.21.15.59:0/425697701 conn(0x7f0fbc0b2130 msgr2=0x7f0fbc15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 shutdown_connections 2024-04-24T06:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:19.216+0000 7f0fc1e00640 1 -- 172.21.15.59:0/425697701 wait complete. 2024-04-24T06:38:19.235 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:38:19.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:19.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:19.307 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[66780]: fuse finished with error 0 and tester_r 0 2024-04-24T06:38:19.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:38:19.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:19.312 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:38:19.469 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:38:19.470 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:38:19.470 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:19.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:38:19.531 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:38:19.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:19.532 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:38:19.591 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:38:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:38:19.592 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:38:19.593 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:19.593 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:19.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:38:19.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:38:19.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:19.654 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:38:19.711 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:38:19.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:38:19.712 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:38:19.712 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:38:19.712 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:19.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:38:19.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:38:19.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:19.775 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:38:19.831 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:38:19.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:38:19.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:20.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.046+0000 7f45a7e00640 1 Processor -- start 2024-04-24T06:38:20.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.047+0000 7f45a7e00640 1 -- start start 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45a80d2630 0x7f45a80b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f45a80b6530 0x7f45a80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f45a80b7000 con 0x7f45a80d2630 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f45a80cf620 con 0x7f45a80b6530 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f45a80cf030 con 0x7f45a815e7c0 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45a80d2630 0x7f45a80b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45a80d2630 0x7f45a80b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57116/0 (socket says 172.21.15.59:57116) 2024-04-24T06:38:20.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.048+0000 7f45a6000640 1 -- 172.21.15.59:0/1085433314 learned_addr learned my addr 172.21.15.59:0/1085433314 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:20.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.049+0000 7f45a6000640 1 -- 172.21.15.59:0/1085433314 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 msgr2=0x7f45a80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:20.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.049+0000 7f45a6000640 1 --2- 172.21.15.59:0/1085433314 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.049+0000 7f45a6000640 1 -- 172.21.15.59:0/1085433314 >> v2:172.21.15.66:3300/0 conn(0x7f45a80b6530 msgr2=0x7f45a80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:20.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.049+0000 7f45a6000640 1 --2- 172.21.15.59:0/1085433314 >> v2:172.21.15.66:3300/0 conn(0x7f45a80b6530 0x7f45a80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.049+0000 7f45a6000640 1 -- 172.21.15.59:0/1085433314 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a80b6a40 con 0x7f45a80d2630 2024-04-24T06:38:20.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.050+0000 7f45a6000640 1 --2- 172.21.15.59:0/1085433314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80d2630 0x7f45a80b5ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f45a80cf590 tx=0x7f459802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118c0527391cc5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.050+0000 7f45a5600640 1 -- 172.21.15.59:0/1085433314 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598036070 con 0x7f45a80d2630 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.050+0000 7f45a5600640 1 -- 172.21.15.59:0/1085433314 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598031600 con 0x7f45a80d2630 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.050+0000 7f45a5600640 1 -- 172.21.15.59:0/1085433314 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980319d0 con 0x7f45a80d2630 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 -- 172.21.15.59:0/1085433314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80d2630 msgr2=0x7f45a80b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 --2- 172.21.15.59:0/1085433314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80d2630 0x7f45a80b5ff0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f45a80cf590 tx=0x7f459802a690 comp rx=0 tx=0).stop 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 -- 172.21.15.59:0/1085433314 shutdown_connections 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 --2- 172.21.15.59:0/1085433314 >> v2:172.21.15.66:3300/0 conn(0x7f45a80b6530 0x7f45a80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 --2- 172.21.15.59:0/1085433314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80d2630 0x7f45a80b5ff0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 --2- 172.21.15.59:0/1085433314 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a80d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 -- 172.21.15.59:0/1085433314 >> 172.21.15.59:0/1085433314 conn(0x7f45a809d920 msgr2=0x7f45a808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 -- 172.21.15.59:0/1085433314 shutdown_connections 2024-04-24T06:38:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.051+0000 7f45a7e00640 1 -- 172.21.15.59:0/1085433314 wait complete. 2024-04-24T06:38:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.053+0000 7f45a7e00640 1 Processor -- start 2024-04-24T06:38:20.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.053+0000 7f45a7e00640 1 -- start start 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45a80d2630 0x7f45a8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80d1a80 con 0x7f45a80b6530 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f45a80cf2b0 con 0x7f45a80d2630 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f45a80d1c00 con 0x7f45a815e7c0 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57128/0 (socket says 172.21.15.59:57128) 2024-04-24T06:38:20.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.054+0000 7f45a6a00640 1 -- 172.21.15.59:0/558659863 learned_addr learned my addr 172.21.15.59:0/558659863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:20.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a7400640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6000640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45a80d2630 0x7f45a8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 -- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 msgr2=0x7f45a815eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a815eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 -- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45a80d2630 msgr2=0x7f45a8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45a80d2630 0x7f45a8159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 -- 172.21.15.59:0/558659863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8159f40 con 0x7f45a80b6530 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a7400640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a815eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f45a6a00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f459000b440 tx=0x7f459000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.055+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590015020 con 0x7f45a80b6530 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.056+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590011890 con 0x7f45a80b6530 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.056+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590011c80 con 0x7f45a80b6530 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.056+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a80cbd20 con 0x7f45a80b6530 2024-04-24T06:38:20.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.056+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81f83c0 con 0x7f45a80b6530 2024-04-24T06:38:20.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.057+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a81630d0 con 0x7f45a80b6530 2024-04-24T06:38:20.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.058+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4590018020 con 0x7f45a80b6530 2024-04-24T06:38:20.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.059+0000 7f459f400640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.059+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f459008bd20 con 0x7f45a80b6530 2024-04-24T06:38:20.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.059+0000 7f45a6000640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.060+0000 7f45a6000640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 0x7f458406edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4598033000 tx=0x7f4598009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.064+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45a81630d0 con 0x7f45a80b6530 2024-04-24T06:38:20.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.310+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45a804fd90 con 0x7f45a80b6530 2024-04-24T06:38:20.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.312+0000 7f459f400640 1 -- 172.21.15.59:0/558659863 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7f459001f020 con 0x7f45a80b6530 2024-04-24T06:38:20.313 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:20.313 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:38:20.313 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.314+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 msgr2=0x7f458406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.314+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 0x7f458406edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4598033000 tx=0x7f4598009000 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.314+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 msgr2=0x7f45a80ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.314+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f459000b440 tx=0x7f459000b900 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 shutdown_connections 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45a815e7c0 0x7f45a815eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45a80d2630 0x7f45a8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 --2- 172.21.15.59:0/558659863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a80b6530 0x7f45a80ca9a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 >> 172.21.15.59:0/558659863 conn(0x7f45a809d920 msgr2=0x7f45a815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 shutdown_connections 2024-04-24T06:38:20.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.315+0000 7f45a7e00640 1 -- 172.21.15.59:0/558659863 wait complete. 2024-04-24T06:38:20.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.550+0000 7fc153e00640 1 Processor -- start 2024-04-24T06:38:20.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.551+0000 7fc153e00640 1 -- start start 2024-04-24T06:38:20.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1540d0d90 0x7fc1540d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1540cf030 con 0x7fc1540d2630 2024-04-24T06:38:20.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc1540b7000 con 0x7fc1540d0d90 2024-04-24T06:38:20.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc1540cf620 con 0x7fc15415e7c0 2024-04-24T06:38:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.552+0000 7fc153400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1540d0d90 0x7fc1540d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1540d0d90 0x7fc1540d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57138/0 (socket says 172.21.15.59:57138) 2024-04-24T06:38:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 -- 172.21.15.59:0/2517890583 learned_addr learned my addr 172.21.15.59:0/2517890583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:20.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc152a00640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 -- 172.21.15.59:0/2517890583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 msgr2=0x7fc1540d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 -- 172.21.15.59:0/2517890583 >> v1:172.21.15.66:6789/0 conn(0x7fc1540d2630 legacy=0x7fc1540b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:20.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.553+0000 7fc153400640 1 -- 172.21.15.59:0/2517890583 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540b6a40 con 0x7fc1540d0d90 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.554+0000 7fc153400640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc1540d1220 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009870 tx=0x7fc14c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66d7bf0a2149cec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.555+0000 7fc151600640 1 -- 172.21.15.59:0/2517890583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c037070 con 0x7fc1540d0d90 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.555+0000 7fc151600640 1 -- 172.21.15.59:0/2517890583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c032440 con 0x7fc1540d0d90 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.555+0000 7fc151600640 1 -- 172.21.15.59:0/2517890583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c032810 con 0x7fc1540d0d90 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.555+0000 7fc153e00640 1 -- 172.21.15.59:0/2517890583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 msgr2=0x7fc1540d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.555+0000 7fc153e00640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc1540d1220 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009870 tx=0x7fc14c02a210 comp rx=0 tx=0).stop 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 -- 172.21.15.59:0/2517890583 shutdown_connections 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc1540d1220 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 --2- 172.21.15.59:0/2517890583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 -- 172.21.15.59:0/2517890583 >> 172.21.15.59:0/2517890583 conn(0x7fc15409d920 msgr2=0x7fc15408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 -- 172.21.15.59:0/2517890583 shutdown_connections 2024-04-24T06:38:20.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.556+0000 7fc153e00640 1 -- 172.21.15.59:0/2517890583 wait complete. 2024-04-24T06:38:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.557+0000 7fc153e00640 1 Processor -- start 2024-04-24T06:38:20.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.557+0000 7fc153e00640 1 -- start start 2024-04-24T06:38:20.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1540d2630 0x7fc1540c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540b6700 con 0x7fc1540d0d90 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540cf2b0 con 0x7fc1540d2630 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.558+0000 7fc153e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc1540d1a80 con 0x7fc15415e7c0 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57144/0 (socket says 172.21.15.59:57144) 2024-04-24T06:38:20.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 -- 172.21.15.59:0/3614974765 learned_addr learned my addr 172.21.15.59:0/3614974765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc153400640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152000640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1540d2630 0x7fc1540c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 -- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 msgr2=0x7fc1540c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 -- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1540d2630 msgr2=0x7fc1540c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1540d2630 0x7fc1540c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc152a00640 1 -- 172.21.15.59:0/3614974765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540c8ba0 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.559+0000 7fc153400640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc152a00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc144007e40 tx=0x7fc14400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144014620 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc144019070 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144014a00 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1540c8e80 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.560+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1540c9440 con 0x7fc1540d0d90 2024-04-24T06:38:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.561+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15404fd90 con 0x7fc1540d0d90 2024-04-24T06:38:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.562+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc144014ba0 con 0x7fc1540d0d90 2024-04-24T06:38:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.563+0000 7fc14aa00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 0x7fc13006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.563+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc14408c540 con 0x7fc1540d0d90 2024-04-24T06:38:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.564+0000 7fc152000640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 0x7fc13006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:20.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.564+0000 7fc152000640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 0x7fc13006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc1541605c0 tx=0x7fc13c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:20.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.568+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc14405a800 con 0x7fc1540d0d90 2024-04-24T06:38:20.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.772+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc15415dc00 con 0x7fc1540d0d90 2024-04-24T06:38:20.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.776+0000 7fc14aa00640 1 -- 172.21.15.59:0/3614974765 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16883 (secure 0 0 0) 0x7fc14405a100 con 0x7fc1540d0d90 2024-04-24T06:38:20.777 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:20.777 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:37:35.076193+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.59:6834/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 msgr2=0x7fc13006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 0x7fc13006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc1541605c0 tx=0x7fc13c009040 comp rx=0 tx=0).stop 2024-04-24T06:38:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 msgr2=0x7fc154153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc144007e40 tx=0x7fc14400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 shutdown_connections 2024-04-24T06:38:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc13006c8f0 0x7fc13006edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc15415e7c0 0x7fc1540c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1540d2630 0x7fc1540c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 --2- 172.21.15.59:0/3614974765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1540d0d90 0x7fc154153ad0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 >> 172.21.15.59:0/3614974765 conn(0x7fc15409d920 msgr2=0x7fc15415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 shutdown_connections 2024-04-24T06:38:20.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.779+0000 7fc153e00640 1 -- 172.21.15.59:0/3614974765 wait complete. 2024-04-24T06:38:20.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:35:52.188853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:35:53.282826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:20.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:21.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.999+0000 7fb7fea00640 1 Processor -- start 2024-04-24T06:38:21.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:20.999+0000 7fb7fea00640 1 -- start start 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f815ed20 0x7fb7f80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb7f80b3600 0x7fb7f80d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7f80b3f10 0x7fb7f80b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f815f100 con 0x7fb7f80b3600 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f80b5da0 con 0x7fb7f80b3f10 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f80d0a20 con 0x7fb7f815ed20 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fcc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb7f80b3600 0x7fb7f80d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fcc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb7f80b3600 0x7fb7f80d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57156/0 (socket says 172.21.15.59:57156) 2024-04-24T06:38:21.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.001+0000 7fb7fcc00640 1 -- 172.21.15.59:0/377335823 learned_addr learned my addr 172.21.15.59:0/377335823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:21.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fd600640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f815ed20 0x7fb7f80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fe000640 1 --2- 172.21.15.59:0/377335823 >> v2:172.21.15.66:3300/0 conn(0x7fb7f80b3f10 0x7fb7f80b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fcc00640 1 -- 172.21.15.59:0/377335823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f815ed20 msgr2=0x7fb7f80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f815ed20 0x7fb7f80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fcc00640 1 -- 172.21.15.59:0/377335823 >> v2:172.21.15.66:3300/0 conn(0x7fb7f80b3f10 msgr2=0x7fb7f80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/377335823 >> v2:172.21.15.66:3300/0 conn(0x7fb7f80b3f10 0x7fb7f80b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.002+0000 7fb7fcc00640 1 -- 172.21.15.59:0/377335823 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f80d2ed0 con 0x7fb7f80b3600 2024-04-24T06:38:21.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.003+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80d13e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb7f808c430 tx=0x7fb7e402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5077d87a9a461b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7f7e00640 1 -- 172.21.15.59:0/377335823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037070 con 0x7fb7f80b3600 2024-04-24T06:38:21.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7f7e00640 1 -- 172.21.15.59:0/377335823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e402ae00 con 0x7fb7f80b3600 2024-04-24T06:38:21.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7f7e00640 1 -- 172.21.15.59:0/377335823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4032660 con 0x7fb7f80b3600 2024-04-24T06:38:21.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7fea00640 1 -- 172.21.15.59:0/377335823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 msgr2=0x7fb7f80d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7fea00640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80d13e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb7f808c430 tx=0x7fb7e402a0b0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.004+0000 7fb7fea00640 1 -- 172.21.15.59:0/377335823 shutdown_connections 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 --2- 172.21.15.59:0/377335823 >> v2:172.21.15.66:3300/0 conn(0x7fb7f80b3f10 0x7fb7f80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80d13e0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 --2- 172.21.15.59:0/377335823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f815ed20 0x7fb7f80d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 -- 172.21.15.59:0/377335823 >> 172.21.15.59:0/377335823 conn(0x7fb7f808bf90 msgr2=0x7fb7f80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 -- 172.21.15.59:0/377335823 shutdown_connections 2024-04-24T06:38:21.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.005+0000 7fb7fea00640 1 -- 172.21.15.59:0/377335823 wait complete. 2024-04-24T06:38:21.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.006+0000 7fb7fea00640 1 Processor -- start 2024-04-24T06:38:21.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.006+0000 7fb7fea00640 1 -- start start 2024-04-24T06:38:21.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f80b3f10 0x7fb7f81e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f815ed20 0x7fb7f8152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f80b48e0 con 0x7fb7f80b3600 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f80d3490 con 0x7fb7f815ed20 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.007+0000 7fb7fea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f80b4a60 con 0x7fb7f80b3f10 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57170/0 (socket says 172.21.15.59:57170) 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 -- 172.21.15.59:0/2670680266 learned_addr learned my addr 172.21.15.59:0/2670680266 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:21.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f80b3f10 0x7fb7f81e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fe000640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f815ed20 0x7fb7f8152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 -- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f80b3f10 msgr2=0x7fb7f81e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f80b3f10 0x7fb7f81e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 -- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f815ed20 msgr2=0x7fb7f8152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f815ed20 0x7fb7f8152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.008+0000 7fb7fd600640 1 -- 172.21.15.59:0/2670680266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f81e4ef0 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7fd600640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d620 tx=0x7fb7ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00b620 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ec019070 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81e5170 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00ba00 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.009+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81e5730 con 0x7fb7f80b3600 2024-04-24T06:38:21.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.010+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f80ba480 con 0x7fb7f80b3600 2024-04-24T06:38:21.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.011+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7ec00bba0 con 0x7fb7f80b3600 2024-04-24T06:38:21.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.013+0000 7fb7f5600640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.013+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fb7ec08bff0 con 0x7fb7f80b3600 2024-04-24T06:38:21.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.013+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.014+0000 7fb7fcc00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 0x7fb7d806edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4034000 tx=0x7fb7e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.017+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7f80ba480 con 0x7fb7f80b3600 2024-04-24T06:38:21.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.211+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7f804fd90 con 0x7fb7f80b3600 2024-04-24T06:38:21.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.212+0000 7fb7f5600640 1 -- 172.21.15.59:0/2670680266 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fb7f81e4ef0 con 0x7fb7f80b3600 2024-04-24T06:38:21.213 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 msgr2=0x7fb7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 0x7fb7d806edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4034000 tx=0x7fb7e402e000 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 msgr2=0x7fb7f80c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d620 tx=0x7fb7ec00dae0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 shutdown_connections 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb7d806c8f0 0x7fb7d806edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f815ed20 0x7fb7f8152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7f80b3f10 0x7fb7f81e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 --2- 172.21.15.59:0/2670680266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7f80b3600 0x7fb7f80c9e70 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.214+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 >> 172.21.15.59:0/2670680266 conn(0x7fb7f808bf90 msgr2=0x7fb7f80d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:21.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.215+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 shutdown_connections 2024-04-24T06:38:21.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.215+0000 7fb7fea00640 1 -- 172.21.15.59:0/2670680266 wait complete. 2024-04-24T06:38:21.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:38:21.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.443+0000 7fbd7be00640 1 Processor -- start 2024-04-24T06:38:21.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.443+0000 7fbd7be00640 1 -- start start 2024-04-24T06:38:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.444+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd7c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.444+0000 7fbd7be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbd7c15d810 0x7fbd7c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.444+0000 7fbd7be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7c15cab0 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.444+0000 7fbd7be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7c0d0f70 con 0x7fbd7c15d810 2024-04-24T06:38:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.444+0000 7fbd7be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c160e40 con 0x7fbd7c1567e0 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd7b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbd7c0d0b40 0x7fbd7c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37204/0 (socket says 172.21.15.59:37204) 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd7b400640 1 -- 172.21.15.59:0/863118287 learned_addr learned my addr 172.21.15.59:0/863118287 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd73e00640 1 --2- 172.21.15.59:0/863118287 >> v2:172.21.15.66:3300/0 conn(0x7fbd7c15d810 0x7fbd7c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd7c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326686721 0 0) 0x7fbd74001490 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.445+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd60003520 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2118308979 0 0) 0x7fbd60003520 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 msgr2=0x7fbd7c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd7c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 >> v2:172.21.15.66:3300/0 conn(0x7fbd7c15d810 msgr2=0x7fbd7c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 --2- 172.21.15.59:0/863118287 >> v2:172.21.15.66:3300/0 conn(0x7fbd7c15d810 0x7fbd7c0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c15cd30 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbd74001bc0 con 0x7fbd7c0d0b40 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 >> v1:172.21.15.59:6789/0 conn(0x7fbd7c0d0b40 legacy=0x7fbd7c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd600039b0 0x7fbd60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0d0f70 con 0x7fbd600039b0 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c160e40 con 0x7fbd60004d40 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6000b9c0 con 0x7fbd7c1567e0 2024-04-24T06:38:21.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.446+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd600039b0 0x7fbd60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7b400640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd73e00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7aa00640 1 -- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 msgr2=0x7fbd60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7aa00640 1 -- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd60004d40 msgr2=0x7fbd600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7aa00640 1 -- 172.21.15.59:0/863118287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd60003520 con 0x7fbd600039b0 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd7b400640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:21.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.447+0000 7fbd73e00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1567e0 0x7fbd60004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.448+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd600039b0 0x7fbd60003dd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009ac0 tx=0x7fbd60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1307c3b07aff16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.448+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c036070 con 0x7fbd600039b0 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.448+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c031400 con 0x7fbd600039b0 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.448+0000 7fbd7a000640 1 -- 172.21.15.59:0/863118287 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c009690 con 0x7fbd600039b0 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 -- 172.21.15.59:0/863118287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd600039b0 msgr2=0x7fbd60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 --2- 172.21.15.59:0/863118287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd600039b0 0x7fbd60003dd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009ac0 tx=0x7fbd60005830 comp rx=0 tx=0).stop 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7b400640 1 -- 172.21.15.59:0/863118287 reap_dead start 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 -- 172.21.15.59:0/863118287 shutdown_connections 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 -- 172.21.15.59:0/863118287 >> 172.21.15.59:0/863118287 conn(0x7fbd7c06bdf0 msgr2=0x7fbd7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 -- 172.21.15.59:0/863118287 shutdown_connections 2024-04-24T06:38:21.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.449+0000 7fbd7be00640 1 -- 172.21.15.59:0/863118287 wait complete. 2024-04-24T06:38:21.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.451+0000 7fbd7be00640 1 Processor -- start 2024-04-24T06:38:21.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.451+0000 7fbd7be00640 1 -- start start 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 0x7fbd7c1f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1f4730 0x7fbd7c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c156bc0 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c15d2f0 con 0x7fbd600039b0 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd7be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c156d40 con 0x7fbd7c1f4730 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57188/0 (socket says 172.21.15.59:57188) 2024-04-24T06:38:21.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.452+0000 7fbd73e00640 1 -- 172.21.15.59:0/3404738346 learned_addr learned my addr 172.21.15.59:0/3404738346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 0x7fbd7c1f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd7b400640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1f4730 0x7fbd7c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 -- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1f4730 msgr2=0x7fbd7c1f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1f4730 0x7fbd7c1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 -- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 msgr2=0x7fbd7c1f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 0x7fbd7c1f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 -- 172.21.15.59:0/3404738346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c1582f0 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 0x7fbd7c1f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.453+0000 7fbd73e00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbd640109d0 tx=0x7fbd64010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.454+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400f040 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.454+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6400ec20 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.454+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6401c400 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.454+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c1585d0 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.454+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c158b90 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.455+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7c04fd90 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.456+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd6401c5a0 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.457+0000 7fbd72a00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 0x7fbd5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.457+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fbd6408c310 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.457+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 0x7fbd5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.458+0000 7fbd7aa00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 0x7fbd5806edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c033000 tx=0x7fbd6c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.462+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd64018030 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.669+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd7c0b5150 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.670+0000 7fbd72a00640 1 -- 172.21.15.59:0/3404738346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fbd6405a660 con 0x7fbd7c06dcb0 2024-04-24T06:38:21.671 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 msgr2=0x7fbd5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 0x7fbd5806edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c033000 tx=0x7fbd6c008030 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 msgr2=0x7fbd7c1f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbd640109d0 tx=0x7fbd64010e90 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 shutdown_connections 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbd5806c8f0 0x7fbd5806edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbd7c1f4730 0x7fbd7c1f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd7c06dcb0 0x7fbd7c1f8040 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 --2- 172.21.15.59:0/3404738346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd600039b0 0x7fbd7c1f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 >> 172.21.15.59:0/3404738346 conn(0x7fbd7c06bdf0 msgr2=0x7fbd7c072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 shutdown_connections 2024-04-24T06:38:21.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.673+0000 7fbd7be00640 1 -- 172.21.15.59:0/3404738346 wait complete. 2024-04-24T06:38:21.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:38:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.901+0000 7f9056800640 1 Processor -- start 2024-04-24T06:38:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.902+0000 7f9056800640 1 -- start start 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.903+0000 7f9056800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9050071c70 0x7f9050163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.903+0000 7f9056800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 0x7f90500d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.903+0000 7f9056800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9050163320 con 0x7f905015d300 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.903+0000 7f9056800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f90500d0fb0 con 0x7f9050071c70 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.903+0000 7f9056800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9050163650 con 0x7f90500d0600 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9050071c70 0x7f9050163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9050071c70 0x7f9050163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57196/0 (socket says 172.21.15.59:57196) 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 -- 172.21.15.59:0/3251714643 learned_addr learned my addr 172.21.15.59:0/3251714643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9055e00640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 0x7f90500d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 -- 172.21.15.59:0/3251714643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 msgr2=0x7f90500d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 0x7f90500d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 -- 172.21.15.59:0/3251714643 >> v1:172.21.15.66:6789/0 conn(0x7f905015d300 legacy=0x7f9050167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9054a00640 1 -- 172.21.15.59:0/3251714643 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90500720d0 con 0x7f9050071c70 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.904+0000 7f9055e00640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 0x7f90500d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.905+0000 7f9054a00640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9050071c70 0x7f9050163ae0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f90500afdb0 tx=0x7f903c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a424c06c4124e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f904fe00640 1 -- 172.21.15.59:0/3251714643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c037070 con 0x7f9050071c70 2024-04-24T06:38:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f904fe00640 1 -- 172.21.15.59:0/3251714643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c032440 con 0x7f9050071c70 2024-04-24T06:38:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f904fe00640 1 -- 172.21.15.59:0/3251714643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c032810 con 0x7f9050071c70 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 -- 172.21.15.59:0/3251714643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9050071c70 msgr2=0x7f9050163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9050071c70 0x7f9050163ae0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f90500afdb0 tx=0x7f903c02a200 comp rx=0 tx=0).stop 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 -- 172.21.15.59:0/3251714643 shutdown_connections 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90500d0600 0x7f90500d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 --2- 172.21.15.59:0/3251714643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9050071c70 0x7f9050163ae0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.906+0000 7f9056800640 1 -- 172.21.15.59:0/3251714643 >> 172.21.15.59:0/3251714643 conn(0x7f905006bdf0 msgr2=0x7f905006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.907+0000 7f9056800640 1 -- 172.21.15.59:0/3251714643 shutdown_connections 2024-04-24T06:38:21.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.907+0000 7f9056800640 1 -- 172.21.15.59:0/3251714643 wait complete. 2024-04-24T06:38:21.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.908+0000 7f9056800640 1 Processor -- start 2024-04-24T06:38:21.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.909+0000 7f9056800640 1 -- start start 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.909+0000 7f9056800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 0x7f905014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9056800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9056800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 0x7f905009b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9056800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9050072940 con 0x7f90500d0600 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9056800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9050072660 con 0x7f9050071c70 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9056800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9050164020 con 0x7f905015d300 2024-04-24T06:38:21.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57200/0 (socket says 172.21.15.59:57200) 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 -- 172.21.15.59:0/3195243420 learned_addr learned my addr 172.21.15.59:0/3195243420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9055400640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 0x7f905014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 -- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 msgr2=0x7f905009b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9055e00640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 0x7f905009b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 0x7f905009b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 -- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 msgr2=0x7f905014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 0x7f905014a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9054a00640 1 -- 172.21.15.59:0/3195243420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9050153f40 con 0x7f90500d0600 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9055400640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 0x7f905014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:21.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.910+0000 7f9055e00640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 0x7f905009b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f9054a00640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f903c000c00 tx=0x7f903c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c041020 con 0x7f90500d0600 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c030990 con 0x7f90500d0600 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90501557e0 con 0x7f90500d0600 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c037040 con 0x7f90500d0600 2024-04-24T06:38:21.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.911+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9050155da0 con 0x7f90500d0600 2024-04-24T06:38:21.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.913+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905004fd90 con 0x7f90500d0600 2024-04-24T06:38:21.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.914+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f903c044020 con 0x7f90500d0600 2024-04-24T06:38:21.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.915+0000 7f904d600640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 0x7f903006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:21.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.915+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f903c0ac6b0 con 0x7f90500d0600 2024-04-24T06:38:21.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.915+0000 7f9055400640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 0x7f903006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:21.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.916+0000 7f9055400640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 0x7f903006edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f904400a810 tx=0x7f9044008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:21.920+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f903c04b030 con 0x7f90500d0600 2024-04-24T06:38:22.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.128+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90500cff50 con 0x7f90500d0600 2024-04-24T06:38:22.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.129+0000 7f904d600640 1 -- 172.21.15.59:0/3195243420 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f903c07a970 con 0x7f90500d0600 2024-04-24T06:38:22.130 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:38:22.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.131+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 msgr2=0x7f903006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.131+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 0x7f903006edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f904400a810 tx=0x7f9044008040 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.131+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 msgr2=0x7f905009ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.131+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f903c000c00 tx=0x7f903c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 shutdown_connections 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f903006c8f0 0x7f903006edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f905015d300 0x7f905009b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90500d0600 0x7f905009ab00 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 --2- 172.21.15.59:0/3195243420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9050071c70 0x7f905014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 >> 172.21.15.59:0/3195243420 conn(0x7f905006bdf0 msgr2=0x7f905009d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 shutdown_connections 2024-04-24T06:38:22.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.132+0000 7f9056800640 1 -- 172.21.15.59:0/3195243420 wait complete. 2024-04-24T06:38:22.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:22.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.363+0000 7f245b200640 1 Processor -- start 2024-04-24T06:38:22.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.363+0000 7f245b200640 1 -- start start 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f245c0cff10 0x7f245c0d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f245c15fa20 con 0x7f245c0d0330 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f245c0cf430 con 0x7f245c0d0ec0 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f245c0b3b60 con 0x7f245c0cff10 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f245c0d0330 0x7f245c15eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37206/0 (socket says 172.21.15.59:37206) 2024-04-24T06:38:22.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.365+0000 7f245a800640 1 -- 172.21.15.59:0/4249826913 learned_addr learned my addr 172.21.15.59:0/4249826913 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2451400640 1 --2- 172.21.15.59:0/4249826913 >> v2:172.21.15.66:3300/0 conn(0x7f245c0cff10 0x7f245c0d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191856704 0 0) 0x7f245c15fa20 con 0x7f245c0d0330 2024-04-24T06:38:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440003520 con 0x7f245c0d0330 2024-04-24T06:38:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1413654915 0 0) 0x7f2440003520 con 0x7f245c0d0330 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 >> v2:172.21.15.66:3300/0 conn(0x7f245c0cff10 msgr2=0x7f245c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 --2- 172.21.15.59:0/4249826913 >> v2:172.21.15.66:3300/0 conn(0x7f245c0cff10 0x7f245c0d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 >> v1:172.21.15.66:6789/0 conn(0x7f245c0d0ec0 legacy=0x7f245c0d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0cf710 con 0x7f245c0d0330 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f24540019f0 con 0x7f245c0d0330 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 >> v1:172.21.15.59:6789/0 conn(0x7f245c0d0330 legacy=0x7f245c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 0x7f2440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f244000c040 con 0x7f24400039b0 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f244000c1f0 con 0x7f24400051c0 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.366+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f244000c3f0 con 0x7f2440004800 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 0x7f2440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2451400640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 -- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2440004800 msgr2=0x7f2440004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2440004800 0x7f2440004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f245a800640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 -- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24400051c0 msgr2=0x7f244000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f2459e00640 1 -- 172.21.15.59:0/4249826913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2440003520 con 0x7f24400039b0 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f245a800640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24400051c0 0x7f244000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:22.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.367+0000 7f245a800640 1 -- 172.21.15.59:0/4249826913 reap_dead start 2024-04-24T06:38:22.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.368+0000 7f2459e00640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 0x7f2440003dd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f244c001750 tx=0x7f2440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9dae3649be1690 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.368+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c001370 con 0x7f24400039b0 2024-04-24T06:38:22.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c0052d0 con 0x7f24400039b0 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f2459400640 1 -- 172.21.15.59:0/4249826913 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c02abc0 con 0x7f24400039b0 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 -- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 msgr2=0x7f2440003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 0x7f2440003dd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f244c001750 tx=0x7f2440005830 comp rx=0 tx=0).stop 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 -- 172.21.15.59:0/4249826913 shutdown_connections 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 --2- 172.21.15.59:0/4249826913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24400039b0 0x7f2440003dd0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 -- 172.21.15.59:0/4249826913 >> 172.21.15.59:0/4249826913 conn(0x7f245c08bf90 msgr2=0x7f245c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 -- 172.21.15.59:0/4249826913 shutdown_connections 2024-04-24T06:38:22.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.369+0000 7f245b200640 1 -- 172.21.15.59:0/4249826913 wait complete. 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.371+0000 7f245b200640 1 Processor -- start 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.371+0000 7f245b200640 1 -- start start 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24400039b0 0x7f245c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f245c15be10 0x7f245c14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0d12a0 con 0x7f245c14e1d0 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f245c15fd00 con 0x7f245c15be10 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.372+0000 7f245b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f245c0d1420 con 0x7f24400039b0 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57208/0 (socket says 172.21.15.59:57208) 2024-04-24T06:38:22.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 -- 172.21.15.59:0/3395711079 learned_addr learned my addr 172.21.15.59:0/3395711079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f2459e00640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24400039b0 0x7f245c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f2451400640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f245c15be10 0x7f245c14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 -- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24400039b0 msgr2=0x7f245c15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24400039b0 0x7f245c15b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 -- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f245c15be10 msgr2=0x7f245c14b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f245c15be10 0x7f245c14b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 -- 172.21.15.59:0/3395711079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0c96f0 con 0x7f245c14e1d0 2024-04-24T06:38:22.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.373+0000 7f245a800640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f245c0741a0 tx=0x7f245400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.374+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454004d60 con 0x7f245c14e1d0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.374+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245400bb80 con 0x7f245c14e1d0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.374+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454007110 con 0x7f245c14e1d0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.374+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c1f7af0 con 0x7f245c14e1d0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.374+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c0c99d0 con 0x7f245c14e1d0 2024-04-24T06:38:22.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.375+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c04fd90 con 0x7f245c14e1d0 2024-04-24T06:38:22.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.376+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f245401b020 con 0x7f245c14e1d0 2024-04-24T06:38:22.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.377+0000 7f2452a00640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 0x7f243806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.377+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f245408be10 con 0x7f245c14e1d0 2024-04-24T06:38:22.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.377+0000 7f2459e00640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 0x7f243806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.378+0000 7f2459e00640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 0x7f243806edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f244c032000 tx=0x7f244c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.382+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f245408f030 con 0x7f245c14e1d0 2024-04-24T06:38:22.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.618+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f245c06ead0 con 0x7f245c14e1d0 2024-04-24T06:38:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.619+0000 7f2452a00640 1 -- 172.21.15.59:0/3395711079 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7f245405a160 con 0x7f245c14e1d0 2024-04-24T06:38:22.620 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:22.620 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:38:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:38:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.621+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 msgr2=0x7f243806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.621+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 0x7f243806edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f244c032000 tx=0x7f244c02e000 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.621+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 msgr2=0x7f245c14baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.621+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f245c0741a0 tx=0x7f245400b4a0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 shutdown_connections 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f243806c8f0 0x7f243806edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245c14e1d0 0x7f245c14baf0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f245c15be10 0x7f245c14b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 --2- 172.21.15.59:0/3395711079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f24400039b0 0x7f245c15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 >> 172.21.15.59:0/3395711079 conn(0x7f245c08bf90 msgr2=0x7f245c161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 shutdown_connections 2024-04-24T06:38:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.622+0000 7f245b200640 1 -- 172.21.15.59:0/3395711079 wait complete. 2024-04-24T06:38:22.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.850+0000 7f27dfe00640 1 Processor -- start 2024-04-24T06:38:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.850+0000 7f27dfe00640 1 -- start start 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27dfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27e00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27dfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e00b3870 con 0x7f27e00cd760 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27dfe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e00b3b50 con 0x7f27e006f3f0 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27dfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27e00b41a0 con 0x7f27e015b450 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27df400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f27e00cd760 0x7f27e00ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37222/0 (socket says 172.21.15.59:37222) 2024-04-24T06:38:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27df400640 1 -- 172.21.15.59:0/1354356390 learned_addr learned my addr 172.21.15.59:0/1354356390 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:22.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.852+0000 7f27dea00640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27e00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497941827 0 0) 0x7f27e00b3870 con 0x7f27e00cd760 2024-04-24T06:38:22.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27c4003520 con 0x7f27e00cd760 2024-04-24T06:38:22.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1674703063 0 0) 0x7f27c4003520 con 0x7f27e00cd760 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 msgr2=0x7f27e00b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27e00b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 >> v1:172.21.15.66:6789/0 conn(0x7f27e006f3f0 legacy=0x7f27e00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e00b3e30 con 0x7f27e00cd760 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f27d80019f0 con 0x7f27e00cd760 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 >> v1:172.21.15.59:6789/0 conn(0x7f27e00cd760 legacy=0x7f27e00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.853+0000 7f27de000640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27de000640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27c4003fb0 0x7f27c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27de000640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27c4004d40 0x7f27c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27e00b41a0 con 0x7f27c4004d40 2024-04-24T06:38:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27c400bb00 con 0x7f27c4003fb0 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27c400df10 con 0x7f27e015b450 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27c4004d40 0x7f27c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27dea00640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27d7e00640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27c4003fb0 0x7f27c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 -- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 msgr2=0x7f27c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e015b450 0x7f27c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 -- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27c4003fb0 msgr2=0x7f27c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27c4003fb0 0x7f27c4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.854+0000 7f27df400640 1 -- 172.21.15.59:0/1354356390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c4003520 con 0x7f27c4004d40 2024-04-24T06:38:22.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.855+0000 7f27df400640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27c4004d40 0x7f27c40051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f27d800a770 tx=0x7f27c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fce5888a8f4c96a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d800a390 con 0x7f27c4004d40 2024-04-24T06:38:22.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d802f070 con 0x7f27c4004d40 2024-04-24T06:38:22.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27de000640 1 -- 172.21.15.59:0/1354356390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d802bc20 con 0x7f27c4004d40 2024-04-24T06:38:22.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27dfe00640 1 -- 172.21.15.59:0/1354356390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27c4004d40 msgr2=0x7f27c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27dfe00640 1 --2- 172.21.15.59:0/1354356390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27c4004d40 0x7f27c40051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f27d800a770 tx=0x7f27c4005830 comp rx=0 tx=0).stop 2024-04-24T06:38:22.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27df400640 1 -- 172.21.15.59:0/1354356390 reap_dead start 2024-04-24T06:38:22.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27dfe00640 1 -- 172.21.15.59:0/1354356390 shutdown_connections 2024-04-24T06:38:22.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.856+0000 7f27dfe00640 1 -- 172.21.15.59:0/1354356390 >> 172.21.15.59:0/1354356390 conn(0x7f27e00b21a0 msgr2=0x7f27e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:22.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.857+0000 7f27dfe00640 1 -- 172.21.15.59:0/1354356390 shutdown_connections 2024-04-24T06:38:22.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.857+0000 7f27dfe00640 1 -- 172.21.15.59:0/1354356390 wait complete. 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.859+0000 7f27dfe00640 1 Processor -- start 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.859+0000 7f27dfe00640 1 -- start start 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 0x7f27e00c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e006c020 0x7f27e00c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27e00b3080 con 0x7f27e009d580 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27e00b4ff0 con 0x7f27e006bc40 2024-04-24T06:38:22.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27dfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27e00b3200 con 0x7f27e006c020 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57230/0 (socket says 172.21.15.59:57230) 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27df400640 1 -- 172.21.15.59:0/3457411929 learned_addr learned my addr 172.21.15.59:0/3457411929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.860+0000 7f27d7e00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e006c020 0x7f27e00c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 -- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e006c020 msgr2=0x7f27e00c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e006c020 0x7f27e00c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27dea00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 0x7f27e00c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 -- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 msgr2=0x7f27e00c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 0x7f27e00c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 -- 172.21.15.59:0/3457411929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e009df40 con 0x7f27e009d580 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27dea00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 0x7f27e00c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27df400640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f27d800a890 tx=0x7f27e00b4110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8045070 con 0x7f27e009d580 2024-04-24T06:38:22.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d802bc40 con 0x7f27e009d580 2024-04-24T06:38:22.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8001ee0 con 0x7f27e009d580 2024-04-24T06:38:22.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.861+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e00c4950 con 0x7f27e009d580 2024-04-24T06:38:22.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.862+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e00c4f10 con 0x7f27e009d580 2024-04-24T06:38:22.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.863+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e004fd90 con 0x7f27e009d580 2024-04-24T06:38:22.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.864+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27d8048070 con 0x7f27e009d580 2024-04-24T06:38:22.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.865+0000 7f27d6a00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 0x7f27bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:22.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.865+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f27d80aba20 con 0x7f27e009d580 2024-04-24T06:38:22.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.865+0000 7f27dea00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 0x7f27bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:22.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.866+0000 7f27dea00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 0x7f27bc06ede0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009a00 tx=0x7f27d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:22.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:22.869+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27e009df40 con 0x7f27e009d580 2024-04-24T06:38:23.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.074+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27e00b47e0 con 0x7f27e009d580 2024-04-24T06:38:23.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.078+0000 7f27d6a00640 1 -- 172.21.15.59:0/3457411929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16883 (secure 0 0 0) 0x7f27d8079e10 con 0x7f27e009d580 2024-04-24T06:38:23.079 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:23.079 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:37:35.076193+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.59:6834/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:23.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.080+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 msgr2=0x7f27bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.080+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 0x7f27bc06ede0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009a00 tx=0x7f27d0008040 comp rx=0 tx=0).stop 2024-04-24T06:38:23.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.080+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 msgr2=0x7f27e009da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.080+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f27d800a890 tx=0x7f27e00b4110 comp rx=0 tx=0).stop 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 shutdown_connections 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27bc06c920 0x7f27bc06ede0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e009d580 0x7f27e009da00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27e006c020 0x7f27e00c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 --2- 172.21.15.59:0/3457411929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27e006bc40 0x7f27e00c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 >> 172.21.15.59:0/3457411929 conn(0x7f27e00b21a0 msgr2=0x7f27e00ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:23.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 shutdown_connections 2024-04-24T06:38:23.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.081+0000 7f27dfe00640 1 -- 172.21.15.59:0/3457411929 wait complete. 2024-04-24T06:38:23.102 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:38:23.103 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:23.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.314+0000 7fcaf1800640 1 Processor -- start 2024-04-24T06:38:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.315+0000 7fcaf1800640 1 -- start start 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcaf1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec15e7c0 0x7fcaec0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcaf1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaec0d2630 0x7fcaec0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcaf1800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0cf030 con 0x7fcaec0b6530 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcaf1800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0b7000 con 0x7fcaec0d2630 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcaf1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0cf620 con 0x7fcaec15e7c0 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.316+0000 7fcae3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaec0d2630 0x7fcaec0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaec0d2630 0x7fcaec0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57236/0 (socket says 172.21.15.59:57236) 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 -- 172.21.15.59:0/2124595475 learned_addr learned my addr 172.21.15.59:0/2124595475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcaebe00640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec15e7c0 0x7fcaec0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 -- 172.21.15.59:0/2124595475 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec15e7c0 msgr2=0x7fcaec0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec15e7c0 0x7fcaec0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 -- 172.21.15.59:0/2124595475 >> v1:172.21.15.66:6789/0 conn(0x7fcaec0b6530 legacy=0x7fcaec0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.317+0000 7fcae3400640 1 -- 172.21.15.59:0/2124595475 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0b6a40 con 0x7fcaec0d2630 2024-04-24T06:38:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.318+0000 7fcae3400640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec0d2630 0x7fcaec0b5ff0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009870 tx=0x7fcad402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed58ff18a6470c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaeb400640 1 -- 172.21.15.59:0/2124595475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4037070 con 0x7fcaec0d2630 2024-04-24T06:38:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaeb400640 1 -- 172.21.15.59:0/2124595475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4032440 con 0x7fcaec0d2630 2024-04-24T06:38:23.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaeb400640 1 -- 172.21.15.59:0/2124595475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4032810 con 0x7fcaec0d2630 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 -- 172.21.15.59:0/2124595475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec0d2630 msgr2=0x7fcaec0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec0d2630 0x7fcaec0b5ff0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009870 tx=0x7fcad402a210 comp rx=0 tx=0).stop 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 -- 172.21.15.59:0/2124595475 shutdown_connections 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec0d2630 0x7fcaec0b5ff0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 --2- 172.21.15.59:0/2124595475 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec15e7c0 0x7fcaec0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.319+0000 7fcaf1800640 1 -- 172.21.15.59:0/2124595475 >> 172.21.15.59:0/2124595475 conn(0x7fcaec09d920 msgr2=0x7fcaec08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:23.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.320+0000 7fcaf1800640 1 -- 172.21.15.59:0/2124595475 shutdown_connections 2024-04-24T06:38:23.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.320+0000 7fcaf1800640 1 -- 172.21.15.59:0/2124595475 wait complete. 2024-04-24T06:38:23.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.321+0000 7fcaf1800640 1 Processor -- start 2024-04-24T06:38:23.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.321+0000 7fcaf1800640 1 -- start start 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaec0b6530 0x7fcaec14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec0d2630 0x7fcaec0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0d1a80 con 0x7fcaec15e7c0 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0cf2b0 con 0x7fcaec0b6530 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.322+0000 7fcaf1800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0d1c00 con 0x7fcaec0d2630 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57246/0 (socket says 172.21.15.59:57246) 2024-04-24T06:38:23.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 -- 172.21.15.59:0/283999145 learned_addr learned my addr 172.21.15.59:0/283999145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcae3400640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec0d2630 0x7fcaec0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaebe00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaec0b6530 0x7fcaec14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 -- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec0d2630 msgr2=0x7fcaec0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec0d2630 0x7fcaec0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 -- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaec0b6530 msgr2=0x7fcaec14fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaec0b6530 0x7fcaec14fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.323+0000 7fcaf0e00640 1 -- 172.21.15.59:0/283999145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec14ff40 con 0x7fcaec15e7c0 2024-04-24T06:38:23.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcaf0e00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c730 tx=0x7fcae400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4016020 con 0x7fcaec15e7c0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae400aa50 con 0x7fcaec15e7c0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae400ce40 con 0x7fcaec15e7c0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec0b17e0 con 0x7fcaec15e7c0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.324+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec0b1da0 con 0x7fcaec15e7c0 2024-04-24T06:38:23.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.325+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec04fd90 con 0x7fcaec15e7c0 2024-04-24T06:38:23.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.326+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcae4019020 con 0x7fcaec15e7c0 2024-04-24T06:38:23.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.327+0000 7fcae8c00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.327+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fcae408bc50 con 0x7fcaec15e7c0 2024-04-24T06:38:23.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.328+0000 7fcaebe00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.328+0000 7fcaebe00640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 0x7fcac806edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.332+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcae4020030 con 0x7fcaec15e7c0 2024-04-24T06:38:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.575+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaec0d1eb0 con 0x7fcaec15e7c0 2024-04-24T06:38:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.576+0000 7fcae8c00640 1 -- 172.21.15.59:0/283999145 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7fcaec14ff40 con 0x7fcaec15e7c0 2024-04-24T06:38:23.577 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:23.577 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:38:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:38:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.578+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 msgr2=0x7fcac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.578+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 0x7fcac806edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2024-04-24T06:38:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.578+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 msgr2=0x7fcaec0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.578+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c730 tx=0x7fcae400cbf0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.578+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 shutdown_connections 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaec15e7c0 0x7fcaec0cbc70 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaec0d2630 0x7fcaec0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 --2- 172.21.15.59:0/283999145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaec0b6530 0x7fcaec14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 >> 172.21.15.59:0/283999145 conn(0x7fcaec09d920 msgr2=0x7fcaec0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 shutdown_connections 2024-04-24T06:38:23.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.579+0000 7fcaf1800640 1 -- 172.21.15.59:0/283999145 wait complete. 2024-04-24T06:38:23.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:23.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.803+0000 7f8975400640 1 Processor -- start 2024-04-24T06:38:23.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.803+0000 7f8975400640 1 -- start start 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f8975400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f89700cd020 0x7f89700b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f8975400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f8970160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f8975400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89700cd520 con 0x7f89701454e0 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f8975400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f89700cddc0 con 0x7f89700cd020 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f8975400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f89701462c0 con 0x7f89700b3e40 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f896e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89701454e0 0x7f8970145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37224/0 (socket says 172.21.15.59:37224) 2024-04-24T06:38:23.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.805+0000 7f896e800640 1 -- 172.21.15.59:0/1905039500 learned_addr learned my addr 172.21.15.59:0/1905039500 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:23.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896de00640 1 --2- 172.21.15.59:0/1905039500 >> v2:172.21.15.66:3300/0 conn(0x7f89700cd020 0x7f89700b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896f200640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f8970160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370062095 0 0) 0x7f89700cd520 con 0x7f89701454e0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8954003520 con 0x7f89701454e0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474601677 0 0) 0x7f8954003520 con 0x7f89701454e0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 msgr2=0x7f8970160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f8970160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 >> v2:172.21.15.66:3300/0 conn(0x7f89700cd020 msgr2=0x7f89700b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 --2- 172.21.15.59:0/1905039500 >> v2:172.21.15.66:3300/0 conn(0x7f89700cd020 0x7f89700b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.806+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89700cd800 con 0x7f89701454e0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8960001670 con 0x7f89701454e0 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 >> v1:172.21.15.59:6789/0 conn(0x7f89701454e0 legacy=0x7f8970145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89540039b0 0x7f8954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f89540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f89700cddc0 con 0x7f8954004800 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701462c0 con 0x7f89540039b0 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f895400b9c0 con 0x7f89700b3e40 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.807+0000 7f896de00640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896e800640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89540039b0 0x7f8954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896f200640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f89540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896de00640 1 -- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 msgr2=0x7f89540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896de00640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f89540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896de00640 1 -- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89540039b0 msgr2=0x7f8954003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896de00640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89540039b0 0x7f8954003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896de00640 1 -- 172.21.15.59:0/1905039500 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8954003520 con 0x7f8954004800 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896e800640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89540039b0 0x7f8954003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:23.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.808+0000 7f896f200640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b3e40 0x7f89540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.809+0000 7f896de00640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8954004c80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f895c009af0 tx=0x7f8954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebac64f0d780448b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.809+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c036070 con 0x7f8954004800 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.809+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c031400 con 0x7f8954004800 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.809+0000 7f896d400640 1 -- 172.21.15.59:0/1905039500 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c009690 con 0x7f8954004800 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.810+0000 7f8975400640 1 -- 172.21.15.59:0/1905039500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 msgr2=0x7f8954004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.810+0000 7f8975400640 1 --2- 172.21.15.59:0/1905039500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8954004c80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f895c009af0 tx=0x7f8954005830 comp rx=0 tx=0).stop 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.810+0000 7f896f200640 1 -- 172.21.15.59:0/1905039500 reap_dead start 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.810+0000 7f8975400640 1 -- 172.21.15.59:0/1905039500 shutdown_connections 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.810+0000 7f8975400640 1 -- 172.21.15.59:0/1905039500 >> 172.21.15.59:0/1905039500 conn(0x7f8970067ea0 msgr2=0x7f897006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.811+0000 7f8975400640 1 -- 172.21.15.59:0/1905039500 shutdown_connections 2024-04-24T06:38:23.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.811+0000 7f8975400640 1 -- 172.21.15.59:0/1905039500 wait complete. 2024-04-24T06:38:23.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.813+0000 7f8975400640 1 Processor -- start 2024-04-24T06:38:23.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.813+0000 7f8975400640 1 -- start start 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.814+0000 7f8975400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f8975400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 0x7f8970142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f8975400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 0x7f89700b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f8975400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8970160910 con 0x7f8954004800 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f8975400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89701603f0 con 0x7f897015f6c0 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f8975400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8970160a90 con 0x7f89700b0310 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57272/0 (socket says 172.21.15.59:57272) 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 -- 172.21.15.59:0/3560813652 learned_addr learned my addr 172.21.15.59:0/3560813652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896f200640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 0x7f89700b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896de00640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 0x7f8970142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 -- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 msgr2=0x7f89700b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 0x7f89700b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 -- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 msgr2=0x7f8970142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 0x7f8970142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896e800640 1 -- 172.21.15.59:0/3560813652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8970132e90 con 0x7f8954004800 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896de00640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 0x7f8970142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:23.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.815+0000 7f896f200640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 0x7f89700b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f896e800640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f896000d8e0 tx=0x7f896000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960019070 con 0x7f8954004800 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89600074e0 con 0x7f8954004800 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600049e0 con 0x7f8954004800 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8970141ac0 con 0x7f8954004800 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.816+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8970142080 con 0x7f8954004800 2024-04-24T06:38:23.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.818+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89700b0cd0 con 0x7f8954004800 2024-04-24T06:38:23.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.819+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f896001c070 con 0x7f8954004800 2024-04-24T06:38:23.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.819+0000 7f8966a00640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 0x7f895006ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:23.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.820+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f896008b780 con 0x7f8954004800 2024-04-24T06:38:23.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.820+0000 7f896de00640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 0x7f895006ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:23.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.821+0000 7f896de00640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 0x7f895006ecf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f895c033000 tx=0x7f895c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:23.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:23.824+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8960059a40 con 0x7f8954004800 2024-04-24T06:38:24.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.071+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8970160030 con 0x7f8954004800 2024-04-24T06:38:24.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.072+0000 7f8966a00640 1 -- 172.21.15.59:0/3560813652 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5181 (secure 0 0 0) 0x7f8960059340 con 0x7f8954004800 2024-04-24T06:38:24.074 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:24.074 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:35:54.320156+0000","modified":"2024-04-24T06:37:40.197960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/3165421940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3165421940},{"type":"v1","addr":"172.21.15.59:6835","nonce":3165421940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-24T06:38:24.074 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 75 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 msgr2=0x7f895006ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 0x7f895006ecf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f895c033000 tx=0x7f895c008030 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 msgr2=0x7f8970132950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f896000d8e0 tx=0x7f896000dda0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 shutdown_connections 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f895006c830 0x7f895006ecf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f89700b0310 0x7f89700b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f897015f6c0 0x7f8970142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 --2- 172.21.15.59:0/3560813652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8954004800 0x7f8970132950 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 >> 172.21.15.59:0/3560813652 conn(0x7f8970067ea0 msgr2=0x7f89700d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:24.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 shutdown_connections 2024-04-24T06:38:24.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.075+0000 7f8975400640 1 -- 172.21.15.59:0/3560813652 wait complete. 2024-04-24T06:38:24.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.306+0000 7ff9bee00640 1 Processor -- start 2024-04-24T06:38:24.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.306+0000 7ff9bee00640 1 -- start start 2024-04-24T06:38:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.307+0000 7ff9bee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9b815e7c0 0x7ff9b80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.307+0000 7ff9bee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.307+0000 7ff9bee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b80cf030 con 0x7ff9b80d2630 2024-04-24T06:38:24.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.307+0000 7ff9bee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b80d1740 con 0x7ff9b815e7c0 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.307+0000 7ff9bee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80b7000 con 0x7ff9b80d0d90 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9b815e7c0 0x7ff9b80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9b815e7c0 0x7ff9b80d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57284/0 (socket says 172.21.15.59:57284) 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 -- 172.21.15.59:0/250819156 learned_addr learned my addr 172.21.15.59:0/250819156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 -- 172.21.15.59:0/250819156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 msgr2=0x7ff9b80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 --2- 172.21.15.59:0/250819156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 -- 172.21.15.59:0/250819156 >> v1:172.21.15.66:6789/0 conn(0x7ff9b80d2630 legacy=0x7ff9b80b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.308+0000 7ff9bda00640 1 -- 172.21.15.59:0/250819156 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80b6a40 con 0x7ff9b815e7c0 2024-04-24T06:38:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.309+0000 7ff9bda00640 1 --2- 172.21.15.59:0/250819156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80d20f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009870 tx=0x7ff9a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75226eea7ab959a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9b7e00640 1 -- 172.21.15.59:0/250819156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8037070 con 0x7ff9b815e7c0 2024-04-24T06:38:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9b7e00640 1 -- 172.21.15.59:0/250819156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8032440 con 0x7ff9b815e7c0 2024-04-24T06:38:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9b7e00640 1 -- 172.21.15.59:0/250819156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8032810 con 0x7ff9b815e7c0 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 -- 172.21.15.59:0/250819156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b815e7c0 msgr2=0x7ff9b80d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 --2- 172.21.15.59:0/250819156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80d20f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009870 tx=0x7ff9a802a210 comp rx=0 tx=0).stop 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 -- 172.21.15.59:0/250819156 shutdown_connections 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 --2- 172.21.15.59:0/250819156 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 --2- 172.21.15.59:0/250819156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80d20f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 -- 172.21.15.59:0/250819156 >> 172.21.15.59:0/250819156 conn(0x7ff9b809d920 msgr2=0x7ff9b808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 -- 172.21.15.59:0/250819156 shutdown_connections 2024-04-24T06:38:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.310+0000 7ff9bee00640 1 -- 172.21.15.59:0/250819156 wait complete. 2024-04-24T06:38:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.312+0000 7ff9bee00640 1 Processor -- start 2024-04-24T06:38:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.312+0000 7ff9bee00640 1 -- start start 2024-04-24T06:38:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80cf2b0 con 0x7ff9b80d2630 2024-04-24T06:38:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80cf620 con 0x7ff9b815e7c0 2024-04-24T06:38:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80b6700 con 0x7ff9b80d0d90 2024-04-24T06:38:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.313+0000 7ff9bd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57296/0 (socket says 172.21.15.59:57296) 2024-04-24T06:38:24.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 -- 172.21.15.59:0/2838289019 learned_addr learned my addr 172.21.15.59:0/2838289019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:24.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9be400640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bda00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 -- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 msgr2=0x7ff9b815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b815d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 -- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 msgr2=0x7ff9b80b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 -- 172.21.15.59:0/2838289019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80ca620 con 0x7ff9b80d2630 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9be400640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.314+0000 7ff9bd000640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c7b0 tx=0x7ff9a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.315+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4016020 con 0x7ff9b80d2630 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.315+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a400aa50 con 0x7ff9b80d2630 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.315+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400ae40 con 0x7ff9b80d2630 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.315+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b80ccc30 con 0x7ff9b80d2630 2024-04-24T06:38:24.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.315+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b80cd1f0 con 0x7ff9b80d2630 2024-04-24T06:38:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.316+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b804fd90 con 0x7ff9b80d2630 2024-04-24T06:38:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.317+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9a4019020 con 0x7ff9b80d2630 2024-04-24T06:38:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.318+0000 7ff9b5600640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.318+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7ff9a408bd10 con 0x7ff9b80d2630 2024-04-24T06:38:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.318+0000 7ff9bda00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.319+0000 7ff9bda00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 0x7ff99806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8034000 tx=0x7ff9a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.323+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9a4091020 con 0x7ff9b80d2630 2024-04-24T06:38:24.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.533+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b806cab0 con 0x7ff9b80d2630 2024-04-24T06:38:24.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.535+0000 7ff9b5600640 1 -- 172.21.15.59:0/2838289019 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16883 (secure 0 0 0) 0x7ff9a4020030 con 0x7ff9b80d2630 2024-04-24T06:38:24.538 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:24.538 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:37:35.076193+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:35:52.188853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-24T06:35:53.282826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.10","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.59:6834/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:24.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 msgr2=0x7ff99806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 0x7ff99806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8034000 tx=0x7ff9a802e000 comp rx=0 tx=0).stop 2024-04-24T06:38:24.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 msgr2=0x7ff9b815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c7b0 tx=0x7ff9a400cc70 comp rx=0 tx=0).stop 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 shutdown_connections 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff99806c8f0 0x7ff99806edb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9b815e7c0 0x7ff9b80b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9b80d2630 0x7ff9b815c690 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 --2- 172.21.15.59:0/2838289019 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff9b80d0d90 0x7ff9b815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 >> 172.21.15.59:0/2838289019 conn(0x7ff9b809d920 msgr2=0x7ff9b80d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 shutdown_connections 2024-04-24T06:38:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.538+0000 7ff9bee00640 1 -- 172.21.15.59:0/2838289019 wait complete. 2024-04-24T06:38:24.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:38:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.767+0000 7f355f600640 1 Processor -- start 2024-04-24T06:38:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.767+0000 7f355f600640 1 -- start start 2024-04-24T06:38:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.769+0000 7f355f600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f35580b44c0 0x7f35580aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.769+0000 7f355f600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3558162de0 0x7f355806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.769+0000 7f355f600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35581632e0 con 0x7f3558161e60 2024-04-24T06:38:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.769+0000 7f355f600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f35580aefd0 con 0x7f35580b44c0 2024-04-24T06:38:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.769+0000 7f355f600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f35580adcd0 con 0x7f3558162de0 2024-04-24T06:38:24.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f35580b44c0 0x7f35580aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f355d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3558162de0 0x7f355806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f35580b44c0 0x7f35580aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:37556/0 (socket says 172.21.15.59:37556) 2024-04-24T06:38:24.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 -- 172.21.15.59:0/1200467323 learned_addr learned my addr 172.21.15.59:0/1200467323 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:24.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 -- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3558162de0 msgr2=0x7f355806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:24.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 --2- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3558162de0 0x7f355806ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.770+0000 7f3557e00640 1 -- 172.21.15.59:0/1200467323 >> v1:172.21.15.66:6789/0 conn(0x7f3558161e60 legacy=0x7f35580b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.771+0000 7f3557e00640 1 -- 172.21.15.59:0/1200467323 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580adfb0 con 0x7f35580b44c0 2024-04-24T06:38:24.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.772+0000 7f3557e00640 1 --2- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35580b44c0 0x7f35580aea90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3544009870 tx=0x7f354402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c94977e4ca683e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.772+0000 7f3557400640 1 -- 172.21.15.59:0/1200467323 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544037070 con 0x7f35580b44c0 2024-04-24T06:38:24.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.772+0000 7f3557400640 1 -- 172.21.15.59:0/1200467323 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3544032440 con 0x7f35580b44c0 2024-04-24T06:38:24.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.772+0000 7f3557400640 1 -- 172.21.15.59:0/1200467323 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544032810 con 0x7f35580b44c0 2024-04-24T06:38:24.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 -- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35580b44c0 msgr2=0x7f35580aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 --2- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35580b44c0 0x7f35580aea90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3544009870 tx=0x7f354402a1c0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 -- 172.21.15.59:0/1200467323 shutdown_connections 2024-04-24T06:38:24.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 --2- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3558162de0 0x7f355806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 --2- 172.21.15.59:0/1200467323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35580b44c0 0x7f35580aea90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 -- 172.21.15.59:0/1200467323 >> 172.21.15.59:0/1200467323 conn(0x7f35580b2130 msgr2=0x7f35580b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:24.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 -- 172.21.15.59:0/1200467323 shutdown_connections 2024-04-24T06:38:24.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.773+0000 7f355f600640 1 -- 172.21.15.59:0/1200467323 wait complete. 2024-04-24T06:38:24.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.775+0000 7f355f600640 1 Processor -- start 2024-04-24T06:38:24.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.775+0000 7f355f600640 1 -- start start 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35580b44c0 0x7f355815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 0x7f355814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35580af480 con 0x7f3558161e60 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35580ae570 con 0x7f3558162de0 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f355f600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f35580af600 con 0x7f35580b44c0 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f3557e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f3557e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57298/0 (socket says 172.21.15.59:57298) 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.776+0000 7f3557e00640 1 -- 172.21.15.59:0/1292779599 learned_addr learned my addr 172.21.15.59:0/1292779599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f355d600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 0x7f355814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 -- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35580b44c0 msgr2=0x7f355815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35580b44c0 0x7f355815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 -- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 msgr2=0x7f355814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 0x7f355814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 -- 172.21.15.59:0/1292779599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355815e740 con 0x7f3558161e60 2024-04-24T06:38:24.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f355d600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 0x7f355814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:24.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3557e00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f35440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.777+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544037040 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.778+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3544042070 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.778+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544030880 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.778+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355815ea20 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.778+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3558156590 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.779+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35580af7e0 con 0x7f3558161e60 2024-04-24T06:38:24.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.780+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f354402ad10 con 0x7f3558161e60 2024-04-24T06:38:24.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.781+0000 7f3554c00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 0x7f353806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.781+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f35440ac100 con 0x7f3558161e60 2024-04-24T06:38:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.781+0000 7f355cc00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 0x7f353806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.782+0000 7f355cc00640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 0x7f353806ede0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f354c000f60 tx=0x7f354c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:24.786+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35580af7e0 con 0x7f3558161e60 2024-04-24T06:38:25.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.030+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f35580b4f10 con 0x7f3558161e60 2024-04-24T06:38:25.032 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:38:25.031+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:38:25.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.034+0000 7f3554c00640 1 -- 172.21.15.59:0/1292779599 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f35580b4f10 con 0x7f3558161e60 2024-04-24T06:38:25.036 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:38:25.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.037+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 msgr2=0x7f353806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.037+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 0x7f353806ede0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f354c000f60 tx=0x7f354c0079d0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.037+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 msgr2=0x7f355814ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.037+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f35440046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.037+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 shutdown_connections 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f353806c920 0x7f353806ede0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3558162de0 0x7f355814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3558161e60 0x7f355814ce10 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 --2- 172.21.15.59:0/1292779599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35580b44c0 0x7f355815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 >> 172.21.15.59:0/1292779599 conn(0x7f35580b2130 msgr2=0x7f355806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 shutdown_connections 2024-04-24T06:38:25.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.038+0000 7f355f600640 1 -- 172.21.15.59:0/1292779599 wait complete. 2024-04-24T06:38:25.050 INFO:tasks.ceph.mds.c.smithi059.stderr: -2515> 2024-04-24T06:37:41.173+0000 7f271bc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:38:25.052 INFO:tasks.ceph.mds.c.smithi059.stderr: -2037> 2024-04-24T06:37:44.743+0000 7f2716c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x555c9de0a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T06:38:25.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:38:25.108 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:38:25.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.248+0000 7fcc23200640 1 Processor -- start 2024-04-24T06:38:25.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.248+0000 7fcc23200640 1 -- start start 2024-04-24T06:38:25.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.249+0000 7fcc23200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.249+0000 7fcc23200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcc240d05d0 0x7fcc240d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.249+0000 7fcc23200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2406fd30 con 0x7fcc2406f8d0 2024-04-24T06:38:25.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.249+0000 7fcc23200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcc240d0f60 con 0x7fcc240d05d0 2024-04-24T06:38:25.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.249+0000 7fcc23200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcc240ae6b0 con 0x7fcc2415b220 2024-04-24T06:38:25.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.250+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.250+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:49226/0 (socket says 172.21.15.59:49226) 2024-04-24T06:38:25.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.250+0000 7fcc21e00640 1 -- 172.21.15.59:0/1590193619 learned_addr learned my addr 172.21.15.59:0/1590193619 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:25.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.250+0000 7fcc22800640 1 --2- 172.21.15.59:0/1590193619 >> v2:172.21.15.66:3300/0 conn(0x7fcc240d05d0 0x7fcc240d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.251+0000 7fcc21e00640 1 -- 172.21.15.59:0/1590193619 >> v2:172.21.15.66:3300/0 conn(0x7fcc240d05d0 msgr2=0x7fcc240d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.251+0000 7fcc21e00640 1 --2- 172.21.15.59:0/1590193619 >> v2:172.21.15.66:3300/0 conn(0x7fcc240d05d0 0x7fcc240d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.251+0000 7fcc21e00640 1 -- 172.21.15.59:0/1590193619 >> v1:172.21.15.66:6789/0 conn(0x7fcc2406f8d0 legacy=0x7fcc240adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.251+0000 7fcc21e00640 1 -- 172.21.15.59:0/1590193619 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc240ae990 con 0x7fcc2415b220 2024-04-24T06:38:25.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.252+0000 7fcc21e00640 1 --2- 172.21.15.59:0/1590193619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc1402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee1323a89cfd91b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:25.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.253+0000 7fcc20a00640 1 -- 172.21.15.59:0/1590193619 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14037070 con 0x7fcc2415b220 2024-04-24T06:38:25.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.253+0000 7fcc20a00640 1 -- 172.21.15.59:0/1590193619 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc14032440 con 0x7fcc2415b220 2024-04-24T06:38:25.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.253+0000 7fcc20a00640 1 -- 172.21.15.59:0/1590193619 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14032810 con 0x7fcc2415b220 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 -- 172.21.15.59:0/1590193619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 msgr2=0x7fcc240af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 --2- 172.21.15.59:0/1590193619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc1402a270 comp rx=0 tx=0).stop 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 -- 172.21.15.59:0/1590193619 shutdown_connections 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 --2- 172.21.15.59:0/1590193619 >> v2:172.21.15.66:3300/0 conn(0x7fcc240d05d0 0x7fcc240d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 --2- 172.21.15.59:0/1590193619 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc2415b220 0x7fcc240af750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 -- 172.21.15.59:0/1590193619 >> 172.21.15.59:0/1590193619 conn(0x7fcc2406bdf0 msgr2=0x7fcc2406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 -- 172.21.15.59:0/1590193619 shutdown_connections 2024-04-24T06:38:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.254+0000 7fcc23200640 1 -- 172.21.15.59:0/1590193619 wait complete. 2024-04-24T06:38:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.256+0000 7fcc23200640 1 Processor -- start 2024-04-24T06:38:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.256+0000 7fcc23200640 1 -- start start 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc240d05d0 0x7fcc241529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc2415b220 0x7fcc241f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc240af230 con 0x7fcc2406f8d0 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcc240aef50 con 0x7fcc2415b220 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc23200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2415b600 con 0x7fcc240d05d0 2024-04-24T06:38:25.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57328/0 (socket says 172.21.15.59:57328) 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 -- 172.21.15.59:0/2847069970 learned_addr learned my addr 172.21.15.59:0/2847069970 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21400640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc240d05d0 0x7fcc241529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc22800640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc2415b220 0x7fcc241f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 -- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc240d05d0 msgr2=0x7fcc241529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc240d05d0 0x7fcc241529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.258+0000 7fcc21e00640 1 -- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc2415b220 msgr2=0x7fcc241f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc21e00640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc2415b220 0x7fcc241f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc21e00640 1 -- 172.21.15.59:0/2847069970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc240afaa0 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc21e00640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcc1402a2b0 tx=0x7fcc140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14041030 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc14030440 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14037040 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc240afd80 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.259+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc240b0340 con 0x7fcc2406f8d0 2024-04-24T06:38:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.260+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2404fd90 con 0x7fcc2406f8d0 2024-04-24T06:38:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.261+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc14044070 con 0x7fcc2406f8d0 2024-04-24T06:38:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.262+0000 7fcc1a000640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 0x7fcc0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.262+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5996+0+0 (secure 0 0 0) 0x7fcc140ac120 con 0x7fcc2406f8d0 2024-04-24T06:38:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.263+0000 7fcc21400640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 0x7fcc0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.263+0000 7fcc21400640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 0x7fcc0006edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcc24168d60 tx=0x7fcc0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:25.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.267+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc1404b070 con 0x7fcc2406f8d0 2024-04-24T06:38:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:25.502+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc240b43b0 con 0x7fcc2406f8d0 2024-04-24T06:38:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.038+0000 7fcc1a000640 1 -- 172.21.15.59:0/2847069970 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7fcc1407a390 con 0x7fcc2406f8d0 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.042+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 msgr2=0x7fcc0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.042+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 0x7fcc0006edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcc24168d60 tx=0x7fcc0c009040 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.042+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 msgr2=0x7fcc24099ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.042+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcc1402a2b0 tx=0x7fcc140046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 shutdown_connections 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcc0006c8f0 0x7fcc0006edb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc2415b220 0x7fcc241f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcc240d05d0 0x7fcc241529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 --2- 172.21.15.59:0/2847069970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc2406f8d0 0x7fcc24099ae0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 >> 172.21.15.59:0/2847069970 conn(0x7fcc2406bdf0 msgr2=0x7fcc24071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:26.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 shutdown_connections 2024-04-24T06:38:26.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.043+0000 7fcc23200640 1 -- 172.21.15.59:0/2847069970 wait complete. 2024-04-24T06:38:26.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:38:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.332+0000 7fb915c00640 1 Processor -- start 2024-04-24T06:38:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.332+0000 7fb915c00640 1 -- start start 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb915c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb9100af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb915c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb9100ccb40 0x7fb9100ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb915c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9100cd4c0 con 0x7fb910162940 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb915c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb910070060 con 0x7fb9100ccb40 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb915c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb910073170 con 0x7fb910162d20 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb90fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb910162940 0x7fb91006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37228/0 (socket says 172.21.15.59:37228) 2024-04-24T06:38:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.334+0000 7fb90fe00640 1 -- 172.21.15.59:0/2703309544 learned_addr learned my addr 172.21.15.59:0/2703309544 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb907e00640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb9100af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb915200640 1 --2- 172.21.15.59:0/2703309544 >> v2:172.21.15.66:3300/0 conn(0x7fb9100ccb40 0x7fb9100ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667428831 0 0) 0x7fb9100cd4c0 con 0x7fb910162940 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8f4003520 con 0x7fb910162940 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 834743994 0 0) 0x7fb8f4003520 con 0x7fb910162940 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 msgr2=0x7fb9100af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb9100af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 >> v2:172.21.15.66:3300/0 conn(0x7fb9100ccb40 msgr2=0x7fb9100ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.335+0000 7fb90f400640 1 --2- 172.21.15.59:0/2703309544 >> v2:172.21.15.66:3300/0 conn(0x7fb9100ccb40 0x7fb9100ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb910073450 con 0x7fb910162940 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb900001670 con 0x7fb910162940 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 >> v1:172.21.15.59:6789/0 conn(0x7fb910162940 legacy=0x7fb91006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8f40039b0 0x7fb8f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb8f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb8f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb910073170 con 0x7fb8f4004d40 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f400b9c0 con 0x7fb8f40039b0 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f400bb40 con 0x7fb910162d20 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb915200640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb8f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.336+0000 7fb907e00640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb8f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb90fe00640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8f40039b0 0x7fb8f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb915200640 1 -- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 msgr2=0x7fb8f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb915200640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb910162d20 0x7fb8f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb915200640 1 -- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8f40039b0 msgr2=0x7fb8f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb915200640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8f40039b0 0x7fb8f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb915200640 1 -- 172.21.15.59:0/2703309544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f4003520 con 0x7fb8f4004d40 2024-04-24T06:38:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.337+0000 7fb90fe00640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8f40039b0 0x7fb8f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.338+0000 7fb915200640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb8f40051c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fc0 tx=0x7fb8f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc788582c78127 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.338+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908009c10 con 0x7fb8f4004d40 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.338+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908036070 con 0x7fb8f4004d40 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.338+0000 7fb90f400640 1 -- 172.21.15.59:0/2703309544 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90802d070 con 0x7fb8f4004d40 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 -- 172.21.15.59:0/2703309544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 msgr2=0x7fb8f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 --2- 172.21.15.59:0/2703309544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb8f40051c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fc0 tx=0x7fb8f4005830 comp rx=0 tx=0).stop 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915200640 1 -- 172.21.15.59:0/2703309544 reap_dead start 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 -- 172.21.15.59:0/2703309544 shutdown_connections 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 -- 172.21.15.59:0/2703309544 >> 172.21.15.59:0/2703309544 conn(0x7fb9100b1f90 msgr2=0x7fb9100b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 -- 172.21.15.59:0/2703309544 shutdown_connections 2024-04-24T06:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.339+0000 7fb915c00640 1 -- 172.21.15.59:0/2703309544 wait complete. 2024-04-24T06:38:26.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.341+0000 7fb915c00640 1 Processor -- start 2024-04-24T06:38:26.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.341+0000 7fb915c00640 1 -- start start 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb91015ed90 0x7fb910161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb91014f710 0x7fb9100b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9100ce180 con 0x7fb8f4004d40 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb910073a10 con 0x7fb91015ed90 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb915c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb9100ce300 con 0x7fb91014f710 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb90fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb90fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57346/0 (socket says 172.21.15.59:57346) 2024-04-24T06:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb90fe00640 1 -- 172.21.15.59:0/3597829272 learned_addr learned my addr 172.21.15.59:0/3597829272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.342+0000 7fb90fe00640 1 -- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb91014f710 msgr2=0x7fb9100b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90fe00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb91014f710 0x7fb9100b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90fe00640 1 -- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb91015ed90 msgr2=0x7fb910161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90fe00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb91015ed90 0x7fb910161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90fe00640 1 -- 172.21.15.59:0/3597829272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb910162090 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90fe00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb90000d7f0 tx=0x7fb90000dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900019070 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9000074e0 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900004a20 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91015f2e0 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.343+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91015d400 con 0x7fb8f4004d40 2024-04-24T06:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.344+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91004fd90 con 0x7fb8f4004d40 2024-04-24T06:38:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.345+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb90001c070 con 0x7fb8f4004d40 2024-04-24T06:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.346+0000 7fb90cc00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 0x7fb8ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.346+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5996+0+0 (secure 0 0 0) 0x7fb90008b730 con 0x7fb8f4004d40 2024-04-24T06:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.347+0000 7fb907e00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 0x7fb8ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:26.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.347+0000 7fb907e00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 0x7fb8ec06ee00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb9101fc170 tx=0x7fb8f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.351+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb910162090 con 0x7fb8f4004d40 2024-04-24T06:38:26.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:26.556+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb9100cfbc0 con 0x7fb8f4004d40 2024-04-24T06:38:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.049+0000 7fb90cc00640 1 -- 172.21.15.59:0/3597829272 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7fb900059340 con 0x7fb8f4004d40 2024-04-24T06:38:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:38:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.053+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 msgr2=0x7fb8ec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.053+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 0x7fb8ec06ee00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb9101fc170 tx=0x7fb8f8008040 comp rx=0 tx=0).stop 2024-04-24T06:38:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 msgr2=0x7fb91014ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb90000d7f0 tx=0x7fb90000dcb0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 shutdown_connections 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb8ec06c940 0x7fb8ec06ee00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb91014f710 0x7fb9100b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb91015ed90 0x7fb910161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 --2- 172.21.15.59:0/3597829272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8f4004d40 0x7fb91014ff30 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.054+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 >> 172.21.15.59:0/3597829272 conn(0x7fb9100b1f90 msgr2=0x7fb91015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:27.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.055+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 shutdown_connections 2024-04-24T06:38:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.056+0000 7fb915c00640 1 -- 172.21.15.59:0/3597829272 wait complete. 2024-04-24T06:38:27.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:38:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.293+0000 7f158a400640 1 Processor -- start 2024-04-24T06:38:27.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.294+0000 7f158a400640 1 -- start start 2024-04-24T06:38:27.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f158a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15840ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f158a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1584160120 0x7f1584072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f158a400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15840cfdb0 con 0x7f15840734a0 2024-04-24T06:38:27.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f158a400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f158415fef0 con 0x7f1584160120 2024-04-24T06:38:27.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f158a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f15840738d0 con 0x7f158415fb10 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f1589a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f15840734a0 0x7f1584071cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37234/0 (socket says 172.21.15.59:37234) 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.295+0000 7f1589a00640 1 -- 172.21.15.59:0/2864927438 learned_addr learned my addr 172.21.15.59:0/2864927438 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1589000640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15840ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583e00640 1 --2- 172.21.15.59:0/2864927438 >> v2:172.21.15.66:3300/0 conn(0x7f1584160120 0x7f1584072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 563442744 0 0) 0x7f15840cfdb0 con 0x7f15840734a0 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1568003520 con 0x7f15840734a0 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2743977653 0 0) 0x7f1568003520 con 0x7f15840734a0 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 msgr2=0x7f15840ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15840ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 >> v2:172.21.15.66:3300/0 conn(0x7f1584160120 msgr2=0x7f1584072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 --2- 172.21.15.59:0/2864927438 >> v2:172.21.15.66:3300/0 conn(0x7f1584160120 0x7f1584072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15840cf390 con 0x7f15840734a0 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f157c001a10 con 0x7f15840734a0 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 >> v1:172.21.15.59:6789/0 conn(0x7f15840734a0 legacy=0x7f1584071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1568004060 0x7f1568004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004d40 0x7f15680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f158415fef0 con 0x7f1568004060 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15840738d0 con 0x7f1568004d40 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.296+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f156800bb50 con 0x7f158415fb10 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1568004060 0x7f1568004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1589000640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 -- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 msgr2=0x7f15680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f158415fb10 0x7f15680039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 -- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004d40 msgr2=0x7f15680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1589a00640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004d40 0x7f15680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004d40 0x7f15680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.297+0000 7f1583e00640 1 -- 172.21.15.59:0/2864927438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1568003520 con 0x7f1568004060 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.298+0000 7f1583e00640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1568004060 0x7f1568004800 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f157000ba00 tx=0x7f1568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87dabf7c356ba2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570030070 con 0x7f1568004060 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157002ade0 con 0x7f1568004060 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f1583400640 1 -- 172.21.15.59:0/2864927438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157000b600 con 0x7f1568004060 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 -- 172.21.15.59:0/2864927438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1568004060 msgr2=0x7f1568004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 --2- 172.21.15.59:0/2864927438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1568004060 0x7f1568004800 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f157000ba00 tx=0x7f1568005830 comp rx=0 tx=0).stop 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f1589a00640 1 -- 172.21.15.59:0/2864927438 reap_dead start 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 -- 172.21.15.59:0/2864927438 shutdown_connections 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 -- 172.21.15.59:0/2864927438 >> 172.21.15.59:0/2864927438 conn(0x7f15840b1f90 msgr2=0x7f15840b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 -- 172.21.15.59:0/2864927438 shutdown_connections 2024-04-24T06:38:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.299+0000 7f158a400640 1 -- 172.21.15.59:0/2864927438 wait complete. 2024-04-24T06:38:27.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.301+0000 7f158a400640 1 Processor -- start 2024-04-24T06:38:27.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.301+0000 7f158a400640 1 -- start start 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004060 0x7f15840cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f15840ca500 0x7f15840ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15840cedd0 con 0x7f15840cd0f0 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15840ceaf0 con 0x7f1568004060 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f158a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f15840cf0b0 con 0x7f15840ca500 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.302+0000 7f1583e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57358/0 (socket says 172.21.15.59:57358) 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 -- 172.21.15.59:0/3791803135 learned_addr learned my addr 172.21.15.59:0/3791803135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 -- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f15840ca500 msgr2=0x7f15840ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f15840ca500 0x7f15840ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 -- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004060 msgr2=0x7f15840cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1589000640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004060 0x7f15840cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004060 0x7f15840cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 -- 172.21.15.59:0/3791803135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15840cd570 con 0x7f15840cd0f0 2024-04-24T06:38:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.303+0000 7f1583e00640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f157000baf0 tx=0x7f15700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.304+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570043030 con 0x7f15840cd0f0 2024-04-24T06:38:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.304+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1570035eb0 con 0x7f15840cd0f0 2024-04-24T06:38:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.304+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570004020 con 0x7f15840cd0f0 2024-04-24T06:38:27.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.304+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15840c9740 con 0x7f15840cd0f0 2024-04-24T06:38:27.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.304+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15840c9d00 con 0x7f15840cd0f0 2024-04-24T06:38:27.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.305+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158404fd90 con 0x7f15840cd0f0 2024-04-24T06:38:27.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.306+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1570046070 con 0x7f15840cd0f0 2024-04-24T06:38:27.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.307+0000 7f1580c00640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:27.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.307+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5558+0+0 (secure 0 0 0) 0x7f15700ac550 con 0x7f15840cd0f0 2024-04-24T06:38:27.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.308+0000 7f1589000640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:27.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.308+0000 7f1589000640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 0x7f156406edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1574006fd0 tx=0x7f1574008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:27.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.312+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15840cd570 con 0x7f15840cd0f0 2024-04-24T06:38:27.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:27.536+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f15840cdd60 con 0x7f15840cd0f0 2024-04-24T06:38:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.063+0000 7f1580c00640 1 -- 172.21.15.59:0/3791803135 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f157003c3a0 con 0x7f15840cd0f0 2024-04-24T06:38:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:38:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.068+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 msgr2=0x7f156406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.069+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 0x7f156406edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1574006fd0 tx=0x7f1574008040 comp rx=0 tx=0).stop 2024-04-24T06:38:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.069+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 msgr2=0x7f158415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.069+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f157000baf0 tx=0x7f15700046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 shutdown_connections 2024-04-24T06:38:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f15840ca500 0x7f15840ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15840cd0f0 0x7f158415e110 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 --2- 172.21.15.59:0/3791803135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1568004060 0x7f15840cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.070+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 >> 172.21.15.59:0/3791803135 conn(0x7f15840b1f90 msgr2=0x7f158406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:28.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.071+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 shutdown_connections 2024-04-24T06:38:28.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.071+0000 7f158a400640 1 -- 172.21.15.59:0/3791803135 wait complete. 2024-04-24T06:38:28.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:38:28.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.301+0000 7f0110a00640 1 Processor -- start 2024-04-24T06:38:28.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.302+0000 7f0110a00640 1 -- start start 2024-04-24T06:38:28.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.303+0000 7f0110a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 0x7f010c06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.303+0000 7f0110a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.303+0000 7f0110a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010c0ae6b0 con 0x7f010c15b220 2024-04-24T06:38:28.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.303+0000 7f0110a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f010c0afbe0 con 0x7f010c0af750 2024-04-24T06:38:28.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.303+0000 7f0110a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f010c0d0f40 con 0x7f010c0d05d0 2024-04-24T06:38:28.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.304+0000 7f010b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.304+0000 7f010b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:49324/0 (socket says 172.21.15.59:49324) 2024-04-24T06:38:28.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.304+0000 7f010b200640 1 -- 172.21.15.59:0/1086502427 learned_addr learned my addr 172.21.15.59:0/1086502427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:28.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.304+0000 7f0101e00640 1 --2- 172.21.15.59:0/1086502427 >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 0x7f010c06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f0109e00640 1 -- 172.21.15.59:0/1086502427 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801869318 0 0) 0x7f010c0ae6b0 con 0x7f010c15b220 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f0109e00640 1 -- 172.21.15.59:0/1086502427 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00f0003520 con 0x7f010c15b220 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f010b200640 1 -- 172.21.15.59:0/1086502427 >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 msgr2=0x7f010c06f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f010b200640 1 --2- 172.21.15.59:0/1086502427 >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 0x7f010c06f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f010b200640 1 -- 172.21.15.59:0/1086502427 >> v1:172.21.15.66:6789/0 conn(0x7f010c15b220 legacy=0x7f010c0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f010b200640 1 -- 172.21.15.59:0/1086502427 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c0ae990 con 0x7f010c0d05d0 2024-04-24T06:38:28.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.305+0000 7f0101e00640 1 --2- 172.21.15.59:0/1086502427 >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 0x7f010c06f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.307+0000 7f010b200640 1 --2- 172.21.15.59:0/1086502427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f010400d150 tx=0x7f010402c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c09451cb94a2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.307+0000 7f0109e00640 1 -- 172.21.15.59:0/1086502427 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104009600 con 0x7f010c0d05d0 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.307+0000 7f0109e00640 1 -- 172.21.15.59:0/1086502427 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0104037070 con 0x7f010c0d05d0 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.307+0000 7f0109e00640 1 -- 172.21.15.59:0/1086502427 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01040099c0 con 0x7f010c0d05d0 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 -- 172.21.15.59:0/1086502427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 msgr2=0x7f010c0d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 --2- 172.21.15.59:0/1086502427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f010400d150 tx=0x7f010402c930 comp rx=0 tx=0).stop 2024-04-24T06:38:28.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 -- 172.21.15.59:0/1086502427 shutdown_connections 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 --2- 172.21.15.59:0/1086502427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c0d05d0 0x7f010c0d0a00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 --2- 172.21.15.59:0/1086502427 >> v2:172.21.15.66:3300/0 conn(0x7f010c0af750 0x7f010c06f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 -- 172.21.15.59:0/1086502427 >> 172.21.15.59:0/1086502427 conn(0x7f010c06bdf0 msgr2=0x7f010c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 -- 172.21.15.59:0/1086502427 shutdown_connections 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.308+0000 7f0110a00640 1 -- 172.21.15.59:0/1086502427 wait complete. 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.310+0000 7f0110a00640 1 Processor -- start 2024-04-24T06:38:28.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.310+0000 7f0110a00640 1 -- start start 2024-04-24T06:38:28.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010c0af750 0x7f010c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c15b220 0x7f010c1efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f010c0af230 con 0x7f010c0d05d0 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f010c0aef50 con 0x7f010c0af750 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.311+0000 7f0110a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f010c15b600 con 0x7f010c15b220 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57366/0 (socket says 172.21.15.59:57366) 2024-04-24T06:38:28.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 -- 172.21.15.59:0/230558197 learned_addr learned my addr 172.21.15.59:0/230558197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f010a800640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010c0af750 0x7f010c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f010b200640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c15b220 0x7f010c1efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 -- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c15b220 msgr2=0x7f010c1efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c15b220 0x7f010c1efc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 -- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010c0af750 msgr2=0x7f010c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010c0af750 0x7f010c15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.312+0000 7f0101e00640 1 -- 172.21.15.59:0/230558197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c15c370 con 0x7f010c0d05d0 2024-04-24T06:38:28.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0101e00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f00f400ec10 tx=0x7f00f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017070 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f400cc30 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4013470 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c15c5e0 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.313+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c15cba0 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.314+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c04fd90 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.315+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00f401a070 con 0x7f010c0d05d0 2024-04-24T06:38:28.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.316+0000 7f0103400640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 0x7f00e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.316+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f00f408b910 con 0x7f010c0d05d0 2024-04-24T06:38:28.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.317+0000 7f010a800640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 0x7f00e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.317+0000 7f010a800640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 0x7f00e806edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000f60 tx=0x7f00fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.321+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00f4059ef0 con 0x7f010c0d05d0 2024-04-24T06:38:28.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.524+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f010c158460 con 0x7f010c0d05d0 2024-04-24T06:38:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.532+0000 7f0103400640 1 -- 172.21.15.59:0/230558197 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f00f40597f0 con 0x7f010c0d05d0 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 msgr2=0x7f00e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 0x7f00e806edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000f60 tx=0x7f00fc009040 comp rx=0 tx=0).stop 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 msgr2=0x7f010c15be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f00f400ec10 tx=0x7f00f400c520 comp rx=0 tx=0).stop 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 shutdown_connections 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f00e806c8f0 0x7f00e806edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f010c15b220 0x7f010c1efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010c0d05d0 0x7f010c15be30 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 --2- 172.21.15.59:0/230558197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f010c0af750 0x7f010c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 >> 172.21.15.59:0/230558197 conn(0x7f010c06bdf0 msgr2=0x7f010c158740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:28.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.535+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 shutdown_connections 2024-04-24T06:38:28.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.536+0000 7f0110a00640 1 -- 172.21.15.59:0/230558197 wait complete. 2024-04-24T06:38:28.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-24T06:38:28.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.770+0000 7fe31a600640 1 Processor -- start 2024-04-24T06:38:28.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.771+0000 7fe31a600640 1 -- start start 2024-04-24T06:38:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.772+0000 7fe31a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe31415e7c0 0x7fe3140d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.772+0000 7fe31a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 0x7fe3140b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.772+0000 7fe31a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3140cf030 con 0x7fe3140b6530 2024-04-24T06:38:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.772+0000 7fe31a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe3140b7000 con 0x7fe31415e7c0 2024-04-24T06:38:28.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.772+0000 7fe31a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3140cf620 con 0x7fe3140d2630 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe31415e7c0 0x7fe3140d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe3140b6530 0x7fe3140d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37250/0 (socket says 172.21.15.59:37250) 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319c00640 1 -- 172.21.15.59:0/4276878949 learned_addr learned my addr 172.21.15.59:0/4276878949 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe30be00640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 0x7fe3140b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe313e00640 1 -- 172.21.15.59:0/4276878949 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606272142 0 0) 0x7fe30c001490 con 0x7fe3140b6530 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe313e00640 1 -- 172.21.15.59:0/4276878949 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2f8003520 con 0x7fe3140b6530 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319200640 1 -- 172.21.15.59:0/4276878949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 msgr2=0x7fe3140b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319200640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 0x7fe3140b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319200640 1 -- 172.21.15.59:0/4276878949 >> v1:172.21.15.59:6789/0 conn(0x7fe3140b6530 legacy=0x7fe3140d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe319200640 1 -- 172.21.15.59:0/4276878949 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3140b6a40 con 0x7fe31415e7c0 2024-04-24T06:38:28.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.773+0000 7fe30be00640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 0x7fe3140b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:28.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.774+0000 7fe319200640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe31415e7c0 0x7fe3140d20f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe3040097a0 tx=0x7fe30402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1bb656e253a7df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe313e00640 1 -- 172.21.15.59:0/4276878949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304037070 con 0x7fe31415e7c0 2024-04-24T06:38:28.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe313e00640 1 -- 172.21.15.59:0/4276878949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304032440 con 0x7fe31415e7c0 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe313e00640 1 -- 172.21.15.59:0/4276878949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304032810 con 0x7fe31415e7c0 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 -- 172.21.15.59:0/4276878949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe31415e7c0 msgr2=0x7fe3140d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe31415e7c0 0x7fe3140d20f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe3040097a0 tx=0x7fe30402a180 comp rx=0 tx=0).stop 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 -- 172.21.15.59:0/4276878949 shutdown_connections 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3140d2630 0x7fe3140b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 --2- 172.21.15.59:0/4276878949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe31415e7c0 0x7fe3140d20f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.775+0000 7fe31a600640 1 -- 172.21.15.59:0/4276878949 >> 172.21.15.59:0/4276878949 conn(0x7fe31409d920 msgr2=0x7fe31408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.776+0000 7fe31a600640 1 -- 172.21.15.59:0/4276878949 shutdown_connections 2024-04-24T06:38:28.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.776+0000 7fe31a600640 1 -- 172.21.15.59:0/4276878949 wait complete. 2024-04-24T06:38:28.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.777+0000 7fe31a600640 1 Processor -- start 2024-04-24T06:38:28.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.777+0000 7fe31a600640 1 -- start start 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3140b6530 0x7fe3140b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe31415e7c0 0x7fe3140cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3140d1a80 con 0x7fe3140d2630 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3140cf2b0 con 0x7fe3140b6530 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.778+0000 7fe31a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3140d1c00 con 0x7fe31415e7c0 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57394/0 (socket says 172.21.15.59:57394) 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 -- 172.21.15.59:0/1963273041 learned_addr learned my addr 172.21.15.59:0/1963273041 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe319200640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3140b6530 0x7fe3140b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 -- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe31415e7c0 msgr2=0x7fe3140cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe31415e7c0 0x7fe3140cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 -- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3140b6530 msgr2=0x7fe3140b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3140b6530 0x7fe3140b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.779+0000 7fe30be00640 1 -- 172.21.15.59:0/1963273041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3140b7710 con 0x7fe3140d2630 2024-04-24T06:38:28.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe30be00640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e810 tx=0x7fe2fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00f040 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc004230 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00c6d0 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3140b1f80 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.780+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3140b2540 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.781+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31404fd90 con 0x7fe3140d2630 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.782+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2fc00c870 con 0x7fe3140d2630 2024-04-24T06:38:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.783+0000 7fe311600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.783+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fe2fc08b670 con 0x7fe3140d2630 2024-04-24T06:38:28.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.784+0000 7fe319200640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:28.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.784+0000 7fe319200640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe304034000 tx=0x7fe30402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:28.788+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2fc017020 con 0x7fe3140d2630 2024-04-24T06:38:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:29.073+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fe3140b4570 con 0x7fe3140d2630 2024-04-24T06:38:30.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.070+0000 7fe311600640 1 -- 172.21.15.59:0/1963273041 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v468) ==== 167+0+0 (secure 0 0 0) 0x7fe2fc059c50 con 0x7fe3140d2630 2024-04-24T06:38:30.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 msgr2=0x7fe2f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 0x7fe2f006edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe304034000 tx=0x7fe30402e000 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 msgr2=0x7fe3140cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00e810 tx=0x7fe2fc00ecd0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 shutdown_connections 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe2f006c8f0 0x7fe2f006edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe31415e7c0 0x7fe3140cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3140d2630 0x7fe3140cde70 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 --2- 172.21.15.59:0/1963273041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3140b6530 0x7fe3140b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.073+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 >> 172.21.15.59:0/1963273041 conn(0x7fe31409d920 msgr2=0x7fe3140b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.074+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 shutdown_connections 2024-04-24T06:38:30.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.074+0000 7fe31a600640 1 -- 172.21.15.59:0/1963273041 wait complete. 2024-04-24T06:38:30.096 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-24T06:38:30.096 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:38:30.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.295+0000 7f252d200640 1 Processor -- start 2024-04-24T06:38:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.296+0000 7f252d200640 1 -- start start 2024-04-24T06:38:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.297+0000 7f252d200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f25280d14a0 0x7f25280b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.297+0000 7f252d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25280cf030 con 0x7f252815e7c0 2024-04-24T06:38:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.297+0000 7f252d200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25280b6a40 con 0x7f25280b5ff0 2024-04-24T06:38:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.297+0000 7f252d200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f25280b7000 con 0x7f25280d14a0 2024-04-24T06:38:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.298+0000 7f2527400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f252815e7c0 0x7f25280d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49624/0 (socket says 172.21.15.59:49624) 2024-04-24T06:38:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.298+0000 7f2527400640 1 -- 172.21.15.59:0/3532296278 learned_addr learned my addr 172.21.15.59:0/3532296278 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.298+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075265574 0 0) 0x7f2518001490 con 0x7f252815e7c0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.298+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f250c003520 con 0x7f252815e7c0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.298+0000 7f2527e00640 1 --2- 172.21.15.59:0/3532296278 >> v2:172.21.15.66:3300/0 conn(0x7f25280d14a0 0x7f25280b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3657354767 0 0) 0x7f250c003520 con 0x7f252815e7c0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 >> v2:172.21.15.66:3300/0 conn(0x7f25280d14a0 msgr2=0x7f25280b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 --2- 172.21.15.59:0/3532296278 >> v2:172.21.15.66:3300/0 conn(0x7f25280d14a0 0x7f25280b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 >> v1:172.21.15.66:6789/0 conn(0x7f25280b5ff0 legacy=0x7f25280d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25280b6d20 con 0x7f252815e7c0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f2518001bc0 con 0x7f252815e7c0 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 >> v1:172.21.15.59:6789/0 conn(0x7f252815e7c0 legacy=0x7f25280d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f250c003a00 0x7f250c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f250c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f250c005210 0x7f250c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25280b7000 con 0x7f250c004800 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f250c00c160 con 0x7f250c003a00 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f250c00c360 con 0x7f250c005210 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.299+0000 7f2526a00640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f250c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2527e00640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f250c005210 0x7f250c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2527400640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f250c003a00 0x7f250c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2526a00640 1 -- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f250c005210 msgr2=0x7f250c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2526a00640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f250c005210 0x7f250c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2526a00640 1 -- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f250c003a00 msgr2=0x7f250c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2526a00640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f250c003a00 0x7f250c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2526a00640 1 -- 172.21.15.59:0/3532296278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c003520 con 0x7f250c004800 2024-04-24T06:38:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.300+0000 7f2527e00640 1 -- 172.21.15.59:0/3532296278 reap_dead start 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.301+0000 7f2526a00640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f250c004c80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2514001750 tx=0x7f250c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457a3d78a2cbbc32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514001370 con 0x7f250c004800 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25140052d0 con 0x7f250c004800 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f2526000640 1 -- 172.21.15.59:0/3532296278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251402abc0 con 0x7f250c004800 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f252d200640 1 -- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 msgr2=0x7f250c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f252d200640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f250c004c80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2514001750 tx=0x7f250c005830 comp rx=0 tx=0).stop 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f252d200640 1 -- 172.21.15.59:0/3532296278 shutdown_connections 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f252d200640 1 --2- 172.21.15.59:0/3532296278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f250c004c80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.302+0000 7f252d200640 1 -- 172.21.15.59:0/3532296278 >> 172.21.15.59:0/3532296278 conn(0x7f252809d920 msgr2=0x7f252808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.303+0000 7f252d200640 1 -- 172.21.15.59:0/3532296278 shutdown_connections 2024-04-24T06:38:30.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.303+0000 7f252d200640 1 -- 172.21.15.59:0/3532296278 wait complete. 2024-04-24T06:38:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.304+0000 7f252d200640 1 Processor -- start 2024-04-24T06:38:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.305+0000 7f252d200640 1 -- start start 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.305+0000 7f252d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f252d200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25281f8870 0x7f25280cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f252d200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25280ce0a0 0x7f252814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f252d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25280d1a80 con 0x7f250c004800 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f252d200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25280cf620 con 0x7f25280ce0a0 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f252d200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25280d1c00 con 0x7f25281f8870 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34384/0 (socket says 172.21.15.59:34384) 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 -- 172.21.15.59:0/1316909868 learned_addr learned my addr 172.21.15.59:0/1316909868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 -- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25281f8870 msgr2=0x7f25280cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25281f8870 0x7f25280cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 -- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25280ce0a0 msgr2=0x7f252814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25280ce0a0 0x7f252814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.306+0000 7f2527400640 1 -- 172.21.15.59:0/1316909868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25280cc0a0 con 0x7f250c004800 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f2527400640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f25180013d0 tx=0x7f251800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800edf0 con 0x7f250c004800 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518018070 con 0x7f250c004800 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518013660 con 0x7f250c004800 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25280cc380 con 0x7f250c004800 2024-04-24T06:38:30.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.307+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2528160ed0 con 0x7f250c004800 2024-04-24T06:38:30.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.308+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252804fd90 con 0x7f250c004800 2024-04-24T06:38:30.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.309+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2518013800 con 0x7f250c004800 2024-04-24T06:38:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.310+0000 7f251f400640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 0x7f250806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.310+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f251808c0c0 con 0x7f250c004800 2024-04-24T06:38:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.310+0000 7f2526a00640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 0x7f250806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.311+0000 7f2526a00640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 0x7f250806edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2514032000 tx=0x7f251402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.315+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25280cf030 con 0x7f250c004800 2024-04-24T06:38:30.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.526+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f252808c130 con 0x7f250c004800 2024-04-24T06:38:30.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.528+0000 7f251f400640 1 -- 172.21.15.59:0/1316909868 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+13195 (secure 0 0 0) 0x7f251805a730 con 0x7f250c004800 2024-04-24T06:38:30.530 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:30.530 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:28.051790+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/3165421940":"2024-04-25T06:38:25.032276+0000","172.21.15.59:6834/3165421940":"2024-04-25T06:38:25.032276+0000","172.21.15.59:6835/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.59:6834/2792490076":"2024-04-25T06:37:35.076161+0000","172.21.15.66:6833/301885470":"2024-04-25T06:36:16.639793+0000","172.21.15.66:6832/301885470":"2024-04-25T06:36:16.639793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:30.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.530+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 msgr2=0x7f250806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.530+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 0x7f250806edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2514032000 tx=0x7f251402c000 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.530+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 msgr2=0x7f2528160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.530+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f25180013d0 tx=0x7f251800e530 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 shutdown_connections 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f250806c8f0 0x7f250806edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25280ce0a0 0x7f252814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25281f8870 0x7f25280cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 --2- 172.21.15.59:0/1316909868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f250c004800 0x7f2528160990 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 >> 172.21.15.59:0/1316909868 conn(0x7f252809d920 msgr2=0x7f25280b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 shutdown_connections 2024-04-24T06:38:30.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.531+0000 7f252d200640 1 -- 172.21.15.59:0/1316909868 wait complete. 2024-04-24T06:38:30.551 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:30.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:30.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.754+0000 7fafcbe00640 1 Processor -- start 2024-04-24T06:38:30.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.754+0000 7fafcbe00640 1 -- start start 2024-04-24T06:38:30.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcbe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fafcc15d300 0x7fafcc071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafcc0d0600 0x7fafcc0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcbe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafcc163320 con 0x7fafcc167180 2024-04-24T06:38:30.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcbe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fafcc0d0f90 con 0x7fafcc0d0600 2024-04-24T06:38:30.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcbe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fafcc163650 con 0x7fafcc15d300 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafcc0d0600 0x7fafcc0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafcc0d0600 0x7fafcc0d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34388/0 (socket says 172.21.15.59:34388) 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.756+0000 7fafcb400640 1 -- 172.21.15.59:0/1897308384 learned_addr learned my addr 172.21.15.59:0/1897308384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.757+0000 7fafcb400640 1 -- 172.21.15.59:0/1897308384 >> v2:172.21.15.66:3300/0 conn(0x7fafcc15d300 msgr2=0x7fafcc071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.757+0000 7fafcb400640 1 --2- 172.21.15.59:0/1897308384 >> v2:172.21.15.66:3300/0 conn(0x7fafcc15d300 0x7fafcc071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.757+0000 7fafcb400640 1 -- 172.21.15.59:0/1897308384 >> v1:172.21.15.66:6789/0 conn(0x7fafcc167180 legacy=0x7fafcc163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.757+0000 7fafcb400640 1 -- 172.21.15.59:0/1897308384 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc072380 con 0x7fafcc0d0600 2024-04-24T06:38:30.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.758+0000 7fafcb400640 1 --2- 172.21.15.59:0/1897308384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc0d0a50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0b42c0 tx=0x7fafc402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952613619927d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.758+0000 7fafca000640 1 -- 172.21.15.59:0/1897308384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4036070 con 0x7fafcc0d0600 2024-04-24T06:38:30.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.758+0000 7fafca000640 1 -- 172.21.15.59:0/1897308384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4031600 con 0x7fafcc0d0600 2024-04-24T06:38:30.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.758+0000 7fafca000640 1 -- 172.21.15.59:0/1897308384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40319d0 con 0x7fafcc0d0600 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 -- 172.21.15.59:0/1897308384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 msgr2=0x7fafcc0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 --2- 172.21.15.59:0/1897308384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc0d0a50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0b42c0 tx=0x7fafc402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 -- 172.21.15.59:0/1897308384 shutdown_connections 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 --2- 172.21.15.59:0/1897308384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc0d0a50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 --2- 172.21.15.59:0/1897308384 >> v2:172.21.15.66:3300/0 conn(0x7fafcc15d300 0x7fafcc071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 -- 172.21.15.59:0/1897308384 >> 172.21.15.59:0/1897308384 conn(0x7fafcc06bdf0 msgr2=0x7fafcc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:30.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 -- 172.21.15.59:0/1897308384 shutdown_connections 2024-04-24T06:38:30.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.759+0000 7fafcbe00640 1 -- 172.21.15.59:0/1897308384 wait complete. 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.761+0000 7fafcbe00640 1 Processor -- start 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.762+0000 7fafcbe00640 1 -- start start 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc15d300 0x7fafcc0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 0x7fafcc161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1675e0 con 0x7fafcc0d0600 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc072940 con 0x7fafcc15d300 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcbe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc06e1e0 con 0x7fafcc167180 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34396/0 (socket says 172.21.15.59:34396) 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcaa00640 1 -- 172.21.15.59:0/3138714847 learned_addr learned my addr 172.21.15.59:0/3138714847 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.763+0000 7fafcb400640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 0x7fafcc161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafc3e00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc15d300 0x7fafcc0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 -- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 msgr2=0x7fafcc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 0x7fafcc161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 -- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc15d300 msgr2=0x7fafcc0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc15d300 0x7fafcc0b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 -- 172.21.15.59:0/3138714847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc160c50 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcb400640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 0x7fafcc161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcaa00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b4a0 tx=0x7fafbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc014070 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc010610 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc010a00 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.764+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc160f30 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.765+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc160940 con 0x7fafcc0d0600 2024-04-24T06:38:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.766+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc161100 con 0x7fafcc0d0600 2024-04-24T06:38:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.767+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafbc017070 con 0x7fafcc0d0600 2024-04-24T06:38:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.768+0000 7fafc2a00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 0x7fafa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.768+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fafbc08b390 con 0x7fafcc0d0600 2024-04-24T06:38:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.768+0000 7fafc3e00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 0x7fafa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.769+0000 7fafc3e00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 0x7fafa806edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fafcc15c150 tx=0x7fafb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.773+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafbc059a10 con 0x7fafcc0d0600 2024-04-24T06:38:30.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.979+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafcc04fd90 con 0x7fafcc0d0600 2024-04-24T06:38:30.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.979+0000 7fafc2a00640 1 -- 172.21.15.59:0/3138714847 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fafbc059310 con 0x7fafcc0d0600 2024-04-24T06:38:30.982 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 msgr2=0x7fafa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 0x7fafa806edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fafcc15c150 tx=0x7fafb4008040 comp rx=0 tx=0).stop 2024-04-24T06:38:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 msgr2=0x7fafcc15d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b4a0 tx=0x7fafbc00b960 comp rx=0 tx=0).stop 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 shutdown_connections 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fafa806c8f0 0x7fafa806edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fafcc167180 0x7fafcc161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafcc15d300 0x7fafcc0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 --2- 172.21.15.59:0/3138714847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc0d0600 0x7fafcc15d6e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.982+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 >> 172.21.15.59:0/3138714847 conn(0x7fafcc06bdf0 msgr2=0x7fafcc166690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.983+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 shutdown_connections 2024-04-24T06:38:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:30.983+0000 7fafcbe00640 1 -- 172.21.15.59:0/3138714847 wait complete. 2024-04-24T06:38:31.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-24T06:38:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.212+0000 7fb4f4c00640 1 Processor -- start 2024-04-24T06:38:31.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.212+0000 7fb4f4c00640 1 -- start start 2024-04-24T06:38:31.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4f4c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4f015b450 0x7fb4f00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4f4c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 0x7fb4f00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4f4c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f00b3870 con 0x7fb4f00b3080 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4f4c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f00b41a0 con 0x7fb4f015b450 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4f4c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f00b3b50 con 0x7fb4f006f3f0 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4efe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb4f00b3080 0x7fb4f00ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49632/0 (socket says 172.21.15.59:49632) 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4efe00640 1 -- 172.21.15.59:0/3559631719 learned_addr learned my addr 172.21.15.59:0/3559631719 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.214+0000 7fb4ef400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb4f015b450 0x7fb4f00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4eea00640 1 --2- 172.21.15.59:0/3559631719 >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 0x7fb4f00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4ef400640 1 -- 172.21.15.59:0/3559631719 >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 msgr2=0x7fb4f00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4ef400640 1 --2- 172.21.15.59:0/3559631719 >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 0x7fb4f00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4ef400640 1 -- 172.21.15.59:0/3559631719 >> v1:172.21.15.59:6789/0 conn(0x7fb4f00b3080 legacy=0x7fb4f00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4ef400640 1 -- 172.21.15.59:0/3559631719 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f00b3e30 con 0x7fb4f015b450 2024-04-24T06:38:31.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.215+0000 7fb4eea00640 1 --2- 172.21.15.59:0/3559631719 >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 0x7fb4f00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.216+0000 7fb4ef400640 1 --2- 172.21.15.59:0/3559631719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f015b450 0x7fb4f00b4630 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00af4a0 tx=0x7fb4e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4310301aaf577e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:31.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.216+0000 7fb4ee000640 1 -- 172.21.15.59:0/3559631719 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0037070 con 0x7fb4f015b450 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.216+0000 7fb4ee000640 1 -- 172.21.15.59:0/3559631719 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e0032440 con 0x7fb4f015b450 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.216+0000 7fb4ee000640 1 -- 172.21.15.59:0/3559631719 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0032810 con 0x7fb4f015b450 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 -- 172.21.15.59:0/3559631719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f015b450 msgr2=0x7fb4f00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/3559631719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f015b450 0x7fb4f00b4630 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00af4a0 tx=0x7fb4e002a230 comp rx=0 tx=0).stop 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 -- 172.21.15.59:0/3559631719 shutdown_connections 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/3559631719 >> v2:172.21.15.66:3300/0 conn(0x7fb4f006f3f0 0x7fb4f00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/3559631719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f015b450 0x7fb4f00b4630 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 -- 172.21.15.59:0/3559631719 >> 172.21.15.59:0/3559631719 conn(0x7fb4f00b21a0 msgr2=0x7fb4f00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 -- 172.21.15.59:0/3559631719 shutdown_connections 2024-04-24T06:38:31.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.217+0000 7fb4f4c00640 1 -- 172.21.15.59:0/3559631719 wait complete. 2024-04-24T06:38:31.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.219+0000 7fb4f4c00640 1 Processor -- start 2024-04-24T06:38:31.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.219+0000 7fb4f4c00640 1 -- start start 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4f006f3f0 0x7fb4f01e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4f015b450 0x7fb4f015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f00b4ff0 con 0x7fb4f00b3080 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f015b830 con 0x7fb4f015b450 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4f4c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f00cd590 con 0x7fb4f006f3f0 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34422/0 (socket says 172.21.15.59:34422) 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.220+0000 7fb4eea00640 1 -- 172.21.15.59:0/1329284726 learned_addr learned my addr 172.21.15.59:0/1329284726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4ef400640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4f006f3f0 0x7fb4f01e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4efe00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4f015b450 0x7fb4f015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 -- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4f006f3f0 msgr2=0x7fb4f01e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4f006f3f0 0x7fb4f01e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 -- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4f015b450 msgr2=0x7fb4f015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4f015b450 0x7fb4f015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 -- 172.21.15.59:0/1329284726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01e4040 con 0x7fb4f00b3080 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.221+0000 7fb4eea00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7f0 tx=0x7fb4d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.222+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8015070 con 0x7fb4f00b3080 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.222+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8007a30 con 0x7fb4f00b3080 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.222+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01e4320 con 0x7fb4f00b3080 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.222+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8007e00 con 0x7fb4f00b3080 2024-04-24T06:38:31.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.222+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01e48e0 con 0x7fb4f00b3080 2024-04-24T06:38:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.223+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f00cf850 con 0x7fb4f00b3080 2024-04-24T06:38:31.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.224+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4d8018070 con 0x7fb4f00b3080 2024-04-24T06:38:31.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.225+0000 7fb4e7400640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 0x7fb4cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:31.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.225+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fb4d808b960 con 0x7fb4f00b3080 2024-04-24T06:38:31.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.225+0000 7fb4ef400640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 0x7fb4cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:31.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.226+0000 7fb4ef400640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 0x7fb4cc06edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0034000 tx=0x7fb4e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.229+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4f00cf850 con 0x7fb4f00b3080 2024-04-24T06:38:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:31.511+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fb4f004fd90 con 0x7fb4f00b3080 2024-04-24T06:38:32.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.083+0000 7fb4e7400640 1 -- 172.21.15.59:0/1329284726 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v470) ==== 149+0+0 (secure 0 0 0) 0x7fb4d8059f40 con 0x7fb4f00b3080 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 msgr2=0x7fb4cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 0x7fb4cc06edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0034000 tx=0x7fb4e002e000 comp rx=0 tx=0).stop 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 msgr2=0x7fb4f0158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7f0 tx=0x7fb4d800cbf0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 shutdown_connections 2024-04-24T06:38:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb4cc06c8f0 0x7fb4cc06edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4f015b450 0x7fb4f015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4f00b3080 0x7fb4f0158200 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 --2- 172.21.15.59:0/1329284726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb4f006f3f0 0x7fb4f01e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 >> 172.21.15.59:0/1329284726 conn(0x7fb4f00b21a0 msgr2=0x7fb4f00ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 shutdown_connections 2024-04-24T06:38:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.087+0000 7fb4f4c00640 1 -- 172.21.15.59:0/1329284726 wait complete. 2024-04-24T06:38:32.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:38:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.332+0000 7fa6a7e00640 1 Processor -- start 2024-04-24T06:38:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.332+0000 7fa6a7e00640 1 -- start start 2024-04-24T06:38:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.333+0000 7fa6a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa6a80ad170 0x7fa6a806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.333+0000 7fa6a7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a8071e10 con 0x7fa6a80d0600 2024-04-24T06:38:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.333+0000 7fa6a7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a80720f0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.333+0000 7fa6a7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a8072a20 con 0x7fa6a80ad170 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa6a80ad170 0x7fa6a806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa6a80d0600 0x7fa6a8162d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49646/0 (socket says 172.21.15.59:49646) 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a6a00640 1 -- 172.21.15.59:0/951252792 learned_addr learned my addr 172.21.15.59:0/951252792 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a7400640 1 -- 172.21.15.59:0/951252792 >> v1:172.21.15.66:6789/0 conn(0x7fa6a806dcb0 legacy=0x7fa6a80d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a7400640 1 -- 172.21.15.59:0/951252792 >> v1:172.21.15.59:6789/0 conn(0x7fa6a80d0600 legacy=0x7fa6a8162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.334+0000 7fa6a7400640 1 -- 172.21.15.59:0/951252792 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a80723d0 con 0x7fa6a80ad170 2024-04-24T06:38:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.335+0000 7fa6a7400640 1 --2- 172.21.15.59:0/951252792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a80ad170 0x7fa6a806e5c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d90 tx=0x7fa6a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0889744ed1dac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.336+0000 7fa6a5600640 1 -- 172.21.15.59:0/951252792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0007740 con 0x7fa6a80ad170 2024-04-24T06:38:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.336+0000 7fa6a5600640 1 -- 172.21.15.59:0/951252792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a00078e0 con 0x7fa6a80ad170 2024-04-24T06:38:32.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.336+0000 7fa6a5600640 1 -- 172.21.15.59:0/951252792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0007cb0 con 0x7fa6a80ad170 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.336+0000 7fa6a7e00640 1 -- 172.21.15.59:0/951252792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a80ad170 msgr2=0x7fa6a806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.336+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/951252792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a80ad170 0x7fa6a806e5c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d90 tx=0x7fa6a002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.337+0000 7fa6a7e00640 1 -- 172.21.15.59:0/951252792 shutdown_connections 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.337+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/951252792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a80ad170 0x7fa6a806e5c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.337+0000 7fa6a7e00640 1 -- 172.21.15.59:0/951252792 >> 172.21.15.59:0/951252792 conn(0x7fa6a806bdf0 msgr2=0x7fa6a806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.337+0000 7fa6a7e00640 1 -- 172.21.15.59:0/951252792 shutdown_connections 2024-04-24T06:38:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.337+0000 7fa6a7e00640 1 -- 172.21.15.59:0/951252792 wait complete. 2024-04-24T06:38:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.338+0000 7fa6a7e00640 1 Processor -- start 2024-04-24T06:38:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.339+0000 7fa6a7e00640 1 -- start start 2024-04-24T06:38:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.339+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.339+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 0x7fa6a80b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.339+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa6a80d0600 0x7fa6a80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a815eaf0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80ad620 con 0x7fa6a80ad170 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a806eb00 con 0x7fa6a80d0600 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34434/0 (socket says 172.21.15.59:34434) 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 -- 172.21.15.59:0/600324093 learned_addr learned my addr 172.21.15.59:0/600324093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6000640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 0x7fa6a80b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a7400640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa6a80d0600 0x7fa6a80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 -- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa6a80d0600 msgr2=0x7fa6a80b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa6a80d0600 0x7fa6a80b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 -- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 msgr2=0x7fa6a80b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 0x7fa6a80b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6a00640 1 -- 172.21.15.59:0/600324093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a81f84b0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.340+0000 7fa6a6000640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 0x7fa6a80b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa6a6a00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa69800c480 tx=0x7fa69800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698017070 con 0x7fa6a806dcb0 2024-04-24T06:38:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa698005ce0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6980072f0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a81f8790 con 0x7fa6a806dcb0 2024-04-24T06:38:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.341+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81f8d50 con 0x7fa6a806dcb0 2024-04-24T06:38:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.343+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a804fd90 con 0x7fa6a806dcb0 2024-04-24T06:38:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.343+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa69801a070 con 0x7fa6a806dcb0 2024-04-24T06:38:32.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.344+0000 7fa69ea00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 0x7fa68806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:32.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.344+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fa69808aba0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.345+0000 7fa6a6000640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 0x7fa68806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:32.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.346+0000 7fa6a6000640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 0x7fa68806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8151af0 tx=0x7fa694009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:32.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.349+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa698058bc0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.554+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6a80cc770 con 0x7fa6a806dcb0 2024-04-24T06:38:32.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.559+0000 7fa69ea00640 1 -- 172.21.15.59:0/600324093 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fa69805cec0 con 0x7fa6a806dcb0 2024-04-24T06:38:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 msgr2=0x7fa68806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 0x7fa68806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8151af0 tx=0x7fa694009040 comp rx=0 tx=0).stop 2024-04-24T06:38:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 msgr2=0x7fa6a80cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa69800c480 tx=0x7fa69800c940 comp rx=0 tx=0).stop 2024-04-24T06:38:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 shutdown_connections 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa68806c8f0 0x7fa68806edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa6a80d0600 0x7fa6a80b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6a80ad170 0x7fa6a80b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 --2- 172.21.15.59:0/600324093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6a806dcb0 0x7fa6a80cbc90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 >> 172.21.15.59:0/600324093 conn(0x7fa6a806bdf0 msgr2=0x7fa6a80af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.562+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 shutdown_connections 2024-04-24T06:38:32.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:32.563+0000 7fa6a7e00640 1 -- 172.21.15.59:0/600324093 wait complete. 2024-04-24T06:38:32.583 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:38:32.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:38:32.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:38:32.810 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:38:32.810 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:32.810 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:32.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:38:32.871 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:38:32.871 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:32.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:38:32.933 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:32.933 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:32.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:38:32.992 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:38:32.992 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:33.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:38:33.054 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:38:33.055 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.266+0000 7fc7e7200640 1 Processor -- start 2024-04-24T06:38:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.266+0000 7fc7e7200640 1 -- start start 2024-04-24T06:38:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc7e00cc7e0 0x7fc7e0167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e0160210 con 0x7fc7e00cd760 2024-04-24T06:38:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e01604f0 con 0x7fc7e015c3d0 2024-04-24T06:38:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e01607d0 con 0x7fc7e00cc7e0 2024-04-24T06:38:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e5400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7e00cd760 0x7fc7e00b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49658/0 (socket says 172.21.15.59:49658) 2024-04-24T06:38:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.268+0000 7fc7e5400640 1 -- 172.21.15.59:0/517704760 learned_addr learned my addr 172.21.15.59:0/517704760 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800112739 0 0) 0x7fc7e0160210 con 0x7fc7e00cd760 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7c8003520 con 0x7fc7e00cd760 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e6800640 1 --2- 172.21.15.59:0/517704760 >> v2:172.21.15.66:3300/0 conn(0x7fc7e00cc7e0 0x7fc7e0167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3123286245 0 0) 0x7fc7c8003520 con 0x7fc7e00cd760 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 >> v2:172.21.15.66:3300/0 conn(0x7fc7e00cc7e0 msgr2=0x7fc7e0167400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 --2- 172.21.15.59:0/517704760 >> v2:172.21.15.66:3300/0 conn(0x7fc7e00cc7e0 0x7fc7e0167400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 >> v1:172.21.15.66:6789/0 conn(0x7fc7e015c3d0 legacy=0x7fc7e00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e0160ab0 con 0x7fc7e00cd760 2024-04-24T06:38:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fc7d0001670 con 0x7fc7e00cd760 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 >> v1:172.21.15.59:6789/0 conn(0x7fc7e00cd760 legacy=0x7fc7e00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c80051c0 0x7fc7c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01607d0 con 0x7fc7c80039b0 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c800c110 con 0x7fc7c8004800 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.269+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c800c310 con 0x7fc7c80051c0 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5400640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e6800640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c80051c0 0x7fc7c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 -- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c80051c0 msgr2=0x7fc7c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:33.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7c80051c0 0x7fc7c800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 -- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c8004800 msgr2=0x7fc7c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:33.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e5e00640 1 -- 172.21.15.59:0/517704760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c8003520 con 0x7fc7c80039b0 2024-04-24T06:38:33.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.270+0000 7fc7e6800640 1 -- 172.21.15.59:0/517704760 reap_dead start 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.272+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4001750 tx=0x7fc7c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97a45c1cf0d8d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.272+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4001370 con 0x7fc7c80039b0 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.272+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d40052d0 con 0x7fc7c80039b0 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.272+0000 7fc7e4a00640 1 -- 172.21.15.59:0/517704760 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d402abe0 con 0x7fc7c80039b0 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 -- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 msgr2=0x7fc7c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4001750 tx=0x7fc7c8005830 comp rx=0 tx=0).stop 2024-04-24T06:38:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 -- 172.21.15.59:0/517704760 shutdown_connections 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 --2- 172.21.15.59:0/517704760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 -- 172.21.15.59:0/517704760 >> 172.21.15.59:0/517704760 conn(0x7fc7e00b21a0 msgr2=0x7fc7e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 -- 172.21.15.59:0/517704760 shutdown_connections 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.273+0000 7fc7e7200640 1 -- 172.21.15.59:0/517704760 wait complete. 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.275+0000 7fc7e7200640 1 Processor -- start 2024-04-24T06:38:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.276+0000 7fc7e7200640 1 -- start start 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 0x7fc7e01e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7e01f2200 0x7fc7e01e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e015c930 con 0x7fc7e01570d0 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e015c7b0 con 0x7fc7c80039b0 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00cd050 con 0x7fc7e01f2200 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34444/0 (socket says 172.21.15.59:34444) 2024-04-24T06:38:33.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.277+0000 7fc7e6800640 1 -- 172.21.15.59:0/1390758767 learned_addr learned my addr 172.21.15.59:0/1390758767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 0x7fc7e01e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e5400640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7e01f2200 0x7fc7e01e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 -- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7e01f2200 msgr2=0x7fc7e01e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7e01f2200 0x7fc7e01e63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 -- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 msgr2=0x7fc7e01e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 0x7fc7e01e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 -- 172.21.15.59:0/1390758767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01564b0 con 0x7fc7e01570d0 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 0x7fc7e01e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e6800640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00cae30 tx=0x7fc7dc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc019070 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc007d40 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01e6a10 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.278+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc014640 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.279+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e015cb20 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.280+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e004fd90 con 0x7fc7e01570d0 2024-04-24T06:38:33.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.281+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7dc005d10 con 0x7fc7e01570d0 2024-04-24T06:38:33.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.282+0000 7fc7da000640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 0x7fc7c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.282+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fc7dc08b420 con 0x7fc7e01570d0 2024-04-24T06:38:33.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.282+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 0x7fc7c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.283+0000 7fc7e5e00640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 0x7fc7c406ede0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4032000 tx=0x7fc7d402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.286+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7dc059a10 con 0x7fc7e01570d0 2024-04-24T06:38:33.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.531+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e006e400 con 0x7fc7e01570d0 2024-04-24T06:38:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.532+0000 7fc7da000640 1 -- 172.21.15.59:0/1390758767 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3863 (secure 0 0 0) 0x7fc7dc059310 con 0x7fc7e01570d0 2024-04-24T06:38:33.534 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:33.534 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/702923672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":702923672},{"type":"v1","addr":"172.21.15.59:6837","nonce":702923672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2166946357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2166946357},{"type":"v1","addr":"172.21.15.66:6835","nonce":2166946357}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/3070215916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3070215916},{"type":"v1","addr":"172.21.15.66:6833","nonce":3070215916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":8415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/846644261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":846644261},{"type":"v1","addr":"172.21.15.59:6835","nonce":846644261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-04-24T06:38:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 77 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.534+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 msgr2=0x7fc7c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.534+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 0x7fc7c406ede0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4032000 tx=0x7fc7d402e000 comp rx=0 tx=0).stop 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.534+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 msgr2=0x7fc7e0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.534+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00cae30 tx=0x7fc7dc009d20 comp rx=0 tx=0).stop 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 shutdown_connections 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc7c406c920 0x7fc7c406ede0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e01570d0 0x7fc7e0157580 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc7e01f2200 0x7fc7e01e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 --2- 172.21.15.59:0/1390758767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80039b0 0x7fc7e01e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 >> 172.21.15.59:0/1390758767 conn(0x7fc7e00b21a0 msgr2=0x7fc7e00b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 shutdown_connections 2024-04-24T06:38:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.535+0000 7fc7e7200640 1 -- 172.21.15.59:0/1390758767 wait complete. 2024-04-24T06:38:33.555 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:38:33.556 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:38:33.556 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:38:33.556 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:38:33.556 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:38:33.556 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:38:33.556 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:38:33.556 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:38:33.557 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:38:33.557 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:38:33.557 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:38:33.557 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:38:33.557 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:38:33.557 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:38:33.558 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:38:33.558 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:38:33.576 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:38:33.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:38:33.579 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:38:33.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:38:33.581 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:38:33.581 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:38:33.588 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:38:33.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:38:33.624 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:38:33.625 DEBUG:teuthology.parallel:result is None 2024-04-24T06:38:33.629 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:38:33.629 DEBUG:teuthology.parallel:result is None 2024-04-24T06:38:33.634 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:38:33.635 DEBUG:teuthology.parallel:result is None 2024-04-24T06:38:33.641 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:38:33.641 DEBUG:teuthology.parallel:result is None 2024-04-24T06:38:33.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:38:33.773 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:38:33.775 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:38:33.805 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:38:33.807 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.817+0000 7f61ba800640 1 Processor -- start 2024-04-24T06:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.817+0000 7f61ba800640 1 -- start start 2024-04-24T06:38:33.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61ba800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 0x7f61b40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61ba800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b415ea40 0x7f61b415f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61ba800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b40cf930 con 0x7f61b40d1360 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61ba800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f61b415fdc0 con 0x7f61b415ea40 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61ba800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f61b40cfc10 con 0x7f61b40d0f80 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61b8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b415ea40 0x7f61b415f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61b8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b415ea40 0x7f61b415f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34486/0 (socket says 172.21.15.59:34486) 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.818+0000 7f61b8a00640 1 -- 172.21.15.59:0/864675903 learned_addr learned my addr 172.21.15.59:0/864675903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b9400640 1 --2- 172.21.15.59:0/864675903 >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 0x7f61b40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b8a00640 1 -- 172.21.15.59:0/864675903 >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 msgr2=0x7f61b40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b8a00640 1 --2- 172.21.15.59:0/864675903 >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 0x7f61b40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b8a00640 1 -- 172.21.15.59:0/864675903 >> v1:172.21.15.66:6789/0 conn(0x7f61b40d1360 legacy=0x7f61b4162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b8a00640 1 -- 172.21.15.59:0/864675903 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b40cfef0 con 0x7f61b415ea40 2024-04-24T06:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.819+0000 7f61b9400640 1 --2- 172.21.15.59:0/864675903 >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 0x7f61b40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.820+0000 7f61b8a00640 1 --2- 172.21.15.59:0/864675903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b415ea40 0x7f61b415f880 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f61b40d1e90 tx=0x7f61a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22c0e292bfe6a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.820+0000 7f61b3e00640 1 -- 172.21.15.59:0/864675903 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0037070 con 0x7f61b415ea40 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.820+0000 7f61b3e00640 1 -- 172.21.15.59:0/864675903 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a0032440 con 0x7f61b415ea40 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.820+0000 7f61b3e00640 1 -- 172.21.15.59:0/864675903 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0032810 con 0x7f61b415ea40 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 -- 172.21.15.59:0/864675903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b415ea40 msgr2=0x7f61b415f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 --2- 172.21.15.59:0/864675903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b415ea40 0x7f61b415f880 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f61b40d1e90 tx=0x7f61a002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 -- 172.21.15.59:0/864675903 shutdown_connections 2024-04-24T06:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 --2- 172.21.15.59:0/864675903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b415ea40 0x7f61b415f880 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 --2- 172.21.15.59:0/864675903 >> v2:172.21.15.66:3300/0 conn(0x7f61b40d0f80 0x7f61b40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 -- 172.21.15.59:0/864675903 >> 172.21.15.59:0/864675903 conn(0x7f61b408bf90 msgr2=0x7f61b40d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 -- 172.21.15.59:0/864675903 shutdown_connections 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.821+0000 7f61ba800640 1 -- 172.21.15.59:0/864675903 wait complete. 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.823+0000 7f61ba800640 1 Processor -- start 2024-04-24T06:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.823+0000 7f61ba800640 1 -- start start 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b40d1360 0x7f61b40b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61b415ea40 0x7f61b40b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61b40d1810 con 0x7f61b40d0f80 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61b40d04b0 con 0x7f61b40d1360 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61ba800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f61b40d3320 con 0x7f61b415ea40 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34494/0 (socket says 172.21.15.59:34494) 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 -- 172.21.15.59:0/3178594494 learned_addr learned my addr 172.21.15.59:0/3178594494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b8a00640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b40d1360 0x7f61b40b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9e00640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61b415ea40 0x7f61b40b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 -- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61b415ea40 msgr2=0x7f61b40b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61b415ea40 0x7f61b40b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 -- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b40d1360 msgr2=0x7f61b40b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b40d1360 0x7f61b40b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.824+0000 7f61b9400640 1 -- 172.21.15.59:0/3178594494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b40c9be0 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61b9400640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c7f0 tx=0x7f61a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8015070 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8007a30 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8007e00 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b40c9ec0 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.825+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b40ca760 con 0x7f61b40d0f80 2024-04-24T06:38:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.826+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b40ca090 con 0x7f61b40d0f80 2024-04-24T06:38:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.827+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61a8018070 con 0x7f61b40d0f80 2024-04-24T06:38:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.828+0000 7f61b1600640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.828+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f61a808b900 con 0x7f61b40d0f80 2024-04-24T06:38:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.828+0000 7f61b8a00640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.829+0000 7f61b8a00640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f61a0034000 tx=0x7f61a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:33.833+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61a8059890 con 0x7f61b40d0f80 2024-04-24T06:38:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.041+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f61b404fd90 con 0x7f61b40d0f80 2024-04-24T06:38:34.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.108+0000 7f61b1600640 1 -- 172.21.15.59:0/3178594494 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) ==== 97+0+0 (secure 0 0 0) 0x7f61a805db90 con 0x7f61b40d0f80 2024-04-24T06:38:34.110 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.111+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 msgr2=0x7f619406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f61a0034000 tx=0x7f61a002e000 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 msgr2=0x7f61b40ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c7f0 tx=0x7f61a800cbf0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 shutdown_connections 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f61b415ea40 0x7f61b40b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61b40d1360 0x7f61b40b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 --2- 172.21.15.59:0/3178594494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b40d0f80 0x7f61b40ce2f0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 >> 172.21.15.59:0/3178594494 conn(0x7f61b408bf90 msgr2=0x7f61b4161630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 shutdown_connections 2024-04-24T06:38:34.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.112+0000 7f61ba800640 1 -- 172.21.15.59:0/3178594494 wait complete. 2024-04-24T06:38:34.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:38:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.336+0000 7f9d87200640 1 Processor -- start 2024-04-24T06:38:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.336+0000 7f9d87200640 1 -- start start 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.337+0000 7f9d87200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d8815c3d0 0x7f9d880cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.337+0000 7f9d87200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 0x7f9d880b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.337+0000 7f9d87200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d88160210 con 0x7f9d880b4690 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.337+0000 7f9d87200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9d881604f0 con 0x7f9d8815c3d0 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.337+0000 7f9d87200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9d881607d0 con 0x7f9d880cd590 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d8815c3d0 0x7f9d880cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d8815c3d0 0x7f9d880cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34496/0 (socket says 172.21.15.59:34496) 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 -- 172.21.15.59:0/3968766731 learned_addr learned my addr 172.21.15.59:0/3968766731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d7fe00640 1 --2- 172.21.15.59:0/3968766731 >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 0x7f9d880b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 -- 172.21.15.59:0/3968766731 >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 msgr2=0x7f9d880b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 --2- 172.21.15.59:0/3968766731 >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 0x7f9d880b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 -- 172.21.15.59:0/3968766731 >> v1:172.21.15.66:6789/0 conn(0x7f9d880b4690 legacy=0x7f9d8806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d85e00640 1 -- 172.21.15.59:0/3968766731 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d88160ab0 con 0x7f9d8815c3d0 2024-04-24T06:38:34.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.338+0000 7f9d7fe00640 1 --2- 172.21.15.59:0/3968766731 >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 0x7f9d880b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.339+0000 7f9d85e00640 1 --2- 172.21.15.59:0/3968766731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cd050 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9d880cff90 tx=0x7f9d7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33c5285ade288b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.339+0000 7f9d85400640 1 -- 172.21.15.59:0/3968766731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78037070 con 0x7f9d8815c3d0 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.339+0000 7f9d85400640 1 -- 172.21.15.59:0/3968766731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d78032440 con 0x7f9d8815c3d0 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.339+0000 7f9d85400640 1 -- 172.21.15.59:0/3968766731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78032810 con 0x7f9d8815c3d0 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 -- 172.21.15.59:0/3968766731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d8815c3d0 msgr2=0x7f9d880cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 --2- 172.21.15.59:0/3968766731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cd050 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9d880cff90 tx=0x7f9d7802a230 comp rx=0 tx=0).stop 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 -- 172.21.15.59:0/3968766731 shutdown_connections 2024-04-24T06:38:34.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 --2- 172.21.15.59:0/3968766731 >> v2:172.21.15.66:3300/0 conn(0x7f9d880cd590 0x7f9d880b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 --2- 172.21.15.59:0/3968766731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cd050 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 -- 172.21.15.59:0/3968766731 >> 172.21.15.59:0/3968766731 conn(0x7f9d880b21a0 msgr2=0x7f9d880b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 -- 172.21.15.59:0/3968766731 shutdown_connections 2024-04-24T06:38:34.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.340+0000 7f9d87200640 1 -- 172.21.15.59:0/3968766731 wait complete. 2024-04-24T06:38:34.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.342+0000 7f9d87200640 1 Processor -- start 2024-04-24T06:38:34.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.342+0000 7f9d87200640 1 -- start start 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9d880cd590 0x7f9d880cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8815c7b0 con 0x7f9d880b4690 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b4ac0 con 0x7f9d8815c3d0 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d87200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8815c930 con 0x7f9d880cd590 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34512/0 (socket says 172.21.15.59:34512) 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 -- 172.21.15.59:0/3616190547 learned_addr learned my addr 172.21.15.59:0/3616190547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 -- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9d880cd590 msgr2=0x7f9d880cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9d880cd590 0x7f9d880cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 -- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815c3d0 msgr2=0x7f9d880cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.344+0000 7f9d85e00640 1 -- 172.21.15.59:0/3616190547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d880c98b0 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d85e00640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78041020 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d780306c0 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78037040 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d880c9b90 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.345+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d880ca150 con 0x7f9d880b4690 2024-04-24T06:38:34.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.346+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8804fd90 con 0x7f9d880b4690 2024-04-24T06:38:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.347+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d78044020 con 0x7f9d880b4690 2024-04-24T06:38:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.348+0000 7f9d7e000640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 0x7f9d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.348+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4857+0+0 (secure 0 0 0) 0x7f9d780ac090 con 0x7f9d880b4690 2024-04-24T06:38:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.349+0000 7f9d7fe00640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 0x7f9d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.349+0000 7f9d7fe00640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 0x7f9d6406edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9d70006e80 tx=0x7f9d70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.353+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d7804b030 con 0x7f9d880b4690 2024-04-24T06:38:34.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.622+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9d8806e400 con 0x7f9d880b4690 2024-04-24T06:38:34.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.624+0000 7f9d7e000640 1 -- 172.21.15.59:0/3616190547 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f9d7807a800 con 0x7f9d880b4690 2024-04-24T06:38:34.625 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:38:34.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:38:34.628 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.629 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.631 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:38:34.633 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.634 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:38:34.636 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:38:34.638 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:38:34.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:34.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:38:34.641 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:38:34.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 msgr2=0x7f9d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 0x7f9d6406edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9d70006e80 tx=0x7f9d70008040 comp rx=0 tx=0).stop 2024-04-24T06:38:34.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 msgr2=0x7f9d880c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d780046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 shutdown_connections 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9d6406c8f0 0x7f9d6406edb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d8815c3d0 0x7f9d880cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9d880cd590 0x7f9d880cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 --2- 172.21.15.59:0/3616190547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d880b4690 0x7f9d880c5fe0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 >> 172.21.15.59:0/3616190547 conn(0x7f9d880b21a0 msgr2=0x7f9d8806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 shutdown_connections 2024-04-24T06:38:34.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.626+0000 7f9d87200640 1 -- 172.21.15.59:0/3616190547 wait complete. 2024-04-24T06:38:34.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.836+0000 7fbf65800640 1 Processor -- start 2024-04-24T06:38:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.836+0000 7fbf65800640 1 -- start start 2024-04-24T06:38:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf65800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbf6015e7c0 0x7fbf600d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf65800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf600d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf65800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf600cf030 con 0x7fbf600d2630 2024-04-24T06:38:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf65800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbf600d1740 con 0x7fbf6015e7c0 2024-04-24T06:38:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf65800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf600b7000 con 0x7fbf600d0d90 2024-04-24T06:38:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf5f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbf600d2630 0x7fbf600b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49664/0 (socket says 172.21.15.59:49664) 2024-04-24T06:38:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.838+0000 7fbf5f400640 1 -- 172.21.15.59:0/2693868763 learned_addr learned my addr 172.21.15.59:0/2693868763 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf64e00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf600d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5fe00640 1 --2- 172.21.15.59:0/2693868763 >> v2:172.21.15.66:3300/0 conn(0x7fbf6015e7c0 0x7fbf600d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364508478 0 0) 0x7fbf4c001110 con 0x7fbf600d2630 2024-04-24T06:38:34.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf44003520 con 0x7fbf600d2630 2024-04-24T06:38:34.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2423425782 0 0) 0x7fbf44003520 con 0x7fbf600d2630 2024-04-24T06:38:34.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 msgr2=0x7fbf600d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf600d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 >> v2:172.21.15.66:3300/0 conn(0x7fbf6015e7c0 msgr2=0x7fbf600d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 --2- 172.21.15.59:0/2693868763 >> v2:172.21.15.66:3300/0 conn(0x7fbf6015e7c0 0x7fbf600d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600b6a40 con 0x7fbf600d2630 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fbf4c001840 con 0x7fbf600d2630 2024-04-24T06:38:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf64e00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf600d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.839+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 >> v1:172.21.15.59:6789/0 conn(0x7fbf600d2630 legacy=0x7fbf600b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf440039b0 0x7fbf44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf44004800 0x7fbf44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600d1740 con 0x7fbf44004800 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600b7000 con 0x7fbf440039b0 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4400b9f0 con 0x7fbf600d0d90 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf44004800 0x7fbf44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 -- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 msgr2=0x7fbf440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf600d0d90 0x7fbf440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 -- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf440039b0 msgr2=0x7fbf44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf440039b0 0x7fbf44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.840+0000 7fbf5f400640 1 -- 172.21.15.59:0/2693868763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf44003520 con 0x7fbf44004800 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.841+0000 7fbf5f400640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf44004800 0x7fbf44004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c2f0 tx=0x7fbf44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75137786cdcc625c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c035070 con 0x7fbf44004800 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c007e50 con 0x7fbf44004800 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf5ea00640 1 -- 172.21.15.59:0/2693868763 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c005ad0 con 0x7fbf44004800 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf65800640 1 -- 172.21.15.59:0/2693868763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf44004800 msgr2=0x7fbf44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf65800640 1 --2- 172.21.15.59:0/2693868763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf44004800 0x7fbf44004c80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c2f0 tx=0x7fbf44005830 comp rx=0 tx=0).stop 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.842+0000 7fbf64e00640 1 -- 172.21.15.59:0/2693868763 reap_dead start 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.843+0000 7fbf65800640 1 -- 172.21.15.59:0/2693868763 shutdown_connections 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.843+0000 7fbf65800640 1 -- 172.21.15.59:0/2693868763 >> 172.21.15.59:0/2693868763 conn(0x7fbf6009d920 msgr2=0x7fbf6008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.843+0000 7fbf65800640 1 -- 172.21.15.59:0/2693868763 shutdown_connections 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.843+0000 7fbf65800640 1 -- 172.21.15.59:0/2693868763 wait complete. 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.844+0000 7fbf65800640 1 Processor -- start 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.845+0000 7fbf65800640 1 -- start start 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf44004800 0x7fbf6015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf600b56a0 0x7fbf600b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600cf2b0 con 0x7fbf600b1b50 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf600cf620 con 0x7fbf600b56a0 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf65800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf600b6700 con 0x7fbf44004800 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34528/0 (socket says 172.21.15.59:34528) 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 -- 172.21.15.59:0/2328127129 learned_addr learned my addr 172.21.15.59:0/2328127129 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf5fe00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf44004800 0x7fbf6015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf5f400640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf600b56a0 0x7fbf600b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 -- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf44004800 msgr2=0x7fbf6015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf44004800 0x7fbf6015dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 -- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf600b56a0 msgr2=0x7fbf600b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf600b56a0 0x7fbf600b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.846+0000 7fbf64e00640 1 -- 172.21.15.59:0/2328127129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf600c8940 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf64e00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fbf600d0b30 tx=0x7fbf5800e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5801a070 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58004250 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800f040 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf600b4a80 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.847+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf600b5040 con 0x7fbf600b1b50 2024-04-24T06:38:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.848+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6004fd90 con 0x7fbf600b1b50 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.850+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf5801d070 con 0x7fbf600b1b50 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.851+0000 7fbf57e00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 0x7fbf4006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.851+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4857+0+0 (secure 0 0 0) 0x7fbf5808a580 con 0x7fbf600b1b50 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.851+0000 7fbf5fe00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 0x7fbf4006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.852+0000 7fbf5fe00640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 0x7fbf4006ee00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf50004b30 tx=0x7fbf5000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:34.856+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf58051b80 con 0x7fbf600b1b50 2024-04-24T06:38:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.063+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf600d1eb0 con 0x7fbf600b1b50 2024-04-24T06:38:35.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.065+0000 7fbf57e00640 1 -- 172.21.15.59:0/2328127129 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+12808 (secure 0 0 0) 0x7fbf58058c50 con 0x7fbf600b1b50 2024-04-24T06:38:35.067 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:35.067 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":94,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:34.129780+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 msgr2=0x7fbf4006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 0x7fbf4006ee00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf50004b30 tx=0x7fbf5000b040 comp rx=0 tx=0).stop 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 msgr2=0x7fbf600b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fbf600d0b30 tx=0x7fbf5800e8b0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 shutdown_connections 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf4006c940 0x7fbf4006ee00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf600b1b50 0x7fbf600b1fd0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf600b56a0 0x7fbf600b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 --2- 172.21.15.59:0/2328127129 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf44004800 0x7fbf6015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 >> 172.21.15.59:0/2328127129 conn(0x7fbf6009d920 msgr2=0x7fbf60160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 shutdown_connections 2024-04-24T06:38:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.068+0000 7fbf65800640 1 -- 172.21.15.59:0/2328127129 wait complete. 2024-04-24T06:38:35.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:35.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.289+0000 7fd0df200640 1 Processor -- start 2024-04-24T06:38:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.289+0000 7fd0df200640 1 -- start start 2024-04-24T06:38:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.290+0000 7fd0df200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0e0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.290+0000 7fd0df200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd0e015d810 0x7fd0e0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.290+0000 7fd0df200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e015cab0 con 0x7fd0e01567e0 2024-04-24T06:38:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.290+0000 7fd0df200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e0160e40 con 0x7fd0e015d810 2024-04-24T06:38:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.290+0000 7fd0df200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e015cd30 con 0x7fd0e00d0d10 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.291+0000 7fd0dde00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd0e01567e0 0x7fd0e00d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49674/0 (socket says 172.21.15.59:49674) 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.291+0000 7fd0dde00640 1 -- 172.21.15.59:0/1919578512 learned_addr learned my addr 172.21.15.59:0/1919578512 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.291+0000 7fd0de800640 1 --2- 172.21.15.59:0/1919578512 >> v2:172.21.15.59:3300/0 conn(0x7fd0e015d810 0x7fd0e0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.291+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0e0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3488850015 0 0) 0x7fd0d0001110 con 0x7fd0e01567e0 2024-04-24T06:38:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c4003520 con 0x7fd0e01567e0 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4065351848 0 0) 0x7fd0c4003520 con 0x7fd0e01567e0 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 msgr2=0x7fd0e0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0e0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 >> v2:172.21.15.59:3300/0 conn(0x7fd0e015d810 msgr2=0x7fd0e0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 --2- 172.21.15.59:0/1919578512 >> v2:172.21.15.59:3300/0 conn(0x7fd0e015d810 0x7fd0e0160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e015d010 con 0x7fd0e01567e0 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd0d0001840 con 0x7fd0e01567e0 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 >> v1:172.21.15.59:6789/0 conn(0x7fd0e01567e0 legacy=0x7fd0e00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0c40039b0 0x7fd0c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0c4004800 0x7fd0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0160e40 con 0x7fd0c40039b0 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e015cd30 con 0x7fd0c4004800 2024-04-24T06:38:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c400b9c0 con 0x7fd0e00d0d10 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.292+0000 7fd0dde00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0c40039b0 0x7fd0c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0de800640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0c4004800 0x7fd0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0dde00640 1 -- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 msgr2=0x7fd0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0dde00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0dde00640 1 -- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0c4004800 msgr2=0x7fd0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0dde00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0c4004800 0x7fd0c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0dde00640 1 -- 172.21.15.59:0/1919578512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c4003520 con 0x7fd0c40039b0 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0de800640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0e00d0d10 0x7fd0c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.293+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0c4004800 0x7fd0c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.294+0000 7fd0dde00640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0c40039b0 0x7fd0c4003dd0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c4c0 tx=0x7fd0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f8b0add24f873e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000c0e0 con 0x7fd0c40039b0 2024-04-24T06:38:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0034070 con 0x7fd0c40039b0 2024-04-24T06:38:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0dd400640 1 -- 172.21.15.59:0/1919578512 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0005d10 con 0x7fd0c40039b0 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0df200640 1 -- 172.21.15.59:0/1919578512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0c40039b0 msgr2=0x7fd0c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0df200640 1 --2- 172.21.15.59:0/1919578512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0c40039b0 0x7fd0c4003dd0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c4c0 tx=0x7fd0c4005830 comp rx=0 tx=0).stop 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0de800640 1 -- 172.21.15.59:0/1919578512 reap_dead start 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0df200640 1 -- 172.21.15.59:0/1919578512 shutdown_connections 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.295+0000 7fd0df200640 1 -- 172.21.15.59:0/1919578512 >> 172.21.15.59:0/1919578512 conn(0x7fd0e006bdf0 msgr2=0x7fd0e006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.296+0000 7fd0df200640 1 -- 172.21.15.59:0/1919578512 shutdown_connections 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.296+0000 7fd0df200640 1 -- 172.21.15.59:0/1919578512 wait complete. 2024-04-24T06:38:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.297+0000 7fd0df200640 1 Processor -- start 2024-04-24T06:38:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.298+0000 7fd0df200640 1 -- start start 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0c40039b0 0x7fd0e0158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0e01f2720 0x7fd0e01f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0156d40 con 0x7fd0e014de10 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0156bc0 con 0x7fd0e01f2720 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0df200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e0156ec0 con 0x7fd0c40039b0 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34566/0 (socket says 172.21.15.59:34566) 2024-04-24T06:38:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 -- 172.21.15.59:0/2090710446 learned_addr learned my addr 172.21.15.59:0/2090710446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0dde00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0c40039b0 0x7fd0e0158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 -- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0c40039b0 msgr2=0x7fd0e0158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0c40039b0 0x7fd0e0158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 -- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0e01f2720 msgr2=0x7fd0e01f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0e01f2720 0x7fd0e01f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.299+0000 7fd0d7e00640 1 -- 172.21.15.59:0/2090710446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e014e7d0 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0d7e00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80108e0 tx=0x7fd0c8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800f040 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c800eba0 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c801c400 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e01591b0 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.300+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e0159770 con 0x7fd0e014de10 2024-04-24T06:38:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.301+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e004fd90 con 0x7fd0e014de10 2024-04-24T06:38:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.302+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0c801c5a0 con 0x7fd0e014de10 2024-04-24T06:38:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.303+0000 7fd0d6000640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.303+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4857+0+0 (secure 0 0 0) 0x7fd0c808bf00 con 0x7fd0e014de10 2024-04-24T06:38:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.303+0000 7fd0dde00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.304+0000 7fd0dde00640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0033000 tx=0x7fd0d0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.308+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0c8018030 con 0x7fd0e014de10 2024-04-24T06:38:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.515+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0e00b2c10 con 0x7fd0e014de10 2024-04-24T06:38:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.516+0000 7fd0d6000640 1 -- 172.21.15.59:0/2090710446 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd0c805a020 con 0x7fd0e014de10 2024-04-24T06:38:35.518 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 msgr2=0x7fd0bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0033000 tx=0x7fd0d0009030 comp rx=0 tx=0).stop 2024-04-24T06:38:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 msgr2=0x7fd0e014e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80108e0 tx=0x7fd0c8010da0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 shutdown_connections 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0e01f2720 0x7fd0e01f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e014de10 0x7fd0e014e290 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 --2- 172.21.15.59:0/2090710446 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd0c40039b0 0x7fd0e0158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 >> 172.21.15.59:0/2090710446 conn(0x7fd0e006bdf0 msgr2=0x7fd0e0073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 shutdown_connections 2024-04-24T06:38:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.519+0000 7fd0df200640 1 -- 172.21.15.59:0/2090710446 wait complete. 2024-04-24T06:38:35.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:38:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.750+0000 7f2b76c00640 1 Processor -- start 2024-04-24T06:38:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.751+0000 7f2b76c00640 1 -- start start 2024-04-24T06:38:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.752+0000 7f2b76c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7015e7c0 0x7f2b700d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.752+0000 7f2b76c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b700d2630 0x7f2b700b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.752+0000 7f2b76c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b700cf030 con 0x7f2b700b6530 2024-04-24T06:38:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.752+0000 7f2b76c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b700b7000 con 0x7f2b7015e7c0 2024-04-24T06:38:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.752+0000 7f2b76c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2b700cf620 con 0x7f2b700d2630 2024-04-24T06:38:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b75800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7015e7c0 0x7f2b700d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b76200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2b700b6530 0x7f2b700d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49682/0 (socket says 172.21.15.59:49682) 2024-04-24T06:38:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b76200640 1 -- 172.21.15.59:0/2274586285 learned_addr learned my addr 172.21.15.59:0/2274586285 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b74e00640 1 --2- 172.21.15.59:0/2274586285 >> v2:172.21.15.66:3300/0 conn(0x7f2b700d2630 0x7f2b700b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b75800640 1 -- 172.21.15.59:0/2274586285 >> v2:172.21.15.66:3300/0 conn(0x7f2b700d2630 msgr2=0x7f2b700b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b75800640 1 --2- 172.21.15.59:0/2274586285 >> v2:172.21.15.66:3300/0 conn(0x7f2b700d2630 0x7f2b700b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b75800640 1 -- 172.21.15.59:0/2274586285 >> v1:172.21.15.59:6789/0 conn(0x7f2b700b6530 legacy=0x7f2b700d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.753+0000 7f2b75800640 1 -- 172.21.15.59:0/2274586285 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b700b6a40 con 0x7f2b7015e7c0 2024-04-24T06:38:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b75800640 1 --2- 172.21.15.59:0/2274586285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700d20f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2b700d55c0 tx=0x7f2b6402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfd11b9773a60aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b6fe00640 1 -- 172.21.15.59:0/2274586285 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64037070 con 0x7f2b7015e7c0 2024-04-24T06:38:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b6fe00640 1 -- 172.21.15.59:0/2274586285 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b64032440 con 0x7f2b7015e7c0 2024-04-24T06:38:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b6fe00640 1 -- 172.21.15.59:0/2274586285 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64032810 con 0x7f2b7015e7c0 2024-04-24T06:38:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b76c00640 1 -- 172.21.15.59:0/2274586285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 msgr2=0x7f2b700d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.755+0000 7f2b76c00640 1 --2- 172.21.15.59:0/2274586285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700d20f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2b700d55c0 tx=0x7f2b6402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 -- 172.21.15.59:0/2274586285 shutdown_connections 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 --2- 172.21.15.59:0/2274586285 >> v2:172.21.15.66:3300/0 conn(0x7f2b700d2630 0x7f2b700b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 --2- 172.21.15.59:0/2274586285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700d20f0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 -- 172.21.15.59:0/2274586285 >> 172.21.15.59:0/2274586285 conn(0x7f2b7009d920 msgr2=0x7f2b7008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 -- 172.21.15.59:0/2274586285 shutdown_connections 2024-04-24T06:38:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.756+0000 7f2b76c00640 1 -- 172.21.15.59:0/2274586285 wait complete. 2024-04-24T06:38:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.757+0000 7f2b76c00640 1 Processor -- start 2024-04-24T06:38:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.757+0000 7f2b76c00640 1 -- start start 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 0x7f2b700ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b700d2630 0x7f2b700cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700d1a80 con 0x7f2b7015e7c0 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700cf2b0 con 0x7f2b700b6530 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.758+0000 7f2b76c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2b700d1c00 con 0x7f2b700d2630 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34596/0 (socket says 172.21.15.59:34596) 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 -- 172.21.15.59:0/915193394 learned_addr learned my addr 172.21.15.59:0/915193394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b74e00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b700d2630 0x7f2b700cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b75800640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 0x7f2b700ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 -- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b700d2630 msgr2=0x7f2b700cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b700d2630 0x7f2b700cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 -- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 msgr2=0x7f2b700ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 0x7f2b700ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 -- 172.21.15.59:0/915193394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b701e9430 con 0x7f2b7015e7c0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b75800640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 0x7f2b700ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.759+0000 7f2b76200640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800cad0 tx=0x7f2b6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.760+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68017070 con 0x7f2b7015e7c0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.760+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6800ac30 con 0x7f2b7015e7c0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.760+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701e9710 con 0x7f2b7015e7c0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.760+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68012620 con 0x7f2b7015e7c0 2024-04-24T06:38:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.760+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b701e99f0 con 0x7f2b7015e7c0 2024-04-24T06:38:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.761+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7004fd90 con 0x7f2b7015e7c0 2024-04-24T06:38:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.762+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b6801a070 con 0x7f2b7015e7c0 2024-04-24T06:38:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.763+0000 7f2b6d600640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.764+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4857+0+0 (secure 0 0 0) 0x7f2b6808b540 con 0x7f2b7015e7c0 2024-04-24T06:38:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.764+0000 7f2b75800640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:35.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.765+0000 7f2b75800640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b64034000 tx=0x7f2b6402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.768+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b700cf030 con 0x7f2b7015e7c0 2024-04-24T06:38:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.973+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2b700d1eb0 con 0x7f2b7015e7c0 2024-04-24T06:38:35.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.974+0000 7f2b6d600640 1 -- 172.21.15.59:0/915193394 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) ==== 84+0+65 (secure 0 0 0) 0x7f2b6800ae20 con 0x7f2b7015e7c0 2024-04-24T06:38:35.975 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:35.976 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:38:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.976+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 msgr2=0x7f2b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.976+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b64034000 tx=0x7f2b6402e000 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.976+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 msgr2=0x7f2b700b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.976+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800cad0 tx=0x7f2b6800cf90 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 shutdown_connections 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7015e7c0 0x7f2b700b21f0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b700d2630 0x7f2b700cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 --2- 172.21.15.59:0/915193394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b700b6530 0x7f2b700ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 >> 172.21.15.59:0/915193394 conn(0x7f2b7009d920 msgr2=0x7f2b7006cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 shutdown_connections 2024-04-24T06:38:35.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:35.977+0000 7f2b76c00640 1 -- 172.21.15.59:0/915193394 wait complete. 2024-04-24T06:38:35.999 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:38:35.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:38:36.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.216+0000 7fa693200640 1 Processor -- start 2024-04-24T06:38:36.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.216+0000 7fa693200640 1 -- start start 2024-04-24T06:38:36.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.217+0000 7fa693200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 0x7fa69406c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa693200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa694157000 con 0x7fa694160100 2024-04-24T06:38:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa693200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6940afb00 con 0x7fa694156c20 2024-04-24T06:38:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa693200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa69415cc20 con 0x7fa69406bdf0 2024-04-24T06:38:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa692800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa694160100 0x7fa694160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49684/0 (socket says 172.21.15.59:49684) 2024-04-24T06:38:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa692800640 1 -- 172.21.15.59:0/2816216931 learned_addr learned my addr 172.21.15.59:0/2816216931 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa68be00640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 0x7fa69406c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3676623052 0 0) 0x7fa694157000 con 0x7fa694160100 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.218+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa678003520 con 0x7fa694160100 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3750531584 0 0) 0x7fa678003520 con 0x7fa694160100 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 msgr2=0x7fa69406c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 0x7fa69406c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 >> v1:172.21.15.66:6789/0 conn(0x7fa694156c20 legacy=0x7fa6940aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69415cf00 con 0x7fa694160100 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa68c0019f0 con 0x7fa694160100 2024-04-24T06:38:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 >> v1:172.21.15.59:6789/0 conn(0x7fa694160100 legacy=0x7fa694160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa678003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 0x7fa678004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415cc20 con 0x7fa6780039e0 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa67800ba70 con 0x7fa678004d40 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa67800bc70 con 0x7fa69406bdf0 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.219+0000 7fa691e00640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa678003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.220+0000 7fa691e00640 1 -- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 msgr2=0x7fa678004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.220+0000 7fa691e00640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406bdf0 0x7fa678004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.220+0000 7fa691e00640 1 -- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa678004d40 msgr2=0x7fa6780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.220+0000 7fa691e00640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:36.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.220+0000 7fa691e00640 1 -- 172.21.15.59:0/2816216931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa678003520 con 0x7fa6780039e0 2024-04-24T06:38:36.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.221+0000 7fa691e00640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa678003e00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa684001720 tx=0x7fa678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ae1ad6504cd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:36.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.221+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684001370 con 0x7fa6780039e0 2024-04-24T06:38:36.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.221+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6840052d0 con 0x7fa6780039e0 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.221+0000 7fa691400640 1 -- 172.21.15.59:0/2816216931 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68402abc0 con 0x7fa6780039e0 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 -- 172.21.15.59:0/2816216931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 msgr2=0x7fa678003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 --2- 172.21.15.59:0/2816216931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa678003e00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa684001720 tx=0x7fa678005830 comp rx=0 tx=0).stop 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa692800640 1 -- 172.21.15.59:0/2816216931 reap_dead start 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 -- 172.21.15.59:0/2816216931 shutdown_connections 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 -- 172.21.15.59:0/2816216931 >> 172.21.15.59:0/2816216931 conn(0x7fa6940b2200 msgr2=0x7fa6940b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 -- 172.21.15.59:0/2816216931 shutdown_connections 2024-04-24T06:38:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.222+0000 7fa693200640 1 -- 172.21.15.59:0/2816216931 wait complete. 2024-04-24T06:38:36.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.223+0000 7fa693200640 1 Processor -- start 2024-04-24T06:38:36.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.224+0000 7fa693200640 1 -- start start 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.224+0000 7fa693200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa693200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 0x7fa69406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa693200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 0x7fa694157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa693200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415d9e0 con 0x7fa6780039e0 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa693200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415d4c0 con 0x7fa694157000 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa693200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa694073ba0 con 0x7fa69406e510 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34610/0 (socket says 172.21.15.59:34610) 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 -- 172.21.15.59:0/2472137296 learned_addr learned my addr 172.21.15.59:0/2472137296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa68be00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 0x7fa69406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa692800640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 0x7fa694157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 -- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 msgr2=0x7fa69406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 0x7fa69406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 -- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 msgr2=0x7fa694157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 0x7fa694157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa691e00640 1 -- 172.21.15.59:0/2472137296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694151ce0 con 0x7fa6780039e0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa68be00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 0x7fa69406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.225+0000 7fa692800640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 0x7fa694157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa691e00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa684008fd0 tx=0x7fa684001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684037040 con 0x7fa6780039e0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa684005480 con 0x7fa6780039e0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa694157be0 con 0x7fa6780039e0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684007a30 con 0x7fa6780039e0 2024-04-24T06:38:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.226+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941581a0 con 0x7fa6780039e0 2024-04-24T06:38:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.227+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69404fd90 con 0x7fa6780039e0 2024-04-24T06:38:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.229+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa684044020 con 0x7fa6780039e0 2024-04-24T06:38:36.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.230+0000 7fa68a000640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 0x7fa67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:36.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.230+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4857+0+0 (secure 0 0 0) 0x7fa6840ab270 con 0x7fa6780039e0 2024-04-24T06:38:36.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.230+0000 7fa68be00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 0x7fa67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:36.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.231+0000 7fa68be00640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 0x7fa67006edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa69409e3e0 tx=0x7fa67c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.240+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa68404b020 con 0x7fa6780039e0 2024-04-24T06:38:36.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:36.448+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa69406cbe0 con 0x7fa6780039e0 2024-04-24T06:38:37.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.142+0000 7fa68a000640 1 -- 172.21.15.59:0/2472137296 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v95) ==== 138+0+0 (secure 0 0 0) 0x7fa694151ce0 con 0x7fa6780039e0 2024-04-24T06:38:37.144 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:38:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 msgr2=0x7fa67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 0x7fa67006edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa69409e3e0 tx=0x7fa67c008040 comp rx=0 tx=0).stop 2024-04-24T06:38:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 msgr2=0x7fa694151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa684008fd0 tx=0x7fa684001120 comp rx=0 tx=0).stop 2024-04-24T06:38:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 shutdown_connections 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa67006c8f0 0x7fa67006edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa694157000 0x7fa694157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa69406e510 0x7fa69406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 --2- 172.21.15.59:0/2472137296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6780039e0 0x7fa694151160 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.147+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 >> 172.21.15.59:0/2472137296 conn(0x7fa6940b2200 msgr2=0x7fa6940b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.148+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 shutdown_connections 2024-04-24T06:38:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.148+0000 7fa693200640 1 -- 172.21.15.59:0/2472137296 wait complete. 2024-04-24T06:38:37.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:38:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.376+0000 7f9eee000640 1 Processor -- start 2024-04-24T06:38:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.376+0000 7f9eee000640 1 -- start start 2024-04-24T06:38:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.378+0000 7f9eee000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ee8166780 0x7f9ee8166be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.378+0000 7f9eee000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ee80c3170 0x7f9ee80c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.378+0000 7f9eee000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee80b33f0 con 0x7f9ee8162d50 2024-04-24T06:38:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.378+0000 7f9eee000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee8167200 con 0x7f9ee80c3170 2024-04-24T06:38:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.378+0000 7f9eee000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee81674e0 con 0x7f9ee8166780 2024-04-24T06:38:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ee80c3170 0x7f9ee80c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ee80c3170 0x7f9ee80c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34616/0 (socket says 172.21.15.59:34616) 2024-04-24T06:38:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 -- 172.21.15.59:0/1986816687 learned_addr learned my addr 172.21.15.59:0/1986816687 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9ee7e00640 1 --2- 172.21.15.59:0/1986816687 >> v2:172.21.15.66:3300/0 conn(0x7f9ee8166780 0x7f9ee8166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 -- 172.21.15.59:0/1986816687 >> v2:172.21.15.66:3300/0 conn(0x7f9ee8166780 msgr2=0x7f9ee8166be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 --2- 172.21.15.59:0/1986816687 >> v2:172.21.15.66:3300/0 conn(0x7f9ee8166780 0x7f9ee8166be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 -- 172.21.15.59:0/1986816687 >> v1:172.21.15.66:6789/0 conn(0x7f9ee8162d50 legacy=0x7f9ee80acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.379+0000 7f9eed600640 1 -- 172.21.15.59:0/1986816687 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee80b3670 con 0x7f9ee80c3170 2024-04-24T06:38:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9eed600640 1 --2- 172.21.15.59:0/1986816687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee80c3600 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9ee806d8c0 tx=0x7f9ee002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410f8425bf1cee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9ee7400640 1 -- 172.21.15.59:0/1986816687 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0036070 con 0x7f9ee80c3170 2024-04-24T06:38:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9ee7400640 1 -- 172.21.15.59:0/1986816687 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee0031600 con 0x7f9ee80c3170 2024-04-24T06:38:37.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9ee7400640 1 -- 172.21.15.59:0/1986816687 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00319d0 con 0x7f9ee80c3170 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9eee000640 1 -- 172.21.15.59:0/1986816687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 msgr2=0x7f9ee80c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.381+0000 7f9eee000640 1 --2- 172.21.15.59:0/1986816687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee80c3600 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9ee806d8c0 tx=0x7f9ee002a6a0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 -- 172.21.15.59:0/1986816687 shutdown_connections 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 --2- 172.21.15.59:0/1986816687 >> v2:172.21.15.66:3300/0 conn(0x7f9ee8166780 0x7f9ee8166be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 --2- 172.21.15.59:0/1986816687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee80c3600 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 -- 172.21.15.59:0/1986816687 >> 172.21.15.59:0/1986816687 conn(0x7f9ee806bbb0 msgr2=0x7f9ee806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 -- 172.21.15.59:0/1986816687 shutdown_connections 2024-04-24T06:38:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.382+0000 7f9eee000640 1 -- 172.21.15.59:0/1986816687 wait complete. 2024-04-24T06:38:37.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.384+0000 7f9eee000640 1 Processor -- start 2024-04-24T06:38:37.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.384+0000 7f9eee000640 1 -- start start 2024-04-24T06:38:37.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ee8162d50 0x7f9ee80b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 0x7f9ee8165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee80af440 con 0x7f9ee80c3170 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee80ad700 con 0x7f9ee8166780 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eee000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee80b3e70 con 0x7f9ee8162d50 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34618/0 (socket says 172.21.15.59:34618) 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.385+0000 7f9eecc00640 1 -- 172.21.15.59:0/1409366718 learned_addr learned my addr 172.21.15.59:0/1409366718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eed600640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 0x7f9ee8165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9ee7e00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ee8162d50 0x7f9ee80b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 -- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ee8162d50 msgr2=0x7f9ee80b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ee8162d50 0x7f9ee80b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 -- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 msgr2=0x7f9ee8165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:37.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 0x7f9ee8165330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:37.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 -- 172.21.15.59:0/1409366718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee8164290 con 0x7f9ee80c3170 2024-04-24T06:38:37.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eed600640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 0x7f9ee8165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.386+0000 7f9eecc00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c400 tx=0x7f9edc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.387+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0052d0 con 0x7f9ee80c3170 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.387+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc005490 con 0x7f9ee80c3170 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.387+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc007260 con 0x7f9ee80c3170 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.387+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee8164570 con 0x7f9ee80c3170 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.387+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee8164e10 con 0x7f9ee80c3170 2024-04-24T06:38:37.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.388+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee8164740 con 0x7f9ee80c3170 2024-04-24T06:38:37.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.389+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9edc007400 con 0x7f9ee80c3170 2024-04-24T06:38:37.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.390+0000 7f9ee4c00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 0x7f9ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:37.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.390+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5227+0+0 (secure 0 0 0) 0x7f9edc08af40 con 0x7f9ee80c3170 2024-04-24T06:38:37.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.391+0000 7f9ee7e00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 0x7f9ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.391+0000 7f9ee7e00640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 0x7f9ec806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4006fd0 tx=0x7f9ed4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.396+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ee80b33f0 con 0x7f9ee80c3170 2024-04-24T06:38:37.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:37.601+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9ee804fd90 con 0x7f9ee80c3170 2024-04-24T06:38:37.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:38:37.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:38:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.158+0000 7f9ee4c00640 1 -- 172.21.15.59:0/1409366718 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v96) ==== 170+0+0 (secure 0 0 0) 0x7f9edc0594b0 con 0x7f9ee80c3170 2024-04-24T06:38:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:38:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 msgr2=0x7f9ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 0x7f9ec806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4006fd0 tx=0x7f9ed4008040 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 msgr2=0x7f9ee8099820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c400 tx=0x7f9edc00c8c0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 shutdown_connections 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9ec806c8f0 0x7f9ec806edb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee8166780 0x7f9ee8165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ee8162d50 0x7f9ee80b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 --2- 172.21.15.59:0/1409366718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ee80c3170 0x7f9ee8099820 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 >> 172.21.15.59:0/1409366718 conn(0x7f9ee806bbb0 msgr2=0x7f9ee806bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:38.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 shutdown_connections 2024-04-24T06:38:38.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.163+0000 7f9eee000640 1 -- 172.21.15.59:0/1409366718 wait complete. 2024-04-24T06:38:38.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:38:38.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.381+0000 7f10ea400640 1 Processor -- start 2024-04-24T06:38:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.381+0000 7f10ea400640 1 -- start start 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10ea400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 0x7f10e40b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10ea400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f10e415ed60 0x7f10e40b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10ea400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e40b2dd0 con 0x7f10e40cbb90 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10ea400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f10e406b9d0 con 0x7f10e415ed60 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10ea400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10e406c250 con 0x7f10e40b24b0 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10e3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f10e40cbb90 0x7f10e40b1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49686/0 (socket says 172.21.15.59:49686) 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.383+0000 7f10e3e00640 1 -- 172.21.15.59:0/300901409 learned_addr learned my addr 172.21.15.59:0/300901409 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2a00640 1 --2- 172.21.15.59:0/300901409 >> v2:172.21.15.66:3300/0 conn(0x7f10e415ed60 0x7f10e40b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e3400640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 0x7f10e40b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180787816 0 0) 0x7f10e40b2dd0 con 0x7f10e40cbb90 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c8003520 con 0x7f10e40cbb90 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1512874965 0 0) 0x7f10c8003520 con 0x7f10e40cbb90 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 msgr2=0x7f10e40b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 0x7f10e40b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 >> v2:172.21.15.66:3300/0 conn(0x7f10e415ed60 msgr2=0x7f10e40b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 --2- 172.21.15.59:0/300901409 >> v2:172.21.15.66:3300/0 conn(0x7f10e415ed60 0x7f10e40b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40cea70 con 0x7f10e40cbb90 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f10dc0019f0 con 0x7f10e40cbb90 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.384+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 >> v1:172.21.15.59:6789/0 conn(0x7f10e40cbb90 legacy=0x7f10e40b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 0x7f10c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406c250 con 0x7f10c8004d40 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10c800b9c0 con 0x7f10c80039b0 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10c800bb40 con 0x7f10e40b24b0 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 -- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 msgr2=0x7f10c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e40b24b0 0x7f10c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 -- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c80039b0 msgr2=0x7f10c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.385+0000 7f10e3e00640 1 -- 172.21.15.59:0/300901409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8003520 con 0x7f10c8004d40 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.386+0000 7f10e3e00640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0013d0 tx=0x7f10c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461e0c1124dd7e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00a410 con 0x7f10c8004d40 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc02f070 con 0x7f10c8004d40 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10e2000640 1 -- 172.21.15.59:0/300901409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02bbc0 con 0x7f10c8004d40 2024-04-24T06:38:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10ea400640 1 -- 172.21.15.59:0/300901409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c8004d40 msgr2=0x7f10c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10ea400640 1 --2- 172.21.15.59:0/300901409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0013d0 tx=0x7f10c8005830 comp rx=0 tx=0).stop 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10e3e00640 1 -- 172.21.15.59:0/300901409 reap_dead start 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10ea400640 1 -- 172.21.15.59:0/300901409 shutdown_connections 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.387+0000 7f10ea400640 1 -- 172.21.15.59:0/300901409 >> 172.21.15.59:0/300901409 conn(0x7f10e4073450 msgr2=0x7f10e409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.388+0000 7f10ea400640 1 -- 172.21.15.59:0/300901409 shutdown_connections 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.388+0000 7f10ea400640 1 -- 172.21.15.59:0/300901409 wait complete. 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.389+0000 7f10ea400640 1 Processor -- start 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.390+0000 7f10ea400640 1 -- start start 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.390+0000 7f10ea400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c8004d40 0x7f10e415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.390+0000 7f10ea400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e4161370 0x7f10e41617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10ea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40b0c80 con 0x7f10e41f4a20 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10ea400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406bee0 con 0x7f10c8004d40 2024-04-24T06:38:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10ea400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40ce4d0 con 0x7f10e4161370 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34634/0 (socket says 172.21.15.59:34634) 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 -- 172.21.15.59:0/731817659 learned_addr learned my addr 172.21.15.59:0/731817659 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e2a00640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e4161370 0x7f10e41617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c8004d40 0x7f10e415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 -- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e4161370 msgr2=0x7f10e41617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e4161370 0x7f10e41617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 -- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c8004d40 msgr2=0x7f10e415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c8004d40 0x7f10e415e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.391+0000 7f10e3e00640 1 -- 172.21.15.59:0/731817659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e415e7b0 con 0x7f10e41f4a20 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10e3e00640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00a880 tx=0x7f10dc001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc001f70 con 0x7f10e41f4a20 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc00a3f0 con 0x7f10e41f4a20 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02cc60 con 0x7f10e41f4a20 2024-04-24T06:38:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e41f7f50 con 0x7f10e41f4a20 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.392+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e41f8510 con 0x7f10e41f4a20 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.394+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e4068a80 con 0x7f10e41f4a20 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.395+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10dc034470 con 0x7f10e41f4a20 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.396+0000 7f10db400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 0x7f10c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.396+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5609+0+0 (secure 0 0 0) 0x7f10dc0abc50 con 0x7f10e41f4a20 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.396+0000 7f10e3400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 0x7f10c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.397+0000 7f10e3400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 0x7f10c406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004770 tx=0x7f10d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.402+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10dc07a040 con 0x7f10e41f4a20 2024-04-24T06:38:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:38.645+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f10e40b2aa0 con 0x7f10e41f4a20 2024-04-24T06:38:39.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.163+0000 7f10db400640 1 -- 172.21.15.59:0/731817659 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v80) ==== 174+0+0 (secure 0 0 0) 0x7f10dc079940 con 0x7f10e41f4a20 2024-04-24T06:38:39.165 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 14 and data pool 15 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 msgr2=0x7f10c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 0x7f10c406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004770 tx=0x7f10d4008040 comp rx=0 tx=0).stop 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 msgr2=0x7f10e41f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00a880 tx=0x7f10dc001360 comp rx=0 tx=0).stop 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 shutdown_connections 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10c406c920 0x7f10c406ede0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10e41f4a20 0x7f10e41f4ed0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10e4161370 0x7f10e41617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 --2- 172.21.15.59:0/731817659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c8004d40 0x7f10e415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 >> 172.21.15.59:0/731817659 conn(0x7f10e4073450 msgr2=0x7f10e40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 shutdown_connections 2024-04-24T06:38:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.167+0000 7f10ea400640 1 -- 172.21.15.59:0/731817659 wait complete. 2024-04-24T06:38:39.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:39.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.385+0000 7f396f000640 1 Processor -- start 2024-04-24T06:38:39.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.385+0000 7f396f000640 1 -- start start 2024-04-24T06:38:39.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.386+0000 7f396f000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f39680d00a0 0x7f396809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.386+0000 7f396f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f396809af40 0x7f3968070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.386+0000 7f396f000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f396806feb0 con 0x7f396815e9c0 2024-04-24T06:38:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.386+0000 7f396f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f39680d04e0 con 0x7f396809af40 2024-04-24T06:38:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.386+0000 7f396f000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f39680707c0 con 0x7f39680d00a0 2024-04-24T06:38:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f396809af40 0x7f3968070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396dc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f396815e9c0 0x7f39680cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51188/0 (socket says 172.21.15.59:51188) 2024-04-24T06:38:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396dc00640 1 -- 172.21.15.59:0/3224629580 learned_addr learned my addr 172.21.15.59:0/3224629580 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:39.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396d200640 1 --2- 172.21.15.59:0/3224629580 >> v2:172.21.15.66:3300/0 conn(0x7f39680d00a0 0x7f396809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396e600640 1 -- 172.21.15.59:0/3224629580 >> v2:172.21.15.66:3300/0 conn(0x7f39680d00a0 msgr2=0x7f396809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:39.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396e600640 1 --2- 172.21.15.59:0/3224629580 >> v2:172.21.15.66:3300/0 conn(0x7f39680d00a0 0x7f396809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396e600640 1 -- 172.21.15.59:0/3224629580 >> v1:172.21.15.59:6789/0 conn(0x7f396815e9c0 legacy=0x7f39680cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f396e600640 1 -- 172.21.15.59:0/3224629580 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680ce380 con 0x7f396809af40 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.387+0000 7f3967e00640 1 -- 172.21.15.59:0/3224629580 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956843753 0 0) 0x7f396806feb0 con 0x7f396815e9c0 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.388+0000 7f396e600640 1 --2- 172.21.15.59:0/3224629580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f3968070c50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f39681622c0 tx=0x7f396002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfa2a2779e4c10b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f3967e00640 1 -- 172.21.15.59:0/3224629580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960007740 con 0x7f396809af40 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f3967e00640 1 -- 172.21.15.59:0/3224629580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39600078e0 con 0x7f396809af40 2024-04-24T06:38:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f3967e00640 1 -- 172.21.15.59:0/3224629580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960007cb0 con 0x7f396809af40 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 -- 172.21.15.59:0/3224629580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 msgr2=0x7f3968070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 --2- 172.21.15.59:0/3224629580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f3968070c50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f39681622c0 tx=0x7f396002a680 comp rx=0 tx=0).stop 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 -- 172.21.15.59:0/3224629580 shutdown_connections 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 --2- 172.21.15.59:0/3224629580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f3968070c50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 --2- 172.21.15.59:0/3224629580 >> v2:172.21.15.66:3300/0 conn(0x7f39680d00a0 0x7f396809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 -- 172.21.15.59:0/3224629580 >> 172.21.15.59:0/3224629580 conn(0x7f39680b3ee0 msgr2=0x7f39680b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 -- 172.21.15.59:0/3224629580 shutdown_connections 2024-04-24T06:38:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.389+0000 7f396f000640 1 -- 172.21.15.59:0/3224629580 wait complete. 2024-04-24T06:38:39.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.391+0000 7f396f000640 1 Processor -- start 2024-04-24T06:38:39.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.391+0000 7f396f000640 1 -- start start 2024-04-24T06:38:39.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 0x7f396815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f396815e9c0 0x7f396815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f396806c820 con 0x7f396809af40 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3968070450 con 0x7f39680d00a0 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396f000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3968071190 con 0x7f396815e9c0 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44334/0 (socket says 172.21.15.59:44334) 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.392+0000 7f396dc00640 1 -- 172.21.15.59:0/1429900967 learned_addr learned my addr 172.21.15.59:0/1429900967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396d200640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 0x7f396815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396e600640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f396815e9c0 0x7f396815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 -- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f396815e9c0 msgr2=0x7f396815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f396815e9c0 0x7f396815fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 -- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 msgr2=0x7f396815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 0x7f396815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 -- 172.21.15.59:0/1429900967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680b5200 con 0x7f396809af40 2024-04-24T06:38:39.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396d200640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 0x7f396815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f396dc00640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c430 tx=0x7f395c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.393+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0074e0 con 0x7f396809af40 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.394+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0047e0 con 0x7f396809af40 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.394+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c004bd0 con 0x7f396809af40 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.394+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39680b54e0 con 0x7f396809af40 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.394+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39680cb450 con 0x7f396809af40 2024-04-24T06:38:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.395+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396804fd90 con 0x7f396809af40 2024-04-24T06:38:39.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.396+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f395c011070 con 0x7f396809af40 2024-04-24T06:38:39.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.397+0000 7f3965600640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 0x7f394806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.397+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f395c08b840 con 0x7f396809af40 2024-04-24T06:38:39.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.397+0000 7f396d200640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 0x7f394806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.398+0000 7f396d200640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 0x7f394806edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f396809f9e0 tx=0x7f3954008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.402+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f395c059c60 con 0x7f396809af40 2024-04-24T06:38:39.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.611+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f396806feb0 con 0x7f396809af40 2024-04-24T06:38:39.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.613+0000 7f3965600640 1 -- 172.21.15.59:0/1429900967 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16537 (secure 0 0 0) 0x7f395c059560 con 0x7f396809af40 2024-04-24T06:38:39.616 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:39.616 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":97,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:38.648035+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:39.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 msgr2=0x7f394806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 0x7f394806edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f396809f9e0 tx=0x7f3954008040 comp rx=0 tx=0).stop 2024-04-24T06:38:39.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 msgr2=0x7f39680cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c430 tx=0x7f395c00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 shutdown_connections 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f394806c8f0 0x7f394806edb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f396815e9c0 0x7f396815fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39680d00a0 0x7f396815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 --2- 172.21.15.59:0/1429900967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f396809af40 0x7f39680cb970 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 >> 172.21.15.59:0/1429900967 conn(0x7f39680b3ee0 msgr2=0x7f39680b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 shutdown_connections 2024-04-24T06:38:39.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.616+0000 7f396f000640 1 -- 172.21.15.59:0/1429900967 wait complete. 2024-04-24T06:38:39.637 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:39.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.845+0000 7f98e6000640 1 Processor -- start 2024-04-24T06:38:39.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.846+0000 7f98e6000640 1 -- start start 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98e6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 0x7f98e00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98e6000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e00cd520 con 0x7f98e01454e0 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98e6000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e00cd800 con 0x7f98e00b3e40 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98e6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f98e00cddc0 con 0x7f98e00cd020 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98df400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f98e01454e0 0x7f98e0145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51190/0 (socket says 172.21.15.59:51190) 2024-04-24T06:38:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.847+0000 7f98df400640 1 -- 172.21.15.59:0/2522855176 learned_addr learned my addr 172.21.15.59:0/2522855176 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98dea00640 1 --2- 172.21.15.59:0/2522855176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 0x7f98e00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984287915 0 0) 0x7f98e00cd520 con 0x7f98e01454e0 2024-04-24T06:38:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98c4003520 con 0x7f98e01454e0 2024-04-24T06:38:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98dea00640 1 -- 172.21.15.59:0/2522855176 >> v1:172.21.15.66:6789/0 conn(0x7f98e00b3e40 legacy=0x7f98e0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1358848368 0 0) 0x7f98c4003520 con 0x7f98e01454e0 2024-04-24T06:38:39.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98dea00640 1 -- 172.21.15.59:0/2522855176 >> v1:172.21.15.59:6789/0 conn(0x7f98e01454e0 legacy=0x7f98e0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.848+0000 7f98dea00640 1 -- 172.21.15.59:0/2522855176 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e00cdae0 con 0x7f98e00cd020 2024-04-24T06:38:39.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.849+0000 7f98dea00640 1 --2- 172.21.15.59:0/2522855176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 0x7f98e00b3900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009dc0 tx=0x7f98cc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b49e93175b27cec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.850+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0001670 con 0x7f98e00cd020 2024-04-24T06:38:39.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.850+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc007540 con 0x7f98e00cd020 2024-04-24T06:38:39.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.850+0000 7f98de000640 1 -- 172.21.15.59:0/2522855176 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc007910 con 0x7f98e00cd020 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 -- 172.21.15.59:0/2522855176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 msgr2=0x7f98e00b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 --2- 172.21.15.59:0/2522855176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 0x7f98e00b3900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009dc0 tx=0x7f98cc02a5d0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 -- 172.21.15.59:0/2522855176 shutdown_connections 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 --2- 172.21.15.59:0/2522855176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e00cd020 0x7f98e00b3900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 -- 172.21.15.59:0/2522855176 >> 172.21.15.59:0/2522855176 conn(0x7f98e0067ea0 msgr2=0x7f98e006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 -- 172.21.15.59:0/2522855176 shutdown_connections 2024-04-24T06:38:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.851+0000 7f98e6000640 1 -- 172.21.15.59:0/2522855176 wait complete. 2024-04-24T06:38:39.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.853+0000 7f98e6000640 1 Processor -- start 2024-04-24T06:38:39.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.853+0000 7f98e6000640 1 -- start start 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e01454e0 0x7f98e013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 0x7f98e0143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0161020 con 0x7f98e00b3e40 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01462c0 con 0x7f98e015d780 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98e6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f98e01611a0 con 0x7f98e01454e0 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44342/0 (socket says 172.21.15.59:44342) 2024-04-24T06:38:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.854+0000 7f98df400640 1 -- 172.21.15.59:0/3819959865 learned_addr learned my addr 172.21.15.59:0/3819959865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98dea00640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e01454e0 0x7f98e013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98dfe00640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 0x7f98e0143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 -- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e01454e0 msgr2=0x7f98e013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e01454e0 0x7f98e013ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 -- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 msgr2=0x7f98e0143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 0x7f98e0143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 -- 172.21.15.59:0/3819959865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01f7fd0 con 0x7f98e00b3e40 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98dfe00640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 0x7f98e0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98df400640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f98d000c5b0 tx=0x7f98d000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0017070 con 0x7f98e00b3e40 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0005ce0 con 0x7f98e00b3e40 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e013f250 con 0x7f98e00b3e40 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.855+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0007330 con 0x7f98e00b3e40 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.856+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01f8550 con 0x7f98e00b3e40 2024-04-24T06:38:39.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.857+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e0072c10 con 0x7f98e00b3e40 2024-04-24T06:38:39.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.858+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98d001a070 con 0x7f98e00b3e40 2024-04-24T06:38:39.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.859+0000 7f98d7400640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 0x7f98c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:39.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.860+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f98d008b050 con 0x7f98e00b3e40 2024-04-24T06:38:39.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.860+0000 7f98dea00640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 0x7f98c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:39.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.860+0000 7f98dea00640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 0x7f98c006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f98cc034000 tx=0x7f98cc02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:39.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:39.865+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98e0072c10 con 0x7f98e00b3e40 2024-04-24T06:38:40.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.061+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98e0160030 con 0x7f98e00b3e40 2024-04-24T06:38:40.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.063+0000 7f98d7400640 1 -- 172.21.15.59:0/3819959865 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16537 (secure 0 0 0) 0x7f98d0059470 con 0x7f98e00b3e40 2024-04-24T06:38:40.066 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:40.066 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":97,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:38.648035+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.065+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 msgr2=0x7f98c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 0x7f98c006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f98cc034000 tx=0x7f98cc02f000 comp rx=0 tx=0).stop 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 msgr2=0x7f98e015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f98d000c5b0 tx=0x7f98d000ca70 comp rx=0 tx=0).stop 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 shutdown_connections 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f98c006c8f0 0x7f98c006edb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015d780 0x7f98e0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f98e01454e0 0x7f98e013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 --2- 172.21.15.59:0/3819959865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00b3e40 0x7f98e015cd20 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 >> 172.21.15.59:0/3819959865 conn(0x7f98e0067ea0 msgr2=0x7f98e006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 shutdown_connections 2024-04-24T06:38:40.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.066+0000 7f98e6000640 1 -- 172.21.15.59:0/3819959865 wait complete. 2024-04-24T06:38:40.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:38:40.127 INFO:tasks.ceph.mds.d.smithi066.stderr:2024-04-24T06:38:40.125+0000 7f46c6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:38:40.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.292+0000 7f5c5be00640 1 Processor -- start 2024-04-24T06:38:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.293+0000 7f5c5be00640 1 -- start start 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c5c0b5da0 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c15f100 con 0x7f5c5c0b3600 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d0a20 con 0x7f5c5c15ed20 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.294+0000 7f5c5a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44350/0 (socket says 172.21.15.59:44350) 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 -- 172.21.15.59:0/2609641676 learned_addr learned my addr 172.21.15.59:0/2609641676 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5aa00640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:40.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 -- 172.21.15.59:0/2609641676 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 msgr2=0x7f5c5c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 -- 172.21.15.59:0/2609641676 >> v1:172.21.15.59:6789/0 conn(0x7f5c5c0b3f10 legacy=0x7f5c5c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.295+0000 7f5c5a000640 1 -- 172.21.15.59:0/2609641676 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c0d2ed0 con 0x7f5c5c0b3600 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.296+0000 7f5c5a000640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009870 tx=0x7f5c4402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7628529acda50f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.296+0000 7f5c59600640 1 -- 172.21.15.59:0/2609641676 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037070 con 0x7f5c5c0b3600 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.296+0000 7f5c59600640 1 -- 172.21.15.59:0/2609641676 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44032440 con 0x7f5c5c0b3600 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.296+0000 7f5c59600640 1 -- 172.21.15.59:0/2609641676 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44032810 con 0x7f5c5c0b3600 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 -- 172.21.15.59:0/2609641676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3600 msgr2=0x7f5c5c0d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009870 tx=0x7f5c4402a130 comp rx=0 tx=0).stop 2024-04-24T06:38:40.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 -- 172.21.15.59:0/2609641676 shutdown_connections 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c0d13e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 --2- 172.21.15.59:0/2609641676 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 -- 172.21.15.59:0/2609641676 >> 172.21.15.59:0/2609641676 conn(0x7f5c5c08bf90 msgr2=0x7f5c5c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 -- 172.21.15.59:0/2609641676 shutdown_connections 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.297+0000 7f5c5be00640 1 -- 172.21.15.59:0/2609641676 wait complete. 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.299+0000 7f5c5be00640 1 Processor -- start 2024-04-24T06:38:40.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.299+0000 7f5c5be00640 1 -- start start 2024-04-24T06:38:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d0290 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d3490 con 0x7f5c5c0b3600 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d0410 con 0x7f5c5c15ed20 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44358/0 (socket says 172.21.15.59:44358) 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 -- 172.21.15.59:0/1068896199 learned_addr learned my addr 172.21.15.59:0/1068896199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 -- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 msgr2=0x7f5c5c0c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 -- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5c0b3600 msgr2=0x7f5c5c154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.300+0000 7f5c5a000640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c5a000640 1 -- 172.21.15.59:0/1068896199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c0cd250 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c5a000640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402a170 tx=0x7f5c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037040 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44042070 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44030980 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c0cd530 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.301+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c0cdaf0 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.303+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c04fd90 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.304+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c44030b20 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.304+0000 7f5c52a00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 0x7f5c3806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.305+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f5c440ac4f0 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.305+0000 7f5c5aa00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 0x7f5c3806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:40.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.306+0000 7f5c5aa00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 0x7f5c3806ee00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0070b0 tx=0x7f5c4c007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:40.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.309+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c4407a900 con 0x7f5c5c0b3f10 2024-04-24T06:38:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:40.545+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5c5c0cc950 con 0x7f5c5c0b3f10 2024-04-24T06:38:41.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.214+0000 7f5c52a00640 1 -- 172.21.15.59:0/1068896199 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7f5c4407a200 con 0x7f5c5c0b3f10 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.217+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 msgr2=0x7f5c3806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.217+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 0x7f5c3806ee00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0070b0 tx=0x7f5c4c007040 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.217+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 msgr2=0x7f5c5c0ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.217+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402a170 tx=0x7f5c440046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 shutdown_connections 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c3806c940 0x7f5c3806ee00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c5c15ed20 0x7f5c5c0c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c5c0b3f10 0x7f5c5c0ccb70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 --2- 172.21.15.59:0/1068896199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5c0b3600 0x7f5c5c154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 >> 172.21.15.59:0/1068896199 conn(0x7f5c5c08bf90 msgr2=0x7f5c5c0d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 shutdown_connections 2024-04-24T06:38:41.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.218+0000 7f5c5be00640 1 -- 172.21.15.59:0/1068896199 wait complete. 2024-04-24T06:38:41.242 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:38:41.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:38:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.456+0000 7f7097200640 1 Processor -- start 2024-04-24T06:38:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.456+0000 7f7097200640 1 -- start start 2024-04-24T06:38:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.458+0000 7f7097200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f709815ed20 0x7f70980d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.458+0000 7f7097200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70980b3600 0x7f70980d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.458+0000 7f7097200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70980b5da0 con 0x7f70980b3f10 2024-04-24T06:38:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.458+0000 7f7097200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f709815f100 con 0x7f70980b3600 2024-04-24T06:38:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.458+0000 7f7097200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f70980d0a20 con 0x7f709815ed20 2024-04-24T06:38:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70980b3600 0x7f70980d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70980b3600 0x7f70980d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44366/0 (socket says 172.21.15.59:44366) 2024-04-24T06:38:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 -- 172.21.15.59:0/2931692863 learned_addr learned my addr 172.21.15.59:0/2931692863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095e00640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f709815ed20 0x7f70980d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 -- 172.21.15.59:0/2931692863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f709815ed20 msgr2=0x7f70980d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f709815ed20 0x7f70980d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 -- 172.21.15.59:0/2931692863 >> v1:172.21.15.66:6789/0 conn(0x7f70980b3f10 legacy=0x7f70980b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.459+0000 7f7095400640 1 -- 172.21.15.59:0/2931692863 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70980d2ed0 con 0x7f70980b3600 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.460+0000 7f7095400640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70980b3600 0x7f70980d13e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7084009870 tx=0x7f708402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1315c8dfaa42dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7094a00640 1 -- 172.21.15.59:0/2931692863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084037070 con 0x7f70980b3600 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7094a00640 1 -- 172.21.15.59:0/2931692863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084032440 con 0x7f70980b3600 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7094a00640 1 -- 172.21.15.59:0/2931692863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084032810 con 0x7f70980b3600 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 -- 172.21.15.59:0/2931692863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70980b3600 msgr2=0x7f70980d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70980b3600 0x7f70980d13e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7084009870 tx=0x7f708402a130 comp rx=0 tx=0).stop 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 -- 172.21.15.59:0/2931692863 shutdown_connections 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70980b3600 0x7f70980d13e0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 --2- 172.21.15.59:0/2931692863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f709815ed20 0x7f70980d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 -- 172.21.15.59:0/2931692863 >> 172.21.15.59:0/2931692863 conn(0x7f709808bf90 msgr2=0x7f70980d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 -- 172.21.15.59:0/2931692863 shutdown_connections 2024-04-24T06:38:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.461+0000 7f7097200640 1 -- 172.21.15.59:0/2931692863 wait complete. 2024-04-24T06:38:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.463+0000 7f7097200640 1 Processor -- start 2024-04-24T06:38:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.463+0000 7f7097200640 1 -- start start 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70980b3600 0x7f709809e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70980b3f10 0x7f709815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70980d0290 con 0x7f709815ed20 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f70980d3490 con 0x7f70980b3f10 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7097200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f70980d0410 con 0x7f70980b3600 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.464+0000 7f7096800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44370/0 (socket says 172.21.15.59:44370) 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 -- 172.21.15.59:0/74640419 learned_addr learned my addr 172.21.15.59:0/74640419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7095400640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70980b3f10 0x7f709815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7095e00640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70980b3600 0x7f709809e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 -- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70980b3600 msgr2=0x7f709809e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70980b3600 0x7f709809e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 -- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70980b3f10 msgr2=0x7f709815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70980b3f10 0x7f709815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.465+0000 7f7096800640 1 -- 172.21.15.59:0/74640419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70981f80b0 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f7096800640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7090001eb0 tx=0x7f709000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090017070 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70900052d0 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090003810 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70981f8390 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.466+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70981f8950 con 0x7f709815ed20 2024-04-24T06:38:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.467+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709804fd90 con 0x7f709815ed20 2024-04-24T06:38:41.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.468+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f709001a070 con 0x7f709815ed20 2024-04-24T06:38:41.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.469+0000 7f708e000640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 0x7f707806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:41.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.469+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f709008af30 con 0x7f709815ed20 2024-04-24T06:38:41.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.469+0000 7f7095e00640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 0x7f707806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:41.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.470+0000 7f7095e00640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 0x7f707806edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f70980ce100 tx=0x7f708800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:41.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.473+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7090059350 con 0x7f709815ed20 2024-04-24T06:38:41.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:41.729+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f70981f8d30 con 0x7f709815ed20 2024-04-24T06:38:42.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.217+0000 7f708e000640 1 -- 172.21.15.59:0/74640419 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f7090058c50 con 0x7f709815ed20 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 msgr2=0x7f707806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 0x7f707806edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f70980ce100 tx=0x7f708800b040 comp rx=0 tx=0).stop 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 msgr2=0x7f70980a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7090001eb0 tx=0x7f709000c890 comp rx=0 tx=0).stop 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 shutdown_connections 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f707806c8f0 0x7f707806edb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f709815ed20 0x7f70980a1a20 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70980b3f10 0x7f709815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 --2- 172.21.15.59:0/74640419 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f70980b3600 0x7f709809e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 >> 172.21.15.59:0/74640419 conn(0x7f709808bf90 msgr2=0x7f70980d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:42.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 shutdown_connections 2024-04-24T06:38:42.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.221+0000 7f7097200640 1 -- 172.21.15.59:0/74640419 wait complete. 2024-04-24T06:38:42.248 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:38:42.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:42.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.460+0000 7f43a0e00640 1 Processor -- start 2024-04-24T06:38:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.460+0000 7f43a0e00640 1 -- start start 2024-04-24T06:38:42.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f439c15e7c0 0x7f439c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f439c0d2630 0x7f439c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0b6530 0x7f439c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f439c0b7000 con 0x7f439c0d2630 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f439c0cf620 con 0x7f439c15e7c0 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.461+0000 7f43a0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f439c0cf030 con 0x7f439c0b6530 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f439c0d2630 0x7f439c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f439c0d2630 0x7f439c0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44378/0 (socket says 172.21.15.59:44378) 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 -- 172.21.15.59:0/1893630576 learned_addr learned my addr 172.21.15.59:0/1893630576 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439b400640 1 --2- 172.21.15.59:0/1893630576 >> v2:172.21.15.66:3300/0 conn(0x7f439c15e7c0 0x7f439c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 -- 172.21.15.59:0/1893630576 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0b6530 msgr2=0x7f439c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 --2- 172.21.15.59:0/1893630576 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0b6530 0x7f439c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 -- 172.21.15.59:0/1893630576 >> v2:172.21.15.66:3300/0 conn(0x7f439c15e7c0 msgr2=0x7f439c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 --2- 172.21.15.59:0/1893630576 >> v2:172.21.15.66:3300/0 conn(0x7f439c15e7c0 0x7f439c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.462+0000 7f439aa00640 1 -- 172.21.15.59:0/1893630576 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c0b6a40 con 0x7f439c0d2630 2024-04-24T06:38:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.463+0000 7f439aa00640 1 --2- 172.21.15.59:0/1893630576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c0d2630 0x7f439c0b5ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f439c08c6d0 tx=0x7f438802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1180cd2ff87776e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.464+0000 7f439a000640 1 -- 172.21.15.59:0/1893630576 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388037070 con 0x7f439c0d2630 2024-04-24T06:38:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.464+0000 7f439a000640 1 -- 172.21.15.59:0/1893630576 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4388032440 con 0x7f439c0d2630 2024-04-24T06:38:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.464+0000 7f439a000640 1 -- 172.21.15.59:0/1893630576 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388032810 con 0x7f439c0d2630 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 -- 172.21.15.59:0/1893630576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c0d2630 msgr2=0x7f439c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 --2- 172.21.15.59:0/1893630576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c0d2630 0x7f439c0b5ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f439c08c6d0 tx=0x7f438802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 -- 172.21.15.59:0/1893630576 shutdown_connections 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 --2- 172.21.15.59:0/1893630576 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0b6530 0x7f439c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 --2- 172.21.15.59:0/1893630576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c0d2630 0x7f439c0b5ff0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 --2- 172.21.15.59:0/1893630576 >> v2:172.21.15.66:3300/0 conn(0x7f439c15e7c0 0x7f439c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 -- 172.21.15.59:0/1893630576 >> 172.21.15.59:0/1893630576 conn(0x7f439c09d920 msgr2=0x7f439c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 -- 172.21.15.59:0/1893630576 shutdown_connections 2024-04-24T06:38:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.465+0000 7f43a0e00640 1 -- 172.21.15.59:0/1893630576 wait complete. 2024-04-24T06:38:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.467+0000 7f43a0e00640 1 Processor -- start 2024-04-24T06:38:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.467+0000 7f43a0e00640 1 -- start start 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 0x7f439c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 0x7f439c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0d1a80 con 0x7f439c15e7c0 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0cf2b0 con 0x7f439c0b6530 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f43a0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f439c0d1c00 con 0x7f439c0d2630 2024-04-24T06:38:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f439be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f439be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44386/0 (socket says 172.21.15.59:44386) 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.468+0000 7f439be00640 1 -- 172.21.15.59:0/349538445 learned_addr learned my addr 172.21.15.59:0/349538445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439aa00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 0x7f439c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439b400640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 0x7f439c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 -- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 msgr2=0x7f439c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 0x7f439c158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 -- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 msgr2=0x7f439c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 0x7f439c0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 -- 172.21.15.59:0/349538445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c0ce440 con 0x7f439c15e7c0 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439aa00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 0x7f439c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439b400640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 0x7f439c0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:42.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f439be00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f439400b970 tx=0x7f439400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394019070 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439400cb60 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394014440 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.469+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c159b20 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.470+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c0b12d0 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.471+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c04fd90 con 0x7f439c15e7c0 2024-04-24T06:38:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.472+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439401c070 con 0x7f439c15e7c0 2024-04-24T06:38:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.473+0000 7f4393400640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 0x7f437c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.473+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f439408b890 con 0x7f439c15e7c0 2024-04-24T06:38:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.473+0000 7f439b400640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 0x7f437c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.474+0000 7f439b400640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 0x7f437c06ede0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f438c006fd0 tx=0x7f438c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.478+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f439c0ce440 con 0x7f439c15e7c0 2024-04-24T06:38:42.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.725+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f439c0b3bd0 con 0x7f439c15e7c0 2024-04-24T06:38:42.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.726+0000 7f4393400640 1 -- 172.21.15.59:0/349538445 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5177 (secure 0 0 0) 0x7f4394059cb0 con 0x7f439c15e7c0 2024-04-24T06:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:42.728 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:38:41.731305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/891418539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":891418539},{"type":"v1","addr":"172.21.15.66:6834","nonce":891418539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":7}]} 2024-04-24T06:38:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 83 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 msgr2=0x7f437c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 0x7f437c06ede0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f438c006fd0 tx=0x7f438c008040 comp rx=0 tx=0).stop 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 msgr2=0x7f439c0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f439400b970 tx=0x7f439400be30 comp rx=0 tx=0).stop 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 shutdown_connections 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f437c06c920 0x7f437c06ede0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f439c15e7c0 0x7f439c0b1db0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f439c0d2630 0x7f439c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 --2- 172.21.15.59:0/349538445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f439c0b6530 0x7f439c0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 >> 172.21.15.59:0/349538445 conn(0x7f439c09d920 msgr2=0x7f439c0d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 shutdown_connections 2024-04-24T06:38:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.729+0000 7f43a0e00640 1 -- 172.21.15.59:0/349538445 wait complete. 2024-04-24T06:38:42.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.962+0000 7f6d3f800640 1 Processor -- start 2024-04-24T06:38:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.963+0000 7f6d3f800640 1 -- start start 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.964+0000 7f6d3f800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d3815cb20 0x7f6d380cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.964+0000 7f6d3f800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6d38163770 0x7f6d380d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.964+0000 7f6d3f800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d380d01c0 con 0x7f6d38160180 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.964+0000 7f6d3f800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3815cf00 con 0x7f6d3815cb20 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.964+0000 7f6d3f800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3815f6a0 con 0x7f6d38163770 2024-04-24T06:38:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d3815cb20 0x7f6d380cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d3815cb20 0x7f6d380cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44390/0 (socket says 172.21.15.59:44390) 2024-04-24T06:38:42.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 -- 172.21.15.59:0/1766346654 learned_addr learned my addr 172.21.15.59:0/1766346654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:42.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d37e00640 1 --2- 172.21.15.59:0/1766346654 >> v2:172.21.15.66:3300/0 conn(0x7f6d38163770 0x7f6d380d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 -- 172.21.15.59:0/1766346654 >> v2:172.21.15.66:3300/0 conn(0x7f6d38163770 msgr2=0x7f6d380d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:42.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 --2- 172.21.15.59:0/1766346654 >> v2:172.21.15.66:3300/0 conn(0x7f6d38163770 0x7f6d380d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 -- 172.21.15.59:0/1766346654 >> v1:172.21.15.66:6789/0 conn(0x7f6d38160180 legacy=0x7f6d380b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.965+0000 7f6d3ce00640 1 -- 172.21.15.59:0/1766346654 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3815f980 con 0x7f6d3815cb20 2024-04-24T06:38:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.967+0000 7f6d3ce00640 1 --2- 172.21.15.59:0/1766346654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3815cb20 0x7f6d380cf840 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6d3806dc70 tx=0x7f6d2802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a128dc60810e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.967+0000 7f6d37400640 1 -- 172.21.15.59:0/1766346654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28036070 con 0x7f6d3815cb20 2024-04-24T06:38:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.967+0000 7f6d37400640 1 -- 172.21.15.59:0/1766346654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28031600 con 0x7f6d3815cb20 2024-04-24T06:38:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.967+0000 7f6d37400640 1 -- 172.21.15.59:0/1766346654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d280319d0 con 0x7f6d3815cb20 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 -- 172.21.15.59:0/1766346654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3815cb20 msgr2=0x7f6d380cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 --2- 172.21.15.59:0/1766346654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3815cb20 0x7f6d380cf840 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6d3806dc70 tx=0x7f6d2802a660 comp rx=0 tx=0).stop 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 -- 172.21.15.59:0/1766346654 shutdown_connections 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 --2- 172.21.15.59:0/1766346654 >> v2:172.21.15.66:3300/0 conn(0x7f6d38163770 0x7f6d380d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 --2- 172.21.15.59:0/1766346654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3815cb20 0x7f6d380cf840 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 -- 172.21.15.59:0/1766346654 >> 172.21.15.59:0/1766346654 conn(0x7f6d3806cc70 msgr2=0x7f6d3806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 -- 172.21.15.59:0/1766346654 shutdown_connections 2024-04-24T06:38:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.968+0000 7f6d3f800640 1 -- 172.21.15.59:0/1766346654 wait complete. 2024-04-24T06:38:42.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.970+0000 7f6d3f800640 1 Processor -- start 2024-04-24T06:38:42.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.970+0000 7f6d3f800640 1 -- start start 2024-04-24T06:38:42.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6d3815cb20 0x7f6d380b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d38160180 0x7f6d381561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d380d1900 con 0x7f6d38163770 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d380d1780 con 0x7f6d38160180 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6d380d1a80 con 0x7f6d3815cb20 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44398/0 (socket says 172.21.15.59:44398) 2024-04-24T06:38:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.972+0000 7f6d3d800640 1 -- 172.21.15.59:0/2726679414 learned_addr learned my addr 172.21.15.59:0/2726679414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 -- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6d3815cb20 msgr2=0x7f6d380b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6d3815cb20 0x7f6d380b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 -- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d38160180 msgr2=0x7f6d381561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d37e00640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d38160180 0x7f6d381561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d38160180 0x7f6d381561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 -- 172.21.15.59:0/2726679414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d380b9060 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3d800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c440 tx=0x7f6d3000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30011790 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d30017070 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30011b70 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d380b9340 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.973+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d38158990 con 0x7f6d38163770 2024-04-24T06:38:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.975+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d380b9860 con 0x7f6d38163770 2024-04-24T06:38:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.976+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d30011d10 con 0x7f6d38163770 2024-04-24T06:38:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.977+0000 7f6d34c00640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 0x7f6d1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.977+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f6d3008ba60 con 0x7f6d38163770 2024-04-24T06:38:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.977+0000 7f6d3ce00640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 0x7f6d1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.978+0000 7f6d3ce00640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 0x7f6d1406edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6d28033000 tx=0x7f6d2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:42.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:42.981+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d380b9860 con 0x7f6d38163770 2024-04-24T06:38:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.186+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d38158b90 con 0x7f6d38163770 2024-04-24T06:38:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.189+0000 7f6d34c00640 1 -- 172.21.15.59:0/2726679414 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16524 (secure 0 0 0) 0x7f6d38158b90 con 0x7f6d38163770 2024-04-24T06:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:43.191 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:40.185327+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 msgr2=0x7f6d1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 0x7f6d1406edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6d28033000 tx=0x7f6d2802e000 comp rx=0 tx=0).stop 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 msgr2=0x7f6d381566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c440 tx=0x7f6d3000c900 comp rx=0 tx=0).stop 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 shutdown_connections 2024-04-24T06:38:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6d1406c8f0 0x7f6d1406edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d38163770 0x7f6d381566f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.191+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d38160180 0x7f6d381561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.192+0000 7f6d3f800640 1 --2- 172.21.15.59:0/2726679414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6d3815cb20 0x7f6d380b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.192+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 >> 172.21.15.59:0/2726679414 conn(0x7f6d3806cc70 msgr2=0x7f6d3806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.192+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 shutdown_connections 2024-04-24T06:38:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.192+0000 7f6d3f800640 1 -- 172.21.15.59:0/2726679414 wait complete. 2024-04-24T06:38:43.211 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:38:43.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:38:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.412+0000 7fc488800640 1 Processor -- start 2024-04-24T06:38:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.412+0000 7fc488800640 1 -- start start 2024-04-24T06:38:43.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.413+0000 7fc488800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc48006f7c0 0x7fc4800b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc488800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc48006ec60 con 0x7fc48006df00 2024-04-24T06:38:43.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc488800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4800b46e0 con 0x7fc48015b840 2024-04-24T06:38:43.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc488800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc4800b49c0 con 0x7fc48006f7c0 2024-04-24T06:38:43.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc485e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc48006f7c0 0x7fc4800b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc486800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc48006df00 0x7fc48006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51206/0 (socket says 172.21.15.59:51206) 2024-04-24T06:38:43.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc486800640 1 -- 172.21.15.59:0/1907892064 learned_addr learned my addr 172.21.15.59:0/1907892064 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:43.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc485e00640 1 -- 172.21.15.59:0/1907892064 >> v1:172.21.15.66:6789/0 conn(0x7fc48015b840 legacy=0x7fc48006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:43.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc485e00640 1 -- 172.21.15.59:0/1907892064 >> v1:172.21.15.59:6789/0 conn(0x7fc48006df00 legacy=0x7fc48006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc485e00640 1 -- 172.21.15.59:0/1907892064 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4800b4ca0 con 0x7fc48006f7c0 2024-04-24T06:38:43.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.414+0000 7fc484a00640 1 -- 172.21.15.59:0/1907892064 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061298225 0 0) 0x7fc48006ec60 con 0x7fc48006df00 2024-04-24T06:38:43.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc485e00640 1 --2- 172.21.15.59:0/1907892064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006f7c0 0x7fc4800b54a0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc47402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93177dff8a468cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc484a00640 1 -- 172.21.15.59:0/1907892064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474007740 con 0x7fc48006f7c0 2024-04-24T06:38:43.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc484a00640 1 -- 172.21.15.59:0/1907892064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4740078e0 con 0x7fc48006f7c0 2024-04-24T06:38:43.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc484a00640 1 -- 172.21.15.59:0/1907892064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474007cb0 con 0x7fc48006f7c0 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc488800640 1 -- 172.21.15.59:0/1907892064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006f7c0 msgr2=0x7fc4800b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.416+0000 7fc488800640 1 --2- 172.21.15.59:0/1907892064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006f7c0 0x7fc4800b54a0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc47402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.417+0000 7fc488800640 1 -- 172.21.15.59:0/1907892064 shutdown_connections 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.417+0000 7fc488800640 1 --2- 172.21.15.59:0/1907892064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006f7c0 0x7fc4800b54a0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.417+0000 7fc488800640 1 -- 172.21.15.59:0/1907892064 >> 172.21.15.59:0/1907892064 conn(0x7fc480067ea0 msgr2=0x7fc48006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.417+0000 7fc488800640 1 -- 172.21.15.59:0/1907892064 shutdown_connections 2024-04-24T06:38:43.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.417+0000 7fc488800640 1 -- 172.21.15.59:0/1907892064 wait complete. 2024-04-24T06:38:43.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.418+0000 7fc488800640 1 Processor -- start 2024-04-24T06:38:43.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.419+0000 7fc488800640 1 -- start start 2024-04-24T06:38:43.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.419+0000 7fc488800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc488800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 0x7fc480169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc488800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4800ce8b0 0x7fc480169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc488800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4800ce340 con 0x7fc48006df00 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc488800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4800ce1c0 con 0x7fc4800ce8b0 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc488800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc4800ce4c0 con 0x7fc48006f7c0 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44428/0 (socket says 172.21.15.59:44428) 2024-04-24T06:38:43.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 -- 172.21.15.59:0/2621768967 learned_addr learned my addr 172.21.15.59:0/2621768967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:43.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485400640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 0x7fc480169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 -- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 msgr2=0x7fc480169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc486800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4800ce8b0 0x7fc480169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 0x7fc480169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 -- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4800ce8b0 msgr2=0x7fc480169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4800ce8b0 0x7fc480169a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485e00640 1 -- 172.21.15.59:0/2621768967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48014a950 con 0x7fc48006df00 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.420+0000 7fc485400640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 0x7fc480169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc485e00640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc474004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474040070 con 0x7fc48006df00 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc474031d00 con 0x7fc48006df00 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48014ac30 con 0x7fc48006df00 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474045680 con 0x7fc48006df00 2024-04-24T06:38:43.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.421+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48014b1f0 con 0x7fc48006df00 2024-04-24T06:38:43.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.422+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc480068a80 con 0x7fc48006df00 2024-04-24T06:38:43.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.424+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc474039040 con 0x7fc48006df00 2024-04-24T06:38:43.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.425+0000 7fc47a000640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.425+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fc4740ad8e0 con 0x7fc48006df00 2024-04-24T06:38:43.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.425+0000 7fc485400640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.426+0000 7fc485400640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 0x7fc46406edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc480142660 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.428+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc47407bea0 con 0x7fc48006df00 2024-04-24T06:38:43.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.632+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc48006f370 con 0x7fc46406c8f0 2024-04-24T06:38:43.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.650+0000 7fc47a000640 1 -- 172.21.15.59:0/2621768967 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7fc48006f370 con 0x7fc46406c8f0 2024-04-24T06:38:43.656 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:43.658 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:38:43.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 msgr2=0x7fc46406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 0x7fc46406edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc480142660 tx=0x7fc470008040 comp rx=0 tx=0).stop 2024-04-24T06:38:43.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 msgr2=0x7fc48016ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc474004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 shutdown_connections 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4800ce8b0 0x7fc480169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc48006f7c0 0x7fc480169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 --2- 172.21.15.59:0/2621768967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc48006df00 0x7fc48016ac50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 >> 172.21.15.59:0/2621768967 conn(0x7fc480067ea0 msgr2=0x7fc48015aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 shutdown_connections 2024-04-24T06:38:43.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.657+0000 7fc488800640 1 -- 172.21.15.59:0/2621768967 wait complete. 2024-04-24T06:38:43.679 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-24T06:38:43.398579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260184,"kb_used_data":10696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714536,"statfs":{"total":773094113280,"available":772827684864,"internally_reserved":0,"allocated":10952704,"data_stored":5378796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001366"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248098+0000","last_change":"2024-04-24T06:38:39.198572+0000","last_active":"2024-04-24T06:38:40.248098+0000","last_peered":"2024-04-24T06:38:40.248098+0000","last_clean":"2024-04-24T06:38:40.248098+0000","last_became_active":"2024-04-24T06:38:39.195884+0000","last_became_peered":"2024-04-24T06:38:39.195884+0000","last_unstale":"2024-04-24T06:38:40.248098+0000","last_undegraded":"2024-04-24T06:38:40.248098+0000","last_fullsized":"2024-04-24T06:38:40.248098+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:57.136731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345613+0000","last_change":"2024-04-24T06:38:39.199816+0000","last_active":"2024-04-24T06:38:40.345613+0000","last_peered":"2024-04-24T06:38:40.345613+0000","last_clean":"2024-04-24T06:38:40.345613+0000","last_became_active":"2024-04-24T06:38:39.199538+0000","last_became_peered":"2024-04-24T06:38:39.199538+0000","last_unstale":"2024-04-24T06:38:40.345613+0000","last_undegraded":"2024-04-24T06:38:40.345613+0000","last_fullsized":"2024-04-24T06:38:40.345613+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:01.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247699+0000","last_change":"2024-04-24T06:38:39.195344+0000","last_active":"2024-04-24T06:38:40.247699+0000","last_peered":"2024-04-24T06:38:40.247699+0000","last_clean":"2024-04-24T06:38:40.247699+0000","last_became_active":"2024-04-24T06:38:39.195073+0000","last_became_peered":"2024-04-24T06:38:39.195073+0000","last_unstale":"2024-04-24T06:38:40.247699+0000","last_undegraded":"2024-04-24T06:38:40.247699+0000","last_fullsized":"2024-04-24T06:38:40.247699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:18.706878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193223+0000","last_change":"2024-04-24T06:38:39.207297+0000","last_active":"2024-04-24T06:38:40.193223+0000","last_peered":"2024-04-24T06:38:40.193223+0000","last_clean":"2024-04-24T06:38:40.193223+0000","last_became_active":"2024-04-24T06:38:39.206852+0000","last_became_peered":"2024-04-24T06:38:39.206852+0000","last_unstale":"2024-04-24T06:38:40.193223+0000","last_undegraded":"2024-04-24T06:38:40.193223+0000","last_fullsized":"2024-04-24T06:38:40.193223+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:16:32.299278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347890+0000","last_change":"2024-04-24T06:38:39.206116+0000","last_active":"2024-04-24T06:38:40.347890+0000","last_peered":"2024-04-24T06:38:40.347890+0000","last_clean":"2024-04-24T06:38:40.347890+0000","last_became_active":"2024-04-24T06:38:39.205430+0000","last_became_peered":"2024-04-24T06:38:39.205430+0000","last_unstale":"2024-04-24T06:38:40.347890+0000","last_undegraded":"2024-04-24T06:38:40.347890+0000","last_fullsized":"2024-04-24T06:38:40.347890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349677+0000","last_change":"2024-04-24T06:38:39.203926+0000","last_active":"2024-04-24T06:38:40.349677+0000","last_peered":"2024-04-24T06:38:40.349677+0000","last_clean":"2024-04-24T06:38:40.349677+0000","last_became_active":"2024-04-24T06:38:39.203577+0000","last_became_peered":"2024-04-24T06:38:39.203577+0000","last_unstale":"2024-04-24T06:38:40.349677+0000","last_undegraded":"2024-04-24T06:38:40.349677+0000","last_fullsized":"2024-04-24T06:38:40.349677+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346089+0000","last_change":"2024-04-24T06:38:39.203297+0000","last_active":"2024-04-24T06:38:40.346089+0000","last_peered":"2024-04-24T06:38:40.346089+0000","last_clean":"2024-04-24T06:38:40.346089+0000","last_became_active":"2024-04-24T06:38:39.202949+0000","last_became_peered":"2024-04-24T06:38:39.202949+0000","last_unstale":"2024-04-24T06:38:40.346089+0000","last_undegraded":"2024-04-24T06:38:40.346089+0000","last_fullsized":"2024-04-24T06:38:40.346089+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:36.001714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343208+0000","last_change":"2024-04-24T06:38:39.192584+0000","last_active":"2024-04-24T06:38:40.343208+0000","last_peered":"2024-04-24T06:38:40.343208+0000","last_clean":"2024-04-24T06:38:40.343208+0000","last_became_active":"2024-04-24T06:38:39.191588+0000","last_became_peered":"2024-04-24T06:38:39.191588+0000","last_unstale":"2024-04-24T06:38:40.343208+0000","last_undegraded":"2024-04-24T06:38:40.343208+0000","last_fullsized":"2024-04-24T06:38:40.343208+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197699+0000","last_change":"2024-04-24T06:38:39.203403+0000","last_active":"2024-04-24T06:38:40.197699+0000","last_peered":"2024-04-24T06:38:40.197699+0000","last_clean":"2024-04-24T06:38:40.197699+0000","last_became_active":"2024-04-24T06:38:39.203123+0000","last_became_peered":"2024-04-24T06:38:39.203123+0000","last_unstale":"2024-04-24T06:38:40.197699+0000","last_undegraded":"2024-04-24T06:38:40.197699+0000","last_fullsized":"2024-04-24T06:38:40.197699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348984+0000","last_change":"2024-04-24T06:38:39.195955+0000","last_active":"2024-04-24T06:38:40.348984+0000","last_peered":"2024-04-24T06:38:40.348984+0000","last_clean":"2024-04-24T06:38:40.348984+0000","last_became_active":"2024-04-24T06:38:39.195719+0000","last_became_peered":"2024-04-24T06:38:39.195719+0000","last_unstale":"2024-04-24T06:38:40.348984+0000","last_undegraded":"2024-04-24T06:38:40.348984+0000","last_fullsized":"2024-04-24T06:38:40.348984+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346307+0000","last_change":"2024-04-24T06:38:39.198347+0000","last_active":"2024-04-24T06:38:40.346307+0000","last_peered":"2024-04-24T06:38:40.346307+0000","last_clean":"2024-04-24T06:38:40.346307+0000","last_became_active":"2024-04-24T06:38:39.197747+0000","last_became_peered":"2024-04-24T06:38:39.197747+0000","last_unstale":"2024-04-24T06:38:40.346307+0000","last_undegraded":"2024-04-24T06:38:40.346307+0000","last_fullsized":"2024-04-24T06:38:40.346307+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197455+0000","last_change":"2024-04-24T06:38:39.188824+0000","last_active":"2024-04-24T06:38:40.197455+0000","last_peered":"2024-04-24T06:38:40.197455+0000","last_clean":"2024-04-24T06:38:40.197455+0000","last_became_active":"2024-04-24T06:38:39.188507+0000","last_became_peered":"2024-04-24T06:38:39.188507+0000","last_unstale":"2024-04-24T06:38:40.197455+0000","last_undegraded":"2024-04-24T06:38:40.197455+0000","last_fullsized":"2024-04-24T06:38:40.197455+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:02.364072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193234+0000","last_change":"2024-04-24T06:38:39.211246+0000","last_active":"2024-04-24T06:38:40.193234+0000","last_peered":"2024-04-24T06:38:40.193234+0000","last_clean":"2024-04-24T06:38:40.193234+0000","last_became_active":"2024-04-24T06:38:39.210923+0000","last_became_peered":"2024-04-24T06:38:39.210923+0000","last_unstale":"2024-04-24T06:38:40.193234+0000","last_undegraded":"2024-04-24T06:38:40.193234+0000","last_fullsized":"2024-04-24T06:38:40.193234+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:01.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343542+0000","last_change":"2024-04-24T06:38:39.191027+0000","last_active":"2024-04-24T06:38:40.343542+0000","last_peered":"2024-04-24T06:38:40.343542+0000","last_clean":"2024-04-24T06:38:40.343542+0000","last_became_active":"2024-04-24T06:38:39.189396+0000","last_became_peered":"2024-04-24T06:38:39.189396+0000","last_unstale":"2024-04-24T06:38:40.343542+0000","last_undegraded":"2024-04-24T06:38:40.343542+0000","last_fullsized":"2024-04-24T06:38:40.343542+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:45.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192771+0000","last_change":"2024-04-24T06:38:39.211694+0000","last_active":"2024-04-24T06:38:40.192771+0000","last_peered":"2024-04-24T06:38:40.192771+0000","last_clean":"2024-04-24T06:38:40.192771+0000","last_became_active":"2024-04-24T06:38:39.211408+0000","last_became_peered":"2024-04-24T06:38:39.211408+0000","last_unstale":"2024-04-24T06:38:40.192771+0000","last_undegraded":"2024-04-24T06:38:40.192771+0000","last_fullsized":"2024-04-24T06:38:40.192771+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346664+0000","last_change":"2024-04-24T06:38:39.193528+0000","last_active":"2024-04-24T06:38:40.346664+0000","last_peered":"2024-04-24T06:38:40.346664+0000","last_clean":"2024-04-24T06:38:40.346664+0000","last_became_active":"2024-04-24T06:38:39.193166+0000","last_became_peered":"2024-04-24T06:38:39.193166+0000","last_unstale":"2024-04-24T06:38:40.346664+0000","last_undegraded":"2024-04-24T06:38:40.346664+0000","last_fullsized":"2024-04-24T06:38:40.346664+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345867+0000","last_change":"2024-04-24T06:38:39.202517+0000","last_active":"2024-04-24T06:38:40.345867+0000","last_peered":"2024-04-24T06:38:40.345867+0000","last_clean":"2024-04-24T06:38:40.345867+0000","last_became_active":"2024-04-24T06:38:39.202228+0000","last_became_peered":"2024-04-24T06:38:39.202228+0000","last_unstale":"2024-04-24T06:38:40.345867+0000","last_undegraded":"2024-04-24T06:38:40.345867+0000","last_fullsized":"2024-04-24T06:38:40.345867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:55.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193674+0000","last_change":"2024-04-24T06:38:39.207496+0000","last_active":"2024-04-24T06:38:40.193674+0000","last_peered":"2024-04-24T06:38:40.193674+0000","last_clean":"2024-04-24T06:38:40.193674+0000","last_became_active":"2024-04-24T06:38:39.207147+0000","last_became_peered":"2024-04-24T06:38:39.207147+0000","last_unstale":"2024-04-24T06:38:40.193674+0000","last_undegraded":"2024-04-24T06:38:40.193674+0000","last_fullsized":"2024-04-24T06:38:40.193674+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:05.276766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343725+0000","last_change":"2024-04-24T06:38:39.192262+0000","last_active":"2024-04-24T06:38:40.343725+0000","last_peered":"2024-04-24T06:38:40.343725+0000","last_clean":"2024-04-24T06:38:40.343725+0000","last_became_active":"2024-04-24T06:38:39.191884+0000","last_became_peered":"2024-04-24T06:38:39.191884+0000","last_unstale":"2024-04-24T06:38:40.343725+0000","last_undegraded":"2024-04-24T06:38:40.343725+0000","last_fullsized":"2024-04-24T06:38:40.343725+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:37.831589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197787+0000","last_change":"2024-04-24T06:38:39.208616+0000","last_active":"2024-04-24T06:38:40.197787+0000","last_peered":"2024-04-24T06:38:40.197787+0000","last_clean":"2024-04-24T06:38:40.197787+0000","last_became_active":"2024-04-24T06:38:39.206997+0000","last_became_peered":"2024-04-24T06:38:39.206997+0000","last_unstale":"2024-04-24T06:38:40.197787+0000","last_undegraded":"2024-04-24T06:38:40.197787+0000","last_fullsized":"2024-04-24T06:38:40.197787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344973+0000","last_change":"2024-04-24T06:38:39.197394+0000","last_active":"2024-04-24T06:38:40.344973+0000","last_peered":"2024-04-24T06:38:40.344973+0000","last_clean":"2024-04-24T06:38:40.344973+0000","last_became_active":"2024-04-24T06:38:39.197125+0000","last_became_peered":"2024-04-24T06:38:39.197125+0000","last_unstale":"2024-04-24T06:38:40.344973+0000","last_undegraded":"2024-04-24T06:38:40.344973+0000","last_fullsized":"2024-04-24T06:38:40.344973+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:20:44.787089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343665+0000","last_change":"2024-04-24T06:38:39.194001+0000","last_active":"2024-04-24T06:38:40.343665+0000","last_peered":"2024-04-24T06:38:40.343665+0000","last_clean":"2024-04-24T06:38:40.343665+0000","last_became_active":"2024-04-24T06:38:39.193724+0000","last_became_peered":"2024-04-24T06:38:39.193724+0000","last_unstale":"2024-04-24T06:38:40.343665+0000","last_undegraded":"2024-04-24T06:38:40.343665+0000","last_fullsized":"2024-04-24T06:38:40.343665+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343257+0000","last_change":"2024-04-24T06:38:39.194212+0000","last_active":"2024-04-24T06:38:40.343257+0000","last_peered":"2024-04-24T06:38:40.343257+0000","last_clean":"2024-04-24T06:38:40.343257+0000","last_became_active":"2024-04-24T06:38:39.192153+0000","last_became_peered":"2024-04-24T06:38:39.192153+0000","last_unstale":"2024-04-24T06:38:40.343257+0000","last_undegraded":"2024-04-24T06:38:40.343257+0000","last_fullsized":"2024-04-24T06:38:40.343257+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346103+0000","last_change":"2024-04-24T06:38:39.194167+0000","last_active":"2024-04-24T06:38:40.346103+0000","last_peered":"2024-04-24T06:38:40.346103+0000","last_clean":"2024-04-24T06:38:40.346103+0000","last_became_active":"2024-04-24T06:38:39.193438+0000","last_became_peered":"2024-04-24T06:38:39.193438+0000","last_unstale":"2024-04-24T06:38:40.346103+0000","last_undegraded":"2024-04-24T06:38:40.346103+0000","last_fullsized":"2024-04-24T06:38:40.346103+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349062+0000","last_change":"2024-04-24T06:38:39.187275+0000","last_active":"2024-04-24T06:38:40.349062+0000","last_peered":"2024-04-24T06:38:40.349062+0000","last_clean":"2024-04-24T06:38:40.349062+0000","last_became_active":"2024-04-24T06:38:39.186183+0000","last_became_peered":"2024-04-24T06:38:39.186183+0000","last_unstale":"2024-04-24T06:38:40.349062+0000","last_undegraded":"2024-04-24T06:38:40.349062+0000","last_fullsized":"2024-04-24T06:38:40.349062+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345708+0000","last_change":"2024-04-24T06:38:39.193254+0000","last_active":"2024-04-24T06:38:40.345708+0000","last_peered":"2024-04-24T06:38:40.345708+0000","last_clean":"2024-04-24T06:38:40.345708+0000","last_became_active":"2024-04-24T06:38:39.192235+0000","last_became_peered":"2024-04-24T06:38:39.192235+0000","last_unstale":"2024-04-24T06:38:40.345708+0000","last_undegraded":"2024-04-24T06:38:40.345708+0000","last_fullsized":"2024-04-24T06:38:40.345708+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:05.276766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346818+0000","last_change":"2024-04-24T06:38:39.190838+0000","last_active":"2024-04-24T06:38:40.346818+0000","last_peered":"2024-04-24T06:38:40.346818+0000","last_clean":"2024-04-24T06:38:40.346818+0000","last_became_active":"2024-04-24T06:38:39.190537+0000","last_became_peered":"2024-04-24T06:38:39.190537+0000","last_unstale":"2024-04-24T06:38:40.346818+0000","last_undegraded":"2024-04-24T06:38:40.346818+0000","last_fullsized":"2024-04-24T06:38:40.346818+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348678+0000","last_change":"2024-04-24T06:38:39.203029+0000","last_active":"2024-04-24T06:38:40.348678+0000","last_peered":"2024-04-24T06:38:40.348678+0000","last_clean":"2024-04-24T06:38:40.348678+0000","last_became_active":"2024-04-24T06:38:39.202726+0000","last_became_peered":"2024-04-24T06:38:39.202726+0000","last_unstale":"2024-04-24T06:38:40.348678+0000","last_undegraded":"2024-04-24T06:38:40.348678+0000","last_fullsized":"2024-04-24T06:38:40.348678+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344085+0000","last_change":"2024-04-24T06:38:39.196567+0000","last_active":"2024-04-24T06:38:40.344085+0000","last_peered":"2024-04-24T06:38:40.344085+0000","last_clean":"2024-04-24T06:38:40.344085+0000","last_became_active":"2024-04-24T06:38:39.189089+0000","last_became_peered":"2024-04-24T06:38:39.189089+0000","last_unstale":"2024-04-24T06:38:40.344085+0000","last_undegraded":"2024-04-24T06:38:40.344085+0000","last_fullsized":"2024-04-24T06:38:40.344085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:16:32.299278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343698+0000","last_change":"2024-04-24T06:38:39.202795+0000","last_active":"2024-04-24T06:38:40.343698+0000","last_peered":"2024-04-24T06:38:40.343698+0000","last_clean":"2024-04-24T06:38:40.343698+0000","last_became_active":"2024-04-24T06:38:39.196550+0000","last_became_peered":"2024-04-24T06:38:39.196550+0000","last_unstale":"2024-04-24T06:38:40.343698+0000","last_undegraded":"2024-04-24T06:38:40.343698+0000","last_fullsized":"2024-04-24T06:38:40.343698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:16.574285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193154+0000","last_change":"2024-04-24T06:38:39.210545+0000","last_active":"2024-04-24T06:38:40.193154+0000","last_peered":"2024-04-24T06:38:40.193154+0000","last_clean":"2024-04-24T06:38:40.193154+0000","last_became_active":"2024-04-24T06:38:39.210048+0000","last_became_peered":"2024-04-24T06:38:39.210048+0000","last_unstale":"2024-04-24T06:38:40.193154+0000","last_undegraded":"2024-04-24T06:38:40.193154+0000","last_fullsized":"2024-04-24T06:38:40.193154+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247266+0000","last_change":"2024-04-24T06:38:39.193897+0000","last_active":"2024-04-24T06:38:40.247266+0000","last_peered":"2024-04-24T06:38:40.247266+0000","last_clean":"2024-04-24T06:38:40.247266+0000","last_became_active":"2024-04-24T06:38:39.193625+0000","last_became_peered":"2024-04-24T06:38:39.193625+0000","last_unstale":"2024-04-24T06:38:40.247266+0000","last_undegraded":"2024-04-24T06:38:40.247266+0000","last_fullsized":"2024-04-24T06:38:40.247266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:42.761887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344269+0000","last_change":"2024-04-24T06:38:39.199835+0000","last_active":"2024-04-24T06:38:40.344269+0000","last_peered":"2024-04-24T06:38:40.344269+0000","last_clean":"2024-04-24T06:38:40.344269+0000","last_became_active":"2024-04-24T06:38:39.196133+0000","last_became_peered":"2024-04-24T06:38:39.196133+0000","last_unstale":"2024-04-24T06:38:40.344269+0000","last_undegraded":"2024-04-24T06:38:40.344269+0000","last_fullsized":"2024-04-24T06:38:40.344269+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:46:20.162332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"97'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345720+0000","last_change":"2024-04-24T06:38:38.198883+0000","last_active":"2024-04-24T06:38:40.345720+0000","last_peered":"2024-04-24T06:38:40.345720+0000","last_clean":"2024-04-24T06:38:40.345720+0000","last_became_active":"2024-04-24T06:38:38.198568+0000","last_became_peered":"2024-04-24T06:38:38.198568+0000","last_unstale":"2024-04-24T06:38:40.345720+0000","last_undegraded":"2024-04-24T06:38:40.345720+0000","last_fullsized":"2024-04-24T06:38:40.345720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:58.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193898+0000","last_change":"2024-04-24T06:38:39.210232+0000","last_active":"2024-04-24T06:38:40.193898+0000","last_peered":"2024-04-24T06:38:40.193898+0000","last_clean":"2024-04-24T06:38:40.193898+0000","last_became_active":"2024-04-24T06:38:39.209896+0000","last_became_peered":"2024-04-24T06:38:39.209896+0000","last_unstale":"2024-04-24T06:38:40.193898+0000","last_undegraded":"2024-04-24T06:38:40.193898+0000","last_fullsized":"2024-04-24T06:38:40.193898+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:55.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350088+0000","last_change":"2024-04-24T06:38:38.196394+0000","last_active":"2024-04-24T06:38:40.350088+0000","last_peered":"2024-04-24T06:38:40.350088+0000","last_clean":"2024-04-24T06:38:40.350088+0000","last_became_active":"2024-04-24T06:38:38.195341+0000","last_became_peered":"2024-04-24T06:38:38.195341+0000","last_unstale":"2024-04-24T06:38:40.350088+0000","last_undegraded":"2024-04-24T06:38:40.350088+0000","last_fullsized":"2024-04-24T06:38:40.350088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248025+0000","last_change":"2024-04-24T06:38:39.204023+0000","last_active":"2024-04-24T06:38:40.248025+0000","last_peered":"2024-04-24T06:38:40.248025+0000","last_clean":"2024-04-24T06:38:40.248025+0000","last_became_active":"2024-04-24T06:38:39.202732+0000","last_became_peered":"2024-04-24T06:38:39.202732+0000","last_unstale":"2024-04-24T06:38:40.248025+0000","last_undegraded":"2024-04-24T06:38:40.248025+0000","last_fullsized":"2024-04-24T06:38:40.248025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:50.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345427+0000","last_change":"2024-04-24T06:38:38.200782+0000","last_active":"2024-04-24T06:38:40.345427+0000","last_peered":"2024-04-24T06:38:40.345427+0000","last_clean":"2024-04-24T06:38:40.345427+0000","last_became_active":"2024-04-24T06:38:38.200307+0000","last_became_peered":"2024-04-24T06:38:38.200307+0000","last_unstale":"2024-04-24T06:38:40.345427+0000","last_undegraded":"2024-04-24T06:38:40.345427+0000","last_fullsized":"2024-04-24T06:38:40.345427+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:21.855834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.198173+0000","last_change":"2024-04-24T06:38:39.199890+0000","last_active":"2024-04-24T06:38:40.198173+0000","last_peered":"2024-04-24T06:38:40.198173+0000","last_clean":"2024-04-24T06:38:40.198173+0000","last_became_active":"2024-04-24T06:38:39.199271+0000","last_became_peered":"2024-04-24T06:38:39.199271+0000","last_unstale":"2024-04-24T06:38:40.198173+0000","last_undegraded":"2024-04-24T06:38:40.198173+0000","last_fullsized":"2024-04-24T06:38:40.198173+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344424+0000","last_change":"2024-04-24T06:38:38.194595+0000","last_active":"2024-04-24T06:38:40.344424+0000","last_peered":"2024-04-24T06:38:40.344424+0000","last_clean":"2024-04-24T06:38:40.344424+0000","last_became_active":"2024-04-24T06:38:38.192661+0000","last_became_peered":"2024-04-24T06:38:38.192661+0000","last_unstale":"2024-04-24T06:38:40.344424+0000","last_undegraded":"2024-04-24T06:38:40.344424+0000","last_fullsized":"2024-04-24T06:38:40.344424+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346578+0000","last_change":"2024-04-24T06:38:39.201176+0000","last_active":"2024-04-24T06:38:40.346578+0000","last_peered":"2024-04-24T06:38:40.346578+0000","last_clean":"2024-04-24T06:38:40.346578+0000","last_became_active":"2024-04-24T06:38:39.200887+0000","last_became_peered":"2024-04-24T06:38:39.200887+0000","last_unstale":"2024-04-24T06:38:40.346578+0000","last_undegraded":"2024-04-24T06:38:40.346578+0000","last_fullsized":"2024-04-24T06:38:40.346578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:34.525473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247516+0000","last_change":"2024-04-24T06:38:38.198972+0000","last_active":"2024-04-24T06:38:40.247516+0000","last_peered":"2024-04-24T06:38:40.247516+0000","last_clean":"2024-04-24T06:38:40.247516+0000","last_became_active":"2024-04-24T06:38:38.198249+0000","last_became_peered":"2024-04-24T06:38:38.198249+0000","last_unstale":"2024-04-24T06:38:40.247516+0000","last_undegraded":"2024-04-24T06:38:40.247516+0000","last_fullsized":"2024-04-24T06:38:40.247516+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:09:07.925689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349760+0000","last_change":"2024-04-24T06:38:39.185960+0000","last_active":"2024-04-24T06:38:40.349760+0000","last_peered":"2024-04-24T06:38:40.349760+0000","last_clean":"2024-04-24T06:38:40.349760+0000","last_became_active":"2024-04-24T06:38:39.185662+0000","last_became_peered":"2024-04-24T06:38:39.185662+0000","last_unstale":"2024-04-24T06:38:40.349760+0000","last_undegraded":"2024-04-24T06:38:40.349760+0000","last_fullsized":"2024-04-24T06:38:40.349760+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343670+0000","last_change":"2024-04-24T06:38:38.194361+0000","last_active":"2024-04-24T06:38:40.343670+0000","last_peered":"2024-04-24T06:38:40.343670+0000","last_clean":"2024-04-24T06:38:40.343670+0000","last_became_active":"2024-04-24T06:38:38.194069+0000","last_became_peered":"2024-04-24T06:38:38.194069+0000","last_unstale":"2024-04-24T06:38:40.343670+0000","last_undegraded":"2024-04-24T06:38:40.343670+0000","last_fullsized":"2024-04-24T06:38:40.343670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:00.907566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192629+0000","last_change":"2024-04-24T06:38:39.198515+0000","last_active":"2024-04-24T06:38:40.192629+0000","last_peered":"2024-04-24T06:38:40.192629+0000","last_clean":"2024-04-24T06:38:40.192629+0000","last_became_active":"2024-04-24T06:38:39.198226+0000","last_became_peered":"2024-04-24T06:38:39.198226+0000","last_unstale":"2024-04-24T06:38:40.192629+0000","last_undegraded":"2024-04-24T06:38:40.192629+0000","last_fullsized":"2024-04-24T06:38:40.192629+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348265+0000","last_change":"2024-04-24T06:38:38.196983+0000","last_active":"2024-04-24T06:38:40.348265+0000","last_peered":"2024-04-24T06:38:40.348265+0000","last_clean":"2024-04-24T06:38:40.348265+0000","last_became_active":"2024-04-24T06:38:38.196630+0000","last_became_peered":"2024-04-24T06:38:38.196630+0000","last_unstale":"2024-04-24T06:38:40.348265+0000","last_undegraded":"2024-04-24T06:38:40.348265+0000","last_fullsized":"2024-04-24T06:38:40.348265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197283+0000","last_change":"2024-04-24T06:38:39.199272+0000","last_active":"2024-04-24T06:38:40.197283+0000","last_peered":"2024-04-24T06:38:40.197283+0000","last_clean":"2024-04-24T06:38:40.197283+0000","last_became_active":"2024-04-24T06:38:39.196423+0000","last_became_peered":"2024-04-24T06:38:39.196423+0000","last_unstale":"2024-04-24T06:38:40.197283+0000","last_undegraded":"2024-04-24T06:38:40.197283+0000","last_fullsized":"2024-04-24T06:38:40.197283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:06.077317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247261+0000","last_change":"2024-04-24T06:38:38.185510+0000","last_active":"2024-04-24T06:38:40.247261+0000","last_peered":"2024-04-24T06:38:40.247261+0000","last_clean":"2024-04-24T06:38:40.247261+0000","last_became_active":"2024-04-24T06:38:38.185175+0000","last_became_peered":"2024-04-24T06:38:38.185175+0000","last_unstale":"2024-04-24T06:38:40.247261+0000","last_undegraded":"2024-04-24T06:38:40.247261+0000","last_fullsized":"2024-04-24T06:38:40.247261+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:12:36.252749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345900+0000","last_change":"2024-04-24T06:38:39.191902+0000","last_active":"2024-04-24T06:38:40.345900+0000","last_peered":"2024-04-24T06:38:40.345900+0000","last_clean":"2024-04-24T06:38:40.345900+0000","last_became_active":"2024-04-24T06:38:39.189373+0000","last_became_peered":"2024-04-24T06:38:39.189373+0000","last_unstale":"2024-04-24T06:38:40.345900+0000","last_undegraded":"2024-04-24T06:38:40.345900+0000","last_fullsized":"2024-04-24T06:38:40.345900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:44.646457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248073+0000","last_change":"2024-04-24T06:38:38.187061+0000","last_active":"2024-04-24T06:38:40.248073+0000","last_peered":"2024-04-24T06:38:40.248073+0000","last_clean":"2024-04-24T06:38:40.248073+0000","last_became_active":"2024-04-24T06:38:38.186579+0000","last_became_peered":"2024-04-24T06:38:38.186579+0000","last_unstale":"2024-04-24T06:38:40.248073+0000","last_undegraded":"2024-04-24T06:38:40.248073+0000","last_fullsized":"2024-04-24T06:38:40.248073+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:20.407609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197838+0000","last_change":"2024-04-24T06:38:39.200872+0000","last_active":"2024-04-24T06:38:40.197838+0000","last_peered":"2024-04-24T06:38:40.197838+0000","last_clean":"2024-04-24T06:38:40.197838+0000","last_became_active":"2024-04-24T06:38:39.200381+0000","last_became_peered":"2024-04-24T06:38:39.200381+0000","last_unstale":"2024-04-24T06:38:40.197838+0000","last_undegraded":"2024-04-24T06:38:40.197838+0000","last_fullsized":"2024-04-24T06:38:40.197838+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193684+0000","last_change":"2024-04-24T06:38:38.190498+0000","last_active":"2024-04-24T06:38:40.193684+0000","last_peered":"2024-04-24T06:38:40.193684+0000","last_clean":"2024-04-24T06:38:40.193684+0000","last_became_active":"2024-04-24T06:38:38.190189+0000","last_became_peered":"2024-04-24T06:38:38.190189+0000","last_unstale":"2024-04-24T06:38:40.193684+0000","last_undegraded":"2024-04-24T06:38:40.193684+0000","last_fullsized":"2024-04-24T06:38:40.193684+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:37.387580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349028+0000","last_change":"2024-04-24T06:38:39.191116+0000","last_active":"2024-04-24T06:38:40.349028+0000","last_peered":"2024-04-24T06:38:40.349028+0000","last_clean":"2024-04-24T06:38:40.349028+0000","last_became_active":"2024-04-24T06:38:39.190170+0000","last_became_peered":"2024-04-24T06:38:39.190170+0000","last_unstale":"2024-04-24T06:38:40.349028+0000","last_undegraded":"2024-04-24T06:38:40.349028+0000","last_fullsized":"2024-04-24T06:38:40.349028+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:36.001714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"97'3","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350286+0000","last_change":"2024-04-24T06:38:38.201487+0000","last_active":"2024-04-24T06:38:40.350286+0000","last_peered":"2024-04-24T06:38:40.350286+0000","last_clean":"2024-04-24T06:38:40.350286+0000","last_became_active":"2024-04-24T06:38:38.201000+0000","last_became_peered":"2024-04-24T06:38:38.201000+0000","last_unstale":"2024-04-24T06:38:40.350286+0000","last_undegraded":"2024-04-24T06:38:40.350286+0000","last_fullsized":"2024-04-24T06:38:40.350286+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349985+0000","last_change":"2024-04-24T06:38:39.203026+0000","last_active":"2024-04-24T06:38:40.349985+0000","last_peered":"2024-04-24T06:38:40.349985+0000","last_clean":"2024-04-24T06:38:40.349985+0000","last_became_active":"2024-04-24T06:38:39.198765+0000","last_became_peered":"2024-04-24T06:38:39.198765+0000","last_unstale":"2024-04-24T06:38:40.349985+0000","last_undegraded":"2024-04-24T06:38:40.349985+0000","last_fullsized":"2024-04-24T06:38:40.349985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"97'6","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197750+0000","last_change":"2024-04-24T06:38:38.190980+0000","last_active":"2024-04-24T06:38:40.197750+0000","last_peered":"2024-04-24T06:38:40.197750+0000","last_clean":"2024-04-24T06:38:40.197750+0000","last_became_active":"2024-04-24T06:38:38.190686+0000","last_became_peered":"2024-04-24T06:38:38.190686+0000","last_unstale":"2024-04-24T06:38:40.197750+0000","last_undegraded":"2024-04-24T06:38:40.197750+0000","last_fullsized":"2024-04-24T06:38:40.197750+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:49.059567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192882+0000","last_change":"2024-04-24T06:38:39.208692+0000","last_active":"2024-04-24T06:38:40.192882+0000","last_peered":"2024-04-24T06:38:40.192882+0000","last_clean":"2024-04-24T06:38:40.192882+0000","last_became_active":"2024-04-24T06:38:39.208429+0000","last_became_peered":"2024-04-24T06:38:39.208429+0000","last_unstale":"2024-04-24T06:38:40.192882+0000","last_undegraded":"2024-04-24T06:38:40.192882+0000","last_fullsized":"2024-04-24T06:38:40.192882+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345133+0000","last_change":"2024-04-24T06:38:38.190669+0000","last_active":"2024-04-24T06:38:40.345133+0000","last_peered":"2024-04-24T06:38:40.345133+0000","last_clean":"2024-04-24T06:38:40.345133+0000","last_became_active":"2024-04-24T06:38:38.189921+0000","last_became_peered":"2024-04-24T06:38:38.189921+0000","last_unstale":"2024-04-24T06:38:40.345133+0000","last_undegraded":"2024-04-24T06:38:40.345133+0000","last_fullsized":"2024-04-24T06:38:40.345133+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:39:35.842533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247221+0000","last_change":"2024-04-24T06:38:39.191212+0000","last_active":"2024-04-24T06:38:40.247221+0000","last_peered":"2024-04-24T06:38:40.247221+0000","last_clean":"2024-04-24T06:38:40.247221+0000","last_became_active":"2024-04-24T06:38:39.190920+0000","last_became_peered":"2024-04-24T06:38:39.190920+0000","last_unstale":"2024-04-24T06:38:40.247221+0000","last_undegraded":"2024-04-24T06:38:40.247221+0000","last_fullsized":"2024-04-24T06:38:40.247221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:21.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343947+0000","last_change":"2024-04-24T06:38:38.187644+0000","last_active":"2024-04-24T06:38:40.343947+0000","last_peered":"2024-04-24T06:38:40.343947+0000","last_clean":"2024-04-24T06:38:40.343947+0000","last_became_active":"2024-04-24T06:38:38.187365+0000","last_became_peered":"2024-04-24T06:38:38.187365+0000","last_unstale":"2024-04-24T06:38:40.343947+0000","last_undegraded":"2024-04-24T06:38:40.343947+0000","last_fullsized":"2024-04-24T06:38:40.343947+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:29.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196278+0000","last_change":"2024-04-24T06:38:39.197715+0000","last_active":"2024-04-24T06:38:40.196278+0000","last_peered":"2024-04-24T06:38:40.196278+0000","last_clean":"2024-04-24T06:38:40.196278+0000","last_became_active":"2024-04-24T06:38:39.197426+0000","last_became_peered":"2024-04-24T06:38:39.197426+0000","last_unstale":"2024-04-24T06:38:40.196278+0000","last_undegraded":"2024-04-24T06:38:40.196278+0000","last_fullsized":"2024-04-24T06:38:40.196278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:59.085923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196464+0000","last_change":"2024-04-24T06:38:38.197445+0000","last_active":"2024-04-24T06:38:40.196464+0000","last_peered":"2024-04-24T06:38:40.196464+0000","last_clean":"2024-04-24T06:38:40.196464+0000","last_became_active":"2024-04-24T06:38:38.197089+0000","last_became_peered":"2024-04-24T06:38:38.197089+0000","last_unstale":"2024-04-24T06:38:40.196464+0000","last_undegraded":"2024-04-24T06:38:40.196464+0000","last_fullsized":"2024-04-24T06:38:40.196464+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:33.508081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343339+0000","last_change":"2024-04-24T06:38:39.206909+0000","last_active":"2024-04-24T06:38:40.343339+0000","last_peered":"2024-04-24T06:38:40.343339+0000","last_clean":"2024-04-24T06:38:40.343339+0000","last_became_active":"2024-04-24T06:38:39.206602+0000","last_became_peered":"2024-04-24T06:38:39.206602+0000","last_unstale":"2024-04-24T06:38:40.343339+0000","last_undegraded":"2024-04-24T06:38:40.343339+0000","last_fullsized":"2024-04-24T06:38:40.343339+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:55.179321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"97'8","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247640+0000","last_change":"2024-04-24T06:38:38.195023+0000","last_active":"2024-04-24T06:38:40.247640+0000","last_peered":"2024-04-24T06:38:40.247640+0000","last_clean":"2024-04-24T06:38:40.247640+0000","last_became_active":"2024-04-24T06:38:38.194727+0000","last_became_peered":"2024-04-24T06:38:38.194727+0000","last_unstale":"2024-04-24T06:38:40.247640+0000","last_undegraded":"2024-04-24T06:38:40.247640+0000","last_fullsized":"2024-04-24T06:38:40.247640+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:47.203530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349294+0000","last_change":"2024-04-24T06:38:39.189634+0000","last_active":"2024-04-24T06:38:40.349294+0000","last_peered":"2024-04-24T06:38:40.349294+0000","last_clean":"2024-04-24T06:38:40.349294+0000","last_became_active":"2024-04-24T06:38:39.189276+0000","last_became_peered":"2024-04-24T06:38:39.189276+0000","last_unstale":"2024-04-24T06:38:40.349294+0000","last_undegraded":"2024-04-24T06:38:40.349294+0000","last_fullsized":"2024-04-24T06:38:40.349294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197754+0000","last_change":"2024-04-24T06:38:38.192153+0000","last_active":"2024-04-24T06:38:40.197754+0000","last_peered":"2024-04-24T06:38:40.197754+0000","last_clean":"2024-04-24T06:38:40.197754+0000","last_became_active":"2024-04-24T06:38:38.191860+0000","last_became_peered":"2024-04-24T06:38:38.191860+0000","last_unstale":"2024-04-24T06:38:40.197754+0000","last_undegraded":"2024-04-24T06:38:40.197754+0000","last_fullsized":"2024-04-24T06:38:40.197754+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347294+0000","last_change":"2024-04-24T06:38:39.190622+0000","last_active":"2024-04-24T06:38:40.347294+0000","last_peered":"2024-04-24T06:38:40.347294+0000","last_clean":"2024-04-24T06:38:40.347294+0000","last_became_active":"2024-04-24T06:38:39.189278+0000","last_became_peered":"2024-04-24T06:38:39.189278+0000","last_unstale":"2024-04-24T06:38:40.347294+0000","last_undegraded":"2024-04-24T06:38:40.347294+0000","last_fullsized":"2024-04-24T06:38:40.347294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:50.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348369+0000","last_change":"2024-04-24T06:38:38.201583+0000","last_active":"2024-04-24T06:38:40.348369+0000","last_peered":"2024-04-24T06:38:40.348369+0000","last_clean":"2024-04-24T06:38:40.348369+0000","last_became_active":"2024-04-24T06:38:38.199954+0000","last_became_peered":"2024-04-24T06:38:38.199954+0000","last_unstale":"2024-04-24T06:38:40.348369+0000","last_undegraded":"2024-04-24T06:38:40.348369+0000","last_fullsized":"2024-04-24T06:38:40.348369+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350010+0000","last_change":"2024-04-24T06:38:39.198039+0000","last_active":"2024-04-24T06:38:40.350010+0000","last_peered":"2024-04-24T06:38:40.350010+0000","last_clean":"2024-04-24T06:38:40.350010+0000","last_became_active":"2024-04-24T06:38:39.196519+0000","last_became_peered":"2024-04-24T06:38:39.196519+0000","last_unstale":"2024-04-24T06:38:40.350010+0000","last_undegraded":"2024-04-24T06:38:40.350010+0000","last_fullsized":"2024-04-24T06:38:40.350010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344689+0000","last_change":"2024-04-24T06:38:38.193288+0000","last_active":"2024-04-24T06:38:40.344689+0000","last_peered":"2024-04-24T06:38:40.344689+0000","last_clean":"2024-04-24T06:38:40.344689+0000","last_became_active":"2024-04-24T06:38:38.192927+0000","last_became_peered":"2024-04-24T06:38:38.192927+0000","last_unstale":"2024-04-24T06:38:40.344689+0000","last_undegraded":"2024-04-24T06:38:40.344689+0000","last_fullsized":"2024-04-24T06:38:40.344689+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343849+0000","last_change":"2024-04-24T06:38:39.207942+0000","last_active":"2024-04-24T06:38:40.343849+0000","last_peered":"2024-04-24T06:38:40.343849+0000","last_clean":"2024-04-24T06:38:40.343849+0000","last_became_active":"2024-04-24T06:38:39.207684+0000","last_became_peered":"2024-04-24T06:38:39.207684+0000","last_unstale":"2024-04-24T06:38:40.343849+0000","last_undegraded":"2024-04-24T06:38:40.343849+0000","last_fullsized":"2024-04-24T06:38:40.343849+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:12.582458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344235+0000","last_change":"2024-04-24T06:38:38.199850+0000","last_active":"2024-04-24T06:38:40.344235+0000","last_peered":"2024-04-24T06:38:40.344235+0000","last_clean":"2024-04-24T06:38:40.344235+0000","last_became_active":"2024-04-24T06:38:38.199550+0000","last_became_peered":"2024-04-24T06:38:38.199550+0000","last_unstale":"2024-04-24T06:38:40.344235+0000","last_undegraded":"2024-04-24T06:38:40.344235+0000","last_fullsized":"2024-04-24T06:38:40.344235+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:52.606985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345057+0000","last_change":"2024-04-24T06:38:39.203367+0000","last_active":"2024-04-24T06:38:40.345057+0000","last_peered":"2024-04-24T06:38:40.345057+0000","last_clean":"2024-04-24T06:38:40.345057+0000","last_became_active":"2024-04-24T06:38:39.202890+0000","last_became_peered":"2024-04-24T06:38:39.202890+0000","last_unstale":"2024-04-24T06:38:40.345057+0000","last_undegraded":"2024-04-24T06:38:40.345057+0000","last_fullsized":"2024-04-24T06:38:40.345057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:35.113576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346450+0000","last_change":"2024-04-24T06:38:38.200141+0000","last_active":"2024-04-24T06:38:40.346450+0000","last_peered":"2024-04-24T06:38:40.346450+0000","last_clean":"2024-04-24T06:38:40.346450+0000","last_became_active":"2024-04-24T06:38:38.199811+0000","last_became_peered":"2024-04-24T06:38:38.199811+0000","last_unstale":"2024-04-24T06:38:40.346450+0000","last_undegraded":"2024-04-24T06:38:40.346450+0000","last_fullsized":"2024-04-24T06:38:40.346450+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:00.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343446+0000","last_change":"2024-04-24T06:38:39.193012+0000","last_active":"2024-04-24T06:38:40.343446+0000","last_peered":"2024-04-24T06:38:40.343446+0000","last_clean":"2024-04-24T06:38:40.343446+0000","last_became_active":"2024-04-24T06:38:39.192559+0000","last_became_peered":"2024-04-24T06:38:39.192559+0000","last_unstale":"2024-04-24T06:38:40.343446+0000","last_undegraded":"2024-04-24T06:38:40.343446+0000","last_fullsized":"2024-04-24T06:38:40.343446+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:20:51.055492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346087+0000","last_change":"2024-04-24T06:38:38.200095+0000","last_active":"2024-04-24T06:38:40.346087+0000","last_peered":"2024-04-24T06:38:40.346087+0000","last_clean":"2024-04-24T06:38:40.346087+0000","last_became_active":"2024-04-24T06:38:38.199766+0000","last_became_peered":"2024-04-24T06:38:38.199766+0000","last_unstale":"2024-04-24T06:38:40.346087+0000","last_undegraded":"2024-04-24T06:38:40.346087+0000","last_fullsized":"2024-04-24T06:38:40.346087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:35.221397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347442+0000","last_change":"2024-04-24T06:38:39.196328+0000","last_active":"2024-04-24T06:38:40.347442+0000","last_peered":"2024-04-24T06:38:40.347442+0000","last_clean":"2024-04-24T06:38:40.347442+0000","last_became_active":"2024-04-24T06:38:39.195995+0000","last_became_peered":"2024-04-24T06:38:39.195995+0000","last_unstale":"2024-04-24T06:38:40.347442+0000","last_undegraded":"2024-04-24T06:38:40.347442+0000","last_fullsized":"2024-04-24T06:38:40.347442+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:02.364072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343171+0000","last_change":"2024-04-24T06:38:38.197579+0000","last_active":"2024-04-24T06:38:40.343171+0000","last_peered":"2024-04-24T06:38:40.343171+0000","last_clean":"2024-04-24T06:38:40.343171+0000","last_became_active":"2024-04-24T06:38:38.197079+0000","last_became_peered":"2024-04-24T06:38:38.197079+0000","last_unstale":"2024-04-24T06:38:40.343171+0000","last_undegraded":"2024-04-24T06:38:40.343171+0000","last_fullsized":"2024-04-24T06:38:40.343171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346060+0000","last_change":"2024-04-24T06:38:39.195363+0000","last_active":"2024-04-24T06:38:40.346060+0000","last_peered":"2024-04-24T06:38:40.346060+0000","last_clean":"2024-04-24T06:38:40.346060+0000","last_became_active":"2024-04-24T06:38:39.195103+0000","last_became_peered":"2024-04-24T06:38:39.195103+0000","last_unstale":"2024-04-24T06:38:40.346060+0000","last_undegraded":"2024-04-24T06:38:40.346060+0000","last_fullsized":"2024-04-24T06:38:40.346060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:09.894221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348476+0000","last_change":"2024-04-24T06:38:38.195715+0000","last_active":"2024-04-24T06:38:40.348476+0000","last_peered":"2024-04-24T06:38:40.348476+0000","last_clean":"2024-04-24T06:38:40.348476+0000","last_became_active":"2024-04-24T06:38:38.195332+0000","last_became_peered":"2024-04-24T06:38:38.195332+0000","last_unstale":"2024-04-24T06:38:40.348476+0000","last_undegraded":"2024-04-24T06:38:40.348476+0000","last_fullsized":"2024-04-24T06:38:40.348476+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:20.596981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247821+0000","last_change":"2024-04-24T06:38:39.205097+0000","last_active":"2024-04-24T06:38:40.247821+0000","last_peered":"2024-04-24T06:38:40.247821+0000","last_clean":"2024-04-24T06:38:40.247821+0000","last_became_active":"2024-04-24T06:38:39.204775+0000","last_became_peered":"2024-04-24T06:38:39.204775+0000","last_unstale":"2024-04-24T06:38:40.247821+0000","last_undegraded":"2024-04-24T06:38:40.247821+0000","last_fullsized":"2024-04-24T06:38:40.247821+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:34.525473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346163+0000","last_change":"2024-04-24T06:38:38.200684+0000","last_active":"2024-04-24T06:38:40.346163+0000","last_peered":"2024-04-24T06:38:40.346163+0000","last_clean":"2024-04-24T06:38:40.346163+0000","last_became_active":"2024-04-24T06:38:38.200389+0000","last_became_peered":"2024-04-24T06:38:38.200389+0000","last_unstale":"2024-04-24T06:38:40.346163+0000","last_undegraded":"2024-04-24T06:38:40.346163+0000","last_fullsized":"2024-04-24T06:38:40.346163+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:30:41.285222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248344+0000","last_change":"2024-04-24T06:38:39.199725+0000","last_active":"2024-04-24T06:38:40.248344+0000","last_peered":"2024-04-24T06:38:40.248344+0000","last_clean":"2024-04-24T06:38:40.248344+0000","last_became_active":"2024-04-24T06:38:39.196222+0000","last_became_peered":"2024-04-24T06:38:39.196222+0000","last_unstale":"2024-04-24T06:38:40.248344+0000","last_undegraded":"2024-04-24T06:38:40.248344+0000","last_fullsized":"2024-04-24T06:38:40.248344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:34.212630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345335+0000","last_change":"2024-04-24T06:38:38.188269+0000","last_active":"2024-04-24T06:38:40.345335+0000","last_peered":"2024-04-24T06:38:40.345335+0000","last_clean":"2024-04-24T06:38:40.345335+0000","last_became_active":"2024-04-24T06:38:38.187831+0000","last_became_peered":"2024-04-24T06:38:38.187831+0000","last_unstale":"2024-04-24T06:38:40.345335+0000","last_undegraded":"2024-04-24T06:38:40.345335+0000","last_fullsized":"2024-04-24T06:38:40.345335+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:57.876709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348478+0000","last_change":"2024-04-24T06:38:39.205151+0000","last_active":"2024-04-24T06:38:40.348478+0000","last_peered":"2024-04-24T06:38:40.348478+0000","last_clean":"2024-04-24T06:38:40.348478+0000","last_became_active":"2024-04-24T06:38:39.204845+0000","last_became_peered":"2024-04-24T06:38:39.204845+0000","last_unstale":"2024-04-24T06:38:40.348478+0000","last_undegraded":"2024-04-24T06:38:40.348478+0000","last_fullsized":"2024-04-24T06:38:40.348478+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348954+0000","last_change":"2024-04-24T06:38:38.199087+0000","last_active":"2024-04-24T06:38:40.348954+0000","last_peered":"2024-04-24T06:38:40.348954+0000","last_clean":"2024-04-24T06:38:40.348954+0000","last_became_active":"2024-04-24T06:38:38.197951+0000","last_became_peered":"2024-04-24T06:38:38.197951+0000","last_unstale":"2024-04-24T06:38:40.348954+0000","last_undegraded":"2024-04-24T06:38:40.348954+0000","last_fullsized":"2024-04-24T06:38:40.348954+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344895+0000","last_change":"2024-04-24T06:38:39.193206+0000","last_active":"2024-04-24T06:38:40.344895+0000","last_peered":"2024-04-24T06:38:40.344895+0000","last_clean":"2024-04-24T06:38:40.344895+0000","last_became_active":"2024-04-24T06:38:39.188996+0000","last_became_peered":"2024-04-24T06:38:39.188996+0000","last_unstale":"2024-04-24T06:38:40.344895+0000","last_undegraded":"2024-04-24T06:38:40.344895+0000","last_fullsized":"2024-04-24T06:38:40.344895+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:26.822649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344411+0000","last_change":"2024-04-24T06:38:38.189257+0000","last_active":"2024-04-24T06:38:40.344411+0000","last_peered":"2024-04-24T06:38:40.344411+0000","last_clean":"2024-04-24T06:38:40.344411+0000","last_became_active":"2024-04-24T06:38:38.188947+0000","last_became_peered":"2024-04-24T06:38:38.188947+0000","last_unstale":"2024-04-24T06:38:40.344411+0000","last_undegraded":"2024-04-24T06:38:40.344411+0000","last_fullsized":"2024-04-24T06:38:40.344411+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:15.213157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196852+0000","last_change":"2024-04-24T06:38:39.186708+0000","last_active":"2024-04-24T06:38:40.196852+0000","last_peered":"2024-04-24T06:38:40.196852+0000","last_clean":"2024-04-24T06:38:40.196852+0000","last_became_active":"2024-04-24T06:38:39.186405+0000","last_became_peered":"2024-04-24T06:38:39.186405+0000","last_unstale":"2024-04-24T06:38:40.196852+0000","last_undegraded":"2024-04-24T06:38:40.196852+0000","last_fullsized":"2024-04-24T06:38:40.196852+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:38.404973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347138+0000","last_change":"2024-04-24T06:38:38.199875+0000","last_active":"2024-04-24T06:38:40.347138+0000","last_peered":"2024-04-24T06:38:40.347138+0000","last_clean":"2024-04-24T06:38:40.347138+0000","last_became_active":"2024-04-24T06:38:38.199365+0000","last_became_peered":"2024-04-24T06:38:38.199365+0000","last_unstale":"2024-04-24T06:38:40.347138+0000","last_undegraded":"2024-04-24T06:38:40.347138+0000","last_fullsized":"2024-04-24T06:38:40.347138+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:49.059567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343879+0000","last_change":"2024-04-24T06:38:39.204035+0000","last_active":"2024-04-24T06:38:40.343879+0000","last_peered":"2024-04-24T06:38:40.343879+0000","last_clean":"2024-04-24T06:38:40.343879+0000","last_became_active":"2024-04-24T06:38:39.203749+0000","last_became_peered":"2024-04-24T06:38:39.203749+0000","last_unstale":"2024-04-24T06:38:40.343879+0000","last_undegraded":"2024-04-24T06:38:40.343879+0000","last_fullsized":"2024-04-24T06:38:40.343879+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:32:34.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193041+0000","last_change":"2024-04-24T06:38:38.186818+0000","last_active":"2024-04-24T06:38:40.193041+0000","last_peered":"2024-04-24T06:38:40.193041+0000","last_clean":"2024-04-24T06:38:40.193041+0000","last_became_active":"2024-04-24T06:38:38.186521+0000","last_became_peered":"2024-04-24T06:38:38.186521+0000","last_unstale":"2024-04-24T06:38:40.193041+0000","last_undegraded":"2024-04-24T06:38:40.193041+0000","last_fullsized":"2024-04-24T06:38:40.193041+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:58.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347890+0000","last_change":"2024-04-24T06:38:39.204573+0000","last_active":"2024-04-24T06:38:40.347890+0000","last_peered":"2024-04-24T06:38:40.347890+0000","last_clean":"2024-04-24T06:38:40.347890+0000","last_became_active":"2024-04-24T06:38:39.204193+0000","last_became_peered":"2024-04-24T06:38:39.204193+0000","last_unstale":"2024-04-24T06:38:40.347890+0000","last_undegraded":"2024-04-24T06:38:40.347890+0000","last_fullsized":"2024-04-24T06:38:40.347890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344146+0000","last_change":"2024-04-24T06:38:38.194934+0000","last_active":"2024-04-24T06:38:40.344146+0000","last_peered":"2024-04-24T06:38:40.344146+0000","last_clean":"2024-04-24T06:38:40.344146+0000","last_became_active":"2024-04-24T06:38:38.194457+0000","last_became_peered":"2024-04-24T06:38:38.194457+0000","last_unstale":"2024-04-24T06:38:40.344146+0000","last_undegraded":"2024-04-24T06:38:40.344146+0000","last_fullsized":"2024-04-24T06:38:40.344146+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:43.925402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344535+0000","last_change":"2024-04-24T06:38:39.210529+0000","last_active":"2024-04-24T06:38:40.344535+0000","last_peered":"2024-04-24T06:38:40.344535+0000","last_clean":"2024-04-24T06:38:40.344535+0000","last_became_active":"2024-04-24T06:38:39.209451+0000","last_became_peered":"2024-04-24T06:38:39.209451+0000","last_unstale":"2024-04-24T06:38:40.344535+0000","last_undegraded":"2024-04-24T06:38:40.344535+0000","last_fullsized":"2024-04-24T06:38:40.344535+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:20:15.554365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350359+0000","last_change":"2024-04-24T06:38:38.200307+0000","last_active":"2024-04-24T06:38:40.350359+0000","last_peered":"2024-04-24T06:38:40.350359+0000","last_clean":"2024-04-24T06:38:40.350359+0000","last_became_active":"2024-04-24T06:38:38.199966+0000","last_became_peered":"2024-04-24T06:38:38.199966+0000","last_unstale":"2024-04-24T06:38:40.350359+0000","last_undegraded":"2024-04-24T06:38:40.350359+0000","last_fullsized":"2024-04-24T06:38:40.350359+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346627+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:38:40.346627+0000","last_peered":"2024-04-24T06:38:40.346627+0000","last_clean":"2024-04-24T06:38:40.346627+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:38:40.346627+0000","last_undegraded":"2024-04-24T06:38:40.346627+0000","last_fullsized":"2024-04-24T06:38:40.346627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1814528,"data_stored":1114635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":524807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":524225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":525891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":525789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":524189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":524155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1810432,"data_stored":1115105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:38:43.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:38:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.935+0000 7f969f200640 1 Processor -- start 2024-04-24T06:38:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.935+0000 7f969f200640 1 -- start start 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a00b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a00d0d90 0x7f96a00d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00cf030 con 0x7f96a015e7c0 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f96a00b7000 con 0x7f96a00d0d90 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f96a00cf620 con 0x7f96a00b5ff0 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a00d0d90 0x7f96a00d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a00d0d90 0x7f96a00d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44436/0 (socket says 172.21.15.59:44436) 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f969e800640 1 -- 172.21.15.59:0/1288548031 learned_addr learned my addr 172.21.15.59:0/1288548031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.937+0000 7f9697e00640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a00b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.938+0000 7f969e800640 1 -- 172.21.15.59:0/1288548031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 msgr2=0x7f96a00b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.938+0000 7f969e800640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a00b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.938+0000 7f969e800640 1 -- 172.21.15.59:0/1288548031 >> v1:172.21.15.66:6789/0 conn(0x7f96a015e7c0 legacy=0x7f96a00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.938+0000 7f969e800640 1 -- 172.21.15.59:0/1288548031 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a00b6a40 con 0x7f96a00d0d90 2024-04-24T06:38:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.939+0000 7f969e800640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00d0d90 0x7f96a00d1220 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9698009e50 tx=0x7f969802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273a993f535c34bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.940+0000 7f969d400640 1 -- 172.21.15.59:0/1288548031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698036070 con 0x7f96a00d0d90 2024-04-24T06:38:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.940+0000 7f969d400640 1 -- 172.21.15.59:0/1288548031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698031600 con 0x7f96a00d0d90 2024-04-24T06:38:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.940+0000 7f969d400640 1 -- 172.21.15.59:0/1288548031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980319d0 con 0x7f96a00d0d90 2024-04-24T06:38:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.940+0000 7f969f200640 1 -- 172.21.15.59:0/1288548031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00d0d90 msgr2=0x7f96a00d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.940+0000 7f969f200640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00d0d90 0x7f96a00d1220 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9698009e50 tx=0x7f969802a710 comp rx=0 tx=0).stop 2024-04-24T06:38:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 -- 172.21.15.59:0/1288548031 shutdown_connections 2024-04-24T06:38:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a00b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 --2- 172.21.15.59:0/1288548031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00d0d90 0x7f96a00d1220 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 -- 172.21.15.59:0/1288548031 >> 172.21.15.59:0/1288548031 conn(0x7f96a009d920 msgr2=0x7f96a008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 -- 172.21.15.59:0/1288548031 shutdown_connections 2024-04-24T06:38:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.941+0000 7f969f200640 1 -- 172.21.15.59:0/1288548031 wait complete. 2024-04-24T06:38:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.942+0000 7f969f200640 1 Processor -- start 2024-04-24T06:38:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.943+0000 7f969f200640 1 -- start start 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.943+0000 7f969f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a0159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.943+0000 7f969f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a00d0d90 0x7f96a0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.943+0000 7f969f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00d2800 con 0x7f96a015e7c0 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00cf2b0 con 0x7f96a00d0d90 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f96a00d1a80 con 0x7f96a00b5ff0 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44442/0 (socket says 172.21.15.59:44442) 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 -- 172.21.15.59:0/2770294420 learned_addr learned my addr 172.21.15.59:0/2770294420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969de00640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a0159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f9697e00640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a00d0d90 0x7f96a0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 -- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 msgr2=0x7f96a0159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a0159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 -- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a00d0d90 msgr2=0x7f96a0160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a00d0d90 0x7f96a0160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.944+0000 7f969e800640 1 -- 172.21.15.59:0/2770294420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a015f0e0 con 0x7f96a015e7c0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f969e800640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9698009e50 tx=0x7f9698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698041020 con 0x7f96a015e7c0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96980077a0 con 0x7f96a015e7c0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698036040 con 0x7f96a015e7c0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a015f360 con 0x7f96a015e7c0 2024-04-24T06:38:43.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.945+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a015f920 con 0x7f96a015e7c0 2024-04-24T06:38:43.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.946+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a004fd90 con 0x7f96a015e7c0 2024-04-24T06:38:43.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.947+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9698044020 con 0x7f96a015e7c0 2024-04-24T06:38:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.948+0000 7f9696000640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 0x7f967c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.948+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f96980ac850 con 0x7f96a015e7c0 2024-04-24T06:38:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.949+0000 7f969de00640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 0x7f967c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:43.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.949+0000 7f969de00640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 0x7f967c06ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9690007fd0 tx=0x7f9690009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:43.953+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f969804b030 con 0x7f96a015e7c0 2024-04-24T06:38:44.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.151+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96a00d1eb0 con 0x7f967c06c920 2024-04-24T06:38:44.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.170+0000 7f9696000640 1 -- 172.21.15.59:0/2770294420 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7f96a00d1eb0 con 0x7f967c06c920 2024-04-24T06:38:44.176 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:44.178 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:38:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 msgr2=0x7f967c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 0x7f967c06ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9690007fd0 tx=0x7f9690009040 comp rx=0 tx=0).stop 2024-04-24T06:38:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 msgr2=0x7f96a015eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9698009e50 tx=0x7f9698004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 shutdown_connections 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f967c06c920 0x7f967c06ede0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a015e7c0 0x7f96a015eba0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a00d0d90 0x7f96a0160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 --2- 172.21.15.59:0/2770294420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f96a00b5ff0 0x7f96a0159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.177+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 >> 172.21.15.59:0/2770294420 conn(0x7f96a009d920 msgr2=0x7f96a006cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.178+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 shutdown_connections 2024-04-24T06:38:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.178+0000 7f969f200640 1 -- 172.21.15.59:0/2770294420 wait complete. 2024-04-24T06:38:44.200 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-24T06:38:43.398579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260184,"kb_used_data":10696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714536,"statfs":{"total":773094113280,"available":772827684864,"internally_reserved":0,"allocated":10952704,"data_stored":5378796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001366"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248098+0000","last_change":"2024-04-24T06:38:39.198572+0000","last_active":"2024-04-24T06:38:40.248098+0000","last_peered":"2024-04-24T06:38:40.248098+0000","last_clean":"2024-04-24T06:38:40.248098+0000","last_became_active":"2024-04-24T06:38:39.195884+0000","last_became_peered":"2024-04-24T06:38:39.195884+0000","last_unstale":"2024-04-24T06:38:40.248098+0000","last_undegraded":"2024-04-24T06:38:40.248098+0000","last_fullsized":"2024-04-24T06:38:40.248098+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:57.136731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345613+0000","last_change":"2024-04-24T06:38:39.199816+0000","last_active":"2024-04-24T06:38:40.345613+0000","last_peered":"2024-04-24T06:38:40.345613+0000","last_clean":"2024-04-24T06:38:40.345613+0000","last_became_active":"2024-04-24T06:38:39.199538+0000","last_became_peered":"2024-04-24T06:38:39.199538+0000","last_unstale":"2024-04-24T06:38:40.345613+0000","last_undegraded":"2024-04-24T06:38:40.345613+0000","last_fullsized":"2024-04-24T06:38:40.345613+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:01.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247699+0000","last_change":"2024-04-24T06:38:39.195344+0000","last_active":"2024-04-24T06:38:40.247699+0000","last_peered":"2024-04-24T06:38:40.247699+0000","last_clean":"2024-04-24T06:38:40.247699+0000","last_became_active":"2024-04-24T06:38:39.195073+0000","last_became_peered":"2024-04-24T06:38:39.195073+0000","last_unstale":"2024-04-24T06:38:40.247699+0000","last_undegraded":"2024-04-24T06:38:40.247699+0000","last_fullsized":"2024-04-24T06:38:40.247699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:18.706878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193223+0000","last_change":"2024-04-24T06:38:39.207297+0000","last_active":"2024-04-24T06:38:40.193223+0000","last_peered":"2024-04-24T06:38:40.193223+0000","last_clean":"2024-04-24T06:38:40.193223+0000","last_became_active":"2024-04-24T06:38:39.206852+0000","last_became_peered":"2024-04-24T06:38:39.206852+0000","last_unstale":"2024-04-24T06:38:40.193223+0000","last_undegraded":"2024-04-24T06:38:40.193223+0000","last_fullsized":"2024-04-24T06:38:40.193223+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:16:32.299278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347890+0000","last_change":"2024-04-24T06:38:39.206116+0000","last_active":"2024-04-24T06:38:40.347890+0000","last_peered":"2024-04-24T06:38:40.347890+0000","last_clean":"2024-04-24T06:38:40.347890+0000","last_became_active":"2024-04-24T06:38:39.205430+0000","last_became_peered":"2024-04-24T06:38:39.205430+0000","last_unstale":"2024-04-24T06:38:40.347890+0000","last_undegraded":"2024-04-24T06:38:40.347890+0000","last_fullsized":"2024-04-24T06:38:40.347890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349677+0000","last_change":"2024-04-24T06:38:39.203926+0000","last_active":"2024-04-24T06:38:40.349677+0000","last_peered":"2024-04-24T06:38:40.349677+0000","last_clean":"2024-04-24T06:38:40.349677+0000","last_became_active":"2024-04-24T06:38:39.203577+0000","last_became_peered":"2024-04-24T06:38:39.203577+0000","last_unstale":"2024-04-24T06:38:40.349677+0000","last_undegraded":"2024-04-24T06:38:40.349677+0000","last_fullsized":"2024-04-24T06:38:40.349677+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346089+0000","last_change":"2024-04-24T06:38:39.203297+0000","last_active":"2024-04-24T06:38:40.346089+0000","last_peered":"2024-04-24T06:38:40.346089+0000","last_clean":"2024-04-24T06:38:40.346089+0000","last_became_active":"2024-04-24T06:38:39.202949+0000","last_became_peered":"2024-04-24T06:38:39.202949+0000","last_unstale":"2024-04-24T06:38:40.346089+0000","last_undegraded":"2024-04-24T06:38:40.346089+0000","last_fullsized":"2024-04-24T06:38:40.346089+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:36.001714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343208+0000","last_change":"2024-04-24T06:38:39.192584+0000","last_active":"2024-04-24T06:38:40.343208+0000","last_peered":"2024-04-24T06:38:40.343208+0000","last_clean":"2024-04-24T06:38:40.343208+0000","last_became_active":"2024-04-24T06:38:39.191588+0000","last_became_peered":"2024-04-24T06:38:39.191588+0000","last_unstale":"2024-04-24T06:38:40.343208+0000","last_undegraded":"2024-04-24T06:38:40.343208+0000","last_fullsized":"2024-04-24T06:38:40.343208+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197699+0000","last_change":"2024-04-24T06:38:39.203403+0000","last_active":"2024-04-24T06:38:40.197699+0000","last_peered":"2024-04-24T06:38:40.197699+0000","last_clean":"2024-04-24T06:38:40.197699+0000","last_became_active":"2024-04-24T06:38:39.203123+0000","last_became_peered":"2024-04-24T06:38:39.203123+0000","last_unstale":"2024-04-24T06:38:40.197699+0000","last_undegraded":"2024-04-24T06:38:40.197699+0000","last_fullsized":"2024-04-24T06:38:40.197699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348984+0000","last_change":"2024-04-24T06:38:39.195955+0000","last_active":"2024-04-24T06:38:40.348984+0000","last_peered":"2024-04-24T06:38:40.348984+0000","last_clean":"2024-04-24T06:38:40.348984+0000","last_became_active":"2024-04-24T06:38:39.195719+0000","last_became_peered":"2024-04-24T06:38:39.195719+0000","last_unstale":"2024-04-24T06:38:40.348984+0000","last_undegraded":"2024-04-24T06:38:40.348984+0000","last_fullsized":"2024-04-24T06:38:40.348984+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346307+0000","last_change":"2024-04-24T06:38:39.198347+0000","last_active":"2024-04-24T06:38:40.346307+0000","last_peered":"2024-04-24T06:38:40.346307+0000","last_clean":"2024-04-24T06:38:40.346307+0000","last_became_active":"2024-04-24T06:38:39.197747+0000","last_became_peered":"2024-04-24T06:38:39.197747+0000","last_unstale":"2024-04-24T06:38:40.346307+0000","last_undegraded":"2024-04-24T06:38:40.346307+0000","last_fullsized":"2024-04-24T06:38:40.346307+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197455+0000","last_change":"2024-04-24T06:38:39.188824+0000","last_active":"2024-04-24T06:38:40.197455+0000","last_peered":"2024-04-24T06:38:40.197455+0000","last_clean":"2024-04-24T06:38:40.197455+0000","last_became_active":"2024-04-24T06:38:39.188507+0000","last_became_peered":"2024-04-24T06:38:39.188507+0000","last_unstale":"2024-04-24T06:38:40.197455+0000","last_undegraded":"2024-04-24T06:38:40.197455+0000","last_fullsized":"2024-04-24T06:38:40.197455+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:02.364072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193234+0000","last_change":"2024-04-24T06:38:39.211246+0000","last_active":"2024-04-24T06:38:40.193234+0000","last_peered":"2024-04-24T06:38:40.193234+0000","last_clean":"2024-04-24T06:38:40.193234+0000","last_became_active":"2024-04-24T06:38:39.210923+0000","last_became_peered":"2024-04-24T06:38:39.210923+0000","last_unstale":"2024-04-24T06:38:40.193234+0000","last_undegraded":"2024-04-24T06:38:40.193234+0000","last_fullsized":"2024-04-24T06:38:40.193234+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:01.176496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343542+0000","last_change":"2024-04-24T06:38:39.191027+0000","last_active":"2024-04-24T06:38:40.343542+0000","last_peered":"2024-04-24T06:38:40.343542+0000","last_clean":"2024-04-24T06:38:40.343542+0000","last_became_active":"2024-04-24T06:38:39.189396+0000","last_became_peered":"2024-04-24T06:38:39.189396+0000","last_unstale":"2024-04-24T06:38:40.343542+0000","last_undegraded":"2024-04-24T06:38:40.343542+0000","last_fullsized":"2024-04-24T06:38:40.343542+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:45.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192771+0000","last_change":"2024-04-24T06:38:39.211694+0000","last_active":"2024-04-24T06:38:40.192771+0000","last_peered":"2024-04-24T06:38:40.192771+0000","last_clean":"2024-04-24T06:38:40.192771+0000","last_became_active":"2024-04-24T06:38:39.211408+0000","last_became_peered":"2024-04-24T06:38:39.211408+0000","last_unstale":"2024-04-24T06:38:40.192771+0000","last_undegraded":"2024-04-24T06:38:40.192771+0000","last_fullsized":"2024-04-24T06:38:40.192771+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346664+0000","last_change":"2024-04-24T06:38:39.193528+0000","last_active":"2024-04-24T06:38:40.346664+0000","last_peered":"2024-04-24T06:38:40.346664+0000","last_clean":"2024-04-24T06:38:40.346664+0000","last_became_active":"2024-04-24T06:38:39.193166+0000","last_became_peered":"2024-04-24T06:38:39.193166+0000","last_unstale":"2024-04-24T06:38:40.346664+0000","last_undegraded":"2024-04-24T06:38:40.346664+0000","last_fullsized":"2024-04-24T06:38:40.346664+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345867+0000","last_change":"2024-04-24T06:38:39.202517+0000","last_active":"2024-04-24T06:38:40.345867+0000","last_peered":"2024-04-24T06:38:40.345867+0000","last_clean":"2024-04-24T06:38:40.345867+0000","last_became_active":"2024-04-24T06:38:39.202228+0000","last_became_peered":"2024-04-24T06:38:39.202228+0000","last_unstale":"2024-04-24T06:38:40.345867+0000","last_undegraded":"2024-04-24T06:38:40.345867+0000","last_fullsized":"2024-04-24T06:38:40.345867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:55.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193674+0000","last_change":"2024-04-24T06:38:39.207496+0000","last_active":"2024-04-24T06:38:40.193674+0000","last_peered":"2024-04-24T06:38:40.193674+0000","last_clean":"2024-04-24T06:38:40.193674+0000","last_became_active":"2024-04-24T06:38:39.207147+0000","last_became_peered":"2024-04-24T06:38:39.207147+0000","last_unstale":"2024-04-24T06:38:40.193674+0000","last_undegraded":"2024-04-24T06:38:40.193674+0000","last_fullsized":"2024-04-24T06:38:40.193674+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:05.276766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343725+0000","last_change":"2024-04-24T06:38:39.192262+0000","last_active":"2024-04-24T06:38:40.343725+0000","last_peered":"2024-04-24T06:38:40.343725+0000","last_clean":"2024-04-24T06:38:40.343725+0000","last_became_active":"2024-04-24T06:38:39.191884+0000","last_became_peered":"2024-04-24T06:38:39.191884+0000","last_unstale":"2024-04-24T06:38:40.343725+0000","last_undegraded":"2024-04-24T06:38:40.343725+0000","last_fullsized":"2024-04-24T06:38:40.343725+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:37.831589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197787+0000","last_change":"2024-04-24T06:38:39.208616+0000","last_active":"2024-04-24T06:38:40.197787+0000","last_peered":"2024-04-24T06:38:40.197787+0000","last_clean":"2024-04-24T06:38:40.197787+0000","last_became_active":"2024-04-24T06:38:39.206997+0000","last_became_peered":"2024-04-24T06:38:39.206997+0000","last_unstale":"2024-04-24T06:38:40.197787+0000","last_undegraded":"2024-04-24T06:38:40.197787+0000","last_fullsized":"2024-04-24T06:38:40.197787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344973+0000","last_change":"2024-04-24T06:38:39.197394+0000","last_active":"2024-04-24T06:38:40.344973+0000","last_peered":"2024-04-24T06:38:40.344973+0000","last_clean":"2024-04-24T06:38:40.344973+0000","last_became_active":"2024-04-24T06:38:39.197125+0000","last_became_peered":"2024-04-24T06:38:39.197125+0000","last_unstale":"2024-04-24T06:38:40.344973+0000","last_undegraded":"2024-04-24T06:38:40.344973+0000","last_fullsized":"2024-04-24T06:38:40.344973+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:20:44.787089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343665+0000","last_change":"2024-04-24T06:38:39.194001+0000","last_active":"2024-04-24T06:38:40.343665+0000","last_peered":"2024-04-24T06:38:40.343665+0000","last_clean":"2024-04-24T06:38:40.343665+0000","last_became_active":"2024-04-24T06:38:39.193724+0000","last_became_peered":"2024-04-24T06:38:39.193724+0000","last_unstale":"2024-04-24T06:38:40.343665+0000","last_undegraded":"2024-04-24T06:38:40.343665+0000","last_fullsized":"2024-04-24T06:38:40.343665+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343257+0000","last_change":"2024-04-24T06:38:39.194212+0000","last_active":"2024-04-24T06:38:40.343257+0000","last_peered":"2024-04-24T06:38:40.343257+0000","last_clean":"2024-04-24T06:38:40.343257+0000","last_became_active":"2024-04-24T06:38:39.192153+0000","last_became_peered":"2024-04-24T06:38:39.192153+0000","last_unstale":"2024-04-24T06:38:40.343257+0000","last_undegraded":"2024-04-24T06:38:40.343257+0000","last_fullsized":"2024-04-24T06:38:40.343257+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346103+0000","last_change":"2024-04-24T06:38:39.194167+0000","last_active":"2024-04-24T06:38:40.346103+0000","last_peered":"2024-04-24T06:38:40.346103+0000","last_clean":"2024-04-24T06:38:40.346103+0000","last_became_active":"2024-04-24T06:38:39.193438+0000","last_became_peered":"2024-04-24T06:38:39.193438+0000","last_unstale":"2024-04-24T06:38:40.346103+0000","last_undegraded":"2024-04-24T06:38:40.346103+0000","last_fullsized":"2024-04-24T06:38:40.346103+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:14:52.652212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349062+0000","last_change":"2024-04-24T06:38:39.187275+0000","last_active":"2024-04-24T06:38:40.349062+0000","last_peered":"2024-04-24T06:38:40.349062+0000","last_clean":"2024-04-24T06:38:40.349062+0000","last_became_active":"2024-04-24T06:38:39.186183+0000","last_became_peered":"2024-04-24T06:38:39.186183+0000","last_unstale":"2024-04-24T06:38:40.349062+0000","last_undegraded":"2024-04-24T06:38:40.349062+0000","last_fullsized":"2024-04-24T06:38:40.349062+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345708+0000","last_change":"2024-04-24T06:38:39.193254+0000","last_active":"2024-04-24T06:38:40.345708+0000","last_peered":"2024-04-24T06:38:40.345708+0000","last_clean":"2024-04-24T06:38:40.345708+0000","last_became_active":"2024-04-24T06:38:39.192235+0000","last_became_peered":"2024-04-24T06:38:39.192235+0000","last_unstale":"2024-04-24T06:38:40.345708+0000","last_undegraded":"2024-04-24T06:38:40.345708+0000","last_fullsized":"2024-04-24T06:38:40.345708+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:11:05.276766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346818+0000","last_change":"2024-04-24T06:38:39.190838+0000","last_active":"2024-04-24T06:38:40.346818+0000","last_peered":"2024-04-24T06:38:40.346818+0000","last_clean":"2024-04-24T06:38:40.346818+0000","last_became_active":"2024-04-24T06:38:39.190537+0000","last_became_peered":"2024-04-24T06:38:39.190537+0000","last_unstale":"2024-04-24T06:38:40.346818+0000","last_undegraded":"2024-04-24T06:38:40.346818+0000","last_fullsized":"2024-04-24T06:38:40.346818+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348678+0000","last_change":"2024-04-24T06:38:39.203029+0000","last_active":"2024-04-24T06:38:40.348678+0000","last_peered":"2024-04-24T06:38:40.348678+0000","last_clean":"2024-04-24T06:38:40.348678+0000","last_became_active":"2024-04-24T06:38:39.202726+0000","last_became_peered":"2024-04-24T06:38:39.202726+0000","last_unstale":"2024-04-24T06:38:40.348678+0000","last_undegraded":"2024-04-24T06:38:40.348678+0000","last_fullsized":"2024-04-24T06:38:40.348678+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344085+0000","last_change":"2024-04-24T06:38:39.196567+0000","last_active":"2024-04-24T06:38:40.344085+0000","last_peered":"2024-04-24T06:38:40.344085+0000","last_clean":"2024-04-24T06:38:40.344085+0000","last_became_active":"2024-04-24T06:38:39.189089+0000","last_became_peered":"2024-04-24T06:38:39.189089+0000","last_unstale":"2024-04-24T06:38:40.344085+0000","last_undegraded":"2024-04-24T06:38:40.344085+0000","last_fullsized":"2024-04-24T06:38:40.344085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:16:32.299278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343698+0000","last_change":"2024-04-24T06:38:39.202795+0000","last_active":"2024-04-24T06:38:40.343698+0000","last_peered":"2024-04-24T06:38:40.343698+0000","last_clean":"2024-04-24T06:38:40.343698+0000","last_became_active":"2024-04-24T06:38:39.196550+0000","last_became_peered":"2024-04-24T06:38:39.196550+0000","last_unstale":"2024-04-24T06:38:40.343698+0000","last_undegraded":"2024-04-24T06:38:40.343698+0000","last_fullsized":"2024-04-24T06:38:40.343698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:16.574285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193154+0000","last_change":"2024-04-24T06:38:39.210545+0000","last_active":"2024-04-24T06:38:40.193154+0000","last_peered":"2024-04-24T06:38:40.193154+0000","last_clean":"2024-04-24T06:38:40.193154+0000","last_became_active":"2024-04-24T06:38:39.210048+0000","last_became_peered":"2024-04-24T06:38:39.210048+0000","last_unstale":"2024-04-24T06:38:40.193154+0000","last_undegraded":"2024-04-24T06:38:40.193154+0000","last_fullsized":"2024-04-24T06:38:40.193154+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247266+0000","last_change":"2024-04-24T06:38:39.193897+0000","last_active":"2024-04-24T06:38:40.247266+0000","last_peered":"2024-04-24T06:38:40.247266+0000","last_clean":"2024-04-24T06:38:40.247266+0000","last_became_active":"2024-04-24T06:38:39.193625+0000","last_became_peered":"2024-04-24T06:38:39.193625+0000","last_unstale":"2024-04-24T06:38:40.247266+0000","last_undegraded":"2024-04-24T06:38:40.247266+0000","last_fullsized":"2024-04-24T06:38:40.247266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:42.761887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344269+0000","last_change":"2024-04-24T06:38:39.199835+0000","last_active":"2024-04-24T06:38:40.344269+0000","last_peered":"2024-04-24T06:38:40.344269+0000","last_clean":"2024-04-24T06:38:40.344269+0000","last_became_active":"2024-04-24T06:38:39.196133+0000","last_became_peered":"2024-04-24T06:38:39.196133+0000","last_unstale":"2024-04-24T06:38:40.344269+0000","last_undegraded":"2024-04-24T06:38:40.344269+0000","last_fullsized":"2024-04-24T06:38:40.344269+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:46:20.162332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"97'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345720+0000","last_change":"2024-04-24T06:38:38.198883+0000","last_active":"2024-04-24T06:38:40.345720+0000","last_peered":"2024-04-24T06:38:40.345720+0000","last_clean":"2024-04-24T06:38:40.345720+0000","last_became_active":"2024-04-24T06:38:38.198568+0000","last_became_peered":"2024-04-24T06:38:38.198568+0000","last_unstale":"2024-04-24T06:38:40.345720+0000","last_undegraded":"2024-04-24T06:38:40.345720+0000","last_fullsized":"2024-04-24T06:38:40.345720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:58.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193898+0000","last_change":"2024-04-24T06:38:39.210232+0000","last_active":"2024-04-24T06:38:40.193898+0000","last_peered":"2024-04-24T06:38:40.193898+0000","last_clean":"2024-04-24T06:38:40.193898+0000","last_became_active":"2024-04-24T06:38:39.209896+0000","last_became_peered":"2024-04-24T06:38:39.209896+0000","last_unstale":"2024-04-24T06:38:40.193898+0000","last_undegraded":"2024-04-24T06:38:40.193898+0000","last_fullsized":"2024-04-24T06:38:40.193898+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:37:55.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350088+0000","last_change":"2024-04-24T06:38:38.196394+0000","last_active":"2024-04-24T06:38:40.350088+0000","last_peered":"2024-04-24T06:38:40.350088+0000","last_clean":"2024-04-24T06:38:40.350088+0000","last_became_active":"2024-04-24T06:38:38.195341+0000","last_became_peered":"2024-04-24T06:38:38.195341+0000","last_unstale":"2024-04-24T06:38:40.350088+0000","last_undegraded":"2024-04-24T06:38:40.350088+0000","last_fullsized":"2024-04-24T06:38:40.350088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248025+0000","last_change":"2024-04-24T06:38:39.204023+0000","last_active":"2024-04-24T06:38:40.248025+0000","last_peered":"2024-04-24T06:38:40.248025+0000","last_clean":"2024-04-24T06:38:40.248025+0000","last_became_active":"2024-04-24T06:38:39.202732+0000","last_became_peered":"2024-04-24T06:38:39.202732+0000","last_unstale":"2024-04-24T06:38:40.248025+0000","last_undegraded":"2024-04-24T06:38:40.248025+0000","last_fullsized":"2024-04-24T06:38:40.248025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:50.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345427+0000","last_change":"2024-04-24T06:38:38.200782+0000","last_active":"2024-04-24T06:38:40.345427+0000","last_peered":"2024-04-24T06:38:40.345427+0000","last_clean":"2024-04-24T06:38:40.345427+0000","last_became_active":"2024-04-24T06:38:38.200307+0000","last_became_peered":"2024-04-24T06:38:38.200307+0000","last_unstale":"2024-04-24T06:38:40.345427+0000","last_undegraded":"2024-04-24T06:38:40.345427+0000","last_fullsized":"2024-04-24T06:38:40.345427+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:59:21.855834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.198173+0000","last_change":"2024-04-24T06:38:39.199890+0000","last_active":"2024-04-24T06:38:40.198173+0000","last_peered":"2024-04-24T06:38:40.198173+0000","last_clean":"2024-04-24T06:38:40.198173+0000","last_became_active":"2024-04-24T06:38:39.199271+0000","last_became_peered":"2024-04-24T06:38:39.199271+0000","last_unstale":"2024-04-24T06:38:40.198173+0000","last_undegraded":"2024-04-24T06:38:40.198173+0000","last_fullsized":"2024-04-24T06:38:40.198173+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:56:35.932802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344424+0000","last_change":"2024-04-24T06:38:38.194595+0000","last_active":"2024-04-24T06:38:40.344424+0000","last_peered":"2024-04-24T06:38:40.344424+0000","last_clean":"2024-04-24T06:38:40.344424+0000","last_became_active":"2024-04-24T06:38:38.192661+0000","last_became_peered":"2024-04-24T06:38:38.192661+0000","last_unstale":"2024-04-24T06:38:40.344424+0000","last_undegraded":"2024-04-24T06:38:40.344424+0000","last_fullsized":"2024-04-24T06:38:40.344424+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346578+0000","last_change":"2024-04-24T06:38:39.201176+0000","last_active":"2024-04-24T06:38:40.346578+0000","last_peered":"2024-04-24T06:38:40.346578+0000","last_clean":"2024-04-24T06:38:40.346578+0000","last_became_active":"2024-04-24T06:38:39.200887+0000","last_became_peered":"2024-04-24T06:38:39.200887+0000","last_unstale":"2024-04-24T06:38:40.346578+0000","last_undegraded":"2024-04-24T06:38:40.346578+0000","last_fullsized":"2024-04-24T06:38:40.346578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:34.525473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247516+0000","last_change":"2024-04-24T06:38:38.198972+0000","last_active":"2024-04-24T06:38:40.247516+0000","last_peered":"2024-04-24T06:38:40.247516+0000","last_clean":"2024-04-24T06:38:40.247516+0000","last_became_active":"2024-04-24T06:38:38.198249+0000","last_became_peered":"2024-04-24T06:38:38.198249+0000","last_unstale":"2024-04-24T06:38:40.247516+0000","last_undegraded":"2024-04-24T06:38:40.247516+0000","last_fullsized":"2024-04-24T06:38:40.247516+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:09:07.925689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349760+0000","last_change":"2024-04-24T06:38:39.185960+0000","last_active":"2024-04-24T06:38:40.349760+0000","last_peered":"2024-04-24T06:38:40.349760+0000","last_clean":"2024-04-24T06:38:40.349760+0000","last_became_active":"2024-04-24T06:38:39.185662+0000","last_became_peered":"2024-04-24T06:38:39.185662+0000","last_unstale":"2024-04-24T06:38:40.349760+0000","last_undegraded":"2024-04-24T06:38:40.349760+0000","last_fullsized":"2024-04-24T06:38:40.349760+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:41:31.768709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343670+0000","last_change":"2024-04-24T06:38:38.194361+0000","last_active":"2024-04-24T06:38:40.343670+0000","last_peered":"2024-04-24T06:38:40.343670+0000","last_clean":"2024-04-24T06:38:40.343670+0000","last_became_active":"2024-04-24T06:38:38.194069+0000","last_became_peered":"2024-04-24T06:38:38.194069+0000","last_unstale":"2024-04-24T06:38:40.343670+0000","last_undegraded":"2024-04-24T06:38:40.343670+0000","last_fullsized":"2024-04-24T06:38:40.343670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:00.907566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192629+0000","last_change":"2024-04-24T06:38:39.198515+0000","last_active":"2024-04-24T06:38:40.192629+0000","last_peered":"2024-04-24T06:38:40.192629+0000","last_clean":"2024-04-24T06:38:40.192629+0000","last_became_active":"2024-04-24T06:38:39.198226+0000","last_became_peered":"2024-04-24T06:38:39.198226+0000","last_unstale":"2024-04-24T06:38:40.192629+0000","last_undegraded":"2024-04-24T06:38:40.192629+0000","last_fullsized":"2024-04-24T06:38:40.192629+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:35:28.179738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348265+0000","last_change":"2024-04-24T06:38:38.196983+0000","last_active":"2024-04-24T06:38:40.348265+0000","last_peered":"2024-04-24T06:38:40.348265+0000","last_clean":"2024-04-24T06:38:40.348265+0000","last_became_active":"2024-04-24T06:38:38.196630+0000","last_became_peered":"2024-04-24T06:38:38.196630+0000","last_unstale":"2024-04-24T06:38:40.348265+0000","last_undegraded":"2024-04-24T06:38:40.348265+0000","last_fullsized":"2024-04-24T06:38:40.348265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197283+0000","last_change":"2024-04-24T06:38:39.199272+0000","last_active":"2024-04-24T06:38:40.197283+0000","last_peered":"2024-04-24T06:38:40.197283+0000","last_clean":"2024-04-24T06:38:40.197283+0000","last_became_active":"2024-04-24T06:38:39.196423+0000","last_became_peered":"2024-04-24T06:38:39.196423+0000","last_unstale":"2024-04-24T06:38:40.197283+0000","last_undegraded":"2024-04-24T06:38:40.197283+0000","last_fullsized":"2024-04-24T06:38:40.197283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:18:06.077317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247261+0000","last_change":"2024-04-24T06:38:38.185510+0000","last_active":"2024-04-24T06:38:40.247261+0000","last_peered":"2024-04-24T06:38:40.247261+0000","last_clean":"2024-04-24T06:38:40.247261+0000","last_became_active":"2024-04-24T06:38:38.185175+0000","last_became_peered":"2024-04-24T06:38:38.185175+0000","last_unstale":"2024-04-24T06:38:40.247261+0000","last_undegraded":"2024-04-24T06:38:40.247261+0000","last_fullsized":"2024-04-24T06:38:40.247261+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:12:36.252749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345900+0000","last_change":"2024-04-24T06:38:39.191902+0000","last_active":"2024-04-24T06:38:40.345900+0000","last_peered":"2024-04-24T06:38:40.345900+0000","last_clean":"2024-04-24T06:38:40.345900+0000","last_became_active":"2024-04-24T06:38:39.189373+0000","last_became_peered":"2024-04-24T06:38:39.189373+0000","last_unstale":"2024-04-24T06:38:40.345900+0000","last_undegraded":"2024-04-24T06:38:40.345900+0000","last_fullsized":"2024-04-24T06:38:40.345900+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:44.646457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248073+0000","last_change":"2024-04-24T06:38:38.187061+0000","last_active":"2024-04-24T06:38:40.248073+0000","last_peered":"2024-04-24T06:38:40.248073+0000","last_clean":"2024-04-24T06:38:40.248073+0000","last_became_active":"2024-04-24T06:38:38.186579+0000","last_became_peered":"2024-04-24T06:38:38.186579+0000","last_unstale":"2024-04-24T06:38:40.248073+0000","last_undegraded":"2024-04-24T06:38:40.248073+0000","last_fullsized":"2024-04-24T06:38:40.248073+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:20:20.407609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197838+0000","last_change":"2024-04-24T06:38:39.200872+0000","last_active":"2024-04-24T06:38:40.197838+0000","last_peered":"2024-04-24T06:38:40.197838+0000","last_clean":"2024-04-24T06:38:40.197838+0000","last_became_active":"2024-04-24T06:38:39.200381+0000","last_became_peered":"2024-04-24T06:38:39.200381+0000","last_unstale":"2024-04-24T06:38:40.197838+0000","last_undegraded":"2024-04-24T06:38:40.197838+0000","last_fullsized":"2024-04-24T06:38:40.197838+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193684+0000","last_change":"2024-04-24T06:38:38.190498+0000","last_active":"2024-04-24T06:38:40.193684+0000","last_peered":"2024-04-24T06:38:40.193684+0000","last_clean":"2024-04-24T06:38:40.193684+0000","last_became_active":"2024-04-24T06:38:38.190189+0000","last_became_peered":"2024-04-24T06:38:38.190189+0000","last_unstale":"2024-04-24T06:38:40.193684+0000","last_undegraded":"2024-04-24T06:38:40.193684+0000","last_fullsized":"2024-04-24T06:38:40.193684+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:37.387580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349028+0000","last_change":"2024-04-24T06:38:39.191116+0000","last_active":"2024-04-24T06:38:40.349028+0000","last_peered":"2024-04-24T06:38:40.349028+0000","last_clean":"2024-04-24T06:38:40.349028+0000","last_became_active":"2024-04-24T06:38:39.190170+0000","last_became_peered":"2024-04-24T06:38:39.190170+0000","last_unstale":"2024-04-24T06:38:40.349028+0000","last_undegraded":"2024-04-24T06:38:40.349028+0000","last_fullsized":"2024-04-24T06:38:40.349028+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:22:36.001714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"97'3","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350286+0000","last_change":"2024-04-24T06:38:38.201487+0000","last_active":"2024-04-24T06:38:40.350286+0000","last_peered":"2024-04-24T06:38:40.350286+0000","last_clean":"2024-04-24T06:38:40.350286+0000","last_became_active":"2024-04-24T06:38:38.201000+0000","last_became_peered":"2024-04-24T06:38:38.201000+0000","last_unstale":"2024-04-24T06:38:40.350286+0000","last_undegraded":"2024-04-24T06:38:40.350286+0000","last_fullsized":"2024-04-24T06:38:40.350286+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349985+0000","last_change":"2024-04-24T06:38:39.203026+0000","last_active":"2024-04-24T06:38:40.349985+0000","last_peered":"2024-04-24T06:38:40.349985+0000","last_clean":"2024-04-24T06:38:40.349985+0000","last_became_active":"2024-04-24T06:38:39.198765+0000","last_became_peered":"2024-04-24T06:38:39.198765+0000","last_unstale":"2024-04-24T06:38:40.349985+0000","last_undegraded":"2024-04-24T06:38:40.349985+0000","last_fullsized":"2024-04-24T06:38:40.349985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"97'6","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197750+0000","last_change":"2024-04-24T06:38:38.190980+0000","last_active":"2024-04-24T06:38:40.197750+0000","last_peered":"2024-04-24T06:38:40.197750+0000","last_clean":"2024-04-24T06:38:40.197750+0000","last_became_active":"2024-04-24T06:38:38.190686+0000","last_became_peered":"2024-04-24T06:38:38.190686+0000","last_unstale":"2024-04-24T06:38:40.197750+0000","last_undegraded":"2024-04-24T06:38:40.197750+0000","last_fullsized":"2024-04-24T06:38:40.197750+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:49.059567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.192882+0000","last_change":"2024-04-24T06:38:39.208692+0000","last_active":"2024-04-24T06:38:40.192882+0000","last_peered":"2024-04-24T06:38:40.192882+0000","last_clean":"2024-04-24T06:38:40.192882+0000","last_became_active":"2024-04-24T06:38:39.208429+0000","last_became_peered":"2024-04-24T06:38:39.208429+0000","last_unstale":"2024-04-24T06:38:40.192882+0000","last_undegraded":"2024-04-24T06:38:40.192882+0000","last_fullsized":"2024-04-24T06:38:40.192882+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345133+0000","last_change":"2024-04-24T06:38:38.190669+0000","last_active":"2024-04-24T06:38:40.345133+0000","last_peered":"2024-04-24T06:38:40.345133+0000","last_clean":"2024-04-24T06:38:40.345133+0000","last_became_active":"2024-04-24T06:38:38.189921+0000","last_became_peered":"2024-04-24T06:38:38.189921+0000","last_unstale":"2024-04-24T06:38:40.345133+0000","last_undegraded":"2024-04-24T06:38:40.345133+0000","last_fullsized":"2024-04-24T06:38:40.345133+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:39:35.842533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247221+0000","last_change":"2024-04-24T06:38:39.191212+0000","last_active":"2024-04-24T06:38:40.247221+0000","last_peered":"2024-04-24T06:38:40.247221+0000","last_clean":"2024-04-24T06:38:40.247221+0000","last_became_active":"2024-04-24T06:38:39.190920+0000","last_became_peered":"2024-04-24T06:38:39.190920+0000","last_unstale":"2024-04-24T06:38:40.247221+0000","last_undegraded":"2024-04-24T06:38:40.247221+0000","last_fullsized":"2024-04-24T06:38:40.247221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:21.614373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343947+0000","last_change":"2024-04-24T06:38:38.187644+0000","last_active":"2024-04-24T06:38:40.343947+0000","last_peered":"2024-04-24T06:38:40.343947+0000","last_clean":"2024-04-24T06:38:40.343947+0000","last_became_active":"2024-04-24T06:38:38.187365+0000","last_became_peered":"2024-04-24T06:38:38.187365+0000","last_unstale":"2024-04-24T06:38:40.343947+0000","last_undegraded":"2024-04-24T06:38:40.343947+0000","last_fullsized":"2024-04-24T06:38:40.343947+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:29.069513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196278+0000","last_change":"2024-04-24T06:38:39.197715+0000","last_active":"2024-04-24T06:38:40.196278+0000","last_peered":"2024-04-24T06:38:40.196278+0000","last_clean":"2024-04-24T06:38:40.196278+0000","last_became_active":"2024-04-24T06:38:39.197426+0000","last_became_peered":"2024-04-24T06:38:39.197426+0000","last_unstale":"2024-04-24T06:38:40.196278+0000","last_undegraded":"2024-04-24T06:38:40.196278+0000","last_fullsized":"2024-04-24T06:38:40.196278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:59.085923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196464+0000","last_change":"2024-04-24T06:38:38.197445+0000","last_active":"2024-04-24T06:38:40.196464+0000","last_peered":"2024-04-24T06:38:40.196464+0000","last_clean":"2024-04-24T06:38:40.196464+0000","last_became_active":"2024-04-24T06:38:38.197089+0000","last_became_peered":"2024-04-24T06:38:38.197089+0000","last_unstale":"2024-04-24T06:38:40.196464+0000","last_undegraded":"2024-04-24T06:38:40.196464+0000","last_fullsized":"2024-04-24T06:38:40.196464+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:33.508081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343339+0000","last_change":"2024-04-24T06:38:39.206909+0000","last_active":"2024-04-24T06:38:40.343339+0000","last_peered":"2024-04-24T06:38:40.343339+0000","last_clean":"2024-04-24T06:38:40.343339+0000","last_became_active":"2024-04-24T06:38:39.206602+0000","last_became_peered":"2024-04-24T06:38:39.206602+0000","last_unstale":"2024-04-24T06:38:40.343339+0000","last_undegraded":"2024-04-24T06:38:40.343339+0000","last_fullsized":"2024-04-24T06:38:40.343339+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:55.179321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"97'8","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247640+0000","last_change":"2024-04-24T06:38:38.195023+0000","last_active":"2024-04-24T06:38:40.247640+0000","last_peered":"2024-04-24T06:38:40.247640+0000","last_clean":"2024-04-24T06:38:40.247640+0000","last_became_active":"2024-04-24T06:38:38.194727+0000","last_became_peered":"2024-04-24T06:38:38.194727+0000","last_unstale":"2024-04-24T06:38:40.247640+0000","last_undegraded":"2024-04-24T06:38:40.247640+0000","last_fullsized":"2024-04-24T06:38:40.247640+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:47.203530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.349294+0000","last_change":"2024-04-24T06:38:39.189634+0000","last_active":"2024-04-24T06:38:40.349294+0000","last_peered":"2024-04-24T06:38:40.349294+0000","last_clean":"2024-04-24T06:38:40.349294+0000","last_became_active":"2024-04-24T06:38:39.189276+0000","last_became_peered":"2024-04-24T06:38:39.189276+0000","last_unstale":"2024-04-24T06:38:40.349294+0000","last_undegraded":"2024-04-24T06:38:40.349294+0000","last_fullsized":"2024-04-24T06:38:40.349294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:55.826700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.197754+0000","last_change":"2024-04-24T06:38:38.192153+0000","last_active":"2024-04-24T06:38:40.197754+0000","last_peered":"2024-04-24T06:38:40.197754+0000","last_clean":"2024-04-24T06:38:40.197754+0000","last_became_active":"2024-04-24T06:38:38.191860+0000","last_became_peered":"2024-04-24T06:38:38.191860+0000","last_unstale":"2024-04-24T06:38:40.197754+0000","last_undegraded":"2024-04-24T06:38:40.197754+0000","last_fullsized":"2024-04-24T06:38:40.197754+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347294+0000","last_change":"2024-04-24T06:38:39.190622+0000","last_active":"2024-04-24T06:38:40.347294+0000","last_peered":"2024-04-24T06:38:40.347294+0000","last_clean":"2024-04-24T06:38:40.347294+0000","last_became_active":"2024-04-24T06:38:39.189278+0000","last_became_peered":"2024-04-24T06:38:39.189278+0000","last_unstale":"2024-04-24T06:38:40.347294+0000","last_undegraded":"2024-04-24T06:38:40.347294+0000","last_fullsized":"2024-04-24T06:38:40.347294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:50.076960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348369+0000","last_change":"2024-04-24T06:38:38.201583+0000","last_active":"2024-04-24T06:38:40.348369+0000","last_peered":"2024-04-24T06:38:40.348369+0000","last_clean":"2024-04-24T06:38:40.348369+0000","last_became_active":"2024-04-24T06:38:38.199954+0000","last_became_peered":"2024-04-24T06:38:38.199954+0000","last_unstale":"2024-04-24T06:38:40.348369+0000","last_undegraded":"2024-04-24T06:38:40.348369+0000","last_fullsized":"2024-04-24T06:38:40.348369+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350010+0000","last_change":"2024-04-24T06:38:39.198039+0000","last_active":"2024-04-24T06:38:40.350010+0000","last_peered":"2024-04-24T06:38:40.350010+0000","last_clean":"2024-04-24T06:38:40.350010+0000","last_became_active":"2024-04-24T06:38:39.196519+0000","last_became_peered":"2024-04-24T06:38:39.196519+0000","last_unstale":"2024-04-24T06:38:40.350010+0000","last_undegraded":"2024-04-24T06:38:40.350010+0000","last_fullsized":"2024-04-24T06:38:40.350010+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:50:07.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344689+0000","last_change":"2024-04-24T06:38:38.193288+0000","last_active":"2024-04-24T06:38:40.344689+0000","last_peered":"2024-04-24T06:38:40.344689+0000","last_clean":"2024-04-24T06:38:40.344689+0000","last_became_active":"2024-04-24T06:38:38.192927+0000","last_became_peered":"2024-04-24T06:38:38.192927+0000","last_unstale":"2024-04-24T06:38:40.344689+0000","last_undegraded":"2024-04-24T06:38:40.344689+0000","last_fullsized":"2024-04-24T06:38:40.344689+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:23.497613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343849+0000","last_change":"2024-04-24T06:38:39.207942+0000","last_active":"2024-04-24T06:38:40.343849+0000","last_peered":"2024-04-24T06:38:40.343849+0000","last_clean":"2024-04-24T06:38:40.343849+0000","last_became_active":"2024-04-24T06:38:39.207684+0000","last_became_peered":"2024-04-24T06:38:39.207684+0000","last_unstale":"2024-04-24T06:38:40.343849+0000","last_undegraded":"2024-04-24T06:38:40.343849+0000","last_fullsized":"2024-04-24T06:38:40.343849+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:12.582458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344235+0000","last_change":"2024-04-24T06:38:38.199850+0000","last_active":"2024-04-24T06:38:40.344235+0000","last_peered":"2024-04-24T06:38:40.344235+0000","last_clean":"2024-04-24T06:38:40.344235+0000","last_became_active":"2024-04-24T06:38:38.199550+0000","last_became_peered":"2024-04-24T06:38:38.199550+0000","last_unstale":"2024-04-24T06:38:40.344235+0000","last_undegraded":"2024-04-24T06:38:40.344235+0000","last_fullsized":"2024-04-24T06:38:40.344235+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:52.606985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345057+0000","last_change":"2024-04-24T06:38:39.203367+0000","last_active":"2024-04-24T06:38:40.345057+0000","last_peered":"2024-04-24T06:38:40.345057+0000","last_clean":"2024-04-24T06:38:40.345057+0000","last_became_active":"2024-04-24T06:38:39.202890+0000","last_became_peered":"2024-04-24T06:38:39.202890+0000","last_unstale":"2024-04-24T06:38:40.345057+0000","last_undegraded":"2024-04-24T06:38:40.345057+0000","last_fullsized":"2024-04-24T06:38:40.345057+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:17:35.113576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346450+0000","last_change":"2024-04-24T06:38:38.200141+0000","last_active":"2024-04-24T06:38:40.346450+0000","last_peered":"2024-04-24T06:38:40.346450+0000","last_clean":"2024-04-24T06:38:40.346450+0000","last_became_active":"2024-04-24T06:38:38.199811+0000","last_became_peered":"2024-04-24T06:38:38.199811+0000","last_unstale":"2024-04-24T06:38:40.346450+0000","last_undegraded":"2024-04-24T06:38:40.346450+0000","last_fullsized":"2024-04-24T06:38:40.346450+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:00.654577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343446+0000","last_change":"2024-04-24T06:38:39.193012+0000","last_active":"2024-04-24T06:38:40.343446+0000","last_peered":"2024-04-24T06:38:40.343446+0000","last_clean":"2024-04-24T06:38:40.343446+0000","last_became_active":"2024-04-24T06:38:39.192559+0000","last_became_peered":"2024-04-24T06:38:39.192559+0000","last_unstale":"2024-04-24T06:38:40.343446+0000","last_undegraded":"2024-04-24T06:38:40.343446+0000","last_fullsized":"2024-04-24T06:38:40.343446+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:20:51.055492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346087+0000","last_change":"2024-04-24T06:38:38.200095+0000","last_active":"2024-04-24T06:38:40.346087+0000","last_peered":"2024-04-24T06:38:40.346087+0000","last_clean":"2024-04-24T06:38:40.346087+0000","last_became_active":"2024-04-24T06:38:38.199766+0000","last_became_peered":"2024-04-24T06:38:38.199766+0000","last_unstale":"2024-04-24T06:38:40.346087+0000","last_undegraded":"2024-04-24T06:38:40.346087+0000","last_fullsized":"2024-04-24T06:38:40.346087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:35.221397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347442+0000","last_change":"2024-04-24T06:38:39.196328+0000","last_active":"2024-04-24T06:38:40.347442+0000","last_peered":"2024-04-24T06:38:40.347442+0000","last_clean":"2024-04-24T06:38:40.347442+0000","last_became_active":"2024-04-24T06:38:39.195995+0000","last_became_peered":"2024-04-24T06:38:39.195995+0000","last_unstale":"2024-04-24T06:38:40.347442+0000","last_undegraded":"2024-04-24T06:38:40.347442+0000","last_fullsized":"2024-04-24T06:38:40.347442+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:02.364072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343171+0000","last_change":"2024-04-24T06:38:38.197579+0000","last_active":"2024-04-24T06:38:40.343171+0000","last_peered":"2024-04-24T06:38:40.343171+0000","last_clean":"2024-04-24T06:38:40.343171+0000","last_became_active":"2024-04-24T06:38:38.197079+0000","last_became_peered":"2024-04-24T06:38:38.197079+0000","last_unstale":"2024-04-24T06:38:40.343171+0000","last_undegraded":"2024-04-24T06:38:40.343171+0000","last_fullsized":"2024-04-24T06:38:40.343171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:56.119339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346060+0000","last_change":"2024-04-24T06:38:39.195363+0000","last_active":"2024-04-24T06:38:40.346060+0000","last_peered":"2024-04-24T06:38:40.346060+0000","last_clean":"2024-04-24T06:38:40.346060+0000","last_became_active":"2024-04-24T06:38:39.195103+0000","last_became_peered":"2024-04-24T06:38:39.195103+0000","last_unstale":"2024-04-24T06:38:40.346060+0000","last_undegraded":"2024-04-24T06:38:40.346060+0000","last_fullsized":"2024-04-24T06:38:40.346060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:09.894221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348476+0000","last_change":"2024-04-24T06:38:38.195715+0000","last_active":"2024-04-24T06:38:40.348476+0000","last_peered":"2024-04-24T06:38:40.348476+0000","last_clean":"2024-04-24T06:38:40.348476+0000","last_became_active":"2024-04-24T06:38:38.195332+0000","last_became_peered":"2024-04-24T06:38:38.195332+0000","last_unstale":"2024-04-24T06:38:40.348476+0000","last_undegraded":"2024-04-24T06:38:40.348476+0000","last_fullsized":"2024-04-24T06:38:40.348476+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:58:20.596981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.247821+0000","last_change":"2024-04-24T06:38:39.205097+0000","last_active":"2024-04-24T06:38:40.247821+0000","last_peered":"2024-04-24T06:38:40.247821+0000","last_clean":"2024-04-24T06:38:40.247821+0000","last_became_active":"2024-04-24T06:38:39.204775+0000","last_became_peered":"2024-04-24T06:38:39.204775+0000","last_unstale":"2024-04-24T06:38:40.247821+0000","last_undegraded":"2024-04-24T06:38:40.247821+0000","last_fullsized":"2024-04-24T06:38:40.247821+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:40:34.525473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346163+0000","last_change":"2024-04-24T06:38:38.200684+0000","last_active":"2024-04-24T06:38:40.346163+0000","last_peered":"2024-04-24T06:38:40.346163+0000","last_clean":"2024-04-24T06:38:40.346163+0000","last_became_active":"2024-04-24T06:38:38.200389+0000","last_became_peered":"2024-04-24T06:38:38.200389+0000","last_unstale":"2024-04-24T06:38:40.346163+0000","last_undegraded":"2024-04-24T06:38:40.346163+0000","last_fullsized":"2024-04-24T06:38:40.346163+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:30:41.285222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.248344+0000","last_change":"2024-04-24T06:38:39.199725+0000","last_active":"2024-04-24T06:38:40.248344+0000","last_peered":"2024-04-24T06:38:40.248344+0000","last_clean":"2024-04-24T06:38:40.248344+0000","last_became_active":"2024-04-24T06:38:39.196222+0000","last_became_peered":"2024-04-24T06:38:39.196222+0000","last_unstale":"2024-04-24T06:38:40.248344+0000","last_undegraded":"2024-04-24T06:38:40.248344+0000","last_fullsized":"2024-04-24T06:38:40.248344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:34.212630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.345335+0000","last_change":"2024-04-24T06:38:38.188269+0000","last_active":"2024-04-24T06:38:40.345335+0000","last_peered":"2024-04-24T06:38:40.345335+0000","last_clean":"2024-04-24T06:38:40.345335+0000","last_became_active":"2024-04-24T06:38:38.187831+0000","last_became_peered":"2024-04-24T06:38:38.187831+0000","last_unstale":"2024-04-24T06:38:40.345335+0000","last_undegraded":"2024-04-24T06:38:40.345335+0000","last_fullsized":"2024-04-24T06:38:40.345335+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:57.876709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348478+0000","last_change":"2024-04-24T06:38:39.205151+0000","last_active":"2024-04-24T06:38:40.348478+0000","last_peered":"2024-04-24T06:38:40.348478+0000","last_clean":"2024-04-24T06:38:40.348478+0000","last_became_active":"2024-04-24T06:38:39.204845+0000","last_became_peered":"2024-04-24T06:38:39.204845+0000","last_unstale":"2024-04-24T06:38:40.348478+0000","last_undegraded":"2024-04-24T06:38:40.348478+0000","last_fullsized":"2024-04-24T06:38:40.348478+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:21:12.440562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.348954+0000","last_change":"2024-04-24T06:38:38.199087+0000","last_active":"2024-04-24T06:38:40.348954+0000","last_peered":"2024-04-24T06:38:40.348954+0000","last_clean":"2024-04-24T06:38:40.348954+0000","last_became_active":"2024-04-24T06:38:38.197951+0000","last_became_peered":"2024-04-24T06:38:38.197951+0000","last_unstale":"2024-04-24T06:38:40.348954+0000","last_undegraded":"2024-04-24T06:38:40.348954+0000","last_fullsized":"2024-04-24T06:38:40.348954+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344895+0000","last_change":"2024-04-24T06:38:39.193206+0000","last_active":"2024-04-24T06:38:40.344895+0000","last_peered":"2024-04-24T06:38:40.344895+0000","last_clean":"2024-04-24T06:38:40.344895+0000","last_became_active":"2024-04-24T06:38:39.188996+0000","last_became_peered":"2024-04-24T06:38:39.188996+0000","last_unstale":"2024-04-24T06:38:40.344895+0000","last_undegraded":"2024-04-24T06:38:40.344895+0000","last_fullsized":"2024-04-24T06:38:40.344895+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:26.822649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344411+0000","last_change":"2024-04-24T06:38:38.189257+0000","last_active":"2024-04-24T06:38:40.344411+0000","last_peered":"2024-04-24T06:38:40.344411+0000","last_clean":"2024-04-24T06:38:40.344411+0000","last_became_active":"2024-04-24T06:38:38.188947+0000","last_became_peered":"2024-04-24T06:38:38.188947+0000","last_unstale":"2024-04-24T06:38:40.344411+0000","last_undegraded":"2024-04-24T06:38:40.344411+0000","last_fullsized":"2024-04-24T06:38:40.344411+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:15.213157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.196852+0000","last_change":"2024-04-24T06:38:39.186708+0000","last_active":"2024-04-24T06:38:40.196852+0000","last_peered":"2024-04-24T06:38:40.196852+0000","last_clean":"2024-04-24T06:38:40.196852+0000","last_became_active":"2024-04-24T06:38:39.186405+0000","last_became_peered":"2024-04-24T06:38:39.186405+0000","last_unstale":"2024-04-24T06:38:40.196852+0000","last_undegraded":"2024-04-24T06:38:40.196852+0000","last_fullsized":"2024-04-24T06:38:40.196852+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:38.404973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347138+0000","last_change":"2024-04-24T06:38:38.199875+0000","last_active":"2024-04-24T06:38:40.347138+0000","last_peered":"2024-04-24T06:38:40.347138+0000","last_clean":"2024-04-24T06:38:40.347138+0000","last_became_active":"2024-04-24T06:38:38.199365+0000","last_became_peered":"2024-04-24T06:38:38.199365+0000","last_unstale":"2024-04-24T06:38:40.347138+0000","last_undegraded":"2024-04-24T06:38:40.347138+0000","last_fullsized":"2024-04-24T06:38:40.347138+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:49.059567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.343879+0000","last_change":"2024-04-24T06:38:39.204035+0000","last_active":"2024-04-24T06:38:40.343879+0000","last_peered":"2024-04-24T06:38:40.343879+0000","last_clean":"2024-04-24T06:38:40.343879+0000","last_became_active":"2024-04-24T06:38:39.203749+0000","last_became_peered":"2024-04-24T06:38:39.203749+0000","last_unstale":"2024-04-24T06:38:40.343879+0000","last_undegraded":"2024-04-24T06:38:40.343879+0000","last_fullsized":"2024-04-24T06:38:40.343879+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:32:34.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.193041+0000","last_change":"2024-04-24T06:38:38.186818+0000","last_active":"2024-04-24T06:38:40.193041+0000","last_peered":"2024-04-24T06:38:40.193041+0000","last_clean":"2024-04-24T06:38:40.193041+0000","last_became_active":"2024-04-24T06:38:38.186521+0000","last_became_peered":"2024-04-24T06:38:38.186521+0000","last_unstale":"2024-04-24T06:38:40.193041+0000","last_undegraded":"2024-04-24T06:38:40.193041+0000","last_fullsized":"2024-04-24T06:38:40.193041+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:58.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.347890+0000","last_change":"2024-04-24T06:38:39.204573+0000","last_active":"2024-04-24T06:38:40.347890+0000","last_peered":"2024-04-24T06:38:40.347890+0000","last_clean":"2024-04-24T06:38:40.347890+0000","last_became_active":"2024-04-24T06:38:39.204193+0000","last_became_peered":"2024-04-24T06:38:39.204193+0000","last_unstale":"2024-04-24T06:38:40.347890+0000","last_undegraded":"2024-04-24T06:38:40.347890+0000","last_fullsized":"2024-04-24T06:38:40.347890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:05.203025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344146+0000","last_change":"2024-04-24T06:38:38.194934+0000","last_active":"2024-04-24T06:38:40.344146+0000","last_peered":"2024-04-24T06:38:40.344146+0000","last_clean":"2024-04-24T06:38:40.344146+0000","last_became_active":"2024-04-24T06:38:38.194457+0000","last_became_peered":"2024-04-24T06:38:38.194457+0000","last_unstale":"2024-04-24T06:38:40.344146+0000","last_undegraded":"2024-04-24T06:38:40.344146+0000","last_fullsized":"2024-04-24T06:38:40.344146+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:56:43.925402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.344535+0000","last_change":"2024-04-24T06:38:39.210529+0000","last_active":"2024-04-24T06:38:40.344535+0000","last_peered":"2024-04-24T06:38:40.344535+0000","last_clean":"2024-04-24T06:38:40.344535+0000","last_became_active":"2024-04-24T06:38:39.209451+0000","last_became_peered":"2024-04-24T06:38:39.209451+0000","last_unstale":"2024-04-24T06:38:40.344535+0000","last_undegraded":"2024-04-24T06:38:40.344535+0000","last_fullsized":"2024-04-24T06:38:40.344535+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:38.144412+0000","last_clean_scrub_stamp":"2024-04-24T06:38:38.144412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:20:15.554365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.350359+0000","last_change":"2024-04-24T06:38:38.200307+0000","last_active":"2024-04-24T06:38:40.350359+0000","last_peered":"2024-04-24T06:38:40.350359+0000","last_clean":"2024-04-24T06:38:40.350359+0000","last_became_active":"2024-04-24T06:38:38.199966+0000","last_became_peered":"2024-04-24T06:38:38.199966+0000","last_unstale":"2024-04-24T06:38:40.350359+0000","last_undegraded":"2024-04-24T06:38:40.350359+0000","last_fullsized":"2024-04-24T06:38:40.350359+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:38:37.127020+0000","last_clean_scrub_stamp":"2024-04-24T06:38:37.127020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:33.195238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T06:38:40.346627+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:38:40.346627+0000","last_peered":"2024-04-24T06:38:40.346627+0000","last_clean":"2024-04-24T06:38:40.346627+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:38:40.346627+0000","last_undegraded":"2024-04-24T06:38:40.346627+0000","last_fullsized":"2024-04-24T06:38:40.346627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1814528,"data_stored":1114635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":524807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":524225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":525891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":525789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":524189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":524155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1810432,"data_stored":1115105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:38:44.203 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:38:44.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:38:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.460+0000 7f9521e00640 1 Processor -- start 2024-04-24T06:38:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.460+0000 7f9521e00640 1 -- start start 2024-04-24T06:38:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9521e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f951c0b3190 0x7f951c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9521e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f951c06dcb0 0x7f951c06e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9521e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c0b3ab0 con 0x7f951c073e00 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9521e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f951c071e80 con 0x7f951c0b3190 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9521e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f951c072160 con 0x7f951c06dcb0 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9520a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f951c0b3190 0x7f951c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9520a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f951c0b3190 0x7f951c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44446/0 (socket says 172.21.15.59:44446) 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.462+0000 7f9520a00640 1 -- 172.21.15.59:0/4033267511 learned_addr learned my addr 172.21.15.59:0/4033267511 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.463+0000 7f9520a00640 1 -- 172.21.15.59:0/4033267511 >> v2:172.21.15.66:3300/0 conn(0x7f951c06dcb0 msgr2=0x7f951c06e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.463+0000 7f9520a00640 1 --2- 172.21.15.59:0/4033267511 >> v2:172.21.15.66:3300/0 conn(0x7f951c06dcb0 0x7f951c06e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.463+0000 7f9520a00640 1 -- 172.21.15.59:0/4033267511 >> v1:172.21.15.66:6789/0 conn(0x7f951c073e00 legacy=0x7f951c15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.463+0000 7f9520a00640 1 -- 172.21.15.59:0/4033267511 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c072440 con 0x7f951c0b3190 2024-04-24T06:38:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.464+0000 7f9520a00640 1 --2- 172.21.15.59:0/4033267511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c0b3570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f951c163280 tx=0x7f950c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8c6496420f2b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.464+0000 7f951be00640 1 -- 172.21.15.59:0/4033267511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c037070 con 0x7f951c0b3190 2024-04-24T06:38:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.464+0000 7f951be00640 1 -- 172.21.15.59:0/4033267511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c032440 con 0x7f951c0b3190 2024-04-24T06:38:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.464+0000 7f951be00640 1 -- 172.21.15.59:0/4033267511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c032810 con 0x7f951c0b3190 2024-04-24T06:38:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 -- 172.21.15.59:0/4033267511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 msgr2=0x7f951c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 --2- 172.21.15.59:0/4033267511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c0b3570 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f951c163280 tx=0x7f950c02a230 comp rx=0 tx=0).stop 2024-04-24T06:38:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 -- 172.21.15.59:0/4033267511 shutdown_connections 2024-04-24T06:38:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 --2- 172.21.15.59:0/4033267511 >> v2:172.21.15.66:3300/0 conn(0x7f951c06dcb0 0x7f951c06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 --2- 172.21.15.59:0/4033267511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c0b3570 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 -- 172.21.15.59:0/4033267511 >> 172.21.15.59:0/4033267511 conn(0x7f951c0b1e90 msgr2=0x7f951c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 -- 172.21.15.59:0/4033267511 shutdown_connections 2024-04-24T06:38:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.465+0000 7f9521e00640 1 -- 172.21.15.59:0/4033267511 wait complete. 2024-04-24T06:38:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.467+0000 7f9521e00640 1 Processor -- start 2024-04-24T06:38:44.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.467+0000 7f9521e00640 1 -- start start 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f951c06dcb0 0x7f951c148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f951c073e00 0x7f951c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15fb60 con 0x7f951c0b3190 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15f9e0 con 0x7f951c06dcb0 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f951c072f20 con 0x7f951c073e00 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44450/0 (socket says 172.21.15.59:44450) 2024-04-24T06:38:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.468+0000 7f9521400640 1 -- 172.21.15.59:0/3973607479 learned_addr learned my addr 172.21.15.59:0/3973607479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:44.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9520a00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f951c06dcb0 0x7f951c148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 -- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f951c073e00 msgr2=0x7f951c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f951c073e00 0x7f951c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 -- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f951c06dcb0 msgr2=0x7f951c148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f951c06dcb0 0x7f951c148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 -- 172.21.15.59:0/3973607479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c159c80 con 0x7f951c0b3190 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521400640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f951400b970 tx=0x7f951400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514019070 con 0x7f951c0b3190 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951400cb00 con 0x7f951c0b3190 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514014440 con 0x7f951c0b3190 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.469+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c159f60 con 0x7f951c0b3190 2024-04-24T06:38:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.470+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c15a520 con 0x7f951c0b3190 2024-04-24T06:38:44.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.471+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c0b54d0 con 0x7f951c0b3190 2024-04-24T06:38:44.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.472+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f951401c070 con 0x7f951c0b3190 2024-04-24T06:38:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.473+0000 7f9519600640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.473+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f951408b890 con 0x7f951c0b3190 2024-04-24T06:38:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.474+0000 7f9520a00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:44.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.474+0000 7f9520a00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 0x7f94f806edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f950c034000 tx=0x7f950c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:44.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.478+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f951c0b54d0 con 0x7f951c0b3190 2024-04-24T06:38:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.758+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f951c0afba0 con 0x7f951c0b3190 2024-04-24T06:38:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.763+0000 7f9519600640 1 -- 172.21.15.59:0/3973607479 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f9514059cb0 con 0x7f951c0b3190 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:38:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 msgr2=0x7f94f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 0x7f94f806edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f950c034000 tx=0x7f950c02e000 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 msgr2=0x7f951c160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f951400b970 tx=0x7f951400be30 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 shutdown_connections 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.766+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f951c0b3190 0x7f951c160ba0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f951c073e00 0x7f951c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 --2- 172.21.15.59:0/3973607479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f951c06dcb0 0x7f951c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 >> 172.21.15.59:0/3973607479 conn(0x7f951c0b1e90 msgr2=0x7f951c15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 shutdown_connections 2024-04-24T06:38:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.767+0000 7f9521e00640 1 -- 172.21.15.59:0/3973607479 wait complete. 2024-04-24T06:38:44.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:38:44.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.994+0000 7f35f5600640 1 Processor -- start 2024-04-24T06:38:44.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.995+0000 7f35f5600640 1 -- start start 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.996+0000 7f35f5600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35f015e7c0 0x7f35f00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.996+0000 7f35f5600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f35f00d2630 0x7f35f00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.996+0000 7f35f5600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f00cf030 con 0x7f35f00b6530 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.996+0000 7f35f5600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f35f00b7000 con 0x7f35f015e7c0 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.996+0000 7f35f5600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f35f00cf620 con 0x7f35f00d2630 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35f015e7c0 0x7f35f00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35f015e7c0 0x7f35f00d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44462/0 (socket says 172.21.15.59:44462) 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35f4c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f35f00b6530 0x7f35f00d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51222/0 (socket says 172.21.15.59:51222) 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 -- 172.21.15.59:0/2348618029 learned_addr learned my addr 172.21.15.59:0/2348618029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:44.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 -- 172.21.15.59:0/2348618029 >> v2:172.21.15.66:3300/0 conn(0x7f35f00d2630 msgr2=0x7f35f00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 --2- 172.21.15.59:0/2348618029 >> v2:172.21.15.66:3300/0 conn(0x7f35f00d2630 0x7f35f00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 -- 172.21.15.59:0/2348618029 >> v1:172.21.15.59:6789/0 conn(0x7f35f00b6530 legacy=0x7f35f00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.997+0000 7f35efe00640 1 -- 172.21.15.59:0/2348618029 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f00b6a40 con 0x7f35f015e7c0 2024-04-24T06:38:45.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.998+0000 7f35efe00640 1 --2- 172.21.15.59:0/2348618029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00d20f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f35f00d55c0 tx=0x7f35e002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf408f53f23aa99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.999+0000 7f35eea00640 1 -- 172.21.15.59:0/2348618029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0036070 con 0x7f35f015e7c0 2024-04-24T06:38:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.999+0000 7f35eea00640 1 -- 172.21.15.59:0/2348618029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0031600 con 0x7f35f015e7c0 2024-04-24T06:38:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:44.999+0000 7f35eea00640 1 -- 172.21.15.59:0/2348618029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00319d0 con 0x7f35f015e7c0 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 -- 172.21.15.59:0/2348618029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 msgr2=0x7f35f00d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 --2- 172.21.15.59:0/2348618029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00d20f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f35f00d55c0 tx=0x7f35e002a740 comp rx=0 tx=0).stop 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 -- 172.21.15.59:0/2348618029 shutdown_connections 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 --2- 172.21.15.59:0/2348618029 >> v2:172.21.15.66:3300/0 conn(0x7f35f00d2630 0x7f35f00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 --2- 172.21.15.59:0/2348618029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00d20f0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 -- 172.21.15.59:0/2348618029 >> 172.21.15.59:0/2348618029 conn(0x7f35f009d920 msgr2=0x7f35f008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 -- 172.21.15.59:0/2348618029 shutdown_connections 2024-04-24T06:38:45.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.000+0000 7f35f5600640 1 -- 172.21.15.59:0/2348618029 wait complete. 2024-04-24T06:38:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.002+0000 7f35f5600640 1 Processor -- start 2024-04-24T06:38:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.002+0000 7f35f5600640 1 -- start start 2024-04-24T06:38:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f00b6530 0x7f35f014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35f00d2630 0x7f35f00b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00d1a80 con 0x7f35f015e7c0 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00cf2b0 con 0x7f35f00b6530 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f5600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f35f00d1c00 con 0x7f35f00d2630 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44466/0 (socket says 172.21.15.59:44466) 2024-04-24T06:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 -- 172.21.15.59:0/9445178 learned_addr learned my addr 172.21.15.59:0/9445178 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35efe00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f00b6530 0x7f35f014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 -- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35f00d2630 msgr2=0x7f35f00b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35f00d2630 0x7f35f00b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.003+0000 7f35f4c00640 1 -- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f00b6530 msgr2=0x7f35f014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35f4c00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f00b6530 0x7f35f014f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35f4c00640 1 -- 172.21.15.59:0/9445178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f00b13f0 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35f4c00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c400 tx=0x7f35e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e80074e0 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e8007680 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f00b16d0 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e80049e0 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.004+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f00b1c90 con 0x7f35f015e7c0 2024-04-24T06:38:45.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.005+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f004fd90 con 0x7f35f015e7c0 2024-04-24T06:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.007+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35e801a020 con 0x7f35f015e7c0 2024-04-24T06:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.007+0000 7f35e7e00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 0x7f35cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.008+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f35e808b330 con 0x7f35f015e7c0 2024-04-24T06:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.008+0000 7f35efe00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 0x7f35cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.009+0000 7f35efe00640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 0x7f35cc06ede0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f35e0033000 tx=0x7f35e0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.013+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35f00cf030 con 0x7f35f015e7c0 2024-04-24T06:38:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.256+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35f00d0890 con 0x7f35f015e7c0 2024-04-24T06:38:45.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.257+0000 7f35e7e00640 1 -- 172.21.15.59:0/9445178 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5177 (secure 0 0 0) 0x7f35e8059760 con 0x7f35f015e7c0 2024-04-24T06:38:45.259 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:45.259 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:38:41.731305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/891418539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":891418539},{"type":"v1","addr":"172.21.15.66:6834","nonce":891418539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":7}]} 2024-04-24T06:38:45.259 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 83 2024-04-24T06:38:45.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.259+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 msgr2=0x7f35cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.259+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 0x7f35cc06ede0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f35e0033000 tx=0x7f35e0009000 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.259+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 msgr2=0x7f35f00cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.259+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c400 tx=0x7f35e800c8c0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 shutdown_connections 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f35cc06c920 0x7f35cc06ede0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f015e7c0 0x7f35f00cbd70 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f35f00d2630 0x7f35f00b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 --2- 172.21.15.59:0/9445178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35f00b6530 0x7f35f014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 >> 172.21.15.59:0/9445178 conn(0x7f35f009d920 msgr2=0x7f35f00b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 shutdown_connections 2024-04-24T06:38:45.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.260+0000 7f35f5600640 1 -- 172.21.15.59:0/9445178 wait complete. 2024-04-24T06:38:45.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:38:38.647945+0000', 'modified': '2024-04-24T06:38:41.731305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8499': {'gid': 8499, 'name': 'd', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6834/891418539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 891418539}, {'type': 'v1', 'addr': '172.21.15.66:6834', 'nonce': 891418539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8499, 'qdb_cluster': [8499]} 2024-04-24T06:38:45.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:38:45.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T06:38:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.909+0000 7fbee5800640 1 Processor -- start 2024-04-24T06:38:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.909+0000 7fbee5800640 1 -- start start 2024-04-24T06:38:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbee015aca0 0x7fbee00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee5800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee015b110 con 0x7fbee00b45d0 2024-04-24T06:38:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee5800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee00b4c30 con 0x7fbee0071e70 2024-04-24T06:38:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee5800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbee00d0cd0 con 0x7fbee015aca0 2024-04-24T06:38:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbee015aca0 0x7fbee00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbee015aca0 0x7fbee00cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44474/0 (socket says 172.21.15.59:44474) 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.911+0000 7fbee4e00640 1 -- 172.21.15.59:0/3755616018 learned_addr learned my addr 172.21.15.59:0/3755616018 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.912+0000 7fbee4e00640 1 -- 172.21.15.59:0/3755616018 >> v1:172.21.15.66:6789/0 conn(0x7fbee0071e70 legacy=0x7fbee00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.912+0000 7fbee4e00640 1 -- 172.21.15.59:0/3755616018 >> v1:172.21.15.59:6789/0 conn(0x7fbee00b45d0 legacy=0x7fbee015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.912+0000 7fbee4e00640 1 -- 172.21.15.59:0/3755616018 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee00b4f10 con 0x7fbee015aca0 2024-04-24T06:38:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.913+0000 7fbee4e00640 1 --2- 172.21.15.59:0/3755616018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee00cc920 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009e50 tx=0x7fbed802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26defd5b5854e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.913+0000 7fbedea00640 1 -- 172.21.15.59:0/3755616018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8036070 con 0x7fbee015aca0 2024-04-24T06:38:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.913+0000 7fbedea00640 1 -- 172.21.15.59:0/3755616018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8031600 con 0x7fbee015aca0 2024-04-24T06:38:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.913+0000 7fbedea00640 1 -- 172.21.15.59:0/3755616018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80319d0 con 0x7fbee015aca0 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 -- 172.21.15.59:0/3755616018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 msgr2=0x7fbee00cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 --2- 172.21.15.59:0/3755616018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee00cc920 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009e50 tx=0x7fbed802a730 comp rx=0 tx=0).stop 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 -- 172.21.15.59:0/3755616018 shutdown_connections 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 --2- 172.21.15.59:0/3755616018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee00cc920 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 -- 172.21.15.59:0/3755616018 >> 172.21.15.59:0/3755616018 conn(0x7fbee006bdf0 msgr2=0x7fbee006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 -- 172.21.15.59:0/3755616018 shutdown_connections 2024-04-24T06:38:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.914+0000 7fbee5800640 1 -- 172.21.15.59:0/3755616018 wait complete. 2024-04-24T06:38:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.916+0000 7fbee5800640 1 Processor -- start 2024-04-24T06:38:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.917+0000 7fbee5800640 1 -- start start 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbee0071e70 0x7fbee015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbee00b45d0 0x7fbee01f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0073e30 con 0x7fbee015aca0 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbee00722d0 con 0x7fbee00b45d0 2024-04-24T06:38:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbee0073800 con 0x7fbee0071e70 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44488/0 (socket says 172.21.15.59:44488) 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.918+0000 7fbee4e00640 1 -- 172.21.15.59:0/21601900 learned_addr learned my addr 172.21.15.59:0/21601900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 -- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbee0071e70 msgr2=0x7fbee015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbee0071e70 0x7fbee015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 -- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbee00b45d0 msgr2=0x7fbee01f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbee00b45d0 0x7fbee01f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 -- 172.21.15.59:0/21601900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01422e0 con 0x7fbee015aca0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee4e00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009f70 tx=0x7fbed8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbed7e00640 1 -- 172.21.15.59:0/21601900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed803f020 con 0x7fbee015aca0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbed7e00640 1 -- 172.21.15.59:0/21601900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8037040 con 0x7fbee015aca0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbed7e00640 1 -- 172.21.15.59:0/21601900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8036070 con 0x7fbee015aca0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.919+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01428a0 con 0x7fbee015aca0 2024-04-24T06:38:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.920+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee0142e60 con 0x7fbee015aca0 2024-04-24T06:38:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.923+0000 7fbed7e00640 1 -- 172.21.15.59:0/21601900 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbed8042020 con 0x7fbee015aca0 2024-04-24T06:38:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.923+0000 7fbed7e00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 0x7fbec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.924+0000 7fbed7e00640 1 -- 172.21.15.59:0/21601900 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fbed80ac0f0 con 0x7fbee015aca0 2024-04-24T06:38:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.924+0000 7fbedfe00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 0x7fbec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.924+0000 7fbee9c1b740 1 Processor -- start 2024-04-24T06:38:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.924+0000 7fbee9c1b740 1 -- start start 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbedfe00640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 0x7fbec006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000f60 tx=0x7fbed00079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4dfa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cac4d64b10 con 0x55cac4e2a360 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cac4d0b2b0 con 0x55cac4e47980 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cac4d60ad0 con 0x55cac4e2aaa0 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44500/0 (socket says 172.21.15.59:44500) 2024-04-24T06:38:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.925+0000 7fbee4e00640 1 -- 172.21.15.59:0/4210657209 learned_addr learned my addr 172.21.15.59:0/4210657209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbedfe00640 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4dfa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 -- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 msgr2=0x55cac4e29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 -- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 msgr2=0x55cac4dfa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4dfa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 -- 172.21.15.59:0/4210657209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac4a772d0 con 0x55cac4e2a360 2024-04-24T06:38:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbedfe00640 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4dfa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbee4e00640 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbed80761e0 tx=0x7fbed80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8008c90d81b186 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbed4c00640 1 -- 172.21.15.59:0/4210657209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8092020 con 0x55cac4e2a360 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbed4c00640 1 -- 172.21.15.59:0/4210657209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed807e130 con 0x55cac4e2a360 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.926+0000 7fbed4c00640 1 -- 172.21.15.59:0/4210657209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8082960 con 0x55cac4e2a360 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 -- 172.21.15.59:0/4210657209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 msgr2=0x55cac4dff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbed80761e0 tx=0x7fbed80095d0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 -- 172.21.15.59:0/4210657209 shutdown_connections 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2a360 0x55cac4dff170 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/4210657209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4dfa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 -- 172.21.15.59:0/4210657209 >> 172.21.15.59:0/4210657209 conn(0x55cac4dfc580 msgr2=0x55cac4e44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 -- 172.21.15.59:0/4210657209 shutdown_connections 2024-04-24T06:38:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.927+0000 7fbee9c1b740 1 -- 172.21.15.59:0/4210657209 wait complete. 2024-04-24T06:38:45.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.928+0000 7fbee9c1b740 1 -- start start 2024-04-24T06:38:45.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.928+0000 7fbee9c1b740 1 Processor -- start 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e17530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cac4d64b10 con 0x55cac4e2af90 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cac4d1e590 con 0x55cac4e47980 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cac4dfe730 con 0x55cac4e2aaa0 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee9c1b740 10 client.0 _ll_register_callbacks cb 0x55cac4d013a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44506/0 (socket says 172.21.15.59:44506) 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.929+0000 7fbee4e00640 1 -- 172.21.15.59:0/1605514233 learned_addr learned my addr 172.21.15.59:0/1605514233 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbedf400640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e17530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 -- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 msgr2=0x55cac4e143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 -- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 msgr2=0x55cac4e17530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e17530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 -- 172.21.15.59:0/1605514233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac4b416b0 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbedf400640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee4e00640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbed80761e0 tx=0x7fbed8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8806471a8a2f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbec9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8098060 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed807e130 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.930+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fbed80763f0 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbed8092040 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 10 client.8643 fetch_fsmap learned FSMap version 83 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55cac4e15660 con 0x55cac4e2af90 2024-04-24T06:38:45.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed807e130 con 0x55cac4e2af90 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fbed8099250 con 0x55cac4e2af90 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 10 client.8643 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 10 client.8643 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 20 client.8643 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e42c60 0x55cac4e43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 4 client.8643 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.931+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac4d72530 con 0x55cac4e42c60 2024-04-24T06:38:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.932+0000 7fbedfe00640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e42c60 0x55cac4e43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.933+0000 7fbedfe00640 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e42c60 0x55cac4e43060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9100cd49d34a95ac server_cookie=c6b5f4430c9a40af in_seq=0 out_seq=0 2024-04-24T06:38:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.933+0000 7fbec9600640 10 client.8643 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:38:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbec9600640 1 -- 172.21.15.59:0/1605514233 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cac4d72530 con 0x55cac4e42c60 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbec9600640 10 client.8643 handle_command_reply: tid=0 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 client.8643 shutdown 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 msgr2=0x55cac4e45f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbed80761e0 tx=0x7fbed8009810 comp rx=0 tx=0).stop 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 shutdown_connections 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e42c60 0x55cac4e43060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2af90 0x55cac4e45f50 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/1605514233 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2aaa0 0x55cac4e143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 >> 172.21.15.59:0/1605514233 conn(0x55cac4e2a360 msgr2=0x55cac4df9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbec9600640 0 client.8643 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 shutdown_connections 2024-04-24T06:38:45.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.935+0000 7fbee9c1b740 1 -- 172.21.15.59:0/1605514233 wait complete. 2024-04-24T06:38:45.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.936+0000 7fbee9c1b740 20 client.8643 trim_cache size 0 max 16384 2024-04-24T06:38:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.956+0000 7fbee9c1b740 1 Processor -- start 2024-04-24T06:38:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.956+0000 7fbee9c1b740 1 -- start start 2024-04-24T06:38:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e43580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e44c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01fde70 con 0x55cac4e2aaa0 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0142110 con 0x55cac4e47980 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbee01fd820 con 0x55cac4e2af90 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44508/0 (socket says 172.21.15.59:44508) 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.957+0000 7fbedfe00640 1 -- 172.21.15.59:0/2105815632 learned_addr learned my addr 172.21.15.59:0/2105815632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 -- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 msgr2=0x55cac4e43580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e43580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 -- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 msgr2=0x55cac4e44c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e44c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 -- 172.21.15.59:0/2105815632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac4e462c0 con 0x55cac4e2aaa0 2024-04-24T06:38:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbedfe00640 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x55cac4e168b0 tx=0x7fbed000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c6ed116ad34542 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbecbe00640 1 -- 172.21.15.59:0/2105815632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0026070 con 0x55cac4e2aaa0 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbecbe00640 1 -- 172.21.15.59:0/2105815632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed0021640 con 0x55cac4e2aaa0 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.958+0000 7fbecbe00640 1 -- 172.21.15.59:0/2105815632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed00160d0 con 0x55cac4e2aaa0 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 -- 172.21.15.59:0/2105815632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 msgr2=0x55cac4e43040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x55cac4e168b0 tx=0x7fbed000f000 comp rx=0 tx=0).stop 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 -- 172.21.15.59:0/2105815632 shutdown_connections 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e47980 0x55cac4e44c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e43580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/2105815632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e2aaa0 0x55cac4e43040 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 -- 172.21.15.59:0/2105815632 >> 172.21.15.59:0/2105815632 conn(0x55cac4e2a360 msgr2=0x55cac4df5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 -- 172.21.15.59:0/2105815632 shutdown_connections 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.959+0000 7fbee9c1b740 1 -- 172.21.15.59:0/2105815632 wait complete. 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.960+0000 7fbee9c1b740 1 -- start start 2024-04-24T06:38:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.960+0000 7fbee9c1b740 1 Processor -- start 2024-04-24T06:38:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e2aaa0 0x55cac4e23b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e68a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cac4d64b10 con 0x55cac4e47980 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cac4dfe730 con 0x55cac4e2aaa0 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cac4d1e590 con 0x55cac4e2af90 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee9c1b740 10 client.0 _ll_register_callbacks cb 0x7fbed80095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44520/0 (socket says 172.21.15.59:44520) 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.961+0000 7fbee4e00640 1 -- 172.21.15.59:0/65778528 learned_addr learned my addr 172.21.15.59:0/65778528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbedfe00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e2aaa0 0x55cac4e23b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 -- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 msgr2=0x55cac4e68a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e68a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 -- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e2aaa0 msgr2=0x55cac4e23b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e2aaa0 0x55cac4e23b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 -- 172.21.15.59:0/65778528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac4e243b0 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee4e00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034040 tx=0x7fbed80aa450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2a704691b9ebfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbec9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80824d0 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fbed80793d0 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed0021640 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbed807e130 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee9c1b740 10 client.8649 fetch_fsmap learned FSMap version 83 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.962+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55cac4e243b0 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed0021640 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fbed8082bb0 con 0x55cac4e47980 2024-04-24T06:38:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 10 client.8649 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:38:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 10 client.8649 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T06:38:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 20 client.8649 populate_metadata read hostname 'smithi059' 2024-04-24T06:38:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e43390 0x55cac4e26330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 4 client.8649 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:38:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "status"}) -- 0x55cac4d6c3a0 con 0x55cac4e43390 2024-04-24T06:38:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.963+0000 7fbedfe00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e43390 0x55cac4e26330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.964+0000 7fbedfe00640 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e43390 0x55cac4e26330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8552fb16027a40 server_cookie=65fa0cb46ec9ba86 in_seq=0 out_seq=0 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.964+0000 7fbec9600640 10 client.8649 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbec9600640 1 -- 172.21.15.59:0/65778528 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cac4d6c3a0 con 0x55cac4e43390 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbec9600640 10 client.8649 handle_command_reply: tid=0 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 client.8649 shutdown 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 msgr2=0x55cac4e3bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034040 tx=0x7fbed80aa450 comp rx=0 tx=0).stop 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 shutdown_connections 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55cac4e43390 0x55cac4e26330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cac4e47980 0x55cac4e3bb10 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cac4e2af90 0x55cac4e68a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 --2- 172.21.15.59:0/65778528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cac4e2aaa0 0x55cac4e23b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbec9600640 0 client.8649 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 >> 172.21.15.59:0/65778528 conn(0x55cac4e2a360 msgr2=0x55cac4dfdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 shutdown_connections 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.965+0000 7fbee9c1b740 1 -- 172.21.15.59:0/65778528 wait complete. 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee9c1b740 20 client.8649 trim_cache size 0 max 16384 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8499, 2024-04-24T06:38:45.970 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7997406710000003, 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 83, 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 98, 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 98, 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.156419572000001 2024-04-24T06:38:45.971 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 msgr2=0x7fbec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 0x7fbec006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000f60 tx=0x7fbed00079d0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 msgr2=0x7fbee01f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009f70 tx=0x7fbed8004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 shutdown_connections 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbec006c8f0 0x7fbec006edb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbee015aca0 0x7fbee01f6620 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbee00b45d0 0x7fbee01f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 --2- 172.21.15.59:0/21601900 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbee0071e70 0x7fbee015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.966+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 >> 172.21.15.59:0/21601900 conn(0x7fbee006bdf0 msgr2=0x7fbee0158290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.967+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 shutdown_connections 2024-04-24T06:38:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:45.967+0000 7fbee5800640 1 -- 172.21.15.59:0/21601900 wait complete. 2024-04-24T06:38:45.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:38:46.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.195+0000 7fa37d400640 1 Processor -- start 2024-04-24T06:38:46.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.196+0000 7fa37d400640 1 -- start start 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa37d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 0x7fa3780cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa37d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3780b3080 0x7fa3780cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa37d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3780b3870 con 0x7fa37815b450 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa37d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa3780b41a0 con 0x7fa3780b3080 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa37d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3780b3b50 con 0x7fa37806f3f0 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa377e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa37815b450 0x7fa3780b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:51240/0 (socket says 172.21.15.59:51240) 2024-04-24T06:38:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.197+0000 7fa377e00640 1 -- 172.21.15.59:0/1083203428 learned_addr learned my addr 172.21.15.59:0/1083203428 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:46.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.198+0000 7fa37ca00640 1 --2- 172.21.15.59:0/1083203428 >> v2:172.21.15.66:3300/0 conn(0x7fa3780b3080 0x7fa3780cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.198+0000 7fa377400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 0x7fa3780cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.198+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763823988 0 0) 0x7fa3780b3870 con 0x7fa37815b450 2024-04-24T06:38:46.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.198+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa358003520 con 0x7fa37815b450 2024-04-24T06:38:46.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.198+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1999232113 0 0) 0x7fa358003520 con 0x7fa37815b450 2024-04-24T06:38:46.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 msgr2=0x7fa3780cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:46.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 0x7fa3780cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 >> v2:172.21.15.66:3300/0 conn(0x7fa3780b3080 msgr2=0x7fa3780cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 --2- 172.21.15.59:0/1083203428 >> v2:172.21.15.66:3300/0 conn(0x7fa3780b3080 0x7fa3780cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3780b3e30 con 0x7fa37815b450 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa368001670 con 0x7fa37815b450 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 >> v1:172.21.15.59:6789/0 conn(0x7fa37815b450 legacy=0x7fa3780b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3580039b0 0x7fa358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004800 0x7fa358004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 0x7fa3580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780b41a0 con 0x7fa358004800 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780b3b50 con 0x7fa3580039b0 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3580099b0 con 0x7fa37806f3f0 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.199+0000 7fa377400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004800 0x7fa358004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.200+0000 7fa377400640 1 -- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 msgr2=0x7fa3580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.200+0000 7fa377400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37806f3f0 0x7fa3580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.200+0000 7fa377400640 1 -- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3580039b0 msgr2=0x7fa358003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.200+0000 7fa377400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3580039b0 0x7fa358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.200+0000 7fa377400640 1 -- 172.21.15.59:0/1083203428 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa358003520 con 0x7fa358004800 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.201+0000 7fa377400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004800 0x7fa358004c80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fc0 tx=0x7fa358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd06639a28d44d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.201+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360009c10 con 0x7fa358004800 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.201+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360036070 con 0x7fa358004800 2024-04-24T06:38:46.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.201+0000 7fa376a00640 1 -- 172.21.15.59:0/1083203428 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36002d070 con 0x7fa358004800 2024-04-24T06:38:46.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 -- 172.21.15.59:0/1083203428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004800 msgr2=0x7fa358004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 --2- 172.21.15.59:0/1083203428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004800 0x7fa358004c80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fc0 tx=0x7fa358005830 comp rx=0 tx=0).stop 2024-04-24T06:38:46.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37ca00640 1 -- 172.21.15.59:0/1083203428 reap_dead start 2024-04-24T06:38:46.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 -- 172.21.15.59:0/1083203428 shutdown_connections 2024-04-24T06:38:46.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 -- 172.21.15.59:0/1083203428 >> 172.21.15.59:0/1083203428 conn(0x7fa3780b21a0 msgr2=0x7fa3780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:46.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 -- 172.21.15.59:0/1083203428 shutdown_connections 2024-04-24T06:38:46.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.202+0000 7fa37d400640 1 -- 172.21.15.59:0/1083203428 wait complete. 2024-04-24T06:38:46.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.204+0000 7fa37d400640 1 Processor -- start 2024-04-24T06:38:46.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.204+0000 7fa37d400640 1 -- start start 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37809cad0 0x7fa3780c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 0x7fa3780c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780cd050 con 0x7fa3780c5ed0 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780b4ff0 con 0x7fa3780c5530 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3780cd1d0 con 0x7fa37809cad0 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44544/0 (socket says 172.21.15.59:44544) 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.205+0000 7fa37ca00640 1 -- 172.21.15.59:0/3518072064 learned_addr learned my addr 172.21.15.59:0/3518072064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa377e00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37809cad0 0x7fa3780c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa377400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 0x7fa3780c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 -- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37809cad0 msgr2=0x7fa3780c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37809cad0 0x7fa3780c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 -- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 msgr2=0x7fa3780c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 0x7fa3780c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 -- 172.21.15.59:0/3518072064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37809dae0 con 0x7fa3780c5ed0 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa377400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 0x7fa3780c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:46.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa37ca00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa37806d900 tx=0x7fa37000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000c680 con 0x7fa3780c5ed0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.206+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37001a070 con 0x7fa3780c5ed0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.207+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3780b15a0 con 0x7fa3780c5ed0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.207+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000ca60 con 0x7fa3780c5ed0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.207+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3780c6690 con 0x7fa3780c5ed0 2024-04-24T06:38:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.208+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37804fd90 con 0x7fa3780c5ed0 2024-04-24T06:38:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.209+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa37000cc00 con 0x7fa3780c5ed0 2024-04-24T06:38:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.210+0000 7fa36fe00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 0x7fa35406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.210+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fa37008b720 con 0x7fa3780c5ed0 2024-04-24T06:38:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.210+0000 7fa377e00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 0x7fa35406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.211+0000 7fa377e00640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 0x7fa35406edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa368005ab0 tx=0x7fa368001510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:46.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.214+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa37809dae0 con 0x7fa3780c5ed0 2024-04-24T06:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.459+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa378158aa0 con 0x7fa3780c5ed0 2024-04-24T06:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.460+0000 7fa36fe00640 1 -- 172.21.15.59:0/3518072064 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7fa370059a20 con 0x7fa3780c5ed0 2024-04-24T06:38:46.462 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:38:46.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.462+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 msgr2=0x7fa35406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.462+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 0x7fa35406edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa368005ab0 tx=0x7fa368001510 comp rx=0 tx=0).stop 2024-04-24T06:38:46.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.462+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 msgr2=0x7fa3780afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:46.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.462+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa37806d900 tx=0x7fa37000ec00 comp rx=0 tx=0).stop 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 shutdown_connections 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa35406c8f0 0x7fa35406edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3780c5ed0 0x7fa3780afb80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3780c5530 0x7fa3780c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 --2- 172.21.15.59:0/3518072064 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa37809cad0 0x7fa3780c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 >> 172.21.15.59:0/3518072064 conn(0x7fa3780b21a0 msgr2=0x7fa37806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 shutdown_connections 2024-04-24T06:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46.463+0000 7fa37d400640 1 -- 172.21.15.59:0/3518072064 wait complete. 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:38:46.484 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:38:46.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:46.485 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:38:46.539 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.540 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.541 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:38:46.542 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:46.542 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:38:46.542 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:38:46.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:46.543 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:38:46.621 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:38:46.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:46.626 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:38:46.684 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:38:46.684 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:38:46.684 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:38:46.684 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:38:46.684 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:38:46.685 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:38:46.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:38:46.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:38:46.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:46.908 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:38:46.960 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:38:46.960 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:38:46.961 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:38:46.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:46.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:38:47.049 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:38:47.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:38:47.049 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:47.104 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:38:47.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:38:47.104 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:47.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:38:47.242 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:38:47.297 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:38:47.298 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:38:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:38:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:47.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:47.300 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:38:47.351 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:38:47.351 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:38:47.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:38:47.430 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:38:47.477 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:38:47.477 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:38:47.478 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:38:47.479 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:38:47.480 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:38:47.480 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:47.480 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:47.480 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:47.480 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:47.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:47.481 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:38:47.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:38:47.535+0000 7f79c7761480 -1 init, newargv = 0x564e1362b4a0 newargc=15 2024-04-24T06:38:47.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[69762]: starting ceph client 2024-04-24T06:38:47.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[69762]: starting fuse 2024-04-24T06:38:48.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:38:48.721 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:38:48.780 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:38:48.781 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:38:48.782 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:38:48.783 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:48.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:48.784 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:38:48.839 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:38:48.840 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:38:48.840 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:38:48.841 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:38:48.842 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:38:48.842 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:38:48.842 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:38:48.842 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:38:49.017 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.69762.asok 2024-04-24T06:38:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:38:49.036 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69762.asok 2024-04-24T06:38:49.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:49.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69762.asok status 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "69762", 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:49.273 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8658, 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8658 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4002038260 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:38:49.274 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4002038260 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8658 192.168.0.1:0/4002038260", 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/4002038260", 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 83, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 98, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:38:49.275 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:38:49.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:49.289 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:38:49.289 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:49.345 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:38:49.345 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:49.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:49.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:49.436 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:49.436 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:49.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:49.492 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-24T06:38:49.749 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-24T06:38:49.749 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-24T06:38:49.749 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.18872 s, 33.3 MB/s 2024-04-24T06:38:49.752 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:49.752 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:49.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:50.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.018+0000 7f3a32c00640 1 Processor -- start 2024-04-24T06:38:50.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.018+0000 7f3a32c00640 1 -- start start 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a2c06f7c0 0x7f3a2c0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c15b840 0x7f3a2c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a2c06df00 0x7f3a2c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2c0700e0 con 0x7f3a2c06df00 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2c06ec60 con 0x7f3a2c06f7c0 2024-04-24T06:38:50.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a32c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2c0b46e0 con 0x7f3a2c15b840 2024-04-24T06:38:50.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a30c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a2c06df00 0x7f3a2c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.020+0000 7f3a30c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a2c06df00 0x7f3a2c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38048/0 (socket says 172.21.15.59:38048) 2024-04-24T06:38:50.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 -- 172.21.15.59:0/3770884609 learned_addr learned my addr 172.21.15.59:0/3770884609 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a2b400640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c15b840 0x7f3a2c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a2be00640 1 --2- 172.21.15.59:0/3770884609 >> v2:172.21.15.66:3300/0 conn(0x7f3a2c06f7c0 0x7f3a2c0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 -- 172.21.15.59:0/3770884609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c15b840 msgr2=0x7f3a2c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c15b840 0x7f3a2c06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 -- 172.21.15.59:0/3770884609 >> v2:172.21.15.66:3300/0 conn(0x7f3a2c06f7c0 msgr2=0x7f3a2c0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 --2- 172.21.15.59:0/3770884609 >> v2:172.21.15.66:3300/0 conn(0x7f3a2c06f7c0 0x7f3a2c0b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.021+0000 7f3a30c00640 1 -- 172.21.15.59:0/3770884609 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c0b49c0 con 0x7f3a2c06df00 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.022+0000 7f3a30c00640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c06df00 0x7f3a2c06e390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06c700 tx=0x7f3a2402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41705cff6a163f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.023+0000 7f3a2aa00640 1 -- 172.21.15.59:0/3770884609 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400b2e0 con 0x7f3a2c06df00 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.023+0000 7f3a2aa00640 1 -- 172.21.15.59:0/3770884609 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24009600 con 0x7f3a2c06df00 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.023+0000 7f3a2aa00640 1 -- 172.21.15.59:0/3770884609 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400b480 con 0x7f3a2c06df00 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 -- 172.21.15.59:0/3770884609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c06df00 msgr2=0x7f3a2c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c06df00 0x7f3a2c06e390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06c700 tx=0x7f3a2402c8a0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 -- 172.21.15.59:0/3770884609 shutdown_connections 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c06df00 0x7f3a2c06e390 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 --2- 172.21.15.59:0/3770884609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c15b840 0x7f3a2c06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 --2- 172.21.15.59:0/3770884609 >> v2:172.21.15.66:3300/0 conn(0x7f3a2c06f7c0 0x7f3a2c0b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 -- 172.21.15.59:0/3770884609 >> 172.21.15.59:0/3770884609 conn(0x7f3a2c067ea0 msgr2=0x7f3a2c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 -- 172.21.15.59:0/3770884609 shutdown_connections 2024-04-24T06:38:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.024+0000 7f3a32c00640 1 -- 172.21.15.59:0/3770884609 wait complete. 2024-04-24T06:38:50.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.026+0000 7f3a32c00640 1 Processor -- start 2024-04-24T06:38:50.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.026+0000 7f3a32c00640 1 -- start start 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 0x7f3a2c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a2c06f7c0 0x7f3a2c1f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c06e8d0 con 0x7f3a2c15db10 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c0b4f80 con 0x7f3a2c06f7c0 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a32c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2c0ce1c0 con 0x7f3a2c06df00 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a30c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a30c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38060/0 (socket says 172.21.15.59:38060) 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.027+0000 7f3a30c00640 1 -- 172.21.15.59:0/4077844492 learned_addr learned my addr 172.21.15.59:0/4077844492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:50.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a2be00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 0x7f3a2c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a2b400640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a2c06f7c0 0x7f3a2c1f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 -- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 msgr2=0x7f3a2c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 0x7f3a2c14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 -- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a2c06f7c0 msgr2=0x7f3a2c1f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a2c06f7c0 0x7f3a2c1f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 -- 172.21.15.59:0/4077844492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c0adac0 con 0x7f3a2c15db10 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a2be00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 0x7f3a2c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:50.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.028+0000 7f3a30c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402ce90 tx=0x7f3a24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.029+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24040030 con 0x7f3a2c15db10 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.029+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2403c640 con 0x7f3a2c15db10 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.029+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2403ca10 con 0x7f3a2c15db10 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.029+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c0adda0 con 0x7f3a2c15db10 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.029+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c0ae360 con 0x7f3a2c15db10 2024-04-24T06:38:50.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.030+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c068a80 con 0x7f3a2c15db10 2024-04-24T06:38:50.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.031+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a24043030 con 0x7f3a2c15db10 2024-04-24T06:38:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.032+0000 7f3a23e00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 0x7f3a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.032+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f3a24030020 con 0x7f3a2c15db10 2024-04-24T06:38:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.032+0000 7f3a2be00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 0x7f3a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.033+0000 7f3a2be00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 0x7f3a0c06edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c157950 tx=0x7f3a1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.037+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a2404a030 con 0x7f3a2c15db10 2024-04-24T06:38:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.241+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a2c0ae530 con 0x7f3a2c15db10 2024-04-24T06:38:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.243+0000 7f3a23e00640 1 -- 172.21.15.59:0/4077844492 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16524 (secure 0 0 0) 0x7f3a2407ac10 con 0x7f3a2c15db10 2024-04-24T06:38:50.246 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:50.246 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:40.185327+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 msgr2=0x7f3a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 0x7f3a0c06edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c157950 tx=0x7f3a1c008040 comp rx=0 tx=0).stop 2024-04-24T06:38:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 msgr2=0x7f3a2c1561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402ce90 tx=0x7f3a24004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 shutdown_connections 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a0c06c8f0 0x7f3a0c06edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a2c15db10 0x7f3a2c1561c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a2c06f7c0 0x7f3a2c1f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 --2- 172.21.15.59:0/4077844492 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a2c06df00 0x7f3a2c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 >> 172.21.15.59:0/4077844492 conn(0x7f3a2c067ea0 msgr2=0x7f3a2c0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.246+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 shutdown_connections 2024-04-24T06:38:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.247+0000 7f3a32c00640 1 -- 172.21.15.59:0/4077844492 wait complete. 2024-04-24T06:38:50.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:38:36.464123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:38:37.616685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:50.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.477+0000 7fe33d400640 1 Processor -- start 2024-04-24T06:38:50.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.477+0000 7fe33d400640 1 -- start start 2024-04-24T06:38:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.479+0000 7fe33d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe33815b750 0x7fe3380cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.479+0000 7fe33d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.479+0000 7fe33d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3380b2f00 con 0x7fe3380b1b90 2024-04-24T06:38:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.479+0000 7fe33d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380d02a0 con 0x7fe33815b750 2024-04-24T06:38:50.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.479+0000 7fe33d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380b2380 con 0x7fe3380cfe40 2024-04-24T06:38:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.480+0000 7fe32de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.480+0000 7fe32de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:56760/0 (socket says 172.21.15.59:56760) 2024-04-24T06:38:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.480+0000 7fe32de00640 1 -- 172.21.15.59:0/2901123800 learned_addr learned my addr 172.21.15.59:0/2901123800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.480+0000 7fe336800640 1 --2- 172.21.15.59:0/2901123800 >> v2:172.21.15.66:3300/0 conn(0x7fe33815b750 0x7fe3380cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.481+0000 7fe32de00640 1 -- 172.21.15.59:0/2901123800 >> v2:172.21.15.66:3300/0 conn(0x7fe33815b750 msgr2=0x7fe3380cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.481+0000 7fe32de00640 1 --2- 172.21.15.59:0/2901123800 >> v2:172.21.15.66:3300/0 conn(0x7fe33815b750 0x7fe3380cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.481+0000 7fe32de00640 1 -- 172.21.15.59:0/2901123800 >> v1:172.21.15.66:6789/0 conn(0x7fe3380b1b90 legacy=0x7fe33815f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.481+0000 7fe32de00640 1 -- 172.21.15.59:0/2901123800 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380b2660 con 0x7fe3380cfe40 2024-04-24T06:38:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.483+0000 7fe32de00640 1 --2- 172.21.15.59:0/2901123800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe32000bb10 tx=0x7fe32002c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cf9124c7495cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe335e00640 1 -- 172.21.15.59:0/2901123800 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320032790 con 0x7fe3380cfe40 2024-04-24T06:38:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe335e00640 1 -- 172.21.15.59:0/2901123800 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe320038070 con 0x7fe3380cfe40 2024-04-24T06:38:50.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe335e00640 1 -- 172.21.15.59:0/2901123800 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320032b50 con 0x7fe3380cfe40 2024-04-24T06:38:50.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 -- 172.21.15.59:0/2901123800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 msgr2=0x7fe3380b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 --2- 172.21.15.59:0/2901123800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe32000bb10 tx=0x7fe32002c890 comp rx=0 tx=0).stop 2024-04-24T06:38:50.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 -- 172.21.15.59:0/2901123800 shutdown_connections 2024-04-24T06:38:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 --2- 172.21.15.59:0/2901123800 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380cfe40 0x7fe3380b1650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 --2- 172.21.15.59:0/2901123800 >> v2:172.21.15.66:3300/0 conn(0x7fe33815b750 0x7fe3380cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.484+0000 7fe33d400640 1 -- 172.21.15.59:0/2901123800 >> 172.21.15.59:0/2901123800 conn(0x7fe338073450 msgr2=0x7fe33809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.485+0000 7fe33d400640 1 -- 172.21.15.59:0/2901123800 shutdown_connections 2024-04-24T06:38:50.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.485+0000 7fe33d400640 1 -- 172.21.15.59:0/2901123800 wait complete. 2024-04-24T06:38:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.486+0000 7fe33d400640 1 Processor -- start 2024-04-24T06:38:50.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.487+0000 7fe33d400640 1 -- start start 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.487+0000 7fe33d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380b1b90 0x7fe338156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe33d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe33d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe33815b750 0x7fe33814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe33d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe33815bb30 con 0x7fe3380cfe40 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe33d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380b2c20 con 0x7fe33815b750 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe33d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380b2020 con 0x7fe3380b1b90 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe32de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe336800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380b1b90 0x7fe338156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe32de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38062/0 (socket says 172.21.15.59:38062) 2024-04-24T06:38:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe32de00640 1 -- 172.21.15.59:0/2845815216 learned_addr learned my addr 172.21.15.59:0/2845815216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:50.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.488+0000 7fe337200640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe33815b750 0x7fe33814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 -- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380b1b90 msgr2=0x7fe338156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380b1b90 0x7fe338156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 -- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe33815b750 msgr2=0x7fe33814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe33815b750 0x7fe33814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 -- 172.21.15.59:0/2845815216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380c96b0 con 0x7fe3380cfe40 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32de00640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.489+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320043070 con 0x7fe3380cfe40 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.490+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32003c660 con 0x7fe3380cfe40 2024-04-24T06:38:50.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.490+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32003ca50 con 0x7fe3380cfe40 2024-04-24T06:38:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.490+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381569a0 con 0x7fe3380cfe40 2024-04-24T06:38:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.490+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3380c9e80 con 0x7fe3380cfe40 2024-04-24T06:38:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.491+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe338068a80 con 0x7fe3380cfe40 2024-04-24T06:38:50.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.492+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe320046070 con 0x7fe3380cfe40 2024-04-24T06:38:50.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.493+0000 7fe32f400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.493+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fe3200ac0b0 con 0x7fe3380cfe40 2024-04-24T06:38:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.493+0000 7fe336800640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.494+0000 7fe336800640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 0x7fe31406edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe328009b40 tx=0x7fe328009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.497+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3380c96b0 con 0x7fe3380cfe40 2024-04-24T06:38:50.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.700+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe33815a520 con 0x7fe3380cfe40 2024-04-24T06:38:50.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.700+0000 7fe32f400640 1 -- 172.21.15.59:0/2845815216 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fe320079df0 con 0x7fe3380cfe40 2024-04-24T06:38:50.702 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:50.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 msgr2=0x7fe31406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 0x7fe31406edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe328009b40 tx=0x7fe328009190 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 msgr2=0x7fe3380c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe320004730 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 shutdown_connections 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe31406c8f0 0x7fe31406edb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe33815b750 0x7fe33814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3380cfe40 0x7fe3380c9170 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 --2- 172.21.15.59:0/2845815216 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3380b1b90 0x7fe338156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 >> 172.21.15.59:0/2845815216 conn(0x7fe338073450 msgr2=0x7fe3380708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 shutdown_connections 2024-04-24T06:38:50.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.703+0000 7fe33d400640 1 -- 172.21.15.59:0/2845815216 wait complete. 2024-04-24T06:38:50.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:38:50.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.928+0000 7f29d0e00640 1 Processor -- start 2024-04-24T06:38:50.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.928+0000 7f29d0e00640 1 -- start start 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29d0e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f29cc0d0d10 0x7f29cc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29d0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 0x7f29cc160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29d0e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29cc15cab0 con 0x7f29cc1567e0 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29d0e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f29cc160e40 con 0x7f29cc0d0d10 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29d0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc15cd30 con 0x7f29cc15d810 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29cb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f29cc1567e0 0x7f29cc0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:38626/0 (socket says 172.21.15.59:38626) 2024-04-24T06:38:50.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29cb400640 1 -- 172.21.15.59:0/30071288 learned_addr learned my addr 172.21.15.59:0/30071288 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:50.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29cbe00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 0x7f29cc160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.930+0000 7f29caa00640 1 --2- 172.21.15.59:0/30071288 >> v2:172.21.15.66:3300/0 conn(0x7f29cc0d0d10 0x7f29cc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302622011 0 0) 0x7f29bc001110 con 0x7f29cc1567e0 2024-04-24T06:38:50.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29b0003520 con 0x7f29cc1567e0 2024-04-24T06:38:50.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4198434043 0 0) 0x7f29b0003520 con 0x7f29cc1567e0 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 msgr2=0x7f29cc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 0x7f29cc160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 >> v2:172.21.15.66:3300/0 conn(0x7f29cc0d0d10 msgr2=0x7f29cc156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 --2- 172.21.15.59:0/30071288 >> v2:172.21.15.66:3300/0 conn(0x7f29cc0d0d10 0x7f29cc156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc15d010 con 0x7f29cc1567e0 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f29bc001840 con 0x7f29cc1567e0 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 >> v1:172.21.15.59:6789/0 conn(0x7f29cc1567e0 legacy=0x7f29cc0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00039b0 0x7f29b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 0x7f29b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc160e40 con 0x7f29b0004d40 2024-04-24T06:38:50.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc15cd30 con 0x7f29b00039b0 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.931+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29b000b9c0 con 0x7f29cc15d810 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cb400640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00039b0 0x7f29b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 -- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 msgr2=0x7f29b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc15d810 0x7f29b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 -- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00039b0 msgr2=0x7f29b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29b00039b0 0x7f29b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.932+0000 7f29cbe00640 1 -- 172.21.15.59:0/30071288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b0003520 con 0x7f29b0004d40 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.933+0000 7f29cbe00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29b00051c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fc0 tx=0x7f29b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b0819fcf5b6095 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.933+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4009c10 con 0x7f29b0004d40 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.933+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c4036070 con 0x7f29b0004d40 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.933+0000 7f29ca000640 1 -- 172.21.15.59:0/30071288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c402d070 con 0x7f29b0004d40 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 -- 172.21.15.59:0/30071288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 msgr2=0x7f29b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 --2- 172.21.15.59:0/30071288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29b00051c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009fc0 tx=0x7f29b0005830 comp rx=0 tx=0).stop 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29cbe00640 1 -- 172.21.15.59:0/30071288 reap_dead start 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 -- 172.21.15.59:0/30071288 shutdown_connections 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 -- 172.21.15.59:0/30071288 >> 172.21.15.59:0/30071288 conn(0x7f29cc06bdf0 msgr2=0x7f29cc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 -- 172.21.15.59:0/30071288 shutdown_connections 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.934+0000 7f29d0e00640 1 -- 172.21.15.59:0/30071288 wait complete. 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.936+0000 7f29d0e00640 1 Processor -- start 2024-04-24T06:38:50.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.936+0000 7f29d0e00640 1 -- start start 2024-04-24T06:38:50.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29cc0b0f60 0x7f29cc0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc151bc0 0x7f29cc152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc160290 con 0x7f29b0004d40 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc160110 con 0x7f29cc0b0f60 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.937+0000 7f29d0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc160410 con 0x7f29cc151bc0 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38090/0 (socket says 172.21.15.59:38090) 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 -- 172.21.15.59:0/3703180308 learned_addr learned my addr 172.21.15.59:0/3703180308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cbe00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc151bc0 0x7f29cc152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 -- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc151bc0 msgr2=0x7f29cc152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29caa00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29cc0b0f60 0x7f29cc0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc151bc0 0x7f29cc152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 -- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29cc0b0f60 msgr2=0x7f29cc0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29cc0b0f60 0x7f29cc0b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 -- 172.21.15.59:0/3703180308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc1f2620 con 0x7f29b0004d40 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.938+0000 7f29cb400640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f29bc001050 tx=0x7f29bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.939+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00c8d0 con 0x7f29b0004d40 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.939+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29bc018070 con 0x7f29b0004d40 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.939+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00ccb0 con 0x7f29b0004d40 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.939+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc1f2900 con 0x7f29b0004d40 2024-04-24T06:38:50.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.939+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc1f2ec0 con 0x7f29b0004d40 2024-04-24T06:38:50.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.940+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc04fd90 con 0x7f29b0004d40 2024-04-24T06:38:50.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.941+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29bc01d460 con 0x7f29b0004d40 2024-04-24T06:38:50.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.942+0000 7f29c3400640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 0x7f29ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:50.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.942+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f29bc08bb70 con 0x7f29b0004d40 2024-04-24T06:38:50.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.942+0000 7f29caa00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 0x7f29ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:50.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.943+0000 7f29caa00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 0x7f29ac06ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004770 tx=0x7f29b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:50.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:50.947+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29cc15cab0 con 0x7f29b0004d40 2024-04-24T06:38:51.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.149+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29cc0b5150 con 0x7f29b0004d40 2024-04-24T06:38:51.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.149+0000 7f29c3400640 1 -- 172.21.15.59:0/3703180308 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f29bc059f00 con 0x7f29b0004d40 2024-04-24T06:38:51.151 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:38:51.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 msgr2=0x7f29ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 0x7f29ac06ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004770 tx=0x7f29b8008040 comp rx=0 tx=0).stop 2024-04-24T06:38:51.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 msgr2=0x7f29cc1e8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f29bc001050 tx=0x7f29bc00ec80 comp rx=0 tx=0).stop 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 shutdown_connections 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f29ac06c920 0x7f29ac06ede0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f29cc151bc0 0x7f29cc152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29cc0b0f60 0x7f29cc0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 --2- 172.21.15.59:0/3703180308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29b0004d40 0x7f29cc1e8740 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 >> 172.21.15.59:0/3703180308 conn(0x7f29cc06bdf0 msgr2=0x7f29cc0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 shutdown_connections 2024-04-24T06:38:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.152+0000 7f29d0e00640 1 -- 172.21.15.59:0/3703180308 wait complete. 2024-04-24T06:38:51.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:38:51.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.384+0000 7f001ae00640 1 Processor -- start 2024-04-24T06:38:51.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.384+0000 7f001ae00640 1 -- start start 2024-04-24T06:38:51.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.385+0000 7f001ae00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0014162c10 0x7f00140d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f001ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f001ae00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00140d0fd0 con 0x7f00140b4670 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f001ae00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f00140b35d0 con 0x7f0014162c10 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f001ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f00140b38b0 con 0x7f0014072480 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f0018e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f0013e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0014162c10 0x7f00140d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f000b400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f00140b4670 0x7f0014071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:33158/0 (socket says 172.21.15.59:33158) 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.386+0000 7f000b400640 1 -- 172.21.15.59:0/254590224 learned_addr learned my addr 172.21.15.59:0/254590224 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0013400640 1 -- 172.21.15.59:0/254590224 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600241718 0 0) 0x7efffc001110 con 0x7f00140b4670 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0013e00640 1 -- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 msgr2=0x7f001406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0013e00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0013e00640 1 -- 172.21.15.59:0/254590224 >> v1:172.21.15.66:6789/0 conn(0x7f00140b4670 legacy=0x7f0014071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0013e00640 1 -- 172.21.15.59:0/254590224 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00140b3b90 con 0x7f0014162c10 2024-04-24T06:38:51.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.387+0000 7f0018e00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:38:51.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.389+0000 7f0013e00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140d0a90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f00040097a0 tx=0x7f000402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c3b257ccd4f07e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.389+0000 7f0013400640 1 -- 172.21.15.59:0/254590224 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004037070 con 0x7f0014162c10 2024-04-24T06:38:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.389+0000 7f0013400640 1 -- 172.21.15.59:0/254590224 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0004032440 con 0x7f0014162c10 2024-04-24T06:38:51.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.389+0000 7f0013400640 1 -- 172.21.15.59:0/254590224 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004032810 con 0x7f0014162c10 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 -- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 msgr2=0x7f00140d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140d0a90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f00040097a0 tx=0x7f000402a180 comp rx=0 tx=0).stop 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 -- 172.21.15.59:0/254590224 shutdown_connections 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 --2- 172.21.15.59:0/254590224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140d0a90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 -- 172.21.15.59:0/254590224 >> 172.21.15.59:0/254590224 conn(0x7f0014067d90 msgr2=0x7f001406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 -- 172.21.15.59:0/254590224 shutdown_connections 2024-04-24T06:38:51.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.390+0000 7f001ae00640 1 -- 172.21.15.59:0/254590224 wait complete. 2024-04-24T06:38:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.392+0000 7f001ae00640 1 Processor -- start 2024-04-24T06:38:51.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.392+0000 7f001ae00640 1 -- start start 2024-04-24T06:38:51.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0014162ff0 con 0x7f00140b4670 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00140b4150 con 0x7f0014162c10 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f001ae00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0014163170 con 0x7f0014072480 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f000b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f000b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38106/0 (socket says 172.21.15.59:38106) 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f000b400640 1 -- 172.21.15.59:0/3601700444 learned_addr learned my addr 172.21.15.59:0/3601700444 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f0013e00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.394+0000 7f0018e00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 -- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 msgr2=0x7f001415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001415d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 -- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 msgr2=0x7f00140aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 -- 172.21.15.59:0/3601700444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00140aaad0 con 0x7f00140b4670 2024-04-24T06:38:51.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f000b400640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c750 tx=0x7efffc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.395+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc018070 con 0x7f00140b4670 2024-04-24T06:38:51.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.396+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00140aa510 con 0x7f00140b4670 2024-04-24T06:38:51.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.396+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efffc0072d0 con 0x7f00140b4670 2024-04-24T06:38:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.396+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc0047e0 con 0x7f00140b4670 2024-04-24T06:38:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.396+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00140ae530 con 0x7f00140b4670 2024-04-24T06:38:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.397+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0014068a80 con 0x7f00140b4670 2024-04-24T06:38:51.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.398+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efffc01b070 con 0x7f00140b4670 2024-04-24T06:38:51.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.399+0000 7f0010c00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.399+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7efffc08ae00 con 0x7f00140b4670 2024-04-24T06:38:51.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.400+0000 7f0013e00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.400+0000 7f0013e00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 0x7efff006edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0004034000 tx=0x7f000402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.406+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00140d0fd0 con 0x7f00140b4670 2024-04-24T06:38:51.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.612+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f001406f1b0 con 0x7f00140b4670 2024-04-24T06:38:51.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.613+0000 7f0010c00640 1 -- 172.21.15.59:0/3601700444 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7efffc058bd0 con 0x7f00140b4670 2024-04-24T06:38:51.615 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.615+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 msgr2=0x7efff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.615+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 0x7efff006edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0004034000 tx=0x7f000402e000 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.615+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 msgr2=0x7f00140aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.615+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c750 tx=0x7efffc00cc10 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 shutdown_connections 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efff006c8f0 0x7efff006edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0014162c10 0x7f00140aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00140b4670 0x7f00140aed30 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 --2- 172.21.15.59:0/3601700444 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0014072480 0x7f001415d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 >> 172.21.15.59:0/3601700444 conn(0x7f0014067d90 msgr2=0x7f00140cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 shutdown_connections 2024-04-24T06:38:51.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.616+0000 7f001ae00640 1 -- 172.21.15.59:0/3601700444 wait complete. 2024-04-24T06:38:51.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:38:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.846+0000 7f618b200640 1 Processor -- start 2024-04-24T06:38:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.846+0000 7f618b200640 1 -- start start 2024-04-24T06:38:51.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.847+0000 7f618b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f618c0cd8b0 0x7f618c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.847+0000 7f618b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f618c0d0b00 0x7f618c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f618b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618c0b4cf0 con 0x7f618c15c670 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f618b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f618c15ca50 con 0x7f618c0cd8b0 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f618b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f618c0ccb80 con 0x7f618c0d0b00 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f618c0cd8b0 0x7f618c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f618c0cd8b0 0x7f618c0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38118/0 (socket says 172.21.15.59:38118) 2024-04-24T06:38:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 -- 172.21.15.59:0/1051544355 learned_addr learned my addr 172.21.15.59:0/1051544355 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 -- 172.21.15.59:0/1051544355 >> v2:172.21.15.66:3300/0 conn(0x7f618c0d0b00 msgr2=0x7f618c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 --2- 172.21.15.59:0/1051544355 >> v2:172.21.15.66:3300/0 conn(0x7f618c0d0b00 0x7f618c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 -- 172.21.15.59:0/1051544355 >> v1:172.21.15.66:6789/0 conn(0x7f618c15c670 legacy=0x7f618c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:51.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.848+0000 7f6181400640 1 -- 172.21.15.59:0/1051544355 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c0ccdd0 con 0x7f618c0cd8b0 2024-04-24T06:38:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f6181400640 1 --2- 172.21.15.59:0/1051544355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0cd8b0 0x7f618c0d05c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f618c06c970 tx=0x7f617402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5596cdf0ec65af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f6189400640 1 -- 172.21.15.59:0/1051544355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174036070 con 0x7f618c0cd8b0 2024-04-24T06:38:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f6189400640 1 -- 172.21.15.59:0/1051544355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174031600 con 0x7f618c0cd8b0 2024-04-24T06:38:51.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f6189400640 1 -- 172.21.15.59:0/1051544355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61740319d0 con 0x7f618c0cd8b0 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f618b200640 1 -- 172.21.15.59:0/1051544355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0cd8b0 msgr2=0x7f618c0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.850+0000 7f618b200640 1 --2- 172.21.15.59:0/1051544355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0cd8b0 0x7f618c0d05c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f618c06c970 tx=0x7f617402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 -- 172.21.15.59:0/1051544355 shutdown_connections 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 --2- 172.21.15.59:0/1051544355 >> v2:172.21.15.66:3300/0 conn(0x7f618c0d0b00 0x7f618c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 --2- 172.21.15.59:0/1051544355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0cd8b0 0x7f618c0d05c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 -- 172.21.15.59:0/1051544355 >> 172.21.15.59:0/1051544355 conn(0x7f618c06bdf0 msgr2=0x7f618c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 -- 172.21.15.59:0/1051544355 shutdown_connections 2024-04-24T06:38:51.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.851+0000 7f618b200640 1 -- 172.21.15.59:0/1051544355 wait complete. 2024-04-24T06:38:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.852+0000 7f618b200640 1 Processor -- start 2024-04-24T06:38:51.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.853+0000 7f618b200640 1 -- start start 2024-04-24T06:38:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.853+0000 7f618b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 0x7f618c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f618b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f618b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c15c670 0x7f618c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f618b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f618c0b3e60 con 0x7f618c0d0b00 2024-04-24T06:38:51.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f618b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f618c0cd390 con 0x7f618c15c670 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f618b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f618c0b3fe0 con 0x7f618c0cd8b0 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38128/0 (socket says 172.21.15.59:38128) 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 -- 172.21.15.59:0/3925736621 learned_addr learned my addr 172.21.15.59:0/3925736621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6189e00640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 0x7f618c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 -- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 msgr2=0x7f618c1509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 0x7f618c1509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 -- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c15c670 msgr2=0x7f618c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c15c670 0x7f618c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6181400640 1 -- 172.21.15.59:0/3925736621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c0af920 con 0x7f618c0d0b00 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.854+0000 7f6189e00640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 0x7f618c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f6181400640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61740093e0 tx=0x7f6174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174041020 con 0x7f618c0d0b00 2024-04-24T06:38:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6174007800 con 0x7f618c0d0b00 2024-04-24T06:38:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174036040 con 0x7f618c0d0b00 2024-04-24T06:38:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c1512f0 con 0x7f618c0d0b00 2024-04-24T06:38:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.855+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c14e3b0 con 0x7f618c0d0b00 2024-04-24T06:38:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.857+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c04fd90 con 0x7f618c0d0b00 2024-04-24T06:38:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.857+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6174044020 con 0x7f618c0d0b00 2024-04-24T06:38:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.858+0000 7f6182a00640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 0x7f616806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.858+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f61740acab0 con 0x7f618c0d0b00 2024-04-24T06:38:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.859+0000 7f6189e00640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 0x7f616806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:51.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.859+0000 7f6189e00640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 0x7f616806edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f617c00a810 tx=0x7f617c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:51.863+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f617404b030 con 0x7f618c0d0b00 2024-04-24T06:38:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.078+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f618c0ceab0 con 0x7f618c0d0b00 2024-04-24T06:38:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.079+0000 7f6182a00640 1 -- 172.21.15.59:0/3925736621 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) ==== 80+0+0 (secure 0 0 0) 0x7f618c0af920 con 0x7f618c0d0b00 2024-04-24T06:38:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 msgr2=0x7f616806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 0x7f616806edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f617c00a810 tx=0x7f617c008040 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 msgr2=0x7f618c14da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61740093e0 tx=0x7f6174004a40 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 shutdown_connections 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f616806c8f0 0x7f616806edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f618c15c670 0x7f618c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f618c0d0b00 0x7f618c14da30 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 --2- 172.21.15.59:0/3925736621 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f618c0cd8b0 0x7f618c1509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 >> 172.21.15.59:0/3925736621 conn(0x7f618c06bdf0 msgr2=0x7f618c15ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:52.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 shutdown_connections 2024-04-24T06:38:52.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.082+0000 7f618b200640 1 -- 172.21.15.59:0/3925736621 wait complete. 2024-04-24T06:38:52.103 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:38:52.103 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:38:52.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:38:52.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.370+0000 7fb34f200640 1 Processor -- start 2024-04-24T06:38:52.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.370+0000 7fb34f200640 1 -- start start 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.371+0000 7fb34f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb3500cd760 0x7fb3500b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.371+0000 7fb34f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 0x7fb3500cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.371+0000 7fb34f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb350160210 con 0x7fb35015c3d0 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.371+0000 7fb34f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb3501604f0 con 0x7fb3500cd760 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.371+0000 7fb34f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb3501607d0 con 0x7fb3500b4690 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb3500cd760 0x7fb3500b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb3500cd760 0x7fb3500b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38140/0 (socket says 172.21.15.59:38140) 2024-04-24T06:38:52.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 -- 172.21.15.59:0/3237370176 learned_addr learned my addr 172.21.15.59:0/3237370176 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb34e800640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 0x7fb3500cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 -- 172.21.15.59:0/3237370176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 msgr2=0x7fb3500cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 0x7fb3500cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 -- 172.21.15.59:0/3237370176 >> v1:172.21.15.66:6789/0 conn(0x7fb35015c3d0 legacy=0x7fb3500cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb345400640 1 -- 172.21.15.59:0/3237370176 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350160ab0 con 0x7fb3500cd760 2024-04-24T06:38:52.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.372+0000 7fb34e800640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 0x7fb3500cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.373+0000 7fb345400640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500cd760 0x7fb3500b4150 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb338009870 tx=0x7fb33802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151ad5f0d53c81f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34d400640 1 -- 172.21.15.59:0/3237370176 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338037070 con 0x7fb3500cd760 2024-04-24T06:38:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34d400640 1 -- 172.21.15.59:0/3237370176 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb338032440 con 0x7fb3500cd760 2024-04-24T06:38:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34d400640 1 -- 172.21.15.59:0/3237370176 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338032810 con 0x7fb3500cd760 2024-04-24T06:38:52.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 -- 172.21.15.59:0/3237370176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500cd760 msgr2=0x7fb3500b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500cd760 0x7fb3500b4150 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb338009870 tx=0x7fb33802a250 comp rx=0 tx=0).stop 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 -- 172.21.15.59:0/3237370176 shutdown_connections 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb3500b4690 0x7fb3500cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 --2- 172.21.15.59:0/3237370176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500cd760 0x7fb3500b4150 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.374+0000 7fb34f200640 1 -- 172.21.15.59:0/3237370176 >> 172.21.15.59:0/3237370176 conn(0x7fb3500b21a0 msgr2=0x7fb3500b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.375+0000 7fb34f200640 1 -- 172.21.15.59:0/3237370176 shutdown_connections 2024-04-24T06:38:52.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.375+0000 7fb34f200640 1 -- 172.21.15.59:0/3237370176 wait complete. 2024-04-24T06:38:52.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.376+0000 7fb34f200640 1 Processor -- start 2024-04-24T06:38:52.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.376+0000 7fb34f200640 1 -- start start 2024-04-24T06:38:52.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 0x7fb3500b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb35015c3d0 0x7fb35009bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb35015c7b0 con 0x7fb3500b4690 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500b4b00 con 0x7fb3500cd760 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.377+0000 7fb34f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb35015c930 con 0x7fb35015c3d0 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38148/0 (socket says 172.21.15.59:38148) 2024-04-24T06:38:52.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 -- 172.21.15.59:0/3413238182 learned_addr learned my addr 172.21.15.59:0/3413238182 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb345400640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 0x7fb3500b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 -- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb35015c3d0 msgr2=0x7fb35009bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34e800640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb35015c3d0 0x7fb35009bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb35015c3d0 0x7fb35009bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 -- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 msgr2=0x7fb3500b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 0x7fb3500b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb34de00640 1 -- 172.21.15.59:0/3413238182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3501514d0 con 0x7fb3500b4690 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.378+0000 7fb345400640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 0x7fb3500b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:52.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb34de00640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c430 tx=0x7fb34000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340017070 con 0x7fb3500b4690 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400052d0 con 0x7fb3500b4690 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340003830 con 0x7fb3500b4690 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500aea30 con 0x7fb3500b4690 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.379+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3500aeff0 con 0x7fb3500b4690 2024-04-24T06:38:52.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.380+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35006e0e0 con 0x7fb3500b4690 2024-04-24T06:38:52.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.382+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb34001a070 con 0x7fb3500b4690 2024-04-24T06:38:52.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.383+0000 7fb346a00640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 0x7fb32c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.383+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fb34008ae90 con 0x7fb3500b4690 2024-04-24T06:38:52.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.383+0000 7fb345400640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 0x7fb32c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.384+0000 7fb345400640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 0x7fb32c06edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb338034000 tx=0x7fb33802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.387+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3400592c0 con 0x7fb3500b4690 2024-04-24T06:38:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.592+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb35004fd90 con 0x7fb3500b4690 2024-04-24T06:38:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.594+0000 7fb346a00640 1 -- 172.21.15.59:0/3413238182 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16524 (secure 0 0 0) 0x7fb340058bc0 con 0x7fb3500b4690 2024-04-24T06:38:52.596 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:38:52.596 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:40.185327+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.596+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 msgr2=0x7fb32c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.596+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 0x7fb32c06edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb338034000 tx=0x7fb33802e000 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.596+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 msgr2=0x7fb350150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.596+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c430 tx=0x7fb34000c8f0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 shutdown_connections 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb32c06c8f0 0x7fb32c06edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb35015c3d0 0x7fb35009bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3500cd760 0x7fb3500b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 --2- 172.21.15.59:0/3413238182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3500b4690 0x7fb350150f90 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 >> 172.21.15.59:0/3413238182 conn(0x7fb3500b21a0 msgr2=0x7fb35015f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:52.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 shutdown_connections 2024-04-24T06:38:52.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.597+0000 7fb34f200640 1 -- 172.21.15.59:0/3413238182 wait complete. 2024-04-24T06:38:52.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:38:36.464123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:38:37.616685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:38:52.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:38:52.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.815+0000 7f0543e00640 1 Processor -- start 2024-04-24T06:38:52.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.815+0000 7f0543e00640 1 -- start start 2024-04-24T06:38:52.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0543e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 0x7f0544156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0543e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054415cab0 con 0x7f054415d810 2024-04-24T06:38:52.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0543e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0544160e40 con 0x7f05440d0d10 2024-04-24T06:38:52.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0543e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f054415cd30 con 0x7f05441567e0 2024-04-24T06:38:52.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f053a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f054415d810 0x7f05440d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:38636/0 (socket says 172.21.15.59:38636) 2024-04-24T06:38:52.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f053a000640 1 -- 172.21.15.59:0/1221119205 learned_addr learned my addr 172.21.15.59:0/1221119205 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277939457 0 0) 0x7f052c001110 con 0x7f054415d810 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.817+0000 7f0542a00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 0x7f0544156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0528003520 con 0x7f054415d810 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1040083956 0 0) 0x7f0528003520 con 0x7f054415d810 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 msgr2=0x7f0544156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 0x7f0544156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 >> v1:172.21.15.66:6789/0 conn(0x7f05440d0d10 legacy=0x7f0544160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054415d010 con 0x7f054415d810 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f052c001840 con 0x7f054415d810 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 >> v1:172.21.15.59:6789/0 conn(0x7f054415d810 legacy=0x7f05440d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f0528003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528004800 0x7f0528004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 0x7f0528005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f054415cd30 con 0x7f05280039e0 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f052800ff60 con 0x7f0528004800 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0528010160 con 0x7f05441567e0 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.818+0000 7f0542a00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f0528003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f0542a00640 1 -- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 msgr2=0x7f0528005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f0542a00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441567e0 0x7f0528005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f0542a00640 1 -- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528004800 msgr2=0x7f0528004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:52.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f053a000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528004800 0x7f0528004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f0542a00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528004800 0x7f0528004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f0542a00640 1 -- 172.21.15.59:0/1221119205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0528003520 con 0x7f05280039e0 2024-04-24T06:38:52.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.819+0000 7f053a000640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0528004800 0x7f0528004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:52.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.820+0000 7f0542a00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f0528003e00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f053400ba30 tx=0x7f0528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399dc6fce46fba88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.820+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534030070 con 0x7f05280039e0 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.820+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053402ae10 con 0x7f05280039e0 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.820+0000 7f0542000640 1 -- 172.21.15.59:0/1221119205 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053400b600 con 0x7f05280039e0 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 -- 172.21.15.59:0/1221119205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 msgr2=0x7f0528003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 --2- 172.21.15.59:0/1221119205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f0528003e00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f053400ba30 tx=0x7f0528005830 comp rx=0 tx=0).stop 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543400640 1 -- 172.21.15.59:0/1221119205 reap_dead start 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 -- 172.21.15.59:0/1221119205 shutdown_connections 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 -- 172.21.15.59:0/1221119205 >> 172.21.15.59:0/1221119205 conn(0x7f054406bdf0 msgr2=0x7f054406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 -- 172.21.15.59:0/1221119205 shutdown_connections 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.821+0000 7f0543e00640 1 -- 172.21.15.59:0/1221119205 wait complete. 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.823+0000 7f0543e00640 1 Processor -- start 2024-04-24T06:38:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.823+0000 7f0543e00640 1 -- start start 2024-04-24T06:38:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 0x7f0544097380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441508e0 0x7f0544150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0544156d40 con 0x7f05280039e0 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0544156bc0 con 0x7f0544096f00 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0543e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0544156ec0 con 0x7f05441508e0 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0542a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0542a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38166/0 (socket says 172.21.15.59:38166) 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f0542a00640 1 -- 172.21.15.59:0/2144236654 learned_addr learned my addr 172.21.15.59:0/2144236654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.824+0000 7f053a000640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 0x7f0544097380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 -- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441508e0 msgr2=0x7f0544150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441508e0 0x7f0544150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 -- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 msgr2=0x7f0544097380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 0x7f0544097380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 -- 172.21.15.59:0/2144236654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05440b27c0 con 0x7f05280039e0 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f053a000640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 0x7f0544097380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0542a00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f053400bb50 tx=0x7f0534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534043070 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0534004020 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053400b600 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.825+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05440b2aa0 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.826+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05440978c0 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.827+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054404fd90 con 0x7f05280039e0 2024-04-24T06:38:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.828+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f053403f650 con 0x7f05280039e0 2024-04-24T06:38:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.829+0000 7f053b400640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.829+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f05340ac290 con 0x7f05280039e0 2024-04-24T06:38:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.829+0000 7f053a000640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.830+0000 7f053a000640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 0x7f052006edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f052c0019b0 tx=0x7f052c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:52.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:52.834+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f053407a590 con 0x7f05280039e0 2024-04-24T06:38:53.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.037+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05440b5150 con 0x7f05280039e0 2024-04-24T06:38:53.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.038+0000 7f053b400640 1 -- 172.21.15.59:0/2144236654 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f0534079e90 con 0x7f05280039e0 2024-04-24T06:38:53.040 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:38:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 msgr2=0x7f052006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 0x7f052006edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f052c0019b0 tx=0x7f052c009040 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 msgr2=0x7f05440af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f053400bb50 tx=0x7f0534004730 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 shutdown_connections 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f05441508e0 0x7f0544150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0544096f00 0x7f0544097380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 --2- 172.21.15.59:0/2144236654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05280039e0 0x7f05440af6b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.040+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 >> 172.21.15.59:0/2144236654 conn(0x7f054406bdf0 msgr2=0x7f05440997d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.041+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 shutdown_connections 2024-04-24T06:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.041+0000 7f0543e00640 1 -- 172.21.15.59:0/2144236654 wait complete. 2024-04-24T06:38:53.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:38:53.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.275+0000 7fb6b9a00640 1 Processor -- start 2024-04-24T06:38:53.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.275+0000 7fb6b9a00640 1 -- start start 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 0x7fb6b40aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 0x7fb6b4162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6b4071d70 0x7fb6b4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b40aef40 con 0x7fb6b4071d70 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b40afd80 con 0x7fb6b406f6f0 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.276+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b4163370 con 0x7fb6b415e790 2024-04-24T06:38:53.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6b4071d70 0x7fb6b4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6b4071d70 0x7fb6b4072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38170/0 (socket says 172.21.15.59:38170) 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 -- 172.21.15.59:0/3009821748 learned_addr learned my addr 172.21.15.59:0/3009821748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6aa000640 1 --2- 172.21.15.59:0/3009821748 >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 0x7fb6b4162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b2a00640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 0x7fb6b40aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 -- 172.21.15.59:0/3009821748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 msgr2=0x7fb6b40aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 0x7fb6b40aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 -- 172.21.15.59:0/3009821748 >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 msgr2=0x7fb6b4162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 --2- 172.21.15.59:0/3009821748 >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 0x7fb6b4162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b3400640 1 -- 172.21.15.59:0/3009821748 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b40adc80 con 0x7fb6b4071d70 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6aa000640 1 --2- 172.21.15.59:0/3009821748 >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 0x7fb6b4162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:53.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.277+0000 7fb6b2a00640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 0x7fb6b40aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.278+0000 7fb6b3400640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b4071d70 0x7fb6b4072200 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4070040 tx=0x7fb6ac02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7abd02a833f048 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.279+0000 7fb6b2000640 1 -- 172.21.15.59:0/3009821748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac037070 con 0x7fb6b4071d70 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.279+0000 7fb6b2000640 1 -- 172.21.15.59:0/3009821748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac032440 con 0x7fb6b4071d70 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.279+0000 7fb6b2000640 1 -- 172.21.15.59:0/3009821748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac032810 con 0x7fb6b4071d70 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.279+0000 7fb6b9a00640 1 -- 172.21.15.59:0/3009821748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b4071d70 msgr2=0x7fb6b4072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.279+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b4071d70 0x7fb6b4072200 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4070040 tx=0x7fb6ac02a1d0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 -- 172.21.15.59:0/3009821748 shutdown_connections 2024-04-24T06:38:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b4071d70 0x7fb6b4072200 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/3009821748 >> v2:172.21.15.66:3300/0 conn(0x7fb6b406f6f0 0x7fb6b4162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/3009821748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b415e790 0x7fb6b40aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 -- 172.21.15.59:0/3009821748 >> 172.21.15.59:0/3009821748 conn(0x7fb6b4067d90 msgr2=0x7fb6b406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 -- 172.21.15.59:0/3009821748 shutdown_connections 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.280+0000 7fb6b9a00640 1 -- 172.21.15.59:0/3009821748 wait complete. 2024-04-24T06:38:53.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.281+0000 7fb6b9a00640 1 Processor -- start 2024-04-24T06:38:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.281+0000 7fb6b9a00640 1 -- start start 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6b406f6f0 0x7fb6b415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b4071d70 0x7fb6b415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b40ae4e0 con 0x7fb6b415e790 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b40ae200 con 0x7fb6b406f6f0 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.282+0000 7fb6b9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b406fbb0 con 0x7fb6b4071d70 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38174/0 (socket says 172.21.15.59:38174) 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 -- 172.21.15.59:0/2391279248 learned_addr learned my addr 172.21.15.59:0/2391279248 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 -- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b4071d70 msgr2=0x7fb6b415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6aa000640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b4071d70 0x7fb6b415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b2a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6b406f6f0 0x7fb6b415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b4071d70 0x7fb6b415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 -- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6b406f6f0 msgr2=0x7fb6b415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6b406f6f0 0x7fb6b415f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.283+0000 7fb6b3400640 1 -- 172.21.15.59:0/2391279248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b415ce90 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6b3400640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac02a210 tx=0x7fb6ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac037040 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac042070 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac030bd0 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b415dae0 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.284+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b415f0a0 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.285+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b4068a80 con 0x7fb6b415e790 2024-04-24T06:38:53.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.286+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6ac047460 con 0x7fb6b415e790 2024-04-24T06:38:53.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.287+0000 7fb6ab400640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 0x7fb69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.287+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fb6ac0ac810 con 0x7fb6b415e790 2024-04-24T06:38:53.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.287+0000 7fb6b2a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 0x7fb69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.288+0000 7fb6b2a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 0x7fb69006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4008e60 tx=0x7fb6a40083b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.291+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6ac07aba0 con 0x7fb6b415e790 2024-04-24T06:38:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.501+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6b40d08c0 con 0x7fb6b415e790 2024-04-24T06:38:53.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.502+0000 7fb6ab400640 1 -- 172.21.15.59:0/2391279248 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fb6ac07a4a0 con 0x7fb6b415e790 2024-04-24T06:38:53.504 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:38:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.504+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 msgr2=0x7fb69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 0x7fb69006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4008e60 tx=0x7fb6a40083b0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 msgr2=0x7fb6b4144d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac02a210 tx=0x7fb6ac0046c0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 shutdown_connections 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb69006c8f0 0x7fb69006edb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b415e790 0x7fb6b4144d60 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb6b4071d70 0x7fb6b415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 --2- 172.21.15.59:0/2391279248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6b406f6f0 0x7fb6b415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 >> 172.21.15.59:0/2391279248 conn(0x7fb6b4067d90 msgr2=0x7fb6b406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 shutdown_connections 2024-04-24T06:38:53.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.505+0000 7fb6b9a00640 1 -- 172.21.15.59:0/2391279248 wait complete. 2024-04-24T06:38:53.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:38:53.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.740+0000 7f5153e00640 1 Processor -- start 2024-04-24T06:38:53.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.740+0000 7f5153e00640 1 -- start start 2024-04-24T06:38:53.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f51540d0d10 0x7f5154156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515415d810 0x7f5154160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515415cab0 con 0x7f51541567e0 2024-04-24T06:38:53.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5154160e40 con 0x7f515415d810 2024-04-24T06:38:53.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f515415cd30 con 0x7f51540d0d10 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515415d810 0x7f5154160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f515415d810 0x7f5154160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38190/0 (socket says 172.21.15.59:38190) 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.742+0000 7f5153400640 1 -- 172.21.15.59:0/3542041236 learned_addr learned my addr 172.21.15.59:0/3542041236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.743+0000 7f514a000640 1 --2- 172.21.15.59:0/3542041236 >> v2:172.21.15.66:3300/0 conn(0x7f51540d0d10 0x7f5154156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.743+0000 7f5153400640 1 -- 172.21.15.59:0/3542041236 >> v2:172.21.15.66:3300/0 conn(0x7f51540d0d10 msgr2=0x7f5154156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.743+0000 7f5153400640 1 --2- 172.21.15.59:0/3542041236 >> v2:172.21.15.66:3300/0 conn(0x7f51540d0d10 0x7f5154156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.743+0000 7f5153400640 1 -- 172.21.15.59:0/3542041236 >> v1:172.21.15.66:6789/0 conn(0x7f51541567e0 legacy=0x7f51540d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.743+0000 7f5153400640 1 -- 172.21.15.59:0/3542041236 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515415d010 con 0x7f515415d810 2024-04-24T06:38:53.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.744+0000 7f5153400640 1 --2- 172.21.15.59:0/3542041236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515415d810 0x7f5154160110 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5154074140 tx=0x7f514c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565fc171f7aca026 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.744+0000 7f5152000640 1 -- 172.21.15.59:0/3542041236 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c037070 con 0x7f515415d810 2024-04-24T06:38:53.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.744+0000 7f5152000640 1 -- 172.21.15.59:0/3542041236 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c032440 con 0x7f515415d810 2024-04-24T06:38:53.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.744+0000 7f5152000640 1 -- 172.21.15.59:0/3542041236 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c032810 con 0x7f515415d810 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 -- 172.21.15.59:0/3542041236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515415d810 msgr2=0x7f5154160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 --2- 172.21.15.59:0/3542041236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515415d810 0x7f5154160110 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5154074140 tx=0x7f514c02a230 comp rx=0 tx=0).stop 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 -- 172.21.15.59:0/3542041236 shutdown_connections 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 --2- 172.21.15.59:0/3542041236 >> v2:172.21.15.66:3300/0 conn(0x7f51540d0d10 0x7f5154156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 --2- 172.21.15.59:0/3542041236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f515415d810 0x7f5154160110 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 -- 172.21.15.59:0/3542041236 >> 172.21.15.59:0/3542041236 conn(0x7f515406bdf0 msgr2=0x7f515406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 -- 172.21.15.59:0/3542041236 shutdown_connections 2024-04-24T06:38:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.745+0000 7f5153e00640 1 -- 172.21.15.59:0/3542041236 wait complete. 2024-04-24T06:38:53.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.747+0000 7f5153e00640 1 Processor -- start 2024-04-24T06:38:53.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.747+0000 7f5153e00640 1 -- start start 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 0x7f5154072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f515415d810 0x7f51540730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51541607d0 con 0x7f51540d0d10 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5154160650 con 0x7f51541567e0 2024-04-24T06:38:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.748+0000 7f5153e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5154160950 con 0x7f515415d810 2024-04-24T06:38:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38198/0 (socket says 172.21.15.59:38198) 2024-04-24T06:38:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 -- 172.21.15.59:0/94951392 learned_addr learned my addr 172.21.15.59:0/94951392 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f514a000640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 0x7f5154072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5153400640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f515415d810 0x7f51540730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 -- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f515415d810 msgr2=0x7f51540730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f515415d810 0x7f51540730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 -- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 msgr2=0x7f5154072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 0x7f5154072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f5152a00640 1 -- 172.21.15.59:0/94951392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515406ec10 con 0x7f51540d0d10 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.749+0000 7f514a000640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 0x7f5154072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f5152a00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f514400c7c0 tx=0x7f514400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144016020 con 0x7f51540d0d10 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514400aa50 con 0x7f51540d0d10 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144012420 con 0x7f51540d0d10 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515406de50 con 0x7f51540d0d10 2024-04-24T06:38:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.750+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515406e410 con 0x7f51540d0d10 2024-04-24T06:38:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.751+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154068290 con 0x7f51540d0d10 2024-04-24T06:38:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.752+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5144019020 con 0x7f51540d0d10 2024-04-24T06:38:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.754+0000 7f514b400640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.754+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f514408b960 con 0x7f51540d0d10 2024-04-24T06:38:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.754+0000 7f514a000640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.755+0000 7f514a000640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 0x7f513006edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f513c006fd0 tx=0x7f513c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.758+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5154068290 con 0x7f51540d0d10 2024-04-24T06:38:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.968+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5154155d80 con 0x7f51540d0d10 2024-04-24T06:38:53.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.969+0000 7f514b400640 1 -- 172.21.15.59:0/94951392 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f5154155d80 con 0x7f51540d0d10 2024-04-24T06:38:53.971 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:38:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 msgr2=0x7f513006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 0x7f513006edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f513c006fd0 tx=0x7f513c008040 comp rx=0 tx=0).stop 2024-04-24T06:38:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 msgr2=0x7f515414b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f514400c7c0 tx=0x7f514400cc80 comp rx=0 tx=0).stop 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 shutdown_connections 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f513006c8f0 0x7f513006edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f515415d810 0x7f51540730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51541567e0 0x7f5154072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 --2- 172.21.15.59:0/94951392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51540d0d10 0x7f515414b830 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.972+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 >> 172.21.15.59:0/94951392 conn(0x7f515406bdf0 msgr2=0x7f5154071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.973+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 shutdown_connections 2024-04-24T06:38:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:53.973+0000 7f5153e00640 1 -- 172.21.15.59:0/94951392 wait complete. 2024-04-24T06:38:53.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:38:54.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.208+0000 7fae2fe00640 1 Processor -- start 2024-04-24T06:38:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.209+0000 7fae2fe00640 1 -- start start 2024-04-24T06:38:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae300d0f80 0x7fae300cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae3015ea40 0x7fae3015f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fae300d1360 0x7fae30162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fae3015fdc0 con 0x7fae300d0f80 2024-04-24T06:38:54.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fae300cf930 con 0x7fae300d1360 2024-04-24T06:38:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fae300cfc10 con 0x7fae3015ea40 2024-04-24T06:38:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.210+0000 7fae2ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae300d0f80 0x7fae300cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae300d0f80 0x7fae300cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38204/0 (socket says 172.21.15.59:38204) 2024-04-24T06:38:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 -- 172.21.15.59:0/1444884356 learned_addr learned my addr 172.21.15.59:0/1444884356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2f400640 1 --2- 172.21.15.59:0/1444884356 >> v2:172.21.15.66:3300/0 conn(0x7fae300d1360 0x7fae30162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2e000640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae3015ea40 0x7fae3015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 -- 172.21.15.59:0/1444884356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae3015ea40 msgr2=0x7fae3015f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae3015ea40 0x7fae3015f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 -- 172.21.15.59:0/1444884356 >> v2:172.21.15.66:3300/0 conn(0x7fae300d1360 msgr2=0x7fae30162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 --2- 172.21.15.59:0/1444884356 >> v2:172.21.15.66:3300/0 conn(0x7fae300d1360 0x7fae30162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.211+0000 7fae2ea00640 1 -- 172.21.15.59:0/1444884356 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae300cfef0 con 0x7fae300d0f80 2024-04-24T06:38:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.213+0000 7fae2ea00640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae300d0f80 0x7fae300cf250 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fae300d0d40 tx=0x7fae2002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ee6308fd98918d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.213+0000 7fae2d600640 1 -- 172.21.15.59:0/1444884356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20037070 con 0x7fae300d0f80 2024-04-24T06:38:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.213+0000 7fae2d600640 1 -- 172.21.15.59:0/1444884356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20032440 con 0x7fae300d0f80 2024-04-24T06:38:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.213+0000 7fae2d600640 1 -- 172.21.15.59:0/1444884356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20032810 con 0x7fae300d0f80 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 -- 172.21.15.59:0/1444884356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae300d0f80 msgr2=0x7fae300cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae300d0f80 0x7fae300cf250 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fae300d0d40 tx=0x7fae2002a140 comp rx=0 tx=0).stop 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 -- 172.21.15.59:0/1444884356 shutdown_connections 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 --2- 172.21.15.59:0/1444884356 >> v2:172.21.15.66:3300/0 conn(0x7fae300d1360 0x7fae30162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae3015ea40 0x7fae3015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 --2- 172.21.15.59:0/1444884356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae300d0f80 0x7fae300cf250 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 -- 172.21.15.59:0/1444884356 >> 172.21.15.59:0/1444884356 conn(0x7fae3008bf90 msgr2=0x7fae300d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 -- 172.21.15.59:0/1444884356 shutdown_connections 2024-04-24T06:38:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.214+0000 7fae2fe00640 1 -- 172.21.15.59:0/1444884356 wait complete. 2024-04-24T06:38:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.216+0000 7fae2fe00640 1 Processor -- start 2024-04-24T06:38:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.216+0000 7fae2fe00640 1 -- start start 2024-04-24T06:38:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae300d0f80 0x7fae300b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae300d1360 0x7fae300b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae300d17e0 con 0x7fae3015ea40 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae300d04b0 con 0x7fae300d0f80 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fae301628b0 con 0x7fae300d1360 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38210/0 (socket says 172.21.15.59:38210) 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2f400640 1 -- 172.21.15.59:0/3804199821 learned_addr learned my addr 172.21.15.59:0/3804199821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.218+0000 7fae2ea00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae300d0f80 0x7fae300b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 -- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae300d1360 msgr2=0x7fae300b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae300d1360 0x7fae300b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 -- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae300d0f80 msgr2=0x7fae300b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae300d0f80 0x7fae300b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 -- 172.21.15.59:0/3804199821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae300b4d10 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2f400640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fae28007e40 tx=0x7fae2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28014620 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae28019070 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28014a20 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.219+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae300b9840 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.220+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae300b9e00 con 0x7fae3015ea40 2024-04-24T06:38:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.221+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3004fd90 con 0x7fae3015ea40 2024-04-24T06:38:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.222+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae28014bc0 con 0x7fae3015ea40 2024-04-24T06:38:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.224+0000 7fae26a00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.224+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fae2808c3d0 con 0x7fae3015ea40 2024-04-24T06:38:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.224+0000 7fae2ea00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.225+0000 7fae2ea00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 0x7fae1006edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.228+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae2805a890 con 0x7fae3015ea40 2024-04-24T06:38:54.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.447+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae300d1ec0 con 0x7fae3015ea40 2024-04-24T06:38:54.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.448+0000 7fae26a00640 1 -- 172.21.15.59:0/3804199821 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) ==== 80+0+0 (secure 0 0 0) 0x7fae2805a190 con 0x7fae3015ea40 2024-04-24T06:38:54.450 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.450+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 msgr2=0x7fae1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.450+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 0x7fae1006edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.450+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 msgr2=0x7fae300b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.450+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fae28007e40 tx=0x7fae2800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 shutdown_connections 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae3015ea40 0x7fae300b3e10 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fae300d1360 0x7fae300b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 --2- 172.21.15.59:0/3804199821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae300d0f80 0x7fae300b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 >> 172.21.15.59:0/3804199821 conn(0x7fae3008bf90 msgr2=0x7fae30161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 shutdown_connections 2024-04-24T06:38:54.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:38:54.451+0000 7fae2fe00640 1 -- 172.21.15.59:0/3804199821 wait complete. 2024-04-24T06:38:54.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:38:54.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:38:54.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:38:54.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[69762]: fuse finished with error 0 and tester_r 0 2024-04-24T06:38:54.553 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:39:00.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:39:00.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:00.656 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:39:00.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:00.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.933+0000 7f9ed4a00640 1 Processor -- start 2024-04-24T06:39:00.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.933+0000 7f9ed4a00640 1 -- start start 2024-04-24T06:39:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.934+0000 7f9ed4a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 0x7f9ed00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.934+0000 7f9ed4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ed00cd590 0x7f9ed00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.934+0000 7f9ed4a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed0160210 con 0x7f9ed00b4690 2024-04-24T06:39:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.934+0000 7f9ed4a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed01604f0 con 0x7f9ed00cd590 2024-04-24T06:39:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.934+0000 7f9ed4a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed01607d0 con 0x7f9ed015c3d0 2024-04-24T06:39:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ed00cd590 0x7f9ed00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecf200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9ed00b4690 0x7f9ed006b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48836/0 (socket says 172.21.15.59:48836) 2024-04-24T06:39:00.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecf200640 1 -- 172.21.15.59:0/2592852170 learned_addr learned my addr 172.21.15.59:0/2592852170 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ece800640 1 --2- 172.21.15.59:0/2592852170 >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 0x7f9ed00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecde00640 1 -- 172.21.15.59:0/2592852170 >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 msgr2=0x7f9ed00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecde00640 1 --2- 172.21.15.59:0/2592852170 >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 0x7f9ed00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecde00640 1 -- 172.21.15.59:0/2592852170 >> v1:172.21.15.59:6789/0 conn(0x7f9ed00b4690 legacy=0x7f9ed006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ecde00640 1 -- 172.21.15.59:0/2592852170 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed0160ab0 con 0x7f9ed00cd590 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.935+0000 7f9ece800640 1 --2- 172.21.15.59:0/2592852170 >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 0x7f9ed00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:00.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.936+0000 7f9ecde00640 1 --2- 172.21.15.59:0/2592852170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed00cd590 0x7f9ed00b4150 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9ed006d9b0 tx=0x7f9ec002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0231ce2c1f0f26c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.936+0000 7f9ecd400640 1 -- 172.21.15.59:0/2592852170 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0037070 con 0x7f9ed00cd590 2024-04-24T06:39:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.936+0000 7f9ecd400640 1 -- 172.21.15.59:0/2592852170 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0032440 con 0x7f9ed00cd590 2024-04-24T06:39:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.936+0000 7f9ecd400640 1 -- 172.21.15.59:0/2592852170 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0032810 con 0x7f9ed00cd590 2024-04-24T06:39:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 -- 172.21.15.59:0/2592852170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed00cd590 msgr2=0x7f9ed00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/2592852170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed00cd590 0x7f9ed00b4150 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9ed006d9b0 tx=0x7f9ec002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 -- 172.21.15.59:0/2592852170 shutdown_connections 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/2592852170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed00cd590 0x7f9ed00b4150 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/2592852170 >> v2:172.21.15.66:3300/0 conn(0x7f9ed015c3d0 0x7f9ed00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 -- 172.21.15.59:0/2592852170 >> 172.21.15.59:0/2592852170 conn(0x7f9ed00b21a0 msgr2=0x7f9ed00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 -- 172.21.15.59:0/2592852170 shutdown_connections 2024-04-24T06:39:00.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.937+0000 7f9ed4a00640 1 -- 172.21.15.59:0/2592852170 wait complete. 2024-04-24T06:39:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.939+0000 7f9ed4a00640 1 Processor -- start 2024-04-24T06:39:00.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.939+0000 7f9ed4a00640 1 -- start start 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ed00b4690 0x7f9ed0072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00cd590 0x7f9ed015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed015c7b0 con 0x7f9ed015c3d0 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00b4ac0 con 0x7f9ed00cd590 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ed4a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed015c930 con 0x7f9ed00b4690 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.940+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38988/0 (socket says 172.21.15.59:38988) 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 -- 172.21.15.59:0/928667973 learned_addr learned my addr 172.21.15.59:0/928667973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ece800640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ed00b4690 0x7f9ed0072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecde00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00cd590 0x7f9ed015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 -- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ed00b4690 msgr2=0x7f9ed0072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:00.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ed00b4690 0x7f9ed0072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 -- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00cd590 msgr2=0x7f9ed015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:00.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00cd590 0x7f9ed015ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:00.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 -- 172.21.15.59:0/928667973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed015f470 con 0x7f9ed015c3d0 2024-04-24T06:39:00.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.941+0000 7f9ecf200640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800cbc0 tx=0x7f9ec800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.942+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8017070 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.942+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec800ad40 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.942+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed015f710 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.942+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8012600 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.942+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed00ad2b0 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.943+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed004fd90 con 0x7f9ed015c3d0 2024-04-24T06:39:00.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.944+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec801a070 con 0x7f9ed015c3d0 2024-04-24T06:39:00.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.945+0000 7f9ec6a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 0x7f9eac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:00.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.945+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec808b830 con 0x7f9ed015c3d0 2024-04-24T06:39:00.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.945+0000 7f9ece800640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 0x7f9eac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:00.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.946+0000 7f9ece800640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 0x7f9eac06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800a950 tx=0x7f9eb8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:00.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:00.950+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ec8059600 con 0x7f9ed015c3d0 2024-04-24T06:39:01.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.196+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed00ce240 con 0x7f9ed015c3d0 2024-04-24T06:39:01.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.197+0000 7f9ec6a00640 1 -- 172.21.15.59:0/928667973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5177 (secure 0 0 0) 0x7f9ec805d900 con 0x7f9ed015c3d0 2024-04-24T06:39:01.199 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:01.199 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:38:41.731305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/891418539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":891418539},{"type":"v1","addr":"172.21.15.66:6834","nonce":891418539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":7}]} 2024-04-24T06:39:01.199 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 83 2024-04-24T06:39:01.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 msgr2=0x7f9eac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 0x7f9eac06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800a950 tx=0x7f9eb8008040 comp rx=0 tx=0).stop 2024-04-24T06:39:01.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 msgr2=0x7f9ed00c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800cbc0 tx=0x7f9ec800a430 comp rx=0 tx=0).stop 2024-04-24T06:39:01.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 shutdown_connections 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9eac06c8f0 0x7f9eac06edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed015c3d0 0x7f9ed00c5460 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00cd590 0x7f9ed015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 --2- 172.21.15.59:0/928667973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9ed00b4690 0x7f9ed0072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 >> 172.21.15.59:0/928667973 conn(0x7f9ed00b21a0 msgr2=0x7f9ed00b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 shutdown_connections 2024-04-24T06:39:01.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.200+0000 7f9ed4a00640 1 -- 172.21.15.59:0/928667973 wait complete. 2024-04-24T06:39:01.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-24T06:39:01.480 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:39:01.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:39:01.481 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.693+0000 7fd91b000640 1 Processor -- start 2024-04-24T06:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.694+0000 7fd91b000640 1 -- start start 2024-04-24T06:39:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.695+0000 7fd91b000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd91406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.695+0000 7fd91b000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91415f160 con 0x7fd91415ece0 2024-04-24T06:39:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.695+0000 7fd91b000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91415f430 con 0x7fd9140b3080 2024-04-24T06:39:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.695+0000 7fd91b000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd9140ced20 con 0x7fd9140cd050 2024-04-24T06:39:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd919200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd91415ece0 0x7fd9140cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48848/0 (socket says 172.21.15.59:48848) 2024-04-24T06:39:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd919200640 1 -- 172.21.15.59:0/2149555540 learned_addr learned my addr 172.21.15.59:0/2149555540 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd919c00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd91406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4194108605 0 0) 0x7fd91415f160 con 0x7fd91415ece0 2024-04-24T06:39:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8fc003520 con 0x7fd91415ece0 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1640705661 0 0) 0x7fd8fc003520 con 0x7fd91415ece0 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 msgr2=0x7fd91406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.696+0000 7fd913e00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd91406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 >> v1:172.21.15.66:6789/0 conn(0x7fd9140b3080 legacy=0x7fd9140cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91415f710 con 0x7fd91415ece0 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd900001670 con 0x7fd91415ece0 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 >> v1:172.21.15.59:6789/0 conn(0x7fd91415ece0 legacy=0x7fd9140cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc0039e0 0x7fd8fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd8fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc004d40 0x7fd8fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc00bb50 con 0x7fd8fc004d40 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc00bcd0 con 0x7fd8fc0039e0 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc00bed0 con 0x7fd9140cd050 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd91a600640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc004d40 0x7fd8fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd919c00640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc0039e0 0x7fd8fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.697+0000 7fd919200640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd8fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.698+0000 7fd91a600640 1 -- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 msgr2=0x7fd8fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.698+0000 7fd91a600640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140cd050 0x7fd8fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.698+0000 7fd91a600640 1 -- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc0039e0 msgr2=0x7fd8fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.698+0000 7fd91a600640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc0039e0 0x7fd8fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.698+0000 7fd91a600640 1 -- 172.21.15.59:0/2149555540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc003520 con 0x7fd8fc004d40 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.699+0000 7fd91a600640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc004d40 0x7fd8fc0051c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001e50 tx=0x7fd8fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9081c9272f757a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.699+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c001a70 con 0x7fd8fc004d40 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.699+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90c003970 con 0x7fd8fc004d40 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.699+0000 7fd913e00640 1 -- 172.21.15.59:0/2149555540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c003d60 con 0x7fd8fc004d40 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 -- 172.21.15.59:0/2149555540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc004d40 msgr2=0x7fd8fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 --2- 172.21.15.59:0/2149555540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc004d40 0x7fd8fc0051c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001e50 tx=0x7fd8fc005830 comp rx=0 tx=0).stop 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91a600640 1 -- 172.21.15.59:0/2149555540 reap_dead start 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 -- 172.21.15.59:0/2149555540 shutdown_connections 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 -- 172.21.15.59:0/2149555540 >> 172.21.15.59:0/2149555540 conn(0x7fd9140b21a0 msgr2=0x7fd9140b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 -- 172.21.15.59:0/2149555540 shutdown_connections 2024-04-24T06:39:01.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.700+0000 7fd91b000640 1 -- 172.21.15.59:0/2149555540 wait complete. 2024-04-24T06:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.702+0000 7fd91b000640 1 Processor -- start 2024-04-24T06:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.702+0000 7fd91b000640 1 -- start start 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc004d40 0x7fd9140c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 0x7fd9140c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd9140cef50 con 0x7fd914094220 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd91415fcd0 con 0x7fd8fc004d40 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91b000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd914159110 con 0x7fd9140afc10 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39010/0 (socket says 172.21.15.59:39010) 2024-04-24T06:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd91a600640 1 -- 172.21.15.59:0/2385860063 learned_addr learned my addr 172.21.15.59:0/2385860063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd919200640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 0x7fd9140c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.703+0000 7fd919c00640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc004d40 0x7fd9140c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 -- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 msgr2=0x7fd9140c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 0x7fd9140c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 -- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc004d40 msgr2=0x7fd9140c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc004d40 0x7fd9140c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 -- 172.21.15.59:0/2385860063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9140b00f0 con 0x7fd914094220 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd919200640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 0x7fd9140c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91a600640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001f70 tx=0x7fd90c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c001a70 con 0x7fd914094220 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90c041070 con 0x7fd914094220 2024-04-24T06:39:01.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c005520 con 0x7fd914094220 2024-04-24T06:39:01.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9140b03d0 con 0x7fd914094220 2024-04-24T06:39:01.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.704+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9140b0990 con 0x7fd914094220 2024-04-24T06:39:01.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.706+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd914068290 con 0x7fd914094220 2024-04-24T06:39:01.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.707+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd90c03a070 con 0x7fd914094220 2024-04-24T06:39:01.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.707+0000 7fd911600640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 0x7fd8f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:01.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.708+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fd90c0ab650 con 0x7fd914094220 2024-04-24T06:39:01.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.708+0000 7fd919c00640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 0x7fd8f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:01.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.708+0000 7fd919c00640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 0x7fd8f406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd908009950 tx=0x7fd908008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:01.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.712+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd914068290 con 0x7fd914094220 2024-04-24T06:39:01.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.959+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91404fd90 con 0x7fd914094220 2024-04-24T06:39:01.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.961+0000 7fd911600640 1 -- 172.21.15.59:0/2385860063 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5177 (secure 0 0 0) 0x7fd90c079950 con 0x7fd914094220 2024-04-24T06:39:01.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:01.963 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:38:41.731305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/891418539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":891418539},{"type":"v1","addr":"172.21.15.66:6834","nonce":891418539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":7}]} 2024-04-24T06:39:01.963 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 83 2024-04-24T06:39:01.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 msgr2=0x7fd8f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 0x7fd8f406ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd908009950 tx=0x7fd908008040 comp rx=0 tx=0).stop 2024-04-24T06:39:01.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 msgr2=0x7fd9140b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001f70 tx=0x7fd90c0014a0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 shutdown_connections 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd8f406c920 0x7fd8f406ede0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd914094220 0x7fd9140b1190 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd9140afc10 0x7fd9140c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 --2- 172.21.15.59:0/2385860063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8fc004d40 0x7fd9140c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.963+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 >> 172.21.15.59:0/2385860063 conn(0x7fd9140b21a0 msgr2=0x7fd9140d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.964+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 shutdown_connections 2024-04-24T06:39:01.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:01.964+0000 7fd91b000640 1 -- 172.21.15.59:0/2385860063 wait complete. 2024-04-24T06:39:01.985 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-24T06:39:02.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.092+0000 7fd3661dbf40 1 Processor -- start 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 -- start start 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4dd5a10 0x559bb4defba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d06be0 0x559bb4d49a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb4d1c9c0 con 0x559bb4d49f40 2024-04-24T06:39:02.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559bb4cc3cc0 con 0x559bb4dd5a10 2024-04-24T06:39:02.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.093+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559bb4d1ab20 con 0x559bb4d06be0 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39026/0 (socket says 172.21.15.59:39026) 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 -- 172.21.15.59:0/1885136391 learned_addr learned my addr 172.21.15.59:0/1885136391 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd362a00640 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d06be0 0x559bb4d49a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363400640 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4dd5a10 0x559bb4defba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 -- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d06be0 msgr2=0x559bb4d49a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d06be0 0x559bb4d49a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 -- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4dd5a10 msgr2=0x559bb4defba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4dd5a10 0x559bb4defba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.094+0000 7fd363e00640 1 -- 172.21.15.59:0/1885136391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb4df1360 con 0x559bb4d49f40 2024-04-24T06:39:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd363e00640 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e320 tx=0x7fd35c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9ddd8d0402f8cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd361600640 1 -- 172.21.15.59:0/1885136391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c016020 con 0x559bb4d49f40 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd361600640 1 -- 172.21.15.59:0/1885136391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c012600 con 0x559bb4d49f40 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd361600640 1 -- 172.21.15.59:0/1885136391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0129f0 con 0x559bb4d49f40 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1885136391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 msgr2=0x559bb4d68ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.095+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e320 tx=0x7fd35c005f90 comp rx=0 tx=0).stop 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1885136391 shutdown_connections 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4d49f40 0x559bb4d68ad0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d06be0 0x559bb4d49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1885136391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4dd5a10 0x559bb4defba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1885136391 >> 172.21.15.59:0/1885136391 conn(0x559bb4d66cc0 msgr2=0x559bb4df3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1885136391 shutdown_connections 2024-04-24T06:39:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.096+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1885136391 wait complete. 2024-04-24T06:39:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.097+0000 7fd3661dbf40 1 Processor -- start 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.097+0000 7fd3661dbf40 1 -- start start 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4d06be0 0x559bb4debc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d49f40 0x559bb4dcd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 0x559bb4dcd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559bb4d65e80 con 0x559bb4dd5a10 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559bb4d659f0 con 0x559bb4d06be0 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd3661dbf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559bb4d49c30 con 0x559bb4d49f40 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 0x559bb4dcd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 0x559bb4dcd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39040/0 (socket says 172.21.15.59:39040) 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd363e00640 1 -- 172.21.15.59:0/1651537996 learned_addr learned my addr 172.21.15.59:0/1651537996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd363400640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4d06be0 0x559bb4debc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd362a00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d49f40 0x559bb4dcd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.098+0000 7fd363e00640 1 -- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d49f40 msgr2=0x559bb4dcd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd363e00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559bb4d49f40 0x559bb4dcd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd363e00640 1 -- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4d06be0 msgr2=0x559bb4debc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd363e00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bb4d06be0 0x559bb4debc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd363e00640 1 -- 172.21.15.59:0/1651537996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb4dd52f0 con 0x559bb4dd5a10 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd363e00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 0x559bb4dcd590 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd35c015040 tx=0x7fd35c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd35be00640 1 -- 172.21.15.59:0/1651537996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01da00 con 0x559bb4dd5a10 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd35be00640 1 -- 172.21.15.59:0/1651537996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c01dba0 con 0x559bb4dd5a10 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd35be00640 1 -- 172.21.15.59:0/1651537996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01b430 con 0x559bb4dd5a10 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559bb4dd5010 con 0x559bb4dd5a10 2024-04-24T06:39:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.099+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb4dd3270 con 0x559bb4dd5a10 2024-04-24T06:39:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.101+0000 7fd35be00640 1 -- 172.21.15.59:0/1651537996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd35c019020 con 0x559bb4dd5a10 2024-04-24T06:39:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.102+0000 7fd35be00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd34406b2d0 0x7fd34406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.102+0000 7fd35be00640 1 -- 172.21.15.59:0/1651537996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fd35c08c3b0 con 0x559bb4dd5a10 2024-04-24T06:39:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.103+0000 7fd363400640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd34406b2d0 0x7fd34406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.103+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x559bb4d4fe70 0x559bb4d64940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.103+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x559bb4d64e80 con 0x559bb4d4fe70 2024-04-24T06:39:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.103+0000 7fd362a00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x559bb4d4fe70 0x559bb4d64940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.103+0000 7fd363400640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd34406b2d0 0x7fd34406d790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd354001050 tx=0x7fd354001c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd362a00640 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x559bb4d4fe70 0x559bb4d64940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd362a00640 1 -- 172.21.15.59:0/1651537996 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fd35004f860 con 0x559bb4d4fe70 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x559bb4d4fe70 msgr2=0x559bb4d64940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x559bb4d4fe70 0x559bb4d64940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd34406b2d0 msgr2=0x7fd34406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd34406b2d0 0x7fd34406d790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd354001050 tx=0x7fd354001c10 comp rx=0 tx=0).stop 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 msgr2=0x559bb4dcd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 --2- 172.21.15.59:0/1651537996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559bb4dd5a10 0x559bb4dcd590 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd35c015040 tx=0x7fd35c003330 comp rx=0 tx=0).stop 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd363e00640 1 -- 172.21.15.59:0/1651537996 reap_dead start 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 shutdown_connections 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.104+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 >> 172.21.15.59:0/1651537996 conn(0x559bb4d66cc0 msgr2=0x559bb4d68ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.105+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 shutdown_connections 2024-04-24T06:39:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.105+0000 7fd3661dbf40 1 -- 172.21.15.59:0/1651537996 wait complete. 2024-04-24T06:39:02.111 INFO:teuthology.orchestra.run.smithi059.stdout:����� 2024-04-24T06:39:02.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:39:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.309+0000 7fed3b200640 1 Processor -- start 2024-04-24T06:39:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.309+0000 7fed3b200640 1 -- start start 2024-04-24T06:39:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.310+0000 7fed3b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed34160960 0x7fed34163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.310+0000 7fed3b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed341644b0 0x7fed340b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.310+0000 7fed3b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 0x7fed340b4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed3b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fed34164270 con 0x7fed34160960 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed3b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fed34160d40 con 0x7fed341668e0 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed3b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fed34166de0 con 0x7fed341644b0 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed33e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed34160960 0x7fed34163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed33e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed34160960 0x7fed34163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39042/0 (socket says 172.21.15.59:39042) 2024-04-24T06:39:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed33e00640 1 -- 172.21.15.59:0/581380656 learned_addr learned my addr 172.21.15.59:0/581380656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed39200640 1 --2- 172.21.15.59:0/581380656 >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 0x7fed340b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.311+0000 7fed33400640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed341644b0 0x7fed340b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed33e00640 1 -- 172.21.15.59:0/581380656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed341644b0 msgr2=0x7fed340b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed33e00640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed341644b0 0x7fed340b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed33e00640 1 -- 172.21.15.59:0/581380656 >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 msgr2=0x7fed340b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed33e00640 1 --2- 172.21.15.59:0/581380656 >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 0x7fed340b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed33e00640 1 -- 172.21.15.59:0/581380656 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed341670c0 con 0x7fed34160960 2024-04-24T06:39:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.312+0000 7fed39200640 1 --2- 172.21.15.59:0/581380656 >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 0x7fed340b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:39:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.313+0000 7fed33e00640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed34160960 0x7fed34163d30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fed3406c7c0 tx=0x7fed1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c490925f4cb44693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed32a00640 1 -- 172.21.15.59:0/581380656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c037070 con 0x7fed34160960 2024-04-24T06:39:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed32a00640 1 -- 172.21.15.59:0/581380656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c032440 con 0x7fed34160960 2024-04-24T06:39:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed32a00640 1 -- 172.21.15.59:0/581380656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c032810 con 0x7fed34160960 2024-04-24T06:39:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 -- 172.21.15.59:0/581380656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed34160960 msgr2=0x7fed34163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed34160960 0x7fed34163d30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fed3406c7c0 tx=0x7fed1c02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 -- 172.21.15.59:0/581380656 shutdown_connections 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 --2- 172.21.15.59:0/581380656 >> v2:172.21.15.66:3300/0 conn(0x7fed341668e0 0x7fed340b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed341644b0 0x7fed340b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 --2- 172.21.15.59:0/581380656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed34160960 0x7fed34163d30 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 -- 172.21.15.59:0/581380656 >> 172.21.15.59:0/581380656 conn(0x7fed34067ea0 msgr2=0x7fed3406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.314+0000 7fed3b200640 1 -- 172.21.15.59:0/581380656 shutdown_connections 2024-04-24T06:39:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.315+0000 7fed3b200640 1 -- 172.21.15.59:0/581380656 wait complete. 2024-04-24T06:39:02.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.316+0000 7fed3b200640 1 Processor -- start 2024-04-24T06:39:02.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.316+0000 7fed3b200640 1 -- start start 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed34160960 0x7fed341516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 0x7fed340b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed34071de0 con 0x7fed341668e0 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed34167680 con 0x7fed341644b0 2024-04-24T06:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.317+0000 7fed3b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fed34071f60 con 0x7fed34160960 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39054/0 (socket says 172.21.15.59:39054) 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 -- 172.21.15.59:0/3496154519 learned_addr learned my addr 172.21.15.59:0/3496154519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed33e00640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed34160960 0x7fed341516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed33400640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 0x7fed340b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 -- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed34160960 msgr2=0x7fed341516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed34160960 0x7fed341516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 -- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 msgr2=0x7fed340b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 0x7fed340b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed39200640 1 -- 172.21.15.59:0/3496154519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed340ce5e0 con 0x7fed341668e0 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.318+0000 7fed33400640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 0x7fed340b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed39200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d5d0 tx=0x7fed2c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0125f0 con 0x7fed341668e0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2c019070 con 0x7fed341668e0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0129b0 con 0x7fed341668e0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed340ce8c0 con 0x7fed341668e0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.319+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed340cee80 con 0x7fed341668e0 2024-04-24T06:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.320+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed34068a80 con 0x7fed341668e0 2024-04-24T06:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.321+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed2c012b50 con 0x7fed341668e0 2024-04-24T06:39:02.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.322+0000 7fed2be00640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:02.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.322+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fed2c08be90 con 0x7fed341668e0 2024-04-24T06:39:02.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.322+0000 7fed33e00640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.323+0000 7fed33e00640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 0x7fed1006edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed1c034000 tx=0x7fed1c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:02.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.326+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed2c011340 con 0x7fed341668e0 2024-04-24T06:39:02.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.573+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fed3406e260 con 0x7fed341668e0 2024-04-24T06:39:02.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.574+0000 7fed2be00640 1 -- 172.21.15.59:0/3496154519 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7fed2c08d070 con 0x7fed341668e0 2024-04-24T06:39:02.576 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 msgr2=0x7fed1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 0x7fed1006edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed1c034000 tx=0x7fed1c02e000 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 msgr2=0x7fed340b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d5d0 tx=0x7fed2c00da90 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 shutdown_connections 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fed1006c8f0 0x7fed1006edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed341668e0 0x7fed340b3cd0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed341644b0 0x7fed340b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 --2- 172.21.15.59:0/3496154519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fed34160960 0x7fed341516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.577+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 >> 172.21.15.59:0/3496154519 conn(0x7fed34067ea0 msgr2=0x7fed3406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.578+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 shutdown_connections 2024-04-24T06:39:02.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02.578+0000 7fed3b200640 1 -- 172.21.15.59:0/3496154519 wait complete. 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:02.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:39:02.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:39:02.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:39:02.598 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:39:02.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:02.598 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:39:02.644 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:39:02.644 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:39:02.644 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:39:02.644 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.644 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:39:02.647 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:02.647 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:39:02.647 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:39:02.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:02.647 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:39:02.770 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:39:02.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:02.775 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:39:02.831 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:39:02.832 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:39:02.832 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:39:02.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:39:02.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:39:02.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:39:02.833 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:39:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:39:03.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:03.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:03.055 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:39:03.107 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:39:03.107 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:39:03.107 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:39:03.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:03.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:39:03.188 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:39:03.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:39:03.188 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:03.245 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:39:03.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:39:03.245 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:03.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:39:03.387 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:39:03.447 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:39:03.448 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:03.449 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:03.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:03.450 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:39:03.502 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:39:03.502 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:39:03.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:39:03.582 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:39:03.631 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:39:03.631 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:39:03.632 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:39:03.633 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:39:03.634 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:39:03.634 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:03.634 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:03.634 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:03.634 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:03.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:03.635 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:39:03.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:39:03.680+0000 7ff965c86480 -1 init, newargv = 0x561dd50cd6b0 newargc=15 2024-04-24T06:39:03.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70872]: starting ceph client 2024-04-24T06:39:03.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70872]: starting fuse 2024-04-24T06:39:04.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:39:04.874 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:39:04.935 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:39:04.936 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:39:04.937 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:04.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:04.938 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:39:04.996 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:39:04.996 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:39:04.996 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:39:04.997 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:04.998 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:39:04.998 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:39:05.180 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.70872.asok 2024-04-24T06:39:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:05.199 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70872.asok 2024-04-24T06:39:05.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:05.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70872.asok status 2024-04-24T06:39:05.438 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:05.438 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:39:05.438 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:39:05.438 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "70872", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8748, 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8748 2024-04-24T06:39:05.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3355905812 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3355905812 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8748 192.168.0.1:0/3355905812", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3355905812", 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:39:05.440 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 83, 2024-04-24T06:39:05.441 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 98, 2024-04-24T06:39:05.441 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:39:05.441 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:39:05.441 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:39:05.441 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:05.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:05.453 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:39:05.454 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:05.509 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:39:05.509 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:05.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:05.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:05.599 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:39:05.599 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:39:05.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:05.654 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-24T06:39:05.910 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-24T06:39:05.911 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-24T06:39:05.911 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.18959 s, 33.2 MB/s 2024-04-24T06:39:05.913 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:39:05.913 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:39:05.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:05.968 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-24T06:39:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-24T06:39:06.191 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-24T06:39:06.192 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.161809 s, 38.9 MB/s 2024-04-24T06:39:06.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:06.194 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-24T06:39:06.262 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-24T06:39:06.263 INFO:teuthology.orchestra.run.smithi059.stdout:./file2_sixmegs 2024-04-24T06:39:06.263 INFO:teuthology.orchestra.run.smithi059.stdout:./file3_sixmegs 2024-04-24T06:39:06.263 INFO:teuthology.orchestra.run.smithi059.stdout:./file1_sixmegs 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-24T06:39:06.264 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:39:06.398 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:39:06.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-24T06:39:06.413 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:39:06.609 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:39:06.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:06.625 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:39:06.626 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:39:06.626 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:39:06.626 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:06.626 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-24T06:39:06.626 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:39:06.820 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-24T06:39:06.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:06.835 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:39:06.835 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:39:06.836 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:39:06.836 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:39:06.836 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-24T06:39:06.836 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:39:07.029 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:39:07.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:39:07.044 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:39:07.045 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:39:07.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:39:07.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.353+0000 7f037a400640 1 Processor -- start 2024-04-24T06:39:07.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.353+0000 7f037a400640 1 -- start start 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f037a400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03740b4130 0x7f037406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f037a400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f03740cfb90 0x7f03740c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f037a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037406ed40 con 0x7f0374162df0 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f037a400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f037406eaf0 con 0x7f03740b4130 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f037a400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f03740b3590 con 0x7f03740cfb90 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f0373e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03740b4130 0x7f037406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f0373e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03740b4130 0x7f037406f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39056/0 (socket says 172.21.15.59:39056) 2024-04-24T06:39:07.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.355+0000 7f0373e00640 1 -- 172.21.15.59:0/85036256 learned_addr learned my addr 172.21.15.59:0/85036256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:07.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.356+0000 7f0373e00640 1 -- 172.21.15.59:0/85036256 >> v2:172.21.15.66:3300/0 conn(0x7f03740cfb90 msgr2=0x7f03740c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:07.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.356+0000 7f0379a00640 1 --2- 172.21.15.59:0/85036256 >> v2:172.21.15.66:3300/0 conn(0x7f03740cfb90 0x7f03740c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.356+0000 7f0373e00640 1 --2- 172.21.15.59:0/85036256 >> v2:172.21.15.66:3300/0 conn(0x7f03740cfb90 0x7f03740c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.356+0000 7f0373e00640 1 -- 172.21.15.59:0/85036256 >> v1:172.21.15.66:6789/0 conn(0x7f0374162df0 legacy=0x7f03740b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:07.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.356+0000 7f0373e00640 1 -- 172.21.15.59:0/85036256 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03740b3220 con 0x7f03740b4130 2024-04-24T06:39:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.357+0000 7f0373e00640 1 --2- 172.21.15.59:0/85036256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03740b4130 0x7f037406f680 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f037406f2c0 tx=0x7f036402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341581360f56b455 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.357+0000 7f0373400640 1 -- 172.21.15.59:0/85036256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364037070 con 0x7f03740b4130 2024-04-24T06:39:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f0373400640 1 -- 172.21.15.59:0/85036256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0364032440 con 0x7f03740b4130 2024-04-24T06:39:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f0373400640 1 -- 172.21.15.59:0/85036256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364032810 con 0x7f03740b4130 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 -- 172.21.15.59:0/85036256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03740b4130 msgr2=0x7f037406f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 --2- 172.21.15.59:0/85036256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03740b4130 0x7f037406f680 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f037406f2c0 tx=0x7f036402a230 comp rx=0 tx=0).stop 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 -- 172.21.15.59:0/85036256 shutdown_connections 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 --2- 172.21.15.59:0/85036256 >> v2:172.21.15.66:3300/0 conn(0x7f03740cfb90 0x7f03740c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 --2- 172.21.15.59:0/85036256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03740b4130 0x7f037406f680 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 -- 172.21.15.59:0/85036256 >> 172.21.15.59:0/85036256 conn(0x7f03740b21a0 msgr2=0x7f03740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:07.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 -- 172.21.15.59:0/85036256 shutdown_connections 2024-04-24T06:39:07.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.358+0000 7f037a400640 1 -- 172.21.15.59:0/85036256 wait complete. 2024-04-24T06:39:07.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.360+0000 7f037a400640 1 Processor -- start 2024-04-24T06:39:07.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.360+0000 7f037a400640 1 -- start start 2024-04-24T06:39:07.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 0x7f0374165c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f03740cfb90 0x7f03740c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03740c3860 con 0x7f0374162df0 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03740c36e0 con 0x7f03740b4130 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.361+0000 7f037a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f03740c39e0 con 0x7f03740cfb90 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39070/0 (socket says 172.21.15.59:39070) 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 -- 172.21.15.59:0/2270075128 learned_addr learned my addr 172.21.15.59:0/2270075128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379000640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 0x7f0374165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0373e00640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f03740cfb90 0x7f03740c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 -- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f03740cfb90 msgr2=0x7f03740c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f03740cfb90 0x7f03740c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 -- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 msgr2=0x7f0374165c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 0x7f0374165c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379a00640 1 -- 172.21.15.59:0/2270075128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0374073260 con 0x7f0374162df0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.362+0000 7f0379000640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 0x7f0374165c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f0379a00640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f036c00cd70 tx=0x7f036c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c003b90 con 0x7f0374162df0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036c003d30 con 0x7f0374162df0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c004240 con 0x7f0374162df0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03740c3f30 con 0x7f0374162df0 2024-04-24T06:39:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.363+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03740c44f0 con 0x7f0374162df0 2024-04-24T06:39:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.364+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f037404fd90 con 0x7f0374162df0 2024-04-24T06:39:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.365+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f036c0043e0 con 0x7f0374162df0 2024-04-24T06:39:07.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.366+0000 7f0370c00640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.366+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f036c08c690 con 0x7f0374162df0 2024-04-24T06:39:07.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.367+0000 7f0379000640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.367+0000 7f0379000640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 0x7f035806edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0368006fd0 tx=0x7f0368008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.373+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f036c01d340 con 0x7f0374162df0 2024-04-24T06:39:07.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.578+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03740cc510 con 0x7f0374162df0 2024-04-24T06:39:07.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.579+0000 7f0370c00640 1 -- 172.21.15.59:0/2270075128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16524 (secure 0 0 0) 0x7f036c05aab0 con 0x7f0374162df0 2024-04-24T06:39:07.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:07.581 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:40.185327+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:39:07.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 msgr2=0x7f035806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 0x7f035806edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0368006fd0 tx=0x7f0368008040 comp rx=0 tx=0).stop 2024-04-24T06:39:07.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 msgr2=0x7f0374164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f036c00cd70 tx=0x7f036c00a660 comp rx=0 tx=0).stop 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 shutdown_connections 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0374162df0 0x7f0374164f60 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f03740cfb90 0x7f03740c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 --2- 172.21.15.59:0/2270075128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03740b4130 0x7f0374165c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.582+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 >> 172.21.15.59:0/2270075128 conn(0x7f03740b21a0 msgr2=0x7f03740b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.583+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 shutdown_connections 2024-04-24T06:39:07.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.583+0000 7f037a400640 1 -- 172.21.15.59:0/2270075128 wait complete. 2024-04-24T06:39:07.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:38:36.464123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:38:37.616685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:39:07.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:39:07.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.808+0000 7f5865200640 1 Processor -- start 2024-04-24T06:39:07.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.808+0000 7f5865200640 1 -- start start 2024-04-24T06:39:07.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.809+0000 7f5865200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f586015ed20 0x7f58600d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.809+0000 7f5865200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58600b5da0 con 0x7f58600b3600 2024-04-24T06:39:07.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.809+0000 7f5865200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58600d0a20 con 0x7f58600d13e0 2024-04-24T06:39:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.809+0000 7f5865200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f586015f100 con 0x7f586015ed20 2024-04-24T06:39:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.810+0000 7f585ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f58600b3600 0x7f58600b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48864/0 (socket says 172.21.15.59:48864) 2024-04-24T06:39:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.810+0000 7f585ea00640 1 -- 172.21.15.59:0/536902573 learned_addr learned my addr 172.21.15.59:0/536902573 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.810+0000 7f585f400640 1 --2- 172.21.15.59:0/536902573 >> v2:172.21.15.66:3300/0 conn(0x7f586015ed20 0x7f58600d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.810+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156296244 0 0) 0x7f58600b5da0 con 0x7f58600b3600 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.810+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5840003520 con 0x7f58600b3600 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4290295892 0 0) 0x7f5840003520 con 0x7f58600b3600 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 >> v2:172.21.15.66:3300/0 conn(0x7f586015ed20 msgr2=0x7f58600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 --2- 172.21.15.59:0/536902573 >> v2:172.21.15.66:3300/0 conn(0x7f586015ed20 0x7f58600d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 >> v1:172.21.15.66:6789/0 conn(0x7f58600d13e0 legacy=0x7f58600b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58600d2ed0 con 0x7f58600b3600 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f58480019f0 con 0x7f58600b3600 2024-04-24T06:39:07.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 >> v1:172.21.15.59:6789/0 conn(0x7f58600b3600 legacy=0x7f58600b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840003a00 0x7f5840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 0x7f5840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58400053c0 0x7f584000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f584000c270 con 0x7f5840004800 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f584000c420 con 0x7f58400053c0 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f584000c620 con 0x7f5840003a00 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.811+0000 7f585ea00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 0x7f5840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585f400640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840003a00 0x7f5840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585ea00640 1 -- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840003a00 msgr2=0x7f5840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585ea00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840003a00 0x7f5840003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585ea00640 1 -- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58400053c0 msgr2=0x7f584000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585fe00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58400053c0 0x7f584000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585ea00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58400053c0 0x7f584000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585ea00640 1 -- 172.21.15.59:0/536902573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5840003520 con 0x7f5840004800 2024-04-24T06:39:07.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585fe00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58400053c0 0x7f584000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:07.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.812+0000 7f585fe00640 1 -- 172.21.15.59:0/536902573 reap_dead start 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.813+0000 7f585ea00640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 0x7f5840004c80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f584800a7b0 tx=0x7f5840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b89f4218dd22e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584800a3d0 con 0x7f5840004800 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584802f070 con 0x7f5840004800 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f585e000640 1 -- 172.21.15.59:0/536902573 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584802bc80 con 0x7f5840004800 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 -- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 msgr2=0x7f5840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 0x7f5840004c80 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f584800a7b0 tx=0x7f5840005830 comp rx=0 tx=0).stop 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 -- 172.21.15.59:0/536902573 shutdown_connections 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 --2- 172.21.15.59:0/536902573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5840004800 0x7f5840004c80 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 -- 172.21.15.59:0/536902573 >> 172.21.15.59:0/536902573 conn(0x7f586008bf90 msgr2=0x7f58600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.814+0000 7f5865200640 1 -- 172.21.15.59:0/536902573 shutdown_connections 2024-04-24T06:39:07.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.815+0000 7f5865200640 1 -- 172.21.15.59:0/536902573 wait complete. 2024-04-24T06:39:07.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.816+0000 7f5865200640 1 Processor -- start 2024-04-24T06:39:07.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.816+0000 7f5865200640 1 -- start start 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 0x7f58600cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58601606d0 0x7f5860158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58600d0ea0 con 0x7f58600b3f10 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58600d3490 con 0x7f58601606d0 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.817+0000 7f5865200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f58600d1020 con 0x7f5840004800 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39098/0 (socket says 172.21.15.59:39098) 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 -- 172.21.15.59:0/1795593037 learned_addr learned my addr 172.21.15.59:0/1795593037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585ea00640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58601606d0 0x7f5860158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585f400640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 0x7f58600cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 -- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 msgr2=0x7f58600cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 0x7f58600cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 -- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58601606d0 msgr2=0x7f5860158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58601606d0 0x7f5860158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585fe00640 1 -- 172.21.15.59:0/1795593037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860160b50 con 0x7f58600b3f10 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.818+0000 7f585f400640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 0x7f58600cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f585fe00640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f586008e600 tx=0x7f585800d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858018070 con 0x7f58600b3f10 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58580035f0 con 0x7f58600b3f10 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580039e0 con 0x7f58600b3f10 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5860160e20 con 0x7f58600b3f10 2024-04-24T06:39:07.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.819+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601613e0 con 0x7f58600b3f10 2024-04-24T06:39:07.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.820+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586004fd90 con 0x7f58600b3f10 2024-04-24T06:39:07.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.821+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f585801b070 con 0x7f58600b3f10 2024-04-24T06:39:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.822+0000 7f5857400640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.822+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f585808b0d0 con 0x7f58600b3f10 2024-04-24T06:39:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.822+0000 7f585f400640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.823+0000 7f585f400640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 0x7f583c06edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5850006fd0 tx=0x7f5850008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:07.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:07.826+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58580594f0 con 0x7f58600b3f10 2024-04-24T06:39:08.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:39:08.030 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:39:08.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.036+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f586015d2f0 con 0x7f58600b3f10 2024-04-24T06:39:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.036+0000 7f5857400640 1 -- 172.21.15.59:0/1795593037 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f5858058df0 con 0x7f58600b3f10 2024-04-24T06:39:08.037 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 msgr2=0x7f583c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 0x7f583c06edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5850006fd0 tx=0x7f5850008040 comp rx=0 tx=0).stop 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 msgr2=0x7f58600b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f586008e600 tx=0x7f585800d680 comp rx=0 tx=0).stop 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 shutdown_connections 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f583c06c8f0 0x7f583c06edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58600b3f10 0x7f58600b4370 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58601606d0 0x7f5860158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 --2- 172.21.15.59:0/1795593037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5840004800 0x7f58600cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 >> 172.21.15.59:0/1795593037 conn(0x7f586008bf90 msgr2=0x7f586006d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 shutdown_connections 2024-04-24T06:39:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.039+0000 7f5865200640 1 -- 172.21.15.59:0/1795593037 wait complete. 2024-04-24T06:39:08.059 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:39:08.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.300+0000 7fcaaa400640 1 Processor -- start 2024-04-24T06:39:08.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.301+0000 7fcaaa400640 1 -- start start 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.302+0000 7fcaaa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa415ed20 0x7fcaa40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.302+0000 7fcaaa400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaa40d13e0 0x7fcaa40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.302+0000 7fcaaa400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa40b5da0 con 0x7fcaa40b3600 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.302+0000 7fcaaa400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa415f100 con 0x7fcaa40d13e0 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.302+0000 7fcaaa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa40d07d0 con 0x7fcaa415ed20 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fca9be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcaa40b3600 0x7fcaa40b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48870/0 (socket says 172.21.15.59:48870) 2024-04-24T06:39:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fca9be00640 1 -- 172.21.15.59:0/3458446281 learned_addr learned my addr 172.21.15.59:0/3458446281 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9a00640 1 --2- 172.21.15.59:0/3458446281 >> v2:172.21.15.59:3300/0 conn(0x7fcaa40d13e0 0x7fcaa40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9000640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa415ed20 0x7fcaa40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9a00640 1 -- 172.21.15.59:0/3458446281 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa415ed20 msgr2=0x7fcaa40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9a00640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa415ed20 0x7fcaa40d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9a00640 1 -- 172.21.15.59:0/3458446281 >> v1:172.21.15.59:6789/0 conn(0x7fcaa40b3600 legacy=0x7fcaa40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.303+0000 7fcaa9a00640 1 -- 172.21.15.59:0/3458446281 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa40d0a20 con 0x7fcaa40d13e0 2024-04-24T06:39:08.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.304+0000 7fcaa9a00640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d13e0 0x7fcaa40d0290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009d90 tx=0x7fca9c02a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3591464ed36322 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaa3e00640 1 -- 172.21.15.59:0/3458446281 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c007740 con 0x7fcaa40d13e0 2024-04-24T06:39:08.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaa3e00640 1 -- 172.21.15.59:0/3458446281 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9c0078e0 con 0x7fcaa40d13e0 2024-04-24T06:39:08.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaa3e00640 1 -- 172.21.15.59:0/3458446281 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c007cb0 con 0x7fcaa40d13e0 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 -- 172.21.15.59:0/3458446281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d13e0 msgr2=0x7fcaa40d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d13e0 0x7fcaa40d0290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009d90 tx=0x7fca9c02a540 comp rx=0 tx=0).stop 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 -- 172.21.15.59:0/3458446281 shutdown_connections 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d13e0 0x7fcaa40d0290 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 --2- 172.21.15.59:0/3458446281 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa415ed20 0x7fcaa40d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.305+0000 7fcaaa400640 1 -- 172.21.15.59:0/3458446281 >> 172.21.15.59:0/3458446281 conn(0x7fcaa408bf90 msgr2=0x7fcaa40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.306+0000 7fcaaa400640 1 -- 172.21.15.59:0/3458446281 shutdown_connections 2024-04-24T06:39:08.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.306+0000 7fcaaa400640 1 -- 172.21.15.59:0/3458446281 wait complete. 2024-04-24T06:39:08.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.307+0000 7fcaaa400640 1 Processor -- start 2024-04-24T06:39:08.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.307+0000 7fcaaa400640 1 -- start start 2024-04-24T06:39:08.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa40d13e0 0x7fcaa4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 0x7fcaa40c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40d3490 con 0x7fcaa40b3600 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa40d31b0 con 0x7fcaa415ed20 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.308+0000 7fcaaa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa40b4620 con 0x7fcaa40d13e0 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39116/0 (socket says 172.21.15.59:39116) 2024-04-24T06:39:08.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 -- 172.21.15.59:0/2491042742 learned_addr learned my addr 172.21.15.59:0/2491042742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:08.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9a00640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 0x7fcaa40c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 -- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa40d13e0 msgr2=0x7fcaa4154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fca9be00640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa40d13e0 0x7fcaa4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa40d13e0 0x7fcaa4154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 -- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 msgr2=0x7fcaa40c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 0x7fcaa40c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9000640 1 -- 172.21.15.59:0/2491042742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa40a0ff0 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.309+0000 7fcaa9a00640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 0x7fcaa40c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaa9000640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d5f0 tx=0x7fca9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940125f0 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca94019070 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940129d0 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa40a12d0 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.310+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa40a1890 con 0x7fcaa40b3600 2024-04-24T06:39:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.311+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa404fd90 con 0x7fcaa40b3600 2024-04-24T06:39:08.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.312+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca94012b70 con 0x7fcaa40b3600 2024-04-24T06:39:08.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.314+0000 7fcaa1600640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 0x7fca8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.314+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fca9408bf40 con 0x7fcaa40b3600 2024-04-24T06:39:08.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.314+0000 7fca9be00640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 0x7fca8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.315+0000 7fca9be00640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 0x7fca8006edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcaa41f7f80 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.318+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca9405a360 con 0x7fcaa40b3600 2024-04-24T06:39:08.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.520+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcaa40b43e0 con 0x7fcaa40b3600 2024-04-24T06:39:08.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.521+0000 7fcaa1600640 1 -- 172.21.15.59:0/2491042742 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fca94059c60 con 0x7fcaa40b3600 2024-04-24T06:39:08.522 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:39:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 msgr2=0x7fca8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 0x7fca8006edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcaa41f7f80 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 msgr2=0x7fcaa40a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d5f0 tx=0x7fca9400dab0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 shutdown_connections 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fca8006c8f0 0x7fca8006edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcaa415ed20 0x7fcaa40c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcaa40d13e0 0x7fcaa4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 --2- 172.21.15.59:0/2491042742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40b3600 0x7fcaa40a0950 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.523+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 >> 172.21.15.59:0/2491042742 conn(0x7fcaa408bf90 msgr2=0x7fcaa40b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.524+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 shutdown_connections 2024-04-24T06:39:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.524+0000 7fcaaa400640 1 -- 172.21.15.59:0/2491042742 wait complete. 2024-04-24T06:39:08.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:39:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.760+0000 7f6fb7e00640 1 Processor -- start 2024-04-24T06:39:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.761+0000 7f6fb7e00640 1 -- start start 2024-04-24T06:39:08.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.762+0000 7f6fb7e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6fb80d0d10 0x7f6fb8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.762+0000 7f6fb7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb815cab0 con 0x7f6fb81567e0 2024-04-24T06:39:08.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.762+0000 7f6fb7e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb8160e40 con 0x7f6fb815d810 2024-04-24T06:39:08.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.762+0000 7f6fb7e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb815cd30 con 0x7f6fb80d0d10 2024-04-24T06:39:08.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.763+0000 7f6fb6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6fb81567e0 0x7f6fb80d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48880/0 (socket says 172.21.15.59:48880) 2024-04-24T06:39:08.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.763+0000 7f6fb6a00640 1 -- 172.21.15.59:0/430485836 learned_addr learned my addr 172.21.15.59:0/430485836 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:08.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.763+0000 7f6fafe00640 1 --2- 172.21.15.59:0/430485836 >> v2:172.21.15.66:3300/0 conn(0x7f6fb80d0d10 0x7f6fb8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.763+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751213513 0 0) 0x7f6fa8001110 con 0x7f6fb81567e0 2024-04-24T06:39:08.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.763+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9c003520 con 0x7f6fb81567e0 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2610840724 0 0) 0x7f6f9c003520 con 0x7f6fb81567e0 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 >> v2:172.21.15.66:3300/0 conn(0x7f6fb80d0d10 msgr2=0x7f6fb8156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 --2- 172.21.15.59:0/430485836 >> v2:172.21.15.66:3300/0 conn(0x7f6fb80d0d10 0x7f6fb8156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 >> v1:172.21.15.66:6789/0 conn(0x7f6fb815d810 legacy=0x7f6fb8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb815d010 con 0x7f6fb81567e0 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6fa8001840 con 0x7f6fb81567e0 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 >> v1:172.21.15.59:6789/0 conn(0x7f6fb81567e0 legacy=0x7f6fb80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c003a00 0x7f6f9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 0x7f6f9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f9c0051c0 0x7f6f9c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb815cd30 con 0x7f6f9c004800 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c0104d0 con 0x7f6f9c0051c0 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9c0106d0 con 0x7f6f9c003a00 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.764+0000 7f6fafe00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 0x7f6f9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fb7400640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f9c0051c0 0x7f6f9c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fb6a00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c003a00 0x7f6f9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fafe00640 1 -- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c003a00 msgr2=0x7f6f9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fafe00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c003a00 0x7f6f9c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fafe00640 1 -- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f9c0051c0 msgr2=0x7f6f9c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fafe00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f9c0051c0 0x7f6f9c00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fafe00640 1 -- 172.21.15.59:0/430485836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c003520 con 0x7f6f9c004800 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fb7400640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f9c0051c0 0x7f6f9c00fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:08.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.765+0000 7f6fb7400640 1 -- 172.21.15.59:0/430485836 reap_dead start 2024-04-24T06:39:08.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.766+0000 7f6fafe00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 0x7f6f9c004c80 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80afdb0 tx=0x7f6f9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31e6455d423c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.766+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0036070 con 0x7f6f9c004800 2024-04-24T06:39:08.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.766+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa0031400 con 0x7f6f9c004800 2024-04-24T06:39:08.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.766+0000 7f6fb6000640 1 -- 172.21.15.59:0/430485836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0009690 con 0x7f6f9c004800 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 -- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 msgr2=0x7f6f9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 0x7f6f9c004c80 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80afdb0 tx=0x7f6f9c005830 comp rx=0 tx=0).stop 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 -- 172.21.15.59:0/430485836 shutdown_connections 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/430485836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f9c004800 0x7f6f9c004c80 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 -- 172.21.15.59:0/430485836 >> 172.21.15.59:0/430485836 conn(0x7f6fb806bdf0 msgr2=0x7f6fb806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 -- 172.21.15.59:0/430485836 shutdown_connections 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.767+0000 7f6fb7e00640 1 -- 172.21.15.59:0/430485836 wait complete. 2024-04-24T06:39:08.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.769+0000 7f6fb7e00640 1 Processor -- start 2024-04-24T06:39:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.769+0000 7f6fb7e00640 1 -- start start 2024-04-24T06:39:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c004800 0x7f6fb8142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 0x7f6fb8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8156d40 con 0x7f6fb8151dc0 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8156bc0 con 0x7f6fb8154fa0 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb8156ec0 con 0x7f6f9c004800 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39132/0 (socket says 172.21.15.59:39132) 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.770+0000 7f6fb7400640 1 -- 172.21.15.59:0/2218676222 learned_addr learned my addr 172.21.15.59:0/2218676222 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fafe00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 0x7f6fb8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb6a00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c004800 0x7f6fb8142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 -- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c004800 msgr2=0x7f6fb8142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c004800 0x7f6fb8142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 -- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 msgr2=0x7f6fb8155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 0x7f6fb8155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 -- 172.21.15.59:0/2218676222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb8150b20 con 0x7f6fb8151dc0 2024-04-24T06:39:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fafe00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 0x7f6fb8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6fb7400640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815a680 tx=0x7f6fb000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.771+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0018070 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.772+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb8150e00 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.772+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb0003da0 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.772+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00052b0 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.772+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb81513c0 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.773+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb804fd90 con 0x7f6fb8151dc0 2024-04-24T06:39:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.774+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fb001b070 con 0x7f6fb8151dc0 2024-04-24T06:39:08.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.775+0000 7f6faea00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:08.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.775+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb008b290 con 0x7f6fb8151dc0 2024-04-24T06:39:08.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.775+0000 7f6fb6a00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:08.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.776+0000 7f6fb6a00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000ff0 tx=0x7f6fa8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:08.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.779+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fb0059620 con 0x7f6fb8151dc0 2024-04-24T06:39:08.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.981+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6fb80b5150 con 0x7f6fb8151dc0 2024-04-24T06:39:08.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.983+0000 7f6faea00640 1 -- 172.21.15.59:0/2218676222 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f6fb0058f20 con 0x7f6fb8151dc0 2024-04-24T06:39:08.983 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:39:08.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.985+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 msgr2=0x7f6f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.985+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000ff0 tx=0x7f6fa8009040 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.985+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 msgr2=0x7f6fb8152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.985+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6fb815a680 tx=0x7f6fb000d9d0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 shutdown_connections 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb8151dc0 0x7f6fb8152240 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb8154fa0 0x7f6fb8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 --2- 172.21.15.59:0/2218676222 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f6f9c004800 0x7f6fb8142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 >> 172.21.15.59:0/2218676222 conn(0x7f6fb806bdf0 msgr2=0x7f6fb815ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 shutdown_connections 2024-04-24T06:39:08.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:08.986+0000 7f6fb7e00640 1 -- 172.21.15.59:0/2218676222 wait complete. 2024-04-24T06:39:09.005 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:39:09.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.213+0000 7f59b0000640 1 Processor -- start 2024-04-24T06:39:09.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.213+0000 7f59b0000640 1 -- start start 2024-04-24T06:39:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59b0000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f59a815bc00 0x7f59a815fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59b0000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59a80ce7a0 0x7f59a80cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59b0000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a8160140 con 0x7f59a806ef00 2024-04-24T06:39:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59b0000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f59a80b2f50 con 0x7f59a80ce7a0 2024-04-24T06:39:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59b0000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f59a815c180 con 0x7f59a815bc00 2024-04-24T06:39:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ae000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59a80ce7a0 0x7f59a80cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ad600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f59a806ef00 0x7f59a806f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40530/0 (socket says 172.21.15.59:40530) 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ad600640 1 -- 172.21.15.59:0/958165496 learned_addr learned my addr 172.21.15.59:0/958165496 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59acc00640 1 --2- 172.21.15.59:0/958165496 >> v2:172.21.15.66:3300/0 conn(0x7f59a815bc00 0x7f59a815fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59a7e00640 1 -- 172.21.15.59:0/958165496 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739838158 0 0) 0x7f59a8160140 con 0x7f59a806ef00 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ae000640 1 -- 172.21.15.59:0/958165496 >> v2:172.21.15.66:3300/0 conn(0x7f59a815bc00 msgr2=0x7f59a815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ae000640 1 --2- 172.21.15.59:0/958165496 >> v2:172.21.15.66:3300/0 conn(0x7f59a815bc00 0x7f59a815fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.215+0000 7f59ae000640 1 -- 172.21.15.59:0/958165496 >> v1:172.21.15.59:6789/0 conn(0x7f59a806ef00 legacy=0x7f59a806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.216+0000 7f59ae000640 1 -- 172.21.15.59:0/958165496 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a815c460 con 0x7f59a80ce7a0 2024-04-24T06:39:09.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59ae000640 1 --2- 172.21.15.59:0/958165496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a80ce7a0 0x7f59a80cebf0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f59a806dcf0 tx=0x7f59a002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56b241704739f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59a7e00640 1 -- 172.21.15.59:0/958165496 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007740 con 0x7f59a80ce7a0 2024-04-24T06:39:09.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59a7e00640 1 -- 172.21.15.59:0/958165496 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a00078e0 con 0x7f59a80ce7a0 2024-04-24T06:39:09.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59a7e00640 1 -- 172.21.15.59:0/958165496 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007cb0 con 0x7f59a80ce7a0 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59b0000640 1 -- 172.21.15.59:0/958165496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a80ce7a0 msgr2=0x7f59a80cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.217+0000 7f59b0000640 1 --2- 172.21.15.59:0/958165496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a80ce7a0 0x7f59a80cebf0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f59a806dcf0 tx=0x7f59a002a680 comp rx=0 tx=0).stop 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 -- 172.21.15.59:0/958165496 shutdown_connections 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 --2- 172.21.15.59:0/958165496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a80ce7a0 0x7f59a80cebf0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 --2- 172.21.15.59:0/958165496 >> v2:172.21.15.66:3300/0 conn(0x7f59a815bc00 0x7f59a815fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 -- 172.21.15.59:0/958165496 >> 172.21.15.59:0/958165496 conn(0x7f59a80b2130 msgr2=0x7f59a80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 -- 172.21.15.59:0/958165496 shutdown_connections 2024-04-24T06:39:09.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.218+0000 7f59b0000640 1 -- 172.21.15.59:0/958165496 wait complete. 2024-04-24T06:39:09.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.220+0000 7f59b0000640 1 Processor -- start 2024-04-24T06:39:09.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.220+0000 7f59b0000640 1 -- start start 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59a806ef00 0x7f59a814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a80ce7a0 0x7f59a8151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59a80ce210 con 0x7f59a815bc00 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59a815ca20 con 0x7f59a80ce7a0 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59b0000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f59a8155ff0 con 0x7f59a806ef00 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59ae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59ae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60776/0 (socket says 172.21.15.59:60776) 2024-04-24T06:39:09.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.221+0000 7f59ae000640 1 -- 172.21.15.59:0/1595084949 learned_addr learned my addr 172.21.15.59:0/1595084949 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59acc00640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a80ce7a0 0x7f59a8151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ad600640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59a806ef00 0x7f59a814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 -- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59a806ef00 msgr2=0x7f59a814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59a806ef00 0x7f59a814f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 -- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a80ce7a0 msgr2=0x7f59a8151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a80ce7a0 0x7f59a8151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 -- 172.21.15.59:0/1595084949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a8151c40 con 0x7f59a815bc00 2024-04-24T06:39:09.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.222+0000 7f59ae000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f59a002ac70 tx=0x7f59a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.223+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0040070 con 0x7f59a815bc00 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.223+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0031bc0 con 0x7f59a815bc00 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.223+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0045460 con 0x7f59a815bc00 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.223+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a814f980 con 0x7f59a815bc00 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.223+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a814ff40 con 0x7f59a815bc00 2024-04-24T06:39:09.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.224+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a8068a80 con 0x7f59a815bc00 2024-04-24T06:39:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.225+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59a0039040 con 0x7f59a815bc00 2024-04-24T06:39:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.226+0000 7f59a5600640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.226+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f59a00ad500 con 0x7f59a815bc00 2024-04-24T06:39:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.227+0000 7f59ad600640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.227+0000 7f59ad600640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 0x7f598806edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f599c007fd0 tx=0x7f599c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.231+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59a007b920 con 0x7f59a815bc00 2024-04-24T06:39:09.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.449+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f59a815dff0 con 0x7f59a815bc00 2024-04-24T06:39:09.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.450+0000 7f59a5600640 1 -- 172.21.15.59:0/1595084949 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) ==== 80+0+0 (secure 0 0 0) 0x7f59a007b220 con 0x7f59a815bc00 2024-04-24T06:39:09.450 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:39:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.452+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 msgr2=0x7f598806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.452+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 0x7f598806edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f599c007fd0 tx=0x7f599c009040 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.452+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 msgr2=0x7f59a8143df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.452+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f59a002ac70 tx=0x7f59a0004a40 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 shutdown_connections 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f598806c8f0 0x7f598806edb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59a815bc00 0x7f59a8143df0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59a80ce7a0 0x7f59a8151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 --2- 172.21.15.59:0/1595084949 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59a806ef00 0x7f59a814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 >> 172.21.15.59:0/1595084949 conn(0x7f59a80b2130 msgr2=0x7f59a806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 shutdown_connections 2024-04-24T06:39:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.453+0000 7f59b0000640 1 -- 172.21.15.59:0/1595084949 wait complete. 2024-04-24T06:39:09.474 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:39:09.474 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:39:09.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:39:09.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.735+0000 7f8b2e400640 1 Processor -- start 2024-04-24T06:39:09.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.736+0000 7f8b2e400640 1 -- start start 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b2e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b280cc7e0 0x7f8b28167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b2e400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b28160210 con 0x7f8b280cd760 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b2e400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b281604f0 con 0x7f8b2815c3d0 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b2e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8b281607d0 con 0x7f8b280cc7e0 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b2da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b280cc7e0 0x7f8b28167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b27e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8b280cd760 0x7f8b280b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40540/0 (socket says 172.21.15.59:40540) 2024-04-24T06:39:09.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.737+0000 7f8b27e00640 1 -- 172.21.15.59:0/842140186 learned_addr learned my addr 172.21.15.59:0/842140186 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:09.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.738+0000 7f8b2da00640 1 -- 172.21.15.59:0/842140186 >> v1:172.21.15.66:6789/0 conn(0x7f8b2815c3d0 legacy=0x7f8b280cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:09.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.738+0000 7f8b2da00640 1 -- 172.21.15.59:0/842140186 >> v1:172.21.15.59:6789/0 conn(0x7f8b280cd760 legacy=0x7f8b280b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.738+0000 7f8b2da00640 1 -- 172.21.15.59:0/842140186 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b28160ab0 con 0x7f8b280cc7e0 2024-04-24T06:39:09.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.739+0000 7f8b2da00640 1 --2- 172.21.15.59:0/842140186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cc7e0 0x7f8b28167400 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009d90 tx=0x7f8b2002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912953319b86c140 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.739+0000 7f8b27400640 1 -- 172.21.15.59:0/842140186 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20007740 con 0x7f8b280cc7e0 2024-04-24T06:39:09.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b27400640 1 -- 172.21.15.59:0/842140186 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b200078e0 con 0x7f8b280cc7e0 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b27400640 1 -- 172.21.15.59:0/842140186 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20007cb0 con 0x7f8b280cc7e0 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 -- 172.21.15.59:0/842140186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cc7e0 msgr2=0x7f8b28167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 --2- 172.21.15.59:0/842140186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cc7e0 0x7f8b28167400 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009d90 tx=0x7f8b2002a6a0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 -- 172.21.15.59:0/842140186 shutdown_connections 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 --2- 172.21.15.59:0/842140186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cc7e0 0x7f8b28167400 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 -- 172.21.15.59:0/842140186 >> 172.21.15.59:0/842140186 conn(0x7f8b280b21a0 msgr2=0x7f8b280b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 -- 172.21.15.59:0/842140186 shutdown_connections 2024-04-24T06:39:09.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.740+0000 7f8b2e400640 1 -- 172.21.15.59:0/842140186 wait complete. 2024-04-24T06:39:09.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.742+0000 7f8b2e400640 1 Processor -- start 2024-04-24T06:39:09.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.742+0000 7f8b2e400640 1 -- start start 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b280cc7e0 0x7f8b2815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b2815c3d0 0x7f8b2815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2815c930 con 0x7f8b280cd760 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2815c7b0 con 0x7f8b2815c3d0 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b2e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8b280b4860 con 0x7f8b280cc7e0 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60780/0 (socket says 172.21.15.59:60780) 2024-04-24T06:39:09.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.743+0000 7f8b27e00640 1 -- 172.21.15.59:0/42653305 learned_addr learned my addr 172.21.15.59:0/42653305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:09.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b2da00640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b2815c3d0 0x7f8b2815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 -- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b280cc7e0 msgr2=0x7f8b2815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b280cc7e0 0x7f8b2815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 -- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b2815c3d0 msgr2=0x7f8b2815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b2815c3d0 0x7f8b2815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 -- 172.21.15.59:0/42653305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2815cff0 con 0x7f8b280cd760 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b27e00640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c400 tx=0x7f8b1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.744+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14017070 con 0x7f8b280cd760 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.745+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14005ce0 con 0x7f8b280cd760 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.745+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140072f0 con 0x7f8b280cd760 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.745+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2815adb0 con 0x7f8b280cd760 2024-04-24T06:39:09.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.745+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2815b370 con 0x7f8b280cd760 2024-04-24T06:39:09.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.746+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2804fd90 con 0x7f8b280cd760 2024-04-24T06:39:09.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.747+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b1401a070 con 0x7f8b280cd760 2024-04-24T06:39:09.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.748+0000 7f8b24c00640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.748+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f8b1408af40 con 0x7f8b280cd760 2024-04-24T06:39:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.748+0000 7f8b2d000640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.749+0000 7f8b2d000640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 0x7f8b0806edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c006fd0 tx=0x7f8b1c006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.753+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b14059360 con 0x7f8b280cd760 2024-04-24T06:39:09.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.960+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2806e6f0 con 0x7f8b280cd760 2024-04-24T06:39:09.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.962+0000 7f8b24c00640 1 -- 172.21.15.59:0/42653305 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16524 (secure 0 0 0) 0x7f8b14058c60 con 0x7f8b280cd760 2024-04-24T06:39:09.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:09.963 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:38:40.185327+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 msgr2=0x7f8b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 0x7f8b0806edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c006fd0 tx=0x7f8b1c006bc0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 msgr2=0x7f8b280c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c400 tx=0x7f8b1400c8c0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 shutdown_connections 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b2815c3d0 0x7f8b2815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b280cd760 0x7f8b280c5a20 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 --2- 172.21.15.59:0/42653305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b280cc7e0 0x7f8b2815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 >> 172.21.15.59:0/42653305 conn(0x7f8b280b21a0 msgr2=0x7f8b2806e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 shutdown_connections 2024-04-24T06:39:09.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:09.965+0000 7f8b2e400640 1 -- 172.21.15.59:0/42653305 wait complete. 2024-04-24T06:39:09.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:38:36.464123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:38:37.616685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:39:09.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:39:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.199+0000 7f411c600640 1 Processor -- start 2024-04-24T06:39:10.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.199+0000 7f411c600640 1 -- start start 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.201+0000 7f411c600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f411415e960 0x7f41140d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f411c600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41141631c0 con 0x7f4114162c80 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f411c600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41140d29b0 con 0x7f41140d1720 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f411c600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f41140d1bb0 con 0x7f411415e960 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f411415e960 0x7f41140d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f411415e960 0x7f41140d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60782/0 (socket says 172.21.15.59:60782) 2024-04-24T06:39:10.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 -- 172.21.15.59:0/3911845698 learned_addr learned my addr 172.21.15.59:0/3911845698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 -- 172.21.15.59:0/3911845698 >> v1:172.21.15.66:6789/0 conn(0x7f41140d1720 legacy=0x7f4114162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 -- 172.21.15.59:0/3911845698 >> v1:172.21.15.59:6789/0 conn(0x7f4114162c80 legacy=0x7f41140d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.202+0000 7f4119c00640 1 -- 172.21.15.59:0/3911845698 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41140d2c90 con 0x7f411415e960 2024-04-24T06:39:10.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.204+0000 7f4119c00640 1 --2- 172.21.15.59:0/3911845698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140d11e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f410802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c593371b71c45b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.204+0000 7f4113e00640 1 -- 172.21.15.59:0/3911845698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108037070 con 0x7f411415e960 2024-04-24T06:39:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.204+0000 7f4113e00640 1 -- 172.21.15.59:0/3911845698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108032440 con 0x7f411415e960 2024-04-24T06:39:10.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.204+0000 7f4113e00640 1 -- 172.21.15.59:0/3911845698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108032810 con 0x7f411415e960 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 -- 172.21.15.59:0/3911845698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 msgr2=0x7f41140d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 --2- 172.21.15.59:0/3911845698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140d11e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f410802a210 comp rx=0 tx=0).stop 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 -- 172.21.15.59:0/3911845698 shutdown_connections 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 --2- 172.21.15.59:0/3911845698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140d11e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 -- 172.21.15.59:0/3911845698 >> 172.21.15.59:0/3911845698 conn(0x7f4114067760 msgr2=0x7f4114067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 -- 172.21.15.59:0/3911845698 shutdown_connections 2024-04-24T06:39:10.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.205+0000 7f411c600640 1 -- 172.21.15.59:0/3911845698 wait complete. 2024-04-24T06:39:10.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.207+0000 7f411c600640 1 Processor -- start 2024-04-24T06:39:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.207+0000 7f411c600640 1 -- start start 2024-04-24T06:39:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41140d1720 0x7f41141e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4114162c80 0x7f41140ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f411415ed40 con 0x7f411415e960 2024-04-24T06:39:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41140b7930 con 0x7f41140d1720 2024-04-24T06:39:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f411c600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f411415eec0 con 0x7f4114162c80 2024-04-24T06:39:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.208+0000 7f4119200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60784/0 (socket says 172.21.15.59:60784) 2024-04-24T06:39:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 -- 172.21.15.59:0/3705069416 learned_addr learned my addr 172.21.15.59:0/3705069416 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 -- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4114162c80 msgr2=0x7f41140ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4114162c80 0x7f41140ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 -- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41140d1720 msgr2=0x7f41141e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41140d1720 0x7f41141e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 -- 172.21.15.59:0/3705069416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41141e85c0 con 0x7f411415e960 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.209+0000 7f4119200640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f410000c460 tx=0x7f410000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.210+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100016020 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.210+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4100012870 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.210+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100012c60 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.210+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41141e88a0 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.210+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41141e8e60 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.211+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4114068a80 con 0x7f411415e960 2024-04-24T06:39:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.212+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4100019020 con 0x7f411415e960 2024-04-24T06:39:10.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.213+0000 7f4111600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 0x7f40f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.213+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f410008b880 con 0x7f411415e960 2024-04-24T06:39:10.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.213+0000 7f4119c00640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 0x7f40f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.214+0000 7f4119c00640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 0x7f40f406edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4108034000 tx=0x7f410802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.218+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4100020030 con 0x7f411415e960 2024-04-24T06:39:10.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.422+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41140d2430 con 0x7f411415e960 2024-04-24T06:39:10.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.423+0000 7f4111600640 1 -- 172.21.15.59:0/3705069416 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f4100059ca0 con 0x7f411415e960 2024-04-24T06:39:10.424 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:39:10.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.426+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 msgr2=0x7f40f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.426+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 0x7f40f406edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4108034000 tx=0x7f410802c000 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.426+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 msgr2=0x7f41140cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.426+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f410000c460 tx=0x7f410000c920 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 shutdown_connections 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40f406c8f0 0x7f40f406edb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4114162c80 0x7f41140ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411415e960 0x7f41140cdda0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 --2- 172.21.15.59:0/3705069416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41140d1720 0x7f41141e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 >> 172.21.15.59:0/3705069416 conn(0x7f4114067760 msgr2=0x7f411415d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 shutdown_connections 2024-04-24T06:39:10.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.427+0000 7f411c600640 1 -- 172.21.15.59:0/3705069416 wait complete. 2024-04-24T06:39:10.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:39:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.665+0000 7f0e22400640 1 Processor -- start 2024-04-24T06:39:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.665+0000 7f0e22400640 1 -- start start 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e22400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 0x7f0e1c0d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e22400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0e1c0b7420 0x7f0e1c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e22400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0d23f0 con 0x7f0e1c15e870 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e22400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0d21a0 con 0x7f0e1c0b7420 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e22400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0d3000 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e21a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0e1c0b7420 0x7f0e1c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e21a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0e1c0b7420 0x7f0e1c0b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60790/0 (socket says 172.21.15.59:60790) 2024-04-24T06:39:10.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.667+0000 7f0e21a00640 1 -- 172.21.15.59:0/2639092335 learned_addr learned my addr 172.21.15.59:0/2639092335 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e1be00640 1 --2- 172.21.15.59:0/2639092335 >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 0x7f0e1c0d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e21a00640 1 -- 172.21.15.59:0/2639092335 >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 msgr2=0x7f0e1c0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e21a00640 1 --2- 172.21.15.59:0/2639092335 >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 0x7f0e1c0d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e21a00640 1 -- 172.21.15.59:0/2639092335 >> v1:172.21.15.66:6789/0 conn(0x7f0e1c15e870 legacy=0x7f0e1c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e21a00640 1 -- 172.21.15.59:0/2639092335 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0d26d0 con 0x7f0e1c0b7420 2024-04-24T06:39:10.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.668+0000 7f0e1be00640 1 --2- 172.21.15.59:0/2639092335 >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 0x7f0e1c0d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:10.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.669+0000 7f0e21a00640 1 --2- 172.21.15.59:0/2639092335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0b7830 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c08da70 tx=0x7f0e1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f1b23a8812bfad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.669+0000 7f0e1b400640 1 -- 172.21.15.59:0/2639092335 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14036070 con 0x7f0e1c0b7420 2024-04-24T06:39:10.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.669+0000 7f0e1b400640 1 -- 172.21.15.59:0/2639092335 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e14031600 con 0x7f0e1c0b7420 2024-04-24T06:39:10.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.669+0000 7f0e1b400640 1 -- 172.21.15.59:0/2639092335 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140319d0 con 0x7f0e1c0b7420 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 -- 172.21.15.59:0/2639092335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0b7420 msgr2=0x7f0e1c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 --2- 172.21.15.59:0/2639092335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0b7830 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c08da70 tx=0x7f0e1402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 -- 172.21.15.59:0/2639092335 shutdown_connections 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 --2- 172.21.15.59:0/2639092335 >> v2:172.21.15.66:3300/0 conn(0x7f0e1c0d0fa0 0x7f0e1c0d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 --2- 172.21.15.59:0/2639092335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0b7830 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 -- 172.21.15.59:0/2639092335 >> 172.21.15.59:0/2639092335 conn(0x7f0e1c08bf90 msgr2=0x7f0e1c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:10.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 -- 172.21.15.59:0/2639092335 shutdown_connections 2024-04-24T06:39:10.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.670+0000 7f0e22400640 1 -- 172.21.15.59:0/2639092335 wait complete. 2024-04-24T06:39:10.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.672+0000 7f0e22400640 1 Processor -- start 2024-04-24T06:39:10.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.672+0000 7f0e22400640 1 -- start start 2024-04-24T06:39:10.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e1c15e870 0x7f0e1c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15ec50 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0d2c90 con 0x7f0e1c0b7420 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e22400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c0d3490 con 0x7f0e1c15e870 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.673+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60792/0 (socket says 172.21.15.59:60792) 2024-04-24T06:39:10.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 -- 172.21.15.59:0/3342527448 learned_addr learned my addr 172.21.15.59:0/3342527448 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 -- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e1c15e870 msgr2=0x7f0e1c0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e21a00640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e1c15e870 0x7f0e1c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e21000640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e1c15e870 0x7f0e1c0c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 -- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e1c0b7420 msgr2=0x7f0e1c0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 -- 172.21.15.59:0/3342527448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0c85e0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.674+0000 7f0e1be00640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c7f0 tx=0x7f0e0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.675+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08015070 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.675+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e08007820 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.675+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08007bd0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.675+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c0caa90 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.675+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c0cb050 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.676+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c0cb4d0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.677+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e08018070 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.678+0000 7f0e18c00640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:10.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.678+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f0e0808bcc0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.678+0000 7f0e21000640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:10.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.679+0000 7f0e21000640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e10003d90 tx=0x7f0e10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.683+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e1c0c85e0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.891+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e1c04fd90 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.892+0000 7f0e18c00640 1 -- 172.21.15.59:0/3342527448 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f0e0805a0f0 con 0x7f0e1c0d0fa0 2024-04-24T06:39:10.893 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:39:10.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 msgr2=0x7f0dfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e10003d90 tx=0x7f0e10009040 comp rx=0 tx=0).stop 2024-04-24T06:39:10.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 msgr2=0x7f0e1c150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c7f0 tx=0x7f0e0800ccb0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 shutdown_connections 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0e1c15e870 0x7f0e1c0c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e1c0d0fa0 0x7f0e1c150900 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 --2- 172.21.15.59:0/3342527448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e1c0b7420 0x7f0e1c0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 >> 172.21.15.59:0/3342527448 conn(0x7f0e1c08bf90 msgr2=0x7f0e1c1621a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 shutdown_connections 2024-04-24T06:39:10.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:10.895+0000 7f0e22400640 1 -- 172.21.15.59:0/3342527448 wait complete. 2024-04-24T06:39:10.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:39:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.134+0000 7fa196600640 1 Processor -- start 2024-04-24T06:39:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.134+0000 7fa196600640 1 -- start start 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa196600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1900d0d10 0x7fa190156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa196600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa19015d810 0x7fa190160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa196600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa19015cab0 con 0x7fa1901567e0 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa196600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa190160e40 con 0x7fa1900d0d10 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa196600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa19015cd30 con 0x7fa19015d810 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa18fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1900d0d10 0x7fa190156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa18fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1900d0d10 0x7fa190156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60796/0 (socket says 172.21.15.59:60796) 2024-04-24T06:39:11.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.136+0000 7fa18fe00640 1 -- 172.21.15.59:0/3462272687 learned_addr learned my addr 172.21.15.59:0/3462272687 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.137+0000 7fa18fe00640 1 -- 172.21.15.59:0/3462272687 >> v2:172.21.15.66:3300/0 conn(0x7fa19015d810 msgr2=0x7fa190160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.137+0000 7fa18fe00640 1 --2- 172.21.15.59:0/3462272687 >> v2:172.21.15.66:3300/0 conn(0x7fa19015d810 0x7fa190160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.137+0000 7fa18fe00640 1 -- 172.21.15.59:0/3462272687 >> v1:172.21.15.66:6789/0 conn(0x7fa1901567e0 legacy=0x7fa1900d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.137+0000 7fa18fe00640 1 -- 172.21.15.59:0/3462272687 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19015d010 con 0x7fa1900d0d10 2024-04-24T06:39:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.138+0000 7fa18fe00640 1 --2- 172.21.15.59:0/3462272687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1900d0d10 0x7fa190156bc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa1900b4cb0 tx=0x7fa17c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeceea4c69d3108 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.138+0000 7fa18f400640 1 -- 172.21.15.59:0/3462272687 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c037070 con 0x7fa1900d0d10 2024-04-24T06:39:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.138+0000 7fa18f400640 1 -- 172.21.15.59:0/3462272687 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c032440 con 0x7fa1900d0d10 2024-04-24T06:39:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.138+0000 7fa18f400640 1 -- 172.21.15.59:0/3462272687 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c032810 con 0x7fa1900d0d10 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 -- 172.21.15.59:0/3462272687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1900d0d10 msgr2=0x7fa190156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 --2- 172.21.15.59:0/3462272687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1900d0d10 0x7fa190156bc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa1900b4cb0 tx=0x7fa17c02a230 comp rx=0 tx=0).stop 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 -- 172.21.15.59:0/3462272687 shutdown_connections 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 --2- 172.21.15.59:0/3462272687 >> v2:172.21.15.66:3300/0 conn(0x7fa19015d810 0x7fa190160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 --2- 172.21.15.59:0/3462272687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1900d0d10 0x7fa190156bc0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 -- 172.21.15.59:0/3462272687 >> 172.21.15.59:0/3462272687 conn(0x7fa19006bdf0 msgr2=0x7fa19006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 -- 172.21.15.59:0/3462272687 shutdown_connections 2024-04-24T06:39:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.139+0000 7fa196600640 1 -- 172.21.15.59:0/3462272687 wait complete. 2024-04-24T06:39:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.141+0000 7fa196600640 1 Processor -- start 2024-04-24T06:39:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.141+0000 7fa196600640 1 -- start start 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1900d0d10 0x7fa190072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa19015d810 0x7fa19008bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1901607d0 con 0x7fa1901567e0 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa190160650 con 0x7fa1900d0d10 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa196600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa190160950 con 0x7fa19015d810 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60808/0 (socket says 172.21.15.59:60808) 2024-04-24T06:39:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 -- 172.21.15.59:0/4076914337 learned_addr learned my addr 172.21.15.59:0/4076914337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 -- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa19015d810 msgr2=0x7fa19008bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa19015d810 0x7fa19008bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 -- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1900d0d10 msgr2=0x7fa190072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1900d0d10 0x7fa190072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.142+0000 7fa18fe00640 1 -- 172.21.15.59:0/4076914337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa190159120 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa18fe00640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009960 tx=0x7fa17c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c041070 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c03c950 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa190158080 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c037040 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.143+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa190158640 con 0x7fa1901567e0 2024-04-24T06:39:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.145+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19004fd90 con 0x7fa1901567e0 2024-04-24T06:39:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.146+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa17c044070 con 0x7fa1901567e0 2024-04-24T06:39:11.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.146+0000 7fa18cc00640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.147+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fa17c0ac840 con 0x7fa1901567e0 2024-04-24T06:39:11.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.147+0000 7fa195200640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.148+0000 7fa195200640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 0x7fa17006edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa184003cc0 tx=0x7fa1840079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.151+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa19015cab0 con 0x7fa1901567e0 2024-04-24T06:39:11.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.366+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1900b5150 con 0x7fa1901567e0 2024-04-24T06:39:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.367+0000 7fa18cc00640 1 -- 172.21.15.59:0/4076914337 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fa17c07ac60 con 0x7fa1901567e0 2024-04-24T06:39:11.367 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 msgr2=0x7fa17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 0x7fa17006edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa184003cc0 tx=0x7fa1840079b0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 msgr2=0x7fa19008e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009960 tx=0x7fa17c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 shutdown_connections 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa17006c8f0 0x7fa17006edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa19015d810 0x7fa19008bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1901567e0 0x7fa19008e920 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.369+0000 7fa196600640 1 --2- 172.21.15.59:0/4076914337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1900d0d10 0x7fa190072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.370+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 >> 172.21.15.59:0/4076914337 conn(0x7fa19006bdf0 msgr2=0x7fa19015bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.370+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 shutdown_connections 2024-04-24T06:39:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.370+0000 7fa196600640 1 -- 172.21.15.59:0/4076914337 wait complete. 2024-04-24T06:39:11.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:39:11.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.602+0000 7f0c5f200640 1 Processor -- start 2024-04-24T06:39:11.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.602+0000 7f0c5f200640 1 -- start start 2024-04-24T06:39:11.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5f200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0c600d05c0 0x7f0c601416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 0x7f0c601555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6015ba70 con 0x7f0c6015f060 2024-04-24T06:39:11.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5f200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0c60141c20 con 0x7f0c600d05c0 2024-04-24T06:39:11.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6015bdc0 con 0x7f0c6015c250 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c57e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0c6015f060 0x7f0c600d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40558/0 (socket says 172.21.15.59:40558) 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c57e00640 1 -- 172.21.15.59:0/2419280208 learned_addr learned my addr 172.21.15.59:0/2419280208 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.604+0000 7f0c5de00640 1 --2- 172.21.15.59:0/2419280208 >> v2:172.21.15.66:3300/0 conn(0x7f0c600d05c0 0x7f0c601416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5e800640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 0x7f0c601555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558799238 0 0) 0x7f0c6015ba70 con 0x7f0c6015f060 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c40003520 con 0x7f0c6015f060 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411002563 0 0) 0x7f0c40003520 con 0x7f0c6015f060 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 msgr2=0x7f0c601555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 0x7f0c601555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 >> v2:172.21.15.66:3300/0 conn(0x7f0c600d05c0 msgr2=0x7f0c601416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 --2- 172.21.15.59:0/2419280208 >> v2:172.21.15.66:3300/0 conn(0x7f0c600d05c0 0x7f0c601416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6015f800 con 0x7f0c6015f060 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0c48001670 con 0x7f0c6015f060 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 >> v1:172.21.15.59:6789/0 conn(0x7f0c6015f060 legacy=0x7f0c600d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c400039b0 0x7f0c40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c40004800 0x7f0c40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 0x7f0c400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60141c20 con 0x7f0c400039b0 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6015bdc0 con 0x7f0c40004800 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4000b9c0 con 0x7f0c6015c250 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.605+0000 7f0c5de00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c400039b0 0x7f0c40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c57e00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c40004800 0x7f0c40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c5de00640 1 -- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 msgr2=0x7f0c400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c5de00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c6015c250 0x7f0c400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c5de00640 1 -- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c40004800 msgr2=0x7f0c40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:11.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c5de00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c40004800 0x7f0c40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c5de00640 1 -- 172.21.15.59:0/2419280208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c40003520 con 0x7f0c400039b0 2024-04-24T06:39:11.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.606+0000 7f0c57e00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c40004800 0x7f0c40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.607+0000 7f0c5de00640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c400039b0 0x7f0c40003dd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009ac0 tx=0x7f0c40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9c29f78e9f1d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.607+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50036070 con 0x7f0c400039b0 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.607+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c50031400 con 0x7f0c400039b0 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.607+0000 7f0c5d400640 1 -- 172.21.15.59:0/2419280208 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50009690 con 0x7f0c400039b0 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 -- 172.21.15.59:0/2419280208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c400039b0 msgr2=0x7f0c40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 --2- 172.21.15.59:0/2419280208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c400039b0 0x7f0c40003dd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009ac0 tx=0x7f0c40005830 comp rx=0 tx=0).stop 2024-04-24T06:39:11.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5e800640 1 -- 172.21.15.59:0/2419280208 reap_dead start 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 -- 172.21.15.59:0/2419280208 shutdown_connections 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 -- 172.21.15.59:0/2419280208 >> 172.21.15.59:0/2419280208 conn(0x7f0c6006bdf0 msgr2=0x7f0c6006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 -- 172.21.15.59:0/2419280208 shutdown_connections 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.608+0000 7f0c5f200640 1 -- 172.21.15.59:0/2419280208 wait complete. 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.610+0000 7f0c5f200640 1 Processor -- start 2024-04-24T06:39:11.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.610+0000 7f0c5f200640 1 -- start start 2024-04-24T06:39:11.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c400039b0 0x7f0c6009d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 0x7f0c600ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6015f4f0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6015fdc0 con 0x7f0c6014e240 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6015c6e0 con 0x7f0c400039b0 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.611+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60816/0 (socket says 172.21.15.59:60816) 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 -- 172.21.15.59:0/1576786571 learned_addr learned my addr 172.21.15.59:0/1576786571 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c57e00640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 0x7f0c600ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 -- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c400039b0 msgr2=0x7f0c6009d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5de00640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c400039b0 0x7f0c6009d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c400039b0 0x7f0c6009d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 -- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 msgr2=0x7f0c600ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 0x7f0c600ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 -- 172.21.15.59:0/1576786571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601500f0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c57e00640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 0x7f0c600ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.612+0000 7f0c5e800640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0c58007c40 tx=0x7f0c5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.613+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800eb60 con 0x7f0c6014e6c0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.613+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5800ed20 con 0x7f0c6014e6c0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.613+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58003bd0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.613+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601503d0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.613+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c60155720 con 0x7f0c6014e6c0 2024-04-24T06:39:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.614+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c60155920 con 0x7f0c6014e6c0 2024-04-24T06:39:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.615+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c5801b020 con 0x7f0c6014e6c0 2024-04-24T06:39:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.616+0000 7f0c56000640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 0x7f0c3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.616+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f0c5808b8f0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.616+0000 7f0c5de00640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 0x7f0c3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:11.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.617+0000 7f0c5de00640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 0x7f0c3c06ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0c50033000 tx=0x7f0c50008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:11.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.620+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c60155920 con 0x7f0c6014e6c0 2024-04-24T06:39:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.840+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0c600cfec0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.841+0000 7f0c56000640 1 -- 172.21.15.59:0/1576786571 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v98) ==== 80+0+0 (secure 0 0 0) 0x7f0c58059dd0 con 0x7f0c6014e6c0 2024-04-24T06:39:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.843+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 msgr2=0x7f0c3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 0x7f0c3c06ede0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0c50033000 tx=0x7f0c50008030 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 msgr2=0x7f0c600cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0c58007c40 tx=0x7f0c5800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 shutdown_connections 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0c3c06c920 0x7f0c3c06ede0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c6014e6c0 0x7f0c600cd0f0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c6014e240 0x7f0c600ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 --2- 172.21.15.59:0/1576786571 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0c400039b0 0x7f0c6009d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 >> 172.21.15.59:0/1576786571 conn(0x7f0c6006bdf0 msgr2=0x7f0c6006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 shutdown_connections 2024-04-24T06:39:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:11.844+0000 7f0c5f200640 1 -- 172.21.15.59:0/1576786571 wait complete. 2024-04-24T06:39:11.863 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:39:11.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:11.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:11.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70872]: fuse finished with error 0 and tester_r 0 2024-04-24T06:39:11.957 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:39:18.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:39:18.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:18.059 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:39:18.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:18.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.346+0000 7fde78e00640 1 Processor -- start 2024-04-24T06:39:18.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.346+0000 7fde78e00640 1 -- start start 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde78e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde7415e7c0 0x7fde740d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde78e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde740cf030 con 0x7fde740d0d90 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde78e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde740b6a40 con 0x7fde740d2630 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde78e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fde740b7000 con 0x7fde7415e7c0 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde73400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde7415e7c0 0x7fde740d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde73400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde7415e7c0 0x7fde740d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60826/0 (socket says 172.21.15.59:60826) 2024-04-24T06:39:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.348+0000 7fde73400640 1 -- 172.21.15.59:0/4233608703 learned_addr learned my addr 172.21.15.59:0/4233608703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:18.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.349+0000 7fde73400640 1 -- 172.21.15.59:0/4233608703 >> v1:172.21.15.66:6789/0 conn(0x7fde740d2630 legacy=0x7fde740b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:18.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.349+0000 7fde73400640 1 -- 172.21.15.59:0/4233608703 >> v1:172.21.15.59:6789/0 conn(0x7fde740d0d90 legacy=0x7fde740cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.349+0000 7fde73400640 1 -- 172.21.15.59:0/4233608703 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde740b6d20 con 0x7fde7415e7c0 2024-04-24T06:39:18.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.350+0000 7fde73400640 1 --2- 172.21.15.59:0/4233608703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7415e7c0 0x7fde740d20f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fde64009870 tx=0x7fde6402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c9c592426e716b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:18.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.350+0000 7fde72000640 1 -- 172.21.15.59:0/4233608703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64037070 con 0x7fde7415e7c0 2024-04-24T06:39:18.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.350+0000 7fde72000640 1 -- 172.21.15.59:0/4233608703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde64032440 con 0x7fde7415e7c0 2024-04-24T06:39:18.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.350+0000 7fde72000640 1 -- 172.21.15.59:0/4233608703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64032810 con 0x7fde7415e7c0 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 -- 172.21.15.59:0/4233608703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7415e7c0 msgr2=0x7fde740d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 --2- 172.21.15.59:0/4233608703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7415e7c0 0x7fde740d20f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fde64009870 tx=0x7fde6402a2a0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 -- 172.21.15.59:0/4233608703 shutdown_connections 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 --2- 172.21.15.59:0/4233608703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde7415e7c0 0x7fde740d20f0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 -- 172.21.15.59:0/4233608703 >> 172.21.15.59:0/4233608703 conn(0x7fde7409d920 msgr2=0x7fde7408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 -- 172.21.15.59:0/4233608703 shutdown_connections 2024-04-24T06:39:18.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.351+0000 7fde78e00640 1 -- 172.21.15.59:0/4233608703 wait complete. 2024-04-24T06:39:18.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.352+0000 7fde78e00640 1 Processor -- start 2024-04-24T06:39:18.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.353+0000 7fde78e00640 1 -- start start 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde740d0d90 0x7fde741f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 0x7fde740cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde740b6700 con 0x7fde740d2630 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde740cf620 con 0x7fde7415e7c0 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde78e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fde740d1a80 con 0x7fde740d0d90 2024-04-24T06:39:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60836/0 (socket says 172.21.15.59:60836) 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 -- 172.21.15.59:0/2218713879 learned_addr learned my addr 172.21.15.59:0/2218713879 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde73400640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde740d0d90 0x7fde741f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde73e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 0x7fde740cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 -- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde740d0d90 msgr2=0x7fde741f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde740d0d90 0x7fde741f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 -- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 msgr2=0x7fde740cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 0x7fde740cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde72a00640 1 -- 172.21.15.59:0/2218713879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde740c8fd0 con 0x7fde740d2630 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.354+0000 7fde73e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 0x7fde740cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde72a00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c400 tx=0x7fde6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60011790 con 0x7fde740d2630 2024-04-24T06:39:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde60017070 con 0x7fde740d2630 2024-04-24T06:39:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60011bb0 con 0x7fde740d2630 2024-04-24T06:39:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde740c9570 con 0x7fde740d2630 2024-04-24T06:39:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.355+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde740c9db0 con 0x7fde740d2630 2024-04-24T06:39:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.356+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7404fd90 con 0x7fde740d2630 2024-04-24T06:39:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.357+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde6001c460 con 0x7fde740d2630 2024-04-24T06:39:18.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.358+0000 7fde6b400640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:18.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.358+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fde6008baa0 con 0x7fde740d2630 2024-04-24T06:39:18.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.359+0000 7fde73400640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:18.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.360+0000 7fde73400640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde64034000 tx=0x7fde6402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:18.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.364+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde740c9570 con 0x7fde740d2630 2024-04-24T06:39:18.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.611+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde740c9f80 con 0x7fde740d2630 2024-04-24T06:39:18.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.612+0000 7fde6b400640 1 -- 172.21.15.59:0/2218713879 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5177 (secure 0 0 0) 0x7fde60059e30 con 0x7fde740d2630 2024-04-24T06:39:18.613 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:18.613 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:38:41.731305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8499":{"gid":8499,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/891418539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":891418539},{"type":"v1","addr":"172.21.15.66:6834","nonce":891418539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8499,"qdb_cluster":[8499]},"id":7}]} 2024-04-24T06:39:18.614 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 83 2024-04-24T06:39:18.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 msgr2=0x7fde5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde64034000 tx=0x7fde6402e000 comp rx=0 tx=0).stop 2024-04-24T06:39:18.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 msgr2=0x7fde740ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:18.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c400 tx=0x7fde6000c8c0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 shutdown_connections 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde5406c8f0 0x7fde5406edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.615+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde7415e7c0 0x7fde740cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.616+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde740d2630 0x7fde740ca2d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.616+0000 7fde78e00640 1 --2- 172.21.15.59:0/2218713879 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde740d0d90 0x7fde741f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.616+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 >> 172.21.15.59:0/2218713879 conn(0x7fde7409d920 msgr2=0x7fde7415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.616+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 shutdown_connections 2024-04-24T06:39:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:18.616+0000 7fde78e00640 1 -- 172.21.15.59:0/2218713879 wait complete. 2024-04-24T06:39:18.636 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-24T06:39:18.896 INFO:teuthology.orchestra.run.smithi066.stdout:{"message":"","return_code":0} 2024-04-24T06:39:18.896 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-24T06:39:18.896 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:39:18.896 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:39:18.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:39:18.956 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi059.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-24T06:39:18.957 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:39:18.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:18.957 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:39:19.013 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-24T06:39:19.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:39:19.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-24T06:39:19.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:19.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-24T06:39:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.209+0000 7f18b7200640 1 Processor -- start 2024-04-24T06:39:19.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.209+0000 7f18b7200640 1 -- start start 2024-04-24T06:39:19.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.210+0000 7f18b7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f18b0156c20 0x7f18b00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.210+0000 7f18b7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 0x7f18b006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.210+0000 7f18b7200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b0157000 con 0x7f18b006c330 2024-04-24T06:39:19.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.210+0000 7f18b7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f18b00afb00 con 0x7f18b0156c20 2024-04-24T06:39:19.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.211+0000 7f18b7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f18b015cc20 con 0x7f18b00af4c0 2024-04-24T06:39:19.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.211+0000 7f18b5400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 0x7f18b006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.211+0000 7f18b6800640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f18b006c330 0x7f18b00b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:50802/0 (socket says 172.21.15.59:50802) 2024-04-24T06:39:19.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.211+0000 7f18b6800640 1 -- 172.21.15.59:0/3424843875 learned_addr learned my addr 172.21.15.59:0/3424843875 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:19.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.211+0000 7f18b5e00640 1 --2- 172.21.15.59:0/3424843875 >> v2:172.21.15.66:3300/0 conn(0x7f18b0156c20 0x7f18b00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b4a00640 1 -- 172.21.15.59:0/3424843875 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011186339 0 0) 0x7f18b0157000 con 0x7f18b006c330 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b4a00640 1 -- 172.21.15.59:0/3424843875 --> v1:172.21.15.66:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1898003520 con 0x7f18b006c330 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b5400640 1 -- 172.21.15.59:0/3424843875 >> v2:172.21.15.66:3300/0 conn(0x7f18b0156c20 msgr2=0x7f18b00aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b5400640 1 --2- 172.21.15.59:0/3424843875 >> v2:172.21.15.66:3300/0 conn(0x7f18b0156c20 0x7f18b00aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b5400640 1 -- 172.21.15.59:0/3424843875 >> v1:172.21.15.66:6789/0 conn(0x7f18b006c330 legacy=0x7f18b00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.212+0000 7f18b5400640 1 -- 172.21.15.59:0/3424843875 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b015cf00 con 0x7f18b00af4c0 2024-04-24T06:39:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.214+0000 7f18b5400640 1 --2- 172.21.15.59:0/3424843875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 0x7f18b006bdf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18a00097a0 tx=0x7f18a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8bc36f316bfd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.214+0000 7f18b4a00640 1 -- 172.21.15.59:0/3424843875 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0037070 con 0x7f18b00af4c0 2024-04-24T06:39:19.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.214+0000 7f18b4a00640 1 -- 172.21.15.59:0/3424843875 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a0032440 con 0x7f18b00af4c0 2024-04-24T06:39:19.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.214+0000 7f18b4a00640 1 -- 172.21.15.59:0/3424843875 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00327f0 con 0x7f18b00af4c0 2024-04-24T06:39:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 -- 172.21.15.59:0/3424843875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 msgr2=0x7f18b006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 --2- 172.21.15.59:0/3424843875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 0x7f18b006bdf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f18a00097a0 tx=0x7f18a002a180 comp rx=0 tx=0).stop 2024-04-24T06:39:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 -- 172.21.15.59:0/3424843875 shutdown_connections 2024-04-24T06:39:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 --2- 172.21.15.59:0/3424843875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b00af4c0 0x7f18b006bdf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 --2- 172.21.15.59:0/3424843875 >> v2:172.21.15.66:3300/0 conn(0x7f18b0156c20 0x7f18b00aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.215+0000 7f18b7200640 1 -- 172.21.15.59:0/3424843875 >> 172.21.15.59:0/3424843875 conn(0x7f18b00b2200 msgr2=0x7f18b00b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.216+0000 7f18b7200640 1 -- 172.21.15.59:0/3424843875 shutdown_connections 2024-04-24T06:39:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.216+0000 7f18b7200640 1 -- 172.21.15.59:0/3424843875 wait complete. 2024-04-24T06:39:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.217+0000 7f18b7200640 1 Processor -- start 2024-04-24T06:39:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.218+0000 7f18b7200640 1 -- start start 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b00af4c0 0x7f18b00c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b0156c20 0x7f18b0154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18b015d9e0 con 0x7f18b006c330 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b015d4c0 con 0x7f18b00af4c0 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f18b0073ba0 con 0x7f18b0156c20 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40020/0 (socket says 172.21.15.59:40020) 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 -- 172.21.15.59:0/698796680 learned_addr learned my addr 172.21.15.59:0/698796680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 -- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b0156c20 msgr2=0x7f18b0154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b0156c20 0x7f18b0154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 -- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b00af4c0 msgr2=0x7f18b00c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b00af4c0 0x7f18b00c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.219+0000 7f18b5e00640 1 -- 172.21.15.59:0/698796680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b006eb50 con 0x7f18b006c330 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18b5e00640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f18a400e9e0 tx=0x7f18a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18aa000640 1 -- 172.21.15.59:0/698796680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4017070 con 0x7f18b006c330 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18aa000640 1 -- 172.21.15.59:0/698796680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a400c9d0 con 0x7f18b006c330 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18aa000640 1 -- 172.21.15.59:0/698796680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400cdc0 con 0x7f18b006c330 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b006ee30 con 0x7f18b006c330 2024-04-24T06:39:19.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.220+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b006f3f0 con 0x7f18b006c330 2024-04-24T06:39:19.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.223+0000 7f18aa000640 1 -- 172.21.15.59:0/698796680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18a401a070 con 0x7f18b006c330 2024-04-24T06:39:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.224+0000 7f18aa000640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 0x7f189406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.224+0000 7f18aa000640 1 -- 172.21.15.59:0/698796680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f18a408bea0 con 0x7f18b006c330 2024-04-24T06:39:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.224+0000 7f18b5400640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 0x7f189406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.224+0000 7f18bb5ef740 1 Processor -- start 2024-04-24T06:39:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 -- start start 2024-04-24T06:39:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18b5400640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 0x7f189406ee00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f18a0034000 tx=0x7f18a002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a402fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a432ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b55a36ce70 con 0x55b55a433020 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b55a313380 con 0x55b55a433760 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.225+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b55a368e30 con 0x55b55a450640 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40034/0 (socket says 172.21.15.59:40034) 2024-04-24T06:39:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 -- 172.21.15.59:0/1623299898 learned_addr learned my addr 172.21.15.59:0/1623299898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 -- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 msgr2=0x55b55a402fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b5e00640 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a402fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b5400640 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a432ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a402fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 -- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 msgr2=0x55b55a432ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a432ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.226+0000 7f18b6800640 1 -- 172.21.15.59:0/1623299898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b55a1477e0 con 0x55b55a433020 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.227+0000 7f18b6800640 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0020d0 tx=0x7f18ac00c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dab7b0ab38a4d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.227+0000 7f189e600640 1 -- 172.21.15.59:0/1623299898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac017070 con 0x55b55a433020 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.227+0000 7f189e600640 1 -- 172.21.15.59:0/1623299898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0072d0 con 0x55b55a433020 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.227+0000 7f189e600640 1 -- 172.21.15.59:0/1623299898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0076c0 con 0x55b55a433020 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1623299898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 msgr2=0x55b55a407e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0020d0 tx=0x7f18ac00c4b0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1623299898 shutdown_connections 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433020 0x55b55a407e30 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a432ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1623299898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a402fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1623299898 >> 172.21.15.59:0/1623299898 conn(0x55b55a405240 msgr2=0x55b55a44dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1623299898 shutdown_connections 2024-04-24T06:39:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.228+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1623299898 wait complete. 2024-04-24T06:39:19.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.229+0000 7f18bb5ef740 1 -- start start 2024-04-24T06:39:19.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.229+0000 7f18bb5ef740 1 Processor -- start 2024-04-24T06:39:19.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a42e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a461450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b55a4073f0 con 0x55b55a461990 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b55a326660 con 0x55b55a433760 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b55a4067e0 con 0x55b55a450640 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18bb5ef740 10 client.0 _ll_register_callbacks cb 0x55b55a314aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.230+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40042/0 (socket says 172.21.15.59:40042) 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 -- 172.21.15.59:0/3536919901 learned_addr learned my addr 172.21.15.59:0/3536919901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b5400640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a461450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b5e00640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a42e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 -- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 msgr2=0x55b55a461450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a461450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 -- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 msgr2=0x55b55a42e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a42e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 -- 172.21.15.59:0/3536919901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b55a1a7f70 con 0x55b55a461990 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b5400640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a461450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.231+0000 7f18b6800640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f18ac015040 tx=0x7f18ac005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1b2806d956f09f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f1892a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac022070 con 0x55b55a461990 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac0072d0 con 0x55b55a461990 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f18ac007a30 con 0x55b55a461990 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f18ac017040 con 0x55b55a461990 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f18bb5ef740 10 client.8838 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:19.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55b55a03f170 con 0x55b55a461990 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.232+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0072d0 con 0x55b55a461990 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f18ac01d040 con 0x55b55a461990 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 10 client.8838 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 10 client.8838 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 20 client.8838 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a434120 0x55b55a45de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 4 client.8838 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.233+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b55a37a890 con 0x55b55a434120 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.234+0000 7f18b5e00640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a434120 0x55b55a45de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.235+0000 7f18b5e00640 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a434120 0x55b55a45de10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e476669120b99565 server_cookie=69c1abc8ace4f62e in_seq=0 out_seq=0 2024-04-24T06:39:19.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.235+0000 7f1892a00640 10 client.8838 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f1892a00640 1 -- 172.21.15.59:0/3536919901 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b55a37a890 con 0x55b55a434120 2024-04-24T06:39:19.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f1892a00640 10 client.8838 handle_command_reply: tid=0 2024-04-24T06:39:19.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 client.8838 shutdown 2024-04-24T06:39:19.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 msgr2=0x55b55a44f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f18ac015040 tx=0x7f18ac005b70 comp rx=0 tx=0).stop 2024-04-24T06:39:19.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 shutdown_connections 2024-04-24T06:39:19.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a434120 0x55b55a45de10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44f2c0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a461450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/3536919901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a42e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f1892a00640 0 client.8838 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 >> 172.21.15.59:0/3536919901 conn(0x55b55a433020 msgr2=0x55b55a44dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 shutdown_connections 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 1 -- 172.21.15.59:0/3536919901 wait complete. 2024-04-24T06:39:19.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.237+0000 7f18bb5ef740 20 client.8838 trim_cache size 0 max 16384 2024-04-24T06:39:19.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.250+0000 7fb1b0200640 1 Processor -- start 2024-04-24T06:39:19.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.250+0000 7fb1b0200640 1 -- start start 2024-04-24T06:39:19.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.251+0000 7fb1b0200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1a815fdb0 0x7fb1a815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.251+0000 7fb1b0200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb1a80cd090 0x7fb1a80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.251+0000 7fb1b0200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a80cf330 con 0x7fb1a8146000 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.251+0000 7fb1b0200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a80cda20 con 0x7fb1a815fdb0 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.251+0000 7fb1b0200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a80cfc00 con 0x7fb1a80cd090 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ad800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1a815fdb0 0x7fb1a815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ace00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb1a8146000 0x7fb1a806ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35602/0 (socket says 172.21.15.59:35602) 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ace00640 1 -- 172.21.15.59:0/3947703229 learned_addr learned my addr 172.21.15.59:0/3947703229 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ae200640 1 --2- 172.21.15.59:0/3947703229 >> v2:172.21.15.66:3300/0 conn(0x7fb1a80cd090 0x7fb1a80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ad800640 1 -- 172.21.15.59:0/3947703229 >> v2:172.21.15.66:3300/0 conn(0x7fb1a80cd090 msgr2=0x7fb1a80cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ad800640 1 --2- 172.21.15.59:0/3947703229 >> v2:172.21.15.66:3300/0 conn(0x7fb1a80cd090 0x7fb1a80cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ad800640 1 -- 172.21.15.59:0/3947703229 >> v1:172.21.15.59:6789/0 conn(0x7fb1a8146000 legacy=0x7fb1a806ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.252+0000 7fb1ad800640 1 -- 172.21.15.59:0/3947703229 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a806f370 con 0x7fb1a815fdb0 2024-04-24T06:39:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.253+0000 7fb1ad800640 1 --2- 172.21.15.59:0/3947703229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a815fdb0 0x7fb1a815cd00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8071c10 tx=0x7fb19c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c131340e069392 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1a7e00640 1 -- 172.21.15.59:0/3947703229 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c037070 con 0x7fb1a815fdb0 2024-04-24T06:39:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1a7e00640 1 -- 172.21.15.59:0/3947703229 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19c032440 con 0x7fb1a815fdb0 2024-04-24T06:39:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1a7e00640 1 -- 172.21.15.59:0/3947703229 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c032810 con 0x7fb1a815fdb0 2024-04-24T06:39:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 -- 172.21.15.59:0/3947703229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a815fdb0 msgr2=0x7fb1a815cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 --2- 172.21.15.59:0/3947703229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a815fdb0 0x7fb1a815cd00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8071c10 tx=0x7fb19c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 -- 172.21.15.59:0/3947703229 shutdown_connections 2024-04-24T06:39:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 --2- 172.21.15.59:0/3947703229 >> v2:172.21.15.66:3300/0 conn(0x7fb1a80cd090 0x7fb1a80cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 --2- 172.21.15.59:0/3947703229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a815fdb0 0x7fb1a815cd00 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 -- 172.21.15.59:0/3947703229 >> 172.21.15.59:0/3947703229 conn(0x7fb1a8067ea0 msgr2=0x7fb1a806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 -- 172.21.15.59:0/3947703229 shutdown_connections 2024-04-24T06:39:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.254+0000 7fb1b0200640 1 -- 172.21.15.59:0/3947703229 wait complete. 2024-04-24T06:39:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.256+0000 7fb1b0200640 1 Processor -- start 2024-04-24T06:39:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.256+0000 7fb1b0200640 1 -- start start 2024-04-24T06:39:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a80cd090 0x7fb1a81e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8146000 0x7fb1a80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a815fdb0 0x7fb1a80c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a80cf890 con 0x7fb1a8146000 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a80cf5b0 con 0x7fb1a815fdb0 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.257+0000 7fb1b0200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a80d0090 con 0x7fb1a80cd090 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8146000 0x7fb1a80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8146000 0x7fb1a80cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40064/0 (socket says 172.21.15.59:40064) 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 -- 172.21.15.59:0/454523889 learned_addr learned my addr 172.21.15.59:0/454523889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 -- 172.21.15.59:0/454523889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a80cd090 msgr2=0x7fb1a81e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb1a80cd090 0x7fb1a81e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 -- 172.21.15.59:0/454523889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a815fdb0 msgr2=0x7fb1a80c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1a815fdb0 0x7fb1a80c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 -- 172.21.15.59:0/454523889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a80ccb30 con 0x7fb1a8146000 2024-04-24T06:39:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7f18bb5ef740 1 Processor -- start 2024-04-24T06:39:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7f18bb5ef740 1 -- start start 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.258+0000 7fb1ace00640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a8146000 0x7fb1a80cb040 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb19400c470 tx=0x7fb194001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194017070 con 0x7fb1a8146000 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb194005ce0 con 0x7fb1a8146000 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940072f0 con 0x7fb1a8146000 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7fb1b0200640 1 -- 172.21.15.59:0/454523889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a80cba90 con 0x7fb1a8146000 2024-04-24T06:39:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7fb1b0200640 1 -- 172.21.15.59:0/454523889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a80cc050 con 0x7fb1a8146000 2024-04-24T06:39:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.259+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a450640 0x55b55a44ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7fb1b0200640 1 -- 172.21.15.59:0/454523889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb1a80cd7e0 con 0x7fb1a8146000 2024-04-24T06:39:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a461990 0x55b55a450f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40076/0 (socket says 172.21.15.59:40076) 2024-04-24T06:39:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0200ae0 con 0x55b55a433760 2024-04-24T06:39:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ffc20 con 0x55b55a450640 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f18b00cb260 con 0x55b55a461990 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 -- 172.21.15.59:0/1850957784 learned_addr learned my addr 172.21.15.59:0/1850957784 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 -- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a461990 msgr2=0x55b55a450f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a461990 0x55b55a450f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 -- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a450640 msgr2=0x55b55a44ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a450640 0x55b55a44ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.260+0000 7f18b5e00640 1 -- 172.21.15.59:0/1850957784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b55a461ef0 con 0x55b55a433760 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.261+0000 7f18b5e00640 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f18a40622f0 tx=0x7f18a400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdee6a0e87703f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.261+0000 7f189dc00640 1 -- 172.21.15.59:0/1850957784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4088070 con 0x55b55a433760 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.261+0000 7f189dc00640 1 -- 172.21.15.59:0/1850957784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4060040 con 0x55b55a433760 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.261+0000 7f189dc00640 1 -- 172.21.15.59:0/1850957784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4021040 con 0x55b55a433760 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 msgr2=0x55b55a44da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f18a40622f0 tx=0x7f18a400efd0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb19401a070 con 0x7fb1a8146000 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 shutdown_connections 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a461990 0x55b55a450f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a450640 0x55b55a44ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/1850957784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a433760 0x55b55a44da50 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 >> 172.21.15.59:0/1850957784 conn(0x55b55a433020 msgr2=0x55b55a42e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 shutdown_connections 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- 172.21.15.59:0/1850957784 wait complete. 2024-04-24T06:39:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7f18bb5ef740 1 -- start start 2024-04-24T06:39:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.262+0000 7fb1a5600640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb18806c8f0 0x7fb18806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.263+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fb19408b660 con 0x7fb1a8146000 2024-04-24T06:39:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.263+0000 7f18bb5ef740 1 Processor -- start 2024-04-24T06:39:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.263+0000 7fb1ad800640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb18806c8f0 0x7fb18806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.263+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+504 (secure 0 0 0) 0x7fb1a80cd7e0 con 0x7fb1a8146000 2024-04-24T06:39:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.263+0000 7fb1b0200640 1 -- 172.21.15.59:0/454523889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fb1a80cc800 con 0x7fb1a8146000 2024-04-24T06:39:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7fb1ad800640 1 --2- 172.21.15.59:0/454523889 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb18806c8f0 0x7fb18806edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb19c034000 tx=0x7fb19c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a4c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a44bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00cdf60 con 0x55b55a461990 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b55a466630 con 0x55b55a433760 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b55a4667d0 con 0x55b55a450640 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18bb5ef740 10 client.0 _ll_register_callbacks cb 0x7f18ac015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18b6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40082/0 (socket says 172.21.15.59:40082) 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.264+0000 7f18b6800640 1 -- 172.21.15.59:0/2561242537 learned_addr learned my addr 172.21.15.59:0/2561242537 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 -- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 msgr2=0x55b55a44bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a44bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 -- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 msgr2=0x55b55a4c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a4c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 -- 172.21.15.59:0/2561242537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b55a44df90 con 0x55b55a461990 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18b6800640 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f18ac027dc0 tx=0x7f18ac0014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2140071c6026077f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f1892a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac022040 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f18ac032af0 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a4060040 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f18ac00ce90 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18bb5ef740 10 client.8853 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.265+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55b55a4c9480 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4060040 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f18ac01d030 con 0x55b55a461990 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 10 client.8853 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 10 client.8853 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 20 client.8853 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a4c8130 0x55b55a450f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 4 client.8853 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-24T06:39:19.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.266+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55b55a447b10 con 0x55b55a4c8130 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.267+0000 7f18b5e00640 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a4c8130 0x55b55a450f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.268+0000 7f18b5e00640 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a4c8130 0x55b55a450f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a42ed5476cc8d2b server_cookie=29d059e9b43ffa6 in_seq=0 out_seq=0 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.268+0000 7f1892a00640 10 client.8853 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.269+0000 7f1892a00640 1 -- 172.21.15.59:0/2561242537 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b55a447b10 con 0x55b55a4c8130 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.269+0000 7f1892a00640 10 client.8853 handle_command_reply: tid=0 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.269+0000 7f18bb5ef740 1 client.8853 shutdown 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.269+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 msgr2=0x55b55a44c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.269+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f18ac027dc0 tx=0x7f18ac0014d0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 shutdown_connections 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55b55a4c8130 0x55b55a450f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b55a461990 0x55b55a44c4c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b55a450640 0x55b55a44bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 --2- 172.21.15.59:0/2561242537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b55a433760 0x55b55a4c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 >> 172.21.15.59:0/2561242537 conn(0x55b55a433020 msgr2=0x55b55a403d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f1892a00640 0 client.8853 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 shutdown_connections 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 1 -- 172.21.15.59:0/2561242537 wait complete. 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.270+0000 7f18bb5ef740 20 client.8853 trim_cache size 0 max 16384 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "6fd9c59b-f930-432b-8ce3-480c7d6becdd", 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:39:19.275 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 msgr2=0x7f189406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 0x7f189406ee00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f18a0034000 tx=0x7f18a002c000 comp rx=0 tx=0).stop 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 msgr2=0x7f18b00cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f18a400e9e0 tx=0x7f18a400eea0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 shutdown_connections 2024-04-24T06:39:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f189406c940 0x7f189406ee00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f18b0156c20 0x7f18b0154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b00af4c0 0x7f18b00c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 --2- 172.21.15.59:0/698796680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18b006c330 0x7f18b00cecf0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 >> 172.21.15.59:0/698796680 conn(0x7f18b00b2200 msgr2=0x7f18b0072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 shutdown_connections 2024-04-24T06:39:19.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.271+0000 7f18b7200640 1 -- 172.21.15.59:0/698796680 wait complete. 2024-04-24T06:39:19.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:19.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T06:39:19.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.488+0000 7fa719200640 1 Processor -- start 2024-04-24T06:39:19.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.488+0000 7fa719200640 1 -- start start 2024-04-24T06:39:19.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa71415b8e0 0x7fa7140cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d0080 0x7fa7140d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa7140d1a00 0x7fa7140cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa7140d1e50 con 0x7fa71415b8e0 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa71415bcc0 con 0x7fa7140d1a00 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa719200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa7140cc9e0 con 0x7fa7140d0080 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa711e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa71415b8e0 0x7fa7140cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa711e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa71415b8e0 0x7fa7140cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40096/0 (socket says 172.21.15.59:40096) 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.490+0000 7fa711e00640 1 -- 172.21.15.59:0/3151772062 learned_addr learned my addr 172.21.15.59:0/3151772062 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.491+0000 7fa711e00640 1 -- 172.21.15.59:0/3151772062 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d0080 msgr2=0x7fa7140d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.491+0000 7fa711e00640 1 --2- 172.21.15.59:0/3151772062 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d0080 0x7fa7140d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.491+0000 7fa711e00640 1 -- 172.21.15.59:0/3151772062 >> v2:172.21.15.66:3300/0 conn(0x7fa7140d1a00 msgr2=0x7fa7140cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.491+0000 7fa711e00640 1 --2- 172.21.15.59:0/3151772062 >> v2:172.21.15.66:3300/0 conn(0x7fa7140d1a00 0x7fa7140cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.491+0000 7fa711e00640 1 -- 172.21.15.59:0/3151772062 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7140cccc0 con 0x7fa71415b8e0 2024-04-24T06:39:19.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.492+0000 7fa711e00640 1 --2- 172.21.15.59:0/3151772062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140cfb40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa7140d22f0 tx=0x7fa70402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe336f83cd1df5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.492+0000 7fa710a00640 1 -- 172.21.15.59:0/3151772062 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704033400 con 0x7fa71415b8e0 2024-04-24T06:39:19.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.492+0000 7fa710a00640 1 -- 172.21.15.59:0/3151772062 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7040335a0 con 0x7fa71415b8e0 2024-04-24T06:39:19.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.492+0000 7fa710a00640 1 -- 172.21.15.59:0/3151772062 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704033970 con 0x7fa71415b8e0 2024-04-24T06:39:19.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 -- 172.21.15.59:0/3151772062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 msgr2=0x7fa7140cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 --2- 172.21.15.59:0/3151772062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140cfb40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa7140d22f0 tx=0x7fa70402c8b0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 -- 172.21.15.59:0/3151772062 shutdown_connections 2024-04-24T06:39:19.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 --2- 172.21.15.59:0/3151772062 >> v2:172.21.15.66:3300/0 conn(0x7fa7140d1a00 0x7fa7140cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 --2- 172.21.15.59:0/3151772062 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d0080 0x7fa7140d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 --2- 172.21.15.59:0/3151772062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140cfb40 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 -- 172.21.15.59:0/3151772062 >> 172.21.15.59:0/3151772062 conn(0x7fa714073430 msgr2=0x7fa71409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 -- 172.21.15.59:0/3151772062 shutdown_connections 2024-04-24T06:39:19.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.493+0000 7fa719200640 1 -- 172.21.15.59:0/3151772062 wait complete. 2024-04-24T06:39:19.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.495+0000 7fa719200640 1 Processor -- start 2024-04-24T06:39:19.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.495+0000 7fa719200640 1 -- start start 2024-04-24T06:39:19.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 0x7fa7140c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 0x7fa7140b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7140b0fe0 con 0x7fa71415b8e0 2024-04-24T06:39:19.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7140cd280 con 0x7fa7140d0080 2024-04-24T06:39:19.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.496+0000 7fa719200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa7140cd560 con 0x7fa7140d1a00 2024-04-24T06:39:19.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40106/0 (socket says 172.21.15.59:40106) 2024-04-24T06:39:19.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 -- 172.21.15.59:0/270579235 learned_addr learned my addr 172.21.15.59:0/270579235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa711e00640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 0x7fa7140c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 -- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 msgr2=0x7fa7140b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa711400640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 0x7fa7140b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 0x7fa7140b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 -- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 msgr2=0x7fa7140c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 0x7fa7140c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 -- 172.21.15.59:0/270579235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7140c9590 con 0x7fa71415b8e0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa711e00640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 0x7fa7140c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa711400640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 0x7fa7140b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.497+0000 7fa712800640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00b410 tx=0x7fa70c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.498+0000 7fa70a000640 1 -- 172.21.15.59:0/270579235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c015020 con 0x7fa71415b8e0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.498+0000 7fa70a000640 1 -- 172.21.15.59:0/270579235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c011850 con 0x7fa71415b8e0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.498+0000 7fa70a000640 1 -- 172.21.15.59:0/270579235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c011c40 con 0x7fa71415b8e0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.498+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7140c8fd0 con 0x7fa71415b8e0 2024-04-24T06:39:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.498+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7140b45b0 con 0x7fa71415b8e0 2024-04-24T06:39:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.501+0000 7fa70a000640 1 -- 172.21.15.59:0/270579235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa70c018020 con 0x7fa71415b8e0 2024-04-24T06:39:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.502+0000 7fa70a000640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 0x7fa6f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.502+0000 7fa70a000640 1 -- 172.21.15.59:0/270579235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fa70c08bae0 con 0x7fa71415b8e0 2024-04-24T06:39:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.503+0000 7fa71c00e740 1 Processor -- start 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.503+0000 7fa711e00640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 0x7fa6f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.503+0000 7fa71c00e740 1 -- start start 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.503+0000 7fa711e00640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 0x7fa6f006edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa704036000 tx=0x7fa704009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dec1c30 0x55ad6de745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dea4610 0x55ad6de79420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ddde1f0 con 0x55ad6dea4d50 2024-04-24T06:39:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ad6dd84350 con 0x55ad6dec1c30 2024-04-24T06:39:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ad6ddda1b0 con 0x55ad6dea4610 2024-04-24T06:39:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa711400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa711400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40118/0 (socket says 172.21.15.59:40118) 2024-04-24T06:39:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa711400640 1 -- 172.21.15.59:0/3709535514 learned_addr learned my addr 172.21.15.59:0/3709535514 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa711e00640 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dec1c30 0x55ad6de745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa712800640 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dea4610 0x55ad6de79420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.504+0000 7fa711400640 1 -- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dea4610 msgr2=0x55ad6de79420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa711400640 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dea4610 0x55ad6de79420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa711400640 1 -- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dec1c30 msgr2=0x55ad6de745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa711400640 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dec1c30 0x55ad6de745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa711400640 1 -- 172.21.15.59:0/3709535514 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6ddd8f50 con 0x55ad6dea4d50 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa711400640 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00a9a0 tx=0x7fa6fc00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447c1c1ace42fc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa702a00640 1 -- 172.21.15.59:0/3709535514 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc015070 con 0x55ad6dea4d50 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa702a00640 1 -- 172.21.15.59:0/3709535514 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc00cc00 con 0x55ad6dea4d50 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.505+0000 7fa702a00640 1 -- 172.21.15.59:0/3709535514 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0113f0 con 0x55ad6dea4d50 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 -- 172.21.15.59:0/3709535514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 msgr2=0x55ad6dea40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00a9a0 tx=0x7fa6fc00ae60 comp rx=0 tx=0).stop 2024-04-24T06:39:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 -- 172.21.15.59:0/3709535514 shutdown_connections 2024-04-24T06:39:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dea4610 0x55ad6de79420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6dea4d50 0x55ad6dea40d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 --2- 172.21.15.59:0/3709535514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dec1c30 0x55ad6de745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 -- 172.21.15.59:0/3709535514 >> 172.21.15.59:0/3709535514 conn(0x55ad6de76830 msgr2=0x55ad6debf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 -- 172.21.15.59:0/3709535514 shutdown_connections 2024-04-24T06:39:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.506+0000 7fa71c00e740 1 -- 172.21.15.59:0/3709535514 wait complete. 2024-04-24T06:39:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.507+0000 7fa71c00e740 1 -- start start 2024-04-24T06:39:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.507+0000 7fa71c00e740 1 Processor -- start 2024-04-24T06:39:19.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6decfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad6de789e0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ad6dd97690 con 0x55ad6dea4d50 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ad6de77dd0 con 0x55ad6dec1c30 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa71c00e740 10 client.0 _ll_register_callbacks cb 0x55ad6ddd3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa712800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa712800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40122/0 (socket says 172.21.15.59:40122) 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.508+0000 7fa712800640 1 -- 172.21.15.59:0/4162316485 learned_addr learned my addr 172.21.15.59:0/4162316485 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 -- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 msgr2=0x55ad6dea2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 -- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 msgr2=0x55ad6decfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6decfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 -- 172.21.15.59:0/4162316485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6dd96920 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa712800640 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa70c089f10 tx=0x7fa70c017400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c92680cedb8b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa6f7400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c088060 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fa70c055de0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.509+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6fc00cc00 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa70c076050 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 10 client.8865 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55ad6dbba7d0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc00cc00 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fa70c058bc0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 10 client.8865 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 10 client.8865 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:19.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 20 client.8865 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6dea7630 0x55ad6dea7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 4 client.8865 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.510+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6ddebc10 con 0x55ad6dea7630 2024-04-24T06:39:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.511+0000 7fa711e00640 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6dea7630 0x55ad6dea7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.512+0000 7fa711e00640 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6dea7630 0x55ad6dea7ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c402901ff8945cd server_cookie=dd831bfc9f3c01d7 in_seq=0 out_seq=0 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.512+0000 7fa6f7400640 10 client.8865 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa6f7400640 1 -- 172.21.15.59:0/4162316485 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad6ddebc10 con 0x55ad6dea7630 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa6f7400640 10 client.8865 handle_command_reply: tid=0 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 client.8865 shutdown 2024-04-24T06:39:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 msgr2=0x55ad6dea6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa70c089f10 tx=0x7fa70c017400 comp rx=0 tx=0).stop 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 shutdown_connections 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6dea7630 0x55ad6dea7ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea6400 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa6f7400640 0 client.8865 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4162316485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6decfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 >> 172.21.15.59:0/4162316485 conn(0x55ad6dea4610 msgr2=0x55ad6dea13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 shutdown_connections 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.514+0000 7fa71c00e740 1 -- 172.21.15.59:0/4162316485 wait complete. 2024-04-24T06:39:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.515+0000 7fa71c00e740 20 client.8865 trim_cache size 0 max 16384 2024-04-24T06:39:19.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.534+0000 7fa71c00e740 1 Processor -- start 2024-04-24T06:39:19.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.534+0000 7fa71c00e740 1 -- start start 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6dea3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa714201100 con 0x55ad6de9fbe0 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa714200100 con 0x55ad6dea4d50 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa7141f8130 con 0x55ad6dec1c30 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa711e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa711e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40134/0 (socket says 172.21.15.59:40134) 2024-04-24T06:39:19.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.535+0000 7fa711e00640 1 -- 172.21.15.59:0/4202780397 learned_addr learned my addr 172.21.15.59:0/4202780397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa712800640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 -- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 msgr2=0x55ad6dea35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 -- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 msgr2=0x55ad6dea3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711400640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6dea3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6dea3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 -- 172.21.15.59:0/4202780397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6de77970 con 0x55ad6de9fbe0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa712800640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa711e00640 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa70402cea0 tx=0x7fa704031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ba419cb56e1fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.536+0000 7fa702000640 1 -- 172.21.15.59:0/4202780397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70403db70 con 0x55ad6de9fbe0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa702000640 1 -- 172.21.15.59:0/4202780397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70403dd10 con 0x55ad6de9fbe0 2024-04-24T06:39:19.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa702000640 1 -- 172.21.15.59:0/4202780397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70404c360 con 0x55ad6de9fbe0 2024-04-24T06:39:19.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 -- 172.21.15.59:0/4202780397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 msgr2=0x55ad6dea7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa70402cea0 tx=0x7fa704031000 comp rx=0 tx=0).stop 2024-04-24T06:39:19.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 -- 172.21.15.59:0/4202780397 shutdown_connections 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6dea35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6dea3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 --2- 172.21.15.59:0/4202780397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6dea7c90 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 -- 172.21.15.59:0/4202780397 >> 172.21.15.59:0/4202780397 conn(0x55ad6dea4610 msgr2=0x55ad6dea61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 -- 172.21.15.59:0/4202780397 shutdown_connections 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.537+0000 7fa71c00e740 1 -- 172.21.15.59:0/4202780397 wait complete. 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.538+0000 7fa71c00e740 1 -- start start 2024-04-24T06:39:19.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.538+0000 7fa71c00e740 1 Processor -- start 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6debbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6df38d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad6de9f240 con 0x55ad6de9fbe0 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141f8780 con 0x55ad6dea4d50 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55ad6de74490 con 0x55ad6dec1c30 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.539+0000 7fa71c00e740 10 client.0 _ll_register_callbacks cb 0x7fa70c060130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40150/0 (socket says 172.21.15.59:40150) 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 -- 172.21.15.59:0/357562398 learned_addr learned my addr 172.21.15.59:0/357562398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa712800640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6df38d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 -- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 msgr2=0x55ad6df38d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6df38d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711400640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6debbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 -- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 msgr2=0x55ad6debbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6debbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa711e00640 1 -- 172.21.15.59:0/357562398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6daf03f0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.540+0000 7fa712800640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6df38d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa711e00640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa7040072a0 tx=0x7fa704004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3acb825522c376d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa6f7400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704003b90 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fa704003d30 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70403dd10 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa70404c160 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa71c00e740 10 client.8871 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.541+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55ad6dab00f0 con 0x55ad6de9fbe0 2024-04-24T06:39:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70403dd10 con 0x55ad6de9fbe0 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fa704008040 con 0x55ad6de9fbe0 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 10 client.8871 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 10 client.8871 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 20 client.8871 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6deb6260 0x55ad6df39260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 4 client.8871 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:39:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.542+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad6dde5a80 con 0x55ad6deb6260 2024-04-24T06:39:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.543+0000 7fa711400640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6deb6260 0x55ad6df39260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa711400640 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6deb6260 0x55ad6df39260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec35d521d3a3c642 server_cookie=87c2474228a1b5e0 in_seq=0 out_seq=0 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa6f7400640 10 client.8871 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa6f7400640 1 -- 172.21.15.59:0/357562398 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55ad6dde5a80 con 0x55ad6deb6260 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa6f7400640 10 client.8871 handle_command_reply: tid=0 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 client.8871 shutdown 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 msgr2=0x55ad6debb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa7040072a0 tx=0x7fa704004ab0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 shutdown_connections 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55ad6dec1c30 0x55ad6df38d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ad6dea4d50 0x55ad6debbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55ad6deb6260 0x55ad6df39260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 --2- 172.21.15.59:0/357562398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad6de9fbe0 0x55ad6debb860 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 >> 172.21.15.59:0/357562398 conn(0x55ad6dea4610 msgr2=0x55ad6dea6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.544+0000 7fa6f7400640 0 client.8871 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 shutdown_connections 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa71c00e740 1 -- 172.21.15.59:0/357562398 wait complete. 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa71c00e740 20 client.8871 trim_cache size 0 max 16384 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout: "6fd9c59b-f930-432b-8ce3-480c7d6becdd": { 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-24T06:39:19.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "6fd9c59b-f930-432b-8ce3-480c7d6becdd", 2024-04-24T06:39:19.549 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:39:19.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:39:19.549 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:39:19.549 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:19.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 msgr2=0x7fa6f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 0x7fa6f006edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa704036000 tx=0x7fa704009000 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 msgr2=0x7fa7140c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00b410 tx=0x7fa70c00b8d0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.545+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 shutdown_connections 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa6f006c8f0 0x7fa6f006edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa71415b8e0 0x7fa7140c8630 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa7140d1a00 0x7fa7140b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 --2- 172.21.15.59:0/270579235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7140d0080 0x7fa7140c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 >> 172.21.15.59:0/270579235 conn(0x7fa714073430 msgr2=0x7fa7140cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 shutdown_connections 2024-04-24T06:39:19.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:19.546+0000 7fa719200640 1 -- 172.21.15.59:0/270579235 wait complete. 2024-04-24T06:39:19.565 INFO:tasks.cephfs.filesystem:scrub status for tag:6fd9c59b-f930-432b-8ce3-480c7d6becdd - {'path': '/', 'tag': '6fd9c59b-f930-432b-8ce3-480c7d6becdd', 'options': 'recursive,repair'} 2024-04-24T06:39:20.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:20.620+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-24T06:39:19.269042+0000) ==== 695+0+0 (secure 0 0 0) 0x7fb194057390 con 0x7fb1a8146000 2024-04-24T06:39:20.621 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-24T06:39:20.621 INFO:teuthology.orchestra.run.smithi059.stdout: id: fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:39:20.621 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 10m), standbys: x 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-24T06:39:20.622 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 30 objects, 19 MiB 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-24T06:39:20.623 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout: client: 1.0 MiB/s wr, 0 op/s rd, 1 op/s wr 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:19.269042+0000 mds.d [INF] scrub queued for path: / 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:19.269050+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:19.269060+0000 mds.d [INF] scrub summary: active paths [/] 2024-04-24T06:39:20.624 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:19.272066+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:39:24.634 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:23.815203+0000 mds.d [INF] scrub summary: idle 2024-04-24T06:39:24.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:24.633+0000 7fb1a5600640 1 -- 172.21.15.59:0/454523889 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-24T06:39:23.815203+0000) ==== 189+0+0 (secure 0 0 0) 0x7fb1940575f0 con 0x7fb1a8146000 2024-04-24T06:39:38.155 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:39:38.163 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:39:49.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:49.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T06:39:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.794+0000 7ff262e00640 1 Processor -- start 2024-04-24T06:39:49.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.795+0000 7ff262e00640 1 -- start start 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.796+0000 7ff262e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c0b1850 0x7ff25c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff262e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff25c09aa00 0x7ff25c09ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff262e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff25c0d15c0 con 0x7ff25c09b7d0 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff262e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff25c0d0a20 con 0x7ff25c09aa00 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff262e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0d00f0 con 0x7ff25c0b1850 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff261000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff25c09b7d0 0x7ff25c0d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34292/0 (socket says 172.21.15.59:34292) 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff261000640 1 -- 172.21.15.59:0/477417732 learned_addr learned my addr 172.21.15.59:0/477417732 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:49.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff262400640 1 --2- 172.21.15.59:0/477417732 >> v2:172.21.15.59:3300/0 conn(0x7ff25c09aa00 0x7ff25c09ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.797+0000 7ff261a00640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c0b1850 0x7ff25c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.798+0000 7ff262400640 1 -- 172.21.15.59:0/477417732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c0b1850 msgr2=0x7ff25c0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.798+0000 7ff262400640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c0b1850 0x7ff25c0cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.798+0000 7ff262400640 1 -- 172.21.15.59:0/477417732 >> v1:172.21.15.59:6789/0 conn(0x7ff25c09b7d0 legacy=0x7ff25c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.798+0000 7ff25be00640 1 -- 172.21.15.59:0/477417732 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672774212 0 0) 0x7ff25c0d15c0 con 0x7ff25c09b7d0 2024-04-24T06:39:49.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.798+0000 7ff262400640 1 -- 172.21.15.59:0/477417732 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c0d03d0 con 0x7ff25c09aa00 2024-04-24T06:39:49.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.799+0000 7ff262400640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c09aa00 0x7ff25c09ae90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff25402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a089c7fc692c74b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.799+0000 7ff25be00640 1 -- 172.21.15.59:0/477417732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254037070 con 0x7ff25c09aa00 2024-04-24T06:39:49.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.799+0000 7ff25be00640 1 -- 172.21.15.59:0/477417732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254032440 con 0x7ff25c09aa00 2024-04-24T06:39:49.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.799+0000 7ff25be00640 1 -- 172.21.15.59:0/477417732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254032810 con 0x7ff25c09aa00 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 -- 172.21.15.59:0/477417732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c09aa00 msgr2=0x7ff25c09ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c09aa00 0x7ff25c09ae90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff25402a180 comp rx=0 tx=0).stop 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 -- 172.21.15.59:0/477417732 shutdown_connections 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c09aa00 0x7ff25c09ae90 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 --2- 172.21.15.59:0/477417732 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c0b1850 0x7ff25c0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 -- 172.21.15.59:0/477417732 >> 172.21.15.59:0/477417732 conn(0x7ff25c0b3f70 msgr2=0x7ff25c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 -- 172.21.15.59:0/477417732 shutdown_connections 2024-04-24T06:39:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.800+0000 7ff262e00640 1 -- 172.21.15.59:0/477417732 wait complete. 2024-04-24T06:39:49.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.802+0000 7ff262e00640 1 Processor -- start 2024-04-24T06:39:49.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.802+0000 7ff262e00640 1 -- start start 2024-04-24T06:39:49.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 0x7ff25c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c09b7d0 0x7ff25c0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0ce480 con 0x7ff25c0b1850 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0ce190 con 0x7ff25c09aa00 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0b1dc0 con 0x7ff25c09b7d0 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51138/0 (socket says 172.21.15.59:51138) 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff262400640 1 -- 172.21.15.59:0/3598860683 learned_addr learned my addr 172.21.15.59:0/3598860683 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.803+0000 7ff261a00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 0x7ff25c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff261000640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c09b7d0 0x7ff25c0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 -- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c09b7d0 msgr2=0x7ff25c0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c09b7d0 0x7ff25c0b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 -- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 msgr2=0x7ff25c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 0x7ff25c152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 -- 172.21.15.59:0/3598860683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c0c9550 con 0x7ff25c0b1850 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff261a00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 0x7ff25c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262400640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff2540096f0 tx=0x7ff2540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff259600640 1 -- 172.21.15.59:0/3598860683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254037040 con 0x7ff25c0b1850 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff259600640 1 -- 172.21.15.59:0/3598860683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2540306f0 con 0x7ff25c0b1850 2024-04-24T06:39:49.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff259600640 1 -- 172.21.15.59:0/3598860683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254030ae0 con 0x7ff25c0b1850 2024-04-24T06:39:49.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c0b5dd0 con 0x7ff25c0b1850 2024-04-24T06:39:49.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.804+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c0b6390 con 0x7ff25c0b1850 2024-04-24T06:39:49.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.807+0000 7ff259600640 1 -- 172.21.15.59:0/3598860683 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff254044020 con 0x7ff25c0b1850 2024-04-24T06:39:49.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.808+0000 7ff259600640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 0x7ff23c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.808+0000 7ff259600640 1 -- 172.21.15.59:0/3598860683 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7ff2540abfe0 con 0x7ff25c0b1850 2024-04-24T06:39:49.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.808+0000 7ff261a00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 0x7ff23c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.809+0000 7ff267207740 1 Processor -- start 2024-04-24T06:39:49.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.809+0000 7ff267207740 1 -- start start 2024-04-24T06:39:49.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.809+0000 7ff261a00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 0x7ff23c06edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff250009f50 tx=0x7ff25000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x5562769812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5562769817e0 0x5562769565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562768bb1f0 con 0x55627699ee00 2024-04-24T06:39:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556276861350 con 0x556276981f20 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5562768b71b0 con 0x5562769817e0 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff261a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff261a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51150/0 (socket says 172.21.15.59:51150) 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.810+0000 7ff261a00640 1 -- 172.21.15.59:0/540073599 learned_addr learned my addr 172.21.15.59:0/540073599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261000640 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x5562769812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 -- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5562769817e0 msgr2=0x5562769565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5562769817e0 0x5562769565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 -- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 msgr2=0x5562769812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x5562769812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 -- 172.21.15.59:0/540073599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562768b5f50 con 0x55627699ee00 2024-04-24T06:39:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff261a00640 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff250003dc0 tx=0x7ff250012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d547059d01b488 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff24e000640 1 -- 172.21.15.59:0/540073599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250014620 con 0x55627699ee00 2024-04-24T06:39:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff24e000640 1 -- 172.21.15.59:0/540073599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2500147c0 con 0x55627699ee00 2024-04-24T06:39:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.811+0000 7ff24e000640 1 -- 172.21.15.59:0/540073599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250014b70 con 0x55627699ee00 2024-04-24T06:39:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 -- 172.21.15.59:0/540073599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 msgr2=0x556276951790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff250003dc0 tx=0x7ff250012000 comp rx=0 tx=0).stop 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 -- 172.21.15.59:0/540073599 shutdown_connections 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5562769817e0 0x5562769565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x5562769812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 --2- 172.21.15.59:0/540073599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x556276951790 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 -- 172.21.15.59:0/540073599 >> 172.21.15.59:0/540073599 conn(0x556276953a00 msgr2=0x55627699c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 -- 172.21.15.59:0/540073599 shutdown_connections 2024-04-24T06:39:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.812+0000 7ff267207740 1 -- 172.21.15.59:0/540073599 wait complete. 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.813+0000 7ff267207740 1 -- start start 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.813+0000 7ff267207740 1 Processor -- start 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x55627697ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x556276984620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556276955bb0 con 0x556276981f20 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556276874690 con 0x5562769ac380 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x556276954fa0 con 0x55627699ee00 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.814+0000 7ff267207740 10 client.0 _ll_register_callbacks cb 0x5562768b0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51152/0 (socket says 172.21.15.59:51152) 2024-04-24T06:39:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 -- 172.21.15.59:0/304672643 learned_addr learned my addr 172.21.15.59:0/304672643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff262400640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x556276984620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261000640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x55627697ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 -- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 msgr2=0x55627697ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x55627697ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 -- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 msgr2=0x556276984620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x556276984620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 -- 172.21.15.59:0/304672643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556276873920 con 0x556276981f20 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff262400640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x556276984620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff261a00640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff250007b90 tx=0x7ff250004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1559820af99f1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.815+0000 7ff242a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:49.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250031060 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7ff250014c40 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2500147c0 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ff250037050 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff267207740 10 client.8886 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5562766977d0 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.816+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2500147c0 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7ff25003a020 con 0x556276981f20 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 10 client.8886 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 10 client.8886 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 20 client.8886 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x5562769afec0 0x5562769b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 4 client.8886 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562768c8c10 con 0x5562769afec0 2024-04-24T06:39:49.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.817+0000 7ff261000640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x5562769afec0 0x5562769b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.819+0000 7ff261000640 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x5562769afec0 0x5562769b02f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85853b2745ca62c6 server_cookie=44d42070f1ec80dd in_seq=0 out_seq=0 2024-04-24T06:39:49.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.819+0000 7ff242a00640 10 client.8886 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff242a00640 1 -- 172.21.15.59:0/304672643 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562768c8c10 con 0x5562769afec0 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff242a00640 10 client.8886 handle_command_reply: tid=0 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 client.8886 shutdown 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 msgr2=0x556276995320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff250007b90 tx=0x7ff250004ab0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 shutdown_connections 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x556276984620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x55627697ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x5562769afec0 0x5562769b02f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 --2- 172.21.15.59:0/304672643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556276981f20 0x556276995320 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 >> 172.21.15.59:0/304672643 conn(0x5562769817e0 msgr2=0x55627699c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff242a00640 0 client.8886 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 shutdown_connections 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 1 -- 172.21.15.59:0/304672643 wait complete. 2024-04-24T06:39:49.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.821+0000 7ff267207740 20 client.8886 trim_cache size 0 max 16384 2024-04-24T06:39:49.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.841+0000 7ff267207740 1 Processor -- start 2024-04-24T06:39:49.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.841+0000 7ff267207740 1 -- start start 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x55627699c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x556276994700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c200b90 con 0x5562769ac380 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0cae00 con 0x556276981f20 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c1f87c0 con 0x55627699ee00 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff262400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51166/0 (socket says 172.21.15.59:51166) 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff262400640 1 -- 172.21.15.59:0/3865472122 learned_addr learned my addr 172.21.15.59:0/3865472122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.842+0000 7ff261a00640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x55627699c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff261000640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x556276994700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 -- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 msgr2=0x556276994700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x556276994700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 -- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 msgr2=0x55627699c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x55627699c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 -- 172.21.15.59:0/3865472122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562769537a0 con 0x5562769ac380 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff261a00640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x55627699c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff262400640 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff254079660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1fd59a5f853215 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff24d600640 1 -- 172.21.15.59:0/3865472122 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540a3e60 con 0x5562769ac380 2024-04-24T06:39:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff24d600640 1 -- 172.21.15.59:0/3865472122 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25407e0b0 con 0x5562769ac380 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.843+0000 7ff24d600640 1 -- 172.21.15.59:0/3865472122 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540828e0 con 0x5562769ac380 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 -- 172.21.15.59:0/3865472122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 msgr2=0x556276994c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff2540097a0 tx=0x7ff254079660 comp rx=0 tx=0).stop 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 -- 172.21.15.59:0/3865472122 shutdown_connections 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562769ac380 0x556276994c40 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55627699ee00 0x556276994700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 --2- 172.21.15.59:0/3865472122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556276981f20 0x55627699c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 -- 172.21.15.59:0/3865472122 >> 172.21.15.59:0/3865472122 conn(0x5562769817e0 msgr2=0x55627697cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 -- 172.21.15.59:0/3865472122 shutdown_connections 2024-04-24T06:39:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.844+0000 7ff267207740 1 -- 172.21.15.59:0/3865472122 wait complete. 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.845+0000 7ff267207740 1 -- start start 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.845+0000 7ff267207740 1 Processor -- start 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556276981f20 0x55627699e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x5562769bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556276891840 con 0x55627699ee00 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0ca7b0 con 0x5562769ac380 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5562769b0580 con 0x556276981f20 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff267207740 10 client.0 _ll_register_callbacks cb 0x7ff250003c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51180/0 (socket says 172.21.15.59:51180) 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 -- 172.21.15.59:0/286461090 learned_addr learned my addr 172.21.15.59:0/286461090 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 -- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556276981f20 msgr2=0x55627699e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556276981f20 0x55627699e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.846+0000 7ff261000640 1 -- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 msgr2=0x5562769bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff261000640 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x5562769bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff261000640 1 -- 172.21.15.59:0/286461090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562765cd3f0 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff261000640 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff2480074c0 tx=0x7ff248018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab3b594cde13d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff242a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248018940 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7ff248017440 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25407e0b0 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ff248027050 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff267207740 10 client.8892 fetch_fsmap learned FSMap version 83 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.847+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55627658d0f0 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25407e0b0 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7ff24802a020 con 0x55627699ee00 2024-04-24T06:39:49.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 10 client.8892 fetch_fsmap finished waiting for FSMap version 83 2024-04-24T06:39:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 10 client.8892 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-24T06:39:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 20 client.8892 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55627699c7c0 0x55627699f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 4 client.8892 mds_command: new command op to 8499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:39:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.848+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 --> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562768c2a80 con 0x55627699c7c0 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.849+0000 7ff261a00640 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55627699c7c0 0x55627699f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.849+0000 7ff261a00640 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55627699c7c0 0x55627699f5b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ab4958a80bc7a0 server_cookie=bd991c7f27da4b07 in_seq=0 out_seq=0 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.849+0000 7ff242a00640 10 client.8892 ms_handle_connect on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff242a00640 1 -- 172.21.15.59:0/286461090 <== mds.0 v2:172.21.15.66:6832/891418539 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562768c2a80 con 0x55627699c7c0 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff242a00640 10 client.8892 handle_command_reply: tid=0 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 client.8892 shutdown 2024-04-24T06:39:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 msgr2=0x5562769beea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff2480074c0 tx=0x7ff248018810 comp rx=0 tx=0).stop 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 shutdown_connections 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562769ac380 0x5562769bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:6832/891418539,v1:172.21.15.66:6834/891418539] conn(0x55627699c7c0 0x55627699f5b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55627699ee00 0x5562769beea0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 --2- 172.21.15.59:0/286461090 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x556276981f20 0x55627699e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 >> 172.21.15.59:0/286461090 conn(0x5562769817e0 msgr2=0x55627697d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff242a00640 0 client.8892 ms_handle_reset on v2:172.21.15.66:6832/891418539 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 shutdown_connections 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.850+0000 7ff267207740 1 -- 172.21.15.59:0/286461090 wait complete. 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff267207740 20 client.8892 trim_cache size 0 max 16384 2024-04-24T06:39:49.854 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:49.855 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:39:49.855 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:39:49.855 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:49.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 msgr2=0x7ff23c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 0x7ff23c06edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff250009f50 tx=0x7ff25000b040 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 msgr2=0x7ff25c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff2540096f0 tx=0x7ff2540046c0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 shutdown_connections 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff23c06c8f0 0x7ff23c06edb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff25c0b1850 0x7ff25c0b7140 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff25c09b7d0 0x7ff25c0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 --2- 172.21.15.59:0/3598860683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff25c09aa00 0x7ff25c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.851+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 >> 172.21.15.59:0/3598860683 conn(0x7ff25c0b3f70 msgr2=0x7ff25c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.852+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 shutdown_connections 2024-04-24T06:39:49.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:49.852+0000 7ff262e00640 1 -- 172.21.15.59:0/3598860683 wait complete. 2024-04-24T06:39:49.871 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:39:49.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:39:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.180+0000 7fbb32600640 1 Processor -- start 2024-04-24T06:39:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.180+0000 7fbb32600640 1 -- start start 2024-04-24T06:39:50.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb32600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb2c06ef00 0x7fbb2c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb32600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb2c15bc00 0x7fbb2c15fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb32600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2c06f820 con 0x7fbb2c071d90 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb32600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2c0b2f50 con 0x7fbb2c15bc00 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb32600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2c160140 con 0x7fbb2c06ef00 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb2aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb2c15bc00 0x7fbb2c15fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb2aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb2c15bc00 0x7fbb2c15fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51188/0 (socket says 172.21.15.59:51188) 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb2aa00640 1 -- 172.21.15.59:0/2005117369 learned_addr learned my addr 172.21.15.59:0/2005117369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.182+0000 7fbb2be00640 1 --1- 172.21.15.59:0/2005117369 >> v1:172.21.15.59:6789/0 conn(0x7fbb2c071d90 0x7fbb2c0721c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34304/0 (socket says 172.21.15.59:34304) 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.183+0000 7fbb2b400640 1 --2- 172.21.15.59:0/2005117369 >> v2:172.21.15.66:3300/0 conn(0x7fbb2c06ef00 0x7fbb2c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.183+0000 7fbb2aa00640 1 -- 172.21.15.59:0/2005117369 >> v2:172.21.15.66:3300/0 conn(0x7fbb2c06ef00 msgr2=0x7fbb2c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.183+0000 7fbb2aa00640 1 --2- 172.21.15.59:0/2005117369 >> v2:172.21.15.66:3300/0 conn(0x7fbb2c06ef00 0x7fbb2c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.183+0000 7fbb2aa00640 1 -- 172.21.15.59:0/2005117369 >> v1:172.21.15.59:6789/0 conn(0x7fbb2c071d90 legacy=0x7fbb2c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.183+0000 7fbb2aa00640 1 -- 172.21.15.59:0/2005117369 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c15c180 con 0x7fbb2c15bc00 2024-04-24T06:39:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.184+0000 7fbb2aa00640 1 --2- 172.21.15.59:0/2005117369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c15bc00 0x7fbb2c15fc00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c063790 tx=0x7fbb1802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2958a87dd1d99a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.184+0000 7fbb2a000640 1 -- 172.21.15.59:0/2005117369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18037070 con 0x7fbb2c15bc00 2024-04-24T06:39:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.184+0000 7fbb2a000640 1 -- 172.21.15.59:0/2005117369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb18032440 con 0x7fbb2c15bc00 2024-04-24T06:39:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.184+0000 7fbb2a000640 1 -- 172.21.15.59:0/2005117369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18032810 con 0x7fbb2c15bc00 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 -- 172.21.15.59:0/2005117369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c15bc00 msgr2=0x7fbb2c15fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 --2- 172.21.15.59:0/2005117369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c15bc00 0x7fbb2c15fc00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c063790 tx=0x7fbb1802a2a0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 -- 172.21.15.59:0/2005117369 shutdown_connections 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 --2- 172.21.15.59:0/2005117369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c15bc00 0x7fbb2c15fc00 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 --2- 172.21.15.59:0/2005117369 >> v2:172.21.15.66:3300/0 conn(0x7fbb2c06ef00 0x7fbb2c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 -- 172.21.15.59:0/2005117369 >> 172.21.15.59:0/2005117369 conn(0x7fbb2c0b2130 msgr2=0x7fbb2c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 -- 172.21.15.59:0/2005117369 shutdown_connections 2024-04-24T06:39:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.185+0000 7fbb32600640 1 -- 172.21.15.59:0/2005117369 wait complete. 2024-04-24T06:39:50.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.187+0000 7fbb32600640 1 Processor -- start 2024-04-24T06:39:50.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.187+0000 7fbb32600640 1 -- start start 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c06ef00 0x7fbb2c14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb2c15bc00 0x7fbb2c143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c15ca20 con 0x7fbb2c071d90 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c15c740 con 0x7fbb2c06ef00 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.188+0000 7fbb32600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c0ce210 con 0x7fbb2c15bc00 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51196/0 (socket says 172.21.15.59:51196) 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 -- 172.21.15.59:0/805921337 learned_addr learned my addr 172.21.15.59:0/805921337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 -- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb2c15bc00 msgr2=0x7fbb2c143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb2c15bc00 0x7fbb2c143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 -- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c06ef00 msgr2=0x7fbb2c14c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c06ef00 0x7fbb2c14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.189+0000 7fbb2aa00640 1 -- 172.21.15.59:0/805921337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c147a30 con 0x7fbb2c071d90 2024-04-24T06:39:50.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb2aa00640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18041020 con 0x7fbb2c071d90 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb180306c0 con 0x7fbb2c071d90 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18037040 con 0x7fbb2c071d90 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c14d880 con 0x7fbb2c071d90 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.190+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c0b3220 con 0x7fbb2c071d90 2024-04-24T06:39:50.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.191+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c14d670 con 0x7fbb2c071d90 2024-04-24T06:39:50.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.192+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb18044020 con 0x7fbb2c071d90 2024-04-24T06:39:50.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.193+0000 7fbb23400640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 0x7fbb0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.193+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fbb180ac3d0 con 0x7fbb2c071d90 2024-04-24T06:39:50.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.194+0000 7fbb2b400640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 0x7fbb0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.195+0000 7fbb2b400640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 0x7fbb0c06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c15fc00 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.199+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb1804b030 con 0x7fbb2c071d90 2024-04-24T06:39:50.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.446+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbb2c068a80 con 0x7fbb2c071d90 2024-04-24T06:39:50.448 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:39:50.447+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:39:50.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.450+0000 7fbb23400640 1 -- 172.21.15.59:0/805921337 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fbb1807a760 con 0x7fbb2c071d90 2024-04-24T06:39:50.452 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:39:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 msgr2=0x7fbb0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 0x7fbb0c06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c15fc00 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 msgr2=0x7fbb2c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb180046c0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 shutdown_connections 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb0c06c8f0 0x7fbb0c06edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb2c15bc00 0x7fbb2c143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb2c071d90 0x7fbb2c154c60 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 --2- 172.21.15.59:0/805921337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb2c06ef00 0x7fbb2c14c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.453+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 >> 172.21.15.59:0/805921337 conn(0x7fbb2c0b2130 msgr2=0x7fbb2c14e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.454+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 shutdown_connections 2024-04-24T06:39:50.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.454+0000 7fbb32600640 1 -- 172.21.15.59:0/805921337 wait complete. 2024-04-24T06:39:50.475 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-24T06:39:50.496 INFO:tasks.ceph.mds.d.smithi066.stderr: -6888> 2024-04-24T06:38:40.125+0000 7f46c6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:39:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.572+0000 7f931e62c280 1 -- start start 2024-04-24T06:39:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.572+0000 7f931e62c280 1 Processor -- start 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c7985c00 0x5566c7af9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 0x5566c7af8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772000 con 0x5566c871c000 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772180 con 0x5566c871c800 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.573+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5566c8772300 con 0x5566c7985c00 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51220/0 (socket says 172.21.15.59:51220) 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 -- 172.21.15.59:0/996802773 learned_addr learned my addr 172.21.15.59:0/996802773 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931c000640 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 0x5566c7af8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 -- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c7985c00 msgr2=0x5566c7af9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c7985c00 0x5566c7af9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 -- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 msgr2=0x5566c7af8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 0x5566c7af8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931b600640 1 -- 172.21.15.59:0/996802773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c79b8780 con 0x5566c871c000 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.574+0000 7f931c000640 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 0x5566c7af8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f931b600640 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x5566c87b2cc0 tx=0x5566c871eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9d79cc66d989a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f9318400640 1 -- 172.21.15.59:0/996802773 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aab60 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f9318400640 1 -- 172.21.15.59:0/996802773 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5566c79b0e00 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f9318400640 1 -- 172.21.15.59:0/996802773 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aa820 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c79b94a0 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5566c79b8960 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f931e62c280 4 waiting for MDS map... 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f9318400640 1 -- 172.21.15.59:0/996802773 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x5566c8850200 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f9318400640 1 -- 172.21.15.59:0/996802773 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x5566c79df180 con 0x5566c871c000 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.575+0000 7f931e62c280 4 Got MDS map 84 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.576+0000 7f931e62c280 10 main: JournalTool::main 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.576+0000 7f931e62c280 4 main: JournalTool: connecting to RADOS... 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.576+0000 7f931e62c280 1 Processor -- start 2024-04-24T06:39:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 -- start start 2024-04-24T06:39:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772480 con 0x5566c8863c00 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772600 con 0x5566c8863400 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.577+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5566c8772780 con 0x5566c8863800 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931c000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:34388/0 (socket says 172.21.15.59:34388) 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931c000640 1 -- 172.21.15.59:0/4179769668 learned_addr learned my addr 172.21.15.59:0/4179769668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 -- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 msgr2=0x5566c7af8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 -- 172.21.15.59:0/4179769668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863c00 msgr2=0x5566c7af9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.578+0000 7f931b600640 1 -- 172.21.15.59:0/4179769668 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c79b83c0 con 0x5566c8863800 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931b600640 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 0x5566c7af9600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5566c87b35f0 tx=0x5566c871ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c48a249780b3678 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f9315c00640 1 -- 172.21.15.59:0/4179769668 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aab60 con 0x5566c8863800 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f9315c00640 1 -- 172.21.15.59:0/4179769668 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5566c79b0c40 con 0x5566c8863800 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f9315c00640 1 -- 172.21.15.59:0/4179769668 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aa820 con 0x5566c8863800 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 -- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 msgr2=0x5566c7af9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 0x5566c7af9600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5566c87b35f0 tx=0x5566c871ff20 comp rx=0 tx=0).stop 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 -- 172.21.15.59:0/4179769668 shutdown_connections 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 --2- 172.21.15.59:0/4179769668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.579+0000 7f931e62c280 1 -- 172.21.15.59:0/4179769668 >> 172.21.15.59:0/4179769668 conn(0x5566c871c400 msgr2=0x5566c7a69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.580+0000 7f931e62c280 1 -- 172.21.15.59:0/4179769668 shutdown_connections 2024-04-24T06:39:50.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.580+0000 7f931e62c280 1 -- 172.21.15.59:0/4179769668 wait complete. 2024-04-24T06:39:50.584 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:39:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.580+0000 7f931e62c280 1 Processor -- start 2024-04-24T06:39:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.580+0000 7f931e62c280 1 -- start start 2024-04-24T06:39:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772a80 con 0x5566c8863400 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5566c8772900 con 0x5566c8863800 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931e62c280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5566c8773500 con 0x5566c8863c00 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.581+0000 7f931c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51238/0 (socket says 172.21.15.59:51238) 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 learned_addr learned my addr 172.21.15.59:0/1433973146 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931ac00640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931b600640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863c00 msgr2=0x5566c7af9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c8863c00 0x5566c7af9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863800 msgr2=0x5566c7af9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c8863800 0x5566c7af9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c79b8780 con 0x5566c8863400 2024-04-24T06:39:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f931c000640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x5566c885b560 tx=0x5566c886acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f9313e00640 1 -- 172.21.15.59:0/1433973146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aab60 con 0x5566c8863400 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f9313e00640 1 -- 172.21.15.59:0/1433973146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5566c79b0700 con 0x5566c8863400 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.582+0000 7f9313e00640 1 -- 172.21.15.59:0/1433973146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5566c79aa820 con 0x5566c8863400 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.583+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566c79b8000 con 0x5566c8863400 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.583+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c79b9680 con 0x5566c8863400 2024-04-24T06:39:50.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.583+0000 7f931e62c280 4 main: JournalTool: resolving pool 14 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.584+0000 7f9313e00640 1 -- 172.21.15.59:0/1433973146 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5566c88cac00 con 0x5566c8863400 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f9313e00640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 0x5566c7afa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f9313e00640 1 -- 172.21.15.59:0/1433973146 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x5566c88bd000 con 0x5566c8863400 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931e62c280 4 main: JournalTool: creating IoCtx.. 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931e62c280 4 main: Executing for rank 0 2024-04-24T06:39:50.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931e62c280 4 Filtering by inode '1099511627777' 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5566c88d6c00 0x5566c7afa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x5566c88d7000 con 0x5566c88d6c00 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.585+0000 7f931b600640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 0x5566c7afa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.586+0000 7f931ac00640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5566c88d6c00 0x5566c7afa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.586+0000 7f931b600640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 0x5566c7afa100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x5566c87b35f0 tx=0x5566c871e9c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.586+0000 7f931ac00640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5566c88d6c00 0x5566c7afa680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.590+0000 7f931ac00640 1 -- 172.21.15.59:0/1433973146 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5566c870e000 con 0x5566c88d6c00 2024-04-24T06:39:50.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.590+0000 7f931e62c280 4 Pointer 400.00000000 is readable 2024-04-24T06:39:50.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.590+0000 7f931e62c280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-24T06:39:50.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.590+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5566c88d7400 0x5566c7afac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.590+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x5566c88d7800 con 0x5566c88d7400 2024-04-24T06:39:50.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.591+0000 7f931c000640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5566c88d7400 0x5566c7afac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.591+0000 7f931c000640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5566c88d7400 0x5566c7afac00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.592+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5566c870e500 con 0x5566c88d7400 2024-04-24T06:39:50.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.592+0000 7f931e62c280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-24T06:39:50.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.592+0000 7f931e62c280 10 Starting journal scan from offset 0x405f0e 2024-04-24T06:39:50.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.592+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x5566c88d7c00 con 0x5566c88d7400 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931c000640 1 -- 172.21.15.59:0/1433973146 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5566c870e500 con 0x5566c88d7400 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 10 Parsing data, 0x37c bytes available 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 10 Attempting decode at 0x405f0e 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 10 Consumed 0x37c bytes 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 10 Valid entry at 0x405f0e 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566c7d1a000 0x5566c7afb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x5566c7d1a400 con 0x5566c7d1a000 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.593+0000 7f931b600640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566c7d1a000 0x5566c7afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.594+0000 7f931b600640 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566c7d1a000 0x5566c7afb180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931b600640 1 -- 172.21.15.59:0/1433973146 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5566c870e780 con 0x5566c7d1a000 2024-04-24T06:39:50.599 INFO:teuthology.orchestra.run.smithi059.stdout:Events by type: 2024-04-24T06:39:50.599 INFO:teuthology.orchestra.run.smithi059.stdout:Errors: 0 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 Reached end of journal objects 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 Scanned objects, 0 missing, 1 valid 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 Events scanned, 0 gaps 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 Found 1 valid events 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 Selected 0 events events for processing 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 4 0 events 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566c7d1a000 msgr2=0x5566c7afb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x5566c7d1a000 0x5566c7afb180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5566c88d7400 msgr2=0x5566c7afac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x5566c88d7400 0x5566c7afac00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5566c88d6c00 msgr2=0x5566c7afa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x5566c88d6c00 0x5566c7afa680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931ac00640 1 -- 172.21.15.59:0/1433973146 reap_dead start 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 msgr2=0x5566c7afa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 0x5566c7afa100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x5566c87b35f0 tx=0x5566c871e9c0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 msgr2=0x5566c7af8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x5566c885b560 tx=0x5566c886acc0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 shutdown_connections 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x5566c88d6800 0x5566c7afa100 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 --2- 172.21.15.59:0/1433973146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c8863400 0x5566c7af8000 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 >> 172.21.15.59:0/1433973146 conn(0x5566c871c400 msgr2=0x5566c7a69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 shutdown_connections 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.597+0000 7f931e62c280 1 -- 172.21.15.59:0/1433973146 wait complete. 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 msgr2=0x5566c7af8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x5566c87b2cc0 tx=0x5566c871eae0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 shutdown_connections 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566c871c800 0x5566c7af8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5566c7985c00 0x5566c7af9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 --2- 172.21.15.59:0/996802773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566c871c000 0x5566c7af8580 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 >> 172.21.15.59:0/996802773 conn(0x5566c7985000 msgr2=0x5566c7a6a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 shutdown_connections 2024-04-24T06:39:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.598+0000 7f931e62c280 1 -- 172.21.15.59:0/996802773 wait complete. 2024-04-24T06:39:50.607 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.132114, result: Events by type: Errors: 0 2024-04-24T06:39:50.607 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-24T06:39:50.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.714+0000 7f9f187e0280 1 -- start start 2024-04-24T06:39:50.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.714+0000 7f9f187e0280 1 Processor -- start 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56cee0c00 0x55b56dcb4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56cee1c00 0x55b56dcb4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc78180 con 0x55b56cee1800 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc78300 con 0x55b56cee1c00 2024-04-24T06:39:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.715+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b56dc78480 con 0x55b56cee0c00 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51248/0 (socket says 172.21.15.59:51248) 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 -- 172.21.15.59:0/2068987197 learned_addr learned my addr 172.21.15.59:0/2068987197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f17200640 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56cee1c00 0x55b56dcb4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f16800640 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56cee0c00 0x55b56dcb4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 -- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56cee0c00 msgr2=0x55b56dcb4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56cee0c00 0x55b56dcb4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 -- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56cee1c00 msgr2=0x55b56dcb4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56cee1c00 0x55b56dcb4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.716+0000 7f9f15e00640 1 -- 172.21.15.59:0/2068987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56cf14780 con 0x55b56cee1800 2024-04-24T06:39:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f15e00640 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55b56dd0d380 tx=0x55b56dc7c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d6be95fb9b8e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f13600640 1 -- 172.21.15.59:0/2068987197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf06680 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f13600640 1 -- 172.21.15.59:0/2068987197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b56cf0ca80 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f13600640 1 -- 172.21.15.59:0/2068987197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf06b60 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56cf154a0 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b56cf14000 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f187e0280 4 waiting for MDS map... 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.717+0000 7f9f13600640 1 -- 172.21.15.59:0/2068987197 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x55b56ddb0600 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.718+0000 7f9f13600640 1 -- 172.21.15.59:0/2068987197 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x55b56cf3b180 con 0x55b56cee1800 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.718+0000 7f9f187e0280 4 Got MDS map 84 2024-04-24T06:39:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.718+0000 7f9f187e0280 10 main: JournalTool::main 2024-04-24T06:39:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.718+0000 7f9f187e0280 4 main: JournalTool: connecting to RADOS... 2024-04-24T06:39:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.719+0000 7f9f187e0280 1 Processor -- start 2024-04-24T06:39:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.719+0000 7f9f187e0280 1 -- start start 2024-04-24T06:39:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc78600 con 0x55b56ddb3c00 2024-04-24T06:39:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc78780 con 0x55b56ddb3800 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b56dc78900 con 0x55b56ddb3400 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51262/0 (socket says 172.21.15.59:51262) 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f15e00640 1 -- 172.21.15.59:0/204812045 learned_addr learned my addr 172.21.15.59:0/204812045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f17200640 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.720+0000 7f9f16800640 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 -- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3400 msgr2=0x55b56dcb5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 -- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3800 msgr2=0x55b56dcb5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 -- 172.21.15.59:0/204812045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56cf143c0 con 0x55b56ddb3c00 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f15e00640 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55b56ddbe690 tx=0x55b56cf95380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b0f9c0e2f74e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f10e00640 1 -- 172.21.15.59:0/204812045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf06820 con 0x55b56ddb3c00 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f10e00640 1 -- 172.21.15.59:0/204812045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b56cf0cfc0 con 0x55b56ddb3c00 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.721+0000 7f9f10e00640 1 -- 172.21.15.59:0/204812045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf06680 con 0x55b56ddb3c00 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 -- 172.21.15.59:0/204812045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 msgr2=0x55b56dcb5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55b56ddbe690 tx=0x55b56cf95380 comp rx=0 tx=0).stop 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 -- 172.21.15.59:0/204812045 shutdown_connections 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 --2- 172.21.15.59:0/204812045 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 -- 172.21.15.59:0/204812045 >> 172.21.15.59:0/204812045 conn(0x55b56ddb3000 msgr2=0x55b56dc67000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 -- 172.21.15.59:0/204812045 shutdown_connections 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.722+0000 7f9f187e0280 1 -- 172.21.15.59:0/204812045 wait complete. 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.723+0000 7f9f187e0280 1 Processor -- start 2024-04-24T06:39:50.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.723+0000 7f9f187e0280 1 -- start start 2024-04-24T06:39:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc79c80 con 0x55b56ddb3c00 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b56dc79b00 con 0x55b56ddb3400 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f187e0280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b56dc79980 con 0x55b56ddb3800 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f15e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51270/0 (socket says 172.21.15.59:51270) 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f15e00640 1 -- 172.21.15.59:0/3737562158 learned_addr learned my addr 172.21.15.59:0/3737562158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f17200640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.724+0000 7f9f16800640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3800 msgr2=0x55b56dcb5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56ddb3800 0x55b56dcb5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3400 msgr2=0x55b56dcb5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56ddb3400 0x55b56dcb5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56cf14780 con 0x55b56ddb3c00 2024-04-24T06:39:50.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.725+0000 7f9f15e00640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x55b56ddbf350 tx=0x55b56ddc2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f0f000640 1 -- 172.21.15.59:0/3737562158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf06b60 con 0x55b56ddb3c00 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f0f000640 1 -- 172.21.15.59:0/3737562158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b56cf0ce00 con 0x55b56ddb3c00 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f0f000640 1 -- 172.21.15.59:0/3737562158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b56cf064e0 con 0x55b56ddb3c00 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b56cf154a0 con 0x55b56ddb3c00 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56cf14960 con 0x55b56ddb3c00 2024-04-24T06:39:50.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.726+0000 7f9f187e0280 4 main: JournalTool: resolving pool 14 2024-04-24T06:39:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.728+0000 7f9f0f000640 1 -- 172.21.15.59:0/3737562158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55b56de22c00 con 0x55b56ddb3c00 2024-04-24T06:39:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.728+0000 7f9f0f000640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 0x55b56dcb6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f0f000640 1 -- 172.21.15.59:0/3737562158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x55b56de26600 con 0x55b56ddb3c00 2024-04-24T06:39:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f187e0280 4 main: JournalTool: creating IoCtx.. 2024-04-24T06:39:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f187e0280 4 main: Executing for rank 0 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f187e0280 4 Filtering by inode '1099511627778' 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55b56de21400 0x55b56dcb6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55b56de21800 con 0x55b56de21400 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f17200640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 0x55b56dcb6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.729+0000 7f9f17200640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 0x55b56dcb6100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55b56dc726f0 tx=0x55b56ddc30e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.730+0000 7f9f16800640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55b56de21400 0x55b56dcb6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.731+0000 7f9f16800640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55b56de21400 0x55b56dcb6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f16800640 1 -- 172.21.15.59:0/3737562158 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55b56dc6a000 con 0x55b56de21400 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f187e0280 4 Pointer 400.00000000 is readable 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f187e0280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55b56de21c00 0x55b56dcb6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55b56d1eac00 con 0x55b56de21c00 2024-04-24T06:39:50.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.732+0000 7f9f17200640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55b56de21c00 0x55b56dcb6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.733+0000 7f9f17200640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55b56de21c00 0x55b56dcb6c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.734+0000 7f9f17200640 1 -- 172.21.15.59:0/3737562158 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55b56dc6a500 con 0x55b56de21c00 2024-04-24T06:39:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.734+0000 7f9f187e0280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-24T06:39:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.734+0000 7f9f187e0280 10 Starting journal scan from offset 0x405f0e 2024-04-24T06:39:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.734+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55b56de21800 con 0x55b56de21c00 2024-04-24T06:39:50.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.735+0000 7f9f17200640 1 -- 172.21.15.59:0/3737562158 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55b56dc6a500 con 0x55b56de21c00 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 10 Parsing data, 0x37c bytes available 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 10 Attempting decode at 0x405f0e 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 10 Consumed 0x37c bytes 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 10 Valid entry at 0x405f0e 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55b56dcc1400 0x55b56dcb7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55b56dcc1000 con 0x55b56dcc1400 2024-04-24T06:39:50.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.736+0000 7f9f16800640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55b56dcc1400 0x55b56dcb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f16800640 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55b56dcc1400 0x55b56dcb7180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.739 INFO:teuthology.orchestra.run.smithi059.stdout:Events by type: 2024-04-24T06:39:50.739 INFO:teuthology.orchestra.run.smithi059.stdout:Errors: 0 2024-04-24T06:39:50.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f16800640 1 -- 172.21.15.59:0/3737562158 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b56dc6a000 con 0x55b56dcc1400 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 Reached end of journal objects 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 Scanned objects, 0 missing, 1 valid 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 Events scanned, 0 gaps 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 Found 1 valid events 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 Selected 0 events events for processing 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 4 0 events 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55b56dcc1400 msgr2=0x55b56dcb7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55b56dcc1400 0x55b56dcb7180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55b56de21c00 msgr2=0x55b56dcb6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55b56de21c00 0x55b56dcb6c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55b56de21400 msgr2=0x55b56dcb6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55b56de21400 0x55b56dcb6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f16800640 1 -- 172.21.15.59:0/3737562158 reap_dead start 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 msgr2=0x55b56dcb6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 0x55b56dcb6100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55b56dc726f0 tx=0x55b56ddc30e0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 msgr2=0x55b56dcb5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x55b56ddbf350 tx=0x55b56ddc2c00 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 shutdown_connections 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55b56de21000 0x55b56dcb6100 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 --2- 172.21.15.59:0/3737562158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56ddb3c00 0x55b56dcb5b80 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.737+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 >> 172.21.15.59:0/3737562158 conn(0x55b56ddb3000 msgr2=0x55b56dc67000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 shutdown_connections 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/3737562158 wait complete. 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 msgr2=0x55b56dcb4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55b56dd0d380 tx=0x55b56dc7c120 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 shutdown_connections 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b56cee1c00 0x55b56dcb4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b56cee1800 0x55b56dcb4580 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 --2- 172.21.15.59:0/2068987197 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b56cee0c00 0x55b56dcb4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 >> 172.21.15.59:0/2068987197 conn(0x55b56cee1000 msgr2=0x55b56dc66800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.738+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 shutdown_connections 2024-04-24T06:39:50.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.739+0000 7f9f187e0280 1 -- 172.21.15.59:0/2068987197 wait complete. 2024-04-24T06:39:50.745 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.137980, result: Events by type: Errors: 0 2024-04-24T06:39:50.745 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-24T06:39:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.833+0000 7fcf22533f40 1 Processor -- start 2024-04-24T06:39:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.833+0000 7fcf22533f40 1 -- start start 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689b2ac0 0x55e868924100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e8688f59c0 con 0x55e8689cdad0 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e86889ccc0 con 0x55e8688f31d0 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf22533f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e8688f3b20 con 0x55e8689b2ac0 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51286/0 (socket says 172.21.15.59:51286) 2024-04-24T06:39:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 -- 172.21.15.59:0/4131566646 learned_addr learned my addr 172.21.15.59:0/4131566646 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 -- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689b2ac0 msgr2=0x55e868924100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689b2ac0 0x55e868924100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 -- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 msgr2=0x55e8689ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.834+0000 7fcf20200640 1 -- 172.21.15.59:0/4131566646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8689c92c0 con 0x55e8689cdad0 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.835+0000 7fcf20200640 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800a3f0 tx=0x7fcf18051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5496662f6a4ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.835+0000 7fcf1da00640 1 -- 172.21.15.59:0/4131566646 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18056050 con 0x55e8689cdad0 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.835+0000 7fcf1da00640 1 -- 172.21.15.59:0/4131566646 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18053970 con 0x55e8689cdad0 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.835+0000 7fcf1da00640 1 -- 172.21.15.59:0/4131566646 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18053d20 con 0x55e8689cdad0 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 -- 172.21.15.59:0/4131566646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 msgr2=0x55e8689ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800a3f0 tx=0x7fcf18051000 comp rx=0 tx=0).stop 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 -- 172.21.15.59:0/4131566646 shutdown_connections 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689cdad0 0x55e8689ceb20 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689b2ac0 0x55e868924100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 --2- 172.21.15.59:0/4131566646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 -- 172.21.15.59:0/4131566646 >> 172.21.15.59:0/4131566646 conn(0x55e86893fcc0 msgr2=0x55e8689cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 -- 172.21.15.59:0/4131566646 shutdown_connections 2024-04-24T06:39:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 -- 172.21.15.59:0/4131566646 wait complete. 2024-04-24T06:39:50.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.836+0000 7fcf22533f40 1 Processor -- start 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.837+0000 7fcf22533f40 1 -- start start 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.837+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.837+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 0x55e8689ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.837+0000 7fcf22533f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689cdad0 0x55e8689c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf22533f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e8689c9530 con 0x55e8689b2ac0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf22533f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e868923c20 con 0x55e8688f31d0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf22533f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e8689cfba0 con 0x55e8689cdad0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 0x55e8689ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 0x55e8689ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51292/0 (socket says 172.21.15.59:51292) 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 -- 172.21.15.59:0/1452214805 learned_addr learned my addr 172.21.15.59:0/1452214805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf20200640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689cdad0 0x55e8689c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1f800640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 -- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689cdad0 msgr2=0x55e8689c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e8689cdad0 0x55e8689c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 -- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 msgr2=0x55e8689ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1ee00640 1 -- 172.21.15.59:0/1452214805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8689aed30 con 0x55e8689b2ac0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.838+0000 7fcf1f800640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e8688f31d0 0x55e8689ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf1ee00640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 0x55e8689ced10 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00ad10 tx=0x7fcf0c00ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf17e00640 1 -- 172.21.15.59:0/1452214805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01a070 con 0x55e8689b2ac0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf17e00640 1 -- 172.21.15.59:0/1452214805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c008aa0 con 0x55e8689b2ac0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf17e00640 1 -- 172.21.15.59:0/1452214805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c015420 con 0x55e8689b2ac0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e868925cc0 con 0x55e8689b2ac0 2024-04-24T06:39:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.839+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8689c6970 con 0x55e8689b2ac0 2024-04-24T06:39:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.841+0000 7fcf17e00640 1 -- 172.21.15.59:0/1452214805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf0c01d070 con 0x55e8689b2ac0 2024-04-24T06:39:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.842+0000 7fcf17e00640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf0006b300 0x7fcf0006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.842+0000 7fcf17e00640 1 -- 172.21.15.59:0/1452214805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fcf0c089b70 con 0x55e8689b2ac0 2024-04-24T06:39:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.842+0000 7fcf1f800640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf0006b300 0x7fcf0006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.843+0000 7fcf22533f40 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e8689cf250 0x55e868e6d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.843+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e8689c75a0 con 0x55e8689cf250 2024-04-24T06:39:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.843+0000 7fcf20200640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e8689cf250 0x55e868e6d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.843+0000 7fcf1f800640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf0006b300 0x7fcf0006d7c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcf10004170 tx=0x7fcf10001390 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.844+0000 7fcf20200640 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e8689cf250 0x55e868e6d830 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf20200640 1 -- 172.21.15.59:0/1452214805 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v99'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fcf18053c20 con 0x55e8689cf250 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e8689cf250 msgr2=0x55e868e6d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e8689cf250 0x55e868e6d830 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf0006b300 msgr2=0x7fcf0006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcf0006b300 0x7fcf0006d7c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcf10004170 tx=0x7fcf10001390 comp rx=0 tx=0).stop 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 msgr2=0x55e8689ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf22533f40 1 --2- 172.21.15.59:0/1452214805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e8689b2ac0 0x55e8689ced10 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00ad10 tx=0x7fcf0c00ae00 comp rx=0 tx=0).stop 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.849+0000 7fcf20200640 1 -- 172.21.15.59:0/1452214805 reap_dead start 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.850+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 shutdown_connections 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.850+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 >> 172.21.15.59:0/1452214805 conn(0x55e86893fcc0 msgr2=0x55e8689d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.850+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 shutdown_connections 2024-04-24T06:39:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:50.850+0000 7fcf22533f40 1 -- 172.21.15.59:0/1452214805 wait complete. 2024-04-24T06:39:50.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:39:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.062+0000 7f8abee00640 1 Processor -- start 2024-04-24T06:39:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.062+0000 7f8abee00640 1 -- start start 2024-04-24T06:39:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.063+0000 7f8abee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f8ab81567e0 0x7f8ab8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.063+0000 7f8abee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.063+0000 7f8abee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab815cab0 con 0x7f8ab80d0b40 2024-04-24T06:39:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.063+0000 7f8abee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab80d0f70 con 0x7f8ab81567e0 2024-04-24T06:39:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.063+0000 7f8abee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab8160e40 con 0x7f8ab815d810 2024-04-24T06:39:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.064+0000 7f8abd000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.064+0000 7f8abd000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:43416/0 (socket says 172.21.15.59:43416) 2024-04-24T06:39:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.064+0000 7f8abd000640 1 -- 172.21.15.59:0/895939638 learned_addr learned my addr 172.21.15.59:0/895939638 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.064+0000 7f8abda00640 1 --2- 172.21.15.59:0/895939638 >> v2:172.21.15.66:3300/0 conn(0x7f8ab81567e0 0x7f8ab8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.065+0000 7f8abd000640 1 -- 172.21.15.59:0/895939638 >> v2:172.21.15.66:3300/0 conn(0x7f8ab81567e0 msgr2=0x7f8ab8156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.065+0000 7f8abd000640 1 --2- 172.21.15.59:0/895939638 >> v2:172.21.15.66:3300/0 conn(0x7f8ab81567e0 0x7f8ab8156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.065+0000 7f8abd000640 1 -- 172.21.15.59:0/895939638 >> v1:172.21.15.66:6789/0 conn(0x7f8ab80d0b40 legacy=0x7f8ab8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.065+0000 7f8abd000640 1 -- 172.21.15.59:0/895939638 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab815cd30 con 0x7f8ab815d810 2024-04-24T06:39:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.066+0000 7f8abd000640 1 --2- 172.21.15.59:0/895939638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009870 tx=0x7f8aa402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef9debaf134c68a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.067+0000 7f8ab7e00640 1 -- 172.21.15.59:0/895939638 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4037070 con 0x7f8ab815d810 2024-04-24T06:39:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.067+0000 7f8ab7e00640 1 -- 172.21.15.59:0/895939638 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4032440 con 0x7f8ab815d810 2024-04-24T06:39:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.067+0000 7f8ab7e00640 1 -- 172.21.15.59:0/895939638 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4032810 con 0x7f8ab815d810 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 -- 172.21.15.59:0/895939638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 msgr2=0x7f8ab80d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 --2- 172.21.15.59:0/895939638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009870 tx=0x7f8aa402a250 comp rx=0 tx=0).stop 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 -- 172.21.15.59:0/895939638 shutdown_connections 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 --2- 172.21.15.59:0/895939638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab815d810 0x7f8ab80d0600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 --2- 172.21.15.59:0/895939638 >> v2:172.21.15.66:3300/0 conn(0x7f8ab81567e0 0x7f8ab8156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 -- 172.21.15.59:0/895939638 >> 172.21.15.59:0/895939638 conn(0x7f8ab806bdf0 msgr2=0x7f8ab806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 -- 172.21.15.59:0/895939638 shutdown_connections 2024-04-24T06:39:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.068+0000 7f8abee00640 1 -- 172.21.15.59:0/895939638 wait complete. 2024-04-24T06:39:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.070+0000 7f8abee00640 1 Processor -- start 2024-04-24T06:39:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.070+0000 7f8abee00640 1 -- start start 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 0x7f8ab808eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab81567e0 0x7f8ab81e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8160820 con 0x7f8ab815d810 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab815d2f0 con 0x7f8ab81567e0 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab81609a0 con 0x7f8ab80d0b40 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abe400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abe400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51300/0 (socket says 172.21.15.59:51300) 2024-04-24T06:39:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.071+0000 7f8abe400640 1 -- 172.21.15.59:0/129417739 learned_addr learned my addr 172.21.15.59:0/129417739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abda00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 0x7f8ab808eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abd000640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab81567e0 0x7f8ab81e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 -- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 msgr2=0x7f8ab808eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 0x7f8ab808eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 -- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab81567e0 msgr2=0x7f8ab81e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab81567e0 0x7f8ab81e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 -- 172.21.15.59:0/129417739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab808f070 con 0x7f8ab815d810 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abda00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 0x7f8ab808eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.072+0000 7f8abe400640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c6b0 tx=0x7f8ab000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.073+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0003b70 con 0x7f8ab815d810 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.073+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab0003d30 con 0x7f8ab815d810 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.073+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0005310 con 0x7f8ab815d810 2024-04-24T06:39:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.073+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab809dca0 con 0x7f8ab815d810 2024-04-24T06:39:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.073+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab806ed60 con 0x7f8ab815d810 2024-04-24T06:39:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.074+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab804fd90 con 0x7f8ab815d810 2024-04-24T06:39:51.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.075+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ab00054b0 con 0x7f8ab815d810 2024-04-24T06:39:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.076+0000 7f8ab5600640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 0x7f8a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.076+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f8ab008bb70 con 0x7f8ab815d810 2024-04-24T06:39:51.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.076+0000 7f8abda00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 0x7f8a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.077+0000 7f8abda00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 0x7f8a9806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009b90 tx=0x7f8aac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.081+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ab815cab0 con 0x7f8ab815d810 2024-04-24T06:39:51.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.342+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8ab815ddf0 con 0x7f8ab815d810 2024-04-24T06:39:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.454+0000 7f8ab5600640 1 -- 172.21.15.59:0/129417739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) ==== 162+0+0 (secure 0 0 0) 0x7f8ab0059e10 con 0x7f8ab815d810 2024-04-24T06:39:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 msgr2=0x7f8a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 0x7f8a9806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009b90 tx=0x7f8aac008040 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 msgr2=0x7f8ab81e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c6b0 tx=0x7f8ab000cb70 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 shutdown_connections 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a9806c8f0 0x7f8a9806edb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab815d810 0x7f8ab81e7150 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab81567e0 0x7f8ab81e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 --2- 172.21.15.59:0/129417739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8ab80d0b40 0x7f8ab808eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 >> 172.21.15.59:0/129417739 conn(0x7f8ab806bdf0 msgr2=0x7f8ab80af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 shutdown_connections 2024-04-24T06:39:51.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.457+0000 7f8abee00640 1 -- 172.21.15.59:0/129417739 wait complete. 2024-04-24T06:39:51.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:51.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.678+0000 7f176a800640 1 Processor -- start 2024-04-24T06:39:51.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.679+0000 7f176a800640 1 -- start start 2024-04-24T06:39:51.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.680+0000 7f176a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17640ce250 0x7f17640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.680+0000 7f176a800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17640cbd00 con 0x7f176415ecc0 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.680+0000 7f176a800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17640cbfe0 con 0x7f17640d0810 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.680+0000 7f176a800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f17640cc2c0 con 0x7f17640ce250 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17640ce250 0x7f17640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17640ce250 0x7f17640ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51316/0 (socket says 172.21.15.59:51316) 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 -- 172.21.15.59:0/339250099 learned_addr learned my addr 172.21.15.59:0/339250099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1762a00640 1 --1- 172.21.15.59:0/339250099 >> v1:172.21.15.59:6789/0 conn(0x7f176415ecc0 0x7f17640ae2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34336/0 (socket says 172.21.15.59:34336) 2024-04-24T06:39:51.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 -- 172.21.15.59:0/339250099 >> v1:172.21.15.66:6789/0 conn(0x7f17640d0810 legacy=0x7f17640d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 -- 172.21.15.59:0/339250099 >> v1:172.21.15.59:6789/0 conn(0x7f176415ecc0 legacy=0x7f17640ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1763e00640 1 -- 172.21.15.59:0/339250099 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17640cc5a0 con 0x7f17640ce250 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.681+0000 7f1762000640 1 -- 172.21.15.59:0/339250099 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122187674 0 0) 0x7f17640cbd00 con 0x7f176415ecc0 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.682+0000 7f1763e00640 1 --2- 172.21.15.59:0/339250099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17640ce700 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec72214dc00c0d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f1762000640 1 -- 172.21.15.59:0/339250099 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c037070 con 0x7f17640ce250 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f1762000640 1 -- 172.21.15.59:0/339250099 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175c032440 con 0x7f17640ce250 2024-04-24T06:39:51.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f1762000640 1 -- 172.21.15.59:0/339250099 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c032810 con 0x7f17640ce250 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 -- 172.21.15.59:0/339250099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 msgr2=0x7f17640ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 --2- 172.21.15.59:0/339250099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17640ce700 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c02a2a0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 -- 172.21.15.59:0/339250099 shutdown_connections 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 --2- 172.21.15.59:0/339250099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17640ce700 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 -- 172.21.15.59:0/339250099 >> 172.21.15.59:0/339250099 conn(0x7f1764073430 msgr2=0x7f176409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:51.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 -- 172.21.15.59:0/339250099 shutdown_connections 2024-04-24T06:39:51.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.683+0000 7f176a800640 1 -- 172.21.15.59:0/339250099 wait complete. 2024-04-24T06:39:51.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.685+0000 7f176a800640 1 Processor -- start 2024-04-24T06:39:51.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.685+0000 7f176a800640 1 -- start start 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17640d0810 0x7f17640b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f176415ecc0 0x7f17640b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17640ccc40 con 0x7f17640ce250 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f17640ccac0 con 0x7f17640d0810 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f176a800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f17640ccdc0 con 0x7f176415ecc0 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.686+0000 7f1763400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51330/0 (socket says 172.21.15.59:51330) 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 -- 172.21.15.59:0/1248902004 learned_addr learned my addr 172.21.15.59:0/1248902004 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1762a00640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17640d0810 0x7f17640b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 -- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f176415ecc0 msgr2=0x7f17640b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f176415ecc0 0x7f17640b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 -- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17640d0810 msgr2=0x7f17640b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17640d0810 0x7f17640b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 -- 172.21.15.59:0/1248902004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17640c51b0 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.687+0000 7f1763400640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f175400c7c0 tx=0x7f175400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.688+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754016020 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.688+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175400aa50 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.688+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17640c5490 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.688+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754012420 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.688+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17640c5a50 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.689+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1764068a80 con 0x7f17640ce250 2024-04-24T06:39:51.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.690+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1754019020 con 0x7f17640ce250 2024-04-24T06:39:51.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.691+0000 7f175b400640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 0x7f174006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:51.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.691+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f175408b970 con 0x7f17640ce250 2024-04-24T06:39:51.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.691+0000 7f1762a00640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 0x7f174006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:51.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.692+0000 7f1762a00640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 0x7f174006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f17641f7a00 tx=0x7f174c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:51.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.695+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1754020070 con 0x7f17640ce250 2024-04-24T06:39:51.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.943+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17640c5c20 con 0x7f17640ce250 2024-04-24T06:39:51.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.944+0000 7f175b400640 1 -- 172.21.15.59:0/1248902004 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5174 (secure 0 0 0) 0x7f1754059d30 con 0x7f17640ce250 2024-04-24T06:39:51.946 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:51.946 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:39:51.460620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8490":{"gid":8490,"name":"a","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:39:51.947 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 86 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 msgr2=0x7f174006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 0x7f174006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f17641f7a00 tx=0x7f174c009040 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 msgr2=0x7f17641552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f175400c7c0 tx=0x7f175400cc80 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 shutdown_connections 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f174006c8f0 0x7f174006edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f176415ecc0 0x7f17640b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17640d0810 0x7f17640b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 --2- 172.21.15.59:0/1248902004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17640ce250 0x7f17641552c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 >> 172.21.15.59:0/1248902004 conn(0x7f1764073430 msgr2=0x7f17640d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 shutdown_connections 2024-04-24T06:39:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:51.947+0000 7f176a800640 1 -- 172.21.15.59:0/1248902004 wait complete. 2024-04-24T06:39:52.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:38:38.647945+0000', 'modified': '2024-04-24T06:39:51.460620+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8490': {'gid': 8490, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/1384442467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 1384442467}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 1384442467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:39:52.369 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8490:up:replay 2024-04-24T06:39:53.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:53.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.592+0000 7f9247200640 1 Processor -- start 2024-04-24T06:39:53.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.592+0000 7f9247200640 1 -- start start 2024-04-24T06:39:53.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.593+0000 7f9247200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 0x7f924809b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.593+0000 7f9247200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92480b2e20 0x7f92480af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9247200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92480b35f0 con 0x7f92480d09e0 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9247200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f92480cd2d0 con 0x7f92480b2e20 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9247200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92480b3890 con 0x7f924815c9d0 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9246800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f92480d09e0 0x7f92480d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34350/0 (socket says 172.21.15.59:34350) 2024-04-24T06:39:53.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9246800640 1 -- 172.21.15.59:0/4091152174 learned_addr learned my addr 172.21.15.59:0/4091152174 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:53.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9245400640 1 --2- 172.21.15.59:0/4091152174 >> v2:172.21.15.66:3300/0 conn(0x7f92480b2e20 0x7f92480af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.594+0000 7f9245e00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 0x7f924809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309634803 0 0) 0x7f92480b35f0 con 0x7f92480d09e0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f922c003520 con 0x7f92480d09e0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3037498846 0 0) 0x7f922c003520 con 0x7f92480d09e0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 msgr2=0x7f924809b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 0x7f924809b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 >> v2:172.21.15.66:3300/0 conn(0x7f92480b2e20 msgr2=0x7f92480af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 --2- 172.21.15.59:0/4091152174 >> v2:172.21.15.66:3300/0 conn(0x7f92480b2e20 0x7f92480af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480b3b70 con 0x7f92480d09e0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f9240001690 con 0x7f92480d09e0 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 >> v1:172.21.15.59:6789/0 conn(0x7f92480d09e0 legacy=0x7f92480d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c0039b0 0x7f922c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f922c004800 0x7f922c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 0x7f922c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92480cd2d0 con 0x7f922c004800 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92480b3890 con 0x7f922c0039b0 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f922c00b9c0 con 0x7f924815c9d0 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.595+0000 7f9245400640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f922c004800 0x7f922c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245e00640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c0039b0 0x7f922c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245400640 1 -- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 msgr2=0x7f922c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245400640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924815c9d0 0x7f922c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245400640 1 -- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c0039b0 msgr2=0x7f922c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245400640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c0039b0 0x7f922c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.596+0000 7f9245400640 1 -- 172.21.15.59:0/4091152174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c003520 con 0x7f922c004800 2024-04-24T06:39:53.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.597+0000 7f9245400640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f922c004800 0x7f922c004c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9234009ac0 tx=0x7f922c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b225f22eeac9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:53.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.597+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234036070 con 0x7f922c004800 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.597+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234031400 con 0x7f922c004800 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.597+0000 7f9244a00640 1 -- 172.21.15.59:0/4091152174 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234009690 con 0x7f922c004800 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 -- 172.21.15.59:0/4091152174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f922c004800 msgr2=0x7f922c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 --2- 172.21.15.59:0/4091152174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f922c004800 0x7f922c004c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f9234009ac0 tx=0x7f922c005830 comp rx=0 tx=0).stop 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9246800640 1 -- 172.21.15.59:0/4091152174 reap_dead start 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 -- 172.21.15.59:0/4091152174 shutdown_connections 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 -- 172.21.15.59:0/4091152174 >> 172.21.15.59:0/4091152174 conn(0x7f92480b3ef0 msgr2=0x7f92480b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 -- 172.21.15.59:0/4091152174 shutdown_connections 2024-04-24T06:39:53.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.598+0000 7f9247200640 1 -- 172.21.15.59:0/4091152174 wait complete. 2024-04-24T06:39:53.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.600+0000 7f9247200640 1 Processor -- start 2024-04-24T06:39:53.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.600+0000 7f9247200640 1 -- start start 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c004800 0x7f924814a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924806e100 0x7f92480c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92480b3280 con 0x7f9248149910 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92480ccef0 con 0x7f922c004800 2024-04-24T06:39:53.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9247200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f924815cdb0 con 0x7f924806e100 2024-04-24T06:39:53.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9245400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9245400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51352/0 (socket says 172.21.15.59:51352) 2024-04-24T06:39:53.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.601+0000 7f9245400640 1 -- 172.21.15.59:0/2732720393 learned_addr learned my addr 172.21.15.59:0/2732720393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 -- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924806e100 msgr2=0x7f92480c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924806e100 0x7f92480c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 -- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c004800 msgr2=0x7f924814a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c004800 0x7f924814a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 -- 172.21.15.59:0/2732720393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924815dba0 con 0x7f9248149910 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9245400640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9234009be0 tx=0x7f9234004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234040020 con 0x7f9248149910 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234003950 con 0x7f9248149910 2024-04-24T06:39:53.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924815de80 con 0x7f9248149910 2024-04-24T06:39:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.602+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234036040 con 0x7f9248149910 2024-04-24T06:39:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.603+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924815e440 con 0x7f9248149910 2024-04-24T06:39:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.604+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924804fd90 con 0x7f9248149910 2024-04-24T06:39:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.605+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9234043020 con 0x7f9248149910 2024-04-24T06:39:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.606+0000 7f923e000640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.606+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f92340abe50 con 0x7f9248149910 2024-04-24T06:39:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.606+0000 7f9245e00640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.607+0000 7f9245e00640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9238009fc0 tx=0x7f9238009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:53.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.610+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f923404a070 con 0x7f9248149910 2024-04-24T06:39:53.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.856+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92480d07c0 con 0x7f9248149910 2024-04-24T06:39:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.857+0000 7f923e000640 1 -- 172.21.15.59:0/2732720393 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5175 (secure 0 0 0) 0x7f923407a180 con 0x7f9248149910 2024-04-24T06:39:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:53.859 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:39:52.473527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8490":{"gid":8490,"name":"a","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:39:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 88 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 msgr2=0x7f922806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9238009fc0 tx=0x7f9238009750 comp rx=0 tx=0).stop 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 msgr2=0x7f924806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9234009be0 tx=0x7f9234004730 comp rx=0 tx=0).stop 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 shutdown_connections 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f924806e100 0x7f92480c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9248149910 0x7f924806dbc0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 --2- 172.21.15.59:0/2732720393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f922c004800 0x7f924814a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 >> 172.21.15.59:0/2732720393 conn(0x7f92480b3ef0 msgr2=0x7f92480af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 shutdown_connections 2024-04-24T06:39:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:53.860+0000 7f9247200640 1 -- 172.21.15.59:0/2732720393 wait complete. 2024-04-24T06:39:54.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:38:38.647945+0000', 'modified': '2024-04-24T06:39:52.473527+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8490': {'gid': 8490, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 22, 'addr': '172.21.15.59:6837/1384442467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 1384442467}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 1384442467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:39:54.286 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8490:up:rejoin 2024-04-24T06:39:55.125 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:39:55.123+0000 7f452e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:39:55.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:39:55.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.508+0000 7fe508200640 1 Processor -- start 2024-04-24T06:39:55.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.509+0000 7fe508200640 1 -- start start 2024-04-24T06:39:55.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.510+0000 7fe508200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5001454e0 0x7fe500145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.510+0000 7fe508200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe500160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.510+0000 7fe508200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5000cd520 con 0x7fe5000cd020 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.510+0000 7fe508200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe5000cddc0 con 0x7fe5001454e0 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.510+0000 7fe508200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe5000cd800 con 0x7fe500160910 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe505800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5001454e0 0x7fe500145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe504e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe5000cd020 0x7fe5000b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34360/0 (socket says 172.21.15.59:34360) 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe504e00640 1 -- 172.21.15.59:0/1328642305 learned_addr learned my addr 172.21.15.59:0/1328642305 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:39:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe505800640 1 -- 172.21.15.59:0/1328642305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 msgr2=0x7fe500160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe505800640 1 --2- 172.21.15.59:0/1328642305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe500160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe505800640 1 -- 172.21.15.59:0/1328642305 >> v1:172.21.15.59:6789/0 conn(0x7fe5000cd020 legacy=0x7fe5000b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe505800640 1 -- 172.21.15.59:0/1328642305 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5000cdae0 con 0x7fe5001454e0 2024-04-24T06:39:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.511+0000 7fe4ffe00640 1 -- 172.21.15.59:0/1328642305 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796128677 0 0) 0x7fe5000cd520 con 0x7fe5000cd020 2024-04-24T06:39:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.512+0000 7fe505800640 1 --2- 172.21.15.59:0/1328642305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5001454e0 0x7fe500145bb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40097a0 tx=0x7fe4f402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae50284de7ce508a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:55.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.513+0000 7fe4ffe00640 1 -- 172.21.15.59:0/1328642305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4037070 con 0x7fe5001454e0 2024-04-24T06:39:55.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.513+0000 7fe4ffe00640 1 -- 172.21.15.59:0/1328642305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f402ae50 con 0x7fe5001454e0 2024-04-24T06:39:55.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.513+0000 7fe4ffe00640 1 -- 172.21.15.59:0/1328642305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4032660 con 0x7fe5001454e0 2024-04-24T06:39:55.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.513+0000 7fe508200640 1 -- 172.21.15.59:0/1328642305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5001454e0 msgr2=0x7fe500145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.513+0000 7fe508200640 1 --2- 172.21.15.59:0/1328642305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5001454e0 0x7fe500145bb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40097a0 tx=0x7fe4f402a100 comp rx=0 tx=0).stop 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 -- 172.21.15.59:0/1328642305 shutdown_connections 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 --2- 172.21.15.59:0/1328642305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe500160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 --2- 172.21.15.59:0/1328642305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5001454e0 0x7fe500145bb0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 -- 172.21.15.59:0/1328642305 >> 172.21.15.59:0/1328642305 conn(0x7fe500067ea0 msgr2=0x7fe50006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 -- 172.21.15.59:0/1328642305 shutdown_connections 2024-04-24T06:39:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.514+0000 7fe508200640 1 -- 172.21.15.59:0/1328642305 wait complete. 2024-04-24T06:39:55.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.515+0000 7fe508200640 1 Processor -- start 2024-04-24T06:39:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.516+0000 7fe508200640 1 -- start start 2024-04-24T06:39:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 0x7fe50015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe50013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5000b4010 con 0x7fe5000cd020 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001460f0 con 0x7fe5001454e0 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe508200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe50013d250 con 0x7fe500160910 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe505800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe505800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51376/0 (socket says 172.21.15.59:51376) 2024-04-24T06:39:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe505800640 1 -- 172.21.15.59:0/276182329 learned_addr learned my addr 172.21.15.59:0/276182329 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe504e00640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 0x7fe50015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe505800640 1 -- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 msgr2=0x7fe50013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.517+0000 7fe506200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe50013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe505800640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe50013f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe505800640 1 -- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 msgr2=0x7fe50015d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe505800640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 0x7fe50015d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe505800640 1 -- 172.21.15.59:0/276182329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50013d4f0 con 0x7fe5000cd020 2024-04-24T06:39:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe504e00640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 0x7fe50015d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe505800640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000c00 tx=0x7fe4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4042070 con 0x7fe5000cd020 2024-04-24T06:39:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f403e670 con 0x7fe5000cd020 2024-04-24T06:39:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4037040 con 0x7fe5000cd020 2024-04-24T06:39:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.518+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50013d7d0 con 0x7fe5000cd020 2024-04-24T06:39:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.519+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50013dd90 con 0x7fe5000cd020 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.520+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe500068a80 con 0x7fe5000cd020 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.521+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4f4045070 con 0x7fe5000cd020 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.521+0000 7fe4fd600640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.522+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fe4f40ac410 con 0x7fe5000cd020 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.522+0000 7fe504e00640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:55.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.522+0000 7fe504e00640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5000694d0 tx=0x7fe4ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:55.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.526+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4f407a860 con 0x7fe5000cd020 2024-04-24T06:39:55.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.766+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe500160030 con 0x7fe5000cd020 2024-04-24T06:39:55.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.767+0000 7fe4fd600640 1 -- 172.21.15.59:0/276182329 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5182 (secure 0 0 0) 0x7fe4f407a160 con 0x7fe5000cd020 2024-04-24T06:39:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:55.769 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:39:54.472200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8490":{"gid":8490,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":23,"addr":"172.21.15.59:6837/1384442467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1384442467},{"type":"v1","addr":"172.21.15.59:6837","nonce":1384442467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8490,"qdb_cluster":[8490]},"id":7}]} 2024-04-24T06:39:55.769 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 89 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 msgr2=0x7fe4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5000694d0 tx=0x7fe4ec009040 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 msgr2=0x7fe50015beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000c00 tx=0x7fe4f40046c0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 shutdown_connections 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe500160910 0x7fe50013f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5001454e0 0x7fe50015d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 --2- 172.21.15.59:0/276182329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000cd020 0x7fe50015beb0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 >> 172.21.15.59:0/276182329 conn(0x7fe500067ea0 msgr2=0x7fe500140d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 shutdown_connections 2024-04-24T06:39:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:55.770+0000 7fe508200640 1 -- 172.21.15.59:0/276182329 wait complete. 2024-04-24T06:39:56.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:38:38.647945+0000', 'modified': '2024-04-24T06:39:54.472200+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8490': {'gid': 8490, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.59:6837/1384442467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 1384442467}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 1384442467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8490, 'qdb_cluster': [8490]} 2024-04-24T06:39:56.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:39:56.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T06:39:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.416+0000 7f9537200640 1 Processor -- start 2024-04-24T06:39:56.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.416+0000 7f9537200640 1 -- start start 2024-04-24T06:39:56.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9537200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95380d0890 0x7f95380b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9537200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f953815eb40 0x7f95380b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9537200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953815ff50 con 0x7f953815e760 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9537200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9538162610 con 0x7f953815eb40 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9537200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f95381628f0 con 0x7f95380d0890 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9536800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f953815eb40 0x7f95380b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9536800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f953815eb40 0x7f95380b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51388/0 (socket says 172.21.15.59:51388) 2024-04-24T06:39:56.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f9536800640 1 -- 172.21.15.59:0/1108465812 learned_addr learned my addr 172.21.15.59:0/1108465812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.418+0000 7f952fe00640 1 --2- 172.21.15.59:0/1108465812 >> v2:172.21.15.66:3300/0 conn(0x7f95380d0890 0x7f95380b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.419+0000 7f9536800640 1 -- 172.21.15.59:0/1108465812 >> v2:172.21.15.66:3300/0 conn(0x7f95380d0890 msgr2=0x7f95380b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.419+0000 7f9536800640 1 --2- 172.21.15.59:0/1108465812 >> v2:172.21.15.66:3300/0 conn(0x7f95380d0890 0x7f95380b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.419+0000 7f9536800640 1 -- 172.21.15.59:0/1108465812 >> v1:172.21.15.66:6789/0 conn(0x7f953815e760 legacy=0x7f95380d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.419+0000 7f9536800640 1 -- 172.21.15.59:0/1108465812 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9538162bd0 con 0x7f953815eb40 2024-04-24T06:39:56.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.420+0000 7f9536800640 1 --2- 172.21.15.59:0/1108465812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815eb40 0x7f95380b44e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f95380b4e40 tx=0x7f953002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04453a4ca91133a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9535400640 1 -- 172.21.15.59:0/1108465812 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530036070 con 0x7f953815eb40 2024-04-24T06:39:56.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9535400640 1 -- 172.21.15.59:0/1108465812 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530031600 con 0x7f953815eb40 2024-04-24T06:39:56.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9535400640 1 -- 172.21.15.59:0/1108465812 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300319d0 con 0x7f953815eb40 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 -- 172.21.15.59:0/1108465812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815eb40 msgr2=0x7f95380b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 --2- 172.21.15.59:0/1108465812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815eb40 0x7f95380b44e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f95380b4e40 tx=0x7f953002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 -- 172.21.15.59:0/1108465812 shutdown_connections 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 --2- 172.21.15.59:0/1108465812 >> v2:172.21.15.66:3300/0 conn(0x7f95380d0890 0x7f95380b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 --2- 172.21.15.59:0/1108465812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815eb40 0x7f95380b44e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.421+0000 7f9537200640 1 -- 172.21.15.59:0/1108465812 >> 172.21.15.59:0/1108465812 conn(0x7f9538067760 msgr2=0x7f9538067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.422+0000 7f9537200640 1 -- 172.21.15.59:0/1108465812 shutdown_connections 2024-04-24T06:39:56.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.422+0000 7f9537200640 1 -- 172.21.15.59:0/1108465812 wait complete. 2024-04-24T06:39:56.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.423+0000 7f9537200640 1 Processor -- start 2024-04-24T06:39:56.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.424+0000 7f9537200640 1 -- start start 2024-04-24T06:39:56.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.424+0000 7f9537200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.424+0000 7f9537200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f953815eb40 0x7f9538157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.424+0000 7f9537200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f953806fd50 0x7f95380701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9537200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95380d0d20 con 0x7f953815e760 2024-04-24T06:39:56.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9537200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95381633d0 con 0x7f953806fd50 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9537200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95380d0ea0 con 0x7f953815eb40 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51392/0 (socket says 172.21.15.59:51392) 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 -- 172.21.15.59:0/944816836 learned_addr learned my addr 172.21.15.59:0/944816836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f952fe00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f953815eb40 0x7f9538157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9536800640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f953806fd50 0x7f95380701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 -- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f953815eb40 msgr2=0x7f9538157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f953815eb40 0x7f9538157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 -- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f953806fd50 msgr2=0x7f95380701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f953806fd50 0x7f95380701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.425+0000 7f9535e00640 1 -- 172.21.15.59:0/944816836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95381f5100 con 0x7f953815e760 2024-04-24T06:39:56.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f9535e00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f952800c430 tx=0x7f952800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f952e000640 1 -- 172.21.15.59:0/944816836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011790 con 0x7f953815e760 2024-04-24T06:39:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f952e000640 1 -- 172.21.15.59:0/944816836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9528017070 con 0x7f953815e760 2024-04-24T06:39:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f952e000640 1 -- 172.21.15.59:0/944816836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011b70 con 0x7f953815e760 2024-04-24T06:39:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95381f7b90 con 0x7f953815e760 2024-04-24T06:39:56.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.426+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95380b4800 con 0x7f953815e760 2024-04-24T06:39:56.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.428+0000 7f952e000640 1 -- 172.21.15.59:0/944816836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9528011d10 con 0x7f953815e760 2024-04-24T06:39:56.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.430+0000 7f952e000640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 0x7f951406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.430+0000 7f952e000640 1 -- 172.21.15.59:0/944816836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f952808ba80 con 0x7f953815e760 2024-04-24T06:39:56.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.430+0000 7f952fe00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 0x7f951406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.430+0000 7f954001e740 1 Processor -- start 2024-04-24T06:39:56.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.430+0000 7f954001e740 1 -- start start 2024-04-24T06:39:56.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f952fe00640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 0x7f951406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f95381617c0 tx=0x7f9520008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeea71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cdee9dbb10 con 0x55cdeeaa1360 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cdee9822b0 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.431+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cdee9d7ad0 con 0x55cdeeabe980 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51404/0 (socket says 172.21.15.59:51404) 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 -- 172.21.15.59:0/3717520100 learned_addr learned my addr 172.21.15.59:0/3717520100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9535e00640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeea71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f952fe00640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 -- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 msgr2=0x55cdeea71310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeea71310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 -- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 msgr2=0x55cdeeaa0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 -- 172.21.15.59:0/3717520100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdee6ee2d0 con 0x55cdeeaa1360 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9535e00640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeea71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.432+0000 7f9536800640 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f9530009f10 tx=0x7f953002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b7159f02ff5215 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.433+0000 7f9526a00640 1 -- 172.21.15.59:0/3717520100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530036040 con 0x55cdeeaa1360 2024-04-24T06:39:56.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.433+0000 7f9526a00640 1 -- 172.21.15.59:0/3717520100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530042070 con 0x55cdeeaa1360 2024-04-24T06:39:56.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.433+0000 7f9526a00640 1 -- 172.21.15.59:0/3717520100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003c8d0 con 0x55cdeeaa1360 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.433+0000 7f954001e740 1 -- 172.21.15.59:0/3717520100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 msgr2=0x55cdeea76170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.433+0000 7f954001e740 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f9530009f10 tx=0x7f953002e000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 -- 172.21.15.59:0/3717520100 shutdown_connections 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1360 0x55cdeea76170 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 --2- 172.21.15.59:0/3717520100 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeea71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 -- 172.21.15.59:0/3717520100 >> 172.21.15.59:0/3717520100 conn(0x55cdeea73580 msgr2=0x55cdeeabbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 -- 172.21.15.59:0/3717520100 shutdown_connections 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 -- 172.21.15.59:0/3717520100 wait complete. 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.434+0000 7f954001e740 1 -- start start 2024-04-24T06:39:56.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.435+0000 7f954001e740 1 Processor -- start 2024-04-24T06:39:56.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeabb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cdeea75730 con 0x55cdeea91a00 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cdee995590 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cdeea74b20 con 0x55cdeeabe980 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f954001e740 10 client.0 _ll_register_callbacks cb 0x55cdee9783a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51420/0 (socket says 172.21.15.59:51420) 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 -- 172.21.15.59:0/3371854521 learned_addr learned my addr 172.21.15.59:0/3371854521 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f952fe00640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeabb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 -- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 msgr2=0x55cdeeabb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeabb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 -- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 msgr2=0x55cdeeaa48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.436+0000 7f9536800640 1 -- 172.21.15.59:0/3371854521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdee7b86b0 con 0x55cdeea91a00 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f9536800640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9530007280 tx=0x7f9530004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da0c331b490a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f951b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953004a070 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f9530004450 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9530042070 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9530056050 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f954001e740 10 client.8979 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55cdeeacfa50 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.437+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530042070 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f9530059020 con 0x55cdeea91a00 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 10 client.8979 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 10 client.8979 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 20 client.8979 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeab9ea0 0x55cdeeaa2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 4 client.8979 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:56.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdee9e9530 con 0x55cdeeab9ea0 2024-04-24T06:39:56.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f9535e00640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeab9ea0 0x55cdeeaa2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f9535e00640 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeab9ea0 0x55cdeeaa2610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a8b7dbb8de35a8 server_cookie=e51281202f70705b in_seq=0 out_seq=0 2024-04-24T06:39:56.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.438+0000 7f951b400640 10 client.8979 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.439+0000 7f951b400640 1 -- 172.21.15.59:0/3371854521 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdee9e9530 con 0x55cdeeab9ea0 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.439+0000 7f951b400640 10 client.8979 handle_command_reply: tid=0 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.439+0000 7f954001e740 1 client.8979 shutdown 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 msgr2=0x55cdeeabbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9530007280 tx=0x7f9530004ab0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 shutdown_connections 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeab9ea0 0x55cdeeaa2610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeea91a00 0x55cdeeabbe20 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeabb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 --2- 172.21.15.59:0/3371854521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeaa48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f951b400640 0 client.8979 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 >> 172.21.15.59:0/3371854521 conn(0x55cdeeaa1360 msgr2=0x55cdeea9dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 shutdown_connections 2024-04-24T06:39:56.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 1 -- 172.21.15.59:0/3371854521 wait complete. 2024-04-24T06:39:56.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.440+0000 7f954001e740 20 client.8979 trim_cache size 0 max 16384 2024-04-24T06:39:56.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.460+0000 7f954001e740 1 Processor -- start 2024-04-24T06:39:56.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.460+0000 7f954001e740 1 -- start start 2024-04-24T06:39:56.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeea76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeab93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95381f63d0 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95381f5730 con 0x55cdeea91a00 2024-04-24T06:39:56.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95381f5d80 con 0x55cdeeabe980 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f952fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f952fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51434/0 (socket says 172.21.15.59:51434) 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f952fe00640 1 -- 172.21.15.59:0/2681054989 learned_addr learned my addr 172.21.15.59:0/2681054989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f9536800640 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeab93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.461+0000 7f9535e00640 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeea76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 -- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 msgr2=0x55cdeeab93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeab93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 -- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 msgr2=0x55cdeea76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeea76e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 -- 172.21.15.59:0/2681054989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdeeacb4d0 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f952fe00640 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f95380d2100 tx=0x7f9520015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a1f6d29620606c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f9526000640 1 -- 172.21.15.59:0/2681054989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520014be0 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f9526000640 1 -- 172.21.15.59:0/2681054989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520027070 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.462+0000 7f9526000640 1 -- 172.21.15.59:0/2681054989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520022480 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 -- 172.21.15.59:0/2681054989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 msgr2=0x55cdeeab8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f95380d2100 tx=0x7f9520015000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 -- 172.21.15.59:0/2681054989 shutdown_connections 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeabe980 0x55cdeeab93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeaa1aa0 0x55cdeeab8e90 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 --2- 172.21.15.59:0/2681054989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeea76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 -- 172.21.15.59:0/2681054989 >> 172.21.15.59:0/2681054989 conn(0x55cdeeaa1360 msgr2=0x55cdeea6c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 -- 172.21.15.59:0/2681054989 shutdown_connections 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.463+0000 7f954001e740 1 -- 172.21.15.59:0/2681054989 wait complete. 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.464+0000 7f954001e740 1 -- start start 2024-04-24T06:39:56.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.464+0000 7f954001e740 1 Processor -- start 2024-04-24T06:39:56.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeeaa46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeaa1aa0 0x55cdeeaa4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cdeeaa3060 con 0x55cdeeabe980 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cdeea75730 con 0x55cdeea91a00 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55cdeeaba2a0 con 0x55cdeeaa1aa0 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f954001e740 10 client.0 _ll_register_callbacks cb 0x7f9530003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51448/0 (socket says 172.21.15.59:51448) 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 -- 172.21.15.59:0/541840911 learned_addr learned my addr 172.21.15.59:0/541840911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 -- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeaa1aa0 msgr2=0x55cdeeaa4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeaa1aa0 0x55cdeeaa4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.465+0000 7f9536800640 1 -- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 msgr2=0x55cdeeaa46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f9536800640 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeeaa46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f9536800640 1 -- 172.21.15.59:0/541840911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdeea753c0 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f9536800640 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f9530002230 tx=0x7f9530009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ac5470810859f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f951b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953004a070 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f953005bb00 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9520027070 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9530036050 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f954001e740 10 client.8985 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.466+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55cdeeaa4270 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520027070 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f9530059070 con 0x55cdeeabe980 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 10 client.8985 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 10 client.8985 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 20 client.8985 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeb36090 0x55cdeeb36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 4 client.8985 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "status"}) -- 0x55cdee9e33a0 con 0x55cdeeb36090 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f9535e00640 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeb36090 0x55cdeeb36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f9535e00640 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeb36090 0x55cdeeb36470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ee20f7dbb57d65 server_cookie=2a59c16fabeea778 in_seq=0 out_seq=0 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.467+0000 7f951b400640 10 client.8985 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f951b400640 1 -- 172.21.15.59:0/541840911 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55cdee9e33a0 con 0x55cdeeb36090 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f951b400640 10 client.8985 handle_command_reply: tid=0 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 client.8985 shutdown 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 msgr2=0x55cdeea9d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f9530002230 tx=0x7f9530009000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 shutdown_connections 2024-04-24T06:39:56.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55cdeeb36090 0x55cdeeb36470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cdeeabe980 0x55cdeea9d430 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55cdeeaa1aa0 0x55cdeeaa4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f951b400640 0 client.8985 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 --2- 172.21.15.59:0/541840911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdeea91a00 0x55cdeeaa46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 >> 172.21.15.59:0/541840911 conn(0x55cdeeaa1360 msgr2=0x55cdeeabb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 shutdown_connections 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 1 -- 172.21.15.59:0/541840911 wait complete. 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.468+0000 7f954001e740 20 client.8985 trim_cache size 0 max 16384 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8490, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.003980984, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 89, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 99, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 99, 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 82.689931697000006 2024-04-24T06:39:56.475 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 msgr2=0x7f951406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 0x7f951406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f95381617c0 tx=0x7f9520008040 comp rx=0 tx=0).stop 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 msgr2=0x7f95381f8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f952800c430 tx=0x7f952800c8f0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 shutdown_connections 2024-04-24T06:39:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f951406c8f0 0x7f951406edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f953806fd50 0x7f95380701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f953815eb40 0x7f9538157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 --2- 172.21.15.59:0/944816836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f953815e760 0x7f95381f8670 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 >> 172.21.15.59:0/944816836 conn(0x7f9538067760 msgr2=0x7f953806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 shutdown_connections 2024-04-24T06:39:56.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.469+0000 7f9537200640 1 -- 172.21.15.59:0/944816836 wait complete. 2024-04-24T06:39:56.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-24T06:39:56.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:56.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-24T06:39:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.672+0000 7fb0c8600640 1 Processor -- start 2024-04-24T06:39:56.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.672+0000 7fb0c8600640 1 -- start start 2024-04-24T06:39:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.674+0000 7fb0c8600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0c015eb50 0x7fb0c00d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.674+0000 7fb0c8600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 0x7fb0c00d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.674+0000 7fb0c8600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c00cd830 con 0x7fb0c00b0a60 2024-04-24T06:39:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.674+0000 7fb0c8600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c00cd5e0 con 0x7fb0c015eb50 2024-04-24T06:39:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.674+0000 7fb0c8600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00703a0 con 0x7fb0c00cdb10 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.675+0000 7fb0c5c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0c015eb50 0x7fb0c00d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.675+0000 7fb0c5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 0x7fb0c00d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.675+0000 7fb0c5c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0c015eb50 0x7fb0c00d0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:34606/0 (socket says 172.21.15.59:34606) 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.675+0000 7fb0c5200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 0x7fb0c00d0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:43512/0 (socket says 172.21.15.59:43512) 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.675+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1073627932 learned_addr learned my addr 172.21.15.59:0/1073627932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.676+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 msgr2=0x7fb0c00d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.676+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 0x7fb0c00d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.676+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1073627932 >> v1:172.21.15.66:6789/0 conn(0x7fb0c00b0a60 legacy=0x7fb0c0162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.676+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1073627932 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c00705f0 con 0x7fb0c015eb50 2024-04-24T06:39:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.677+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c015eb50 0x7fb0c00d0730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009870 tx=0x7fb0b402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7e2506b36abfef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.677+0000 7fb0bfe00640 1 -- 172.21.15.59:0/1073627932 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4037070 con 0x7fb0c015eb50 2024-04-24T06:39:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.677+0000 7fb0bfe00640 1 -- 172.21.15.59:0/1073627932 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032440 con 0x7fb0c015eb50 2024-04-24T06:39:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.677+0000 7fb0bfe00640 1 -- 172.21.15.59:0/1073627932 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4032810 con 0x7fb0c015eb50 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 -- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c015eb50 msgr2=0x7fb0c00d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c015eb50 0x7fb0c00d0730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009870 tx=0x7fb0b402a130 comp rx=0 tx=0).stop 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 -- 172.21.15.59:0/1073627932 shutdown_connections 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00cdb10 0x7fb0c00d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1073627932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c015eb50 0x7fb0c00d0730 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 -- 172.21.15.59:0/1073627932 >> 172.21.15.59:0/1073627932 conn(0x7fb0c009aa00 msgr2=0x7fb0c00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 -- 172.21.15.59:0/1073627932 shutdown_connections 2024-04-24T06:39:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.678+0000 7fb0c8600640 1 -- 172.21.15.59:0/1073627932 wait complete. 2024-04-24T06:39:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.680+0000 7fb0c8600640 1 Processor -- start 2024-04-24T06:39:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.680+0000 7fb0c8600640 1 -- start start 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00b0a60 0x7fb0c01f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c00cdb10 0x7fb0c00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00d11b0 con 0x7fb0c015eb50 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0070c40 con 0x7fb0c00cdb10 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c8600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00b3bd0 con 0x7fb0c00b0a60 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51450/0 (socket says 172.21.15.59:51450) 2024-04-24T06:39:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.681+0000 7fb0c6600640 1 -- 172.21.15.59:0/1976531071 learned_addr learned my addr 172.21.15.59:0/1976531071 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00b0a60 0x7fb0c01f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c5200640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c00cdb10 0x7fb0c00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 -- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00b0a60 msgr2=0x7fb0c01f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00b0a60 0x7fb0c01f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 -- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c00cdb10 msgr2=0x7fb0c00c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c00cdb10 0x7fb0c00c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 -- 172.21.15.59:0/1976531071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c00c6060 con 0x7fb0c015eb50 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.682+0000 7fb0c6600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c7a0 tx=0x7fb0b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.683+0000 7fb0bd600640 1 -- 172.21.15.59:0/1976531071 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8003b70 con 0x7fb0c015eb50 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.683+0000 7fb0bd600640 1 -- 172.21.15.59:0/1976531071 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b8003d30 con 0x7fb0c015eb50 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.683+0000 7fb0bd600640 1 -- 172.21.15.59:0/1976531071 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800aa90 con 0x7fb0c015eb50 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.683+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c00c6340 con 0x7fb0c015eb50 2024-04-24T06:39:56.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.683+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c00c6900 con 0x7fb0c015eb50 2024-04-24T06:39:56.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.686+0000 7fb0bd600640 1 -- 172.21.15.59:0/1976531071 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0b800ac30 con 0x7fb0c015eb50 2024-04-24T06:39:56.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.687+0000 7fb0cb5c0740 1 Processor -- start 2024-04-24T06:39:56.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.687+0000 7fb0cb5c0740 1 -- start start 2024-04-24T06:39:56.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0bd600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 0x7fb0a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0bd600640 1 -- 172.21.15.59:0/1976531071 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fb0b808bb50 con 0x7fb0c015eb50 2024-04-24T06:39:56.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0c5200640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 0x7fb0a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8228f7580 0x55b8228f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f6e40 0x55b8228cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.688+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b822830e70 con 0x55b822914460 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8227d7380 con 0x55b8228f7580 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b82282ce30 con 0x55b8228f6e40 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5200640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 0x7fb0a006ede0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb0c006fb70 tx=0x7fb0ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51456/0 (socket says 172.21.15.59:51456) 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1297237212 learned_addr learned my addr 172.21.15.59:0/1297237212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5200640 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8228f7580 0x55b8228f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c6600640 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f6e40 0x55b8228cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f6e40 msgr2=0x55b8228cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f6e40 0x55b8228cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8228f7580 msgr2=0x55b8228f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8228f7580 0x55b8228f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.689+0000 7fb0c5c00640 1 -- 172.21.15.59:0/1297237212 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82260b7e0 con 0x55b822914460 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.690+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402a170 tx=0x7fb0b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d7bd0882194a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.690+0000 7fb0b2000640 1 -- 172.21.15.59:0/1297237212 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4037040 con 0x55b822914460 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.690+0000 7fb0b2000640 1 -- 172.21.15.59:0/1297237212 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032740 con 0x55b822914460 2024-04-24T06:39:56.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.690+0000 7fb0b2000640 1 -- 172.21.15.59:0/1297237212 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4032af0 con 0x55b822914460 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/1297237212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 msgr2=0x55b8228c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402a170 tx=0x7fb0b402c000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/1297237212 shutdown_connections 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f6e40 0x55b8228cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8228f7580 0x55b8228f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/1297237212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b822914460 0x55b8228c6df0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/1297237212 >> 172.21.15.59:0/1297237212 conn(0x55b8228c9060 msgr2=0x55b822911900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/1297237212 shutdown_connections 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.691+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/1297237212 wait complete. 2024-04-24T06:39:56.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.692+0000 7fb0cb5c0740 1 -- start start 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.692+0000 7fb0cb5c0740 1 Processor -- start 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b8228f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8228cb210 con 0x55b8228f9100 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8227ea660 con 0x55b822914460 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b8228ca600 con 0x55b8228f7580 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0cb5c0740 10 client.0 _ll_register_callbacks cb 0x55b8227d8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0c6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0c6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51470/0 (socket says 172.21.15.59:51470) 2024-04-24T06:39:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0c6600640 1 -- 172.21.15.59:0/531307583 learned_addr learned my addr 172.21.15.59:0/531307583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b8228f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.693+0000 7fb0c5200640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 -- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 msgr2=0x55b8228f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b8228f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 -- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 msgr2=0x55b822921a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 -- 172.21.15.59:0/531307583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82266bf70 con 0x55b8228f9100 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0c6600640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800a430 tx=0x7fb0b8059380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7cf440571eff6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8088060 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fb0b8058bd0 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b4032740 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb0b8076050 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0cb5c0740 10 client.8994 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.694+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55b822503170 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032740 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fb0b80554d0 con 0x55b8228f9100 2024-04-24T06:39:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fa53ec00640 1 Processor -- start 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 10 client.8994 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 10 client.8994 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 20 client.8994 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229085c0 0x55b8229089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 4 client.8994 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82283e890 con 0x55b8229085c0 2024-04-24T06:39:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fa53ec00640 1 -- start start 2024-04-24T06:39:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.695+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229085c0 0x55b8229089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229085c0 0x55b8229089f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d7f71611e10cdd server_cookie=b13829bc779a48f6 in_seq=0 out_seq=0 2024-04-24T06:39:56.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fb0a6a00640 10 client.8994 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fa53ec00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa53815e880 0x7fa5380cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fa53ec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa538163040 0x7fa5380d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fa53ec00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5380d2240 con 0x7fa5380d3500 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fa53ec00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa5380d05a0 con 0x7fa53815e880 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.696+0000 7fa53ec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa5380d24e0 con 0x7fa538163040 2024-04-24T06:39:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa53815e880 0x7fa5380cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0a6a00640 1 -- 172.21.15.59:0/531307583 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b82283e890 con 0x55b8229085c0 2024-04-24T06:39:56.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0a6a00640 10 client.8994 handle_command_reply: tid=0 2024-04-24T06:39:56.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa53815e880 0x7fa5380cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51478/0 (socket says 172.21.15.59:51478) 2024-04-24T06:39:56.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 -- 172.21.15.59:0/1570823276 learned_addr learned my addr 172.21.15.59:0/1570823276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53e200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa5380d3500 0x7fa5380d4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:34366/0 (socket says 172.21.15.59:34366) 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 client.8994 shutdown 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53ce00640 1 --2- 172.21.15.59:0/1570823276 >> v2:172.21.15.66:3300/0 conn(0x7fa538163040 0x7fa5380d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 -- 172.21.15.59:0/1570823276 >> v2:172.21.15.66:3300/0 conn(0x7fa538163040 msgr2=0x7fa5380d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 --2- 172.21.15.59:0/1570823276 >> v2:172.21.15.66:3300/0 conn(0x7fa538163040 0x7fa5380d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 -- 172.21.15.59:0/1570823276 >> v1:172.21.15.59:6789/0 conn(0x7fa5380d3500 legacy=0x7fa5380d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fa53d800640 1 -- 172.21.15.59:0/1570823276 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5380d27c0 con 0x7fa53815e880 2024-04-24T06:39:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 msgr2=0x55b8228fa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800a430 tx=0x7fb0b8059380 comp rx=0 tx=0).stop 2024-04-24T06:39:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 shutdown_connections 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229085c0 0x55b8229089f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228fa140 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/531307583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b8228f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 >> 172.21.15.59:0/531307583 conn(0x55b8228f6e40 msgr2=0x55b8228f3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.697+0000 7fb0a6a00640 0 client.8994 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.698+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 shutdown_connections 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.698+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/531307583 wait complete. 2024-04-24T06:39:56.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.698+0000 7fb0cb5c0740 20 client.8994 trim_cache size 0 max 16384 2024-04-24T06:39:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.699+0000 7fa53d800640 1 --2- 172.21.15.59:0/1570823276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5380cfd70 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa5380d2150 tx=0x7fa52c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ec438252d95d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.699+0000 7fa537e00640 1 -- 172.21.15.59:0/1570823276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c037070 con 0x7fa53815e880 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.699+0000 7fa537e00640 1 -- 172.21.15.59:0/1570823276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c032440 con 0x7fa53815e880 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.699+0000 7fa537e00640 1 -- 172.21.15.59:0/1570823276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c032810 con 0x7fa53815e880 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 -- 172.21.15.59:0/1570823276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 msgr2=0x7fa5380cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 --2- 172.21.15.59:0/1570823276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5380cfd70 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa5380d2150 tx=0x7fa52c02a270 comp rx=0 tx=0).stop 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 -- 172.21.15.59:0/1570823276 shutdown_connections 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 --2- 172.21.15.59:0/1570823276 >> v2:172.21.15.66:3300/0 conn(0x7fa538163040 0x7fa5380d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 --2- 172.21.15.59:0/1570823276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5380cfd70 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 -- 172.21.15.59:0/1570823276 >> 172.21.15.59:0/1570823276 conn(0x7fa53808e0d0 msgr2=0x7fa53809d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 -- 172.21.15.59:0/1570823276 shutdown_connections 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.700+0000 7fa53ec00640 1 -- 172.21.15.59:0/1570823276 wait complete. 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.703+0000 7fa53ec00640 1 Processor -- start 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.703+0000 7fa53ec00640 1 -- start start 2024-04-24T06:39:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5380d3500 0x7fa5381f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5381576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa538163040 0x7fa538157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380d02b0 con 0x7fa53815e880 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380d2aa0 con 0x7fa538163040 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa53815ec60 con 0x7fa5380d3500 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5381576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5381576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51488/0 (socket says 172.21.15.59:51488) 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53ce00640 1 -- 172.21.15.59:0/484783400 learned_addr learned my addr 172.21.15.59:0/484783400 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.705+0000 7fa53d800640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5380d3500 0x7fa5381f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 -- 172.21.15.59:0/484783400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5380d3500 msgr2=0x7fa5381f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5380d3500 0x7fa5381f3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 -- 172.21.15.59:0/484783400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa538163040 msgr2=0x7fa538157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa538163040 0x7fa538157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 -- 172.21.15.59:0/484783400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5380c83e0 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ce00640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa53815e880 0x7fa5381576e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fa52400d5f0 tx=0x7fa52400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240125f0 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524019070 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ec00640 1 -- 172.21.15.59:0/484783400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa538160b30 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240129d0 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.706+0000 7fa53ec00640 1 -- 172.21.15.59:0/484783400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381610f0 con 0x7fa53815e880 2024-04-24T06:39:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.707+0000 7fa53ec00640 1 -- 172.21.15.59:0/484783400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa53804fe00 con 0x7fa53815e880 2024-04-24T06:39:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.708+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa524012b70 con 0x7fa53815e880 2024-04-24T06:39:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.710+0000 7fa535600640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa51806c8f0 0x7fa51806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.710+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fa52408c0a0 con 0x7fa53815e880 2024-04-24T06:39:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.710+0000 7fa53d800640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa51806c8f0 0x7fa51806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.710+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7fa52408e640 con 0x7fa53815e880 2024-04-24T06:39:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.711+0000 7fa53ec00640 1 -- 172.21.15.59:0/484783400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fa5380b5af0 con 0x7fa53815e880 2024-04-24T06:39:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.711+0000 7fa53d800640 1 --2- 172.21.15.59:0/484783400 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa51806c8f0 0x7fa51806edb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa52c034000 tx=0x7fa52c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.722+0000 7fb0cb5c0740 1 Processor -- start 2024-04-24T06:39:56.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.722+0000 7fb0cb5c0740 1 -- start start 2024-04-24T06:39:56.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.722+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.722+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f9100 0x55b8228f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.722+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c02010c0 con 0x55b8228f7580 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c02000e0 con 0x55b822914460 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c01ff620 con 0x55b8228f9100 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 -- 172.21.15.59:0/4040117128 learned_addr learned my addr 172.21.15.59:0/4040117128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 -- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f9100 msgr2=0x55b8228f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5200640 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f9100 0x55b8228f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c6600640 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f9100 0x55b8228f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 -- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 msgr2=0x55b822921360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 -- 172.21.15.59:0/4040117128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8228f6aa0 con 0x55b8228f7580 2024-04-24T06:39:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.723+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402a170 tx=0x7fb0b4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc20a8d917e77a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0b1600640 1 -- 172.21.15.59:0/4040117128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4037040 con 0x55b8228f7580 2024-04-24T06:39:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0b1600640 1 -- 172.21.15.59:0/4040117128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032900 con 0x55b8228f7580 2024-04-24T06:39:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0b1600640 1 -- 172.21.15.59:0/4040117128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4032cf0 con 0x55b8228f7580 2024-04-24T06:39:56.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4040117128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 msgr2=0x55b8228cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402a170 tx=0x7fb0b4040040 comp rx=0 tx=0).stop 2024-04-24T06:39:56.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4040117128 shutdown_connections 2024-04-24T06:39:56.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822921360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f9100 0x55b8228f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4040117128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f7580 0x55b8228cbdd0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.724+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4040117128 >> 172.21.15.59:0/4040117128 conn(0x55b8228f6e40 msgr2=0x55b8228c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.725+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4040117128 shutdown_connections 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.725+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4040117128 wait complete. 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.725+0000 7fb0cb5c0740 1 -- start start 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.725+0000 7fb0cb5c0740 1 Processor -- start 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b82296f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822912a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8228c5d80 con 0x55b8228f9100 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01ffac0 con 0x55b822914460 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b8228f2a50 con 0x55b8228f7580 2024-04-24T06:39:56.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.726+0000 7fb0cb5c0740 10 client.0 _ll_register_callbacks cb 0x7fb0b800ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51514/0 (socket says 172.21.15.59:51514) 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 -- 172.21.15.59:0/4024454087 learned_addr learned my addr 172.21.15.59:0/4024454087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b82296f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c6600640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822912a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 -- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 msgr2=0x55b82296f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b82296f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 -- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 msgr2=0x55b822912a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822912a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 -- 172.21.15.59:0/4024454087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b822912f70 con 0x55b8228f9100 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0c5200640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009990 tx=0x7fb0ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f1488c63e03eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.727+0000 7fb0a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0236d0 con 0x55b8228f9100 2024-04-24T06:39:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fb0ac023870 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b4032900 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac033050 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 10 client.9009 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55b822971bb0 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032900 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fb0ac036020 con 0x55b8228f9100 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 10 client.9009 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 10 client.9009 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 20 client.9009 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229720d0 0x55b8228f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 4 client.9009 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.728+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55b822838700 con 0x55b8229720d0 2024-04-24T06:39:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.729+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229720d0 0x55b8228f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.729+0000 7fb0c5c00640 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229720d0 0x55b8228f07e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff34f3ebde287a3 server_cookie=432ca717446a5d26 in_seq=0 out_seq=0 2024-04-24T06:39:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.729+0000 7fb0a6a00640 10 client.9009 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0a6a00640 1 -- 172.21.15.59:0/4024454087 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b822838700 con 0x55b8229720d0 2024-04-24T06:39:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0a6a00640 10 client.9009 handle_command_reply: tid=0 2024-04-24T06:39:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 client.9009 shutdown 2024-04-24T06:39:56.735 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-24T06:39:56.735 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-24T06:39:56.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 msgr2=0x55b8228ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009990 tx=0x7fb0ac0046c0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.735 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-24T06:39:56.735 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-24T06:39:56.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 shutdown_connections 2024-04-24T06:39:56.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x55b8229720d0 0x55b8228f07e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b822914460 0x55b822912a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8228f9100 0x55b8228ce0c0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 --2- 172.21.15.59:0/4024454087 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b8228f7580 0x55b82296f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 >> 172.21.15.59:0/4024454087 conn(0x55b8228f6e40 msgr2=0x55b822935660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.730+0000 7fb0a6a00640 0 client.9009 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.731+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 shutdown_connections 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.731+0000 7fb0cb5c0740 1 -- 172.21.15.59:0/4024454087 wait complete. 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.731+0000 7fb0cb5c0740 20 client.9009 trim_cache size 0 max 16384 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "3d5bb4b8-a154-40c4-84b7-fe1ec47559ff", 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-24T06:39:56.737 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:56.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.731+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 msgr2=0x7fb0a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 0x7fb0a006ede0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb0c006fb70 tx=0x7fb0ac008040 comp rx=0 tx=0).stop 2024-04-24T06:39:56.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 msgr2=0x7fb0c00cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c7a0 tx=0x7fb0b800cc60 comp rx=0 tx=0).stop 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 shutdown_connections 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a006c920 0x7fb0a006ede0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c015eb50 0x7fb0c00cbd90 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c00cdb10 0x7fb0c00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 --2- 172.21.15.59:0/1976531071 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0c00b0a60 0x7fb0c01f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 >> 172.21.15.59:0/1976531071 conn(0x7fb0c009aa00 msgr2=0x7fb0c00b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 shutdown_connections 2024-04-24T06:39:56.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.732+0000 7fb0c8600640 1 -- 172.21.15.59:0/1976531071 wait complete. 2024-04-24T06:39:56.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:39:56.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T06:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.927+0000 7fe17d400640 1 Processor -- start 2024-04-24T06:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.928+0000 7fe17d400640 1 -- start start 2024-04-24T06:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.929+0000 7fe17d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe17815c410 0x7fe1780d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.929+0000 7fe17d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1780af790 0x7fe178071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.929+0000 7fe17d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe178160120 con 0x7fe1780d0b10 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.929+0000 7fe17d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780d0f70 con 0x7fe1780af790 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.929+0000 7fe17d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe178160400 con 0x7fe17815c410 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1780af790 0x7fe178071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1780af790 0x7fe178071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51522/0 (socket says 172.21.15.59:51522) 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 -- 172.21.15.59:0/197578622 learned_addr learned my addr 172.21.15.59:0/197578622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe177e00640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe17815c410 0x7fe1780d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 -- 172.21.15.59:0/197578622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe17815c410 msgr2=0x7fe1780d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe17815c410 0x7fe1780d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 -- 172.21.15.59:0/197578622 >> v1:172.21.15.66:6789/0 conn(0x7fe1780d0b10 legacy=0x7fe1780af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.930+0000 7fe17ca00640 1 -- 172.21.15.59:0/197578622 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1781606e0 con 0x7fe1780af790 2024-04-24T06:39:56.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.931+0000 7fe17ca00640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780af790 0x7fe178071c70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe170009e50 tx=0x7fe17002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd41d0b5e2dfc2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe177400640 1 -- 172.21.15.59:0/197578622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036070 con 0x7fe1780af790 2024-04-24T06:39:56.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe177400640 1 -- 172.21.15.59:0/197578622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170031600 con 0x7fe1780af790 2024-04-24T06:39:56.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe177400640 1 -- 172.21.15.59:0/197578622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1700319d0 con 0x7fe1780af790 2024-04-24T06:39:56.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 -- 172.21.15.59:0/197578622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780af790 msgr2=0x7fe178071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780af790 0x7fe178071c70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe170009e50 tx=0x7fe17002a6c0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 -- 172.21.15.59:0/197578622 shutdown_connections 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780af790 0x7fe178071c70 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 --2- 172.21.15.59:0/197578622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe17815c410 0x7fe1780d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.932+0000 7fe17d400640 1 -- 172.21.15.59:0/197578622 >> 172.21.15.59:0/197578622 conn(0x7fe17806bdf0 msgr2=0x7fe17806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.933+0000 7fe17d400640 1 -- 172.21.15.59:0/197578622 shutdown_connections 2024-04-24T06:39:56.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.933+0000 7fe17d400640 1 -- 172.21.15.59:0/197578622 wait complete. 2024-04-24T06:39:56.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.934+0000 7fe17d400640 1 Processor -- start 2024-04-24T06:39:56.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.934+0000 7fe17d400640 1 -- start start 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 0x7fe1780b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1780d0b10 0x7fe1780cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe17815c7f0 con 0x7fe17815c410 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780721b0 con 0x7fe1780d0b10 2024-04-24T06:39:56.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780b2d00 con 0x7fe1780af790 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51526/0 (socket says 172.21.15.59:51526) 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.935+0000 7fe17ca00640 1 -- 172.21.15.59:0/1769419239 learned_addr learned my addr 172.21.15.59:0/1769419239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe16f400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1780d0b10 0x7fe1780cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe177e00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 0x7fe1780b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 -- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 msgr2=0x7fe1780b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 0x7fe1780b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 -- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1780d0b10 msgr2=0x7fe1780cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1780d0b10 0x7fe1780cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 -- 172.21.15.59:0/1769419239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe178158520 con 0x7fe17815c410 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe177e00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 0x7fe1780b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe17ca00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe170009f70 tx=0x7fe170004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.936+0000 7fe174c00640 1 -- 172.21.15.59:0/1769419239 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170041020 con 0x7fe17815c410 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.937+0000 7fe174c00640 1 -- 172.21.15.59:0/1769419239 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170008710 con 0x7fe17815c410 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.937+0000 7fe174c00640 1 -- 172.21.15.59:0/1769419239 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036040 con 0x7fe17815c410 2024-04-24T06:39:56.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.937+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1780cdc20 con 0x7fe17815c410 2024-04-24T06:39:56.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.937+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1780ce480 con 0x7fe17815c410 2024-04-24T06:39:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.939+0000 7fe174c00640 1 -- 172.21.15.59:0/1769419239 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe170044020 con 0x7fe17815c410 2024-04-24T06:39:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.940+0000 7fe174c00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 0x7fe15406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.940+0000 7fe174c00640 1 -- 172.21.15.59:0/1769419239 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fe1700ac8d0 con 0x7fe17815c410 2024-04-24T06:39:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.941+0000 7fe177e00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 0x7fe15406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.941+0000 7fe18185a740 1 Processor -- start 2024-04-24T06:39:56.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.941+0000 7fe18185a740 1 -- start start 2024-04-24T06:39:56.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.941+0000 7fe177e00640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 0x7fe15406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe168006fd0 tx=0x7fe168008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72dea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72dea7e0 0x557c72dbf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c72d241f0 con 0x557c72e07e00 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c72cca350 con 0x557c72dea7e0 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557c72d201b0 con 0x557c72deaf20 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51530/0 (socket says 172.21.15.59:51530) 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.942+0000 7fe177e00640 1 -- 172.21.15.59:0/4060390085 learned_addr learned my addr 172.21.15.59:0/4060390085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 -- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 msgr2=0x557c72dea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72dea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 -- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72dea7e0 msgr2=0x557c72dbf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72dea7e0 0x557c72dbf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 -- 172.21.15.59:0/4060390085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c72d1ef50 con 0x557c72e07e00 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe177e00640 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe1680043b0 tx=0x7fe168011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e28dd2a60fd70d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe167e00640 1 -- 172.21.15.59:0/4060390085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800f040 con 0x557c72e07e00 2024-04-24T06:39:56.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe167e00640 1 -- 172.21.15.59:0/4060390085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1680043e0 con 0x557c72e07e00 2024-04-24T06:39:56.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.943+0000 7fe167e00640 1 -- 172.21.15.59:0/4060390085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800ee80 con 0x557c72e07e00 2024-04-24T06:39:56.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 -- 172.21.15.59:0/4060390085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 msgr2=0x557c72dba790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe1680043b0 tx=0x7fe168011000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 -- 172.21.15.59:0/4060390085 shutdown_connections 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72dea7e0 0x557c72dbf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72dea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 --2- 172.21.15.59:0/4060390085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72dba790 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 -- 172.21.15.59:0/4060390085 >> 172.21.15.59:0/4060390085 conn(0x557c72dbca00 msgr2=0x557c72e052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 -- 172.21.15.59:0/4060390085 shutdown_connections 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.944+0000 7fe18185a740 1 -- 172.21.15.59:0/4060390085 wait complete. 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.945+0000 7fe18185a740 1 -- start start 2024-04-24T06:39:56.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.945+0000 7fe18185a740 1 Processor -- start 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72ded460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e07e00 0x557c72de5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c72dbebb0 con 0x557c72e04700 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c72cdd690 con 0x557c72deaf20 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557c72dbdfa0 con 0x557c72e07e00 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe18185a740 10 client.0 _ll_register_callbacks cb 0x557c72d19650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51536/0 (socket says 172.21.15.59:51536) 2024-04-24T06:39:56.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 -- 172.21.15.59:0/2988270878 learned_addr learned my addr 172.21.15.59:0/2988270878 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 -- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e07e00 msgr2=0x557c72de5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e07e00 0x557c72de5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 -- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 msgr2=0x557c72ded460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.946+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72ded460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe17ca00640 1 -- 172.21.15.59:0/2988270878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c72cdc920 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe17007a520 tx=0x7fe170079ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b500443d77c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17007e940 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fe1700a5e00 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1680043e0 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe170083170 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe18185a740 10 client.9021 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x557c72b007d0 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1680043e0 con 0x557c72e04700 2024-04-24T06:39:56.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fe1700799e0 con 0x557c72e04700 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe18185a740 10 client.9021 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.947+0000 7fe18185a740 10 client.9021 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe18185a740 20 client.9021 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72dfcc30 0x557c72dfd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe18185a740 4 client.9021 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c72d31c10 con 0x557c72dfcc30 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe177e00640 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72dfcc30 0x557c72dfd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe177e00640 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72dfcc30 0x557c72dfd010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4d0c156f7855b3 server_cookie=b6bf520b3a4c6278 in_seq=0 out_seq=0 2024-04-24T06:39:56.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.948+0000 7fe164c00640 10 client.9021 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.949+0000 7fe164c00640 1 -- 172.21.15.59:0/2988270878 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c72d31c10 con 0x557c72dfcc30 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.949+0000 7fe164c00640 10 client.9021 handle_command_reply: tid=0 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.949+0000 7fe18185a740 1 client.9021 shutdown 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.949+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 msgr2=0x557c72de6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.949+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe17007a520 tx=0x7fe170079ef0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 shutdown_connections 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72dfcc30 0x557c72dfd010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72de6ec0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e07e00 0x557c72de5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 --2- 172.21.15.59:0/2988270878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72ded460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 >> 172.21.15.59:0/2988270878 conn(0x557c72dea7e0 msgr2=0x557c72e052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe164c00640 0 client.9021 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 shutdown_connections 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 1 -- 172.21.15.59:0/2988270878 wait complete. 2024-04-24T06:39:56.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.950+0000 7fe18185a740 20 client.9021 trim_cache size 0 max 16384 2024-04-24T06:39:56.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.969+0000 7fe18185a740 1 Processor -- start 2024-04-24T06:39:56.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.969+0000 7fe18185a740 1 -- start start 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72de6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e04700 0x557c72e050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe178200d70 con 0x557c72e07e00 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781f8bd0 con 0x557c72deaf20 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1781f7f30 con 0x557c72e04700 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe17ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51542/0 (socket says 172.21.15.59:51542) 2024-04-24T06:39:56.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.970+0000 7fe17ca00640 1 -- 172.21.15.59:0/2814081611 learned_addr learned my addr 172.21.15.59:0/2814081611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 -- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e04700 msgr2=0x557c72e050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e04700 0x557c72e050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 -- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 msgr2=0x557c72de6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72de6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 -- 172.21.15.59:0/2814081611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c72dfd380 con 0x557c72e07e00 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe170083330 tx=0x7fe170047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d5da9888af9fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe167400640 1 -- 172.21.15.59:0/2814081611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17007e940 con 0x557c72e07e00 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe167400640 1 -- 172.21.15.59:0/2814081611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1700799e0 con 0x557c72e07e00 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.971+0000 7fe167400640 1 -- 172.21.15.59:0/2814081611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170008d20 con 0x557c72e07e00 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- 172.21.15.59:0/2814081611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 msgr2=0x557c72e055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe170083330 tx=0x7fe170047000 comp rx=0 tx=0).stop 2024-04-24T06:39:56.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- 172.21.15.59:0/2814081611 shutdown_connections 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e07e00 0x557c72e055e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72e04700 0x557c72e050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 --2- 172.21.15.59:0/2814081611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72deaf20 0x557c72de6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- 172.21.15.59:0/2814081611 >> 172.21.15.59:0/2814081611 conn(0x557c72dea7e0 msgr2=0x557c72ded200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- 172.21.15.59:0/2814081611 shutdown_connections 2024-04-24T06:39:56.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- 172.21.15.59:0/2814081611 wait complete. 2024-04-24T06:39:56.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.972+0000 7fe18185a740 1 -- start start 2024-04-24T06:39:56.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.973+0000 7fe18185a740 1 Processor -- start 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72e27c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72e07e00 0x557c72e7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c72cd8d80 con 0x557c72e04700 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781f8580 con 0x557c72e07e00 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x557c72dfd380 con 0x557c72deaf20 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe18185a740 10 client.0 _ll_register_callbacks cb 0x7fe170008320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51558/0 (socket says 172.21.15.59:51558) 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 -- 172.21.15.59:0/2830234388 learned_addr learned my addr 172.21.15.59:0/2830234388 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe177e00640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72e27c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe17ca00640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72e07e00 0x557c72e7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 -- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 msgr2=0x557c72e27c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72e27c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 -- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72e07e00 msgr2=0x557c72e7efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72e07e00 0x557c72e7efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.974+0000 7fe16f400640 1 -- 172.21.15.59:0/2830234388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c72a363f0 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe16f400640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fe160005320 tx=0x7fe16000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a73b32c1a224f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe164c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160016020 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fe1600123f0 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1700799e0 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe160020050 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe18185a740 10 client.9027 fetch_fsmap learned FSMap version 89 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x557c729f60f0 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.975+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1700799e0 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fe160023020 con 0x557c72e04700 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 10 client.9027 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:39:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 10 client.9027 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:39:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 20 client.9027 populate_metadata read hostname 'smithi059' 2024-04-24T06:39:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72db9d20 0x557c72e7f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 4 client.9027 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:39:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c72d2ba80 con 0x557c72db9d20 2024-04-24T06:39:56.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe177e00640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72db9d20 0x557c72e7f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.976+0000 7fe177e00640 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72db9d20 0x557c72e7f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae5198602494ff9 server_cookie=367932fcc420ef4f in_seq=0 out_seq=0 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe164c00640 10 client.9027 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe164c00640 1 -- 172.21.15.59:0/2830234388 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x557c72d2ba80 con 0x557c72db9d20 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe164c00640 10 client.9027 handle_command_reply: tid=0 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 client.9027 shutdown 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 msgr2=0x557c72e67ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fe160005320 tx=0x7fe16000d520 comp rx=0 tx=0).stop 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 shutdown_connections 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x557c72db9d20 0x557c72e7f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c72e07e00 0x557c72e7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c72e04700 0x557c72e67ee0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 --2- 172.21.15.59:0/2830234388 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x557c72deaf20 0x557c72e27c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe164c00640 0 client.9027 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 >> 172.21.15.59:0/2830234388 conn(0x557c72dea7e0 msgr2=0x557c72e17980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 shutdown_connections 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 1 -- 172.21.15.59:0/2830234388 wait complete. 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.977+0000 7fe18185a740 20 client.9027 trim_cache size 0 max 16384 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:39:56.982 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: "3d5bb4b8-a154-40c4-84b7-fe1ec47559ff": { 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "3d5bb4b8-a154-40c4-84b7-fe1ec47559ff", 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:39:56.983 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 msgr2=0x7fe15406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 0x7fe15406edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe168006fd0 tx=0x7fe168008040 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 msgr2=0x7fe17815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe170009f70 tx=0x7fe170004a40 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 shutdown_connections 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe15406c8f0 0x7fe15406edb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe17815c410 0x7fe17815e0d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1780d0b10 0x7fe1780cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 --2- 172.21.15.59:0/1769419239 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1780af790 0x7fe1780b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 >> 172.21.15.59:0/1769419239 conn(0x7fe17806bdf0 msgr2=0x7fe17806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 shutdown_connections 2024-04-24T06:39:56.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:56.978+0000 7fe17d400640 1 -- 172.21.15.59:0/1769419239 wait complete. 2024-04-24T06:39:56.996 INFO:tasks.cephfs.filesystem:scrub status for tag:3d5bb4b8-a154-40c4-84b7-fe1ec47559ff - {'path': '/', 'tag': '3d5bb4b8-a154-40c4-84b7-fe1ec47559ff', 'options': 'recursive,repair'} 2024-04-24T06:39:57.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:57.494+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-24T06:39:56.731266+0000) ==== 1407+0+0 (secure 0 0 0) 0x7fa524012e60 con 0x7fa53815e880 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: id: fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-24T06:39:57.497 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 11m) 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 11m), standbys: x 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 11m), 8 in (since 11m) 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 30 objects, 19 MiB 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-24T06:39:57.498 INFO:teuthology.orchestra.run.smithi059.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.731266+0000 mds.a [INF] scrub queued for path: / 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.731269+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.731273+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.732070+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.732077+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.732197+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.732203+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-24T06:39:57.499 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:56.733379+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-24T06:39:59.520 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:39:58.785566+0000 mds.a [INF] scrub summary: idle 2024-04-24T06:39:59.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:39:59.518+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-24T06:39:58.785566+0000) ==== 189+0+0 (secure 0 0 0) 0x7fa524057d70 con 0x7fa53815e880 2024-04-24T06:40:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:00.522+0000 7fa535600640 1 -- 172.21.15.59:0/484783400 <== mon.0 v2:172.21.15.59:3300/0 9 ==== log(1 entries from seq 1080 at 2024-04-24T06:40:00.000180+0000) ==== 187+0+0 (secure 0 0 0) 0x7fa524057fd0 con 0x7fa53815e880 2024-04-24T06:40:00.524 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:40:00.000180+0000 mon.a [INF] overall HEALTH_OK 2024-04-24T06:40:08.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:40:08.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:40:26.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:26.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-24T06:40:27.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.223+0000 7f5d9d600640 1 Processor -- start 2024-04-24T06:40:27.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.224+0000 7f5d9d600640 1 -- start start 2024-04-24T06:40:27.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.225+0000 7f5d9d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d980d0ce0 0x7f5d980af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.225+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.225+0000 7f5d9d600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d98160120 con 0x7f5d9815c410 2024-04-24T06:40:27.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.225+0000 7f5d9d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5d98160400 con 0x7f5d980d0ce0 2024-04-24T06:40:27.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.225+0000 7f5d9d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5d981606e0 con 0x7f5d980af5c0 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d97400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d980d0ce0 0x7f5d980af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d97400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5d980d0ce0 0x7f5d980af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:59446/0 (socket says 172.21.15.59:59446) 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:49178/0 (socket says 172.21.15.59:49178) 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d97400640 1 -- 172.21.15.59:0/2277056967 learned_addr learned my addr 172.21.15.59:0/2277056967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.226+0000 7f5d97e00640 1 --1- 172.21.15.59:0/2277056967 >> v1:172.21.15.66:6789/0 conn(0x7f5d9815c410 0x7f5d980d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:39732/0 (socket says 172.21.15.59:39732) 2024-04-24T06:40:27.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.227+0000 7f5d97400640 1 -- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 msgr2=0x7f5d98071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.227+0000 7f5d97400640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.227+0000 7f5d97400640 1 -- 172.21.15.59:0/2277056967 >> v1:172.21.15.66:6789/0 conn(0x7f5d9815c410 legacy=0x7f5d980d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.227+0000 7f5d97400640 1 -- 172.21.15.59:0/2277056967 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d981609c0 con 0x7f5d980d0ce0 2024-04-24T06:40:27.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.228+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:27.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.229+0000 7f5d97400640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d980d0ce0 0x7f5d980af080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009e50 tx=0x7f5d8002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2006bfffb633e281 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.230+0000 7f5d96a00640 1 -- 172.21.15.59:0/2277056967 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80036070 con 0x7f5d980d0ce0 2024-04-24T06:40:27.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.230+0000 7f5d96a00640 1 -- 172.21.15.59:0/2277056967 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80031600 con 0x7f5d980d0ce0 2024-04-24T06:40:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.230+0000 7f5d96a00640 1 -- 172.21.15.59:0/2277056967 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d800319d0 con 0x7f5d980d0ce0 2024-04-24T06:40:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.230+0000 7f5d9d600640 1 -- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d980d0ce0 msgr2=0x7f5d980af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.230+0000 7f5d9d600640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d980d0ce0 0x7f5d980af080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009e50 tx=0x7f5d8002a710 comp rx=0 tx=0).stop 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 -- 172.21.15.59:0/2277056967 shutdown_connections 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d98071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 --2- 172.21.15.59:0/2277056967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d980d0ce0 0x7f5d980af080 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 -- 172.21.15.59:0/2277056967 >> 172.21.15.59:0/2277056967 conn(0x7f5d9806bdf0 msgr2=0x7f5d9806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 -- 172.21.15.59:0/2277056967 shutdown_connections 2024-04-24T06:40:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.231+0000 7f5d9d600640 1 -- 172.21.15.59:0/2277056967 wait complete. 2024-04-24T06:40:27.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.233+0000 7f5d9d600640 1 Processor -- start 2024-04-24T06:40:27.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.233+0000 7f5d9d600640 1 -- start start 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d9815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9815c410 0x7f5d981442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980afb80 con 0x7f5d980d0ce0 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980afa00 con 0x7f5d9815c410 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.234+0000 7f5d9d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5d980721b0 con 0x7f5d980af5c0 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48428/0 (socket says 172.21.15.59:48428) 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 -- 172.21.15.59:0/298964679 learned_addr learned my addr 172.21.15.59:0/298964679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9815c410 0x7f5d981442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97e00640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d9815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 -- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 msgr2=0x7f5d9815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d9815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 -- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9815c410 msgr2=0x7f5d981442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9815c410 0x7f5d981442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.235+0000 7f5d97400640 1 -- 172.21.15.59:0/298964679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980cf770 con 0x7f5d980d0ce0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d97400640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5d80034040 tx=0x7f5d80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d8fe00640 1 -- 172.21.15.59:0/298964679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80036050 con 0x7f5d980d0ce0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d8fe00640 1 -- 172.21.15.59:0/298964679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80042070 con 0x7f5d980d0ce0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d8fe00640 1 -- 172.21.15.59:0/298964679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80009510 con 0x7f5d980d0ce0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d98156050 con 0x7f5d980d0ce0 2024-04-24T06:40:27.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.236+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d98156610 con 0x7f5d980d0ce0 2024-04-24T06:40:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.239+0000 7f5d8fe00640 1 -- 172.21.15.59:0/298964679 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d80007c70 con 0x7f5d980d0ce0 2024-04-24T06:40:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.240+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 0x7f5d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.240+0000 7f5d8fe00640 1 -- 172.21.15.59:0/298964679 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f5d800aca40 con 0x7f5d980d0ce0 2024-04-24T06:40:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.240+0000 7f5d97e00640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 0x7f5d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.241+0000 7f5da19d6740 1 Processor -- start 2024-04-24T06:40:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.241+0000 7f5da19d6740 1 -- start start 2024-04-24T06:40:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.241+0000 7f5d97e00640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 0x7f5d7406edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000f60 tx=0x7f5d88007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.241+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b24b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b24b7e0 0x56190b2205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5da19d6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56190b1851f0 con 0x56190b268e00 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56190b12b350 con 0x56190b24bf20 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56190b1811b0 con 0x56190b24b7e0 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48434/0 (socket says 172.21.15.59:48434) 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 -- 172.21.15.59:0/3839818377 learned_addr learned my addr 172.21.15.59:0/3839818377 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97400640 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b24b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b24b7e0 0x56190b2205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 -- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b24b7e0 msgr2=0x56190b2205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b24b7e0 0x56190b2205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 -- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 msgr2=0x56190b24b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b24b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.242+0000 7f5d97e00640 1 -- 172.21.15.59:0/3839818377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190b17ff50 con 0x56190b268e00 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.243+0000 7f5d97e00640 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007720 tx=0x7f5d88011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aefdb378688c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.243+0000 7f5d87e00640 1 -- 172.21.15.59:0/3839818377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88001b70 con 0x56190b268e00 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.243+0000 7f5d87e00640 1 -- 172.21.15.59:0/3839818377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d88001d10 con 0x56190b268e00 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.243+0000 7f5d87e00640 1 -- 172.21.15.59:0/3839818377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88005250 con 0x56190b268e00 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 -- 172.21.15.59:0/3839818377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 msgr2=0x56190b21b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007720 tx=0x7f5d88011000 comp rx=0 tx=0).stop 2024-04-24T06:40:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 -- 172.21.15.59:0/3839818377 shutdown_connections 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b24b7e0 0x56190b2205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b24b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3839818377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b21b790 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 -- 172.21.15.59:0/3839818377 >> 172.21.15.59:0/3839818377 conn(0x56190b21da00 msgr2=0x56190b2662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 -- 172.21.15.59:0/3839818377 shutdown_connections 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.244+0000 7f5da19d6740 1 -- 172.21.15.59:0/3839818377 wait complete. 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.245+0000 7f5da19d6740 1 -- start start 2024-04-24T06:40:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.245+0000 7f5da19d6740 1 Processor -- start 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b25eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b2606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56190b21fbb0 con 0x56190b268e00 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56190b13e690 con 0x56190b24bf20 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56190b21efa0 con 0x56190b246f20 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5da19d6740 10 client.0 _ll_register_callbacks cb 0x56190b17a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5d97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5d97400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48446/0 (socket says 172.21.15.59:48446) 2024-04-24T06:40:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.246+0000 7f5d97400640 1 -- 172.21.15.59:0/4034374427 learned_addr learned my addr 172.21.15.59:0/4034374427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97e00640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b25eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 -- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 msgr2=0x56190b2606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b2606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 -- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 msgr2=0x56190b25eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b25eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 -- 172.21.15.59:0/4034374427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190b13d920 con 0x56190b268e00 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97e00640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b25eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d97400640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f5d800aaf00 tx=0x7f5d8007a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfc69cb6b1d1a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80091070 con 0x56190b268e00 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f5d80079d50 con 0x56190b268e00 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.247+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d88001d10 con 0x56190b268e00 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5d80031600 con 0x56190b268e00 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 10 client.9036 fetch_fsmap learned FSMap version 89 2024-04-24T06:40:27.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x56190af617d0 con 0x56190b268e00 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d88001d10 con 0x56190b268e00 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f5d80076450 con 0x56190b268e00 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 10 client.9036 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 10 client.9036 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 20 client.9036 populate_metadata read hostname 'smithi059' 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b24c8e0 0x56190b24cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 4 client.9036 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190b192c10 con 0x56190b24c8e0 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.248+0000 7f5d97e00640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b24c8e0 0x56190b24cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.249+0000 7f5d97e00640 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b24c8e0 0x56190b24cd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6de16b221d301ee server_cookie=37b9f26142c8b125 in_seq=0 out_seq=0 2024-04-24T06:40:27.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.249+0000 7f5d84c00640 10 client.9036 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5d84c00640 1 -- 172.21.15.59:0/4034374427 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56190b192c10 con 0x56190b24c8e0 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5d84c00640 10 client.9036 handle_command_reply: tid=0 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 client.9036 shutdown 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 msgr2=0x56190b25cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f5d800aaf00 tx=0x7f5d8007a5e0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 shutdown_connections 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b24c8e0 0x56190b24cd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b2606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b25cf60 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 --2- 172.21.15.59:0/4034374427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b25eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5d84c00640 0 client.9036 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 >> 172.21.15.59:0/4034374427 conn(0x56190b24b7e0 msgr2=0x56190b2673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 shutdown_connections 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.250+0000 7f5da19d6740 1 -- 172.21.15.59:0/4034374427 wait complete. 2024-04-24T06:40:27.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.251+0000 7f5da19d6740 20 client.9036 trim_cache size 0 max 16384 2024-04-24T06:40:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.270+0000 7f5da19d6740 1 Processor -- start 2024-04-24T06:40:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.270+0000 7f5da19d6740 1 -- start start 2024-04-24T06:40:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b21efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b21f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981fca50 con 0x56190b268e00 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981fbab0 con 0x56190b24bf20 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5d981fae40 con 0x56190b246f20 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48452/0 (socket says 172.21.15.59:48452) 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.271+0000 7f5d9cc00640 1 -- 172.21.15.59:0/3702036287 learned_addr learned my addr 172.21.15.59:0/3702036287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d97e00640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b21efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d97400640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b21f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 -- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 msgr2=0x56190b21efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b21efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 -- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 msgr2=0x56190b21f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b21f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 -- 172.21.15.59:0/3702036287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190b24dff0 con 0x56190b268e00 2024-04-24T06:40:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d97400640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b21f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000dbb0 tx=0x7f5d9000d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586edee74c1e46ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.272+0000 7f5d87400640 1 -- 172.21.15.59:0/3702036287 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000bc10 con 0x56190b268e00 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5d87400640 1 -- 172.21.15.59:0/3702036287 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9001c070 con 0x56190b268e00 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5d87400640 1 -- 172.21.15.59:0/3702036287 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90017400 con 0x56190b268e00 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 -- 172.21.15.59:0/3702036287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 msgr2=0x56190b24e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000dbb0 tx=0x7f5d9000d5a0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 -- 172.21.15.59:0/3702036287 shutdown_connections 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b24e710 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b21f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 --2- 172.21.15.59:0/3702036287 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b21efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 -- 172.21.15.59:0/3702036287 >> 172.21.15.59:0/3702036287 conn(0x56190b24b7e0 msgr2=0x56190b25ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 -- 172.21.15.59:0/3702036287 shutdown_connections 2024-04-24T06:40:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.273+0000 7f5da19d6740 1 -- 172.21.15.59:0/3702036287 wait complete. 2024-04-24T06:40:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.274+0000 7f5da19d6740 1 -- start start 2024-04-24T06:40:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.274+0000 7f5da19d6740 1 Processor -- start 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b24e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b245b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56190b246da0 con 0x56190b268e00 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981fb460 con 0x56190b24bf20 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x56190b24b410 con 0x56190b246f20 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5da19d6740 10 client.0 _ll_register_callbacks cb 0x7f5d800071f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.275+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48460/0 (socket says 172.21.15.59:48460) 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 -- 172.21.15.59:0/272949998 learned_addr learned my addr 172.21.15.59:0/272949998 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d97e00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b24e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d97400640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b245b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 -- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 msgr2=0x56190b24e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b24e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 -- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 msgr2=0x56190b245b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b245b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 -- 172.21.15.59:0/272949998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190ae973f0 con 0x56190b268e00 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d9cc00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b5d0 tx=0x7f5d9000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1746fa880cb85cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.276+0000 7f5d84c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:40:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90005b90 con 0x56190b268e00 2024-04-24T06:40:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7f5d90005d30 con 0x56190b268e00 2024-04-24T06:40:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9001c070 con 0x56190b268e00 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5d90030050 con 0x56190b268e00 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 10 client.9042 fetch_fsmap learned FSMap version 89 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x56190ae570f0 con 0x56190b268e00 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9001c070 con 0x56190b268e00 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f5d90033020 con 0x56190b268e00 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 10 client.9042 fetch_fsmap finished waiting for FSMap version 89 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 10 client.9042 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 20 client.9042 populate_metadata read hostname 'smithi059' 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b2e2d60 0x56190b2447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 4 client.9042 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.277+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 --> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56190b18ca80 con 0x56190b2e2d60 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.278+0000 7f5d97e00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b2e2d60 0x56190b2447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.278+0000 7f5d97e00640 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b2e2d60 0x56190b2447f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c992435e1cde1a9 server_cookie=89fd905d242daaf8 in_seq=0 out_seq=0 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.278+0000 7f5d84c00640 10 client.9042 ms_handle_connect on v2:172.21.15.59:6835/1384442467 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.278+0000 7f5d84c00640 1 -- 172.21.15.59:0/272949998 <== mds.0 v2:172.21.15.59:6835/1384442467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56190b18ca80 con 0x56190b2e2d60 2024-04-24T06:40:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.278+0000 7f5d84c00640 10 client.9042 handle_command_reply: tid=0 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 client.9042 shutdown 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 msgr2=0x56190b2442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b5d0 tx=0x7f5d9000f040 comp rx=0 tx=0).stop 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 shutdown_connections 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:6835/1384442467,v1:172.21.15.59:6837/1384442467] conn(0x56190b2e2d60 0x56190b2447f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56190b268e00 0x56190b2442b0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56190b24bf20 0x56190b245b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 --2- 172.21.15.59:0/272949998 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x56190b246f20 0x56190b24e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5d84c00640 0 client.9042 ms_handle_reset on v2:172.21.15.59:6835/1384442467 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 >> 172.21.15.59:0/272949998 conn(0x56190b24b7e0 msgr2=0x56190b265930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 shutdown_connections 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 1 -- 172.21.15.59:0/272949998 wait complete. 2024-04-24T06:40:27.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.279+0000 7f5da19d6740 20 client.9042 trim_cache size 0 max 16384 2024-04-24T06:40:27.283 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:40:27.283 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-24T06:40:27.283 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-24T06:40:27.283 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 msgr2=0x7f5d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 0x7f5d7406edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000f60 tx=0x7f5d88007ce0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 msgr2=0x7f5d98152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5d80034040 tx=0x7f5d80004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 shutdown_connections 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d7406c8f0 0x7f5d7406edb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d9815c410 0x7f5d981442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d980d0ce0 0x7f5d98152a30 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 --2- 172.21.15.59:0/298964679 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5d980af5c0 0x7f5d9815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 >> 172.21.15.59:0/298964679 conn(0x7f5d9806bdf0 msgr2=0x7f5d980b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 shutdown_connections 2024-04-24T06:40:27.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.280+0000 7f5d9d600640 1 -- 172.21.15.59:0/298964679 wait complete. 2024-04-24T06:40:27.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:40:27.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:40:27.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.612+0000 7fcb3d800640 1 Processor -- start 2024-04-24T06:40:27.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.612+0000 7fcb3d800640 1 -- start start 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb380cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb3d800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb381676a0 con 0x7fcb380d0830 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb3d800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb38166b20 con 0x7fcb380d0c10 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb3d800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb38166e00 con 0x7fcb380af720 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb37e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcb380d0830 0x7fcb38163bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50704/0 (socket says 172.21.15.59:50704) 2024-04-24T06:40:27.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.614+0000 7fcb37e00640 1 -- 172.21.15.59:0/1439253866 learned_addr learned my addr 172.21.15.59:0/1439253866 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:27.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb3ce00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb380cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361179728 0 0) 0x7fcb381676a0 con 0x7fcb380d0830 2024-04-24T06:40:27.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c003520 con 0x7fcb380d0830 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1158473252 0 0) 0x7fcb1c003520 con 0x7fcb380d0830 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 msgr2=0x7fcb380cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb380cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 >> v1:172.21.15.66:6789/0 conn(0x7fcb380d0c10 legacy=0x7fcb380af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381670e0 con 0x7fcb380d0830 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fcb28001670 con 0x7fcb380d0830 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 >> v1:172.21.15.59:6789/0 conn(0x7fcb380d0830 legacy=0x7fcb38163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb1c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38166e00 con 0x7fcb1c004800 2024-04-24T06:40:27.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c00bb20 con 0x7fcb1c0039e0 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.615+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c00bd20 con 0x7fcb380af720 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37e00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb3ce00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb1c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 -- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 msgr2=0x7fcb1c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb1c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 -- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c0039e0 msgr2=0x7fcb1c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c0039e0 0x7fcb1c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb37400640 1 -- 172.21.15.59:0/1439253866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c003520 con 0x7fcb1c004800 2024-04-24T06:40:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.616+0000 7fcb3ce00640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb380af720 0x7fcb1c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.617+0000 7fcb37400640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcb24001720 tx=0x7fcb1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389babeb05f9b04b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.617+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24001370 con 0x7fcb1c004800 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.617+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240052d0 con 0x7fcb1c004800 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.617+0000 7fcb36a00640 1 -- 172.21.15.59:0/1439253866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2402abc0 con 0x7fcb1c004800 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 -- 172.21.15.59:0/1439253866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 msgr2=0x7fcb1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1439253866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb1c004c80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcb24001720 tx=0x7fcb1c005830 comp rx=0 tx=0).stop 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3ce00640 1 -- 172.21.15.59:0/1439253866 reap_dead start 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 -- 172.21.15.59:0/1439253866 shutdown_connections 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 -- 172.21.15.59:0/1439253866 >> 172.21.15.59:0/1439253866 conn(0x7fcb3806bdf0 msgr2=0x7fcb3806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 -- 172.21.15.59:0/1439253866 shutdown_connections 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.618+0000 7fcb3d800640 1 -- 172.21.15.59:0/1439253866 wait complete. 2024-04-24T06:40:27.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.620+0000 7fcb3d800640 1 Processor -- start 2024-04-24T06:40:27.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.620+0000 7fcb3d800640 1 -- start start 2024-04-24T06:40:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb3815e650 0x7fcb3815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 0x7fcb380ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38166740 con 0x7fcb1c004800 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380b45f0 con 0x7fcb380ae040 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb3d800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb381668c0 con 0x7fcb3815e650 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48482/0 (socket says 172.21.15.59:48482) 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.621+0000 7fcb37e00640 1 -- 172.21.15.59:0/1777512244 learned_addr learned my addr 172.21.15.59:0/1777512244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37400640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb3815e650 0x7fcb3815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb3ce00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 0x7fcb380ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 -- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb3815e650 msgr2=0x7fcb3815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb3815e650 0x7fcb3815ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 -- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 msgr2=0x7fcb380ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 0x7fcb380ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 -- 172.21.15.59:0/1777512244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381588e0 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb3ce00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 0x7fcb380ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.622+0000 7fcb37e00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fcb28001050 tx=0x7fcb2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.623+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800c870 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.623+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb28018070 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.623+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb38158bc0 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.623+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800cc50 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.623+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb38159180 con 0x7fcb1c004800 2024-04-24T06:40:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.624+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb380b2e00 con 0x7fcb1c004800 2024-04-24T06:40:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.625+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb2801d460 con 0x7fcb1c004800 2024-04-24T06:40:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.626+0000 7fcb2fe00640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 0x7fcb1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.626+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5795+0+0 (secure 0 0 0) 0x7fcb2808baf0 con 0x7fcb1c004800 2024-04-24T06:40:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.626+0000 7fcb37400640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 0x7fcb1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.627+0000 7fcb37400640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 0x7fcb1806edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb24032000 tx=0x7fcb2402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:27.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.636+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb380b2e00 con 0x7fcb1c004800 2024-04-24T06:40:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.885+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb38159910 con 0x7fcb1c004800 2024-04-24T06:40:27.887 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:40:27.887+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:40:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.890+0000 7fcb2fe00640 1 -- 172.21.15.59:0/1777512244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7fcb28059e20 con 0x7fcb1c004800 2024-04-24T06:40:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:40:27.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 msgr2=0x7fcb1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 0x7fcb1806edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb24032000 tx=0x7fcb2402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:27.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 msgr2=0x7fcb38157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fcb28001050 tx=0x7fcb2800ec80 comp rx=0 tx=0).stop 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 shutdown_connections 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb1806c8f0 0x7fcb1806edb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb380ae040 0x7fcb380ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb3815e650 0x7fcb3815ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.895+0000 7fcb3d800640 1 --2- 172.21.15.59:0/1777512244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb1c004800 0x7fcb38157030 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.896+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 >> 172.21.15.59:0/1777512244 conn(0x7fcb3806bdf0 msgr2=0x7fcb3806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.896+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 shutdown_connections 2024-04-24T06:40:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:27.896+0000 7fcb3d800640 1 -- 172.21.15.59:0/1777512244 wait complete. 2024-04-24T06:40:27.907 INFO:tasks.ceph.mds.a.smithi059.stderr: -1366> 2024-04-24T06:39:55.123+0000 7f452e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:40:27.908 INFO:tasks.ceph.mds.a.smithi059.stderr: -1147> 2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-24T06:40:27.908 INFO:tasks.ceph.mds.a.smithi059.stderr: -1144> 2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-24T06:40:27.908 INFO:tasks.ceph.mds.a.smithi059.stderr: -1128> 2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-24T06:40:27.908 INFO:tasks.ceph.mds.a.smithi059.stderr: -1125> 2024-04-24T06:39:56.730+0000 7f4529600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-24T06:40:27.922 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-24T06:40:27.968 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:40:28.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.007+0000 7f16d10f7280 1 -- start start 2024-04-24T06:40:28.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.007+0000 7f16d10f7280 1 Processor -- start 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155b30400 0x55e155b18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155b31c00 0x55e155b18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e155aae000 con 0x55e155aa8000 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e155aae180 con 0x55e155b30400 2024-04-24T06:40:28.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e155aae300 con 0x55e155b31c00 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16cfc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16cfc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48514/0 (socket says 172.21.15.59:48514) 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16cfc00640 1 -- 172.21.15.59:0/1110971127 learned_addr learned my addr 172.21.15.59:0/1110971127 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16cf200640 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155b30400 0x55e155b18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.008+0000 7f16ce800640 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155b31c00 0x55e155b18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 -- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155b31c00 msgr2=0x55e155b18000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155b31c00 0x55e155b18000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 -- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155b30400 msgr2=0x55e155b18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155b30400 0x55e155b18b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 -- 172.21.15.59:0/1110971127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e154d78780 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cfc00640 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x55e155b82ff0 tx=0x55e155ae0120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffb61c401b349c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cc000640 1 -- 172.21.15.59:0/1110971127 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6ad00 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cc000640 1 -- 172.21.15.59:0/1110971127 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e154d70700 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16cc000640 1 -- 172.21.15.59:0/1110971127 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6a9c0 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e154d78000 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e154d79680 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.009+0000 7f16d10f7280 4 waiting for MDS map... 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.010+0000 7f16cc000640 1 -- 172.21.15.59:0/1110971127 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x55e155b56600 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.010+0000 7f16cc000640 1 -- 172.21.15.59:0/1110971127 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 90) ==== 2885+0+0 (secure 0 0 0) 0x55e154d9f180 con 0x55e155aa8000 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.010+0000 7f16d10f7280 4 Got MDS map 90 2024-04-24T06:40:28.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.010+0000 7f16d10f7280 10 main: main 2024-04-24T06:40:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.010+0000 7f16d10f7280 4 main: connecting to RADOS... 2024-04-24T06:40:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.011+0000 7f16d10f7280 1 Processor -- start 2024-04-24T06:40:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.011+0000 7f16d10f7280 1 -- start start 2024-04-24T06:40:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e155aae480 con 0x55e155aa9000 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e155aae600 con 0x55e155aa8800 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e155aae780 con 0x55e155aa8c00 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48516/0 (socket says 172.21.15.59:48516) 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 -- 172.21.15.59:0/1739807410 learned_addr learned my addr 172.21.15.59:0/1739807410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16cf200640 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 -- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8c00 msgr2=0x55e155b19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 -- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa8800 msgr2=0x55e155b19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.012+0000 7f16ce800640 1 -- 172.21.15.59:0/1739807410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e154d79860 con 0x55e155aa9000 2024-04-24T06:40:28.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.013+0000 7f16ce800640 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x55e155c7c060 tx=0x55e155ade720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3fd8388a1c29c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.013+0000 7f16c9800640 1 -- 172.21.15.59:0/1739807410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6ab60 con 0x55e155aa9000 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.013+0000 7f16c9800640 1 -- 172.21.15.59:0/1739807410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e154d70e00 con 0x55e155aa9000 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.013+0000 7f16c9800640 1 -- 172.21.15.59:0/1739807410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6a820 con 0x55e155aa9000 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 -- 172.21.15.59:0/1739807410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 msgr2=0x55e155b1a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x55e155c7c060 tx=0x55e155ade720 comp rx=0 tx=0).stop 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 -- 172.21.15.59:0/1739807410 shutdown_connections 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1739807410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 -- 172.21.15.59:0/1739807410 >> 172.21.15.59:0/1739807410 conn(0x55e155aa8400 msgr2=0x55e154e4d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 -- 172.21.15.59:0/1739807410 shutdown_connections 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 -- 172.21.15.59:0/1739807410 wait complete. 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.014+0000 7f16d10f7280 1 Processor -- start 2024-04-24T06:40:28.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 -- start start 2024-04-24T06:40:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e155aafc80 con 0x55e155aa8c00 2024-04-24T06:40:28.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e155aafb00 con 0x55e155aa9000 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.015+0000 7f16d10f7280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55e155aaf980 con 0x55e155aa8800 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 0x55e155b19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48520/0 (socket says 172.21.15.59:48520) 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 -- 172.21.15.59:0/3891948321 learned_addr learned my addr 172.21.15.59:0/3891948321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 -- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8800 msgr2=0x55e155b19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 -- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa9000 msgr2=0x55e155b1a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155aa9000 0x55e155b1a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16cf200640 1 -- 172.21.15.59:0/3891948321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e154d794a0 con 0x55e155aa8c00 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.016+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155aa8800 0x55e155b19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:28.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16cf200640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 0x55e155b19b80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x55e155c7d410 tx=0x55e155c28c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16c7a00640 1 -- 172.21.15.59:0/3891948321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6ab60 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16c7a00640 1 -- 172.21.15.59:0/3891948321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e154d70c40 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16c7a00640 1 -- 172.21.15.59:0/3891948321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e154d6a820 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e154d78960 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e154d783c0 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.017+0000 7f16d10f7280 4 main: resolving pool 14 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16c7a00640 1 -- 172.21.15.59:0/3891948321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55e155c16c00 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16c7a00640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e155c8ac00 0x55e155b1a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16c7a00640 1 -- 172.21.15.59:0/3891948321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x55e155c86800 con 0x55e155aa8c00 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16d10f7280 4 main: creating IoCtx.. 2024-04-24T06:40:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e155c8ac00 0x55e155b1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.019+0000 7f16d10f7280 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e155c8b000 0x55e155b1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.020+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55e155c8b400 con 0x55e155c8b000 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.020+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e155c8b000 0x55e155b1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.020+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e155c8ac00 0x55e155b1a680 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55e155c7c270 tx=0x55e155ade780 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.021+0000 7f16ce800640 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e155c8b000 0x55e155b1ac00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.021+0000 7f16ce800640 1 -- 172.21.15.59:0/3891948321 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55e155ace000 con 0x55e155c8b000 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "0": { 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "data": { 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 5, 2024-04-24T06:40:28.024 INFO:teuthology.orchestra.run.smithi059.stdout: "inotable": { 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "projected_free": [ 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "start": 1099511627779, 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "len": 1099511627773 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "free": [ 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "start": 1099511627779, 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: "len": 1099511627773 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:28.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: "result": 0 2024-04-24T06:40:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:28.026 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:40:28.026 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.021+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e155c8b000 msgr2=0x55e155b1ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.021+0000 7f16d10f7280 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55e155c8b000 0x55e155b1ac00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e155c8ac00 msgr2=0x55e155b1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55e155c8ac00 0x55e155b1a680 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55e155c7c270 tx=0x55e155ade780 comp rx=0 tx=0).stop 2024-04-24T06:40:28.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 msgr2=0x55e155b19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/3891948321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8c00 0x55e155b19b80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x55e155c7d410 tx=0x55e155c28c00 comp rx=0 tx=0).stop 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16cf200640 1 -- 172.21.15.59:0/3891948321 reap_dead start 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 shutdown_connections 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 >> 172.21.15.59:0/3891948321 conn(0x55e155aa8400 msgr2=0x55e154e4d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 shutdown_connections 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/3891948321 wait complete. 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 msgr2=0x55e155b19080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x55e155b82ff0 tx=0x55e155ae0120 comp rx=0 tx=0).stop 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 shutdown_connections 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e155aa8000 0x55e155b19080 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55e155b31c00 0x55e155b18000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 --2- 172.21.15.59:0/1110971127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e155b30400 0x55e155b18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.022+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 >> 172.21.15.59:0/1110971127 conn(0x55e154d45000 msgr2=0x55e154e4c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:28.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.023+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 shutdown_connections 2024-04-24T06:40:28.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.023+0000 7f16d10f7280 1 -- 172.21.15.59:0/1110971127 wait complete. 2024-04-24T06:40:28.030 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.107626, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-24T06:40:28.030 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:40:28.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:40:28.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:40:28.240 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:40:28.240 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:40:28.240 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:40:28.241 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:28.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:40:28.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:40:28.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:28.301 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:40:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-24T06:40:28.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:40:28.361 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:40:28.361 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:40:28.361 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:28.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:40:28.422 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:40:28.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:28.422 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:40:28.478 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:40:28.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:40:28.480 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:40:28.480 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:40:28.480 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:28.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:40:28.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:40:28.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:28.542 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:40:28.598 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:40:28.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:40:28.599 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:40:28.600 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:40:28.600 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:28.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:40:28.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:40:28.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:28.663 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:40:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:40:28.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:40:28.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:28.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.942+0000 7fc0f3200640 1 Processor -- start 2024-04-24T06:40:28.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.943+0000 7fc0f3200640 1 -- start start 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0f40cd050 0x7fc0f406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f415f160 con 0x7fc0f415ece0 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f415f430 con 0x7fc0f40b3080 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f40ced20 con 0x7fc0f40cd050 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0f40cd050 0x7fc0f406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f1400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc0f415ece0 0x7fc0f40cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56152/0 (socket says 172.21.15.59:56152) 2024-04-24T06:40:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.944+0000 7fc0f1400640 1 -- 172.21.15.59:0/716417586 learned_addr learned my addr 172.21.15.59:0/716417586 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:28.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.945+0000 7fc0f1e00640 1 -- 172.21.15.59:0/716417586 >> v1:172.21.15.66:6789/0 conn(0x7fc0f40b3080 legacy=0x7fc0f40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:28.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.945+0000 7fc0f1e00640 1 -- 172.21.15.59:0/716417586 >> v1:172.21.15.59:6789/0 conn(0x7fc0f415ece0 legacy=0x7fc0f40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.945+0000 7fc0f1e00640 1 -- 172.21.15.59:0/716417586 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f415f710 con 0x7fc0f40cd050 2024-04-24T06:40:28.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.946+0000 7fc0f1e00640 1 --2- 172.21.15.59:0/716417586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40cd050 0x7fc0f406f620 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40097a0 tx=0x7fc0e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09d6800f9bc0a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.946+0000 7fc0f0a00640 1 -- 172.21.15.59:0/716417586 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4037070 con 0x7fc0f40cd050 2024-04-24T06:40:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.946+0000 7fc0f0a00640 1 -- 172.21.15.59:0/716417586 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e402ae50 con 0x7fc0f40cd050 2024-04-24T06:40:28.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.946+0000 7fc0f0a00640 1 -- 172.21.15.59:0/716417586 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4032660 con 0x7fc0f40cd050 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 -- 172.21.15.59:0/716417586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40cd050 msgr2=0x7fc0f406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 --2- 172.21.15.59:0/716417586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40cd050 0x7fc0f406f620 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40097a0 tx=0x7fc0e402a140 comp rx=0 tx=0).stop 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 -- 172.21.15.59:0/716417586 shutdown_connections 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 --2- 172.21.15.59:0/716417586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40cd050 0x7fc0f406f620 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 -- 172.21.15.59:0/716417586 >> 172.21.15.59:0/716417586 conn(0x7fc0f40b21a0 msgr2=0x7fc0f40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 -- 172.21.15.59:0/716417586 shutdown_connections 2024-04-24T06:40:28.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.947+0000 7fc0f3200640 1 -- 172.21.15.59:0/716417586 wait complete. 2024-04-24T06:40:28.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.949+0000 7fc0f3200640 1 Processor -- start 2024-04-24T06:40:28.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.949+0000 7fc0f3200640 1 -- start start 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0f40cd050 0x7fc0f415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0f415ece0 0x7fc0f409d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f40cef50 con 0x7fc0f40b3080 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f415fcd0 con 0x7fc0f415ece0 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f4159110 con 0x7fc0f40cd050 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55286/0 (socket says 172.21.15.59:55286) 2024-04-24T06:40:28.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.950+0000 7fc0f1e00640 1 -- 172.21.15.59:0/3449890976 learned_addr learned my addr 172.21.15.59:0/3449890976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:28.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 -- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0f40cd050 msgr2=0x7fc0f415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0f40cd050 0x7fc0f415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 -- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0f415ece0 msgr2=0x7fc0f409d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0f415ece0 0x7fc0f409d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 -- 172.21.15.59:0/3449890976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f40c45e0 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0f1e00640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc0e402a180 tx=0x7fc0e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.951+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4042070 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.952+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e403e670 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.952+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f40c48c0 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.952+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4037040 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.952+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f415c690 con 0x7fc0f40b3080 2024-04-24T06:40:28.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.953+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f404fd90 con 0x7fc0f40b3080 2024-04-24T06:40:28.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.954+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0e4045070 con 0x7fc0f40b3080 2024-04-24T06:40:28.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.955+0000 7fc0ea000640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 0x7fc0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:28.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.955+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7fc0e40ac470 con 0x7fc0f40b3080 2024-04-24T06:40:28.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.955+0000 7fc0f1400640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 0x7fc0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:28.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.956+0000 7fc0f1400640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 0x7fc0d006edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc000f60 tx=0x7fc0dc007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:28.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:28.960+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0e407a900 con 0x7fc0f40b3080 2024-04-24T06:40:29.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.207+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0f40aef60 con 0x7fc0f40b3080 2024-04-24T06:40:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.209+0000 7fc0ea000640 1 -- 172.21.15.59:0/3449890976 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5164 (secure 0 0 0) 0x7fc0e407a200 con 0x7fc0f40b3080 2024-04-24T06:40:29.210 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:29.210 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/271137136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":271137136},{"type":"v1","addr":"172.21.15.59:6837","nonce":271137136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:40:27.887210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:40:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 91 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 msgr2=0x7fc0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 0x7fc0d006edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc000f60 tx=0x7fc0dc007a80 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 msgr2=0x7fc0f40c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc0e402a180 tx=0x7fc0e40046c0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 shutdown_connections 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc0d006c8f0 0x7fc0d006edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0f415ece0 0x7fc0f409d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc0f40cd050 0x7fc0f415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 --2- 172.21.15.59:0/3449890976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0f40b3080 0x7fc0f40c3f00 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 >> 172.21.15.59:0/3449890976 conn(0x7fc0f40b21a0 msgr2=0x7fc0f415c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 shutdown_connections 2024-04-24T06:40:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.212+0000 7fc0f3200640 1 -- 172.21.15.59:0/3449890976 wait complete. 2024-04-24T06:40:29.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.438+0000 7f9f4be00640 1 Processor -- start 2024-04-24T06:40:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.438+0000 7f9f4be00640 1 -- start start 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.439+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 0x7f9f4c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.439+0000 7f9f4be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.439+0000 7f9f4be00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c160210 con 0x7f9f4c15c3d0 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.439+0000 7f9f4be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4c1604f0 con 0x7f9f4c0b4690 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.439+0000 7f9f4be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c1607d0 con 0x7f9f4c0cd760 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55302/0 (socket says 172.21.15.59:55302) 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 -- 172.21.15.59:0/875865547 learned_addr learned my addr 172.21.15.59:0/875865547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:29.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4a000640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 0x7f9f4c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 -- 172.21.15.59:0/875865547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 msgr2=0x7f9f4c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 0x7f9f4c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 -- 172.21.15.59:0/875865547 >> v1:172.21.15.66:6789/0 conn(0x7f9f4c15c3d0 legacy=0x7f9f4c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4b400640 1 -- 172.21.15.59:0/875865547 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c160ab0 con 0x7f9f4c0b4690 2024-04-24T06:40:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.440+0000 7f9f4a000640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 0x7f9f4c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.441+0000 7f9f4b400640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009e50 tx=0x7f9f4402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ddc12573f3754e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f49600640 1 -- 172.21.15.59:0/875865547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44036070 con 0x7f9f4c0b4690 2024-04-24T06:40:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f49600640 1 -- 172.21.15.59:0/875865547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f44031600 con 0x7f9f4c0b4690 2024-04-24T06:40:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f49600640 1 -- 172.21.15.59:0/875865547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440319d0 con 0x7f9f4c0b4690 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 -- 172.21.15.59:0/875865547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0b4690 msgr2=0x7f9f4c0cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009e50 tx=0x7f9f4402a6c0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 -- 172.21.15.59:0/875865547 shutdown_connections 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c0cd760 0x7f9f4c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 --2- 172.21.15.59:0/875865547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cb7d0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 -- 172.21.15.59:0/875865547 >> 172.21.15.59:0/875865547 conn(0x7f9f4c0b21a0 msgr2=0x7f9f4c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 -- 172.21.15.59:0/875865547 shutdown_connections 2024-04-24T06:40:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.442+0000 7f9f4be00640 1 -- 172.21.15.59:0/875865547 wait complete. 2024-04-24T06:40:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.444+0000 7f9f4be00640 1 Processor -- start 2024-04-24T06:40:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.444+0000 7f9f4be00640 1 -- start start 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c15c3d0 0x7f9f4c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c15c7b0 con 0x7f9f4c0cd760 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c0b4ac0 con 0x7f9f4c0b4690 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c15c930 con 0x7f9f4c15c3d0 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.445+0000 7f9f4a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55316/0 (socket says 172.21.15.59:55316) 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 -- 172.21.15.59:0/182232664 learned_addr learned my addr 172.21.15.59:0/182232664 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4b400640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c15c3d0 0x7f9f4c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 -- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c15c3d0 msgr2=0x7f9f4c158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4aa00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c15c3d0 0x7f9f4c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 -- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f4c0b4690 msgr2=0x7f9f4c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 -- 172.21.15.59:0/182232664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c15ac00 con 0x7f9f4c0cd760 2024-04-24T06:40:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f4a000640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c7f0 tx=0x7f9f3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.446+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34015070 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.447+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f34007a30 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.447+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34007e00 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.447+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c0c91c0 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.447+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c15ae80 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.448+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c04fd90 con 0x7f9f4c0cd760 2024-04-24T06:40:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.449+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f34018070 con 0x7f9f4c0cd760 2024-04-24T06:40:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.450+0000 7f9f42a00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.450+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f9f3408bc10 con 0x7f9f4c0cd760 2024-04-24T06:40:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.450+0000 7f9f4aa00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.451+0000 7f9f4aa00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 0x7f9f2806edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00a750 tx=0x7f9f3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.454+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f4c15ac00 con 0x7f9f4c0cd760 2024-04-24T06:40:29.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.664+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f4c06e400 con 0x7f9f4c0cd760 2024-04-24T06:40:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.666+0000 7f9f42a00640 1 -- 172.21.15.59:0/182232664 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16782 (secure 0 0 0) 0x7f9f34059ef0 con 0x7f9f4c0cd760 2024-04-24T06:40:29.667 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:29.668 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:27.886768+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.59:6835/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.66:6834/891418539":"2024-04-25T06:39:50.448272+0000","172.21.15.66:6832/891418539":"2024-04-25T06:39:50.448272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:29.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.669+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 msgr2=0x7f9f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.669+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 0x7f9f2806edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00a750 tx=0x7f9f3c008040 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.669+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 msgr2=0x7f9f4c15a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.669+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c7f0 tx=0x7f9f3400cbf0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.669+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 shutdown_connections 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9f2806c8f0 0x7f9f2806edb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9f4c15c3d0 0x7f9f4c158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4c0cd760 0x7f9f4c15a6c0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 --2- 172.21.15.59:0/182232664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f4c0b4690 0x7f9f4c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 >> 172.21.15.59:0/182232664 conn(0x7f9f4c0b21a0 msgr2=0x7f9f4c099550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 shutdown_connections 2024-04-24T06:40:29.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.670+0000 7f9f4be00640 1 -- 172.21.15.59:0/182232664 wait complete. 2024-04-24T06:40:29.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:38:36.464123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:38:37.616685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:29.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.906+0000 7f642cc00640 1 Processor -- start 2024-04-24T06:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.906+0000 7f642cc00640 1 -- start start 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f642cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f6428161c50 0x7f64280d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f642cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280d0de0 0x7f64280d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f642cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64280d22e0 con 0x7f64280cfc50 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f642cc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f64280d2ba0 con 0x7f6428161c50 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f642cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f64280d2550 con 0x7f64280d0de0 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f6426a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f64280cfc50 0x7f64280d3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56160/0 (socket says 172.21.15.59:56160) 2024-04-24T06:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.908+0000 7f6426a00640 1 -- 172.21.15.59:0/3102054726 learned_addr learned my addr 172.21.15.59:0/3102054726 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246574572 0 0) 0x7f64280d22e0 con 0x7f64280cfc50 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6427e00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280d0de0 0x7f64280d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6427400640 1 --2- 172.21.15.59:0/3102054726 >> v2:172.21.15.66:3300/0 conn(0x7f6428161c50 0x7f64280d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c003520 con 0x7f64280cfc50 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 434001359 0 0) 0x7f640c003520 con 0x7f64280cfc50 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280d0de0 msgr2=0x7f64280d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280d0de0 0x7f64280d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 >> v2:172.21.15.66:3300/0 conn(0x7f6428161c50 msgr2=0x7f64280d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 --2- 172.21.15.59:0/3102054726 >> v2:172.21.15.66:3300/0 conn(0x7f6428161c50 0x7f64280d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64280d2830 con 0x7f64280cfc50 2024-04-24T06:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f6414001670 con 0x7f64280cfc50 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 >> v1:172.21.15.59:6789/0 conn(0x7f64280cfc50 legacy=0x7f64280d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f640c0039b0 0x7f640c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f640c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f640c0051c0 0x7f640c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d2ba0 con 0x7f640c004800 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d2550 con 0x7f640c0051c0 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f640c00c060 con 0x7f640c0039b0 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.909+0000 7f6426a00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f640c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.910+0000 7f6426a00640 1 -- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f640c0039b0 msgr2=0x7f640c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.910+0000 7f6426a00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f640c0039b0 0x7f640c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.910+0000 7f6426a00640 1 -- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f640c0051c0 msgr2=0x7f640c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.910+0000 7f6426a00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f640c0051c0 0x7f640c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.910+0000 7f6426a00640 1 -- 172.21.15.59:0/3102054726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c003520 con 0x7f640c004800 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.911+0000 7f6426a00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f640c004c80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f641400c190 tx=0x7f640c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdfd1b901080f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.911+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414007c80 con 0x7f640c004800 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.911+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6414005ad0 con 0x7f640c004800 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.911+0000 7f6426000640 1 -- 172.21.15.59:0/3102054726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414007070 con 0x7f640c004800 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 -- 172.21.15.59:0/3102054726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 msgr2=0x7f640c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 --2- 172.21.15.59:0/3102054726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f640c004c80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f641400c190 tx=0x7f640c005830 comp rx=0 tx=0).stop 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f6427e00640 1 -- 172.21.15.59:0/3102054726 reap_dead start 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 -- 172.21.15.59:0/3102054726 shutdown_connections 2024-04-24T06:40:29.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 -- 172.21.15.59:0/3102054726 >> 172.21.15.59:0/3102054726 conn(0x7f642808bf90 msgr2=0x7f64280d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:29.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 -- 172.21.15.59:0/3102054726 shutdown_connections 2024-04-24T06:40:29.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.912+0000 7f642cc00640 1 -- 172.21.15.59:0/3102054726 wait complete. 2024-04-24T06:40:29.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.915+0000 7f642cc00640 1 Processor -- start 2024-04-24T06:40:29.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.915+0000 7f642cc00640 1 -- start start 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 0x7f64280cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280cc0e0 0x7f642814c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d0090 con 0x7f640c004800 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d0460 con 0x7f64280cb740 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f642cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f64280d0210 con 0x7f64280cc0e0 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f6427400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f6427400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55342/0 (socket says 172.21.15.59:55342) 2024-04-24T06:40:29.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.916+0000 7f6427400640 1 -- 172.21.15.59:0/2517330989 learned_addr learned my addr 172.21.15.59:0/2517330989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6426a00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 0x7f64280cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 -- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280cc0e0 msgr2=0x7f642814c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280cc0e0 0x7f642814c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 -- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 msgr2=0x7f64280cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 0x7f64280cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 -- 172.21.15.59:0/2517330989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64280cc560 con 0x7f640c004800 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6426a00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 0x7f64280cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f6427400640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f641800d830 tx=0x7f641800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641800b660 con 0x7f640c004800 2024-04-24T06:40:29.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6418019070 con 0x7f640c004800 2024-04-24T06:40:29.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641800ba40 con 0x7f640c004800 2024-04-24T06:40:29.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.917+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64280ba220 con 0x7f640c004800 2024-04-24T06:40:29.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.918+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642814ba60 con 0x7f640c004800 2024-04-24T06:40:29.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.919+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642804fd90 con 0x7f640c004800 2024-04-24T06:40:29.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.920+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f641801e460 con 0x7f640c004800 2024-04-24T06:40:29.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.921+0000 7f641f400640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 0x7f640806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:29.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.921+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f641808c2e0 con 0x7f640c004800 2024-04-24T06:40:29.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.921+0000 7f6426a00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 0x7f640806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:29.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.922+0000 7f6426a00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 0x7f640806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6414032000 tx=0x7f6414009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:29.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:29.926+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f641805a5c0 con 0x7f640c004800 2024-04-24T06:40:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.133+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64280cb3e0 con 0x7f640c004800 2024-04-24T06:40:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.134+0000 7f641f400640 1 -- 172.21.15.59:0/2517330989 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f6418059ec0 con 0x7f640c004800 2024-04-24T06:40:30.135 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.136+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 msgr2=0x7f640806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.136+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 0x7f640806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6414032000 tx=0x7f6414009030 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 msgr2=0x7f64281f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f641800d830 tx=0x7f641800dcf0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 shutdown_connections 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f640806c8f0 0x7f640806edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f64280cc0e0 0x7f642814c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64280cb740 0x7f64280cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 --2- 172.21.15.59:0/2517330989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f640c004800 0x7f64281f4520 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 >> 172.21.15.59:0/2517330989 conn(0x7f642808bf90 msgr2=0x7f6428161240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:30.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 shutdown_connections 2024-04-24T06:40:30.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.137+0000 7f642cc00640 1 -- 172.21.15.59:0/2517330989 wait complete. 2024-04-24T06:40:30.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:40:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.366+0000 7f1a6a400640 1 Processor -- start 2024-04-24T06:40:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.366+0000 7f1a6a400640 1 -- start start 2024-04-24T06:40:30.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.367+0000 7f1a6a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a64156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.367+0000 7f1a6a400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6415cab0 con 0x7f1a6415d810 2024-04-24T06:40:30.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.367+0000 7f1a6a400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a64160e40 con 0x7f1a641567e0 2024-04-24T06:40:30.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.367+0000 7f1a6a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6415cd30 con 0x7f1a640d0d10 2024-04-24T06:40:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.368+0000 7f1a69a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1a6415d810 0x7f1a64160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56168/0 (socket says 172.21.15.59:56168) 2024-04-24T06:40:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.368+0000 7f1a69a00640 1 -- 172.21.15.59:0/1624242685 learned_addr learned my addr 172.21.15.59:0/1624242685 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.368+0000 7f1a63e00640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a64156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.368+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373659924 0 0) 0x7f1a5c001490 con 0x7f1a6415d810 2024-04-24T06:40:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.368+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a4c003520 con 0x7f1a6415d810 2024-04-24T06:40:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3861892273 0 0) 0x7f1a4c003520 con 0x7f1a6415d810 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 msgr2=0x7f1a64156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a64156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 >> v1:172.21.15.66:6789/0 conn(0x7f1a641567e0 legacy=0x7f1a640d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6415d010 con 0x7f1a6415d810 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1a5c001bc0 con 0x7f1a6415d810 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 >> v1:172.21.15.59:6789/0 conn(0x7f1a6415d810 legacy=0x7f1a64160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a4c004800 0x7f1a4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6415cd30 con 0x7f1a4c0039e0 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c00baa0 con 0x7f1a4c004800 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c00bca0 con 0x7f1a640d0d10 2024-04-24T06:40:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a69000640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.369+0000 7f1a63e00640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a4c004800 0x7f1a4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69a00640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69000640 1 -- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 msgr2=0x7f1a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69000640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a640d0d10 0x7f1a4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69000640 1 -- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a4c004800 msgr2=0x7f1a4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69000640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a4c004800 0x7f1a4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.370+0000 7f1a69000640 1 -- 172.21.15.59:0/1624242685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c003520 con 0x7f1a4c0039e0 2024-04-24T06:40:30.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.371+0000 7f1a69000640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f1a58001720 tx=0x7f1a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6e351ac70a45c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.371+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58001370 con 0x7f1a4c0039e0 2024-04-24T06:40:30.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.371+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a580052d0 con 0x7f1a4c0039e0 2024-04-24T06:40:30.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.371+0000 7f1a63400640 1 -- 172.21.15.59:0/1624242685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5802ab90 con 0x7f1a4c0039e0 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 -- 172.21.15.59:0/1624242685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 msgr2=0x7f1a4c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 --2- 172.21.15.59:0/1624242685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f1a58001720 tx=0x7f1a4c005830 comp rx=0 tx=0).stop 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a69a00640 1 -- 172.21.15.59:0/1624242685 reap_dead start 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 -- 172.21.15.59:0/1624242685 shutdown_connections 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 -- 172.21.15.59:0/1624242685 >> 172.21.15.59:0/1624242685 conn(0x7f1a6406bdf0 msgr2=0x7f1a6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 -- 172.21.15.59:0/1624242685 shutdown_connections 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.372+0000 7f1a6a400640 1 -- 172.21.15.59:0/1624242685 wait complete. 2024-04-24T06:40:30.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.374+0000 7f1a6a400640 1 Processor -- start 2024-04-24T06:40:30.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.374+0000 7f1a6a400640 1 -- start start 2024-04-24T06:40:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a64159080 0x7f1a641594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a6414b200 0x7f1a6414b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64156d40 con 0x7f1a4c0039e0 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64156bc0 con 0x7f1a6414b200 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a6a400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1a64156ec0 con 0x7f1a64159080 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.375+0000 7f1a69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55360/0 (socket says 172.21.15.59:55360) 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 -- 172.21.15.59:0/3841244661 learned_addr learned my addr 172.21.15.59:0/3841244661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a63e00640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a64159080 0x7f1a641594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69a00640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a6414b200 0x7f1a6414b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 -- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a64159080 msgr2=0x7f1a641594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a64159080 0x7f1a641594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 -- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a6414b200 msgr2=0x7f1a6414b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a6414b200 0x7f1a6414b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 -- 172.21.15.59:0/3841244661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6414bbc0 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.376+0000 7f1a69000640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.377+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58040020 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.377+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5802be40 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.377+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58001370 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.377+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a641e6360 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.377+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641e6920 con 0x7f1a4c0039e0 2024-04-24T06:40:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.378+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6404fd90 con 0x7f1a4c0039e0 2024-04-24T06:40:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.379+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a58043020 con 0x7f1a4c0039e0 2024-04-24T06:40:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.380+0000 7f1a60c00640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 0x7f1a4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.380+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f1a580abbd0 con 0x7f1a4c0039e0 2024-04-24T06:40:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.380+0000 7f1a63e00640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 0x7f1a4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.381+0000 7f1a63e00640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 0x7f1a4406ede0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1a6415bcd0 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.385+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a5804a030 con 0x7f1a4c0039e0 2024-04-24T06:40:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.588+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a640b5150 con 0x7f1a4c0039e0 2024-04-24T06:40:30.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.589+0000 7f1a60c00640 1 -- 172.21.15.59:0/3841244661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f1a6414bbc0 con 0x7f1a4c0039e0 2024-04-24T06:40:30.590 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:40:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 msgr2=0x7f1a4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 0x7f1a4406ede0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1a6415bcd0 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2024-04-24T06:40:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 msgr2=0x7f1a64154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58001120 comp rx=0 tx=0).stop 2024-04-24T06:40:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 shutdown_connections 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1a4406c920 0x7f1a4406ede0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a6414b200 0x7f1a6414b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1a64159080 0x7f1a641594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 --2- 172.21.15.59:0/3841244661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a4c0039e0 0x7f1a64154f80 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 >> 172.21.15.59:0/3841244661 conn(0x7f1a6406bdf0 msgr2=0x7f1a6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 shutdown_connections 2024-04-24T06:40:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.592+0000 7f1a6a400640 1 -- 172.21.15.59:0/3841244661 wait complete. 2024-04-24T06:40:30.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:40:30.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.824+0000 7f1d97200640 1 Processor -- start 2024-04-24T06:40:30.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.824+0000 7f1d97200640 1 -- start start 2024-04-24T06:40:30.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.826+0000 7f1d97200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f1d9015e790 0x7f1d900aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.826+0000 7f1d97200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d90162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.826+0000 7f1d97200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d90163370 con 0x7f1d90071d70 2024-04-24T06:40:30.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.826+0000 7f1d97200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f1d900aef40 con 0x7f1d9015e790 2024-04-24T06:40:30.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.826+0000 7f1d97200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1d900afd80 con 0x7f1d9006f6f0 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d95200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1d90071d70 0x7f1d900b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56184/0 (socket says 172.21.15.59:56184) 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d95200640 1 -- 172.21.15.59:0/4055362513 learned_addr learned my addr 172.21.15.59:0/4055362513 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/4055362513 >> v2:172.21.15.66:3300/0 conn(0x7f1d9015e790 0x7f1d900aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d87e00640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d90162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360870220 0 0) 0x7f1d88001490 con 0x7f1d90071d70 2024-04-24T06:40:30.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.827+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d74003520 con 0x7f1d90071d70 2024-04-24T06:40:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 589365879 0 0) 0x7f1d74003520 con 0x7f1d90071d70 2024-04-24T06:40:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 msgr2=0x7f1d90162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d90162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 >> v2:172.21.15.66:3300/0 conn(0x7f1d9015e790 msgr2=0x7f1d900aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 --2- 172.21.15.59:0/4055362513 >> v2:172.21.15.66:3300/0 conn(0x7f1d9015e790 0x7f1d900aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d900adc80 con 0x7f1d90071d70 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f1d88001bc0 con 0x7f1d90071d70 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 >> v1:172.21.15.59:6789/0 conn(0x7f1d90071d70 legacy=0x7f1d900b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d74003f80 0x7f1d74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74004d40 0x7f1d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900aef40 con 0x7f1d74004d40 2024-04-24T06:40:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7400b9c0 con 0x7f1d74003f80 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7400bb40 con 0x7f1d9006f6f0 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.828+0000 7f1d95200640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74004d40 0x7f1d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d87e00640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d74003f80 0x7f1d74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d95200640 1 -- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 msgr2=0x7f1d740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d95200640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d9006f6f0 0x7f1d740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d95200640 1 -- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d74003f80 msgr2=0x7f1d74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d95200640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d74003f80 0x7f1d74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d95200640 1 -- 172.21.15.59:0/4055362513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d74003520 con 0x7f1d74004d40 2024-04-24T06:40:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.829+0000 7f1d87e00640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d74003f80 0x7f1d74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.830+0000 7f1d95200640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74004d40 0x7f1d740051c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800a9d0 tx=0x7f1d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d14e12617573c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.830+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800a5c0 con 0x7f1d74004d40 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.830+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8802f070 con 0x7f1d74004d40 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.830+0000 7f1d8f400640 1 -- 172.21.15.59:0/4055362513 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8802be40 con 0x7f1d74004d40 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 -- 172.21.15.59:0/4055362513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74004d40 msgr2=0x7f1d740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 --2- 172.21.15.59:0/4055362513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74004d40 0x7f1d740051c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800a9d0 tx=0x7f1d74005830 comp rx=0 tx=0).stop 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d95200640 1 -- 172.21.15.59:0/4055362513 reap_dead start 2024-04-24T06:40:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 -- 172.21.15.59:0/4055362513 shutdown_connections 2024-04-24T06:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 -- 172.21.15.59:0/4055362513 >> 172.21.15.59:0/4055362513 conn(0x7f1d90067d90 msgr2=0x7f1d9006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 -- 172.21.15.59:0/4055362513 shutdown_connections 2024-04-24T06:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.831+0000 7f1d97200640 1 -- 172.21.15.59:0/4055362513 wait complete. 2024-04-24T06:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.833+0000 7f1d97200640 1 Processor -- start 2024-04-24T06:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.833+0000 7f1d97200640 1 -- start start 2024-04-24T06:40:30.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 0x7f1d9015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d901f4b00 0x7f1d90158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900ae4e0 con 0x7f1d90156690 2024-04-24T06:40:30.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900ae200 con 0x7f1d901f4b00 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.834+0000 7f1d97200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9006fbb0 con 0x7f1d74004d40 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55372/0 (socket says 172.21.15.59:55372) 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 -- 172.21.15.59:0/1973418922 learned_addr learned my addr 172.21.15.59:0/1973418922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 0x7f1d9015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d95200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d901f4b00 0x7f1d90158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 -- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 msgr2=0x7f1d9015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 0x7f1d9015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 -- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d901f4b00 msgr2=0x7f1d90158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d901f4b00 0x7f1d90158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:30.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 -- 172.21.15.59:0/1973418922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d90154b90 con 0x7f1d90156690 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 0x7f1d9015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.835+0000 7f1d87e00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f1d780108c0 tx=0x7f1d78010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.836+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800f040 con 0x7f1d90156690 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.836+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7800eb00 con 0x7f1d90156690 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.836+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7801c400 con 0x7f1d90156690 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.836+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d901548b0 con 0x7f1d90156690 2024-04-24T06:40:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.836+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d90151a70 con 0x7f1d90156690 2024-04-24T06:40:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.837+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d90068a80 con 0x7f1d90156690 2024-04-24T06:40:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.838+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d7801c5a0 con 0x7f1d90156690 2024-04-24T06:40:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.839+0000 7f1d8cc00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 0x7f1d6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.839+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f1d7808c140 con 0x7f1d90156690 2024-04-24T06:40:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.839+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 0x7f1d6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:30.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.840+0000 7f1d8fe00640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 0x7f1d6c06ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009a80 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:30.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:30.843+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d78018030 con 0x7f1d90156690 2024-04-24T06:40:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.054+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d90162190 con 0x7f1d90156690 2024-04-24T06:40:31.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.055+0000 7f1d8cc00640 1 -- 172.21.15.59:0/1973418922 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f1d78059dd0 con 0x7f1d90156690 2024-04-24T06:40:31.056 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:40:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 msgr2=0x7f1d6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 0x7f1d6c06ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009a80 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2024-04-24T06:40:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 msgr2=0x7f1d90153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f1d780108c0 tx=0x7f1d78010d80 comp rx=0 tx=0).stop 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 shutdown_connections 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f1d6c06c920 0x7f1d6c06ede0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d901f4b00 0x7f1d90158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d90156690 0x7f1d90153f80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 --2- 172.21.15.59:0/1973418922 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f1d74004d40 0x7f1d9015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 >> 172.21.15.59:0/1973418922 conn(0x7f1d90067d90 msgr2=0x7f1d900b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 shutdown_connections 2024-04-24T06:40:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.058+0000 7f1d97200640 1 -- 172.21.15.59:0/1973418922 wait complete. 2024-04-24T06:40:31.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:31.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.293+0000 7f4369a00640 1 Processor -- start 2024-04-24T06:40:31.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.293+0000 7f4369a00640 1 -- start start 2024-04-24T06:40:31.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4369a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f43640cff10 0x7f43640d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4369a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f43640d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4369a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436415fa20 con 0x7f43640d0ec0 2024-04-24T06:40:31.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4369a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f43640b3b60 con 0x7f43640cff10 2024-04-24T06:40:31.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4369a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f436415fd00 con 0x7f43640d0330 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4363400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f43640cff10 0x7f43640d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4363400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f43640cff10 0x7f43640d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55384/0 (socket says 172.21.15.59:55384) 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.294+0000 7f4363400640 1 -- 172.21.15.59:0/1298106331 learned_addr learned my addr 172.21.15.59:0/1298106331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4369000640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f43640d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4363400640 1 -- 172.21.15.59:0/1298106331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 msgr2=0x7f43640d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4363400640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f43640d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4363400640 1 -- 172.21.15.59:0/1298106331 >> v1:172.21.15.66:6789/0 conn(0x7f43640d0ec0 legacy=0x7f43640d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4363400640 1 -- 172.21.15.59:0/1298106331 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640cf430 con 0x7f43640cff10 2024-04-24T06:40:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.295+0000 7f4369000640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f43640d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:31.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.296+0000 7f4363400640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640d21a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f6bcd03bf1ae5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.296+0000 7f4362a00640 1 -- 172.21.15.59:0/1298106331 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350037070 con 0x7f43640cff10 2024-04-24T06:40:31.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.296+0000 7f4362a00640 1 -- 172.21.15.59:0/1298106331 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4350032440 con 0x7f43640cff10 2024-04-24T06:40:31.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.296+0000 7f4362a00640 1 -- 172.21.15.59:0/1298106331 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350032810 con 0x7f43640cff10 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 -- 172.21.15.59:0/1298106331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 msgr2=0x7f43640d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640d21a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002a2a0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 -- 172.21.15.59:0/1298106331 shutdown_connections 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f43640d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 --2- 172.21.15.59:0/1298106331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640d21a0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 -- 172.21.15.59:0/1298106331 >> 172.21.15.59:0/1298106331 conn(0x7f436408bf90 msgr2=0x7f43640d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 -- 172.21.15.59:0/1298106331 shutdown_connections 2024-04-24T06:40:31.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.297+0000 7f4369a00640 1 -- 172.21.15.59:0/1298106331 wait complete. 2024-04-24T06:40:31.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.299+0000 7f4369a00640 1 Processor -- start 2024-04-24T06:40:31.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.299+0000 7f4369a00640 1 -- start start 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f436414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43640d0ec0 0x7f436414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f436415eb10 con 0x7f43640cff10 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f43640cf9f0 con 0x7f43640d0ec0 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4369a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f436415ec90 con 0x7f43640d0330 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4363e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55392/0 (socket says 172.21.15.59:55392) 2024-04-24T06:40:31.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.300+0000 7f4363e00640 1 -- 172.21.15.59:0/3555799702 learned_addr learned my addr 172.21.15.59:0/3555799702 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:31.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4369000640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43640d0ec0 0x7f436414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363400640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f436414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 -- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 msgr2=0x7f436414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f436414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 -- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43640d0ec0 msgr2=0x7f436414ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43640d0ec0 0x7f436414ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 -- 172.21.15.59:0/3555799702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640c8fa0 con 0x7f43640cff10 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363400640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f436414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4363e00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f435400b420 tx=0x7f435400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354011830 con 0x7f43640cff10 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4354016070 con 0x7f43640cff10 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354011c10 con 0x7f43640cff10 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.301+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43640c9280 con 0x7f43640cff10 2024-04-24T06:40:31.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.302+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43640c9840 con 0x7f43640cff10 2024-04-24T06:40:31.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.303+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436404fd90 con 0x7f43640cff10 2024-04-24T06:40:31.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.304+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f435401b460 con 0x7f43640cff10 2024-04-24T06:40:31.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.305+0000 7f435be00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.305+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f435408b710 con 0x7f43640cff10 2024-04-24T06:40:31.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.305+0000 7f4363400640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.306+0000 7f4363400640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 0x7f434406edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4350034000 tx=0x7f435002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.310+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4354020020 con 0x7f43640cff10 2024-04-24T06:40:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.556+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43640d1f00 con 0x7f43640cff10 2024-04-24T06:40:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.557+0000 7f435be00640 1 -- 172.21.15.59:0/3555799702 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5164 (secure 0 0 0) 0x7f4354059a90 con 0x7f43640cff10 2024-04-24T06:40:31.558 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:31.558 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/271137136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":271137136},{"type":"v1","addr":"172.21.15.59:6837","nonce":271137136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:40:27.887210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:40:31.559 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 91 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 msgr2=0x7f434406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 0x7f434406edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4350034000 tx=0x7f435002e000 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 msgr2=0x7f43640ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f435400b420 tx=0x7f435400b8e0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 shutdown_connections 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43640d0ec0 0x7f436414ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f43640d0330 0x7f436414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 --2- 172.21.15.59:0/3555799702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43640cff10 0x7f43640ca3c0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 >> 172.21.15.59:0/3555799702 conn(0x7f436408bf90 msgr2=0x7f4364160780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:31.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 shutdown_connections 2024-04-24T06:40:31.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.560+0000 7f4369a00640 1 -- 172.21.15.59:0/3555799702 wait complete. 2024-04-24T06:40:31.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:31.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.785+0000 7fcba2200640 1 Processor -- start 2024-04-24T06:40:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.786+0000 7fcba2200640 1 -- start start 2024-04-24T06:40:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba2200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c15e7c0 0x7fcb9c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba2200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcb9c0d0d90 0x7fcb9c0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba2200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9c0cf030 con 0x7fcb9c0d2630 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba2200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9c0b7000 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba2200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cf620 con 0x7fcb9c15e7c0 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcb9be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcb9c0d2630 0x7fcb9c0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56196/0 (socket says 172.21.15.59:56196) 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcb9be00640 1 -- 172.21.15.59:0/924887745 learned_addr learned my addr 172.21.15.59:0/924887745 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.787+0000 7fcba1800640 1 --2- 172.21.15.59:0/924887745 >> v2:172.21.15.59:3300/0 conn(0x7fcb9c0d0d90 0x7fcb9c0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.788+0000 7fcba0e00640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c15e7c0 0x7fcb9c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.788+0000 7fcba1800640 1 -- 172.21.15.59:0/924887745 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c15e7c0 msgr2=0x7fcb9c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.788+0000 7fcba1800640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c15e7c0 0x7fcb9c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.788+0000 7fcba1800640 1 -- 172.21.15.59:0/924887745 >> v1:172.21.15.59:6789/0 conn(0x7fcb9c0d2630 legacy=0x7fcb9c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.788+0000 7fcba1800640 1 -- 172.21.15.59:0/924887745 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c0b6a40 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.789+0000 7fcba1800640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d0d90 0x7fcb9c0d1220 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009d90 tx=0x7fcb9402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a704c5ccb89b8b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.789+0000 7fcb9b400640 1 -- 172.21.15.59:0/924887745 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007740 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.789+0000 7fcb9b400640 1 -- 172.21.15.59:0/924887745 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb940078e0 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.789+0000 7fcb9b400640 1 -- 172.21.15.59:0/924887745 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007cb0 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 -- 172.21.15.59:0/924887745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d0d90 msgr2=0x7fcb9c0d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d0d90 0x7fcb9c0d1220 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009d90 tx=0x7fcb9402a660 comp rx=0 tx=0).stop 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 -- 172.21.15.59:0/924887745 shutdown_connections 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d0d90 0x7fcb9c0d1220 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 --2- 172.21.15.59:0/924887745 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c15e7c0 0x7fcb9c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 -- 172.21.15.59:0/924887745 >> 172.21.15.59:0/924887745 conn(0x7fcb9c09d920 msgr2=0x7fcb9c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 -- 172.21.15.59:0/924887745 shutdown_connections 2024-04-24T06:40:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.790+0000 7fcba2200640 1 -- 172.21.15.59:0/924887745 wait complete. 2024-04-24T06:40:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.792+0000 7fcba2200640 1 Processor -- start 2024-04-24T06:40:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.792+0000 7fcba2200640 1 -- start start 2024-04-24T06:40:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c0d0d90 0x7fcb9c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb9c15e7c0 0x7fcb9c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0b6700 con 0x7fcb9c0d2630 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cf2b0 con 0x7fcb9c15e7c0 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba2200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c0d1a80 con 0x7fcb9c0d0d90 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcb9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcb9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55416/0 (socket says 172.21.15.59:55416) 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcb9be00640 1 -- 172.21.15.59:0/2241440550 learned_addr learned my addr 172.21.15.59:0/2241440550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.793+0000 7fcba0e00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c0d0d90 0x7fcb9c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 -- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c0d0d90 msgr2=0x7fcb9c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c0d0d90 0x7fcb9c14f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 -- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb9c15e7c0 msgr2=0x7fcb9c0b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcba1800640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb9c15e7c0 0x7fcb9c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb9c15e7c0 0x7fcb9c0b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 -- 172.21.15.59:0/2241440550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1f4b40 con 0x7fcb9c0d2630 2024-04-24T06:40:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb9be00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c440 tx=0x7fcb8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c017070 con 0x7fcb9c0d2630 2024-04-24T06:40:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c005ce0 con 0x7fcb9c0d2630 2024-04-24T06:40:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0072f0 con 0x7fcb9c0d2630 2024-04-24T06:40:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1f4e20 con 0x7fcb9c0d2630 2024-04-24T06:40:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.794+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1f56e0 con 0x7fcb9c0d2630 2024-04-24T06:40:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.795+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c04fd90 con 0x7fcb9c0d2630 2024-04-24T06:40:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.797+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb8c01a070 con 0x7fcb9c0d2630 2024-04-24T06:40:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.798+0000 7fcb98c00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 0x7fcb7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.798+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7fcb8c08b090 con 0x7fcb9c0d2630 2024-04-24T06:40:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.798+0000 7fcba0e00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 0x7fcb7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.799+0000 7fcba0e00640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 0x7fcb7806edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006fd0 tx=0x7fcb84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:31.802+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb9c0cf030 con 0x7fcb9c0d2630 2024-04-24T06:40:32.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.008+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb9c0d1eb0 con 0x7fcb9c0d2630 2024-04-24T06:40:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.010+0000 7fcb98c00640 1 -- 172.21.15.59:0/2241440550 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16782 (secure 0 0 0) 0x7fcb8c05f070 con 0x7fcb9c0d2630 2024-04-24T06:40:32.011 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:32.011 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:27.886768+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.59:6835/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.66:6834/891418539":"2024-04-25T06:39:50.448272+0000","172.21.15.66:6832/891418539":"2024-04-25T06:39:50.448272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.012+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 msgr2=0x7fcb7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.012+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 0x7fcb7806edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006fd0 tx=0x7fcb84008040 comp rx=0 tx=0).stop 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 msgr2=0x7fcb9c1f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c440 tx=0x7fcb8c001e70 comp rx=0 tx=0).stop 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 shutdown_connections 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcb7806c8f0 0x7fcb7806edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb9c15e7c0 0x7fcb9c0b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb9c0d2630 0x7fcb9c1f40a0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 --2- 172.21.15.59:0/2241440550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcb9c0d0d90 0x7fcb9c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 >> 172.21.15.59:0/2241440550 conn(0x7fcb9c09d920 msgr2=0x7fcb9c15f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 shutdown_connections 2024-04-24T06:40:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.013+0000 7fcba2200640 1 -- 172.21.15.59:0/2241440550 wait complete. 2024-04-24T06:40:32.035 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:40:32.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.247+0000 7fb8ac800640 1 Processor -- start 2024-04-24T06:40:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.247+0000 7fb8ac800640 1 -- start start 2024-04-24T06:40:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8ac800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb8a40afaf0 0x7fb8a40d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8ac800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a415f0f0 con 0x7fb8a415c5e0 2024-04-24T06:40:32.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8ac800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a415f3d0 con 0x7fb8a40d06e0 2024-04-24T06:40:32.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8ac800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a415fa20 con 0x7fb8a40afaf0 2024-04-24T06:40:32.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8a9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb8a415c5e0 0x7fb8a40af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56198/0 (socket says 172.21.15.59:56198) 2024-04-24T06:40:32.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8a9e00640 1 -- 172.21.15.59:0/2182910530 learned_addr learned my addr 172.21.15.59:0/2182910530 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8a9400640 1 --2- 172.21.15.59:0/2182910530 >> v2:172.21.15.66:3300/0 conn(0x7fb8a40afaf0 0x7fb8a40d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.249+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604468753 0 0) 0x7fb8a415f0f0 con 0x7fb8a415c5e0 2024-04-24T06:40:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb88c003520 con 0x7fb8a415c5e0 2024-04-24T06:40:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 753092035 0 0) 0x7fb88c003520 con 0x7fb8a415c5e0 2024-04-24T06:40:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 >> v2:172.21.15.66:3300/0 conn(0x7fb8a40afaf0 msgr2=0x7fb8a40d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 --2- 172.21.15.59:0/2182910530 >> v2:172.21.15.66:3300/0 conn(0x7fb8a40afaf0 0x7fb8a40d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 >> v1:172.21.15.66:6789/0 conn(0x7fb8a40d06e0 legacy=0x7fb8a41624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a415f6b0 con 0x7fb8a415c5e0 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb898001670 con 0x7fb8a415c5e0 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 >> v1:172.21.15.59:6789/0 conn(0x7fb8a415c5e0 legacy=0x7fb8a40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb88c0039b0 0x7fb88c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb88c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb88c0051c0 0x7fb88c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a415fa20 con 0x7fb88c004800 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c00c060 con 0x7fb88c0039b0 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c00c260 con 0x7fb88c0051c0 2024-04-24T06:40:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.250+0000 7fb8a9400640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb88c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9e00640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb88c0039b0 0x7fb88c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8aa800640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb88c0051c0 0x7fb88c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9400640 1 -- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb88c0051c0 msgr2=0x7fb88c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9400640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb88c0051c0 0x7fb88c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9400640 1 -- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb88c0039b0 msgr2=0x7fb88c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9400640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb88c0039b0 0x7fb88c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8a9400640 1 -- 172.21.15.59:0/2182910530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c003520 con 0x7fb88c004800 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8aa800640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb88c0051c0 0x7fb88c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.251+0000 7fb8aa800640 1 -- 172.21.15.59:0/2182910530 reap_dead start 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.252+0000 7fb8a9400640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb88c004c80 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb894009ac0 tx=0x7fb88c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0a1201f620ed1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894036070 con 0x7fb88c004800 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894031400 con 0x7fb88c004800 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8a8a00640 1 -- 172.21.15.59:0/2182910530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894009690 con 0x7fb88c004800 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 -- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 msgr2=0x7fb88c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb88c004c80 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb894009ac0 tx=0x7fb88c005830 comp rx=0 tx=0).stop 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 -- 172.21.15.59:0/2182910530 shutdown_connections 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 --2- 172.21.15.59:0/2182910530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb88c004c80 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 -- 172.21.15.59:0/2182910530 >> 172.21.15.59:0/2182910530 conn(0x7fb8a4067ea0 msgr2=0x7fb8a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 -- 172.21.15.59:0/2182910530 shutdown_connections 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.253+0000 7fb8ac800640 1 -- 172.21.15.59:0/2182910530 wait complete. 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.255+0000 7fb8ac800640 1 Processor -- start 2024-04-24T06:40:32.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.255+0000 7fb8ac800640 1 -- start start 2024-04-24T06:40:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a41e84c0 0x7fb8a41e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 0x7fb8a40cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a406e600 con 0x7fb88c004800 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a406e480 con 0x7fb8a41e84c0 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.256+0000 7fb8ac800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40d01a0 con 0x7fb8a40cceb0 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55430/0 (socket says 172.21.15.59:55430) 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 -- 172.21.15.59:0/68552414 learned_addr learned my addr 172.21.15.59:0/68552414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8aa800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 0x7fb8a40cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9400640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a41e84c0 0x7fb8a41e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 -- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 msgr2=0x7fb8a40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 0x7fb8a40cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 -- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a41e84c0 msgr2=0x7fb8a41e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a41e84c0 0x7fb8a41e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8a9e00640 1 -- 172.21.15.59:0/68552414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a41e9270 con 0x7fb88c004800 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.257+0000 7fb8aa800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 0x7fb8a40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb8a9e00640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb89800e710 tx=0x7fb89800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89800ee20 con 0x7fb88c004800 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb898018070 con 0x7fb88c004800 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89800cb10 con 0x7fb88c004800 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a41e9550 con 0x7fb88c004800 2024-04-24T06:40:32.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.258+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a41e9b10 con 0x7fb88c004800 2024-04-24T06:40:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.259+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a4068a80 con 0x7fb88c004800 2024-04-24T06:40:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.260+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb89800ccb0 con 0x7fb88c004800 2024-04-24T06:40:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.261+0000 7fb89e000640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.261+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7fb89808bae0 con 0x7fb88c004800 2024-04-24T06:40:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.262+0000 7fb8a9400640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.262+0000 7fb8a9400640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 0x7fb88806edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb894033000 tx=0x7fb894008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.266+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb898059dc0 con 0x7fb88c004800 2024-04-24T06:40:32.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.510+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a415ee30 con 0x7fb88c004800 2024-04-24T06:40:32.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.511+0000 7fb89e000640 1 -- 172.21.15.59:0/68552414 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5164 (secure 0 0 0) 0x7fb8980596c0 con 0x7fb88c004800 2024-04-24T06:40:32.513 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:32.513 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/271137136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":271137136},{"type":"v1","addr":"172.21.15.59:6837","nonce":271137136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:40:27.887210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:40:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 91 2024-04-24T06:40:32.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 msgr2=0x7fb88806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 0x7fb88806edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb894033000 tx=0x7fb894008030 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 msgr2=0x7fb8a4153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb89800e710 tx=0x7fb89800ebd0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 shutdown_connections 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb8a40cceb0 0x7fb8a40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8a41e84c0 0x7fb8a41e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 --2- 172.21.15.59:0/68552414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb88c004800 0x7fb8a4153aa0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.514+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 >> 172.21.15.59:0/68552414 conn(0x7fb8a4067ea0 msgr2=0x7fb8a4162460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.515+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 shutdown_connections 2024-04-24T06:40:32.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.515+0000 7fb8ac800640 1 -- 172.21.15.59:0/68552414 wait complete. 2024-04-24T06:40:32.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.741+0000 7f25ca000640 1 Processor -- start 2024-04-24T06:40:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.741+0000 7f25ca000640 1 -- start start 2024-04-24T06:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.742+0000 7f25ca000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25c415c3d0 0x7f25c40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.742+0000 7f25ca000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25ca000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c4160210 con 0x7f25c40b4690 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25ca000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f25c41604f0 con 0x7f25c415c3d0 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25ca000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25c41607d0 con 0x7f25c40cd590 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25c8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25c415c3d0 0x7f25c40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25c9600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f25c40b4690 0x7f25c406b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56212/0 (socket says 172.21.15.59:56212) 2024-04-24T06:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25c9600640 1 -- 172.21.15.59:0/1522226649 learned_addr learned my addr 172.21.15.59:0/1522226649 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.743+0000 7f25c3e00640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.744+0000 7f25c8c00640 1 -- 172.21.15.59:0/1522226649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 msgr2=0x7f25c40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.744+0000 7f25c8c00640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.744+0000 7f25c8c00640 1 -- 172.21.15.59:0/1522226649 >> v1:172.21.15.59:6789/0 conn(0x7f25c40b4690 legacy=0x7f25c406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.744+0000 7f25c8c00640 1 -- 172.21.15.59:0/1522226649 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c4160ab0 con 0x7f25c415c3d0 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.744+0000 7f25c3400640 1 -- 172.21.15.59:0/1522226649 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862563409 0 0) 0x7f25c4160210 con 0x7f25c40b4690 2024-04-24T06:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.745+0000 7f25c8c00640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40cd050 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f25b80097a0 tx=0x7f25b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1237e1485c6f8ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.745+0000 7f25c3400640 1 -- 172.21.15.59:0/1522226649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8037070 con 0x7f25c415c3d0 2024-04-24T06:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.745+0000 7f25c3400640 1 -- 172.21.15.59:0/1522226649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b8032440 con 0x7f25c415c3d0 2024-04-24T06:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.745+0000 7f25c3400640 1 -- 172.21.15.59:0/1522226649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8032810 con 0x7f25c415c3d0 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 -- 172.21.15.59:0/1522226649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 msgr2=0x7f25c40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40cd050 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f25b80097a0 tx=0x7f25b802a180 comp rx=0 tx=0).stop 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 -- 172.21.15.59:0/1522226649 shutdown_connections 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 --2- 172.21.15.59:0/1522226649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40cd050 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 -- 172.21.15.59:0/1522226649 >> 172.21.15.59:0/1522226649 conn(0x7f25c40b21a0 msgr2=0x7f25c40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 -- 172.21.15.59:0/1522226649 shutdown_connections 2024-04-24T06:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.746+0000 7f25ca000640 1 -- 172.21.15.59:0/1522226649 wait complete. 2024-04-24T06:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.748+0000 7f25ca000640 1 Processor -- start 2024-04-24T06:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.748+0000 7f25ca000640 1 -- start start 2024-04-24T06:40:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 0x7f25c415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25c415c7b0 con 0x7f25c415c3d0 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25c40b4b00 con 0x7f25c40b4690 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25ca000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f25c415c930 con 0x7f25c40cd590 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55444/0 (socket says 172.21.15.59:55444) 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 -- 172.21.15.59:0/509879035 learned_addr learned my addr 172.21.15.59:0/509879035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c8c00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 0x7f25c415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c3e00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 -- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 msgr2=0x7f25c40ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 -- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 msgr2=0x7f25c415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 0x7f25c415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c9600640 1 -- 172.21.15.59:0/509879035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c415fb10 con 0x7f25c415c3d0 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.749+0000 7f25c8c00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 0x7f25c415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25c9600640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c7a0 tx=0x7f25bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc018070 con 0x7f25c415c3d0 2024-04-24T06:40:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25bc00a640 con 0x7f25c415c3d0 2024-04-24T06:40:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00aab0 con 0x7f25c415c3d0 2024-04-24T06:40:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c415ef90 con 0x7f25c415c3d0 2024-04-24T06:40:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.750+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c415f550 con 0x7f25c415c3d0 2024-04-24T06:40:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.751+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c404fd90 con 0x7f25c415c3d0 2024-04-24T06:40:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.752+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25bc00ac50 con 0x7f25c415c3d0 2024-04-24T06:40:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.753+0000 7f25c0c00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 0x7f25a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.754+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f25bc08b1e0 con 0x7f25c415c3d0 2024-04-24T06:40:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.754+0000 7f25c8c00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 0x7f25a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:32.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.754+0000 7f25c8c00640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 0x7f25a406ede0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f25b8034000 tx=0x7f25b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:32.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:32.758+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25bc0594c0 con 0x7f25c415c3d0 2024-04-24T06:40:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.009+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25c40af870 con 0x7f25c415c3d0 2024-04-24T06:40:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.010+0000 7f25c0c00640 1 -- 172.21.15.59:0/509879035 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5164 (secure 0 0 0) 0x7f25bc058dc0 con 0x7f25c415c3d0 2024-04-24T06:40:33.011 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:33.012 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/271137136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":271137136},{"type":"v1","addr":"172.21.15.59:6837","nonce":271137136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:38:38.647945+0000","modified":"2024-04-24T06:40:27.887210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-24T06:40:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 91 2024-04-24T06:40:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.013+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 msgr2=0x7f25a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.013+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 0x7f25a406ede0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f25b8034000 tx=0x7f25b802e000 comp rx=0 tx=0).stop 2024-04-24T06:40:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.013+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 msgr2=0x7f25c40ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.013+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00c7a0 tx=0x7f25bc00cc60 comp rx=0 tx=0).stop 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 shutdown_connections 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f25a406c920 0x7f25a406ede0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25c415c3d0 0x7f25c40ae5f0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f25c40cd590 0x7f25c40ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 --2- 172.21.15.59:0/509879035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25c40b4690 0x7f25c415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 >> 172.21.15.59:0/509879035 conn(0x7f25c40b21a0 msgr2=0x7f25c40d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 shutdown_connections 2024-04-24T06:40:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.014+0000 7f25ca000640 1 -- 172.21.15.59:0/509879035 wait complete. 2024-04-24T06:40:33.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.244+0000 7f957de00640 1 Processor -- start 2024-04-24T06:40:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.244+0000 7f957de00640 1 -- start start 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.246+0000 7f957de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f957815e7c0 0x7f95780d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.246+0000 7f957de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95780d0d90 0x7f95780d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.246+0000 7f957de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95780cf030 con 0x7f95780d2630 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.246+0000 7f957de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f95780b7000 con 0x7f95780d0d90 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.246+0000 7f957de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95780cf620 con 0x7f957815e7c0 2024-04-24T06:40:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.247+0000 7f957d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95780d0d90 0x7f95780d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.247+0000 7f957ca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f957815e7c0 0x7f95780d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.247+0000 7f957d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95780d0d90 0x7f95780d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:57370/0 (socket says 172.21.15.59:57370) 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.247+0000 7f957d400640 1 -- 172.21.15.59:0/1333179869 learned_addr learned my addr 172.21.15.59:0/1333179869 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.248+0000 7f957d400640 1 -- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f957815e7c0 msgr2=0x7f95780d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.248+0000 7f957d400640 1 --2- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f957815e7c0 0x7f95780d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.248+0000 7f957d400640 1 -- 172.21.15.59:0/1333179869 >> v1:172.21.15.66:6789/0 conn(0x7f95780d2630 legacy=0x7f95780b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.248+0000 7f957d400640 1 -- 172.21.15.59:0/1333179869 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95780b6a40 con 0x7f95780d0d90 2024-04-24T06:40:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.249+0000 7f957d400640 1 --2- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95780d0d90 0x7f95780d1220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aa33fda5cf60eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.250+0000 7f9577400640 1 -- 172.21.15.59:0/1333179869 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570036070 con 0x7f95780d0d90 2024-04-24T06:40:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.250+0000 7f9577400640 1 -- 172.21.15.59:0/1333179869 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570031600 con 0x7f95780d0d90 2024-04-24T06:40:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.250+0000 7f9577400640 1 -- 172.21.15.59:0/1333179869 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700319d0 con 0x7f95780d0d90 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 -- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95780d0d90 msgr2=0x7f95780d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 --2- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95780d0d90 0x7f95780d1220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a710 comp rx=0 tx=0).stop 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 -- 172.21.15.59:0/1333179869 shutdown_connections 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 --2- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95780d0d90 0x7f95780d1220 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 --2- 172.21.15.59:0/1333179869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f957815e7c0 0x7f95780d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 -- 172.21.15.59:0/1333179869 >> 172.21.15.59:0/1333179869 conn(0x7f957809d920 msgr2=0x7f957808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 -- 172.21.15.59:0/1333179869 shutdown_connections 2024-04-24T06:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.251+0000 7f957de00640 1 -- 172.21.15.59:0/1333179869 wait complete. 2024-04-24T06:40:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.253+0000 7f957de00640 1 Processor -- start 2024-04-24T06:40:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.253+0000 7f957de00640 1 -- start start 2024-04-24T06:40:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95780d0d90 0x7f957815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f957815e7c0 0x7f95780b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95780b6700 con 0x7f95780d2630 2024-04-24T06:40:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95780cf2b0 con 0x7f957815e7c0 2024-04-24T06:40:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f957de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95780d1a80 con 0x7f95780d0d90 2024-04-24T06:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f9577e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f9577e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55450/0 (socket says 172.21.15.59:55450) 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.254+0000 7f9577e00640 1 -- 172.21.15.59:0/3587165027 learned_addr learned my addr 172.21.15.59:0/3587165027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f957d400640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f957815e7c0 0x7f95780b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f957ca00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95780d0d90 0x7f957815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 -- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95780d0d90 msgr2=0x7f957815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95780d0d90 0x7f957815d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 -- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f957815e7c0 msgr2=0x7f95780b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f957815e7c0 0x7f95780b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 -- 172.21.15.59:0/3587165027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9578147850 con 0x7f95780d2630 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9577e00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f956400c440 tx=0x7f9564001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640052d0 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9564005470 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564007260 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.255+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9578147b30 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.256+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781480f0 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.257+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95780d0b50 con 0x7f95780d2630 2024-04-24T06:40:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.258+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9564007400 con 0x7f95780d2630 2024-04-24T06:40:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.259+0000 7f9574c00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.259+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f956408b1f0 con 0x7f95780d2630 2024-04-24T06:40:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.259+0000 7f957ca00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.260+0000 7f957ca00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 0x7f955806edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f95781f7ec0 tx=0x7f956c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.263+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95780d0b50 con 0x7f95780d2630 2024-04-24T06:40:33.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.469+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f957804fd90 con 0x7f95780d2630 2024-04-24T06:40:33.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.472+0000 7f9574c00640 1 -- 172.21.15.59:0/3587165027 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16782 (secure 0 0 0) 0x7f95780cf030 con 0x7f95780d2630 2024-04-24T06:40:33.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:33.473 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:27.886768+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:38:36.464123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-24T06:38:37.616685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.59:6835/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.66:6834/891418539":"2024-04-25T06:39:50.448272+0000","172.21.15.66:6832/891418539":"2024-04-25T06:39:50.448272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 msgr2=0x7f955806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 0x7f955806edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f95781f7ec0 tx=0x7f956c008040 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 msgr2=0x7f95781f8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f956400c440 tx=0x7f9564001e70 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 shutdown_connections 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f957815e7c0 0x7f95780b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95780d2630 0x7f95781f8920 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 --2- 172.21.15.59:0/3587165027 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95780d0d90 0x7f957815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 >> 172.21.15.59:0/3587165027 conn(0x7f957809d920 msgr2=0x7f957815f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:33.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 shutdown_connections 2024-04-24T06:40:33.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.475+0000 7f957de00640 1 -- 172.21.15.59:0/3587165027 wait complete. 2024-04-24T06:40:33.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:40:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.708+0000 7f90fa400640 1 Processor -- start 2024-04-24T06:40:33.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.708+0000 7f90fa400640 1 -- start start 2024-04-24T06:40:33.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.709+0000 7f90fa400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90f40cf410 0x7f90f415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90fa400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f415dda0 con 0x7f90f406ddf0 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90fa400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f40cd880 con 0x7f90f406e250 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90fa400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f90f4071ec0 con 0x7f90f40cf410 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90f2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f90f406ddf0 0x7f90f4072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56222/0 (socket says 172.21.15.59:56222) 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90f2a00640 1 -- 172.21.15.59:0/313380967 learned_addr learned my addr 172.21.15.59:0/313380967 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.710+0000 7f90f3400640 1 --2- 172.21.15.59:0/313380967 >> v2:172.21.15.59:3300/0 conn(0x7f90f40cf410 0x7f90f415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.711+0000 7f90f3400640 1 -- 172.21.15.59:0/313380967 >> v1:172.21.15.66:6789/0 conn(0x7f90f406e250 legacy=0x7f90f40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.711+0000 7f90f3400640 1 -- 172.21.15.59:0/313380967 >> v1:172.21.15.59:6789/0 conn(0x7f90f406ddf0 legacy=0x7f90f4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.711+0000 7f90f3400640 1 -- 172.21.15.59:0/313380967 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f40cdb20 con 0x7f90f40cf410 2024-04-24T06:40:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.711+0000 7f90f2000640 1 -- 172.21.15.59:0/313380967 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875754369 0 0) 0x7f90dc001490 con 0x7f90f406ddf0 2024-04-24T06:40:33.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.712+0000 7f90f3400640 1 --2- 172.21.15.59:0/313380967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f415d860 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f90e40097a0 tx=0x7f90e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85d0c60283e63d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.712+0000 7f90f2000640 1 -- 172.21.15.59:0/313380967 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4037070 con 0x7f90f40cf410 2024-04-24T06:40:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.712+0000 7f90f2000640 1 -- 172.21.15.59:0/313380967 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e402ae50 con 0x7f90f40cf410 2024-04-24T06:40:33.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.712+0000 7f90f2000640 1 -- 172.21.15.59:0/313380967 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4032660 con 0x7f90f40cf410 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 -- 172.21.15.59:0/313380967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 msgr2=0x7f90f415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 --2- 172.21.15.59:0/313380967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f415d860 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f90e40097a0 tx=0x7f90e402a140 comp rx=0 tx=0).stop 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 -- 172.21.15.59:0/313380967 shutdown_connections 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 --2- 172.21.15.59:0/313380967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f415d860 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 -- 172.21.15.59:0/313380967 >> 172.21.15.59:0/313380967 conn(0x7f90f40b2130 msgr2=0x7f90f40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 -- 172.21.15.59:0/313380967 shutdown_connections 2024-04-24T06:40:33.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.713+0000 7f90fa400640 1 -- 172.21.15.59:0/313380967 wait complete. 2024-04-24T06:40:33.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.715+0000 7f90fa400640 1 Processor -- start 2024-04-24T06:40:33.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.715+0000 7f90fa400640 1 -- start start 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90f406ddf0 0x7f90f4145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90f406e250 0x7f90f4154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f415a6a0 con 0x7f90f40cf410 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f90f40cde00 con 0x7f90f406e250 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90fa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f90f415a820 con 0x7f90f406ddf0 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55468/0 (socket says 172.21.15.59:55468) 2024-04-24T06:40:33.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.716+0000 7f90f3e00640 1 -- 172.21.15.59:0/1539653196 learned_addr learned my addr 172.21.15.59:0/1539653196 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:33.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f2a00640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90f406e250 0x7f90f4154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90f406ddf0 0x7f90f4145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 -- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90f406ddf0 msgr2=0x7f90f4145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90f406ddf0 0x7f90f4145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 -- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90f406e250 msgr2=0x7f90f4154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90f406e250 0x7f90f4154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 -- 172.21.15.59:0/1539653196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f4156350 con 0x7f90f40cf410 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90f3e00640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c440 tx=0x7f90ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec011790 con 0x7f90f40cf410 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ec017070 con 0x7f90f40cf410 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f4156630 con 0x7f90f40cf410 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec011b70 con 0x7f90f40cf410 2024-04-24T06:40:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.717+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f4156bf0 con 0x7f90f40cf410 2024-04-24T06:40:33.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.719+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f4068a80 con 0x7f90f40cf410 2024-04-24T06:40:33.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.720+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90ec011d10 con 0x7f90f40cf410 2024-04-24T06:40:33.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.721+0000 7f90eb400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:33.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.721+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f90ec08bab0 con 0x7f90f40cf410 2024-04-24T06:40:33.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.721+0000 7f90f3400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:33.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.722+0000 7f90f3400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 0x7f90d006edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:33.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.726+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90ec059e30 con 0x7f90f40cf410 2024-04-24T06:40:33.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:33.976+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f90f406f940 con 0x7f90f40cf410 2024-04-24T06:40:34.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.946+0000 7f90eb400640 1 -- 172.21.15.59:0/1539653196 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 161+0+0 (secure 0 0 0) 0x7f90ec059730 con 0x7f90f40cf410 2024-04-24T06:40:34.947 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 msgr2=0x7f90d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 0x7f90d006edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 msgr2=0x7f90f40aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c440 tx=0x7f90ec00c900 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 shutdown_connections 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90d006c8f0 0x7f90d006edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f40cf410 0x7f90f40aed00 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90f406e250 0x7f90f4154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 --2- 172.21.15.59:0/1539653196 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f90f406ddf0 0x7f90f4145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 >> 172.21.15.59:0/1539653196 conn(0x7f90f40b2130 msgr2=0x7f90f40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 shutdown_connections 2024-04-24T06:40:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:34.950+0000 7f90fa400640 1 -- 172.21.15.59:0/1539653196 wait complete. 2024-04-24T06:40:34.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:40:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.187+0000 7fec84e00640 1 Processor -- start 2024-04-24T06:40:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.188+0000 7fec84e00640 1 -- start start 2024-04-24T06:40:35.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec84e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fec8015eb60 0x7fec800d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec84e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec800ba1e0 con 0x7fec800d2d60 2024-04-24T06:40:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec84e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec800ba4c0 con 0x7fec800b99a0 2024-04-24T06:40:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec84e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fec800ba7a0 con 0x7fec8015eb60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec7d400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fec800d2d60 0x7fec800b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56234/0 (socket says 172.21.15.59:56234) 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.189+0000 7fec7d400640 1 -- 172.21.15.59:0/1476392948 learned_addr learned my addr 172.21.15.59:0/1476392948 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3351058396 0 0) 0x7fec800ba1e0 con 0x7fec800d2d60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec64003520 con 0x7fec800d2d60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7de00640 1 --2- 172.21.15.59:0/1476392948 >> v2:172.21.15.66:3300/0 conn(0x7fec8015eb60 0x7fec800d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191685164 0 0) 0x7fec64003520 con 0x7fec800d2d60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 >> v2:172.21.15.66:3300/0 conn(0x7fec8015eb60 msgr2=0x7fec800d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 --2- 172.21.15.59:0/1476392948 >> v2:172.21.15.66:3300/0 conn(0x7fec8015eb60 0x7fec800d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 >> v1:172.21.15.66:6789/0 conn(0x7fec800b99a0 legacy=0x7fec800b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec800b8a90 con 0x7fec800d2d60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fec6c001670 con 0x7fec800d2d60 2024-04-24T06:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 >> v1:172.21.15.59:6789/0 conn(0x7fec800d2d60 legacy=0x7fec800b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec64003a00 0x7fec64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec64004800 0x7fec64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec800ba7a0 con 0x7fec64005210 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400c160 con 0x7fec64003a00 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.190+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fec6400c360 con 0x7fec64004800 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7d400640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec64004800 0x7fec64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7de00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec64003a00 0x7fec64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 -- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec64004800 msgr2=0x7fec64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec64004800 0x7fec64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 -- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec64003a00 msgr2=0x7fec64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec64003a00 0x7fec64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.191+0000 7fec7e800640 1 -- 172.21.15.59:0/1476392948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec64003520 con 0x7fec64005210 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.192+0000 7fec7e800640 1 -- 172.21.15.59:0/1476392948 reap_dead start 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.192+0000 7fec7e800640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fec78001e50 tx=0x7fec64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cd4bcf272422fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78001a70 con 0x7fec64005210 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec78003970 con 0x7fec64005210 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec7ca00640 1 -- 172.21.15.59:0/1476392948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78003d60 con 0x7fec64005210 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 -- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 msgr2=0x7fec6400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fec78001e50 tx=0x7fec64005830 comp rx=0 tx=0).stop 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 -- 172.21.15.59:0/1476392948 shutdown_connections 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 --2- 172.21.15.59:0/1476392948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec6400b9c0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 -- 172.21.15.59:0/1476392948 >> 172.21.15.59:0/1476392948 conn(0x7fec80067b50 msgr2=0x7fec80067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 -- 172.21.15.59:0/1476392948 shutdown_connections 2024-04-24T06:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.193+0000 7fec84e00640 1 -- 172.21.15.59:0/1476392948 wait complete. 2024-04-24T06:40:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.195+0000 7fec84e00640 1 Processor -- start 2024-04-24T06:40:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.195+0000 7fec84e00640 1 -- start start 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 0x7fec80156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec801f7af0 0x7fec801f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec800b3490 con 0x7fec64005210 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fec800b3c80 con 0x7fec801f7af0 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.196+0000 7fec84e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fec800b3610 con 0x7fec80154930 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55478/0 (socket says 172.21.15.59:55478) 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 -- 172.21.15.59:0/820313698 learned_addr learned my addr 172.21.15.59:0/820313698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7d400640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 0x7fec80156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7e800640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec801f7af0 0x7fec801f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 -- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 msgr2=0x7fec80156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 0x7fec80156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 -- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec801f7af0 msgr2=0x7fec801f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec801f7af0 0x7fec801f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 -- 172.21.15.59:0/820313698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec801580f0 con 0x7fec64005210 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7d400640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 0x7fec80156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.197+0000 7fec7de00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fec7000f580 tx=0x7fec7000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.198+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70013810 con 0x7fec64005210 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.198+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7001a070 con 0x7fec64005210 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.198+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70013bb0 con 0x7fec64005210 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.198+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec801583d0 con 0x7fec64005210 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.198+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec800b6140 con 0x7fec64005210 2024-04-24T06:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.199+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec80068a80 con 0x7fec64005210 2024-04-24T06:40:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.200+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec70012070 con 0x7fec64005210 2024-04-24T06:40:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.201+0000 7fec76000640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 0x7fec6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.201+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7fec7008bc40 con 0x7fec64005210 2024-04-24T06:40:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.201+0000 7fec7d400640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 0x7fec6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.202+0000 7fec7d400640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 0x7fec6006ede0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058b0 tx=0x7fec6c001430 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.206+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec70059f30 con 0x7fec64005210 2024-04-24T06:40:35.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.450+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fec800b3200 con 0x7fec64005210 2024-04-24T06:40:35.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.961+0000 7fec76000640 1 -- 172.21.15.59:0/820313698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) ==== 104+0+0 (secure 0 0 0) 0x7fec70059830 con 0x7fec64005210 2024-04-24T06:40:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 msgr2=0x7fec6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 0x7fec6006ede0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058b0 tx=0x7fec6c001430 comp rx=0 tx=0).stop 2024-04-24T06:40:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 msgr2=0x7fec800b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fec7000f580 tx=0x7fec7000fa40 comp rx=0 tx=0).stop 2024-04-24T06:40:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 shutdown_connections 2024-04-24T06:40:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fec6006c920 0x7fec6006ede0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec801f7af0 0x7fec801f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fec80154930 0x7fec80156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 --2- 172.21.15.59:0/820313698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec64005210 0x7fec800b6660 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 >> 172.21.15.59:0/820313698 conn(0x7fec80067b50 msgr2=0x7fec8015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 shutdown_connections 2024-04-24T06:40:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:35.965+0000 7fec84e00640 1 -- 172.21.15.59:0/820313698 wait complete. 2024-04-24T06:40:35.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:40:36.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.203+0000 7f11b0a00640 1 Processor -- start 2024-04-24T06:40:36.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.203+0000 7f11b0a00640 1 -- start start 2024-04-24T06:40:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11b0a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f11ac162d20 0x7f11ac0af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11b0a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11ac0ccb40 0x7f11ac0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11b0a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11ac0cd4c0 con 0x7f11ac162940 2024-04-24T06:40:36.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11b0a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f11ac070060 con 0x7f11ac0ccb40 2024-04-24T06:40:36.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11b0a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f11ac073170 con 0x7f11ac162d20 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11ac0ccb40 0x7f11ac0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11ac0ccb40 0x7f11ac0ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55486/0 (socket says 172.21.15.59:55486) 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.205+0000 7f11abe00640 1 -- 172.21.15.59:0/1652406189 learned_addr learned my addr 172.21.15.59:0/1652406189 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.206+0000 7f11aaa00640 1 --2- 172.21.15.59:0/1652406189 >> v2:172.21.15.66:3300/0 conn(0x7f11ac162d20 0x7f11ac0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.206+0000 7f11abe00640 1 -- 172.21.15.59:0/1652406189 >> v2:172.21.15.66:3300/0 conn(0x7f11ac162d20 msgr2=0x7f11ac0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.206+0000 7f11abe00640 1 --2- 172.21.15.59:0/1652406189 >> v2:172.21.15.66:3300/0 conn(0x7f11ac162d20 0x7f11ac0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.206+0000 7f11abe00640 1 -- 172.21.15.59:0/1652406189 >> v1:172.21.15.66:6789/0 conn(0x7f11ac162940 legacy=0x7f11ac06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.206+0000 7f11abe00640 1 -- 172.21.15.59:0/1652406189 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac073450 con 0x7f11ac0ccb40 2024-04-24T06:40:36.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.207+0000 7f11abe00640 1 --2- 172.21.15.59:0/1652406189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac0ccb40 0x7f11ac0ccf80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f11ac074200 tx=0x7f11a402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5875f5d56ee6b18d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.207+0000 7f11aa000640 1 -- 172.21.15.59:0/1652406189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4036070 con 0x7f11ac0ccb40 2024-04-24T06:40:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.207+0000 7f11aa000640 1 -- 172.21.15.59:0/1652406189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a4031600 con 0x7f11ac0ccb40 2024-04-24T06:40:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.207+0000 7f11aa000640 1 -- 172.21.15.59:0/1652406189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a40319d0 con 0x7f11ac0ccb40 2024-04-24T06:40:36.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 -- 172.21.15.59:0/1652406189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac0ccb40 msgr2=0x7f11ac0ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 --2- 172.21.15.59:0/1652406189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac0ccb40 0x7f11ac0ccf80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f11ac074200 tx=0x7f11a402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 -- 172.21.15.59:0/1652406189 shutdown_connections 2024-04-24T06:40:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 --2- 172.21.15.59:0/1652406189 >> v2:172.21.15.66:3300/0 conn(0x7f11ac162d20 0x7f11ac0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 --2- 172.21.15.59:0/1652406189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac0ccb40 0x7f11ac0ccf80 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 -- 172.21.15.59:0/1652406189 >> 172.21.15.59:0/1652406189 conn(0x7f11ac0b1f90 msgr2=0x7f11ac0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:36.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 -- 172.21.15.59:0/1652406189 shutdown_connections 2024-04-24T06:40:36.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.208+0000 7f11b0a00640 1 -- 172.21.15.59:0/1652406189 wait complete. 2024-04-24T06:40:36.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.210+0000 7f11b0a00640 1 Processor -- start 2024-04-24T06:40:36.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.210+0000 7f11b0a00640 1 -- start start 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11ac0ccb40 0x7f11ac14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11ac162940 0x7f11ac1ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac0ce180 con 0x7f11ac162d20 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac073a10 con 0x7f11ac162940 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11b0a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac0ce300 con 0x7f11ac0ccb40 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55488/0 (socket says 172.21.15.59:55488) 2024-04-24T06:40:36.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.211+0000 7f11abe00640 1 -- 172.21.15.59:0/3974590184 learned_addr learned my addr 172.21.15.59:0/3974590184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11aaa00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11ac162940 0x7f11ac1ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 -- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11ac0ccb40 msgr2=0x7f11ac14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11ac0ccb40 0x7f11ac14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 -- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11ac162940 msgr2=0x7f11ac1ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11ac162940 0x7f11ac1ebcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 -- 172.21.15.59:0/3974590184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac164220 con 0x7f11ac162d20 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11abe00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f11a402ace0 tx=0x7f11a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4036050 con 0x7f11ac162d20 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.212+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a4042070 con 0x7f11ac162d20 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.213+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4009510 con 0x7f11ac162d20 2024-04-24T06:40:36.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.213+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac164500 con 0x7f11ac162d20 2024-04-24T06:40:36.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.213+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac164ac0 con 0x7f11ac162d20 2024-04-24T06:40:36.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.214+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac04fd90 con 0x7f11ac162d20 2024-04-24T06:40:36.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.215+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11a4007bd0 con 0x7f11ac162d20 2024-04-24T06:40:36.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.216+0000 7f11a3400640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.216+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5881+0+0 (secure 0 0 0) 0x7f11a40acb40 con 0x7f11ac162d20 2024-04-24T06:40:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.216+0000 7f11ab400640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:36.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.217+0000 7f11ab400640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 0x7f118c06edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f11ac15ea50 tx=0x7f119c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:36.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.220+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11a4045360 con 0x7f11ac162d20 2024-04-24T06:40:36.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.426+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11ac06cc90 con 0x7f11ac162d20 2024-04-24T06:40:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.965+0000 7f11a3400640 1 -- 172.21.15.59:0/3974590184 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f11a407ae20 con 0x7f11ac162d20 2024-04-24T06:40:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:40:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.968+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 msgr2=0x7f118c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.968+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 0x7f118c06edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f11ac15ea50 tx=0x7f119c0079b0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.968+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 msgr2=0x7f11ac1652c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:36.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.968+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f11a402ace0 tx=0x7f11a4004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 shutdown_connections 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f118c06c8f0 0x7f118c06edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ac162d20 0x7f11ac1652c0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11ac162940 0x7f11ac1ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 --2- 172.21.15.59:0/3974590184 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f11ac0ccb40 0x7f11ac14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.969+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 >> 172.21.15.59:0/3974590184 conn(0x7f11ac0b1f90 msgr2=0x7f11ac0adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:36.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.973+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 shutdown_connections 2024-04-24T06:40:36.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:36.973+0000 7f11b0a00640 1 -- 172.21.15.59:0/3974590184 wait complete. 2024-04-24T06:40:36.998 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:40:37.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.208+0000 7f10d0c00640 1 Processor -- start 2024-04-24T06:40:37.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.208+0000 7f10d0c00640 1 -- start start 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.209+0000 7f10d0c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 0x7f10cc0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.209+0000 7f10d0c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10cc160120 0x7f10cc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.209+0000 7f10d0c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0cfdb0 con 0x7f10cc15fb10 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.209+0000 7f10d0c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc15fef0 con 0x7f10cc160120 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.209+0000 7f10d0c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc0cf390 con 0x7f10cc073670 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f10cc15fb10 0x7f10cc072f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56250/0 (socket says 172.21.15.59:56250) 2024-04-24T06:40:37.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cb400640 1 -- 172.21.15.59:0/770575672 learned_addr learned my addr 172.21.15.59:0/770575672 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cbe00640 1 --2- 172.21.15.59:0/770575672 >> v2:172.21.15.59:3300/0 conn(0x7f10cc160120 0x7f10cc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10c3e00640 1 --2- 172.21.15.59:0/770575672 >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 0x7f10cc0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cbe00640 1 -- 172.21.15.59:0/770575672 >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 msgr2=0x7f10cc0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10caa00640 1 -- 172.21.15.59:0/770575672 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447274035 0 0) 0x7f10cc0cfdb0 con 0x7f10cc15fb10 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cbe00640 1 --2- 172.21.15.59:0/770575672 >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 0x7f10cc0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cbe00640 1 -- 172.21.15.59:0/770575672 >> v1:172.21.15.59:6789/0 conn(0x7f10cc15fb10 legacy=0x7f10cc072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10cbe00640 1 -- 172.21.15.59:0/770575672 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0ce810 con 0x7f10cc160120 2024-04-24T06:40:37.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.210+0000 7f10c3e00640 1 --2- 172.21.15.59:0/770575672 >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 0x7f10cc0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:37.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.211+0000 7f10cbe00640 1 --2- 172.21.15.59:0/770575672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc160120 0x7f10cc0b47b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0cdd30 tx=0x7f10c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e083d4a05d8ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:37.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.212+0000 7f10caa00640 1 -- 172.21.15.59:0/770575672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4007740 con 0x7f10cc160120 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.212+0000 7f10caa00640 1 -- 172.21.15.59:0/770575672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c40078e0 con 0x7f10cc160120 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.212+0000 7f10caa00640 1 -- 172.21.15.59:0/770575672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4007cb0 con 0x7f10cc160120 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.212+0000 7f10d0c00640 1 -- 172.21.15.59:0/770575672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc160120 msgr2=0x7f10cc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.212+0000 7f10d0c00640 1 --2- 172.21.15.59:0/770575672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc160120 0x7f10cc0b47b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0cdd30 tx=0x7f10c402a680 comp rx=0 tx=0).stop 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 -- 172.21.15.59:0/770575672 shutdown_connections 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 --2- 172.21.15.59:0/770575672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc160120 0x7f10cc0b47b0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 --2- 172.21.15.59:0/770575672 >> v2:172.21.15.66:3300/0 conn(0x7f10cc073670 0x7f10cc0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 -- 172.21.15.59:0/770575672 >> 172.21.15.59:0/770575672 conn(0x7f10cc0b1f90 msgr2=0x7f10cc0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 -- 172.21.15.59:0/770575672 shutdown_connections 2024-04-24T06:40:37.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.213+0000 7f10d0c00640 1 -- 172.21.15.59:0/770575672 wait complete. 2024-04-24T06:40:37.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.215+0000 7f10d0c00640 1 Processor -- start 2024-04-24T06:40:37.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.215+0000 7f10d0c00640 1 -- start start 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10cc15fb10 0x7f10cc0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10cc160120 0x7f10cc15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0cf0b0 con 0x7f10cc073670 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0cedd0 con 0x7f10cc15fb10 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10d0c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0ad270 con 0x7f10cc160120 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55506/0 (socket says 172.21.15.59:55506) 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 -- 172.21.15.59:0/2350088125 learned_addr learned my addr 172.21.15.59:0/2350088125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 -- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10cc160120 msgr2=0x7f10cc15e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10cc160120 0x7f10cc15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 -- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10cc15fb10 msgr2=0x7f10cc0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10cc15fb10 0x7f10cc0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.216+0000 7f10cb400640 1 -- 172.21.15.59:0/2350088125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc06e2d0 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10cb400640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c430 tx=0x7f10bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc005ce0 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc007310 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc06e5b0 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.217+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc06eb70 con 0x7f10cc073670 2024-04-24T06:40:37.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.219+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc04fd90 con 0x7f10cc073670 2024-04-24T06:40:37.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.219+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10bc01a070 con 0x7f10cc073670 2024-04-24T06:40:37.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.220+0000 7f10c3400640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 0x7f10a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.220+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5443+0+0 (secure 0 0 0) 0x7f10bc08ae80 con 0x7f10cc073670 2024-04-24T06:40:37.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.221+0000 7f10c3e00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 0x7f10a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.221+0000 7f10c3e00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 0x7f10a806ee00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:37.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.225+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10bc059400 con 0x7f10cc073670 2024-04-24T06:40:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.431+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f10cc0cdd60 con 0x7f10cc073670 2024-04-24T06:40:37.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.975+0000 7f10c3400640 1 -- 172.21.15.59:0/2350088125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f10bc058d00 con 0x7f10cc073670 2024-04-24T06:40:37.976 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:40:37.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.978+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 msgr2=0x7f10a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.978+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 0x7f10a806ee00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4008040 comp rx=0 tx=0).stop 2024-04-24T06:40:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.978+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 msgr2=0x7f10cc0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.978+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c430 tx=0x7f10bc00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 shutdown_connections 2024-04-24T06:40:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f10a806c940 0x7f10a806ee00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f10cc160120 0x7f10cc15e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10cc15fb10 0x7f10cc0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 --2- 172.21.15.59:0/2350088125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc073670 0x7f10cc0ccc20 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 >> 172.21.15.59:0/2350088125 conn(0x7f10cc0b1f90 msgr2=0x7f10cc0d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 shutdown_connections 2024-04-24T06:40:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:37.979+0000 7f10d0c00640 1 -- 172.21.15.59:0/2350088125 wait complete. 2024-04-24T06:40:38.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:40:38.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.215+0000 7f23aa400640 1 Processor -- start 2024-04-24T06:40:38.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.216+0000 7f23aa400640 1 -- start start 2024-04-24T06:40:38.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.217+0000 7f23aa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.217+0000 7f23aa400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23a40b4f20 0x7f23a40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.217+0000 7f23aa400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a40b5da0 con 0x7f23a415ed20 2024-04-24T06:40:38.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.217+0000 7f23aa400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f23a415f100 con 0x7f23a40b4f20 2024-04-24T06:40:38.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.217+0000 7f23aa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40d0a20 con 0x7f23a40d15b0 2024-04-24T06:40:38.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23a40b4f20 0x7f23a40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23a40b4f20 0x7f23a40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55522/0 (socket says 172.21.15.59:55522) 2024-04-24T06:40:38.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 -- 172.21.15.59:0/512222821 learned_addr learned my addr 172.21.15.59:0/512222821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a3e00640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 -- 172.21.15.59:0/512222821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 msgr2=0x7f23a40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 -- 172.21.15.59:0/512222821 >> v1:172.21.15.66:6789/0 conn(0x7f23a415ed20 legacy=0x7f23a40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a9a00640 1 -- 172.21.15.59:0/512222821 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a40d2ed0 con 0x7f23a40b4f20 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.218+0000 7f23a3e00640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.219+0000 7f23a9a00640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a40b4f20 0x7f23a40cf7c0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f239c009e50 tx=0x7f239c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af8e3e22892382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23a3400640 1 -- 172.21.15.59:0/512222821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c036070 con 0x7f23a40b4f20 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23a3400640 1 -- 172.21.15.59:0/512222821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c031600 con 0x7f23a40b4f20 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23a3400640 1 -- 172.21.15.59:0/512222821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c0319d0 con 0x7f23a40b4f20 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 -- 172.21.15.59:0/512222821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a40b4f20 msgr2=0x7f23a40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a40b4f20 0x7f23a40cf7c0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f239c009e50 tx=0x7f239c02a5b0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 -- 172.21.15.59:0/512222821 shutdown_connections 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 --2- 172.21.15.59:0/512222821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a40b4f20 0x7f23a40cf7c0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 -- 172.21.15.59:0/512222821 >> 172.21.15.59:0/512222821 conn(0x7f23a408bf90 msgr2=0x7f23a40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 -- 172.21.15.59:0/512222821 shutdown_connections 2024-04-24T06:40:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.220+0000 7f23aa400640 1 -- 172.21.15.59:0/512222821 wait complete. 2024-04-24T06:40:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.222+0000 7f23aa400640 1 Processor -- start 2024-04-24T06:40:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.222+0000 7f23aa400640 1 -- start start 2024-04-24T06:40:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 0x7f23a4155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40b3600 con 0x7f23a415ed20 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40d3490 con 0x7f23a40b4f20 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.223+0000 7f23aa400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40b3780 con 0x7f23a40d15b0 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55524/0 (socket says 172.21.15.59:55524) 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 -- 172.21.15.59:0/1331014384 learned_addr learned my addr 172.21.15.59:0/1331014384 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9000640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 0x7f23a4155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 -- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 msgr2=0x7f23a40a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 -- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 msgr2=0x7f23a4155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 0x7f23a4155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 -- 172.21.15.59:0/1331014384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a40a0540 con 0x7f23a415ed20 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9000640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 0x7f23a4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.224+0000 7f23a9a00640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f239c009f70 tx=0x7f239c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.225+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c041020 con 0x7f23a415ed20 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.225+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c008710 con 0x7f23a415ed20 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.225+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c036040 con 0x7f23a415ed20 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.225+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a409ff80 con 0x7f23a415ed20 2024-04-24T06:40:38.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.225+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41f46f0 con 0x7f23a415ed20 2024-04-24T06:40:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.226+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a404fd90 con 0x7f23a415ed20 2024-04-24T06:40:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.227+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f239c044020 con 0x7f23a415ed20 2024-04-24T06:40:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.228+0000 7f23a0c00640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 0x7f238806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.228+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f239c0ac690 con 0x7f23a415ed20 2024-04-24T06:40:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.228+0000 7f23a9000640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 0x7f238806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.229+0000 7f23a9000640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 0x7f238806ee00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f239800a830 tx=0x7f2398008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.232+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f239c04b030 con 0x7f23a415ed20 2024-04-24T06:40:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.437+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f23a40d0460 con 0x7f23a415ed20 2024-04-24T06:40:38.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.444+0000 7f23a0c00640 1 -- 172.21.15.59:0/1331014384 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f239c07ad40 con 0x7f23a415ed20 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 msgr2=0x7f238806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 0x7f238806ee00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f239800a830 tx=0x7f2398008040 comp rx=0 tx=0).stop 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 msgr2=0x7f23a41f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f239c009f70 tx=0x7f239c004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 shutdown_connections 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f238806c940 0x7f238806ee00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23a415ed20 0x7f23a41f4ef0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23a40d15b0 0x7f23a40a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 --2- 172.21.15.59:0/1331014384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a40b4f20 0x7f23a4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 >> 172.21.15.59:0/1331014384 conn(0x7f23a408bf90 msgr2=0x7f23a415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:38.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 shutdown_connections 2024-04-24T06:40:38.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.447+0000 7f23aa400640 1 -- 172.21.15.59:0/1331014384 wait complete. 2024-04-24T06:40:38.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-24T06:40:38.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:40:38.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:40:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.667+0000 7f2811600640 1 Processor -- start 2024-04-24T06:40:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.667+0000 7f2811600640 1 -- start start 2024-04-24T06:40:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.668+0000 7f2811600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.668+0000 7f2811600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0b3870 con 0x7f280c15b450 2024-04-24T06:40:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.668+0000 7f2811600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0b3b50 con 0x7f280c0b3080 2024-04-24T06:40:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.668+0000 7f2811600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f280c0b41a0 con 0x7f280c06f3f0 2024-04-24T06:40:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.669+0000 7f280be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f280c15b450 0x7f280c0b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56266/0 (socket says 172.21.15.59:56266) 2024-04-24T06:40:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.669+0000 7f280be00640 1 -- 172.21.15.59:0/1117865191 learned_addr learned my addr 172.21.15.59:0/1117865191 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.669+0000 7f280b400640 1 --2- 172.21.15.59:0/1117865191 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.669+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310917727 0 0) 0x7f280c0b3870 con 0x7f280c15b450 2024-04-24T06:40:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.669+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f0005730 con 0x7f280c15b450 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095181731 0 0) 0x7f27f0005730 con 0x7f280c15b450 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 msgr2=0x7f280c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 --2- 172.21.15.59:0/1117865191 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 >> v1:172.21.15.66:6789/0 conn(0x7f280c0b3080 legacy=0x7f280c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0b3e30 con 0x7f280c15b450 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f27fc001670 con 0x7f280c15b450 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 >> v1:172.21.15.59:6789/0 conn(0x7f280c15b450 legacy=0x7f280c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f0005bc0 0x7f27f0006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f0006a10 0x7f27f0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f27f000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b41a0 con 0x7f27f00073d0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27f000e280 con 0x7f27f0006a10 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27f000e480 con 0x7f27f0005bc0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f280b400640 1 --2- 172.21.15.59:0/1117865191 >> v2:172.21.15.66:3300/0 conn(0x7f280c06f3f0 0x7f280c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.670+0000 7f2810c00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f27f000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f280b400640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f0006a10 0x7f27f0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f280be00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f0005bc0 0x7f27f0006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f2810c00640 1 -- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f0005bc0 msgr2=0x7f27f0006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f2810c00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27f0005bc0 0x7f27f0006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f2810c00640 1 -- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f0006a10 msgr2=0x7f27f0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f2810c00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f0006a10 0x7f27f0006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f2810c00640 1 -- 172.21.15.59:0/1117865191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0005730 con 0x7f27f00073d0 2024-04-24T06:40:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.671+0000 7f280b400640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27f0006a10 0x7f27f0006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:40:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.672+0000 7f2810c00640 1 -- 172.21.15.59:0/1117865191 reap_dead start 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.672+0000 7f2810c00640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f27f000dbd0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2804001e50 tx=0x7f27f0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339f753005102820 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.672+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804001a70 con 0x7f27f00073d0 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.672+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804003970 con 0x7f27f00073d0 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.672+0000 7f280aa00640 1 -- 172.21.15.59:0/1117865191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804003d60 con 0x7f27f00073d0 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 -- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 msgr2=0x7f27f000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f27f000dbd0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2804001e50 tx=0x7f27f0007a40 comp rx=0 tx=0).stop 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 -- 172.21.15.59:0/1117865191 shutdown_connections 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 --2- 172.21.15.59:0/1117865191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f27f000dbd0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 -- 172.21.15.59:0/1117865191 >> 172.21.15.59:0/1117865191 conn(0x7f280c0b21a0 msgr2=0x7f280c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 -- 172.21.15.59:0/1117865191 shutdown_connections 2024-04-24T06:40:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.673+0000 7f2811600640 1 -- 172.21.15.59:0/1117865191 wait complete. 2024-04-24T06:40:38.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.675+0000 7f2811600640 1 Processor -- start 2024-04-24T06:40:38.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.675+0000 7f2811600640 1 -- start start 2024-04-24T06:40:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 0x7f280c158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c0cb6c0 0x7f280c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b4630 con 0x7f27f00073d0 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b4ff0 con 0x7f280c1f7d30 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f2811600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0b47b0 con 0x7f280c0cb6c0 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f280be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f280be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55550/0 (socket says 172.21.15.59:55550) 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.676+0000 7f280be00640 1 -- 172.21.15.59:0/4034963364 learned_addr learned my addr 172.21.15.59:0/4034963364 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 -- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c0cb6c0 msgr2=0x7f280c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c0cb6c0 0x7f280c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 -- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 msgr2=0x7f280c158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280b400640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 0x7f280c158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 0x7f280c158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 -- 172.21.15.59:0/4034963364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1f8560 con 0x7f27f00073d0 2024-04-24T06:40:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280b400640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 0x7f280c158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f280be00640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e7f0 tx=0x7f27fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00eea0 con 0x7f27f00073d0 2024-04-24T06:40:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc018070 con 0x7f27f00073d0 2024-04-24T06:40:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00ca90 con 0x7f27f00073d0 2024-04-24T06:40:38.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.677+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1f8840 con 0x7f27f00073d0 2024-04-24T06:40:38.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.678+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c15cb00 con 0x7f27f00073d0 2024-04-24T06:40:38.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.679+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c04fd90 con 0x7f27f00073d0 2024-04-24T06:40:38.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.679+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27fc00cc30 con 0x7f27f00073d0 2024-04-24T06:40:38.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.680+0000 7f2803e00640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:38.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.681+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f27fc08b780 con 0x7f27f00073d0 2024-04-24T06:40:38.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.681+0000 7f280b400640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:38.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.681+0000 7f280b400640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006f80 tx=0x7f27f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.685+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27fc059db0 con 0x7f27f00073d0 2024-04-24T06:40:38.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.968+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f280c0ae960 con 0x7f27f00073d0 2024-04-24T06:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.978+0000 7f2803e00640 1 -- 172.21.15.59:0/4034963364 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v557) ==== 146+0+0 (secure 0 0 0) 0x7f27fc0596b0 con 0x7f27f00073d0 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 msgr2=0x7f27ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006f80 tx=0x7f27f8008040 comp rx=0 tx=0).stop 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 msgr2=0x7f280c0c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e7f0 tx=0x7f27fc00ecb0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 shutdown_connections 2024-04-24T06:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f27ec06c8f0 0x7f27ec06edb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f280c0cb6c0 0x7f280c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f280c1f7d30 0x7f280c158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 --2- 172.21.15.59:0/4034963364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27f00073d0 0x7f280c0c4020 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 >> 172.21.15.59:0/4034963364 conn(0x7f280c0b21a0 msgr2=0x7f280c158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 shutdown_connections 2024-04-24T06:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:38.981+0000 7f2811600640 1 -- 172.21.15.59:0/4034963364 wait complete. 2024-04-24T06:40:39.002 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-24T06:40:39.002 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-24T06:40:39.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.265+0000 7f405e800640 1 Processor -- start 2024-04-24T06:40:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.265+0000 7f405e800640 1 -- start start 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.266+0000 7f405e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f405815c3d0 0x7f40580cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40580b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f40580b4690 0x7f405806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4058160210 con 0x7f405815c3d0 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f40581604f0 con 0x7f40580b4690 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40581607d0 con 0x7f40580cd590 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f405815c3d0 0x7f40580cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f405815c3d0 0x7f40580cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49288/0 (socket says 172.21.15.59:49288) 2024-04-24T06:40:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405d400640 1 -- 172.21.15.59:0/1084653617 learned_addr learned my addr 172.21.15.59:0/1084653617 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405ca00640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40580b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.267+0000 7f405de00640 1 --2- 172.21.15.59:0/1084653617 >> v2:172.21.15.66:3300/0 conn(0x7f40580b4690 0x7f405806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.268+0000 7f405d400640 1 -- 172.21.15.59:0/1084653617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 msgr2=0x7f40580b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.268+0000 7f405d400640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40580b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.268+0000 7f405d400640 1 -- 172.21.15.59:0/1084653617 >> v2:172.21.15.66:3300/0 conn(0x7f40580b4690 msgr2=0x7f405806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.268+0000 7f405d400640 1 --2- 172.21.15.59:0/1084653617 >> v2:172.21.15.66:3300/0 conn(0x7f40580b4690 0x7f405806b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.268+0000 7f405d400640 1 -- 172.21.15.59:0/1084653617 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4058160ab0 con 0x7f405815c3d0 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.269+0000 7f405d400640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f40580cd050 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4058161170 tx=0x7f404c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bbf840f05f6a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.270+0000 7f4057e00640 1 -- 172.21.15.59:0/1084653617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c037070 con 0x7f405815c3d0 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.270+0000 7f4057e00640 1 -- 172.21.15.59:0/1084653617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c032440 con 0x7f405815c3d0 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.270+0000 7f4057e00640 1 -- 172.21.15.59:0/1084653617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c032810 con 0x7f405815c3d0 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.270+0000 7f405e800640 1 -- 172.21.15.59:0/1084653617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 msgr2=0x7f40580cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.270+0000 7f405e800640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f40580cd050 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4058161170 tx=0x7f404c02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 -- 172.21.15.59:0/1084653617 shutdown_connections 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 --2- 172.21.15.59:0/1084653617 >> v2:172.21.15.66:3300/0 conn(0x7f40580b4690 0x7f405806b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40580b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 --2- 172.21.15.59:0/1084653617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f40580cd050 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 -- 172.21.15.59:0/1084653617 >> 172.21.15.59:0/1084653617 conn(0x7f40580b21a0 msgr2=0x7f40580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 -- 172.21.15.59:0/1084653617 shutdown_connections 2024-04-24T06:40:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.271+0000 7f405e800640 1 -- 172.21.15.59:0/1084653617 wait complete. 2024-04-24T06:40:39.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.272+0000 7f405e800640 1 Processor -- start 2024-04-24T06:40:39.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.273+0000 7f405e800640 1 -- start start 2024-04-24T06:40:39.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 0x7f405815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40581f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f405815c7b0 con 0x7f405815c3d0 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40580b4b00 con 0x7f40580b4690 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f405815c930 con 0x7f40580cd590 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49292/0 (socket says 172.21.15.59:49292) 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 -- 172.21.15.59:0/402094739 learned_addr learned my addr 172.21.15.59:0/402094739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405d400640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 0x7f405815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405ca00640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40581f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 -- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 msgr2=0x7f40581f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40581f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 -- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 msgr2=0x7f405815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 0x7f405815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405de00640 1 -- 172.21.15.59:0/402094739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405815d4a0 con 0x7f405815c3d0 2024-04-24T06:40:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.274+0000 7f405d400640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 0x7f405815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f405de00640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4050007e40 tx=0x7f405000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050014620 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4050019070 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405815d780 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050014a20 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.275+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405815dd40 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.277+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405804fd90 con 0x7f405815c3d0 2024-04-24T06:40:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.277+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4050014bc0 con 0x7f405815c3d0 2024-04-24T06:40:39.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.279+0000 7f4055600640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.279+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f405008c180 con 0x7f405815c3d0 2024-04-24T06:40:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.279+0000 7f405d400640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.280+0000 7f405d400640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f404c034000 tx=0x7f404c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.283+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f405005a840 con 0x7f405815c3d0 2024-04-24T06:40:39.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.489+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f405806e400 con 0x7f405815c3d0 2024-04-24T06:40:39.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.491+0000 7f4055600640 1 -- 172.21.15.59:0/402094739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+13066 (secure 0 0 0) 0x7f405005a140 con 0x7f405815c3d0 2024-04-24T06:40:39.492 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:39.492 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":102,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:37.965697+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.59:6835/1384442467":"2024-04-25T06:40:27.886704+0000","172.21.15.66:6834/891418539":"2024-04-25T06:39:50.448272+0000","172.21.15.66:6832/891418539":"2024-04-25T06:39:50.448272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:39.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 msgr2=0x7f403c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 0x7f403c06edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f404c034000 tx=0x7f404c02e000 comp rx=0 tx=0).stop 2024-04-24T06:40:39.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 msgr2=0x7f405815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4050007e40 tx=0x7f405000e4b0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 shutdown_connections 2024-04-24T06:40:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f403c06c8f0 0x7f403c06edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f405815c3d0 0x7f405815aab0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f40580cd590 0x7f40581f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 --2- 172.21.15.59:0/402094739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40580b4690 0x7f405815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 >> 172.21.15.59:0/402094739 conn(0x7f40580b21a0 msgr2=0x7f40580b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 shutdown_connections 2024-04-24T06:40:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.494+0000 7f405e800640 1 -- 172.21.15.59:0/402094739 wait complete. 2024-04-24T06:40:39.518 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:39.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:40:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.714+0000 7faba5000640 1 Processor -- start 2024-04-24T06:40:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.714+0000 7faba5000640 1 -- start start 2024-04-24T06:40:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7faba5000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faba015ed20 0x7faba00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7faba5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7faba00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7faba5000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba00b5da0 con 0x7faba00b3f10 2024-04-24T06:40:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7faba5000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faba015f100 con 0x7faba015ed20 2024-04-24T06:40:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7faba5000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faba00d0a20 con 0x7faba00b3600 2024-04-24T06:40:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7fab9fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faba00b3f10 0x7faba00b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33550/0 (socket says 172.21.15.59:33550) 2024-04-24T06:40:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.716+0000 7fab9fe00640 1 -- 172.21.15.59:0/2628078052 learned_addr learned my addr 172.21.15.59:0/2628078052 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9ea00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7faba00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9f400640 1 --2- 172.21.15.59:0/2628078052 >> v2:172.21.15.66:3300/0 conn(0x7faba015ed20 0x7faba00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870169224 0 0) 0x7faba00b5da0 con 0x7faba00b3f10 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab84003520 con 0x7faba00b3f10 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2016244122 0 0) 0x7fab84003520 con 0x7faba00b3f10 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 msgr2=0x7faba00d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7faba00d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 >> v2:172.21.15.66:3300/0 conn(0x7faba015ed20 msgr2=0x7faba00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 --2- 172.21.15.59:0/2628078052 >> v2:172.21.15.66:3300/0 conn(0x7faba015ed20 0x7faba00d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba00d2ed0 con 0x7faba00b3f10 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fab980019f0 con 0x7faba00b3f10 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 >> v1:172.21.15.59:6789/0 conn(0x7faba00b3f10 legacy=0x7faba00b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab840039b0 0x7fab84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.717+0000 7fab9e000640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab84004800 0x7fab84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9e000640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7fab840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faba00d0a20 con 0x7fab84004800 2024-04-24T06:40:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fab8400b9c0 con 0x7fab840039b0 2024-04-24T06:40:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fab8400bbc0 con 0x7faba00b3600 2024-04-24T06:40:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab84004800 0x7fab84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9fe00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7fab840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 -- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 msgr2=0x7fab840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7fab840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 -- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab840039b0 msgr2=0x7fab84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab840039b0 0x7fab84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9ea00640 1 -- 172.21.15.59:0/2628078052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab84003520 con 0x7fab84004800 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.718+0000 7fab9fe00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00b3600 0x7fab840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7fab9ea00640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab84004800 0x7fab84004c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009ac0 tx=0x7fab84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205c532e99417b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c036070 con 0x7fab84004800 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c031400 con 0x7fab84004800 2024-04-24T06:40:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7fab9e000640 1 -- 172.21.15.59:0/2628078052 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c009690 con 0x7fab84004800 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7faba5000640 1 -- 172.21.15.59:0/2628078052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab84004800 msgr2=0x7fab84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7faba5000640 1 --2- 172.21.15.59:0/2628078052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab84004800 0x7fab84004c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009ac0 tx=0x7fab84005830 comp rx=0 tx=0).stop 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7fab9fe00640 1 -- 172.21.15.59:0/2628078052 reap_dead start 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7faba5000640 1 -- 172.21.15.59:0/2628078052 shutdown_connections 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.720+0000 7faba5000640 1 -- 172.21.15.59:0/2628078052 >> 172.21.15.59:0/2628078052 conn(0x7faba008bf90 msgr2=0x7faba00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.721+0000 7faba5000640 1 -- 172.21.15.59:0/2628078052 shutdown_connections 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.721+0000 7faba5000640 1 -- 172.21.15.59:0/2628078052 wait complete. 2024-04-24T06:40:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.722+0000 7faba5000640 1 Processor -- start 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.723+0000 7faba5000640 1 -- start start 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.723+0000 7faba5000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 0x7faba00b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.723+0000 7faba5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00ca9e0 0x7faba0153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.723+0000 7faba5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.723+0000 7faba5000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faba00d13e0 con 0x7faba00cf240 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7faba5000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba00d3490 con 0x7fab84004800 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7faba5000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faba00d1560 con 0x7faba00ca9e0 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49308/0 (socket says 172.21.15.59:49308) 2024-04-24T06:40:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 -- 172.21.15.59:0/1554659289 learned_addr learned my addr 172.21.15.59:0/1554659289 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9f400640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 0x7faba00b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9ea00640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00ca9e0 0x7faba0153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 -- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00ca9e0 msgr2=0x7faba0153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00ca9e0 0x7faba0153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 -- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 msgr2=0x7faba00b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 0x7faba00b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9fe00640 1 -- 172.21.15.59:0/1554659289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba00b7800 con 0x7faba00cf240 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.724+0000 7fab9f400640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 0x7faba00b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7fab9fe00640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7faba00d1c80 tx=0x7fab9800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800b8d0 con 0x7faba00cf240 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9800ba70 con 0x7faba00cf240 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800be40 con 0x7faba00cf240 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba00b7ae0 con 0x7faba00cf240 2024-04-24T06:40:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.725+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba00b80a0 con 0x7faba00cf240 2024-04-24T06:40:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.727+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba004fd90 con 0x7faba00cf240 2024-04-24T06:40:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.728+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab9801b020 con 0x7faba00cf240 2024-04-24T06:40:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.728+0000 7fab97400640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 0x7fab8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.729+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7fab9808b9e0 con 0x7faba00cf240 2024-04-24T06:40:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.729+0000 7fab9f400640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 0x7fab8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.730+0000 7fab9f400640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 0x7fab8006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fab900047b0 tx=0x7fab90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.733+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab98059a60 con 0x7faba00cf240 2024-04-24T06:40:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.940+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faba015dcf0 con 0x7faba00cf240 2024-04-24T06:40:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.941+0000 7fab97400640 1 -- 172.21.15.59:0/1554659289 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fab9805dd60 con 0x7faba00cf240 2024-04-24T06:40:39.941 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.943+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 msgr2=0x7fab8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.943+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 0x7fab8006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fab900047b0 tx=0x7fab90008040 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.943+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 msgr2=0x7faba00cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.943+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7faba00d1c80 tx=0x7fab9800b0d0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 shutdown_connections 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fab8006c8f0 0x7fab8006edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba00cf240 0x7faba00cf6c0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faba00ca9e0 0x7faba0153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 --2- 172.21.15.59:0/1554659289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab84004800 0x7faba00b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 >> 172.21.15.59:0/1554659289 conn(0x7faba008bf90 msgr2=0x7faba006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 shutdown_connections 2024-04-24T06:40:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:39.944+0000 7faba5000640 1 -- 172.21.15.59:0/1554659289 wait complete. 2024-04-24T06:40:39.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-24T06:40:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.178+0000 7f26f6e00640 1 Processor -- start 2024-04-24T06:40:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.178+0000 7f26f6e00640 1 -- start start 2024-04-24T06:40:40.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.179+0000 7f26f6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26f015b450 0x7f26f00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f6e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f00b3870 con 0x7f26f006f3f0 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f6e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f00b3b50 con 0x7f26f00cd760 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f26f00b41a0 con 0x7f26f015b450 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26f015b450 0x7f26f00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26f015b450 0x7f26f00b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49314/0 (socket says 172.21.15.59:49314) 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 -- 172.21.15.59:0/887978371 learned_addr learned my addr 172.21.15.59:0/887978371 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 -- 172.21.15.59:0/887978371 >> v1:172.21.15.66:6789/0 conn(0x7f26f00cd760 legacy=0x7f26f00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 -- 172.21.15.59:0/887978371 >> v1:172.21.15.59:6789/0 conn(0x7f26f006f3f0 legacy=0x7f26f00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.180+0000 7f26f5a00640 1 -- 172.21.15.59:0/887978371 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f00b3e30 con 0x7f26f015b450 2024-04-24T06:40:40.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.181+0000 7f26f5a00640 1 --2- 172.21.15.59:0/887978371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f00b4630 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009870 tx=0x7f26e402a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df4d19fd00aced0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:40.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26efe00640 1 -- 172.21.15.59:0/887978371 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4037070 con 0x7f26f015b450 2024-04-24T06:40:40.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26efe00640 1 -- 172.21.15.59:0/887978371 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e4032640 con 0x7f26f015b450 2024-04-24T06:40:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26efe00640 1 -- 172.21.15.59:0/887978371 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4032a10 con 0x7f26f015b450 2024-04-24T06:40:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26f6e00640 1 -- 172.21.15.59:0/887978371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 msgr2=0x7f26f00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26f6e00640 1 --2- 172.21.15.59:0/887978371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f00b4630 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009870 tx=0x7f26e402a330 comp rx=0 tx=0).stop 2024-04-24T06:40:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26f6e00640 1 -- 172.21.15.59:0/887978371 shutdown_connections 2024-04-24T06:40:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26f6e00640 1 --2- 172.21.15.59:0/887978371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f00b4630 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.182+0000 7f26f6e00640 1 -- 172.21.15.59:0/887978371 >> 172.21.15.59:0/887978371 conn(0x7f26f00b21a0 msgr2=0x7f26f00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.183+0000 7f26f6e00640 1 -- 172.21.15.59:0/887978371 shutdown_connections 2024-04-24T06:40:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.183+0000 7f26f6e00640 1 -- 172.21.15.59:0/887978371 wait complete. 2024-04-24T06:40:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.184+0000 7f26f6e00640 1 Processor -- start 2024-04-24T06:40:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.185+0000 7f26f6e00640 1 -- start start 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.185+0000 7f26f6e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f006f3f0 0x7f26f00c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26f00cd760 0x7f26f00b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26f00b4ff0 con 0x7f26f015b450 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26f015b830 con 0x7f26f006f3f0 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f26f00b3080 con 0x7f26f00cd760 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49318/0 (socket says 172.21.15.59:49318) 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f6400640 1 -- 172.21.15.59:0/1262814490 learned_addr learned my addr 172.21.15.59:0/1262814490 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f5a00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f006f3f0 0x7f26f00c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.186+0000 7f26f5000640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26f00cd760 0x7f26f00b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 -- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26f00cd760 msgr2=0x7f26f00b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26f00cd760 0x7f26f00b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 -- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f006f3f0 msgr2=0x7f26f00c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f006f3f0 0x7f26f00c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 -- 172.21.15.59:0/1262814490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f01f5260 con 0x7f26f015b450 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6400640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c780 tx=0x7f26e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8017070 con 0x7f26f015b450 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8003da0 con 0x7f26f015b450 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80052b0 con 0x7f26f015b450 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.187+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f01f5540 con 0x7f26f015b450 2024-04-24T06:40:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.188+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f01f5b00 con 0x7f26f015b450 2024-04-24T06:40:40.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.189+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f004fd90 con 0x7f26f015b450 2024-04-24T06:40:40.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.190+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26e8005450 con 0x7f26f015b450 2024-04-24T06:40:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.191+0000 7f26ed600640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.191+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f26e808ae60 con 0x7f26f015b450 2024-04-24T06:40:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.192+0000 7f26f5a00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.193+0000 7f26f5a00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 0x7f26d006edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f26e4034000 tx=0x7f26e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:40.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.197+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26e8059520 con 0x7f26f015b450 2024-04-24T06:40:40.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.479+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f26f006ef70 con 0x7f26f015b450 2024-04-24T06:40:40.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:40.996+0000 7f26ed600640 1 -- 172.21.15.59:0/1262814490 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v559) ==== 147+0+0 (secure 0 0 0) 0x7f26e8058e20 con 0x7f26f015b450 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 msgr2=0x7f26d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 0x7f26d006edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f26e4034000 tx=0x7f26e402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 msgr2=0x7f26f0157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c780 tx=0x7f26e800cc40 comp rx=0 tx=0).stop 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 shutdown_connections 2024-04-24T06:40:41.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f26d006c8f0 0x7f26d006edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f015b450 0x7f26f0157d40 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f26f00cd760 0x7f26f00b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 --2- 172.21.15.59:0/1262814490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26f006f3f0 0x7f26f00c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 >> 172.21.15.59:0/1262814490 conn(0x7f26f00b21a0 msgr2=0x7f26f0072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 shutdown_connections 2024-04-24T06:40:41.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.001+0000 7f26f6e00640 1 -- 172.21.15.59:0/1262814490 wait complete. 2024-04-24T06:40:41.038 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T06:40:41.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.258+0000 7f212ea00640 1 Processor -- start 2024-04-24T06:40:41.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.258+0000 7f212ea00640 1 -- start start 2024-04-24T06:40:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.260+0000 7f212ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 0x7f2128156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.260+0000 7f212ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 0x7f2128160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.260+0000 7f212ea00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212815cab0 con 0x7f212815d810 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.260+0000 7f212ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2128160e40 con 0x7f21280d0d10 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.260+0000 7f212ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f212815cd30 con 0x7f21281567e0 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.261+0000 7f212cc00640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f212815d810 0x7f21280d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:41570/0 (socket says 172.21.15.59:41570) 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.261+0000 7f212cc00640 1 -- 172.21.15.59:0/2799800656 learned_addr learned my addr 172.21.15.59:0/2799800656 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:41.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.261+0000 7f212d600640 1 --2- 172.21.15.59:0/2799800656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 0x7f2128156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.261+0000 7f212e000640 1 --2- 172.21.15.59:0/2799800656 >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 0x7f2128160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f212d600640 1 -- 172.21.15.59:0/2799800656 >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 msgr2=0x7f2128160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:41.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f212d600640 1 --2- 172.21.15.59:0/2799800656 >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 0x7f2128160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f212d600640 1 -- 172.21.15.59:0/2799800656 >> v1:172.21.15.66:6789/0 conn(0x7f212815d810 legacy=0x7f21280d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f212d600640 1 -- 172.21.15.59:0/2799800656 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212815d010 con 0x7f21281567e0 2024-04-24T06:40:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f212e000640 1 --2- 172.21.15.59:0/2799800656 >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 0x7f2128160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.262+0000 7f2127e00640 1 -- 172.21.15.59:0/2799800656 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917241075 0 0) 0x7f2114001110 con 0x7f212815d810 2024-04-24T06:40:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.263+0000 7f212d600640 1 --2- 172.21.15.59:0/2799800656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 0x7f2128156bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f211c0097a0 tx=0x7f211c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a9f1133b7eba10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f2127e00640 1 -- 172.21.15.59:0/2799800656 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c037070 con 0x7f21281567e0 2024-04-24T06:40:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f2127e00640 1 -- 172.21.15.59:0/2799800656 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c032440 con 0x7f21281567e0 2024-04-24T06:40:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f2127e00640 1 -- 172.21.15.59:0/2799800656 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c032810 con 0x7f21281567e0 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f212ea00640 1 -- 172.21.15.59:0/2799800656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 msgr2=0x7f2128156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f212ea00640 1 --2- 172.21.15.59:0/2799800656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 0x7f2128156bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f211c0097a0 tx=0x7f211c02a180 comp rx=0 tx=0).stop 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f212ea00640 1 -- 172.21.15.59:0/2799800656 shutdown_connections 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f212ea00640 1 --2- 172.21.15.59:0/2799800656 >> v2:172.21.15.66:3300/0 conn(0x7f21280d0d10 0x7f2128160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.264+0000 7f212ea00640 1 --2- 172.21.15.59:0/2799800656 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f21281567e0 0x7f2128156bc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.265+0000 7f212ea00640 1 -- 172.21.15.59:0/2799800656 >> 172.21.15.59:0/2799800656 conn(0x7f212806bdf0 msgr2=0x7f212806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:41.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.265+0000 7f212ea00640 1 -- 172.21.15.59:0/2799800656 shutdown_connections 2024-04-24T06:40:41.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.265+0000 7f212ea00640 1 -- 172.21.15.59:0/2799800656 wait complete. 2024-04-24T06:40:41.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.266+0000 7f212ea00640 1 Processor -- start 2024-04-24T06:40:41.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.267+0000 7f212ea00640 1 -- start start 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21280d0d10 0x7f2128151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 0x7f21281eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21281607d0 con 0x7f21281567e0 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2128160650 con 0x7f21280d0d10 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2128160950 con 0x7f212815d810 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49330/0 (socket says 172.21.15.59:49330) 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 -- 172.21.15.59:0/911962353 learned_addr learned my addr 172.21.15.59:0/911962353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:41.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212d600640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21280d0d10 0x7f2128151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212e000640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 0x7f21281eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 -- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 msgr2=0x7f21281eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 0x7f21281eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 -- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21280d0d10 msgr2=0x7f2128151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.268+0000 7f212cc00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21280d0d10 0x7f2128151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f212cc00640 1 -- 172.21.15.59:0/911962353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2128154580 con 0x7f21281567e0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f212e000640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 0x7f21281eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f212cc00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f211400c7b0 tx=0x7f211400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114018070 con 0x7f21281567e0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21140072d0 con 0x7f21281567e0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2128154860 con 0x7f21281567e0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21140047e0 con 0x7f21281567e0 2024-04-24T06:40:41.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.269+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2128154e20 con 0x7f21281567e0 2024-04-24T06:40:41.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.270+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212804fd90 con 0x7f21281567e0 2024-04-24T06:40:41.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.271+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f211401b070 con 0x7f21281567e0 2024-04-24T06:40:41.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.272+0000 7f2125600640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:41.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.272+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f211408abf0 con 0x7f21281567e0 2024-04-24T06:40:41.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.273+0000 7f212d600640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:41.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.273+0000 7f212d600640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f211c034000 tx=0x7f211c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:41.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.277+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f212815cab0 con 0x7f21281567e0 2024-04-24T06:40:41.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.482+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f21280b5150 con 0x7f21281567e0 2024-04-24T06:40:41.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.490+0000 7f2125600640 1 -- 172.21.15.59:0/911962353 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f21140592b0 con 0x7f21281567e0 2024-04-24T06:40:41.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 msgr2=0x7f210806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 0x7f210806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f211c034000 tx=0x7f211c02e000 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 msgr2=0x7f21281ea590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f211400c7b0 tx=0x7f211400cc70 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 shutdown_connections 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f210806c8f0 0x7f210806edb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f212815d810 0x7f21281eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21281567e0 0x7f21281ea590 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.492+0000 7f212ea00640 1 --2- 172.21.15.59:0/911962353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21280d0d10 0x7f2128151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.493+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 >> 172.21.15.59:0/911962353 conn(0x7f212806bdf0 msgr2=0x7f2128071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.493+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 shutdown_connections 2024-04-24T06:40:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:41.493+0000 7f212ea00640 1 -- 172.21.15.59:0/911962353 wait complete. 2024-04-24T06:40:41.513 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:40:41.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:40:41.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:40:41.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:40:41.726 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:40:41.726 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:41.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:40:41.788 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:40:41.789 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:41.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:40:41.849 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:40:41.849 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:41.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:40:41.911 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:40:41.911 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:41.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:40:41.973 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T06:40:41.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:42.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.189+0000 7ff42de00640 1 Processor -- start 2024-04-24T06:40:42.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.189+0000 7ff42de00640 1 -- start start 2024-04-24T06:40:42.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.190+0000 7ff42de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.190+0000 7ff42de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff428160910 0x7ff428160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.190+0000 7ff42de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280cd520 con 0x7ff4280cd020 2024-04-24T06:40:42.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.190+0000 7ff42de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280cddc0 con 0x7ff428160910 2024-04-24T06:40:42.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.190+0000 7ff42de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280cd800 con 0x7ff4281454e0 2024-04-24T06:40:42.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.191+0000 7ff427400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7ff428160910 0x7ff428160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.191+0000 7ff426a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.191+0000 7ff426a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:44478/0 (socket says 172.21.15.59:44478) 2024-04-24T06:40:42.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.191+0000 7ff426a00640 1 -- 172.21.15.59:0/4271087105 learned_addr learned my addr 172.21.15.59:0/4271087105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.192+0000 7ff426a00640 1 -- 172.21.15.59:0/4271087105 >> v2:172.21.15.66:3300/0 conn(0x7ff428160910 msgr2=0x7ff428160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.192+0000 7ff426a00640 1 --2- 172.21.15.59:0/4271087105 >> v2:172.21.15.66:3300/0 conn(0x7ff428160910 0x7ff428160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.192+0000 7ff426a00640 1 -- 172.21.15.59:0/4271087105 >> v1:172.21.15.66:6789/0 conn(0x7ff4280cd020 legacy=0x7ff4280b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.192+0000 7ff426a00640 1 -- 172.21.15.59:0/4271087105 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280cdae0 con 0x7ff4281454e0 2024-04-24T06:40:42.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.193+0000 7ff426a00640 1 --2- 172.21.15.59:0/4271087105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6178c8737b7eb81d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.194+0000 7ff426000640 1 -- 172.21.15.59:0/4271087105 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037070 con 0x7ff4281454e0 2024-04-24T06:40:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff426000640 1 -- 172.21.15.59:0/4271087105 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff418032440 con 0x7ff4281454e0 2024-04-24T06:40:42.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff426000640 1 -- 172.21.15.59:0/4271087105 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418032810 con 0x7ff4281454e0 2024-04-24T06:40:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 -- 172.21.15.59:0/4271087105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 msgr2=0x7ff428145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 --2- 172.21.15.59:0/4271087105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802a1c0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 -- 172.21.15.59:0/4271087105 shutdown_connections 2024-04-24T06:40:42.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 --2- 172.21.15.59:0/4271087105 >> v2:172.21.15.66:3300/0 conn(0x7ff428160910 0x7ff428160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 --2- 172.21.15.59:0/4271087105 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4281454e0 0x7ff428145bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.195+0000 7ff42de00640 1 -- 172.21.15.59:0/4271087105 >> 172.21.15.59:0/4271087105 conn(0x7ff428067ea0 msgr2=0x7ff42806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.196+0000 7ff42de00640 1 -- 172.21.15.59:0/4271087105 shutdown_connections 2024-04-24T06:40:42.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.196+0000 7ff42de00640 1 -- 172.21.15.59:0/4271087105 wait complete. 2024-04-24T06:40:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.197+0000 7ff42de00640 1 Processor -- start 2024-04-24T06:40:42.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.198+0000 7ff42de00640 1 -- start start 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4280cd020 0x7ff42815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4281454e0 0x7ff42815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280b4010 con 0x7ff428160910 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281460f0 con 0x7ff4281454e0 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff42de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280ad870 con 0x7ff4280cd020 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff427400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff427400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49334/0 (socket says 172.21.15.59:49334) 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff427400640 1 -- 172.21.15.59:0/329238729 learned_addr learned my addr 172.21.15.59:0/329238729 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff426a00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4280cd020 0x7ff42815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.199+0000 7ff41fe00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4281454e0 0x7ff42815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 -- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4280cd020 msgr2=0x7ff42815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4280cd020 0x7ff42815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 -- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4281454e0 msgr2=0x7ff42815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4281454e0 0x7ff42815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 -- 172.21.15.59:0/329238729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280adb50 con 0x7ff428160910 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff427400640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff42000efc0 tx=0x7ff42000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420018020 con 0x7ff428160910 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420014480 con 0x7ff428160910 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420014870 con 0x7ff428160910 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4280ade30 con 0x7ff428160910 2024-04-24T06:40:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.200+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4280ae3f0 con 0x7ff428160910 2024-04-24T06:40:42.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.202+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42806bec0 con 0x7ff428160910 2024-04-24T06:40:42.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.203+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff420005d10 con 0x7ff428160910 2024-04-24T06:40:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.204+0000 7ff41f400640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.204+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7ff42008c680 con 0x7ff428160910 2024-04-24T06:40:42.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.204+0000 7ff426a00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.205+0000 7ff426a00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 0x7ff40406edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff41802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.209+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff42806bec0 con 0x7ff428160910 2024-04-24T06:40:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.460+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff428068a80 con 0x7ff428160910 2024-04-24T06:40:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.461+0000 7ff41f400640 1 -- 172.21.15.59:0/329238729 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+3863 (secure 0 0 0) 0x7ff420022030 con 0x7ff428160910 2024-04-24T06:40:42.462 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:42.462 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4273680999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":4273680999},{"type":"v1","addr":"172.21.15.66:6835","nonce":4273680999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6496,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/4049630536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":4049630536},{"type":"v1","addr":"172.21.15.66:6834","nonce":4049630536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/967602743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":967602743},{"type":"v1","addr":"172.21.15.59:6836","nonce":967602743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/271137136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":271137136},{"type":"v1","addr":"172.21.15.59:6837","nonce":271137136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-04-24T06:40:42.463 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 93 2024-04-24T06:40:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.464+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 msgr2=0x7ff40406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.464+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 0x7ff40406edb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff41802e000 comp rx=0 tx=0).stop 2024-04-24T06:40:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.464+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 msgr2=0x7ff42815b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.464+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff42000efc0 tx=0x7ff42000c490 comp rx=0 tx=0).stop 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 shutdown_connections 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff428160910 0x7ff42815b330 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4281454e0 0x7ff42815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 --2- 172.21.15.59:0/329238729 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff4280cd020 0x7ff42815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 >> 172.21.15.59:0/329238729 conn(0x7ff428067ea0 msgr2=0x7ff4280b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 shutdown_connections 2024-04-24T06:40:42.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.465+0000 7ff42de00640 1 -- 172.21.15.59:0/329238729 wait complete. 2024-04-24T06:40:42.485 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T06:40:42.486 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T06:40:42.486 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:40:42.486 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:40:42.486 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T06:40:42.486 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T06:40:42.486 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:40:42.487 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:40:42.487 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T06:40:42.487 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T06:40:42.487 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:40:42.487 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:40:42.487 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T06:40:42.487 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T06:40:42.487 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:40:42.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:40:42.506 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:40:42.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T06:40:42.507 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:40:42.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T06:40:42.511 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:40:42.511 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T06:40:42.512 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:40:42.512 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T06:40:42.554 INFO:tasks.ceph.mds.a:Started 2024-04-24T06:40:42.555 DEBUG:teuthology.parallel:result is None 2024-04-24T06:40:42.559 INFO:tasks.ceph.mds.d:Started 2024-04-24T06:40:42.559 DEBUG:teuthology.parallel:result is None 2024-04-24T06:40:42.561 INFO:tasks.ceph.mds.c:Started 2024-04-24T06:40:42.561 DEBUG:teuthology.parallel:result is None 2024-04-24T06:40:42.564 INFO:tasks.ceph.mds.b:Started 2024-04-24T06:40:42.564 DEBUG:teuthology.parallel:result is None 2024-04-24T06:40:42.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T06:40:42.708 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-24T06:40:42.708 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:40:42.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-24T06:40:42.722 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:40:42.722 INFO:tasks.ceph.mds.d.smithi066.stdout:starting mds.d at 2024-04-24T06:40:42.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.748+0000 7f95b8c00640 1 Processor -- start 2024-04-24T06:40:42.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.748+0000 7f95b8c00640 1 -- start start 2024-04-24T06:40:42.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.749+0000 7f95b8c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f95b40cf410 0x7f95b415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.749+0000 7f95b8c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b40cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.749+0000 7f95b8c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95b415dda0 con 0x7f95b406ddf0 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.749+0000 7f95b8c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f95b40cd880 con 0x7f95b40cf410 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.749+0000 7f95b8c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95b4071ec0 con 0x7f95b406e250 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95b406ddf0 0x7f95b4072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33558/0 (socket says 172.21.15.59:33558) 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1400640 1 -- 172.21.15.59:0/4158384171 learned_addr learned my addr 172.21.15.59:0/4158384171 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1e00640 1 --2- 172.21.15.59:0/4158384171 >> v2:172.21.15.66:3300/0 conn(0x7f95b40cf410 0x7f95b415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b2800640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b40cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159940514 0 0) 0x7f95a0001110 con 0x7f95b406ddf0 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9598003520 con 0x7f95b406ddf0 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1e00640 1 -- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 msgr2=0x7f95b40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1e00640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b40cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1e00640 1 -- 172.21.15.59:0/4158384171 >> v1:172.21.15.59:6789/0 conn(0x7f95b406ddf0 legacy=0x7f95b4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1680450137 0 0) 0x7f9598003520 con 0x7f95b406ddf0 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.750+0000 7f95b1e00640 1 -- 172.21.15.59:0/4158384171 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b40cdb20 con 0x7f95b40cf410 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.751+0000 7f95b2800640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b1e00640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf410 0x7f95b415d860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f95a40097a0 tx=0x7f95a402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42a478d56d92f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4037070 con 0x7f95b40cf410 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a402ab10 con 0x7f95b40cf410 2024-04-24T06:40:42.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b0a00640 1 -- 172.21.15.59:0/4158384171 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0001840 con 0x7f95b40cf410 2024-04-24T06:40:42.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b8c00640 1 -- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf410 msgr2=0x7f95b415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.752+0000 7f95b8c00640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf410 0x7f95b415d860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f95a40097a0 tx=0x7f95a402a080 comp rx=0 tx=0).stop 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 -- 172.21.15.59:0/4158384171 shutdown_connections 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 --2- 172.21.15.59:0/4158384171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b40cf410 0x7f95b415d860 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 -- 172.21.15.59:0/4158384171 >> 172.21.15.59:0/4158384171 conn(0x7f95b40b2130 msgr2=0x7f95b40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 -- 172.21.15.59:0/4158384171 shutdown_connections 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.753+0000 7f95b8c00640 1 -- 172.21.15.59:0/4158384171 wait complete. 2024-04-24T06:40:42.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.754+0000 7f95b8c00640 1 Processor -- start 2024-04-24T06:40:42.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 -- start start 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 0x7f95b4145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b41f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95b415a6a0 con 0x7f95b40cf410 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95b40cde00 con 0x7f95b406ddf0 2024-04-24T06:40:42.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.755+0000 7f95b8c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f95b415a820 con 0x7f95b406e250 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b1e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 0x7f95b4145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b1400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b41f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49370/0 (socket says 172.21.15.59:49370) 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 -- 172.21.15.59:0/632087273 learned_addr learned my addr 172.21.15.59:0/632087273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 -- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 msgr2=0x7f95b41f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b41f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 -- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 msgr2=0x7f95b4145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 0x7f95b4145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b2800640 1 -- 172.21.15.59:0/632087273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41f8730 con 0x7f95b40cf410 2024-04-24T06:40:42.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b1e00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 0x7f95b4145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:42.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.756+0000 7f95b1400640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b41f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:42.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95b2800640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00efc0 tx=0x7f95ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac018030 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac00cd30 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac003bb0 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41f8a10 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.757+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b4156e90 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.758+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b4157090 con 0x7f95b40cf410 2024-04-24T06:40:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.759+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95ac01b070 con 0x7f95b40cf410 2024-04-24T06:40:42.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.760+0000 7f95aa000640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 0x7f959406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:42.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.760+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5029+0+0 (secure 0 0 0) 0x7f95ac08c1b0 con 0x7f95b40cf410 2024-04-24T06:40:42.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.761+0000 7f95b1e00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 0x7f959406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:42.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.761+0000 7f95b1e00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 0x7f959406edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034000 tx=0x7f95a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:42.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.764+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95b4157090 con 0x7f95b40cf410 2024-04-24T06:40:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:42.982+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f95b4068a80 con 0x7f95b40cf410 2024-04-24T06:40:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.026+0000 7f95aa000640 1 -- 172.21.15.59:0/632087273 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) ==== 97+0+0 (secure 0 0 0) 0x7f95ac010040 con 0x7f95b40cf410 2024-04-24T06:40:43.028 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 msgr2=0x7f959406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 0x7f959406edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034000 tx=0x7f95a402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 msgr2=0x7f95b415aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00efc0 tx=0x7f95ac00c490 comp rx=0 tx=0).stop 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 shutdown_connections 2024-04-24T06:40:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f959406c8f0 0x7f959406edb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b40cf410 0x7f95b415aa40 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f95b406e250 0x7f95b41f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 --2- 172.21.15.59:0/632087273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b406ddf0 0x7f95b4145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 >> 172.21.15.59:0/632087273 conn(0x7f95b40b2130 msgr2=0x7f95b415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.030+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 shutdown_connections 2024-04-24T06:40:43.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.031+0000 7f95b8c00640 1 -- 172.21.15.59:0/632087273 wait complete. 2024-04-24T06:40:43.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T06:40:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.255+0000 7f04f3200640 1 Processor -- start 2024-04-24T06:40:43.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.256+0000 7f04f3200640 1 -- start start 2024-04-24T06:40:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.257+0000 7f04f3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04f4072fc0 0x7f04f415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.257+0000 7f04f3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f415fcb0 con 0x7f04f4158f10 2024-04-24T06:40:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.257+0000 7f04f3200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f415f410 con 0x7f04f40aeb10 2024-04-24T06:40:43.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.257+0000 7f04f3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f04f415f6f0 con 0x7f04f4072fc0 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f04f4158f10 0x7f04f40ae400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33562/0 (socket says 172.21.15.59:33562) 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f1e00640 1 -- 172.21.15.59:0/3467738864 learned_addr learned my addr 172.21.15.59:0/3467738864 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f2800640 1 --2- 172.21.15.59:0/3467738864 >> v2:172.21.15.59:3300/0 conn(0x7f04f4072fc0 0x7f04f415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f1400640 1 -- 172.21.15.59:0/3467738864 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902014546 0 0) 0x7f04f415fcb0 con 0x7f04f4158f10 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f1400640 1 -- 172.21.15.59:0/3467738864 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04d8003520 con 0x7f04f4158f10 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f2800640 1 -- 172.21.15.59:0/3467738864 >> v1:172.21.15.66:6789/0 conn(0x7f04f40aeb10 legacy=0x7f04f40b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:43.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f2800640 1 -- 172.21.15.59:0/3467738864 >> v1:172.21.15.59:6789/0 conn(0x7f04f4158f10 legacy=0x7f04f40ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.258+0000 7f04f2800640 1 -- 172.21.15.59:0/3467738864 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f415f9d0 con 0x7f04f4072fc0 2024-04-24T06:40:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.259+0000 7f04f2800640 1 --2- 172.21.15.59:0/3467738864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f415ed10 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d90 tx=0x7f04ec02a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7077e20bdb82179 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f1400640 1 -- 172.21.15.59:0/3467738864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec007740 con 0x7f04f4072fc0 2024-04-24T06:40:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f1400640 1 -- 172.21.15.59:0/3467738864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec0078e0 con 0x7f04f4072fc0 2024-04-24T06:40:43.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f1400640 1 -- 172.21.15.59:0/3467738864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec007cb0 con 0x7f04f4072fc0 2024-04-24T06:40:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f3200640 1 -- 172.21.15.59:0/3467738864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 msgr2=0x7f04f415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f3200640 1 --2- 172.21.15.59:0/3467738864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f415ed10 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009d90 tx=0x7f04ec02a580 comp rx=0 tx=0).stop 2024-04-24T06:40:43.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.260+0000 7f04f3200640 1 -- 172.21.15.59:0/3467738864 shutdown_connections 2024-04-24T06:40:43.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.261+0000 7f04f3200640 1 --2- 172.21.15.59:0/3467738864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f415ed10 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.261+0000 7f04f3200640 1 -- 172.21.15.59:0/3467738864 >> 172.21.15.59:0/3467738864 conn(0x7f04f40b2310 msgr2=0x7f04f40b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:43.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.261+0000 7f04f3200640 1 -- 172.21.15.59:0/3467738864 shutdown_connections 2024-04-24T06:40:43.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.261+0000 7f04f3200640 1 -- 172.21.15.59:0/3467738864 wait complete. 2024-04-24T06:40:43.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.262+0000 7f04f3200640 1 Processor -- start 2024-04-24T06:40:43.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.263+0000 7f04f3200640 1 -- start start 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f40aeb10 0x7f04f4151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 0x7f04f41f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40b4370 con 0x7f04f4072fc0 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f40d0350 con 0x7f04f40aeb10 2024-04-24T06:40:43.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40b44f0 con 0x7f04f4158f10 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49384/0 (socket says 172.21.15.59:49384) 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 -- 172.21.15.59:0/2763550921 learned_addr learned my addr 172.21.15.59:0/2763550921 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04e9400640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f40aeb10 0x7f04f4151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f2800640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 0x7f04f41f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 -- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 msgr2=0x7f04f41f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 0x7f04f41f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 -- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f40aeb10 msgr2=0x7f04f4151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f40aeb10 0x7f04f4151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f1e00640 1 -- 172.21.15.59:0/2763550921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f41f4b70 con 0x7f04f4072fc0 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.264+0000 7f04f2800640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 0x7f04f41f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04f1e00640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c3c0 tx=0x7f04e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x7f04f4072fc0 2024-04-24T06:40:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4005ce0 con 0x7f04f4072fc0 2024-04-24T06:40:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f4151b10 con 0x7f04f4072fc0 2024-04-24T06:40:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40072f0 con 0x7f04f4072fc0 2024-04-24T06:40:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.265+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41520d0 con 0x7f04f4072fc0 2024-04-24T06:40:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.266+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f406c960 con 0x7f04f4072fc0 2024-04-24T06:40:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.267+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04e401a070 con 0x7f04f4072fc0 2024-04-24T06:40:43.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.268+0000 7f04eaa00640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.268+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4857+0+0 (secure 0 0 0) 0x7f04e408abe0 con 0x7f04f4072fc0 2024-04-24T06:40:43.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.269+0000 7f04e9400640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.269+0000 7f04e9400640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 0x7f04d006edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f04f4151320 tx=0x7f04dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.273+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04f41f4b70 con 0x7f04f4072fc0 2024-04-24T06:40:43.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.549+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f04f4152310 con 0x7f04f4072fc0 2024-04-24T06:40:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.550+0000 7f04eaa00640 1 -- 172.21.15.59:0/2763550921 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f04f4152310 con 0x7f04f4072fc0 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihm3nGJLRAATGO2E86tb+kXWwlnUz/3ng==", 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:40:43.552 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmw5rBDhAA0udpz2vccmrwLPmgkvOPuQ==", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmdEZ4ARAAtg5lkwgcsRKhqJHab//gWQ==", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmVbO2GhAAw21/KVi1iAf0NA/M1TucUQ==", 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.554 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBfpihmFUlZJBAA8ZsO+RElAcOyA0dCeiOelA==", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.555 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBipihmLKTnMRAA7+83+ccoaJAKnoCNt6oAPw==", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBmpihmtzhWERAA8Vp+qohCQkYD9awFqhyyUA==", 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.556 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBppihmDuvNLBAANHvUhEWifUL+VM2fB5hE/w==", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBwpihmy3sIGBAAvPFge2ESP9NgEA6IGkglkg==", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBzpihmkUUGMhAAOeumYT7vNl9gLq2y4TKObg==", 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.558 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB3pihmqn2UEBAAxgmAEWkuITTbZ0C1+4i9ow==", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQB6pihmMqTKLBAAHEai5pXZIXbMVv3PSJoFjg==", 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-24T06:40:43.560 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA==", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmbd8bARAA3kssWzFqTYSlKSrHr6OLpQ==", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:40:43.561 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBbpihm3N0IMhAAm5CxSCYYU3yw880z+D09rw==", 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBcpihmkem9CxAAUNSymELY1FiETel36udx7g==", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBZpihmvFIRKRAAYDFBMr6NJDIMa8MOUc21dw==", 2024-04-24T06:40:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmTykeIxAAOlW02gW/ArrO4Be9PjiGsA==", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQBapihmYh0NGRAAmOFTX4xbXuK/21wVYenoNA==", 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-24T06:40:43.565 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:43.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-24T06:40:43.566 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:40:43.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 msgr2=0x7f04d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 0x7f04d006edb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f04f4151320 tx=0x7f04dc009040 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 msgr2=0x7f04f41f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c3c0 tx=0x7f04e400c880 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 shutdown_connections 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d006c8f0 0x7f04d006edb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f4158f10 0x7f04f41f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f40aeb10 0x7f04f4151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 --2- 172.21.15.59:0/2763550921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f4072fc0 0x7f04f41f8e00 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 >> 172.21.15.59:0/2763550921 conn(0x7f04f40b2310 msgr2=0x7f04f40b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 shutdown_connections 2024-04-24T06:40:43.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.553+0000 7f04f3200640 1 -- 172.21.15.59:0/2763550921 wait complete. 2024-04-24T06:40:43.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.782+0000 7f507f200640 1 Processor -- start 2024-04-24T06:40:43.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.782+0000 7f507f200640 1 -- start start 2024-04-24T06:40:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.783+0000 7f507f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 0x7f50800b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.783+0000 7f507f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50800b3870 con 0x7f50800cd760 2024-04-24T06:40:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.783+0000 7f507f200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50800b3b50 con 0x7f508006f3f0 2024-04-24T06:40:43.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.783+0000 7f507f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50800b41a0 con 0x7f508015b450 2024-04-24T06:40:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.784+0000 7f507e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50800cd760 0x7f50800ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33568/0 (socket says 172.21.15.59:33568) 2024-04-24T06:40:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.784+0000 7f507e800640 1 -- 172.21.15.59:0/2632542097 learned_addr learned my addr 172.21.15.59:0/2632542097 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:43.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.784+0000 7f507de00640 1 --2- 172.21.15.59:0/2632542097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 0x7f50800b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.784+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276790314 0 0) 0x7f50800b3870 con 0x7f50800cd760 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.784+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5064003520 con 0x7f50800cd760 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.785+0000 7f507de00640 1 -- 172.21.15.59:0/2632542097 >> v1:172.21.15.66:6789/0 conn(0x7f508006f3f0 legacy=0x7f50800cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.785+0000 7f507de00640 1 -- 172.21.15.59:0/2632542097 >> v1:172.21.15.59:6789/0 conn(0x7f50800cd760 legacy=0x7f50800ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.785+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4133199833 0 0) 0x7f5064003520 con 0x7f50800cd760 2024-04-24T06:40:43.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.785+0000 7f507de00640 1 -- 172.21.15.59:0/2632542097 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50800b3e30 con 0x7f508015b450 2024-04-24T06:40:43.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.786+0000 7f507de00640 1 --2- 172.21.15.59:0/2632542097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 0x7f50800b4630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50700097d0 tx=0x7f507002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e156297fba27f6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070037070 con 0x7f508015b450 2024-04-24T06:40:43.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507002ac30 con 0x7f508015b450 2024-04-24T06:40:43.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507ca00640 1 -- 172.21.15.59:0/2632542097 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780019f0 con 0x7f508015b450 2024-04-24T06:40:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 -- 172.21.15.59:0/2632542097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 msgr2=0x7f50800b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 --2- 172.21.15.59:0/2632542097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 0x7f50800b4630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50700097d0 tx=0x7f507002a1a0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 -- 172.21.15.59:0/2632542097 shutdown_connections 2024-04-24T06:40:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 --2- 172.21.15.59:0/2632542097 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508015b450 0x7f50800b4630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 -- 172.21.15.59:0/2632542097 >> 172.21.15.59:0/2632542097 conn(0x7f50800b21a0 msgr2=0x7f50800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 -- 172.21.15.59:0/2632542097 shutdown_connections 2024-04-24T06:40:43.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.787+0000 7f507f200640 1 -- 172.21.15.59:0/2632542097 wait complete. 2024-04-24T06:40:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.789+0000 7f507f200640 1 Processor -- start 2024-04-24T06:40:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.789+0000 7f507f200640 1 -- start start 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 0x7f508015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 0x7f5080157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50800b4ff0 con 0x7f508015b450 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f508015b830 con 0x7f50800cd760 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50800b3080 con 0x7f508006f3f0 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.790+0000 7f507e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49390/0 (socket says 172.21.15.59:49390) 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 -- 172.21.15.59:0/2376053562 learned_addr learned my addr 172.21.15.59:0/2376053562 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507de00640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 0x7f508015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507d400640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 0x7f5080157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 -- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 msgr2=0x7f508015e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 0x7f508015e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 -- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 msgr2=0x7f5080157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 0x7f5080157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 -- 172.21.15.59:0/2376053562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508015c9c0 con 0x7f508015b450 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507de00640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 0x7f508015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507d400640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 0x7f5080157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.791+0000 7f507e800640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5078001b60 tx=0x7f507800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.792+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078017070 con 0x7f508015b450 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.792+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5078012400 con 0x7f508015b450 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.792+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780127b0 con 0x7f508015b450 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.792+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508015cca0 con 0x7f508015b450 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.792+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508015d260 con 0x7f508015b450 2024-04-24T06:40:43.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.793+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508004fd90 con 0x7f508015b450 2024-04-24T06:40:43.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.794+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f507801a070 con 0x7f508015b450 2024-04-24T06:40:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.795+0000 7f5076000640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 0x7f505c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.795+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4857+0+0 (secure 0 0 0) 0x7f507808b710 con 0x7f508015b450 2024-04-24T06:40:43.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.795+0000 7f507de00640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 0x7f505c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:43.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.796+0000 7f507de00640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 0x7f505c06edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:43.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:43.800+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50780597a0 con 0x7f508015b450 2024-04-24T06:40:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.006+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50800d0a40 con 0x7f508015b450 2024-04-24T06:40:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.009+0000 7f5076000640 1 -- 172.21.15.59:0/2376053562 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+12809 (secure 0 0 0) 0x7f507805daa0 con 0x7f508015b450 2024-04-24T06:40:44.010 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:44.010 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":106,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:43.038818+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.011+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 msgr2=0x7f505c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.011+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 0x7f505c06edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).stop 2024-04-24T06:40:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.011+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 msgr2=0x7f508015da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.011+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5078001b60 tx=0x7f507800a430 comp rx=0 tx=0).stop 2024-04-24T06:40:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 shutdown_connections 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f505c06c8f0 0x7f505c06edb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f508015b450 0x7f508015da60 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50800cd760 0x7f5080157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 --2- 172.21.15.59:0/2376053562 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f508006f3f0 0x7f508015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 >> 172.21.15.59:0/2376053562 conn(0x7f50800b21a0 msgr2=0x7f50801590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 shutdown_connections 2024-04-24T06:40:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.012+0000 7f507f200640 1 -- 172.21.15.59:0/2376053562 wait complete. 2024-04-24T06:40:44.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:44.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:40:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.240+0000 7f9b96800640 1 Processor -- start 2024-04-24T06:40:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.240+0000 7f9b96800640 1 -- start start 2024-04-24T06:40:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b96800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b900d15b0 0x7f9b900cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b96800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9b900b4f20 0x7f9b900cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b96800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b900b5da0 con 0x7f9b9015ed20 2024-04-24T06:40:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b96800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9015f100 con 0x7f9b900d15b0 2024-04-24T06:40:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b96800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9b900d0a20 con 0x7f9b900b4f20 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.242+0000 7f9b94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b900d15b0 0x7f9b900cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b900d15b0 0x7f9b900cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49400/0 (socket says 172.21.15.59:49400) 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 -- 172.21.15.59:0/137515484 learned_addr learned my addr 172.21.15.59:0/137515484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b95e00640 1 --2- 172.21.15.59:0/137515484 >> v2:172.21.15.66:3300/0 conn(0x7f9b900b4f20 0x7f9b900cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 -- 172.21.15.59:0/137515484 >> v2:172.21.15.66:3300/0 conn(0x7f9b900b4f20 msgr2=0x7f9b900cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 --2- 172.21.15.59:0/137515484 >> v2:172.21.15.66:3300/0 conn(0x7f9b900b4f20 0x7f9b900cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 -- 172.21.15.59:0/137515484 >> v1:172.21.15.66:6789/0 conn(0x7f9b9015ed20 legacy=0x7f9b900d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.243+0000 7f9b94a00640 1 -- 172.21.15.59:0/137515484 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900d2ed0 con 0x7f9b900d15b0 2024-04-24T06:40:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.244+0000 7f9b94a00640 1 --2- 172.21.15.59:0/137515484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900d15b0 0x7f9b900cf280 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9b9008dad0 tx=0x7f9b7c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559f61dd4f2800f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b8fe00640 1 -- 172.21.15.59:0/137515484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c037070 con 0x7f9b900d15b0 2024-04-24T06:40:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b8fe00640 1 -- 172.21.15.59:0/137515484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c02ae20 con 0x7f9b900d15b0 2024-04-24T06:40:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b8fe00640 1 -- 172.21.15.59:0/137515484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c032660 con 0x7f9b900d15b0 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 -- 172.21.15.59:0/137515484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900d15b0 msgr2=0x7f9b900cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 --2- 172.21.15.59:0/137515484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900d15b0 0x7f9b900cf280 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9b9008dad0 tx=0x7f9b7c02a110 comp rx=0 tx=0).stop 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 -- 172.21.15.59:0/137515484 shutdown_connections 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 --2- 172.21.15.59:0/137515484 >> v2:172.21.15.66:3300/0 conn(0x7f9b900b4f20 0x7f9b900cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 --2- 172.21.15.59:0/137515484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900d15b0 0x7f9b900cf280 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 -- 172.21.15.59:0/137515484 >> 172.21.15.59:0/137515484 conn(0x7f9b9008bf90 msgr2=0x7f9b900d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.245+0000 7f9b96800640 1 -- 172.21.15.59:0/137515484 shutdown_connections 2024-04-24T06:40:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.246+0000 7f9b96800640 1 -- 172.21.15.59:0/137515484 wait complete. 2024-04-24T06:40:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.247+0000 7f9b96800640 1 Processor -- start 2024-04-24T06:40:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.247+0000 7f9b96800640 1 -- start start 2024-04-24T06:40:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 0x7f9b900b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b9015ed20 0x7f9b9006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900b3600 con 0x7f9b900b4f20 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900d3490 con 0x7f9b900d15b0 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.248+0000 7f9b96800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900b3780 con 0x7f9b9015ed20 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49416/0 (socket says 172.21.15.59:49416) 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 -- 172.21.15.59:0/179132978 learned_addr learned my addr 172.21.15.59:0/179132978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b94a00640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 0x7f9b900b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95e00640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b9015ed20 0x7f9b9006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 -- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b9015ed20 msgr2=0x7f9b9006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b9015ed20 0x7f9b9006cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 -- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 msgr2=0x7f9b900b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 0x7f9b900b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b95400640 1 -- 172.21.15.59:0/179132978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900b6090 con 0x7f9b900b4f20 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.249+0000 7f9b94a00640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 0x7f9b900b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b95400640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c430 tx=0x7f9b8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84011790 con 0x7f9b900b4f20 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84017070 con 0x7f9b900b4f20 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84011bb0 con 0x7f9b900b4f20 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b901f4650 con 0x7f9b900b4f20 2024-04-24T06:40:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.250+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b901f48f0 con 0x7f9b900b4f20 2024-04-24T06:40:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.251+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9004fd90 con 0x7f9b900b4f20 2024-04-24T06:40:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.252+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b8401c460 con 0x7f9b900b4f20 2024-04-24T06:40:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.254+0000 7f9b8d600640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.254+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4857+0+0 (secure 0 0 0) 0x7f9b8408b6b0 con 0x7f9b900b4f20 2024-04-24T06:40:44.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.254+0000 7f9b94a00640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.255+0000 7f9b94a00640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 0x7f9b7006ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c034000 tx=0x7f9b7c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.258+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b84059e20 con 0x7f9b900b4f20 2024-04-24T06:40:44.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.463+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b900cfcf0 con 0x7f9b900b4f20 2024-04-24T06:40:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.464+0000 7f9b8d600640 1 -- 172.21.15.59:0/179132978 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f9b84059720 con 0x7f9b900b4f20 2024-04-24T06:40:44.464 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:40:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 msgr2=0x7f9b7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 0x7f9b7006ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c034000 tx=0x7f9b7c02e000 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 msgr2=0x7f9b9015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c430 tx=0x7f9b8400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 shutdown_connections 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b7006c920 0x7f9b7006ede0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9b9015ed20 0x7f9b9006cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b900d15b0 0x7f9b900b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 --2- 172.21.15.59:0/179132978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b900b4f20 0x7f9b9015bd80 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.466+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 >> 172.21.15.59:0/179132978 conn(0x7f9b9008bf90 msgr2=0x7f9b900d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.467+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 shutdown_connections 2024-04-24T06:40:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.467+0000 7f9b96800640 1 -- 172.21.15.59:0/179132978 wait complete. 2024-04-24T06:40:44.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:40:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.687+0000 7f566b000640 1 Processor -- start 2024-04-24T06:40:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.688+0000 7f566b000640 1 -- start start 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f566b000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f566415e7c0 0x7f56640d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f566b000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f56640d0d90 0x7f56640d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f566b000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56640cf030 con 0x7f56640d2630 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f566b000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f56640b7000 con 0x7f566415e7c0 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f566b000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f56640cf620 con 0x7f56640d0d90 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f5669c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f566415e7c0 0x7f56640d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f5669c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f566415e7c0 0x7f56640d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49424/0 (socket says 172.21.15.59:49424) 2024-04-24T06:40:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.689+0000 7f5669c00640 1 -- 172.21.15.59:0/2616364588 learned_addr learned my addr 172.21.15.59:0/2616364588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.690+0000 7f566a600640 1 --2- 172.21.15.59:0/2616364588 >> v2:172.21.15.66:3300/0 conn(0x7f56640d0d90 0x7f56640d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.690+0000 7f5669c00640 1 -- 172.21.15.59:0/2616364588 >> v2:172.21.15.66:3300/0 conn(0x7f56640d0d90 msgr2=0x7f56640d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.690+0000 7f5669c00640 1 --2- 172.21.15.59:0/2616364588 >> v2:172.21.15.66:3300/0 conn(0x7f56640d0d90 0x7f56640d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.690+0000 7f5669c00640 1 -- 172.21.15.59:0/2616364588 >> v1:172.21.15.66:6789/0 conn(0x7f56640d2630 legacy=0x7f56640b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.690+0000 7f5669c00640 1 -- 172.21.15.59:0/2616364588 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56640b6a40 con 0x7f566415e7c0 2024-04-24T06:40:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.691+0000 7f5669c00640 1 --2- 172.21.15.59:0/2616364588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640d20f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56640cb580 tx=0x7f565802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b9a1d127787756 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.691+0000 7f5663e00640 1 -- 172.21.15.59:0/2616364588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658036070 con 0x7f566415e7c0 2024-04-24T06:40:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.691+0000 7f5663e00640 1 -- 172.21.15.59:0/2616364588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5658031600 con 0x7f566415e7c0 2024-04-24T06:40:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.691+0000 7f5663e00640 1 -- 172.21.15.59:0/2616364588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580319d0 con 0x7f566415e7c0 2024-04-24T06:40:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 -- 172.21.15.59:0/2616364588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 msgr2=0x7f56640d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 --2- 172.21.15.59:0/2616364588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640d20f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56640cb580 tx=0x7f565802a6f0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 -- 172.21.15.59:0/2616364588 shutdown_connections 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 --2- 172.21.15.59:0/2616364588 >> v2:172.21.15.66:3300/0 conn(0x7f56640d0d90 0x7f56640d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 --2- 172.21.15.59:0/2616364588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640d20f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 -- 172.21.15.59:0/2616364588 >> 172.21.15.59:0/2616364588 conn(0x7f566409d920 msgr2=0x7f566408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 -- 172.21.15.59:0/2616364588 shutdown_connections 2024-04-24T06:40:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.692+0000 7f566b000640 1 -- 172.21.15.59:0/2616364588 wait complete. 2024-04-24T06:40:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.694+0000 7f566b000640 1 Processor -- start 2024-04-24T06:40:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.694+0000 7f566b000640 1 -- start start 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 0x7f566415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 0x7f56640cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56640cf2b0 con 0x7f566415e7c0 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f56640d1700 con 0x7f56640d2630 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566b000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f56640b6700 con 0x7f56640d0d90 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49438/0 (socket says 172.21.15.59:49438) 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f566a600640 1 -- 172.21.15.59:0/2017264844 learned_addr learned my addr 172.21.15.59:0/2017264844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.695+0000 7f5669c00640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 0x7f566415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5669200640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 0x7f56640cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 -- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 msgr2=0x7f566415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 0x7f566415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 -- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 msgr2=0x7f56640cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 0x7f56640cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 -- 172.21.15.59:0/2017264844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56640cb720 con 0x7f566415e7c0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5669c00640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 0x7f566415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5669200640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 0x7f56640cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566a600640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c580 tx=0x7f565c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015070 con 0x7f566415e7c0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565c004220 con 0x7f566415e7c0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c003610 con 0x7f566415e7c0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.696+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56640cb9c0 con 0x7f566415e7c0 2024-04-24T06:40:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.697+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56640cbf80 con 0x7f566415e7c0 2024-04-24T06:40:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.698+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566404fd90 con 0x7f566415e7c0 2024-04-24T06:40:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.699+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f565c0037b0 con 0x7f566415e7c0 2024-04-24T06:40:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.700+0000 7f5661600640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 0x7f564406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.700+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4857+0+0 (secure 0 0 0) 0x7f565c08b5e0 con 0x7f566415e7c0 2024-04-24T06:40:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.700+0000 7f5669c00640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 0x7f564406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.701+0000 7f5669c00640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 0x7f564406ee00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5658033000 tx=0x7f5658009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.704+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f565c059d50 con 0x7f566415e7c0 2024-04-24T06:40:44.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.914+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f56640d23c0 con 0x7f566415e7c0 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.915+0000 7f5661600640 1 -- 172.21.15.59:0/2017264844 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) ==== 84+0+65 (secure 0 0 0) 0x7f565c059650 con 0x7f566415e7c0 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-24T06:40:44.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:44.917 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-24T06:40:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.918+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 msgr2=0x7f564406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.918+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 0x7f564406ee00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5658033000 tx=0x7f5658009000 comp rx=0 tx=0).stop 2024-04-24T06:40:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.918+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 msgr2=0x7f56640cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.918+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c580 tx=0x7f565c00ca40 comp rx=0 tx=0).stop 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 shutdown_connections 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f564406c940 0x7f564406ee00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566415e7c0 0x7f56640cc780 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56640d2630 0x7f56640cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 --2- 172.21.15.59:0/2017264844 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f56640d0d90 0x7f566415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 >> 172.21.15.59:0/2017264844 conn(0x7f566409d920 msgr2=0x7f566406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 shutdown_connections 2024-04-24T06:40:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:44.919+0000 7f566b000640 1 -- 172.21.15.59:0/2017264844 wait complete. 2024-04-24T06:40:44.939 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:40:44.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:40:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.157+0000 7fe133200640 1 Processor -- start 2024-04-24T06:40:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.157+0000 7fe133200640 1 -- start start 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe133200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe13415e990 0x7fe1340d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe133200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1340cfc70 con 0x7fe1340cec40 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe133200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1340cff50 con 0x7fe1340d3c20 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe133200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe1340d0230 con 0x7fe13415e990 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe131400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe1340cec40 0x7fe13406cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33582/0 (socket says 172.21.15.59:33582) 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe131400640 1 -- 172.21.15.59:0/3197909052 learned_addr learned my addr 172.21.15.59:0/3197909052 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.159+0000 7fe131e00640 1 --2- 172.21.15.59:0/3197909052 >> v2:172.21.15.66:3300/0 conn(0x7fe13415e990 0x7fe1340d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615930281 0 0) 0x7fe1340cfc70 con 0x7fe1340cec40 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe118003520 con 0x7fe1340cec40 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3017036196 0 0) 0x7fe118003520 con 0x7fe1340cec40 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 >> v2:172.21.15.66:3300/0 conn(0x7fe13415e990 msgr2=0x7fe1340d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 --2- 172.21.15.59:0/3197909052 >> v2:172.21.15.66:3300/0 conn(0x7fe13415e990 0x7fe1340d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 >> v1:172.21.15.66:6789/0 conn(0x7fe1340d3c20 legacy=0x7fe13406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1340d2410 con 0x7fe1340cec40 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe11c001810 con 0x7fe1340cec40 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 >> v1:172.21.15.59:6789/0 conn(0x7fe1340cec40 legacy=0x7fe13406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe118003a00 0x7fe118003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1180051c0 0x7fe11800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800c090 con 0x7fe118004800 2024-04-24T06:40:45.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe11800c240 con 0x7fe118003a00 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.160+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe11800c440 con 0x7fe1180051c0 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131e00640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe118003a00 0x7fe118003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe132800640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1180051c0 0x7fe11800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 -- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1180051c0 msgr2=0x7fe11800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1180051c0 0x7fe11800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 -- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe118003a00 msgr2=0x7fe118003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe118003a00 0x7fe118003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe131400640 1 -- 172.21.15.59:0/3197909052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe118003520 con 0x7fe118004800 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe132800640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe1180051c0 0x7fe11800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.161+0000 7fe132800640 1 -- 172.21.15.59:0/3197909052 reap_dead start 2024-04-24T06:40:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.162+0000 7fe131400640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 0x7fe118004c80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c0e0 tx=0x7fe118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2024e9de2e8fb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007c30 con 0x7fe118004800 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c007270 con 0x7fe118004800 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe130a00640 1 -- 172.21.15.59:0/3197909052 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0057a0 con 0x7fe118004800 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 -- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 msgr2=0x7fe118004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 0x7fe118004c80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c0e0 tx=0x7fe118005830 comp rx=0 tx=0).stop 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 -- 172.21.15.59:0/3197909052 shutdown_connections 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 --2- 172.21.15.59:0/3197909052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe118004800 0x7fe118004c80 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 -- 172.21.15.59:0/3197909052 >> 172.21.15.59:0/3197909052 conn(0x7fe13408bf90 msgr2=0x7fe1340d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.163+0000 7fe133200640 1 -- 172.21.15.59:0/3197909052 shutdown_connections 2024-04-24T06:40:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.164+0000 7fe133200640 1 -- 172.21.15.59:0/3197909052 wait complete. 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.165+0000 7fe133200640 1 Processor -- start 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.166+0000 7fe133200640 1 -- start start 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.166+0000 7fe133200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe118004800 0x7fe1341608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.166+0000 7fe133200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe133200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1340b9550 0x7fe1340b99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe133200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe13406e1f0 con 0x7fe134157df0 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe133200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1340d21c0 con 0x7fe1340b9550 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe133200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe1340cc640 con 0x7fe118004800 2024-04-24T06:40:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49452/0 (socket says 172.21.15.59:49452) 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 -- 172.21.15.59:0/1115852703 learned_addr learned my addr 172.21.15.59:0/1115852703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe132800640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1340b9550 0x7fe1340b99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 -- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe118004800 msgr2=0x7fe1341608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe118004800 0x7fe1341608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 -- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1340b9550 msgr2=0x7fe1340b99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1340b9550 0x7fe1340b99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.167+0000 7fe131400640 1 -- 172.21.15.59:0/1115852703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1341611d0 con 0x7fe134157df0 2024-04-24T06:40:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe131400640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe11c000c00 tx=0x7fe1340d0120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c03f020 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c0015d0 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007420 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1341614b0 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.168+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1340cc2d0 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.169+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe13404fd90 con 0x7fe134157df0 2024-04-24T06:40:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.171+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe11c042020 con 0x7fe134157df0 2024-04-24T06:40:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.171+0000 7fe12a000640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 0x7fe11006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.172+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4857+0+0 (secure 0 0 0) 0x7fe11c0ab2b0 con 0x7fe134157df0 2024-04-24T06:40:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.172+0000 7fe131e00640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 0x7fe11006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.172+0000 7fe131e00640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 0x7fe11006ede0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe124009fd0 tx=0x7fe124009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.176+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe11c079990 con 0x7fe134157df0 2024-04-24T06:40:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:45.382+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe13406da70 con 0x7fe134157df0 2024-04-24T06:40:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.049+0000 7fe12a000640 1 -- 172.21.15.59:0/1115852703 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) ==== 138+0+0 (secure 0 0 0) 0x7fe11c079290 con 0x7fe134157df0 2024-04-24T06:40:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-24T06:40:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 msgr2=0x7fe11006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 0x7fe11006ede0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe124009fd0 tx=0x7fe124009750 comp rx=0 tx=0).stop 2024-04-24T06:40:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 msgr2=0x7fe13414e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe11c000c00 tx=0x7fe1340d0120 comp rx=0 tx=0).stop 2024-04-24T06:40:46.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 shutdown_connections 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe11006c920 0x7fe11006ede0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1340b9550 0x7fe1340b99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe134157df0 0x7fe13414e4b0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 --2- 172.21.15.59:0/1115852703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe118004800 0x7fe1341608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.053+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 >> 172.21.15.59:0/1115852703 conn(0x7fe13408bf90 msgr2=0x7fe1340b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.054+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 shutdown_connections 2024-04-24T06:40:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.055+0000 7fe133200640 1 -- 172.21.15.59:0/1115852703 wait complete. 2024-04-24T06:40:46.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:40:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.283+0000 7f9988a00640 1 Processor -- start 2024-04-24T06:40:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.283+0000 7f9988a00640 1 -- start start 2024-04-24T06:40:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.285+0000 7f9988a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9984145e10 0x7f998406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.285+0000 7f9988a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9984071d70 0x7f9984072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.285+0000 7f9988a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99840cfc20 con 0x7f99840d0cc0 2024-04-24T06:40:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.285+0000 7f9988a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f99840cff00 con 0x7f9984071d70 2024-04-24T06:40:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.285+0000 7f9988a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f99840d01e0 con 0x7f9984145e10 2024-04-24T06:40:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9984071d70 0x7f9984072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9984071d70 0x7f9984072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49468/0 (socket says 172.21.15.59:49468) 2024-04-24T06:40:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 -- 172.21.15.59:0/3451416711 learned_addr learned my addr 172.21.15.59:0/3451416711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9981e00640 1 --2- 172.21.15.59:0/3451416711 >> v2:172.21.15.66:3300/0 conn(0x7f9984145e10 0x7f998406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 -- 172.21.15.59:0/3451416711 >> v2:172.21.15.66:3300/0 conn(0x7f9984145e10 msgr2=0x7f998406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 --2- 172.21.15.59:0/3451416711 >> v2:172.21.15.66:3300/0 conn(0x7f9984145e10 0x7f998406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 -- 172.21.15.59:0/3451416711 >> v1:172.21.15.66:6789/0 conn(0x7f99840d0cc0 legacy=0x7f99840cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.286+0000 7f9982800640 1 -- 172.21.15.59:0/3451416711 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99840d04c0 con 0x7f9984071d70 2024-04-24T06:40:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9982800640 1 --2- 172.21.15.59:0/3451416711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9984071d70 0x7f9984072200 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f997c009e50 tx=0x7f997c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cfae8c625e2343 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9981400640 1 -- 172.21.15.59:0/3451416711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c036070 con 0x7f9984071d70 2024-04-24T06:40:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9981400640 1 -- 172.21.15.59:0/3451416711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c031600 con 0x7f9984071d70 2024-04-24T06:40:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9981400640 1 -- 172.21.15.59:0/3451416711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c0319d0 con 0x7f9984071d70 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9988a00640 1 -- 172.21.15.59:0/3451416711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9984071d70 msgr2=0x7f9984072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.288+0000 7f9988a00640 1 --2- 172.21.15.59:0/3451416711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9984071d70 0x7f9984072200 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f997c009e50 tx=0x7f997c02a760 comp rx=0 tx=0).stop 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 -- 172.21.15.59:0/3451416711 shutdown_connections 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 --2- 172.21.15.59:0/3451416711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9984071d70 0x7f9984072200 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 --2- 172.21.15.59:0/3451416711 >> v2:172.21.15.66:3300/0 conn(0x7f9984145e10 0x7f998406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 -- 172.21.15.59:0/3451416711 >> 172.21.15.59:0/3451416711 conn(0x7f9984067d90 msgr2=0x7f998406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 -- 172.21.15.59:0/3451416711 shutdown_connections 2024-04-24T06:40:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.289+0000 7f9988a00640 1 -- 172.21.15.59:0/3451416711 wait complete. 2024-04-24T06:40:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.290+0000 7f9988a00640 1 Processor -- start 2024-04-24T06:40:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.291+0000 7f9988a00640 1 -- start start 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 0x7f99840a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9984145e10 0x7f99841e8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99841461f0 con 0x7f99840d0cc0 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f998415cdd0 con 0x7f9984071d70 2024-04-24T06:40:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9988a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f99840b3fe0 con 0x7f9984145e10 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49474/0 (socket says 172.21.15.59:49474) 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 -- 172.21.15.59:0/3458301342 learned_addr learned my addr 172.21.15.59:0/3458301342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9981e00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 0x7f99840a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9982800640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9984145e10 0x7f99841e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 -- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9984145e10 msgr2=0x7f99841e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9984145e10 0x7f99841e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 -- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 msgr2=0x7f99840a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 0x7f99840a9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9979400640 1 -- 172.21.15.59:0/3458301342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99840b42c0 con 0x7f99840d0cc0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.292+0000 7f9981e00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 0x7f99840a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f9979400640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c480 tx=0x7f996c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0052d0 con 0x7f99840d0cc0 2024-04-24T06:40:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c005470 con 0x7f99840d0cc0 2024-04-24T06:40:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c012870 con 0x7f99840d0cc0 2024-04-24T06:40:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99840b45a0 con 0x7f99840d0cc0 2024-04-24T06:40:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.293+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99840aa440 con 0x7f99840d0cc0 2024-04-24T06:40:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.294+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9984068a80 con 0x7f99840d0cc0 2024-04-24T06:40:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.295+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f996c012a10 con 0x7f99840d0cc0 2024-04-24T06:40:46.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.297+0000 7f997aa00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:46.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.297+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5227+0+0 (secure 0 0 0) 0x7f996c08b620 con 0x7f99840d0cc0 2024-04-24T06:40:46.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.297+0000 7f9981e00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:46.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.298+0000 7f9981e00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 0x7f996006edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9974009a20 tx=0x7f9974008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.302+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f996c059c20 con 0x7f99840d0cc0 2024-04-24T06:40:46.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:46.510+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f99840d1190 con 0x7f99840d0cc0 2024-04-24T06:40:47.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.056+0000 7f997aa00640 1 -- 172.21.15.59:0/3458301342 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) ==== 170+0+0 (secure 0 0 0) 0x7f996c059520 con 0x7f99840d0cc0 2024-04-24T06:40:47.057 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-24T06:40:47.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 msgr2=0x7f996006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 0x7f996006edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9974009a20 tx=0x7f9974008040 comp rx=0 tx=0).stop 2024-04-24T06:40:47.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 msgr2=0x7f99840a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c480 tx=0x7f996c00c940 comp rx=0 tx=0).stop 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 shutdown_connections 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f996006c8f0 0x7f996006edb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9984145e10 0x7f99841e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99840d0cc0 0x7f99840a9460 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 --2- 172.21.15.59:0/3458301342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9984071d70 0x7f99840a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.060+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 >> 172.21.15.59:0/3458301342 conn(0x7f9984067d90 msgr2=0x7f9984072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.061+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 shutdown_connections 2024-04-24T06:40:47.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.061+0000 7f9988a00640 1 -- 172.21.15.59:0/3458301342 wait complete. 2024-04-24T06:40:47.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:40:47.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.277+0000 7fa5ac000640 1 Processor -- start 2024-04-24T06:40:47.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.277+0000 7fa5ac000640 1 -- start start 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5ac000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5ac000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40cfc20 con 0x7fa5a4071d70 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5ac000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a40cff00 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5ac000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40d01e0 con 0x7fa5a4145e10 2024-04-24T06:40:47.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5a8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5a8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a40cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49480/0 (socket says 172.21.15.59:49480) 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.279+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1758602613 learned_addr learned my addr 172.21.15.59:0/1758602613 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.280+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1758602613 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 msgr2=0x7fa5a406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.280+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1758602613 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.280+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1758602613 >> v1:172.21.15.66:6789/0 conn(0x7fa5a4071d70 legacy=0x7fa5a415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.280+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1758602613 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40d04c0 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.281+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1758602613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cf340 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa5900098a0 tx=0x7fa59002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59773d99e3ae41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.281+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1758602613 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590037070 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.281+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1758602613 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590032440 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.281+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1758602613 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590032810 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 -- 172.21.15.59:0/1758602613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a40d0cc0 msgr2=0x7fa5a40cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 --2- 172.21.15.59:0/1758602613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cf340 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa5900098a0 tx=0x7fa59002a250 comp rx=0 tx=0).stop 2024-04-24T06:40:47.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 -- 172.21.15.59:0/1758602613 shutdown_connections 2024-04-24T06:40:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 --2- 172.21.15.59:0/1758602613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cf340 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 --2- 172.21.15.59:0/1758602613 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 -- 172.21.15.59:0/1758602613 >> 172.21.15.59:0/1758602613 conn(0x7fa5a4067d90 msgr2=0x7fa5a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 -- 172.21.15.59:0/1758602613 shutdown_connections 2024-04-24T06:40:47.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.282+0000 7fa5ac000640 1 -- 172.21.15.59:0/1758602613 wait complete. 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.283+0000 7fa5ac000640 1 Processor -- start 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.284+0000 7fa5ac000640 1 -- start start 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a41417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cfa00 con 0x7fa5a4071d70 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cf880 con 0x7fa5a40d0cc0 2024-04-24T06:40:47.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5ac000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a41461f0 con 0x7fa5a4145e10 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49486/0 (socket says 172.21.15.59:49486) 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a9600640 1 -- 172.21.15.59:0/2090344560 learned_addr learned my addr 172.21.15.59:0/2090344560 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5aa000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a41417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a9600640 1 -- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 msgr2=0x7fa5a41417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.285+0000 7fa5a9600640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a41417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a9600640 1 -- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5a40d0cc0 msgr2=0x7fa5a40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a9600640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a9600640 1 -- 172.21.15.59:0/2090344560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40b53c0 con 0x7fa5a4071d70 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a9600640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa59400ba50 tx=0x7fa59400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594019070 con 0x7fa5a4071d70 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59400cc00 con 0x7fa5a4071d70 2024-04-24T06:40:47.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a40b38f0 con 0x7fa5a4071d70 2024-04-24T06:40:47.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594014440 con 0x7fa5a4071d70 2024-04-24T06:40:47.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.286+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40b3eb0 con 0x7fa5a4071d70 2024-04-24T06:40:47.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.287+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a4068a80 con 0x7fa5a4071d70 2024-04-24T06:40:47.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.289+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa59401c070 con 0x7fa5a4071d70 2024-04-24T06:40:47.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.290+0000 7fa5a1600640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 0x7fa58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.290+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5609+0+0 (secure 0 0 0) 0x7fa59408c070 con 0x7fa5a4071d70 2024-04-24T06:40:47.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.290+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 0x7fa58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.291+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 0x7fa58406edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa590034000 tx=0x7fa59002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:47.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.294+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa594023070 con 0x7fa5a4071d70 2024-04-24T06:40:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:47.538+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa5a40692b0 con 0x7fa5a4071d70 2024-04-24T06:40:48.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.109+0000 7fa5a1600640 1 -- 172.21.15.59:0/2090344560 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v96) ==== 174+0+0 (secure 0 0 0) 0x7fa5a40b53c0 con 0x7fa5a4071d70 2024-04-24T06:40:48.114 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 16 and data pool 17 2024-04-24T06:40:48.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.116+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 msgr2=0x7fa58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.116+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 0x7fa58406edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa590034000 tx=0x7fa59002e000 comp rx=0 tx=0).stop 2024-04-24T06:40:48.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.116+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 msgr2=0x7fa5a413ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.116+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fa59400ba50 tx=0x7fa59400bf10 comp rx=0 tx=0).stop 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 shutdown_connections 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa58406c8f0 0x7fa58406edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa5a4145e10 0x7fa5a41417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5a40d0cc0 0x7fa5a40cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 --2- 172.21.15.59:0/2090344560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5a4071d70 0x7fa5a413ec70 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.117+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 >> 172.21.15.59:0/2090344560 conn(0x7fa5a4067d90 msgr2=0x7fa5a415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:48.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.118+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 shutdown_connections 2024-04-24T06:40:48.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.119+0000 7fa5ac000640 1 -- 172.21.15.59:0/2090344560 wait complete. 2024-04-24T06:40:48.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:48.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.327+0000 7f215e400640 1 Processor -- start 2024-04-24T06:40:48.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.327+0000 7f215e400640 1 -- start start 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215e400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f21580b5ff0 0x7f21580b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21580d0d90 0x7f21580d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215e400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21580cf030 con 0x7f215815e7c0 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f21580b7000 con 0x7f21580d0d90 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215e400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f21580d1720 con 0x7f21580b5ff0 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.328+0000 7f215da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21580d0d90 0x7f21580d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21580d0d90 0x7f21580d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49496/0 (socket says 172.21.15.59:49496) 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 -- 172.21.15.59:0/2370971511 learned_addr learned my addr 172.21.15.59:0/2370971511 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f2157e00640 1 --2- 172.21.15.59:0/2370971511 >> v2:172.21.15.66:3300/0 conn(0x7f21580b5ff0 0x7f21580b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 -- 172.21.15.59:0/2370971511 >> v2:172.21.15.66:3300/0 conn(0x7f21580b5ff0 msgr2=0x7f21580b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 --2- 172.21.15.59:0/2370971511 >> v2:172.21.15.66:3300/0 conn(0x7f21580b5ff0 0x7f21580b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 -- 172.21.15.59:0/2370971511 >> v1:172.21.15.66:6789/0 conn(0x7f215815e7c0 legacy=0x7f21580d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:48.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.329+0000 7f215da00640 1 -- 172.21.15.59:0/2370971511 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21580b6a40 con 0x7f21580d0d90 2024-04-24T06:40:48.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.330+0000 7f215da00640 1 --2- 172.21.15.59:0/2370971511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580d0d90 0x7f21580d11e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f21580d55c0 tx=0x7f215002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2208b64acbcd773e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.331+0000 7f2157400640 1 -- 172.21.15.59:0/2370971511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150037070 con 0x7f21580d0d90 2024-04-24T06:40:48.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.331+0000 7f2157400640 1 -- 172.21.15.59:0/2370971511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2150032440 con 0x7f21580d0d90 2024-04-24T06:40:48.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.331+0000 7f2157400640 1 -- 172.21.15.59:0/2370971511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150032810 con 0x7f21580d0d90 2024-04-24T06:40:48.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.331+0000 7f215e400640 1 -- 172.21.15.59:0/2370971511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580d0d90 msgr2=0x7f21580d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.331+0000 7f215e400640 1 --2- 172.21.15.59:0/2370971511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580d0d90 0x7f21580d11e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f21580d55c0 tx=0x7f215002a230 comp rx=0 tx=0).stop 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 -- 172.21.15.59:0/2370971511 shutdown_connections 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 --2- 172.21.15.59:0/2370971511 >> v2:172.21.15.66:3300/0 conn(0x7f21580b5ff0 0x7f21580b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 --2- 172.21.15.59:0/2370971511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580d0d90 0x7f21580d11e0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 -- 172.21.15.59:0/2370971511 >> 172.21.15.59:0/2370971511 conn(0x7f215809d920 msgr2=0x7f215808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 -- 172.21.15.59:0/2370971511 shutdown_connections 2024-04-24T06:40:48.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.332+0000 7f215e400640 1 -- 172.21.15.59:0/2370971511 wait complete. 2024-04-24T06:40:48.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.334+0000 7f215e400640 1 Processor -- start 2024-04-24T06:40:48.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.334+0000 7f215e400640 1 -- start start 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21580d0d90 0x7f21580cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215815e7c0 0x7f21581553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21580cf2b0 con 0x7f21580b5ff0 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21580cf620 con 0x7f21580d0d90 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215e400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f21580d2800 con 0x7f215815e7c0 2024-04-24T06:40:48.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49500/0 (socket says 172.21.15.59:49500) 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.335+0000 7f215d000640 1 -- 172.21.15.59:0/4141092680 learned_addr learned my addr 172.21.15.59:0/4141092680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 -- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215815e7c0 msgr2=0x7f21581553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215815e7c0 0x7f21581553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 -- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21580d0d90 msgr2=0x7f21580cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21580d0d90 0x7f21580cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 -- 172.21.15.59:0/4141092680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21580cd7b0 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215d000640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f214000b3f0 tx=0x7f214000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140015020 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2140011830 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21580cbbf0 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140011c20 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.336+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21580cc1b0 con 0x7f21580b5ff0 2024-04-24T06:40:48.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.338+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215804fd90 con 0x7f21580b5ff0 2024-04-24T06:40:48.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.339+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2140018020 con 0x7f21580b5ff0 2024-04-24T06:40:48.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.340+0000 7f2154c00640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 0x7f213406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.340+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f214008b820 con 0x7f21580b5ff0 2024-04-24T06:40:48.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.340+0000 7f2157e00640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 0x7f213406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.341+0000 7f2157e00640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 0x7f213406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f21581610c0 tx=0x7f21480098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.344+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f214001f030 con 0x7f21580b5ff0 2024-04-24T06:40:48.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.541+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21580d52c0 con 0x7f21580b5ff0 2024-04-24T06:40:48.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.545+0000 7f2154c00640 1 -- 172.21.15.59:0/4141092680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16556 (secure 0 0 0) 0x7f21400172c0 con 0x7f21580b5ff0 2024-04-24T06:40:48.546 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:48.546 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":109,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:47.539674+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:48.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 msgr2=0x7f213406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 0x7f213406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f21581610c0 tx=0x7f21480098d0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 msgr2=0x7f21580cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f214000b3f0 tx=0x7f214000b8b0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 shutdown_connections 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f213406c8f0 0x7f213406edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f215815e7c0 0x7f21581553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21580d0d90 0x7f21580cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 --2- 172.21.15.59:0/4141092680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21580b5ff0 0x7f21580cb5f0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.547+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 >> 172.21.15.59:0/4141092680 conn(0x7f215809d920 msgr2=0x7f215815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.548+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 shutdown_connections 2024-04-24T06:40:48.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.548+0000 7f215e400640 1 -- 172.21.15.59:0/4141092680 wait complete. 2024-04-24T06:40:48.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.776+0000 7f238dc00640 1 Processor -- start 2024-04-24T06:40:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.776+0000 7f238dc00640 1 -- start start 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f238dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23880cd020 0x7f23880b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f238dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23880cd520 con 0x7f23880b3e40 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f238dc00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23880cd800 con 0x7f23881454e0 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f238dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f23880cddc0 con 0x7f23880cd020 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f2387400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f23880b3e40 0x7f2388160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33592/0 (socket says 172.21.15.59:33592) 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f2387400640 1 -- 172.21.15.59:0/3663827653 learned_addr learned my addr 172.21.15.59:0/3663827653 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.778+0000 7f237e000640 1 --2- 172.21.15.59:0/3663827653 >> v2:172.21.15.59:3300/0 conn(0x7f23880cd020 0x7f23880b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.779+0000 7f2386000640 1 -- 172.21.15.59:0/3663827653 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565370908 0 0) 0x7f23880cd520 con 0x7f23880b3e40 2024-04-24T06:40:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.779+0000 7f237e000640 1 -- 172.21.15.59:0/3663827653 >> v1:172.21.15.66:6789/0 conn(0x7f23881454e0 legacy=0x7f2388145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.779+0000 7f237e000640 1 -- 172.21.15.59:0/3663827653 >> v1:172.21.15.59:6789/0 conn(0x7f23880b3e40 legacy=0x7f2388160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.779+0000 7f237e000640 1 -- 172.21.15.59:0/3663827653 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23880cdae0 con 0x7f23880cd020 2024-04-24T06:40:48.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.780+0000 7f237e000640 1 --2- 172.21.15.59:0/3663827653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880cd020 0x7f23880b3900 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f2370009d90 tx=0x7f237002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7c693fc3d5a23b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.780+0000 7f2386000640 1 -- 172.21.15.59:0/3663827653 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370007740 con 0x7f23880cd020 2024-04-24T06:40:48.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.780+0000 7f2386000640 1 -- 172.21.15.59:0/3663827653 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23700078e0 con 0x7f23880cd020 2024-04-24T06:40:48.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.780+0000 7f2386000640 1 -- 172.21.15.59:0/3663827653 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370007cb0 con 0x7f23880cd020 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 -- 172.21.15.59:0/3663827653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880cd020 msgr2=0x7f23880b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 --2- 172.21.15.59:0/3663827653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880cd020 0x7f23880b3900 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f2370009d90 tx=0x7f237002a660 comp rx=0 tx=0).stop 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 -- 172.21.15.59:0/3663827653 shutdown_connections 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 --2- 172.21.15.59:0/3663827653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880cd020 0x7f23880b3900 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 -- 172.21.15.59:0/3663827653 >> 172.21.15.59:0/3663827653 conn(0x7f2388067ea0 msgr2=0x7f238806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 -- 172.21.15.59:0/3663827653 shutdown_connections 2024-04-24T06:40:48.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.781+0000 7f238dc00640 1 -- 172.21.15.59:0/3663827653 wait complete. 2024-04-24T06:40:48.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.783+0000 7f238dc00640 1 Processor -- start 2024-04-24T06:40:48.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.783+0000 7f238dc00640 1 -- start start 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23880cd020 0x7f23881f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 0x7f238815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2388161020 con 0x7f23880b3e40 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23881462c0 con 0x7f23881454e0 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f238dc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f23881611a0 con 0x7f23880cd020 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f2386a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f2386a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49510/0 (socket says 172.21.15.59:49510) 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.784+0000 7f2386a00640 1 -- 172.21.15.59:0/407412253 learned_addr learned my addr 172.21.15.59:0/407412253 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2387400640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 0x7f238815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f237e000640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23880cd020 0x7f23881f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 -- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23880cd020 msgr2=0x7f23881f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23880cd020 0x7f23881f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 -- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 msgr2=0x7f238815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 0x7f238815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 -- 172.21.15.59:0/407412253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23880ae5a0 con 0x7f23880b3e40 2024-04-24T06:40:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2387400640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 0x7f238815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f2386a00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f237800c3c0 tx=0x7f237800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780052d0 con 0x7f23880b3e40 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2378005490 con 0x7f23880b3e40 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.785+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378007260 con 0x7f23880b3e40 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.786+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238806df10 con 0x7f23880b3e40 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.786+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881f4900 con 0x7f23880b3e40 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.787+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2388068a80 con 0x7f23880b3e40 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.788+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2378007400 con 0x7f23880b3e40 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.789+0000 7f237f400640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.789+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f237808afc0 con 0x7f23880b3e40 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.790+0000 7f237e000640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.791+0000 7f237e000640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2370034000 tx=0x7f237002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:48.794+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2378059480 con 0x7f23880b3e40 2024-04-24T06:40:49.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.001+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23881f4b90 con 0x7f23880b3e40 2024-04-24T06:40:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.002+0000 7f237f400640 1 -- 172.21.15.59:0/407412253 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16556 (secure 0 0 0) 0x7f2378058d80 con 0x7f23880b3e40 2024-04-24T06:40:49.004 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:49.004 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":109,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:47.539674+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 msgr2=0x7f236406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2370034000 tx=0x7f237002f000 comp rx=0 tx=0).stop 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 msgr2=0x7f238815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f237800c3c0 tx=0x7f237800c880 comp rx=0 tx=0).stop 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 shutdown_connections 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23881454e0 0x7f238815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f23880cd020 0x7f23881f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 --2- 172.21.15.59:0/407412253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23880b3e40 0x7f238815bc20 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.005+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 >> 172.21.15.59:0/407412253 conn(0x7f2388067ea0 msgr2=0x7f23880b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.006+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 shutdown_connections 2024-04-24T06:40:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.006+0000 7f238dc00640 1 -- 172.21.15.59:0/407412253 wait complete. 2024-04-24T06:40:49.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:40:49.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.237+0000 7f4123c00640 1 Processor -- start 2024-04-24T06:40:49.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.237+0000 7f4123c00640 1 -- start start 2024-04-24T06:40:49.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4123c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f411c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4123c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f411c06f6f0 0x7f411c162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4123c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c163370 con 0x7f411c071d70 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4123c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0aef40 con 0x7f411c06f6f0 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4123c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0afd80 con 0x7f411c15e790 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4121c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f411c071d70 0x7f411c0b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58314/0 (socket says 172.21.15.59:58314) 2024-04-24T06:40:49.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f4121c00640 1 -- 172.21.15.59:0/21387063 learned_addr learned my addr 172.21.15.59:0/21387063 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.239+0000 7f411be00640 1 --2- 172.21.15.59:0/21387063 >> v2:172.21.15.66:3300/0 conn(0x7f411c06f6f0 0x7f411c162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f4121200640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f411c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984751163 0 0) 0x7f4114001110 con 0x7f411c071d70 2024-04-24T06:40:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4100003520 con 0x7f411c071d70 2024-04-24T06:40:49.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2961627132 0 0) 0x7f4100003520 con 0x7f411c071d70 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 msgr2=0x7f411c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f411c0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 >> v2:172.21.15.66:3300/0 conn(0x7f411c06f6f0 msgr2=0x7f411c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 --2- 172.21.15.59:0/21387063 >> v2:172.21.15.66:3300/0 conn(0x7f411c06f6f0 0x7f411c162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0adc80 con 0x7f411c071d70 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4114001840 con 0x7f411c071d70 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 >> v1:172.21.15.59:6789/0 conn(0x7f411c071d70 legacy=0x7f411c0b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f41000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4100003f30 0x7f4100004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0aef40 con 0x7f4100003f30 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f410000b9c0 con 0x7f4100004d40 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.240+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f410000bb40 con 0x7f411c15e790 2024-04-24T06:40:49.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4100003f30 0x7f4100004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f4121200640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f41000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f4121c00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 -- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 msgr2=0x7f41000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f411c15e790 0x7f41000039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 -- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4100004d40 msgr2=0x7f41000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f411be00640 1 -- 172.21.15.59:0/21387063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100003520 con 0x7f4100003f30 2024-04-24T06:40:49.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.241+0000 7f4121c00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.242+0000 7f411be00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4100003f30 0x7f4100004800 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4108009ac0 tx=0x7f4100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6bb72581ddedc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108036070 con 0x7f4100003f30 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108031400 con 0x7f4100003f30 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f411b400640 1 -- 172.21.15.59:0/21387063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108009690 con 0x7f4100003f30 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4123c00640 1 -- 172.21.15.59:0/21387063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4100003f30 msgr2=0x7f4100004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4123c00640 1 --2- 172.21.15.59:0/21387063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4100003f30 0x7f4100004800 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4108009ac0 tx=0x7f4100005830 comp rx=0 tx=0).stop 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4121c00640 1 -- 172.21.15.59:0/21387063 reap_dead start 2024-04-24T06:40:49.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4123c00640 1 -- 172.21.15.59:0/21387063 shutdown_connections 2024-04-24T06:40:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4123c00640 1 -- 172.21.15.59:0/21387063 >> 172.21.15.59:0/21387063 conn(0x7f411c067d90 msgr2=0x7f411c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.243+0000 7f4123c00640 1 -- 172.21.15.59:0/21387063 shutdown_connections 2024-04-24T06:40:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.244+0000 7f4123c00640 1 -- 172.21.15.59:0/21387063 wait complete. 2024-04-24T06:40:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.245+0000 7f4123c00640 1 Processor -- start 2024-04-24T06:40:49.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.245+0000 7f4123c00640 1 -- start start 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.246+0000 7f4123c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4100003f30 0x7f411c15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.246+0000 7f4123c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.246+0000 7f4123c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f411c152870 0x7f411c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f4123c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0ae4e0 con 0x7f411c150080 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f4123c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0ae200 con 0x7f411c152870 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f4123c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f411c06fbb0 con 0x7f4100003f30 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55782/0 (socket says 172.21.15.59:55782) 2024-04-24T06:40:49.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 -- 172.21.15.59:0/108331890 learned_addr learned my addr 172.21.15.59:0/108331890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:49.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 -- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4100003f30 msgr2=0x7f411c15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4100003f30 0x7f411c15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f4121c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f411c152870 0x7f411c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 -- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f411c152870 msgr2=0x7f411c152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f411c152870 0x7f411c152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.247+0000 7f411be00640 1 -- 172.21.15.59:0/108331890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c1602c0 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f411be00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4108034040 tx=0x7f4108004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108040020 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108003c60 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c1605a0 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108036040 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.248+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c160b60 con 0x7f411c150080 2024-04-24T06:40:49.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.249+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c068a80 con 0x7f411c150080 2024-04-24T06:40:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.250+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4108043020 con 0x7f411c150080 2024-04-24T06:40:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.251+0000 7f4118c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.252+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f41080abff0 con 0x7f411c150080 2024-04-24T06:40:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.252+0000 7f4121200640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.252+0000 7f4121200640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 0x7f40fc06ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f410c009fc0 tx=0x7f410c009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.256+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f410804a030 con 0x7f411c150080 2024-04-24T06:40:49.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-24T06:40:49.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:49.499+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f411c162190 con 0x7f411c150080 2024-04-24T06:40:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.127+0000 7f4118c00640 1 -- 172.21.15.59:0/108331890 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7f41080044e0 con 0x7f411c150080 2024-04-24T06:40:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 msgr2=0x7f40fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 0x7f40fc06ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f410c009fc0 tx=0x7f410c009750 comp rx=0 tx=0).stop 2024-04-24T06:40:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 msgr2=0x7f411c150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4108034040 tx=0x7f4108004730 comp rx=0 tx=0).stop 2024-04-24T06:40:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 shutdown_connections 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f40fc06c920 0x7f40fc06ede0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f411c152870 0x7f411c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f411c150080 0x7f411c150500 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 --2- 172.21.15.59:0/108331890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4100003f30 0x7f411c15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 >> 172.21.15.59:0/108331890 conn(0x7f411c067d90 msgr2=0x7f411c0b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.132+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 shutdown_connections 2024-04-24T06:40:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.133+0000 7f4123c00640 1 -- 172.21.15.59:0/108331890 wait complete. 2024-04-24T06:40:50.160 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T06:40:50.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T06:40:50.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.370+0000 7f0157200640 1 Processor -- start 2024-04-24T06:40:50.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.370+0000 7f0157200640 1 -- start start 2024-04-24T06:40:50.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.371+0000 7f0157200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f015815c720 0x7f01580cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.371+0000 7f0157200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.371+0000 7f0157200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015815cb00 con 0x7f01580cdf30 2024-04-24T06:40:50.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.371+0000 7f0157200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f01580d1430 con 0x7f015815c720 2024-04-24T06:40:50.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.371+0000 7f0157200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f015809aa00 con 0x7f01580d0c10 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.372+0000 7f0155e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f015815c720 0x7f01580cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.372+0000 7f0156800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.372+0000 7f014d400640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f01580cdf30 0x7f01580d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:48188/0 (socket says 172.21.15.59:48188) 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.372+0000 7f014d400640 1 -- 172.21.15.59:0/2032640548 learned_addr learned my addr 172.21.15.59:0/2032640548 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.372+0000 7f0156800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:40094/0 (socket says 172.21.15.59:40094) 2024-04-24T06:40:50.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.373+0000 7f0156800640 1 -- 172.21.15.59:0/2032640548 >> v2:172.21.15.66:3300/0 conn(0x7f015815c720 msgr2=0x7f01580cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:50.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.373+0000 7f0156800640 1 --2- 172.21.15.59:0/2032640548 >> v2:172.21.15.66:3300/0 conn(0x7f015815c720 0x7f01580cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.373+0000 7f0155400640 1 -- 172.21.15.59:0/2032640548 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671310877 0 0) 0x7f015815cb00 con 0x7f01580cdf30 2024-04-24T06:40:50.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.373+0000 7f0156800640 1 -- 172.21.15.59:0/2032640548 >> v1:172.21.15.66:6789/0 conn(0x7f01580cdf30 legacy=0x7f01580d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.373+0000 7f0156800640 1 -- 172.21.15.59:0/2032640548 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015809ac50 con 0x7f01580d0c10 2024-04-24T06:40:50.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.374+0000 7f0156800640 1 --2- 172.21.15.59:0/2032640548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0150009d90 tx=0x7f015002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9afcf5003fe15b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:50.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0155400640 1 -- 172.21.15.59:0/2032640548 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150007740 con 0x7f01580d0c10 2024-04-24T06:40:50.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0155400640 1 -- 172.21.15.59:0/2032640548 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01500078e0 con 0x7f01580d0c10 2024-04-24T06:40:50.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0155400640 1 -- 172.21.15.59:0/2032640548 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150007cb0 con 0x7f01580d0c10 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 -- 172.21.15.59:0/2032640548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 msgr2=0x7f01580b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 --2- 172.21.15.59:0/2032640548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0150009d90 tx=0x7f015002a660 comp rx=0 tx=0).stop 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 -- 172.21.15.59:0/2032640548 shutdown_connections 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 --2- 172.21.15.59:0/2032640548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580b3140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 --2- 172.21.15.59:0/2032640548 >> v2:172.21.15.66:3300/0 conn(0x7f015815c720 0x7f01580cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.375+0000 7f0157200640 1 -- 172.21.15.59:0/2032640548 >> 172.21.15.59:0/2032640548 conn(0x7f01580b3f70 msgr2=0x7f01580b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:50.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.376+0000 7f0157200640 1 -- 172.21.15.59:0/2032640548 shutdown_connections 2024-04-24T06:40:50.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.376+0000 7f0157200640 1 -- 172.21.15.59:0/2032640548 wait complete. 2024-04-24T06:40:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.377+0000 7f0157200640 1 Processor -- start 2024-04-24T06:40:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.377+0000 7f0157200640 1 -- start start 2024-04-24T06:40:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01580cdf30 0x7f015814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f015809b4f0 con 0x7f015815c720 2024-04-24T06:40:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f015809b210 con 0x7f01580cdf30 2024-04-24T06:40:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.378+0000 7f0157200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f015809b7d0 con 0x7f01580d0c10 2024-04-24T06:40:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55794/0 (socket says 172.21.15.59:55794) 2024-04-24T06:40:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 -- 172.21.15.59:0/3960279975 learned_addr learned my addr 172.21.15.59:0/3960279975 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0155e00640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01580cdf30 0x7f015814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f014d400640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 -- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 msgr2=0x7f01580ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 -- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01580cdf30 msgr2=0x7f015814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.379+0000 7f0156800640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01580cdf30 0x7f015814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f0156800640 1 -- 172.21.15.59:0/3960279975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01580cb220 con 0x7f015815c720 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f0156800640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0150009eb0 tx=0x7f0150004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150040070 con 0x7f015815c720 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150031bc0 con 0x7f015815c720 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150009450 con 0x7f015815c720 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01580cb500 con 0x7f015815c720 2024-04-24T06:40:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.380+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01580cbac0 con 0x7f015815c720 2024-04-24T06:40:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.382+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015804fd90 con 0x7f015815c720 2024-04-24T06:40:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.382+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0150039040 con 0x7f015815c720 2024-04-24T06:40:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.383+0000 7f014ea00640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.383+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f01500ad450 con 0x7f015815c720 2024-04-24T06:40:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.384+0000 7f0155e00640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.384+0000 7f0155e00640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 0x7f013406edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0148009a30 tx=0x7f0148008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.388+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f015007b7f0 con 0x7f015815c720 2024-04-24T06:40:50.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:50.636+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f01580d0500 con 0x7f015815c720 2024-04-24T06:40:51.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.131+0000 7f014ea00640 1 -- 172.21.15.59:0/3960279975 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f015007b0f0 con 0x7f015815c720 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.134+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 msgr2=0x7f013406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.134+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 0x7f013406edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f0148009a30 tx=0x7f0148008040 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.134+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 msgr2=0x7f01581e7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.134+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f0150009eb0 tx=0x7f0150004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 shutdown_connections 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f013406c8f0 0x7f013406edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f015815c720 0x7f01581e7d60 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f01580d0c10 0x7f01580ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 --2- 172.21.15.59:0/3960279975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01580cdf30 0x7f015814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 >> 172.21.15.59:0/3960279975 conn(0x7f01580b3f70 msgr2=0x7f01580cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 shutdown_connections 2024-04-24T06:40:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.135+0000 7f0157200640 1 -- 172.21.15.59:0/3960279975 wait complete. 2024-04-24T06:40:51.157 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T06:40:51.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:51.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.369+0000 7f3736400640 1 Processor -- start 2024-04-24T06:40:51.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.369+0000 7f3736400640 1 -- start start 2024-04-24T06:40:51.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.370+0000 7f3736400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f373015e7c0 0x7f37300d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.370+0000 7f3736400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37300d0d90 0x7f37300d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.370+0000 7f3736400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37300cf030 con 0x7f37300d2630 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.370+0000 7f3736400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f37300b7000 con 0x7f37300d0d90 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.370+0000 7f3736400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f37300cf620 con 0x7f373015e7c0 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37300d0d90 0x7f37300d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f372fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f37300d2630 0x7f37300b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58320/0 (socket says 172.21.15.59:58320) 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f372fe00640 1 -- 172.21.15.59:0/1122636015 learned_addr learned my addr 172.21.15.59:0/1122636015 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:51.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735000640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f373015e7c0 0x7f37300d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735a00640 1 -- 172.21.15.59:0/1122636015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f373015e7c0 msgr2=0x7f37300d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735a00640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f373015e7c0 0x7f37300d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735a00640 1 -- 172.21.15.59:0/1122636015 >> v1:172.21.15.59:6789/0 conn(0x7f37300d2630 legacy=0x7f37300b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.371+0000 7f3735a00640 1 -- 172.21.15.59:0/1122636015 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37300b6a40 con 0x7f37300d0d90 2024-04-24T06:40:51.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.372+0000 7f3735a00640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0d90 0x7f37300d1220 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f3728009d90 tx=0x7f372802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd067142efd5dc0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.373+0000 7f372f400640 1 -- 172.21.15.59:0/1122636015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728007740 con 0x7f37300d0d90 2024-04-24T06:40:51.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.373+0000 7f372f400640 1 -- 172.21.15.59:0/1122636015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37280078e0 con 0x7f37300d0d90 2024-04-24T06:40:51.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.373+0000 7f372f400640 1 -- 172.21.15.59:0/1122636015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728007cb0 con 0x7f37300d0d90 2024-04-24T06:40:51.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.373+0000 7f3736400640 1 -- 172.21.15.59:0/1122636015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0d90 msgr2=0x7f37300d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.373+0000 7f3736400640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0d90 0x7f37300d1220 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f3728009d90 tx=0x7f372802a660 comp rx=0 tx=0).stop 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 -- 172.21.15.59:0/1122636015 shutdown_connections 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0d90 0x7f37300d1220 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 --2- 172.21.15.59:0/1122636015 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f373015e7c0 0x7f37300d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 -- 172.21.15.59:0/1122636015 >> 172.21.15.59:0/1122636015 conn(0x7f373009d920 msgr2=0x7f373008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 -- 172.21.15.59:0/1122636015 shutdown_connections 2024-04-24T06:40:51.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.374+0000 7f3736400640 1 -- 172.21.15.59:0/1122636015 wait complete. 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.375+0000 7f3736400640 1 Processor -- start 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.376+0000 7f3736400640 1 -- start start 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.376+0000 7f3736400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37300d0d90 0x7f373014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.376+0000 7f3736400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37300d2630 0x7f37301f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3736400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3736400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37300b6700 con 0x7f373015e7c0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3736400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37300cf2b0 con 0x7f37300d0d90 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3736400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f37300d1a80 con 0x7f37300d2630 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55822/0 (socket says 172.21.15.59:55822) 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 -- 172.21.15.59:0/2718220244 learned_addr learned my addr 172.21.15.59:0/2718220244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 -- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37300d2630 msgr2=0x7f37301f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37300d2630 0x7f37301f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 -- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37300d0d90 msgr2=0x7f373014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37300d0d90 0x7f373014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.377+0000 7f3735a00640 1 -- 172.21.15.59:0/2718220244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301f4b40 con 0x7f373015e7c0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f3735a00640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3728009eb0 tx=0x7f3728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728040070 con 0x7f373015e7c0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3728031bc0 con 0x7f373015e7c0 2024-04-24T06:40:51.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728045460 con 0x7f373015e7c0 2024-04-24T06:40:51.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301f4e20 con 0x7f373015e7c0 2024-04-24T06:40:51.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.378+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301f56e0 con 0x7f373015e7c0 2024-04-24T06:40:51.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.379+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373004fd90 con 0x7f373015e7c0 2024-04-24T06:40:51.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.380+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3728039040 con 0x7f373015e7c0 2024-04-24T06:40:51.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.381+0000 7f372cc00640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 0x7f370c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.381+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f37280ad570 con 0x7f373015e7c0 2024-04-24T06:40:51.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.382+0000 7f3735000640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 0x7f370c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.382+0000 7f3735000640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 0x7f370c06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3720006fd0 tx=0x7f3720008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.386+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f372807b900 con 0x7f373015e7c0 2024-04-24T06:40:51.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.631+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37300d1eb0 con 0x7f373015e7c0 2024-04-24T06:40:51.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.633+0000 7f372cc00640 1 -- 172.21.15.59:0/2718220244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5182 (secure 0 0 0) 0x7f37280abe20 con 0x7f373015e7c0 2024-04-24T06:40:51.634 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:51.634 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:40:50.638009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9228":{"gid":9228,"name":"b","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2441853137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2441853137},{"type":"v1","addr":"172.21.15.66:6835","nonce":2441853137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9228,"qdb_cluster":[9228]},"id":8}]} 2024-04-24T06:40:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 100 2024-04-24T06:40:51.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.635+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 msgr2=0x7f370c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.635+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 0x7f370c06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3720006fd0 tx=0x7f3720008040 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.635+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 msgr2=0x7f37300b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.635+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3728009eb0 tx=0x7f3728004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 shutdown_connections 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f370c06c8f0 0x7f370c06edb0 secure :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3720006fd0 tx=0x7f3720008040 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015e7c0 0x7f37300b21f0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37300d2630 0x7f37301f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 --2- 172.21.15.59:0/2718220244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37300d0d90 0x7f373014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 >> 172.21.15.59:0/2718220244 conn(0x7f373009d920 msgr2=0x7f373015f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 shutdown_connections 2024-04-24T06:40:51.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.636+0000 7f3736400640 1 -- 172.21.15.59:0/2718220244 wait complete. 2024-04-24T06:40:51.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.856+0000 7fb992e00640 1 Processor -- start 2024-04-24T06:40:51.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.857+0000 7fb992e00640 1 -- start start 2024-04-24T06:40:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c15e7c0 0x7fb98c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 0x7fb98c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c0b6530 0x7fb98c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c0b7000 con 0x7fb98c0b6530 2024-04-24T06:40:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c0cf620 con 0x7fb98c0d2630 2024-04-24T06:40:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c0cf030 con 0x7fb98c15e7c0 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c0b6530 0x7fb98c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c0b6530 0x7fb98c0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55836/0 (socket says 172.21.15.59:55836) 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.858+0000 7fb992400640 1 -- 172.21.15.59:0/896763278 learned_addr learned my addr 172.21.15.59:0/896763278 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb991000640 1 --2- 172.21.15.59:0/896763278 >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 0x7fb98c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb992400640 1 -- 172.21.15.59:0/896763278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c15e7c0 msgr2=0x7fb98c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb991a00640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c15e7c0 0x7fb98c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb992400640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c15e7c0 0x7fb98c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb992400640 1 -- 172.21.15.59:0/896763278 >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 msgr2=0x7fb98c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:51.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb992400640 1 --2- 172.21.15.59:0/896763278 >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 0x7fb98c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb992400640 1 -- 172.21.15.59:0/896763278 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c0b6a40 con 0x7fb98c0b6530 2024-04-24T06:40:51.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.859+0000 7fb991000640 1 --2- 172.21.15.59:0/896763278 >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 0x7fb98c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.860+0000 7fb992400640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0b6530 0x7fb98c0d0d90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0b71a0 tx=0x7fb98402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb7136ba5df0313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.860+0000 7fb98be00640 1 -- 172.21.15.59:0/896763278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984036070 con 0x7fb98c0b6530 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb98be00640 1 -- 172.21.15.59:0/896763278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb984031600 con 0x7fb98c0b6530 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb98be00640 1 -- 172.21.15.59:0/896763278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840319d0 con 0x7fb98c0b6530 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 -- 172.21.15.59:0/896763278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0b6530 msgr2=0x7fb98c0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0b6530 0x7fb98c0d0d90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0b71a0 tx=0x7fb98402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 -- 172.21.15.59:0/896763278 shutdown_connections 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0b6530 0x7fb98c0d0d90 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 --2- 172.21.15.59:0/896763278 >> v2:172.21.15.66:3300/0 conn(0x7fb98c0d2630 0x7fb98c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 --2- 172.21.15.59:0/896763278 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c15e7c0 0x7fb98c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 -- 172.21.15.59:0/896763278 >> 172.21.15.59:0/896763278 conn(0x7fb98c09d920 msgr2=0x7fb98c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 -- 172.21.15.59:0/896763278 shutdown_connections 2024-04-24T06:40:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.861+0000 7fb992e00640 1 -- 172.21.15.59:0/896763278 wait complete. 2024-04-24T06:40:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.863+0000 7fb992e00640 1 Processor -- start 2024-04-24T06:40:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.863+0000 7fb992e00640 1 -- start start 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 0x7fb98c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c0d2630 0x7fb98c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0d1a80 con 0x7fb98c15e7c0 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0cf2b0 con 0x7fb98c0b6530 2024-04-24T06:40:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.864+0000 7fb992e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c0d1c00 con 0x7fb98c0d2630 2024-04-24T06:40:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55844/0 (socket says 172.21.15.59:55844) 2024-04-24T06:40:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 -- 172.21.15.59:0/2731136035 learned_addr learned my addr 172.21.15.59:0/2731136035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb991a00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 0x7fb98c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb991000640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c0d2630 0x7fb98c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 -- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c0d2630 msgr2=0x7fb98c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c0d2630 0x7fb98c15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 -- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 msgr2=0x7fb98c0b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 0x7fb98c0b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 -- 172.21.15.59:0/2731136035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1f8870 con 0x7fb98c15e7c0 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb991a00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 0x7fb98c0b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:51.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.865+0000 7fb992400640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb984034040 tx=0x7fb984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.866+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984041020 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.866+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9840077a0 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.866+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984036040 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.866+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c1f8b50 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.866+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1f9110 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.867+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c04fd90 con 0x7fb98c15e7c0 2024-04-24T06:40:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.868+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb984044020 con 0x7fb98c15e7c0 2024-04-24T06:40:51.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.869+0000 7fb989600640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:51.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.869+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fb9840acb60 con 0x7fb98c15e7c0 2024-04-24T06:40:51.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.869+0000 7fb991a00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:51.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.870+0000 7fb991a00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 0x7fb96c06edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00b800 tx=0x7fb97c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:51.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:51.874+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb98404b030 con 0x7fb98c15e7c0 2024-04-24T06:40:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.079+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98c15d9a0 con 0x7fb98c15e7c0 2024-04-24T06:40:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.083+0000 7fb989600640 1 -- 172.21.15.59:0/2731136035 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16543 (secure 0 0 0) 0x7fb98407afb0 con 0x7fb98c15e7c0 2024-04-24T06:40:52.084 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:52.084 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:49.106564+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:52.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 msgr2=0x7fb96c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 0x7fb96c06edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00b800 tx=0x7fb97c008040 comp rx=0 tx=0).stop 2024-04-24T06:40:52.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 msgr2=0x7fb98c0b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb984034040 tx=0x7fb984004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 shutdown_connections 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15e7c0 0x7fb98c0b7b90 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb98c0d2630 0x7fb98c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.085+0000 7fb992e00640 1 --2- 172.21.15.59:0/2731136035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb98c0b6530 0x7fb98c0b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.086+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 >> 172.21.15.59:0/2731136035 conn(0x7fb98c09d920 msgr2=0x7fb98c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.086+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 shutdown_connections 2024-04-24T06:40:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.086+0000 7fb992e00640 1 -- 172.21.15.59:0/2731136035 wait complete. 2024-04-24T06:40:52.105 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:40:52.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:40:52.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.305+0000 7ffb5f200640 1 Processor -- start 2024-04-24T06:40:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.306+0000 7ffb5f200640 1 -- start start 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6015b450 0x7ffb600b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb5f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffb600cd760 0x7ffb600ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb5f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb600b3870 con 0x7ffb6006f3f0 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb5f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffb600b41a0 con 0x7ffb600cd760 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb5f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffb600b3b50 con 0x7ffb6015b450 2024-04-24T06:40:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb55400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffb6006f3f0 0x7ffb600cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58336/0 (socket says 172.21.15.59:58336) 2024-04-24T06:40:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.307+0000 7ffb55400640 1 -- 172.21.15.59:0/3134965509 learned_addr learned my addr 172.21.15.59:0/3134965509 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3134965509 >> v2:172.21.15.59:3300/0 conn(0x7ffb600cd760 0x7ffb600ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5de00640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6015b450 0x7ffb600b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5d400640 1 -- 172.21.15.59:0/3134965509 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 303025210 0 0) 0x7ffb600b3870 con 0x7ffb6006f3f0 2024-04-24T06:40:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5d400640 1 -- 172.21.15.59:0/3134965509 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb44003520 con 0x7ffb6006f3f0 2024-04-24T06:40:52.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5e800640 1 -- 172.21.15.59:0/3134965509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6015b450 msgr2=0x7ffb600b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:52.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6015b450 0x7ffb600b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5e800640 1 -- 172.21.15.59:0/3134965509 >> v1:172.21.15.59:6789/0 conn(0x7ffb6006f3f0 legacy=0x7ffb600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.308+0000 7ffb5e800640 1 -- 172.21.15.59:0/3134965509 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb600b3e30 con 0x7ffb600cd760 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.309+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb600cd760 0x7ffb600ad060 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009d90 tx=0x7ffb5802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb1c1af3f7f3dce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5d400640 1 -- 172.21.15.59:0/3134965509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58007740 con 0x7ffb600cd760 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5d400640 1 -- 172.21.15.59:0/3134965509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb580078e0 con 0x7ffb600cd760 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5d400640 1 -- 172.21.15.59:0/3134965509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58007cb0 con 0x7ffb600cd760 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 -- 172.21.15.59:0/3134965509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb600cd760 msgr2=0x7ffb600ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb600cd760 0x7ffb600ad060 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009d90 tx=0x7ffb5802a660 comp rx=0 tx=0).stop 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 -- 172.21.15.59:0/3134965509 shutdown_connections 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb600cd760 0x7ffb600ad060 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3134965509 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6015b450 0x7ffb600b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 -- 172.21.15.59:0/3134965509 >> 172.21.15.59:0/3134965509 conn(0x7ffb600b21a0 msgr2=0x7ffb600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 -- 172.21.15.59:0/3134965509 shutdown_connections 2024-04-24T06:40:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.310+0000 7ffb5f200640 1 -- 172.21.15.59:0/3134965509 wait complete. 2024-04-24T06:40:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.312+0000 7ffb5f200640 1 Processor -- start 2024-04-24T06:40:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.312+0000 7ffb5f200640 1 -- start start 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6006f3f0 0x7ffb600c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 0x7ffb600c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600b4ff0 con 0x7ffb6015b450 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6015b830 con 0x7ffb600cd760 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5f200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffb600b3080 con 0x7ffb6006f3f0 2024-04-24T06:40:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55860/0 (socket says 172.21.15.59:55860) 2024-04-24T06:40:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.313+0000 7ffb5e800640 1 -- 172.21.15.59:0/3480762548 learned_addr learned my addr 172.21.15.59:0/3480762548 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5de00640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6006f3f0 0x7ffb600c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb55400640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 0x7ffb600c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 -- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6006f3f0 msgr2=0x7ffb600c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6006f3f0 0x7ffb600c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 -- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 msgr2=0x7ffb600c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 0x7ffb600c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 -- 172.21.15.59:0/3480762548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb600b0540 con 0x7ffb6015b450 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb55400640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 0x7ffb600c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.314+0000 7ffb5e800640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009eb0 tx=0x7ffb58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.315+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58040070 con 0x7ffb6015b450 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.315+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb58031bc0 con 0x7ffb6015b450 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.315+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58009450 con 0x7ffb6015b450 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.315+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb600b0820 con 0x7ffb6015b450 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.315+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb600b0de0 con 0x7ffb6015b450 2024-04-24T06:40:52.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.316+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6004fd90 con 0x7ffb6015b450 2024-04-24T06:40:52.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.317+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb58039040 con 0x7ffb6015b450 2024-04-24T06:40:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.318+0000 7ffb56a00640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.318+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7ffb580ad4d0 con 0x7ffb6015b450 2024-04-24T06:40:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.318+0000 7ffb5de00640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.319+0000 7ffb5de00640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffb50006fd0 tx=0x7ffb50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.322+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb5807b860 con 0x7ffb6015b450 2024-04-24T06:40:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.522+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb600c44d0 con 0x7ffb3c06c8f0 2024-04-24T06:40:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.541+0000 7ffb56a00640 1 -- 172.21.15.59:0/3480762548 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252415 (secure 0 0 0) 0x7ffb600c44d0 con 0x7ffb3c06c8f0 2024-04-24T06:40:52.546 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.546+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 msgr2=0x7ffb3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.546+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffb50006fd0 tx=0x7ffb50008040 comp rx=0 tx=0).stop 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.546+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 msgr2=0x7ffb6015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.546+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009eb0 tx=0x7ffb58004a40 comp rx=0 tx=0).stop 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 shutdown_connections 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb6015b450 0x7ffb6015e3c0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb600cd760 0x7ffb600c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 --2- 172.21.15.59:0/3480762548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb6006f3f0 0x7ffb600c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 >> 172.21.15.59:0/3480762548 conn(0x7ffb600b21a0 msgr2=0x7ffb600ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 shutdown_connections 2024-04-24T06:40:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.547+0000 7ffb5f200640 1 -- 172.21.15.59:0/3480762548 wait complete. 2024-04-24T06:40:52.568 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-04-24T06:40:51.445952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261188,"kb_used_data":11720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713532,"statfs":{"total":773094113280,"available":772826656768,"internally_reserved":0,"allocated":12001280,"data_stored":5849109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.391542"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.128859+0000","last_change":"2024-04-24T06:40:48.126867+0000","last_active":"2024-04-24T06:40:48.128859+0000","last_peered":"2024-04-24T06:40:48.128859+0000","last_clean":"2024-04-24T06:40:48.128859+0000","last_became_active":"2024-04-24T06:40:48.125645+0000","last_became_peered":"2024-04-24T06:40:48.125645+0000","last_unstale":"2024-04-24T06:40:48.128859+0000","last_undegraded":"2024-04-24T06:40:48.128859+0000","last_fullsized":"2024-04-24T06:40:48.128859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:28:04.665828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127140+0000","last_change":"2024-04-24T06:40:48.110584+0000","last_active":"2024-04-24T06:40:48.127140+0000","last_peered":"2024-04-24T06:40:48.127140+0000","last_clean":"2024-04-24T06:40:48.127140+0000","last_became_active":"2024-04-24T06:40:48.109829+0000","last_became_peered":"2024-04-24T06:40:48.109829+0000","last_unstale":"2024-04-24T06:40:48.127140+0000","last_undegraded":"2024-04-24T06:40:48.127140+0000","last_fullsized":"2024-04-24T06:40:48.127140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:46:15.073278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137640+0000","last_change":"2024-04-24T06:40:48.133984+0000","last_active":"2024-04-24T06:40:48.137640+0000","last_peered":"2024-04-24T06:40:48.137640+0000","last_clean":"2024-04-24T06:40:48.137640+0000","last_became_active":"2024-04-24T06:40:48.133634+0000","last_became_peered":"2024-04-24T06:40:48.133634+0000","last_unstale":"2024-04-24T06:40:48.137640+0000","last_undegraded":"2024-04-24T06:40:48.137640+0000","last_fullsized":"2024-04-24T06:40:48.137640+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.115815+0000","last_change":"2024-04-24T06:40:48.111574+0000","last_active":"2024-04-24T06:40:48.115815+0000","last_peered":"2024-04-24T06:40:48.115815+0000","last_clean":"2024-04-24T06:40:48.115815+0000","last_became_active":"2024-04-24T06:40:48.111275+0000","last_became_peered":"2024-04-24T06:40:48.111275+0000","last_unstale":"2024-04-24T06:40:48.115815+0000","last_undegraded":"2024-04-24T06:40:48.115815+0000","last_fullsized":"2024-04-24T06:40:48.115815+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:04:45.054578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.124037+0000","last_change":"2024-04-24T06:40:48.118794+0000","last_active":"2024-04-24T06:40:48.124037+0000","last_peered":"2024-04-24T06:40:48.124037+0000","last_clean":"2024-04-24T06:40:48.124037+0000","last_became_active":"2024-04-24T06:40:48.118488+0000","last_became_peered":"2024-04-24T06:40:48.118488+0000","last_unstale":"2024-04-24T06:40:48.124037+0000","last_undegraded":"2024-04-24T06:40:48.124037+0000","last_fullsized":"2024-04-24T06:40:48.124037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:01:04.287043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126568+0000","last_change":"2024-04-24T06:40:48.118094+0000","last_active":"2024-04-24T06:40:48.126568+0000","last_peered":"2024-04-24T06:40:48.126568+0000","last_clean":"2024-04-24T06:40:48.126568+0000","last_became_active":"2024-04-24T06:40:48.117819+0000","last_became_peered":"2024-04-24T06:40:48.117819+0000","last_unstale":"2024-04-24T06:40:48.126568+0000","last_undegraded":"2024-04-24T06:40:48.126568+0000","last_fullsized":"2024-04-24T06:40:48.126568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:14.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126139+0000","last_change":"2024-04-24T06:40:48.124361+0000","last_active":"2024-04-24T06:40:48.126139+0000","last_peered":"2024-04-24T06:40:48.126139+0000","last_clean":"2024-04-24T06:40:48.126139+0000","last_became_active":"2024-04-24T06:40:48.123999+0000","last_became_peered":"2024-04-24T06:40:48.123999+0000","last_unstale":"2024-04-24T06:40:48.126139+0000","last_undegraded":"2024-04-24T06:40:48.126139+0000","last_fullsized":"2024-04-24T06:40:48.126139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:09.151981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.139509+0000","last_change":"2024-04-24T06:40:48.137338+0000","last_active":"2024-04-24T06:40:48.139509+0000","last_peered":"2024-04-24T06:40:48.139509+0000","last_clean":"2024-04-24T06:40:48.139509+0000","last_became_active":"2024-04-24T06:40:48.135891+0000","last_became_peered":"2024-04-24T06:40:48.135891+0000","last_unstale":"2024-04-24T06:40:48.139509+0000","last_undegraded":"2024-04-24T06:40:48.139509+0000","last_fullsized":"2024-04-24T06:40:48.139509+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122441+0000","last_change":"2024-04-24T06:40:48.120791+0000","last_active":"2024-04-24T06:40:48.122441+0000","last_peered":"2024-04-24T06:40:48.122441+0000","last_clean":"2024-04-24T06:40:48.122441+0000","last_became_active":"2024-04-24T06:40:48.120447+0000","last_became_peered":"2024-04-24T06:40:48.120447+0000","last_unstale":"2024-04-24T06:40:48.122441+0000","last_undegraded":"2024-04-24T06:40:48.122441+0000","last_fullsized":"2024-04-24T06:40:48.122441+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:26:28.453024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.123396+0000","last_change":"2024-04-24T06:40:48.119795+0000","last_active":"2024-04-24T06:40:48.123396+0000","last_peered":"2024-04-24T06:40:48.123396+0000","last_clean":"2024-04-24T06:40:48.123396+0000","last_became_active":"2024-04-24T06:40:48.119148+0000","last_became_peered":"2024-04-24T06:40:48.119148+0000","last_unstale":"2024-04-24T06:40:48.123396+0000","last_undegraded":"2024-04-24T06:40:48.123396+0000","last_fullsized":"2024-04-24T06:40:48.123396+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:17:23.083995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129401+0000","last_change":"2024-04-24T06:40:48.126500+0000","last_active":"2024-04-24T06:40:48.129401+0000","last_peered":"2024-04-24T06:40:48.129401+0000","last_clean":"2024-04-24T06:40:48.129401+0000","last_became_active":"2024-04-24T06:40:48.126103+0000","last_became_peered":"2024-04-24T06:40:48.126103+0000","last_unstale":"2024-04-24T06:40:48.129401+0000","last_undegraded":"2024-04-24T06:40:48.129401+0000","last_fullsized":"2024-04-24T06:40:48.129401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122001+0000","last_change":"2024-04-24T06:40:48.119612+0000","last_active":"2024-04-24T06:40:48.122001+0000","last_peered":"2024-04-24T06:40:48.122001+0000","last_clean":"2024-04-24T06:40:48.122001+0000","last_became_active":"2024-04-24T06:40:48.119271+0000","last_became_peered":"2024-04-24T06:40:48.119271+0000","last_unstale":"2024-04-24T06:40:48.122001+0000","last_undegraded":"2024-04-24T06:40:48.122001+0000","last_fullsized":"2024-04-24T06:40:48.122001+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:06.695374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135989+0000","last_change":"2024-04-24T06:40:48.132345+0000","last_active":"2024-04-24T06:40:48.135989+0000","last_peered":"2024-04-24T06:40:48.135989+0000","last_clean":"2024-04-24T06:40:48.135989+0000","last_became_active":"2024-04-24T06:40:48.132069+0000","last_became_peered":"2024-04-24T06:40:48.132069+0000","last_unstale":"2024-04-24T06:40:48.135989+0000","last_undegraded":"2024-04-24T06:40:48.135989+0000","last_fullsized":"2024-04-24T06:40:48.135989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:08.906752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137139+0000","last_change":"2024-04-24T06:40:48.134106+0000","last_active":"2024-04-24T06:40:48.137139+0000","last_peered":"2024-04-24T06:40:48.137139+0000","last_clean":"2024-04-24T06:40:48.137139+0000","last_became_active":"2024-04-24T06:40:48.133787+0000","last_became_peered":"2024-04-24T06:40:48.133787+0000","last_unstale":"2024-04-24T06:40:48.137139+0000","last_undegraded":"2024-04-24T06:40:48.137139+0000","last_fullsized":"2024-04-24T06:40:48.137139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126042+0000","last_change":"2024-04-24T06:40:48.118609+0000","last_active":"2024-04-24T06:40:48.126042+0000","last_peered":"2024-04-24T06:40:48.126042+0000","last_clean":"2024-04-24T06:40:48.126042+0000","last_became_active":"2024-04-24T06:40:48.118319+0000","last_became_peered":"2024-04-24T06:40:48.118319+0000","last_unstale":"2024-04-24T06:40:48.126042+0000","last_undegraded":"2024-04-24T06:40:48.126042+0000","last_fullsized":"2024-04-24T06:40:48.126042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:14:12.164105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.142214+0000","last_change":"2024-04-24T06:40:48.135651+0000","last_active":"2024-04-24T06:40:48.142214+0000","last_peered":"2024-04-24T06:40:48.142214+0000","last_clean":"2024-04-24T06:40:48.142214+0000","last_became_active":"2024-04-24T06:40:48.135089+0000","last_became_peered":"2024-04-24T06:40:48.135089+0000","last_unstale":"2024-04-24T06:40:48.142214+0000","last_undegraded":"2024-04-24T06:40:48.142214+0000","last_fullsized":"2024-04-24T06:40:48.142214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:00:46.334314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.131715+0000","last_change":"2024-04-24T06:40:48.128636+0000","last_active":"2024-04-24T06:40:48.131715+0000","last_peered":"2024-04-24T06:40:48.131715+0000","last_clean":"2024-04-24T06:40:48.131715+0000","last_became_active":"2024-04-24T06:40:48.128286+0000","last_became_peered":"2024-04-24T06:40:48.128286+0000","last_unstale":"2024-04-24T06:40:48.131715+0000","last_undegraded":"2024-04-24T06:40:48.131715+0000","last_fullsized":"2024-04-24T06:40:48.131715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.134370+0000","last_change":"2024-04-24T06:40:48.129619+0000","last_active":"2024-04-24T06:40:48.134370+0000","last_peered":"2024-04-24T06:40:48.134370+0000","last_clean":"2024-04-24T06:40:48.134370+0000","last_became_active":"2024-04-24T06:40:48.129168+0000","last_became_peered":"2024-04-24T06:40:48.129168+0000","last_unstale":"2024-04-24T06:40:48.134370+0000","last_undegraded":"2024-04-24T06:40:48.134370+0000","last_fullsized":"2024-04-24T06:40:48.134370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137622+0000","last_change":"2024-04-24T06:40:48.133208+0000","last_active":"2024-04-24T06:40:48.137622+0000","last_peered":"2024-04-24T06:40:48.137622+0000","last_clean":"2024-04-24T06:40:48.137622+0000","last_became_active":"2024-04-24T06:40:48.132883+0000","last_became_peered":"2024-04-24T06:40:48.132883+0000","last_unstale":"2024-04-24T06:40:48.137622+0000","last_undegraded":"2024-04-24T06:40:48.137622+0000","last_fullsized":"2024-04-24T06:40:48.137622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:51.908947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129883+0000","last_change":"2024-04-24T06:40:48.122684+0000","last_active":"2024-04-24T06:40:48.129883+0000","last_peered":"2024-04-24T06:40:48.129883+0000","last_clean":"2024-04-24T06:40:48.129883+0000","last_became_active":"2024-04-24T06:40:48.122325+0000","last_became_peered":"2024-04-24T06:40:48.122325+0000","last_unstale":"2024-04-24T06:40:48.129883+0000","last_undegraded":"2024-04-24T06:40:48.129883+0000","last_fullsized":"2024-04-24T06:40:48.129883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:51.908947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129120+0000","last_change":"2024-04-24T06:40:48.118989+0000","last_active":"2024-04-24T06:40:48.129120+0000","last_peered":"2024-04-24T06:40:48.129120+0000","last_clean":"2024-04-24T06:40:48.129120+0000","last_became_active":"2024-04-24T06:40:48.118685+0000","last_became_peered":"2024-04-24T06:40:48.118685+0000","last_unstale":"2024-04-24T06:40:48.129120+0000","last_undegraded":"2024-04-24T06:40:48.129120+0000","last_fullsized":"2024-04-24T06:40:48.129120+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:08.906752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125148+0000","last_change":"2024-04-24T06:40:48.123620+0000","last_active":"2024-04-24T06:40:48.125148+0000","last_peered":"2024-04-24T06:40:48.125148+0000","last_clean":"2024-04-24T06:40:48.125148+0000","last_became_active":"2024-04-24T06:40:48.123309+0000","last_became_peered":"2024-04-24T06:40:48.123309+0000","last_unstale":"2024-04-24T06:40:48.125148+0000","last_undegraded":"2024-04-24T06:40:48.125148+0000","last_fullsized":"2024-04-24T06:40:48.125148+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:54:15.220666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129359+0000","last_change":"2024-04-24T06:40:48.125429+0000","last_active":"2024-04-24T06:40:48.129359+0000","last_peered":"2024-04-24T06:40:48.129359+0000","last_clean":"2024-04-24T06:40:48.129359+0000","last_became_active":"2024-04-24T06:40:48.125024+0000","last_became_peered":"2024-04-24T06:40:48.125024+0000","last_unstale":"2024-04-24T06:40:48.129359+0000","last_undegraded":"2024-04-24T06:40:48.129359+0000","last_fullsized":"2024-04-24T06:40:48.129359+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130924+0000","last_change":"2024-04-24T06:40:48.128637+0000","last_active":"2024-04-24T06:40:48.130924+0000","last_peered":"2024-04-24T06:40:48.130924+0000","last_clean":"2024-04-24T06:40:48.130924+0000","last_became_active":"2024-04-24T06:40:48.128085+0000","last_became_peered":"2024-04-24T06:40:48.128085+0000","last_unstale":"2024-04-24T06:40:48.130924+0000","last_undegraded":"2024-04-24T06:40:48.130924+0000","last_fullsized":"2024-04-24T06:40:48.130924+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:36:48.126891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130025+0000","last_change":"2024-04-24T06:40:48.127656+0000","last_active":"2024-04-24T06:40:48.130025+0000","last_peered":"2024-04-24T06:40:48.130025+0000","last_clean":"2024-04-24T06:40:48.130025+0000","last_became_active":"2024-04-24T06:40:48.127317+0000","last_became_peered":"2024-04-24T06:40:48.127317+0000","last_unstale":"2024-04-24T06:40:48.130025+0000","last_undegraded":"2024-04-24T06:40:48.130025+0000","last_fullsized":"2024-04-24T06:40:48.130025+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130987+0000","last_change":"2024-04-24T06:40:48.128071+0000","last_active":"2024-04-24T06:40:48.130987+0000","last_peered":"2024-04-24T06:40:48.130987+0000","last_clean":"2024-04-24T06:40:48.130987+0000","last_became_active":"2024-04-24T06:40:48.127765+0000","last_became_peered":"2024-04-24T06:40:48.127765+0000","last_unstale":"2024-04-24T06:40:48.130987+0000","last_undegraded":"2024-04-24T06:40:48.130987+0000","last_fullsized":"2024-04-24T06:40:48.130987+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:32.689270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126022+0000","last_change":"2024-04-24T06:40:48.115437+0000","last_active":"2024-04-24T06:40:48.126022+0000","last_peered":"2024-04-24T06:40:48.126022+0000","last_clean":"2024-04-24T06:40:48.126022+0000","last_became_active":"2024-04-24T06:40:48.115112+0000","last_became_peered":"2024-04-24T06:40:48.115112+0000","last_unstale":"2024-04-24T06:40:48.126022+0000","last_undegraded":"2024-04-24T06:40:48.126022+0000","last_fullsized":"2024-04-24T06:40:48.126022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:16:21.372839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122122+0000","last_change":"2024-04-24T06:40:48.116711+0000","last_active":"2024-04-24T06:40:48.122122+0000","last_peered":"2024-04-24T06:40:48.122122+0000","last_clean":"2024-04-24T06:40:48.122122+0000","last_became_active":"2024-04-24T06:40:48.116405+0000","last_became_peered":"2024-04-24T06:40:48.116405+0000","last_unstale":"2024-04-24T06:40:48.122122+0000","last_undegraded":"2024-04-24T06:40:48.122122+0000","last_fullsized":"2024-04-24T06:40:48.122122+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:15.580661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130744+0000","last_change":"2024-04-24T06:40:48.128764+0000","last_active":"2024-04-24T06:40:48.130744+0000","last_peered":"2024-04-24T06:40:48.130744+0000","last_clean":"2024-04-24T06:40:48.130744+0000","last_became_active":"2024-04-24T06:40:48.128461+0000","last_became_peered":"2024-04-24T06:40:48.128461+0000","last_unstale":"2024-04-24T06:40:48.130744+0000","last_undegraded":"2024-04-24T06:40:48.130744+0000","last_fullsized":"2024-04-24T06:40:48.130744+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.128818+0000","last_change":"2024-04-24T06:40:48.124981+0000","last_active":"2024-04-24T06:40:48.128818+0000","last_peered":"2024-04-24T06:40:48.128818+0000","last_clean":"2024-04-24T06:40:48.128818+0000","last_became_active":"2024-04-24T06:40:48.122688+0000","last_became_peered":"2024-04-24T06:40:48.122688+0000","last_unstale":"2024-04-24T06:40:48.128818+0000","last_undegraded":"2024-04-24T06:40:48.128818+0000","last_fullsized":"2024-04-24T06:40:48.128818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:35.196169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133060+0000","last_change":"2024-04-24T06:40:48.128061+0000","last_active":"2024-04-24T06:40:48.133060+0000","last_peered":"2024-04-24T06:40:48.133060+0000","last_clean":"2024-04-24T06:40:48.133060+0000","last_became_active":"2024-04-24T06:40:48.127698+0000","last_became_peered":"2024-04-24T06:40:48.127698+0000","last_unstale":"2024-04-24T06:40:48.133060+0000","last_undegraded":"2024-04-24T06:40:48.133060+0000","last_fullsized":"2024-04-24T06:40:48.133060+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:24.935770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126015+0000","last_change":"2024-04-24T06:40:48.115874+0000","last_active":"2024-04-24T06:40:48.126015+0000","last_peered":"2024-04-24T06:40:48.126015+0000","last_clean":"2024-04-24T06:40:48.126015+0000","last_became_active":"2024-04-24T06:40:48.115594+0000","last_became_peered":"2024-04-24T06:40:48.115594+0000","last_unstale":"2024-04-24T06:40:48.126015+0000","last_undegraded":"2024-04-24T06:40:48.126015+0000","last_fullsized":"2024-04-24T06:40:48.126015+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:16:54.481838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126630+0000","last_change":"2024-04-24T06:40:48.123127+0000","last_active":"2024-04-24T06:40:48.126630+0000","last_peered":"2024-04-24T06:40:48.126630+0000","last_clean":"2024-04-24T06:40:48.126630+0000","last_became_active":"2024-04-24T06:40:48.121282+0000","last_became_peered":"2024-04-24T06:40:48.121282+0000","last_unstale":"2024-04-24T06:40:48.126630+0000","last_undegraded":"2024-04-24T06:40:48.126630+0000","last_fullsized":"2024-04-24T06:40:48.126630+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:09.151981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"109'4","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.142568+0000","last_change":"2024-04-24T06:40:47.089244+0000","last_active":"2024-04-24T06:40:48.142568+0000","last_peered":"2024-04-24T06:40:48.142568+0000","last_clean":"2024-04-24T06:40:48.142568+0000","last_became_active":"2024-04-24T06:40:47.087988+0000","last_became_peered":"2024-04-24T06:40:47.087988+0000","last_unstale":"2024-04-24T06:40:48.142568+0000","last_undegraded":"2024-04-24T06:40:48.142568+0000","last_fullsized":"2024-04-24T06:40:48.142568+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:16:53.466013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.136741+0000","last_change":"2024-04-24T06:40:48.134224+0000","last_active":"2024-04-24T06:40:48.136741+0000","last_peered":"2024-04-24T06:40:48.136741+0000","last_clean":"2024-04-24T06:40:48.136741+0000","last_became_active":"2024-04-24T06:40:48.133923+0000","last_became_peered":"2024-04-24T06:40:48.133923+0000","last_unstale":"2024-04-24T06:40:48.136741+0000","last_undegraded":"2024-04-24T06:40:48.136741+0000","last_fullsized":"2024-04-24T06:40:48.136741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143194+0000","last_change":"2024-04-24T06:40:47.091853+0000","last_active":"2024-04-24T06:40:48.143194+0000","last_peered":"2024-04-24T06:40:48.143194+0000","last_clean":"2024-04-24T06:40:48.143194+0000","last_became_active":"2024-04-24T06:40:47.091515+0000","last_became_peered":"2024-04-24T06:40:47.091515+0000","last_unstale":"2024-04-24T06:40:48.143194+0000","last_undegraded":"2024-04-24T06:40:48.143194+0000","last_fullsized":"2024-04-24T06:40:48.143194+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:42.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126221+0000","last_change":"2024-04-24T06:40:48.119015+0000","last_active":"2024-04-24T06:40:48.126221+0000","last_peered":"2024-04-24T06:40:48.126221+0000","last_clean":"2024-04-24T06:40:48.126221+0000","last_became_active":"2024-04-24T06:40:48.118700+0000","last_became_peered":"2024-04-24T06:40:48.118700+0000","last_unstale":"2024-04-24T06:40:48.126221+0000","last_undegraded":"2024-04-24T06:40:48.126221+0000","last_fullsized":"2024-04-24T06:40:48.126221+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:02.455813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.140833+0000","last_change":"2024-04-24T06:40:47.091215+0000","last_active":"2024-04-24T06:40:48.140833+0000","last_peered":"2024-04-24T06:40:48.140833+0000","last_clean":"2024-04-24T06:40:48.140833+0000","last_became_active":"2024-04-24T06:40:47.090608+0000","last_became_peered":"2024-04-24T06:40:47.090608+0000","last_unstale":"2024-04-24T06:40:48.140833+0000","last_undegraded":"2024-04-24T06:40:48.140833+0000","last_fullsized":"2024-04-24T06:40:48.140833+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:17.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137658+0000","last_change":"2024-04-24T06:40:48.135160+0000","last_active":"2024-04-24T06:40:48.137658+0000","last_peered":"2024-04-24T06:40:48.137658+0000","last_clean":"2024-04-24T06:40:48.137658+0000","last_became_active":"2024-04-24T06:40:48.131989+0000","last_became_peered":"2024-04-24T06:40:48.131989+0000","last_unstale":"2024-04-24T06:40:48.137658+0000","last_undegraded":"2024-04-24T06:40:48.137658+0000","last_fullsized":"2024-04-24T06:40:48.137658+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.141458+0000","last_change":"2024-04-24T06:40:47.098451+0000","last_active":"2024-04-24T06:40:48.141458+0000","last_peered":"2024-04-24T06:40:48.141458+0000","last_clean":"2024-04-24T06:40:48.141458+0000","last_became_active":"2024-04-24T06:40:47.098050+0000","last_became_peered":"2024-04-24T06:40:47.098050+0000","last_unstale":"2024-04-24T06:40:48.141458+0000","last_undegraded":"2024-04-24T06:40:48.141458+0000","last_fullsized":"2024-04-24T06:40:48.141458+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:13.166385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.121495+0000","last_change":"2024-04-24T06:40:48.118228+0000","last_active":"2024-04-24T06:40:48.121495+0000","last_peered":"2024-04-24T06:40:48.121495+0000","last_clean":"2024-04-24T06:40:48.121495+0000","last_became_active":"2024-04-24T06:40:48.117873+0000","last_became_peered":"2024-04-24T06:40:48.117873+0000","last_unstale":"2024-04-24T06:40:48.121495+0000","last_undegraded":"2024-04-24T06:40:48.121495+0000","last_fullsized":"2024-04-24T06:40:48.121495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:30.921561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143788+0000","last_change":"2024-04-24T06:40:47.093531+0000","last_active":"2024-04-24T06:40:48.143788+0000","last_peered":"2024-04-24T06:40:48.143788+0000","last_clean":"2024-04-24T06:40:48.143788+0000","last_became_active":"2024-04-24T06:40:47.093182+0000","last_became_peered":"2024-04-24T06:40:47.093182+0000","last_unstale":"2024-04-24T06:40:48.143788+0000","last_undegraded":"2024-04-24T06:40:48.143788+0000","last_fullsized":"2024-04-24T06:40:48.143788+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:16:20.357014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129097+0000","last_change":"2024-04-24T06:40:48.113449+0000","last_active":"2024-04-24T06:40:48.129097+0000","last_peered":"2024-04-24T06:40:48.129097+0000","last_clean":"2024-04-24T06:40:48.129097+0000","last_became_active":"2024-04-24T06:40:48.113047+0000","last_became_peered":"2024-04-24T06:40:48.113047+0000","last_unstale":"2024-04-24T06:40:48.129097+0000","last_undegraded":"2024-04-24T06:40:48.129097+0000","last_fullsized":"2024-04-24T06:40:48.129097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:51:15.033592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.145316+0000","last_change":"2024-04-24T06:40:47.096810+0000","last_active":"2024-04-24T06:40:48.145316+0000","last_peered":"2024-04-24T06:40:48.145316+0000","last_clean":"2024-04-24T06:40:48.145316+0000","last_became_active":"2024-04-24T06:40:47.096242+0000","last_became_peered":"2024-04-24T06:40:47.096242+0000","last_unstale":"2024-04-24T06:40:48.145316+0000","last_undegraded":"2024-04-24T06:40:48.145316+0000","last_fullsized":"2024-04-24T06:40:48.145316+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:39.881878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.114897+0000","last_change":"2024-04-24T06:40:48.108510+0000","last_active":"2024-04-24T06:40:48.114897+0000","last_peered":"2024-04-24T06:40:48.114897+0000","last_clean":"2024-04-24T06:40:48.114897+0000","last_became_active":"2024-04-24T06:40:48.107998+0000","last_became_peered":"2024-04-24T06:40:48.107998+0000","last_unstale":"2024-04-24T06:40:48.114897+0000","last_undegraded":"2024-04-24T06:40:48.114897+0000","last_fullsized":"2024-04-24T06:40:48.114897+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:05.270070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.098577+0000","last_change":"2024-04-24T06:40:47.092064+0000","last_active":"2024-04-24T06:40:48.098577+0000","last_peered":"2024-04-24T06:40:48.098577+0000","last_clean":"2024-04-24T06:40:48.098577+0000","last_became_active":"2024-04-24T06:40:47.091762+0000","last_became_peered":"2024-04-24T06:40:47.091762+0000","last_unstale":"2024-04-24T06:40:48.098577+0000","last_undegraded":"2024-04-24T06:40:48.098577+0000","last_fullsized":"2024-04-24T06:40:48.098577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:42.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129123+0000","last_change":"2024-04-24T06:40:48.125108+0000","last_active":"2024-04-24T06:40:48.129123+0000","last_peered":"2024-04-24T06:40:48.129123+0000","last_clean":"2024-04-24T06:40:48.129123+0000","last_became_active":"2024-04-24T06:40:48.124786+0000","last_became_peered":"2024-04-24T06:40:48.124786+0000","last_unstale":"2024-04-24T06:40:48.129123+0000","last_undegraded":"2024-04-24T06:40:48.129123+0000","last_fullsized":"2024-04-24T06:40:48.129123+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:31:29.588166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.091688+0000","last_change":"2024-04-24T06:40:47.091669+0000","last_active":"2024-04-24T06:40:48.091688+0000","last_peered":"2024-04-24T06:40:48.091688+0000","last_clean":"2024-04-24T06:40:48.091688+0000","last_became_active":"2024-04-24T06:40:47.091053+0000","last_became_peered":"2024-04-24T06:40:47.091053+0000","last_unstale":"2024-04-24T06:40:48.091688+0000","last_undegraded":"2024-04-24T06:40:48.091688+0000","last_fullsized":"2024-04-24T06:40:48.091688+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:21.996797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125824+0000","last_change":"2024-04-24T06:40:48.120214+0000","last_active":"2024-04-24T06:40:48.125824+0000","last_peered":"2024-04-24T06:40:48.125824+0000","last_clean":"2024-04-24T06:40:48.125824+0000","last_became_active":"2024-04-24T06:40:48.119866+0000","last_became_peered":"2024-04-24T06:40:48.119866+0000","last_unstale":"2024-04-24T06:40:48.125824+0000","last_undegraded":"2024-04-24T06:40:48.125824+0000","last_fullsized":"2024-04-24T06:40:48.125824+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:14:12.164105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.117306+0000","last_change":"2024-04-24T06:40:47.091932+0000","last_active":"2024-04-24T06:40:48.117306+0000","last_peered":"2024-04-24T06:40:48.117306+0000","last_clean":"2024-04-24T06:40:48.117306+0000","last_became_active":"2024-04-24T06:40:47.091657+0000","last_became_peered":"2024-04-24T06:40:47.091657+0000","last_unstale":"2024-04-24T06:40:48.117306+0000","last_undegraded":"2024-04-24T06:40:48.117306+0000","last_fullsized":"2024-04-24T06:40:48.117306+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:29.716387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135381+0000","last_change":"2024-04-24T06:40:48.130197+0000","last_active":"2024-04-24T06:40:48.135381+0000","last_peered":"2024-04-24T06:40:48.135381+0000","last_clean":"2024-04-24T06:40:48.135381+0000","last_became_active":"2024-04-24T06:40:48.129767+0000","last_became_peered":"2024-04-24T06:40:48.129767+0000","last_unstale":"2024-04-24T06:40:48.135381+0000","last_undegraded":"2024-04-24T06:40:48.135381+0000","last_fullsized":"2024-04-24T06:40:48.135381+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.109115+0000","last_change":"2024-04-24T06:40:47.097414+0000","last_active":"2024-04-24T06:40:48.109115+0000","last_peered":"2024-04-24T06:40:48.109115+0000","last_clean":"2024-04-24T06:40:48.109115+0000","last_became_active":"2024-04-24T06:40:47.097060+0000","last_became_peered":"2024-04-24T06:40:47.097060+0000","last_unstale":"2024-04-24T06:40:48.109115+0000","last_undegraded":"2024-04-24T06:40:48.109115+0000","last_fullsized":"2024-04-24T06:40:48.109115+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:04.990156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.131058+0000","last_change":"2024-04-24T06:40:48.125834+0000","last_active":"2024-04-24T06:40:48.131058+0000","last_peered":"2024-04-24T06:40:48.131058+0000","last_clean":"2024-04-24T06:40:48.131058+0000","last_became_active":"2024-04-24T06:40:48.125437+0000","last_became_peered":"2024-04-24T06:40:48.125437+0000","last_unstale":"2024-04-24T06:40:48.131058+0000","last_undegraded":"2024-04-24T06:40:48.131058+0000","last_fullsized":"2024-04-24T06:40:48.131058+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:23.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148186+0000","last_change":"2024-04-24T06:40:47.098827+0000","last_active":"2024-04-24T06:40:48.148186+0000","last_peered":"2024-04-24T06:40:48.148186+0000","last_clean":"2024-04-24T06:40:48.148186+0000","last_became_active":"2024-04-24T06:40:47.098186+0000","last_became_peered":"2024-04-24T06:40:47.098186+0000","last_unstale":"2024-04-24T06:40:48.148186+0000","last_undegraded":"2024-04-24T06:40:48.148186+0000","last_fullsized":"2024-04-24T06:40:48.148186+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:38:07.079165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126034+0000","last_change":"2024-04-24T06:40:48.122246+0000","last_active":"2024-04-24T06:40:48.126034+0000","last_peered":"2024-04-24T06:40:48.126034+0000","last_clean":"2024-04-24T06:40:48.126034+0000","last_became_active":"2024-04-24T06:40:48.121934+0000","last_became_peered":"2024-04-24T06:40:48.121934+0000","last_unstale":"2024-04-24T06:40:48.126034+0000","last_undegraded":"2024-04-24T06:40:48.126034+0000","last_fullsized":"2024-04-24T06:40:48.126034+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:02.455813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.150230+0000","last_change":"2024-04-24T06:40:47.096544+0000","last_active":"2024-04-24T06:40:48.150230+0000","last_peered":"2024-04-24T06:40:48.150230+0000","last_clean":"2024-04-24T06:40:48.150230+0000","last_became_active":"2024-04-24T06:40:47.096229+0000","last_became_peered":"2024-04-24T06:40:47.096229+0000","last_unstale":"2024-04-24T06:40:48.150230+0000","last_undegraded":"2024-04-24T06:40:48.150230+0000","last_fullsized":"2024-04-24T06:40:48.150230+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:36:47.111066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130656+0000","last_change":"2024-04-24T06:40:48.126769+0000","last_active":"2024-04-24T06:40:48.130656+0000","last_peered":"2024-04-24T06:40:48.130656+0000","last_clean":"2024-04-24T06:40:48.130656+0000","last_became_active":"2024-04-24T06:40:48.126448+0000","last_became_peered":"2024-04-24T06:40:48.126448+0000","last_unstale":"2024-04-24T06:40:48.130656+0000","last_undegraded":"2024-04-24T06:40:48.130656+0000","last_fullsized":"2024-04-24T06:40:48.130656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:44:54.462990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.147665+0000","last_change":"2024-04-24T06:40:47.095389+0000","last_active":"2024-04-24T06:40:48.147665+0000","last_peered":"2024-04-24T06:40:48.147665+0000","last_clean":"2024-04-24T06:40:48.147665+0000","last_became_active":"2024-04-24T06:40:47.093957+0000","last_became_peered":"2024-04-24T06:40:47.093957+0000","last_unstale":"2024-04-24T06:40:48.147665+0000","last_undegraded":"2024-04-24T06:40:48.147665+0000","last_fullsized":"2024-04-24T06:40:48.147665+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:55.608711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133139+0000","last_change":"2024-04-24T06:40:48.129858+0000","last_active":"2024-04-24T06:40:48.133139+0000","last_peered":"2024-04-24T06:40:48.133139+0000","last_clean":"2024-04-24T06:40:48.133139+0000","last_became_active":"2024-04-24T06:40:48.129401+0000","last_became_peered":"2024-04-24T06:40:48.129401+0000","last_unstale":"2024-04-24T06:40:48.133139+0000","last_undegraded":"2024-04-24T06:40:48.133139+0000","last_fullsized":"2024-04-24T06:40:48.133139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:09.868926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.117302+0000","last_change":"2024-04-24T06:40:47.103968+0000","last_active":"2024-04-24T06:40:48.117302+0000","last_peered":"2024-04-24T06:40:48.117302+0000","last_clean":"2024-04-24T06:40:48.117302+0000","last_became_active":"2024-04-24T06:40:47.103729+0000","last_became_peered":"2024-04-24T06:40:47.103729+0000","last_unstale":"2024-04-24T06:40:48.117302+0000","last_undegraded":"2024-04-24T06:40:48.117302+0000","last_fullsized":"2024-04-24T06:40:48.117302+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:19.554237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133027+0000","last_change":"2024-04-24T06:40:48.129870+0000","last_active":"2024-04-24T06:40:48.133027+0000","last_peered":"2024-04-24T06:40:48.133027+0000","last_clean":"2024-04-24T06:40:48.133027+0000","last_became_active":"2024-04-24T06:40:48.129567+0000","last_became_peered":"2024-04-24T06:40:48.129567+0000","last_unstale":"2024-04-24T06:40:48.133027+0000","last_undegraded":"2024-04-24T06:40:48.133027+0000","last_fullsized":"2024-04-24T06:40:48.133027+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:05.270070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"109'8","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148155+0000","last_change":"2024-04-24T06:40:47.103190+0000","last_active":"2024-04-24T06:40:48.148155+0000","last_peered":"2024-04-24T06:40:48.148155+0000","last_clean":"2024-04-24T06:40:48.148155+0000","last_became_active":"2024-04-24T06:40:47.102898+0000","last_became_peered":"2024-04-24T06:40:47.102898+0000","last_unstale":"2024-04-24T06:40:48.148155+0000","last_undegraded":"2024-04-24T06:40:48.148155+0000","last_fullsized":"2024-04-24T06:40:48.148155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:40:03.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127923+0000","last_change":"2024-04-24T06:40:48.124330+0000","last_active":"2024-04-24T06:40:48.127923+0000","last_peered":"2024-04-24T06:40:48.127923+0000","last_clean":"2024-04-24T06:40:48.127923+0000","last_became_active":"2024-04-24T06:40:48.124032+0000","last_became_peered":"2024-04-24T06:40:48.124032+0000","last_unstale":"2024-04-24T06:40:48.127923+0000","last_undegraded":"2024-04-24T06:40:48.127923+0000","last_fullsized":"2024-04-24T06:40:48.127923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:24:46.001179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143008+0000","last_change":"2024-04-24T06:40:47.098869+0000","last_active":"2024-04-24T06:40:48.143008+0000","last_peered":"2024-04-24T06:40:48.143008+0000","last_clean":"2024-04-24T06:40:48.143008+0000","last_became_active":"2024-04-24T06:40:47.098575+0000","last_became_peered":"2024-04-24T06:40:47.098575+0000","last_unstale":"2024-04-24T06:40:48.143008+0000","last_undegraded":"2024-04-24T06:40:48.143008+0000","last_fullsized":"2024-04-24T06:40:48.143008+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.152120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125973+0000","last_change":"2024-04-24T06:40:48.116020+0000","last_active":"2024-04-24T06:40:48.125973+0000","last_peered":"2024-04-24T06:40:48.125973+0000","last_clean":"2024-04-24T06:40:48.125973+0000","last_became_active":"2024-04-24T06:40:48.115743+0000","last_became_peered":"2024-04-24T06:40:48.115743+0000","last_unstale":"2024-04-24T06:40:48.125973+0000","last_undegraded":"2024-04-24T06:40:48.125973+0000","last_fullsized":"2024-04-24T06:40:48.125973+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:47:33.417856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.141364+0000","last_change":"2024-04-24T06:40:47.088524+0000","last_active":"2024-04-24T06:40:48.141364+0000","last_peered":"2024-04-24T06:40:48.141364+0000","last_clean":"2024-04-24T06:40:48.141364+0000","last_became_active":"2024-04-24T06:40:47.088235+0000","last_became_peered":"2024-04-24T06:40:47.088235+0000","last_unstale":"2024-04-24T06:40:48.141364+0000","last_undegraded":"2024-04-24T06:40:48.141364+0000","last_fullsized":"2024-04-24T06:40:48.141364+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:52.536076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135701+0000","last_change":"2024-04-24T06:40:48.125609+0000","last_active":"2024-04-24T06:40:48.135701+0000","last_peered":"2024-04-24T06:40:48.135701+0000","last_clean":"2024-04-24T06:40:48.135701+0000","last_became_active":"2024-04-24T06:40:48.125303+0000","last_became_peered":"2024-04-24T06:40:48.125303+0000","last_unstale":"2024-04-24T06:40:48.135701+0000","last_undegraded":"2024-04-24T06:40:48.135701+0000","last_fullsized":"2024-04-24T06:40:48.135701+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:46:15.073278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.144187+0000","last_change":"2024-04-24T06:40:47.091489+0000","last_active":"2024-04-24T06:40:48.144187+0000","last_peered":"2024-04-24T06:40:48.144187+0000","last_clean":"2024-04-24T06:40:48.144187+0000","last_became_active":"2024-04-24T06:40:47.090831+0000","last_became_peered":"2024-04-24T06:40:47.090831+0000","last_unstale":"2024-04-24T06:40:48.144187+0000","last_undegraded":"2024-04-24T06:40:48.144187+0000","last_fullsized":"2024-04-24T06:40:48.144187+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:07.890927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133087+0000","last_change":"2024-04-24T06:40:48.126927+0000","last_active":"2024-04-24T06:40:48.133087+0000","last_peered":"2024-04-24T06:40:48.133087+0000","last_clean":"2024-04-24T06:40:48.133087+0000","last_became_active":"2024-04-24T06:40:48.126591+0000","last_became_peered":"2024-04-24T06:40:48.126591+0000","last_unstale":"2024-04-24T06:40:48.133087+0000","last_undegraded":"2024-04-24T06:40:48.133087+0000","last_fullsized":"2024-04-24T06:40:48.133087+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:51:15.033592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.107130+0000","last_change":"2024-04-24T06:40:47.096704+0000","last_active":"2024-04-24T06:40:48.107130+0000","last_peered":"2024-04-24T06:40:48.107130+0000","last_clean":"2024-04-24T06:40:48.107130+0000","last_became_active":"2024-04-24T06:40:47.096457+0000","last_became_peered":"2024-04-24T06:40:47.096457+0000","last_unstale":"2024-04-24T06:40:48.107130+0000","last_undegraded":"2024-04-24T06:40:48.107130+0000","last_fullsized":"2024-04-24T06:40:48.107130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.981291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137117+0000","last_change":"2024-04-24T06:40:48.134028+0000","last_active":"2024-04-24T06:40:48.137117+0000","last_peered":"2024-04-24T06:40:48.137117+0000","last_clean":"2024-04-24T06:40:48.137117+0000","last_became_active":"2024-04-24T06:40:48.133706+0000","last_became_peered":"2024-04-24T06:40:48.133706+0000","last_unstale":"2024-04-24T06:40:48.137117+0000","last_undegraded":"2024-04-24T06:40:48.137117+0000","last_fullsized":"2024-04-24T06:40:48.137117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148159+0000","last_change":"2024-04-24T06:40:47.090765+0000","last_active":"2024-04-24T06:40:48.148159+0000","last_peered":"2024-04-24T06:40:48.148159+0000","last_clean":"2024-04-24T06:40:48.148159+0000","last_became_active":"2024-04-24T06:40:47.090354+0000","last_became_peered":"2024-04-24T06:40:47.090354+0000","last_unstale":"2024-04-24T06:40:48.148159+0000","last_undegraded":"2024-04-24T06:40:48.148159+0000","last_fullsized":"2024-04-24T06:40:48.148159+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:42.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130847+0000","last_change":"2024-04-24T06:40:48.127337+0000","last_active":"2024-04-24T06:40:48.130847+0000","last_peered":"2024-04-24T06:40:48.130847+0000","last_clean":"2024-04-24T06:40:48.130847+0000","last_became_active":"2024-04-24T06:40:48.127018+0000","last_became_peered":"2024-04-24T06:40:48.127018+0000","last_unstale":"2024-04-24T06:40:48.130847+0000","last_undegraded":"2024-04-24T06:40:48.130847+0000","last_fullsized":"2024-04-24T06:40:48.130847+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:50:18.350578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116914+0000","last_change":"2024-04-24T06:40:47.096758+0000","last_active":"2024-04-24T06:40:48.116914+0000","last_peered":"2024-04-24T06:40:48.116914+0000","last_clean":"2024-04-24T06:40:48.116914+0000","last_became_active":"2024-04-24T06:40:47.096436+0000","last_became_peered":"2024-04-24T06:40:47.096436+0000","last_unstale":"2024-04-24T06:40:48.116914+0000","last_undegraded":"2024-04-24T06:40:48.116914+0000","last_fullsized":"2024-04-24T06:40:48.116914+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:48.917117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137564+0000","last_change":"2024-04-24T06:40:48.130551+0000","last_active":"2024-04-24T06:40:48.137564+0000","last_peered":"2024-04-24T06:40:48.137564+0000","last_clean":"2024-04-24T06:40:48.137564+0000","last_became_active":"2024-04-24T06:40:48.129810+0000","last_became_peered":"2024-04-24T06:40:48.129810+0000","last_unstale":"2024-04-24T06:40:48.137564+0000","last_undegraded":"2024-04-24T06:40:48.137564+0000","last_fullsized":"2024-04-24T06:40:48.137564+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:20.570062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.105678+0000","last_change":"2024-04-24T06:40:47.092845+0000","last_active":"2024-04-24T06:40:48.105678+0000","last_peered":"2024-04-24T06:40:48.105678+0000","last_clean":"2024-04-24T06:40:48.105678+0000","last_became_active":"2024-04-24T06:40:47.092434+0000","last_became_peered":"2024-04-24T06:40:47.092434+0000","last_unstale":"2024-04-24T06:40:48.105678+0000","last_undegraded":"2024-04-24T06:40:48.105678+0000","last_fullsized":"2024-04-24T06:40:48.105678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:22.568625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122444+0000","last_change":"2024-04-24T06:40:48.116975+0000","last_active":"2024-04-24T06:40:48.122444+0000","last_peered":"2024-04-24T06:40:48.122444+0000","last_clean":"2024-04-24T06:40:48.122444+0000","last_became_active":"2024-04-24T06:40:48.116706+0000","last_became_peered":"2024-04-24T06:40:48.116706+0000","last_unstale":"2024-04-24T06:40:48.122444+0000","last_undegraded":"2024-04-24T06:40:48.122444+0000","last_fullsized":"2024-04-24T06:40:48.122444+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:38:08.094990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.090462+0000","last_change":"2024-04-24T06:40:47.101290+0000","last_active":"2024-04-24T06:40:48.090462+0000","last_peered":"2024-04-24T06:40:48.090462+0000","last_clean":"2024-04-24T06:40:48.090462+0000","last_became_active":"2024-04-24T06:40:47.100995+0000","last_became_peered":"2024-04-24T06:40:47.100995+0000","last_unstale":"2024-04-24T06:40:48.090462+0000","last_undegraded":"2024-04-24T06:40:48.090462+0000","last_fullsized":"2024-04-24T06:40:48.090462+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:14.564836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129484+0000","last_change":"2024-04-24T06:40:48.126465+0000","last_active":"2024-04-24T06:40:48.129484+0000","last_peered":"2024-04-24T06:40:48.129484+0000","last_clean":"2024-04-24T06:40:48.129484+0000","last_became_active":"2024-04-24T06:40:48.126089+0000","last_became_peered":"2024-04-24T06:40:48.126089+0000","last_unstale":"2024-04-24T06:40:48.129484+0000","last_undegraded":"2024-04-24T06:40:48.129484+0000","last_fullsized":"2024-04-24T06:40:48.129484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.145553+0000","last_change":"2024-04-24T06:40:47.103687+0000","last_active":"2024-04-24T06:40:48.145553+0000","last_peered":"2024-04-24T06:40:48.145553+0000","last_clean":"2024-04-24T06:40:48.145553+0000","last_became_active":"2024-04-24T06:40:47.102617+0000","last_became_peered":"2024-04-24T06:40:47.102617+0000","last_unstale":"2024-04-24T06:40:48.145553+0000","last_undegraded":"2024-04-24T06:40:48.145553+0000","last_fullsized":"2024-04-24T06:40:48.145553+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:10.746491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.114601+0000","last_change":"2024-04-24T06:40:48.110130+0000","last_active":"2024-04-24T06:40:48.114601+0000","last_peered":"2024-04-24T06:40:48.114601+0000","last_clean":"2024-04-24T06:40:48.114601+0000","last_became_active":"2024-04-24T06:40:48.109546+0000","last_became_peered":"2024-04-24T06:40:48.109546+0000","last_unstale":"2024-04-24T06:40:48.114601+0000","last_undegraded":"2024-04-24T06:40:48.114601+0000","last_fullsized":"2024-04-24T06:40:48.114601+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:53:52.407256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.118983+0000","last_change":"2024-04-24T06:40:47.090448+0000","last_active":"2024-04-24T06:40:48.118983+0000","last_peered":"2024-04-24T06:40:48.118983+0000","last_clean":"2024-04-24T06:40:48.118983+0000","last_became_active":"2024-04-24T06:40:47.090089+0000","last_became_peered":"2024-04-24T06:40:47.090089+0000","last_unstale":"2024-04-24T06:40:48.118983+0000","last_undegraded":"2024-04-24T06:40:48.118983+0000","last_fullsized":"2024-04-24T06:40:48.118983+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:50.893122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.115840+0000","last_change":"2024-04-24T06:40:48.109616+0000","last_active":"2024-04-24T06:40:48.115840+0000","last_peered":"2024-04-24T06:40:48.115840+0000","last_clean":"2024-04-24T06:40:48.115840+0000","last_became_active":"2024-04-24T06:40:48.109286+0000","last_became_peered":"2024-04-24T06:40:48.109286+0000","last_unstale":"2024-04-24T06:40:48.115840+0000","last_undegraded":"2024-04-24T06:40:48.115840+0000","last_fullsized":"2024-04-24T06:40:48.115840+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:01:36.273184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.090965+0000","last_change":"2024-04-24T06:40:47.101373+0000","last_active":"2024-04-24T06:40:48.090965+0000","last_peered":"2024-04-24T06:40:48.090965+0000","last_clean":"2024-04-24T06:40:48.090965+0000","last_became_active":"2024-04-24T06:40:47.101006+0000","last_became_peered":"2024-04-24T06:40:47.101006+0000","last_unstale":"2024-04-24T06:40:48.090965+0000","last_undegraded":"2024-04-24T06:40:48.090965+0000","last_fullsized":"2024-04-24T06:40:48.090965+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:54:14.204841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127234+0000","last_change":"2024-04-24T06:40:48.125702+0000","last_active":"2024-04-24T06:40:48.127234+0000","last_peered":"2024-04-24T06:40:48.127234+0000","last_clean":"2024-04-24T06:40:48.127234+0000","last_became_active":"2024-04-24T06:40:48.125294+0000","last_became_peered":"2024-04-24T06:40:48.125294+0000","last_unstale":"2024-04-24T06:40:48.127234+0000","last_undegraded":"2024-04-24T06:40:48.127234+0000","last_fullsized":"2024-04-24T06:40:48.127234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.108835+0000","last_change":"2024-04-24T06:40:47.095708+0000","last_active":"2024-04-24T06:40:48.108835+0000","last_peered":"2024-04-24T06:40:48.108835+0000","last_clean":"2024-04-24T06:40:48.108835+0000","last_became_active":"2024-04-24T06:40:47.095330+0000","last_became_peered":"2024-04-24T06:40:47.095330+0000","last_unstale":"2024-04-24T06:40:48.108835+0000","last_undegraded":"2024-04-24T06:40:48.108835+0000","last_fullsized":"2024-04-24T06:40:48.108835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:03.347264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126811+0000","last_change":"2024-04-24T06:40:48.113751+0000","last_active":"2024-04-24T06:40:48.126811+0000","last_peered":"2024-04-24T06:40:48.126811+0000","last_clean":"2024-04-24T06:40:48.126811+0000","last_became_active":"2024-04-24T06:40:48.113444+0000","last_became_peered":"2024-04-24T06:40:48.113444+0000","last_unstale":"2024-04-24T06:40:48.126811+0000","last_undegraded":"2024-04-24T06:40:48.126811+0000","last_fullsized":"2024-04-24T06:40:48.126811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:20.570062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.102911+0000","last_change":"2024-04-24T06:40:47.084334+0000","last_active":"2024-04-24T06:40:48.102911+0000","last_peered":"2024-04-24T06:40:48.102911+0000","last_clean":"2024-04-24T06:40:48.102911+0000","last_became_active":"2024-04-24T06:40:47.083740+0000","last_became_peered":"2024-04-24T06:40:47.083740+0000","last_unstale":"2024-04-24T06:40:48.102911+0000","last_undegraded":"2024-04-24T06:40:48.102911+0000","last_fullsized":"2024-04-24T06:40:48.102911+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:42.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129140+0000","last_change":"2024-04-24T06:40:48.124343+0000","last_active":"2024-04-24T06:40:48.129140+0000","last_peered":"2024-04-24T06:40:48.129140+0000","last_clean":"2024-04-24T06:40:48.129140+0000","last_became_active":"2024-04-24T06:40:48.124003+0000","last_became_peered":"2024-04-24T06:40:48.124003+0000","last_unstale":"2024-04-24T06:40:48.129140+0000","last_undegraded":"2024-04-24T06:40:48.129140+0000","last_fullsized":"2024-04-24T06:40:48.129140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.097970+0000","last_change":"2024-04-24T06:40:47.093061+0000","last_active":"2024-04-24T06:40:48.097970+0000","last_peered":"2024-04-24T06:40:48.097970+0000","last_clean":"2024-04-24T06:40:48.097970+0000","last_became_active":"2024-04-24T06:40:47.092152+0000","last_became_peered":"2024-04-24T06:40:47.092152+0000","last_unstale":"2024-04-24T06:40:48.097970+0000","last_undegraded":"2024-04-24T06:40:48.097970+0000","last_fullsized":"2024-04-24T06:40:48.097970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:41.300615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.138132+0000","last_change":"2024-04-24T06:40:48.136226+0000","last_active":"2024-04-24T06:40:48.138132+0000","last_peered":"2024-04-24T06:40:48.138132+0000","last_clean":"2024-04-24T06:40:48.138132+0000","last_became_active":"2024-04-24T06:40:48.134282+0000","last_became_peered":"2024-04-24T06:40:48.134282+0000","last_unstale":"2024-04-24T06:40:48.138132+0000","last_undegraded":"2024-04-24T06:40:48.138132+0000","last_fullsized":"2024-04-24T06:40:48.138132+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116400+0000","last_change":"2024-04-24T06:40:47.097573+0000","last_active":"2024-04-24T06:40:48.116400+0000","last_peered":"2024-04-24T06:40:48.116400+0000","last_clean":"2024-04-24T06:40:48.116400+0000","last_became_active":"2024-04-24T06:40:47.097293+0000","last_became_peered":"2024-04-24T06:40:47.097293+0000","last_unstale":"2024-04-24T06:40:48.116400+0000","last_undegraded":"2024-04-24T06:40:48.116400+0000","last_fullsized":"2024-04-24T06:40:48.116400+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:37:44.278900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.113859+0000","last_change":"2024-04-24T06:40:48.107774+0000","last_active":"2024-04-24T06:40:48.113859+0000","last_peered":"2024-04-24T06:40:48.113859+0000","last_clean":"2024-04-24T06:40:48.113859+0000","last_became_active":"2024-04-24T06:40:48.107355+0000","last_became_peered":"2024-04-24T06:40:48.107355+0000","last_unstale":"2024-04-24T06:40:48.113859+0000","last_undegraded":"2024-04-24T06:40:48.113859+0000","last_fullsized":"2024-04-24T06:40:48.113859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:09.868926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.092210+0000","last_change":"2024-04-24T06:40:47.101096+0000","last_active":"2024-04-24T06:40:48.092210+0000","last_peered":"2024-04-24T06:40:48.092210+0000","last_clean":"2024-04-24T06:40:48.092210+0000","last_became_active":"2024-04-24T06:40:47.100795+0000","last_became_peered":"2024-04-24T06:40:47.100795+0000","last_unstale":"2024-04-24T06:40:48.092210+0000","last_undegraded":"2024-04-24T06:40:48.092210+0000","last_fullsized":"2024-04-24T06:40:48.092210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:44:53.447165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130903+0000","last_change":"2024-04-24T06:40:48.128357+0000","last_active":"2024-04-24T06:40:48.130903+0000","last_peered":"2024-04-24T06:40:48.130903+0000","last_clean":"2024-04-24T06:40:48.130903+0000","last_became_active":"2024-04-24T06:40:48.127698+0000","last_became_peered":"2024-04-24T06:40:48.127698+0000","last_unstale":"2024-04-24T06:40:48.130903+0000","last_undegraded":"2024-04-24T06:40:48.130903+0000","last_fullsized":"2024-04-24T06:40:48.130903+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:43.167945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.105232+0000","last_change":"2024-04-24T06:40:47.099550+0000","last_active":"2024-04-24T06:40:48.105232+0000","last_peered":"2024-04-24T06:40:48.105232+0000","last_clean":"2024-04-24T06:40:48.105232+0000","last_became_active":"2024-04-24T06:40:47.099300+0000","last_became_peered":"2024-04-24T06:40:47.099300+0000","last_unstale":"2024-04-24T06:40:48.105232+0000","last_undegraded":"2024-04-24T06:40:48.105232+0000","last_fullsized":"2024-04-24T06:40:48.105232+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:47:32.402031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":333,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116497+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:40:48.116497+0000","last_peered":"2024-04-24T06:40:48.116497+0000","last_clean":"2024-04-24T06:40:48.116497+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:40:48.116497+0000","last_undegraded":"2024-04-24T06:40:48.116497+0000","last_fullsized":"2024-04-24T06:40:48.116497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1171143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":587550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1363968,"data_stored":587674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":1179642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:40:52.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:40:52.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.825+0000 7f4bfe000640 1 Processor -- start 2024-04-24T06:40:52.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.825+0000 7f4bfe000640 1 -- start start 2024-04-24T06:40:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.826+0000 7f4bfe000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4bf815c8a0 0x7f4bf80ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.826+0000 7f4bfe000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4bf809b410 0x7f4bf809b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.826+0000 7f4bfe000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bf815cc80 con 0x7f4bf80d0bc0 2024-04-24T06:40:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.826+0000 7f4bfe000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4bf80cd440 con 0x7f4bf809b410 2024-04-24T06:40:52.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.826+0000 7f4bfe000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4bf80cd720 con 0x7f4bf815c8a0 2024-04-24T06:40:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4bf809b410 0x7f4bf809b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4bf809b410 0x7f4bf809b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55868/0 (socket says 172.21.15.59:55868) 2024-04-24T06:40:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 -- 172.21.15.59:0/4280270589 learned_addr learned my addr 172.21.15.59:0/4280270589 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf7400640 1 --2- 172.21.15.59:0/4280270589 >> v2:172.21.15.66:3300/0 conn(0x7f4bf815c8a0 0x7f4bf80ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 -- 172.21.15.59:0/4280270589 >> v2:172.21.15.66:3300/0 conn(0x7f4bf815c8a0 msgr2=0x7f4bf80ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 --2- 172.21.15.59:0/4280270589 >> v2:172.21.15.66:3300/0 conn(0x7f4bf815c8a0 0x7f4bf80ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 -- 172.21.15.59:0/4280270589 >> v1:172.21.15.66:6789/0 conn(0x7f4bf80d0bc0 legacy=0x7f4bf80d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.827+0000 7f4bf6a00640 1 -- 172.21.15.59:0/4280270589 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf80cda00 con 0x7f4bf809b410 2024-04-24T06:40:52.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.828+0000 7f4bf6a00640 1 --2- 172.21.15.59:0/4280270589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf809b8a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8073230 tx=0x7f4be402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57dbdab2ea1f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.829+0000 7f4bf6000640 1 -- 172.21.15.59:0/4280270589 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4037070 con 0x7f4bf809b410 2024-04-24T06:40:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.829+0000 7f4bf6000640 1 -- 172.21.15.59:0/4280270589 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be4032440 con 0x7f4bf809b410 2024-04-24T06:40:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.829+0000 7f4bf6000640 1 -- 172.21.15.59:0/4280270589 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4032810 con 0x7f4bf809b410 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 -- 172.21.15.59:0/4280270589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 msgr2=0x7f4bf809b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 --2- 172.21.15.59:0/4280270589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf809b8a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8073230 tx=0x7f4be402a1a0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 -- 172.21.15.59:0/4280270589 shutdown_connections 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 --2- 172.21.15.59:0/4280270589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf809b8a0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 --2- 172.21.15.59:0/4280270589 >> v2:172.21.15.66:3300/0 conn(0x7f4bf815c8a0 0x7f4bf80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 -- 172.21.15.59:0/4280270589 >> 172.21.15.59:0/4280270589 conn(0x7f4bf80b40f0 msgr2=0x7f4bf80b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 -- 172.21.15.59:0/4280270589 shutdown_connections 2024-04-24T06:40:52.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.830+0000 7f4bfe000640 1 -- 172.21.15.59:0/4280270589 wait complete. 2024-04-24T06:40:52.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.832+0000 7f4bfe000640 1 Processor -- start 2024-04-24T06:40:52.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.832+0000 7f4bfe000640 1 -- start start 2024-04-24T06:40:52.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4bf80d0bc0 0x7f4bf8156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c8a0 0x7f4bf815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf80d0380 con 0x7f4bf809b410 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf80cd110 con 0x7f4bf815c8a0 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bfe000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf80d0500 con 0x7f4bf80d0bc0 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bf7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bf7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55880/0 (socket says 172.21.15.59:55880) 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.833+0000 7f4bf7400640 1 -- 172.21.15.59:0/2516015693 learned_addr learned my addr 172.21.15.59:0/2516015693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf6a00640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4bf80d0bc0 0x7f4bf8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 -- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4bf80d0bc0 msgr2=0x7f4bf8156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4bf80d0bc0 0x7f4bf8156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 -- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c8a0 msgr2=0x7f4bf815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c8a0 0x7f4bf815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 -- 172.21.15.59:0/2516015693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf815d710 con 0x7f4bf809b410 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bf7400640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4be800d800 tx=0x7f4be800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be800b6c0 con 0x7f4bf809b410 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be8019070 con 0x7f4bf809b410 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf815d9f0 con 0x7f4bf809b410 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.834+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be800ba80 con 0x7f4bf809b410 2024-04-24T06:40:52.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.835+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf815dfb0 con 0x7f4bf809b410 2024-04-24T06:40:52.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.836+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf8068a80 con 0x7f4bf809b410 2024-04-24T06:40:52.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.837+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4be800bc20 con 0x7f4bf809b410 2024-04-24T06:40:52.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.837+0000 7f4bef400640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 0x7f4bd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.838+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4be808bff0 con 0x7f4bf809b410 2024-04-24T06:40:52.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.838+0000 7f4bf6a00640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 0x7f4bd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.838+0000 7f4bf6a00640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 0x7f4bd806ede0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4be4034000 tx=0x7f4be402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:52.842+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4be801c360 con 0x7f4bf809b410 2024-04-24T06:40:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.043+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bf815af00 con 0x7f4bd806c920 2024-04-24T06:40:53.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.062+0000 7f4bef400640 1 -- 172.21.15.59:0/2516015693 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252415 (secure 0 0 0) 0x7f4bf815af00 con 0x7f4bd806c920 2024-04-24T06:40:53.066 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:53.067 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:40:53.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 msgr2=0x7f4bd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 0x7f4bd806ede0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4be4034000 tx=0x7f4be402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:53.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 msgr2=0x7f4bf815e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4be800d800 tx=0x7f4be800dcc0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 shutdown_connections 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4bd806c920 0x7f4bd806ede0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bf815c8a0 0x7f4bf815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4bf80d0bc0 0x7f4bf8156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 --2- 172.21.15.59:0/2516015693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4bf809b410 0x7f4bf815e7b0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.067+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 >> 172.21.15.59:0/2516015693 conn(0x7f4bf80b40f0 msgr2=0x7f4bf80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.068+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 shutdown_connections 2024-04-24T06:40:53.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.068+0000 7f4bfe000640 1 -- 172.21.15.59:0/2516015693 wait complete. 2024-04-24T06:40:53.088 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-04-24T06:40:51.445952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261188,"kb_used_data":11720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713532,"statfs":{"total":773094113280,"available":772826656768,"internally_reserved":0,"allocated":12001280,"data_stored":5849109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.391542"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.128859+0000","last_change":"2024-04-24T06:40:48.126867+0000","last_active":"2024-04-24T06:40:48.128859+0000","last_peered":"2024-04-24T06:40:48.128859+0000","last_clean":"2024-04-24T06:40:48.128859+0000","last_became_active":"2024-04-24T06:40:48.125645+0000","last_became_peered":"2024-04-24T06:40:48.125645+0000","last_unstale":"2024-04-24T06:40:48.128859+0000","last_undegraded":"2024-04-24T06:40:48.128859+0000","last_fullsized":"2024-04-24T06:40:48.128859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:28:04.665828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127140+0000","last_change":"2024-04-24T06:40:48.110584+0000","last_active":"2024-04-24T06:40:48.127140+0000","last_peered":"2024-04-24T06:40:48.127140+0000","last_clean":"2024-04-24T06:40:48.127140+0000","last_became_active":"2024-04-24T06:40:48.109829+0000","last_became_peered":"2024-04-24T06:40:48.109829+0000","last_unstale":"2024-04-24T06:40:48.127140+0000","last_undegraded":"2024-04-24T06:40:48.127140+0000","last_fullsized":"2024-04-24T06:40:48.127140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:46:15.073278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137640+0000","last_change":"2024-04-24T06:40:48.133984+0000","last_active":"2024-04-24T06:40:48.137640+0000","last_peered":"2024-04-24T06:40:48.137640+0000","last_clean":"2024-04-24T06:40:48.137640+0000","last_became_active":"2024-04-24T06:40:48.133634+0000","last_became_peered":"2024-04-24T06:40:48.133634+0000","last_unstale":"2024-04-24T06:40:48.137640+0000","last_undegraded":"2024-04-24T06:40:48.137640+0000","last_fullsized":"2024-04-24T06:40:48.137640+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.115815+0000","last_change":"2024-04-24T06:40:48.111574+0000","last_active":"2024-04-24T06:40:48.115815+0000","last_peered":"2024-04-24T06:40:48.115815+0000","last_clean":"2024-04-24T06:40:48.115815+0000","last_became_active":"2024-04-24T06:40:48.111275+0000","last_became_peered":"2024-04-24T06:40:48.111275+0000","last_unstale":"2024-04-24T06:40:48.115815+0000","last_undegraded":"2024-04-24T06:40:48.115815+0000","last_fullsized":"2024-04-24T06:40:48.115815+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:04:45.054578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.124037+0000","last_change":"2024-04-24T06:40:48.118794+0000","last_active":"2024-04-24T06:40:48.124037+0000","last_peered":"2024-04-24T06:40:48.124037+0000","last_clean":"2024-04-24T06:40:48.124037+0000","last_became_active":"2024-04-24T06:40:48.118488+0000","last_became_peered":"2024-04-24T06:40:48.118488+0000","last_unstale":"2024-04-24T06:40:48.124037+0000","last_undegraded":"2024-04-24T06:40:48.124037+0000","last_fullsized":"2024-04-24T06:40:48.124037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:01:04.287043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126568+0000","last_change":"2024-04-24T06:40:48.118094+0000","last_active":"2024-04-24T06:40:48.126568+0000","last_peered":"2024-04-24T06:40:48.126568+0000","last_clean":"2024-04-24T06:40:48.126568+0000","last_became_active":"2024-04-24T06:40:48.117819+0000","last_became_peered":"2024-04-24T06:40:48.117819+0000","last_unstale":"2024-04-24T06:40:48.126568+0000","last_undegraded":"2024-04-24T06:40:48.126568+0000","last_fullsized":"2024-04-24T06:40:48.126568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:14.997116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126139+0000","last_change":"2024-04-24T06:40:48.124361+0000","last_active":"2024-04-24T06:40:48.126139+0000","last_peered":"2024-04-24T06:40:48.126139+0000","last_clean":"2024-04-24T06:40:48.126139+0000","last_became_active":"2024-04-24T06:40:48.123999+0000","last_became_peered":"2024-04-24T06:40:48.123999+0000","last_unstale":"2024-04-24T06:40:48.126139+0000","last_undegraded":"2024-04-24T06:40:48.126139+0000","last_fullsized":"2024-04-24T06:40:48.126139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:09.151981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.139509+0000","last_change":"2024-04-24T06:40:48.137338+0000","last_active":"2024-04-24T06:40:48.139509+0000","last_peered":"2024-04-24T06:40:48.139509+0000","last_clean":"2024-04-24T06:40:48.139509+0000","last_became_active":"2024-04-24T06:40:48.135891+0000","last_became_peered":"2024-04-24T06:40:48.135891+0000","last_unstale":"2024-04-24T06:40:48.139509+0000","last_undegraded":"2024-04-24T06:40:48.139509+0000","last_fullsized":"2024-04-24T06:40:48.139509+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122441+0000","last_change":"2024-04-24T06:40:48.120791+0000","last_active":"2024-04-24T06:40:48.122441+0000","last_peered":"2024-04-24T06:40:48.122441+0000","last_clean":"2024-04-24T06:40:48.122441+0000","last_became_active":"2024-04-24T06:40:48.120447+0000","last_became_peered":"2024-04-24T06:40:48.120447+0000","last_unstale":"2024-04-24T06:40:48.122441+0000","last_undegraded":"2024-04-24T06:40:48.122441+0000","last_fullsized":"2024-04-24T06:40:48.122441+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:26:28.453024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.123396+0000","last_change":"2024-04-24T06:40:48.119795+0000","last_active":"2024-04-24T06:40:48.123396+0000","last_peered":"2024-04-24T06:40:48.123396+0000","last_clean":"2024-04-24T06:40:48.123396+0000","last_became_active":"2024-04-24T06:40:48.119148+0000","last_became_peered":"2024-04-24T06:40:48.119148+0000","last_unstale":"2024-04-24T06:40:48.123396+0000","last_undegraded":"2024-04-24T06:40:48.123396+0000","last_fullsized":"2024-04-24T06:40:48.123396+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:17:23.083995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129401+0000","last_change":"2024-04-24T06:40:48.126500+0000","last_active":"2024-04-24T06:40:48.129401+0000","last_peered":"2024-04-24T06:40:48.129401+0000","last_clean":"2024-04-24T06:40:48.129401+0000","last_became_active":"2024-04-24T06:40:48.126103+0000","last_became_peered":"2024-04-24T06:40:48.126103+0000","last_unstale":"2024-04-24T06:40:48.129401+0000","last_undegraded":"2024-04-24T06:40:48.129401+0000","last_fullsized":"2024-04-24T06:40:48.129401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122001+0000","last_change":"2024-04-24T06:40:48.119612+0000","last_active":"2024-04-24T06:40:48.122001+0000","last_peered":"2024-04-24T06:40:48.122001+0000","last_clean":"2024-04-24T06:40:48.122001+0000","last_became_active":"2024-04-24T06:40:48.119271+0000","last_became_peered":"2024-04-24T06:40:48.119271+0000","last_unstale":"2024-04-24T06:40:48.122001+0000","last_undegraded":"2024-04-24T06:40:48.122001+0000","last_fullsized":"2024-04-24T06:40:48.122001+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:06.695374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135989+0000","last_change":"2024-04-24T06:40:48.132345+0000","last_active":"2024-04-24T06:40:48.135989+0000","last_peered":"2024-04-24T06:40:48.135989+0000","last_clean":"2024-04-24T06:40:48.135989+0000","last_became_active":"2024-04-24T06:40:48.132069+0000","last_became_peered":"2024-04-24T06:40:48.132069+0000","last_unstale":"2024-04-24T06:40:48.135989+0000","last_undegraded":"2024-04-24T06:40:48.135989+0000","last_fullsized":"2024-04-24T06:40:48.135989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:08.906752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137139+0000","last_change":"2024-04-24T06:40:48.134106+0000","last_active":"2024-04-24T06:40:48.137139+0000","last_peered":"2024-04-24T06:40:48.137139+0000","last_clean":"2024-04-24T06:40:48.137139+0000","last_became_active":"2024-04-24T06:40:48.133787+0000","last_became_peered":"2024-04-24T06:40:48.133787+0000","last_unstale":"2024-04-24T06:40:48.137139+0000","last_undegraded":"2024-04-24T06:40:48.137139+0000","last_fullsized":"2024-04-24T06:40:48.137139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126042+0000","last_change":"2024-04-24T06:40:48.118609+0000","last_active":"2024-04-24T06:40:48.126042+0000","last_peered":"2024-04-24T06:40:48.126042+0000","last_clean":"2024-04-24T06:40:48.126042+0000","last_became_active":"2024-04-24T06:40:48.118319+0000","last_became_peered":"2024-04-24T06:40:48.118319+0000","last_unstale":"2024-04-24T06:40:48.126042+0000","last_undegraded":"2024-04-24T06:40:48.126042+0000","last_fullsized":"2024-04-24T06:40:48.126042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:14:12.164105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.142214+0000","last_change":"2024-04-24T06:40:48.135651+0000","last_active":"2024-04-24T06:40:48.142214+0000","last_peered":"2024-04-24T06:40:48.142214+0000","last_clean":"2024-04-24T06:40:48.142214+0000","last_became_active":"2024-04-24T06:40:48.135089+0000","last_became_peered":"2024-04-24T06:40:48.135089+0000","last_unstale":"2024-04-24T06:40:48.142214+0000","last_undegraded":"2024-04-24T06:40:48.142214+0000","last_fullsized":"2024-04-24T06:40:48.142214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:00:46.334314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.131715+0000","last_change":"2024-04-24T06:40:48.128636+0000","last_active":"2024-04-24T06:40:48.131715+0000","last_peered":"2024-04-24T06:40:48.131715+0000","last_clean":"2024-04-24T06:40:48.131715+0000","last_became_active":"2024-04-24T06:40:48.128286+0000","last_became_peered":"2024-04-24T06:40:48.128286+0000","last_unstale":"2024-04-24T06:40:48.131715+0000","last_undegraded":"2024-04-24T06:40:48.131715+0000","last_fullsized":"2024-04-24T06:40:48.131715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.134370+0000","last_change":"2024-04-24T06:40:48.129619+0000","last_active":"2024-04-24T06:40:48.134370+0000","last_peered":"2024-04-24T06:40:48.134370+0000","last_clean":"2024-04-24T06:40:48.134370+0000","last_became_active":"2024-04-24T06:40:48.129168+0000","last_became_peered":"2024-04-24T06:40:48.129168+0000","last_unstale":"2024-04-24T06:40:48.134370+0000","last_undegraded":"2024-04-24T06:40:48.134370+0000","last_fullsized":"2024-04-24T06:40:48.134370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:00:49.725792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137622+0000","last_change":"2024-04-24T06:40:48.133208+0000","last_active":"2024-04-24T06:40:48.137622+0000","last_peered":"2024-04-24T06:40:48.137622+0000","last_clean":"2024-04-24T06:40:48.137622+0000","last_became_active":"2024-04-24T06:40:48.132883+0000","last_became_peered":"2024-04-24T06:40:48.132883+0000","last_unstale":"2024-04-24T06:40:48.137622+0000","last_undegraded":"2024-04-24T06:40:48.137622+0000","last_fullsized":"2024-04-24T06:40:48.137622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:51.908947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129883+0000","last_change":"2024-04-24T06:40:48.122684+0000","last_active":"2024-04-24T06:40:48.129883+0000","last_peered":"2024-04-24T06:40:48.129883+0000","last_clean":"2024-04-24T06:40:48.129883+0000","last_became_active":"2024-04-24T06:40:48.122325+0000","last_became_peered":"2024-04-24T06:40:48.122325+0000","last_unstale":"2024-04-24T06:40:48.129883+0000","last_undegraded":"2024-04-24T06:40:48.129883+0000","last_fullsized":"2024-04-24T06:40:48.129883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:51.908947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129120+0000","last_change":"2024-04-24T06:40:48.118989+0000","last_active":"2024-04-24T06:40:48.129120+0000","last_peered":"2024-04-24T06:40:48.129120+0000","last_clean":"2024-04-24T06:40:48.129120+0000","last_became_active":"2024-04-24T06:40:48.118685+0000","last_became_peered":"2024-04-24T06:40:48.118685+0000","last_unstale":"2024-04-24T06:40:48.129120+0000","last_undegraded":"2024-04-24T06:40:48.129120+0000","last_fullsized":"2024-04-24T06:40:48.129120+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:08.906752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125148+0000","last_change":"2024-04-24T06:40:48.123620+0000","last_active":"2024-04-24T06:40:48.125148+0000","last_peered":"2024-04-24T06:40:48.125148+0000","last_clean":"2024-04-24T06:40:48.125148+0000","last_became_active":"2024-04-24T06:40:48.123309+0000","last_became_peered":"2024-04-24T06:40:48.123309+0000","last_unstale":"2024-04-24T06:40:48.125148+0000","last_undegraded":"2024-04-24T06:40:48.125148+0000","last_fullsized":"2024-04-24T06:40:48.125148+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:54:15.220666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129359+0000","last_change":"2024-04-24T06:40:48.125429+0000","last_active":"2024-04-24T06:40:48.129359+0000","last_peered":"2024-04-24T06:40:48.129359+0000","last_clean":"2024-04-24T06:40:48.129359+0000","last_became_active":"2024-04-24T06:40:48.125024+0000","last_became_peered":"2024-04-24T06:40:48.125024+0000","last_unstale":"2024-04-24T06:40:48.129359+0000","last_undegraded":"2024-04-24T06:40:48.129359+0000","last_fullsized":"2024-04-24T06:40:48.129359+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130924+0000","last_change":"2024-04-24T06:40:48.128637+0000","last_active":"2024-04-24T06:40:48.130924+0000","last_peered":"2024-04-24T06:40:48.130924+0000","last_clean":"2024-04-24T06:40:48.130924+0000","last_became_active":"2024-04-24T06:40:48.128085+0000","last_became_peered":"2024-04-24T06:40:48.128085+0000","last_unstale":"2024-04-24T06:40:48.130924+0000","last_undegraded":"2024-04-24T06:40:48.130924+0000","last_fullsized":"2024-04-24T06:40:48.130924+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:36:48.126891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130025+0000","last_change":"2024-04-24T06:40:48.127656+0000","last_active":"2024-04-24T06:40:48.130025+0000","last_peered":"2024-04-24T06:40:48.130025+0000","last_clean":"2024-04-24T06:40:48.130025+0000","last_became_active":"2024-04-24T06:40:48.127317+0000","last_became_peered":"2024-04-24T06:40:48.127317+0000","last_unstale":"2024-04-24T06:40:48.130025+0000","last_undegraded":"2024-04-24T06:40:48.130025+0000","last_fullsized":"2024-04-24T06:40:48.130025+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130987+0000","last_change":"2024-04-24T06:40:48.128071+0000","last_active":"2024-04-24T06:40:48.130987+0000","last_peered":"2024-04-24T06:40:48.130987+0000","last_clean":"2024-04-24T06:40:48.130987+0000","last_became_active":"2024-04-24T06:40:48.127765+0000","last_became_peered":"2024-04-24T06:40:48.127765+0000","last_unstale":"2024-04-24T06:40:48.130987+0000","last_undegraded":"2024-04-24T06:40:48.130987+0000","last_fullsized":"2024-04-24T06:40:48.130987+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:32.689270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126022+0000","last_change":"2024-04-24T06:40:48.115437+0000","last_active":"2024-04-24T06:40:48.126022+0000","last_peered":"2024-04-24T06:40:48.126022+0000","last_clean":"2024-04-24T06:40:48.126022+0000","last_became_active":"2024-04-24T06:40:48.115112+0000","last_became_peered":"2024-04-24T06:40:48.115112+0000","last_unstale":"2024-04-24T06:40:48.126022+0000","last_undegraded":"2024-04-24T06:40:48.126022+0000","last_fullsized":"2024-04-24T06:40:48.126022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:16:21.372839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122122+0000","last_change":"2024-04-24T06:40:48.116711+0000","last_active":"2024-04-24T06:40:48.122122+0000","last_peered":"2024-04-24T06:40:48.122122+0000","last_clean":"2024-04-24T06:40:48.122122+0000","last_became_active":"2024-04-24T06:40:48.116405+0000","last_became_peered":"2024-04-24T06:40:48.116405+0000","last_unstale":"2024-04-24T06:40:48.122122+0000","last_undegraded":"2024-04-24T06:40:48.122122+0000","last_fullsized":"2024-04-24T06:40:48.122122+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:15.580661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130744+0000","last_change":"2024-04-24T06:40:48.128764+0000","last_active":"2024-04-24T06:40:48.130744+0000","last_peered":"2024-04-24T06:40:48.130744+0000","last_clean":"2024-04-24T06:40:48.130744+0000","last_became_active":"2024-04-24T06:40:48.128461+0000","last_became_peered":"2024-04-24T06:40:48.128461+0000","last_unstale":"2024-04-24T06:40:48.130744+0000","last_undegraded":"2024-04-24T06:40:48.130744+0000","last_fullsized":"2024-04-24T06:40:48.130744+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.128818+0000","last_change":"2024-04-24T06:40:48.124981+0000","last_active":"2024-04-24T06:40:48.128818+0000","last_peered":"2024-04-24T06:40:48.128818+0000","last_clean":"2024-04-24T06:40:48.128818+0000","last_became_active":"2024-04-24T06:40:48.122688+0000","last_became_peered":"2024-04-24T06:40:48.122688+0000","last_unstale":"2024-04-24T06:40:48.128818+0000","last_undegraded":"2024-04-24T06:40:48.128818+0000","last_fullsized":"2024-04-24T06:40:48.128818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:35.196169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133060+0000","last_change":"2024-04-24T06:40:48.128061+0000","last_active":"2024-04-24T06:40:48.133060+0000","last_peered":"2024-04-24T06:40:48.133060+0000","last_clean":"2024-04-24T06:40:48.133060+0000","last_became_active":"2024-04-24T06:40:48.127698+0000","last_became_peered":"2024-04-24T06:40:48.127698+0000","last_unstale":"2024-04-24T06:40:48.133060+0000","last_undegraded":"2024-04-24T06:40:48.133060+0000","last_fullsized":"2024-04-24T06:40:48.133060+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:24.935770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126015+0000","last_change":"2024-04-24T06:40:48.115874+0000","last_active":"2024-04-24T06:40:48.126015+0000","last_peered":"2024-04-24T06:40:48.126015+0000","last_clean":"2024-04-24T06:40:48.126015+0000","last_became_active":"2024-04-24T06:40:48.115594+0000","last_became_peered":"2024-04-24T06:40:48.115594+0000","last_unstale":"2024-04-24T06:40:48.126015+0000","last_undegraded":"2024-04-24T06:40:48.126015+0000","last_fullsized":"2024-04-24T06:40:48.126015+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:16:54.481838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126630+0000","last_change":"2024-04-24T06:40:48.123127+0000","last_active":"2024-04-24T06:40:48.126630+0000","last_peered":"2024-04-24T06:40:48.126630+0000","last_clean":"2024-04-24T06:40:48.126630+0000","last_became_active":"2024-04-24T06:40:48.121282+0000","last_became_peered":"2024-04-24T06:40:48.121282+0000","last_unstale":"2024-04-24T06:40:48.126630+0000","last_undegraded":"2024-04-24T06:40:48.126630+0000","last_fullsized":"2024-04-24T06:40:48.126630+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:09.151981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"109'4","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.142568+0000","last_change":"2024-04-24T06:40:47.089244+0000","last_active":"2024-04-24T06:40:48.142568+0000","last_peered":"2024-04-24T06:40:48.142568+0000","last_clean":"2024-04-24T06:40:48.142568+0000","last_became_active":"2024-04-24T06:40:47.087988+0000","last_became_peered":"2024-04-24T06:40:47.087988+0000","last_unstale":"2024-04-24T06:40:48.142568+0000","last_undegraded":"2024-04-24T06:40:48.142568+0000","last_fullsized":"2024-04-24T06:40:48.142568+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:16:53.466013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.136741+0000","last_change":"2024-04-24T06:40:48.134224+0000","last_active":"2024-04-24T06:40:48.136741+0000","last_peered":"2024-04-24T06:40:48.136741+0000","last_clean":"2024-04-24T06:40:48.136741+0000","last_became_active":"2024-04-24T06:40:48.133923+0000","last_became_peered":"2024-04-24T06:40:48.133923+0000","last_unstale":"2024-04-24T06:40:48.136741+0000","last_undegraded":"2024-04-24T06:40:48.136741+0000","last_fullsized":"2024-04-24T06:40:48.136741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143194+0000","last_change":"2024-04-24T06:40:47.091853+0000","last_active":"2024-04-24T06:40:48.143194+0000","last_peered":"2024-04-24T06:40:48.143194+0000","last_clean":"2024-04-24T06:40:48.143194+0000","last_became_active":"2024-04-24T06:40:47.091515+0000","last_became_peered":"2024-04-24T06:40:47.091515+0000","last_unstale":"2024-04-24T06:40:48.143194+0000","last_undegraded":"2024-04-24T06:40:48.143194+0000","last_fullsized":"2024-04-24T06:40:48.143194+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:42.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126221+0000","last_change":"2024-04-24T06:40:48.119015+0000","last_active":"2024-04-24T06:40:48.126221+0000","last_peered":"2024-04-24T06:40:48.126221+0000","last_clean":"2024-04-24T06:40:48.126221+0000","last_became_active":"2024-04-24T06:40:48.118700+0000","last_became_peered":"2024-04-24T06:40:48.118700+0000","last_unstale":"2024-04-24T06:40:48.126221+0000","last_undegraded":"2024-04-24T06:40:48.126221+0000","last_fullsized":"2024-04-24T06:40:48.126221+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:02.455813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.140833+0000","last_change":"2024-04-24T06:40:47.091215+0000","last_active":"2024-04-24T06:40:48.140833+0000","last_peered":"2024-04-24T06:40:48.140833+0000","last_clean":"2024-04-24T06:40:48.140833+0000","last_became_active":"2024-04-24T06:40:47.090608+0000","last_became_peered":"2024-04-24T06:40:47.090608+0000","last_unstale":"2024-04-24T06:40:48.140833+0000","last_undegraded":"2024-04-24T06:40:48.140833+0000","last_fullsized":"2024-04-24T06:40:48.140833+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:20:17.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137658+0000","last_change":"2024-04-24T06:40:48.135160+0000","last_active":"2024-04-24T06:40:48.137658+0000","last_peered":"2024-04-24T06:40:48.137658+0000","last_clean":"2024-04-24T06:40:48.137658+0000","last_became_active":"2024-04-24T06:40:48.131989+0000","last_became_peered":"2024-04-24T06:40:48.131989+0000","last_unstale":"2024-04-24T06:40:48.137658+0000","last_undegraded":"2024-04-24T06:40:48.137658+0000","last_fullsized":"2024-04-24T06:40:48.137658+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.141458+0000","last_change":"2024-04-24T06:40:47.098451+0000","last_active":"2024-04-24T06:40:48.141458+0000","last_peered":"2024-04-24T06:40:48.141458+0000","last_clean":"2024-04-24T06:40:48.141458+0000","last_became_active":"2024-04-24T06:40:47.098050+0000","last_became_peered":"2024-04-24T06:40:47.098050+0000","last_unstale":"2024-04-24T06:40:48.141458+0000","last_undegraded":"2024-04-24T06:40:48.141458+0000","last_fullsized":"2024-04-24T06:40:48.141458+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:13.166385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.121495+0000","last_change":"2024-04-24T06:40:48.118228+0000","last_active":"2024-04-24T06:40:48.121495+0000","last_peered":"2024-04-24T06:40:48.121495+0000","last_clean":"2024-04-24T06:40:48.121495+0000","last_became_active":"2024-04-24T06:40:48.117873+0000","last_became_peered":"2024-04-24T06:40:48.117873+0000","last_unstale":"2024-04-24T06:40:48.121495+0000","last_undegraded":"2024-04-24T06:40:48.121495+0000","last_fullsized":"2024-04-24T06:40:48.121495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:30.921561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143788+0000","last_change":"2024-04-24T06:40:47.093531+0000","last_active":"2024-04-24T06:40:48.143788+0000","last_peered":"2024-04-24T06:40:48.143788+0000","last_clean":"2024-04-24T06:40:48.143788+0000","last_became_active":"2024-04-24T06:40:47.093182+0000","last_became_peered":"2024-04-24T06:40:47.093182+0000","last_unstale":"2024-04-24T06:40:48.143788+0000","last_undegraded":"2024-04-24T06:40:48.143788+0000","last_fullsized":"2024-04-24T06:40:48.143788+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:16:20.357014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129097+0000","last_change":"2024-04-24T06:40:48.113449+0000","last_active":"2024-04-24T06:40:48.129097+0000","last_peered":"2024-04-24T06:40:48.129097+0000","last_clean":"2024-04-24T06:40:48.129097+0000","last_became_active":"2024-04-24T06:40:48.113047+0000","last_became_peered":"2024-04-24T06:40:48.113047+0000","last_unstale":"2024-04-24T06:40:48.129097+0000","last_undegraded":"2024-04-24T06:40:48.129097+0000","last_fullsized":"2024-04-24T06:40:48.129097+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:51:15.033592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.145316+0000","last_change":"2024-04-24T06:40:47.096810+0000","last_active":"2024-04-24T06:40:48.145316+0000","last_peered":"2024-04-24T06:40:48.145316+0000","last_clean":"2024-04-24T06:40:48.145316+0000","last_became_active":"2024-04-24T06:40:47.096242+0000","last_became_peered":"2024-04-24T06:40:47.096242+0000","last_unstale":"2024-04-24T06:40:48.145316+0000","last_undegraded":"2024-04-24T06:40:48.145316+0000","last_fullsized":"2024-04-24T06:40:48.145316+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:39.881878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.114897+0000","last_change":"2024-04-24T06:40:48.108510+0000","last_active":"2024-04-24T06:40:48.114897+0000","last_peered":"2024-04-24T06:40:48.114897+0000","last_clean":"2024-04-24T06:40:48.114897+0000","last_became_active":"2024-04-24T06:40:48.107998+0000","last_became_peered":"2024-04-24T06:40:48.107998+0000","last_unstale":"2024-04-24T06:40:48.114897+0000","last_undegraded":"2024-04-24T06:40:48.114897+0000","last_fullsized":"2024-04-24T06:40:48.114897+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:05.270070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.098577+0000","last_change":"2024-04-24T06:40:47.092064+0000","last_active":"2024-04-24T06:40:48.098577+0000","last_peered":"2024-04-24T06:40:48.098577+0000","last_clean":"2024-04-24T06:40:48.098577+0000","last_became_active":"2024-04-24T06:40:47.091762+0000","last_became_peered":"2024-04-24T06:40:47.091762+0000","last_unstale":"2024-04-24T06:40:48.098577+0000","last_undegraded":"2024-04-24T06:40:48.098577+0000","last_fullsized":"2024-04-24T06:40:48.098577+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:42.516797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129123+0000","last_change":"2024-04-24T06:40:48.125108+0000","last_active":"2024-04-24T06:40:48.129123+0000","last_peered":"2024-04-24T06:40:48.129123+0000","last_clean":"2024-04-24T06:40:48.129123+0000","last_became_active":"2024-04-24T06:40:48.124786+0000","last_became_peered":"2024-04-24T06:40:48.124786+0000","last_unstale":"2024-04-24T06:40:48.129123+0000","last_undegraded":"2024-04-24T06:40:48.129123+0000","last_fullsized":"2024-04-24T06:40:48.129123+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:31:29.588166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.091688+0000","last_change":"2024-04-24T06:40:47.091669+0000","last_active":"2024-04-24T06:40:48.091688+0000","last_peered":"2024-04-24T06:40:48.091688+0000","last_clean":"2024-04-24T06:40:48.091688+0000","last_became_active":"2024-04-24T06:40:47.091053+0000","last_became_peered":"2024-04-24T06:40:47.091053+0000","last_unstale":"2024-04-24T06:40:48.091688+0000","last_undegraded":"2024-04-24T06:40:48.091688+0000","last_fullsized":"2024-04-24T06:40:48.091688+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:21.996797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125824+0000","last_change":"2024-04-24T06:40:48.120214+0000","last_active":"2024-04-24T06:40:48.125824+0000","last_peered":"2024-04-24T06:40:48.125824+0000","last_clean":"2024-04-24T06:40:48.125824+0000","last_became_active":"2024-04-24T06:40:48.119866+0000","last_became_peered":"2024-04-24T06:40:48.119866+0000","last_unstale":"2024-04-24T06:40:48.125824+0000","last_undegraded":"2024-04-24T06:40:48.125824+0000","last_fullsized":"2024-04-24T06:40:48.125824+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:14:12.164105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.117306+0000","last_change":"2024-04-24T06:40:47.091932+0000","last_active":"2024-04-24T06:40:48.117306+0000","last_peered":"2024-04-24T06:40:48.117306+0000","last_clean":"2024-04-24T06:40:48.117306+0000","last_became_active":"2024-04-24T06:40:47.091657+0000","last_became_peered":"2024-04-24T06:40:47.091657+0000","last_unstale":"2024-04-24T06:40:48.117306+0000","last_undegraded":"2024-04-24T06:40:48.117306+0000","last_fullsized":"2024-04-24T06:40:48.117306+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:29.716387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135381+0000","last_change":"2024-04-24T06:40:48.130197+0000","last_active":"2024-04-24T06:40:48.135381+0000","last_peered":"2024-04-24T06:40:48.135381+0000","last_clean":"2024-04-24T06:40:48.135381+0000","last_became_active":"2024-04-24T06:40:48.129767+0000","last_became_peered":"2024-04-24T06:40:48.129767+0000","last_unstale":"2024-04-24T06:40:48.135381+0000","last_undegraded":"2024-04-24T06:40:48.135381+0000","last_fullsized":"2024-04-24T06:40:48.135381+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:04.363089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.109115+0000","last_change":"2024-04-24T06:40:47.097414+0000","last_active":"2024-04-24T06:40:48.109115+0000","last_peered":"2024-04-24T06:40:48.109115+0000","last_clean":"2024-04-24T06:40:48.109115+0000","last_became_active":"2024-04-24T06:40:47.097060+0000","last_became_peered":"2024-04-24T06:40:47.097060+0000","last_unstale":"2024-04-24T06:40:48.109115+0000","last_undegraded":"2024-04-24T06:40:48.109115+0000","last_fullsized":"2024-04-24T06:40:48.109115+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:04.990156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.131058+0000","last_change":"2024-04-24T06:40:48.125834+0000","last_active":"2024-04-24T06:40:48.131058+0000","last_peered":"2024-04-24T06:40:48.131058+0000","last_clean":"2024-04-24T06:40:48.131058+0000","last_became_active":"2024-04-24T06:40:48.125437+0000","last_became_peered":"2024-04-24T06:40:48.125437+0000","last_unstale":"2024-04-24T06:40:48.131058+0000","last_undegraded":"2024-04-24T06:40:48.131058+0000","last_fullsized":"2024-04-24T06:40:48.131058+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:23.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148186+0000","last_change":"2024-04-24T06:40:47.098827+0000","last_active":"2024-04-24T06:40:48.148186+0000","last_peered":"2024-04-24T06:40:48.148186+0000","last_clean":"2024-04-24T06:40:48.148186+0000","last_became_active":"2024-04-24T06:40:47.098186+0000","last_became_peered":"2024-04-24T06:40:47.098186+0000","last_unstale":"2024-04-24T06:40:48.148186+0000","last_undegraded":"2024-04-24T06:40:48.148186+0000","last_fullsized":"2024-04-24T06:40:48.148186+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:38:07.079165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126034+0000","last_change":"2024-04-24T06:40:48.122246+0000","last_active":"2024-04-24T06:40:48.126034+0000","last_peered":"2024-04-24T06:40:48.126034+0000","last_clean":"2024-04-24T06:40:48.126034+0000","last_became_active":"2024-04-24T06:40:48.121934+0000","last_became_peered":"2024-04-24T06:40:48.121934+0000","last_unstale":"2024-04-24T06:40:48.126034+0000","last_undegraded":"2024-04-24T06:40:48.126034+0000","last_fullsized":"2024-04-24T06:40:48.126034+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:59:02.455813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.150230+0000","last_change":"2024-04-24T06:40:47.096544+0000","last_active":"2024-04-24T06:40:48.150230+0000","last_peered":"2024-04-24T06:40:48.150230+0000","last_clean":"2024-04-24T06:40:48.150230+0000","last_became_active":"2024-04-24T06:40:47.096229+0000","last_became_peered":"2024-04-24T06:40:47.096229+0000","last_unstale":"2024-04-24T06:40:48.150230+0000","last_undegraded":"2024-04-24T06:40:48.150230+0000","last_fullsized":"2024-04-24T06:40:48.150230+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:36:47.111066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130656+0000","last_change":"2024-04-24T06:40:48.126769+0000","last_active":"2024-04-24T06:40:48.130656+0000","last_peered":"2024-04-24T06:40:48.130656+0000","last_clean":"2024-04-24T06:40:48.130656+0000","last_became_active":"2024-04-24T06:40:48.126448+0000","last_became_peered":"2024-04-24T06:40:48.126448+0000","last_unstale":"2024-04-24T06:40:48.130656+0000","last_undegraded":"2024-04-24T06:40:48.130656+0000","last_fullsized":"2024-04-24T06:40:48.130656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:44:54.462990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.147665+0000","last_change":"2024-04-24T06:40:47.095389+0000","last_active":"2024-04-24T06:40:48.147665+0000","last_peered":"2024-04-24T06:40:48.147665+0000","last_clean":"2024-04-24T06:40:48.147665+0000","last_became_active":"2024-04-24T06:40:47.093957+0000","last_became_peered":"2024-04-24T06:40:47.093957+0000","last_unstale":"2024-04-24T06:40:48.147665+0000","last_undegraded":"2024-04-24T06:40:48.147665+0000","last_fullsized":"2024-04-24T06:40:48.147665+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:32:55.608711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133139+0000","last_change":"2024-04-24T06:40:48.129858+0000","last_active":"2024-04-24T06:40:48.133139+0000","last_peered":"2024-04-24T06:40:48.133139+0000","last_clean":"2024-04-24T06:40:48.133139+0000","last_became_active":"2024-04-24T06:40:48.129401+0000","last_became_peered":"2024-04-24T06:40:48.129401+0000","last_unstale":"2024-04-24T06:40:48.133139+0000","last_undegraded":"2024-04-24T06:40:48.133139+0000","last_fullsized":"2024-04-24T06:40:48.133139+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:09.868926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.117302+0000","last_change":"2024-04-24T06:40:47.103968+0000","last_active":"2024-04-24T06:40:48.117302+0000","last_peered":"2024-04-24T06:40:48.117302+0000","last_clean":"2024-04-24T06:40:48.117302+0000","last_became_active":"2024-04-24T06:40:47.103729+0000","last_became_peered":"2024-04-24T06:40:47.103729+0000","last_unstale":"2024-04-24T06:40:48.117302+0000","last_undegraded":"2024-04-24T06:40:48.117302+0000","last_fullsized":"2024-04-24T06:40:48.117302+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:19.554237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133027+0000","last_change":"2024-04-24T06:40:48.129870+0000","last_active":"2024-04-24T06:40:48.133027+0000","last_peered":"2024-04-24T06:40:48.133027+0000","last_clean":"2024-04-24T06:40:48.133027+0000","last_became_active":"2024-04-24T06:40:48.129567+0000","last_became_peered":"2024-04-24T06:40:48.129567+0000","last_unstale":"2024-04-24T06:40:48.133027+0000","last_undegraded":"2024-04-24T06:40:48.133027+0000","last_fullsized":"2024-04-24T06:40:48.133027+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:05.270070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"109'8","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148155+0000","last_change":"2024-04-24T06:40:47.103190+0000","last_active":"2024-04-24T06:40:48.148155+0000","last_peered":"2024-04-24T06:40:48.148155+0000","last_clean":"2024-04-24T06:40:48.148155+0000","last_became_active":"2024-04-24T06:40:47.102898+0000","last_became_peered":"2024-04-24T06:40:47.102898+0000","last_unstale":"2024-04-24T06:40:48.148155+0000","last_undegraded":"2024-04-24T06:40:48.148155+0000","last_fullsized":"2024-04-24T06:40:48.148155+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:40:03.716434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127923+0000","last_change":"2024-04-24T06:40:48.124330+0000","last_active":"2024-04-24T06:40:48.127923+0000","last_peered":"2024-04-24T06:40:48.127923+0000","last_clean":"2024-04-24T06:40:48.127923+0000","last_became_active":"2024-04-24T06:40:48.124032+0000","last_became_peered":"2024-04-24T06:40:48.124032+0000","last_unstale":"2024-04-24T06:40:48.127923+0000","last_undegraded":"2024-04-24T06:40:48.127923+0000","last_fullsized":"2024-04-24T06:40:48.127923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:24:46.001179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.143008+0000","last_change":"2024-04-24T06:40:47.098869+0000","last_active":"2024-04-24T06:40:48.143008+0000","last_peered":"2024-04-24T06:40:48.143008+0000","last_clean":"2024-04-24T06:40:48.143008+0000","last_became_active":"2024-04-24T06:40:47.098575+0000","last_became_peered":"2024-04-24T06:40:47.098575+0000","last_unstale":"2024-04-24T06:40:48.143008+0000","last_undegraded":"2024-04-24T06:40:48.143008+0000","last_fullsized":"2024-04-24T06:40:48.143008+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:42.152120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.125973+0000","last_change":"2024-04-24T06:40:48.116020+0000","last_active":"2024-04-24T06:40:48.125973+0000","last_peered":"2024-04-24T06:40:48.125973+0000","last_clean":"2024-04-24T06:40:48.125973+0000","last_became_active":"2024-04-24T06:40:48.115743+0000","last_became_peered":"2024-04-24T06:40:48.115743+0000","last_unstale":"2024-04-24T06:40:48.125973+0000","last_undegraded":"2024-04-24T06:40:48.125973+0000","last_fullsized":"2024-04-24T06:40:48.125973+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:47:33.417856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.141364+0000","last_change":"2024-04-24T06:40:47.088524+0000","last_active":"2024-04-24T06:40:48.141364+0000","last_peered":"2024-04-24T06:40:48.141364+0000","last_clean":"2024-04-24T06:40:48.141364+0000","last_became_active":"2024-04-24T06:40:47.088235+0000","last_became_peered":"2024-04-24T06:40:47.088235+0000","last_unstale":"2024-04-24T06:40:48.141364+0000","last_undegraded":"2024-04-24T06:40:48.141364+0000","last_fullsized":"2024-04-24T06:40:48.141364+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:52.536076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.135701+0000","last_change":"2024-04-24T06:40:48.125609+0000","last_active":"2024-04-24T06:40:48.135701+0000","last_peered":"2024-04-24T06:40:48.135701+0000","last_clean":"2024-04-24T06:40:48.135701+0000","last_became_active":"2024-04-24T06:40:48.125303+0000","last_became_peered":"2024-04-24T06:40:48.125303+0000","last_unstale":"2024-04-24T06:40:48.135701+0000","last_undegraded":"2024-04-24T06:40:48.135701+0000","last_fullsized":"2024-04-24T06:40:48.135701+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:46:15.073278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.144187+0000","last_change":"2024-04-24T06:40:47.091489+0000","last_active":"2024-04-24T06:40:48.144187+0000","last_peered":"2024-04-24T06:40:48.144187+0000","last_clean":"2024-04-24T06:40:48.144187+0000","last_became_active":"2024-04-24T06:40:47.090831+0000","last_became_peered":"2024-04-24T06:40:47.090831+0000","last_unstale":"2024-04-24T06:40:48.144187+0000","last_undegraded":"2024-04-24T06:40:48.144187+0000","last_fullsized":"2024-04-24T06:40:48.144187+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:57:07.890927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.133087+0000","last_change":"2024-04-24T06:40:48.126927+0000","last_active":"2024-04-24T06:40:48.133087+0000","last_peered":"2024-04-24T06:40:48.133087+0000","last_clean":"2024-04-24T06:40:48.133087+0000","last_became_active":"2024-04-24T06:40:48.126591+0000","last_became_peered":"2024-04-24T06:40:48.126591+0000","last_unstale":"2024-04-24T06:40:48.133087+0000","last_undegraded":"2024-04-24T06:40:48.133087+0000","last_fullsized":"2024-04-24T06:40:48.133087+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:51:15.033592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.107130+0000","last_change":"2024-04-24T06:40:47.096704+0000","last_active":"2024-04-24T06:40:48.107130+0000","last_peered":"2024-04-24T06:40:48.107130+0000","last_clean":"2024-04-24T06:40:48.107130+0000","last_became_active":"2024-04-24T06:40:47.096457+0000","last_became_peered":"2024-04-24T06:40:47.096457+0000","last_unstale":"2024-04-24T06:40:48.107130+0000","last_undegraded":"2024-04-24T06:40:48.107130+0000","last_fullsized":"2024-04-24T06:40:48.107130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.981291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137117+0000","last_change":"2024-04-24T06:40:48.134028+0000","last_active":"2024-04-24T06:40:48.137117+0000","last_peered":"2024-04-24T06:40:48.137117+0000","last_clean":"2024-04-24T06:40:48.137117+0000","last_became_active":"2024-04-24T06:40:48.133706+0000","last_became_peered":"2024-04-24T06:40:48.133706+0000","last_unstale":"2024-04-24T06:40:48.137117+0000","last_undegraded":"2024-04-24T06:40:48.137117+0000","last_fullsized":"2024-04-24T06:40:48.137117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.148159+0000","last_change":"2024-04-24T06:40:47.090765+0000","last_active":"2024-04-24T06:40:48.148159+0000","last_peered":"2024-04-24T06:40:48.148159+0000","last_clean":"2024-04-24T06:40:48.148159+0000","last_became_active":"2024-04-24T06:40:47.090354+0000","last_became_peered":"2024-04-24T06:40:47.090354+0000","last_unstale":"2024-04-24T06:40:48.148159+0000","last_undegraded":"2024-04-24T06:40:48.148159+0000","last_fullsized":"2024-04-24T06:40:48.148159+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:42.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130847+0000","last_change":"2024-04-24T06:40:48.127337+0000","last_active":"2024-04-24T06:40:48.130847+0000","last_peered":"2024-04-24T06:40:48.130847+0000","last_clean":"2024-04-24T06:40:48.130847+0000","last_became_active":"2024-04-24T06:40:48.127018+0000","last_became_peered":"2024-04-24T06:40:48.127018+0000","last_unstale":"2024-04-24T06:40:48.130847+0000","last_undegraded":"2024-04-24T06:40:48.130847+0000","last_fullsized":"2024-04-24T06:40:48.130847+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:50:18.350578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116914+0000","last_change":"2024-04-24T06:40:47.096758+0000","last_active":"2024-04-24T06:40:48.116914+0000","last_peered":"2024-04-24T06:40:48.116914+0000","last_clean":"2024-04-24T06:40:48.116914+0000","last_became_active":"2024-04-24T06:40:47.096436+0000","last_became_peered":"2024-04-24T06:40:47.096436+0000","last_unstale":"2024-04-24T06:40:48.116914+0000","last_undegraded":"2024-04-24T06:40:48.116914+0000","last_fullsized":"2024-04-24T06:40:48.116914+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:48.917117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.137564+0000","last_change":"2024-04-24T06:40:48.130551+0000","last_active":"2024-04-24T06:40:48.137564+0000","last_peered":"2024-04-24T06:40:48.137564+0000","last_clean":"2024-04-24T06:40:48.137564+0000","last_became_active":"2024-04-24T06:40:48.129810+0000","last_became_peered":"2024-04-24T06:40:48.129810+0000","last_unstale":"2024-04-24T06:40:48.137564+0000","last_undegraded":"2024-04-24T06:40:48.137564+0000","last_fullsized":"2024-04-24T06:40:48.137564+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:20.570062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.105678+0000","last_change":"2024-04-24T06:40:47.092845+0000","last_active":"2024-04-24T06:40:48.105678+0000","last_peered":"2024-04-24T06:40:48.105678+0000","last_clean":"2024-04-24T06:40:48.105678+0000","last_became_active":"2024-04-24T06:40:47.092434+0000","last_became_peered":"2024-04-24T06:40:47.092434+0000","last_unstale":"2024-04-24T06:40:48.105678+0000","last_undegraded":"2024-04-24T06:40:48.105678+0000","last_fullsized":"2024-04-24T06:40:48.105678+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:22.568625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.122444+0000","last_change":"2024-04-24T06:40:48.116975+0000","last_active":"2024-04-24T06:40:48.122444+0000","last_peered":"2024-04-24T06:40:48.122444+0000","last_clean":"2024-04-24T06:40:48.122444+0000","last_became_active":"2024-04-24T06:40:48.116706+0000","last_became_peered":"2024-04-24T06:40:48.116706+0000","last_unstale":"2024-04-24T06:40:48.122444+0000","last_undegraded":"2024-04-24T06:40:48.122444+0000","last_fullsized":"2024-04-24T06:40:48.122444+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:38:08.094990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.090462+0000","last_change":"2024-04-24T06:40:47.101290+0000","last_active":"2024-04-24T06:40:48.090462+0000","last_peered":"2024-04-24T06:40:48.090462+0000","last_clean":"2024-04-24T06:40:48.090462+0000","last_became_active":"2024-04-24T06:40:47.100995+0000","last_became_peered":"2024-04-24T06:40:47.100995+0000","last_unstale":"2024-04-24T06:40:48.090462+0000","last_undegraded":"2024-04-24T06:40:48.090462+0000","last_fullsized":"2024-04-24T06:40:48.090462+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:27:14.564836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129484+0000","last_change":"2024-04-24T06:40:48.126465+0000","last_active":"2024-04-24T06:40:48.129484+0000","last_peered":"2024-04-24T06:40:48.129484+0000","last_clean":"2024-04-24T06:40:48.129484+0000","last_became_active":"2024-04-24T06:40:48.126089+0000","last_became_peered":"2024-04-24T06:40:48.126089+0000","last_unstale":"2024-04-24T06:40:48.129484+0000","last_undegraded":"2024-04-24T06:40:48.129484+0000","last_fullsized":"2024-04-24T06:40:48.129484+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:23.489914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"109'2","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.145553+0000","last_change":"2024-04-24T06:40:47.103687+0000","last_active":"2024-04-24T06:40:48.145553+0000","last_peered":"2024-04-24T06:40:48.145553+0000","last_clean":"2024-04-24T06:40:48.145553+0000","last_became_active":"2024-04-24T06:40:47.102617+0000","last_became_peered":"2024-04-24T06:40:47.102617+0000","last_unstale":"2024-04-24T06:40:48.145553+0000","last_undegraded":"2024-04-24T06:40:48.145553+0000","last_fullsized":"2024-04-24T06:40:48.145553+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:10.746491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.114601+0000","last_change":"2024-04-24T06:40:48.110130+0000","last_active":"2024-04-24T06:40:48.114601+0000","last_peered":"2024-04-24T06:40:48.114601+0000","last_clean":"2024-04-24T06:40:48.114601+0000","last_became_active":"2024-04-24T06:40:48.109546+0000","last_became_peered":"2024-04-24T06:40:48.109546+0000","last_unstale":"2024-04-24T06:40:48.114601+0000","last_undegraded":"2024-04-24T06:40:48.114601+0000","last_fullsized":"2024-04-24T06:40:48.114601+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:53:52.407256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.118983+0000","last_change":"2024-04-24T06:40:47.090448+0000","last_active":"2024-04-24T06:40:48.118983+0000","last_peered":"2024-04-24T06:40:48.118983+0000","last_clean":"2024-04-24T06:40:48.118983+0000","last_became_active":"2024-04-24T06:40:47.090089+0000","last_became_peered":"2024-04-24T06:40:47.090089+0000","last_unstale":"2024-04-24T06:40:48.118983+0000","last_undegraded":"2024-04-24T06:40:48.118983+0000","last_fullsized":"2024-04-24T06:40:48.118983+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:20:50.893122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.115840+0000","last_change":"2024-04-24T06:40:48.109616+0000","last_active":"2024-04-24T06:40:48.115840+0000","last_peered":"2024-04-24T06:40:48.115840+0000","last_clean":"2024-04-24T06:40:48.115840+0000","last_became_active":"2024-04-24T06:40:48.109286+0000","last_became_peered":"2024-04-24T06:40:48.109286+0000","last_unstale":"2024-04-24T06:40:48.115840+0000","last_undegraded":"2024-04-24T06:40:48.115840+0000","last_fullsized":"2024-04-24T06:40:48.115840+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:01:36.273184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.090965+0000","last_change":"2024-04-24T06:40:47.101373+0000","last_active":"2024-04-24T06:40:48.090965+0000","last_peered":"2024-04-24T06:40:48.090965+0000","last_clean":"2024-04-24T06:40:48.090965+0000","last_became_active":"2024-04-24T06:40:47.101006+0000","last_became_peered":"2024-04-24T06:40:47.101006+0000","last_unstale":"2024-04-24T06:40:48.090965+0000","last_undegraded":"2024-04-24T06:40:48.090965+0000","last_fullsized":"2024-04-24T06:40:48.090965+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:54:14.204841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.127234+0000","last_change":"2024-04-24T06:40:48.125702+0000","last_active":"2024-04-24T06:40:48.127234+0000","last_peered":"2024-04-24T06:40:48.127234+0000","last_clean":"2024-04-24T06:40:48.127234+0000","last_became_active":"2024-04-24T06:40:48.125294+0000","last_became_peered":"2024-04-24T06:40:48.125294+0000","last_unstale":"2024-04-24T06:40:48.127234+0000","last_undegraded":"2024-04-24T06:40:48.127234+0000","last_fullsized":"2024-04-24T06:40:48.127234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.108835+0000","last_change":"2024-04-24T06:40:47.095708+0000","last_active":"2024-04-24T06:40:48.108835+0000","last_peered":"2024-04-24T06:40:48.108835+0000","last_clean":"2024-04-24T06:40:48.108835+0000","last_became_active":"2024-04-24T06:40:47.095330+0000","last_became_peered":"2024-04-24T06:40:47.095330+0000","last_unstale":"2024-04-24T06:40:48.108835+0000","last_undegraded":"2024-04-24T06:40:48.108835+0000","last_fullsized":"2024-04-24T06:40:48.108835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:01:03.347264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.126811+0000","last_change":"2024-04-24T06:40:48.113751+0000","last_active":"2024-04-24T06:40:48.126811+0000","last_peered":"2024-04-24T06:40:48.126811+0000","last_clean":"2024-04-24T06:40:48.126811+0000","last_became_active":"2024-04-24T06:40:48.113444+0000","last_became_peered":"2024-04-24T06:40:48.113444+0000","last_unstale":"2024-04-24T06:40:48.126811+0000","last_undegraded":"2024-04-24T06:40:48.126811+0000","last_fullsized":"2024-04-24T06:40:48.126811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:20.570062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.102911+0000","last_change":"2024-04-24T06:40:47.084334+0000","last_active":"2024-04-24T06:40:48.102911+0000","last_peered":"2024-04-24T06:40:48.102911+0000","last_clean":"2024-04-24T06:40:48.102911+0000","last_became_active":"2024-04-24T06:40:47.083740+0000","last_became_peered":"2024-04-24T06:40:47.083740+0000","last_unstale":"2024-04-24T06:40:48.102911+0000","last_undegraded":"2024-04-24T06:40:48.102911+0000","last_fullsized":"2024-04-24T06:40:48.102911+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:50:42.366685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.129140+0000","last_change":"2024-04-24T06:40:48.124343+0000","last_active":"2024-04-24T06:40:48.129140+0000","last_peered":"2024-04-24T06:40:48.129140+0000","last_clean":"2024-04-24T06:40:48.129140+0000","last_became_active":"2024-04-24T06:40:48.124003+0000","last_became_peered":"2024-04-24T06:40:48.124003+0000","last_unstale":"2024-04-24T06:40:48.129140+0000","last_undegraded":"2024-04-24T06:40:48.129140+0000","last_fullsized":"2024-04-24T06:40:48.129140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:42.336316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.097970+0000","last_change":"2024-04-24T06:40:47.093061+0000","last_active":"2024-04-24T06:40:48.097970+0000","last_peered":"2024-04-24T06:40:48.097970+0000","last_clean":"2024-04-24T06:40:48.097970+0000","last_became_active":"2024-04-24T06:40:47.092152+0000","last_became_peered":"2024-04-24T06:40:47.092152+0000","last_unstale":"2024-04-24T06:40:48.097970+0000","last_undegraded":"2024-04-24T06:40:48.097970+0000","last_fullsized":"2024-04-24T06:40:48.097970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:41.300615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.138132+0000","last_change":"2024-04-24T06:40:48.136226+0000","last_active":"2024-04-24T06:40:48.138132+0000","last_peered":"2024-04-24T06:40:48.138132+0000","last_clean":"2024-04-24T06:40:48.138132+0000","last_became_active":"2024-04-24T06:40:48.134282+0000","last_became_peered":"2024-04-24T06:40:48.134282+0000","last_unstale":"2024-04-24T06:40:48.138132+0000","last_undegraded":"2024-04-24T06:40:48.138132+0000","last_fullsized":"2024-04-24T06:40:48.138132+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:06.005981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116400+0000","last_change":"2024-04-24T06:40:47.097573+0000","last_active":"2024-04-24T06:40:48.116400+0000","last_peered":"2024-04-24T06:40:48.116400+0000","last_clean":"2024-04-24T06:40:48.116400+0000","last_became_active":"2024-04-24T06:40:47.097293+0000","last_became_peered":"2024-04-24T06:40:47.097293+0000","last_unstale":"2024-04-24T06:40:48.116400+0000","last_undegraded":"2024-04-24T06:40:48.116400+0000","last_fullsized":"2024-04-24T06:40:48.116400+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:37:44.278900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.113859+0000","last_change":"2024-04-24T06:40:48.107774+0000","last_active":"2024-04-24T06:40:48.113859+0000","last_peered":"2024-04-24T06:40:48.113859+0000","last_clean":"2024-04-24T06:40:48.113859+0000","last_became_active":"2024-04-24T06:40:48.107355+0000","last_became_peered":"2024-04-24T06:40:48.107355+0000","last_unstale":"2024-04-24T06:40:48.113859+0000","last_undegraded":"2024-04-24T06:40:48.113859+0000","last_fullsized":"2024-04-24T06:40:48.113859+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:03:09.868926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.092210+0000","last_change":"2024-04-24T06:40:47.101096+0000","last_active":"2024-04-24T06:40:48.092210+0000","last_peered":"2024-04-24T06:40:48.092210+0000","last_clean":"2024-04-24T06:40:48.092210+0000","last_became_active":"2024-04-24T06:40:47.100795+0000","last_became_peered":"2024-04-24T06:40:47.100795+0000","last_unstale":"2024-04-24T06:40:48.092210+0000","last_undegraded":"2024-04-24T06:40:48.092210+0000","last_fullsized":"2024-04-24T06:40:48.092210+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:44:53.447165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.130903+0000","last_change":"2024-04-24T06:40:48.128357+0000","last_active":"2024-04-24T06:40:48.130903+0000","last_peered":"2024-04-24T06:40:48.130903+0000","last_clean":"2024-04-24T06:40:48.130903+0000","last_became_active":"2024-04-24T06:40:48.127698+0000","last_became_peered":"2024-04-24T06:40:48.127698+0000","last_unstale":"2024-04-24T06:40:48.130903+0000","last_undegraded":"2024-04-24T06:40:48.130903+0000","last_fullsized":"2024-04-24T06:40:48.130903+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:47.049856+0000","last_clean_scrub_stamp":"2024-04-24T06:40:47.049856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:43.167945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.105232+0000","last_change":"2024-04-24T06:40:47.099550+0000","last_active":"2024-04-24T06:40:48.105232+0000","last_peered":"2024-04-24T06:40:48.105232+0000","last_clean":"2024-04-24T06:40:48.105232+0000","last_became_active":"2024-04-24T06:40:47.099300+0000","last_became_peered":"2024-04-24T06:40:47.099300+0000","last_unstale":"2024-04-24T06:40:48.105232+0000","last_undegraded":"2024-04-24T06:40:48.105232+0000","last_fullsized":"2024-04-24T06:40:48.105232+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:40:46.034031+0000","last_clean_scrub_stamp":"2024-04-24T06:40:46.034031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:47:32.402031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":333,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-24T06:40:48.116497+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:40:48.116497+0000","last_peered":"2024-04-24T06:40:48.116497+0000","last_clean":"2024-04-24T06:40:48.116497+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:40:48.116497+0000","last_undegraded":"2024-04-24T06:40:48.116497+0000","last_fullsized":"2024-04-24T06:40:48.116497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1171143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":587550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1363968,"data_stored":587674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":1179642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:40:53.093 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:40:53.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T06:40:53.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.359+0000 7fe982400640 1 Processor -- start 2024-04-24T06:40:53.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.360+0000 7fe982400640 1 -- start start 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.361+0000 7fe982400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 0x7fe97c169820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.361+0000 7fe982400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c0d32c0 con 0x7fe97c0b3630 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.361+0000 7fe982400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c0cf6d0 con 0x7fe97c15ec70 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.361+0000 7fe982400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c0d02e0 con 0x7fe97c06db50 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.362+0000 7fe97be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe97c0b3630 0x7fe97c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58348/0 (socket says 172.21.15.59:58348) 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.362+0000 7fe97be00640 1 -- 172.21.15.59:0/1535512670 learned_addr learned my addr 172.21.15.59:0/1535512670 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.362+0000 7fe981a00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 0x7fe97c169820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.362+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212328804 0 0) 0x7fe97c0d32c0 con 0x7fe97c0b3630 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.362+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe960003520 con 0x7fe97c0b3630 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3201403422 0 0) 0x7fe960003520 con 0x7fe97c0b3630 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 msgr2=0x7fe97c169820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 0x7fe97c169820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 >> v1:172.21.15.66:6789/0 conn(0x7fe97c15ec70 legacy=0x7fe97c0d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c0cf9b0 con 0x7fe97c0b3630 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe964001670 con 0x7fe97c0b3630 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 >> v1:172.21.15.59:6789/0 conn(0x7fe97c0b3630 legacy=0x7fe97c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 0x7fe9600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe960003fb0 0x7fe960004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe960004d40 0x7fe9600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c0d02e0 con 0x7fe960003fb0 2024-04-24T06:40:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe96000bb00 con 0x7fe960004d40 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.363+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe96000bd00 con 0x7fe97c06db50 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe960003fb0 0x7fe960004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe981a00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe960004d40 0x7fe9600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 -- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 msgr2=0x7fe9600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c06db50 0x7fe9600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 -- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe960004d40 msgr2=0x7fe9600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe960004d40 0x7fe9600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.364+0000 7fe97be00640 1 -- 172.21.15.59:0/1535512670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe960003520 con 0x7fe960003fb0 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.365+0000 7fe97be00640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe960003fb0 0x7fe960004800 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c190 tx=0x7fe960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eeede872d22689 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964007c60 con 0x7fe960003fb0 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964005af0 con 0x7fe960003fb0 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe97b400640 1 -- 172.21.15.59:0/1535512670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964007070 con 0x7fe960003fb0 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 -- 172.21.15.59:0/1535512670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe960003fb0 msgr2=0x7fe960004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 --2- 172.21.15.59:0/1535512670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe960003fb0 0x7fe960004800 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c190 tx=0x7fe960005830 comp rx=0 tx=0).stop 2024-04-24T06:40:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe981a00640 1 -- 172.21.15.59:0/1535512670 reap_dead start 2024-04-24T06:40:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 -- 172.21.15.59:0/1535512670 shutdown_connections 2024-04-24T06:40:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 -- 172.21.15.59:0/1535512670 >> 172.21.15.59:0/1535512670 conn(0x7fe97c08bf90 msgr2=0x7fe97c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 -- 172.21.15.59:0/1535512670 shutdown_connections 2024-04-24T06:40:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.366+0000 7fe982400640 1 -- 172.21.15.59:0/1535512670 wait complete. 2024-04-24T06:40:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.368+0000 7fe982400640 1 Processor -- start 2024-04-24T06:40:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.368+0000 7fe982400640 1 -- start start 2024-04-24T06:40:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c1578d0 0x7fe97c0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97c0b74b0 0x7fe97c161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c15f050 con 0x7fe97c0b8090 2024-04-24T06:40:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c0cff70 con 0x7fe97c0b74b0 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe982400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c0cf260 con 0x7fe97c1578d0 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.369+0000 7fe97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55898/0 (socket says 172.21.15.59:55898) 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 -- 172.21.15.59:0/3136761677 learned_addr learned my addr 172.21.15.59:0/3136761677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe981000640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c1578d0 0x7fe97c0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe981a00640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97c0b74b0 0x7fe97c161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 -- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c1578d0 msgr2=0x7fe97c0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c1578d0 0x7fe97c0b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 -- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97c0b74b0 msgr2=0x7fe97c161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97c0b74b0 0x7fe97c161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 -- 172.21.15.59:0/3136761677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c0c7e70 con 0x7fe97c0b8090 2024-04-24T06:40:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.370+0000 7fe97be00640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fe96402ac20 tx=0x7fe964001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.371+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964041020 con 0x7fe97c0b8090 2024-04-24T06:40:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.371+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964038040 con 0x7fe97c0b8090 2024-04-24T06:40:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.371+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964007290 con 0x7fe97c0b8090 2024-04-24T06:40:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.371+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c0c8150 con 0x7fe97c0b8090 2024-04-24T06:40:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.371+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c0c8710 con 0x7fe97c0b8090 2024-04-24T06:40:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.372+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c04fd90 con 0x7fe97c0b8090 2024-04-24T06:40:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.374+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe964044020 con 0x7fe97c0b8090 2024-04-24T06:40:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.375+0000 7fe978c00640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.375+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fe9640abbd0 con 0x7fe97c0b8090 2024-04-24T06:40:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.375+0000 7fe981000640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.376+0000 7fe981000640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 0x7fe95806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe96c007fd0 tx=0x7fe96c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.379+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe964005e20 con 0x7fe97c0b8090 2024-04-24T06:40:53.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.657+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe97c0b12f0 con 0x7fe97c0b8090 2024-04-24T06:40:53.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.663+0000 7fe978c00640 1 -- 172.21.15.59:0/3136761677 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fe964005ac0 con 0x7fe97c0b8090 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBbpihmo5tNKBAAG1PnUS+Nuu47woihGIlToA== 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T06:40:53.664 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.665+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 msgr2=0x7fe95806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.665+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 0x7fe95806edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe96c007fd0 tx=0x7fe96c009040 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.665+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 msgr2=0x7fe97c161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.665+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fe96402ac20 tx=0x7fe964001970 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 shutdown_connections 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97c0b74b0 0x7fe97c161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c0b8090 0x7fe97c161230 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 --2- 172.21.15.59:0/3136761677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97c1578d0 0x7fe97c0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 >> 172.21.15.59:0/3136761677 conn(0x7fe97c08bf90 msgr2=0x7fe97c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 shutdown_connections 2024-04-24T06:40:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.666+0000 7fe982400640 1 -- 172.21.15.59:0/3136761677 wait complete. 2024-04-24T06:40:53.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.893+0000 7efecf200640 1 Processor -- start 2024-04-24T06:40:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.893+0000 7efecf200640 1 -- start start 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efed015c700 0x7efed00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7efed006bf10 0x7efed00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efed00b46b0 con 0x7efed015c700 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7efed00d0c70 con 0x7efed006bf10 2024-04-24T06:40:53.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efed00d0f50 con 0x7efed0160d50 2024-04-24T06:40:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.894+0000 7efecde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efed015c700 0x7efed00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efed015c700 0x7efed00d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55908/0 (socket says 172.21.15.59:55908) 2024-04-24T06:40:53.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 -- 172.21.15.59:0/4166237905 learned_addr learned my addr 172.21.15.59:0/4166237905 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efece800640 1 --2- 172.21.15.59:0/4166237905 >> v2:172.21.15.66:3300/0 conn(0x7efed006bf10 0x7efed00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecd400640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 -- 172.21.15.59:0/4166237905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 msgr2=0x7efed006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed006b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 -- 172.21.15.59:0/4166237905 >> v2:172.21.15.66:3300/0 conn(0x7efed006bf10 msgr2=0x7efed00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 --2- 172.21.15.59:0/4166237905 >> v2:172.21.15.66:3300/0 conn(0x7efed006bf10 0x7efed00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecde00640 1 -- 172.21.15.59:0/4166237905 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed00b4340 con 0x7efed015c700 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.895+0000 7efecd400640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.896+0000 7efecde00640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed00d0600 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7efed00b2da0 tx=0x7efec002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d7c5810272d5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecca00640 1 -- 172.21.15.59:0/4166237905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0037070 con 0x7efed015c700 2024-04-24T06:40:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecca00640 1 -- 172.21.15.59:0/4166237905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0032440 con 0x7efed015c700 2024-04-24T06:40:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecca00640 1 -- 172.21.15.59:0/4166237905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0032810 con 0x7efed015c700 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 -- 172.21.15.59:0/4166237905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 msgr2=0x7efed00d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed00d0600 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7efed00b2da0 tx=0x7efec002a140 comp rx=0 tx=0).stop 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 -- 172.21.15.59:0/4166237905 shutdown_connections 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 --2- 172.21.15.59:0/4166237905 >> v2:172.21.15.66:3300/0 conn(0x7efed006bf10 0x7efed00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 --2- 172.21.15.59:0/4166237905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed00d0600 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 -- 172.21.15.59:0/4166237905 >> 172.21.15.59:0/4166237905 conn(0x7efed009aa00 msgr2=0x7efed00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 -- 172.21.15.59:0/4166237905 shutdown_connections 2024-04-24T06:40:53.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.897+0000 7efecf200640 1 -- 172.21.15.59:0/4166237905 wait complete. 2024-04-24T06:40:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.899+0000 7efecf200640 1 Processor -- start 2024-04-24T06:40:53.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.899+0000 7efecf200640 1 -- start start 2024-04-24T06:40:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed006bf10 0x7efed014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed00b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efed00ce210 con 0x7efed015c700 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efed01611b0 con 0x7efed006bf10 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efed00ce390 con 0x7efed0160d50 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.900+0000 7efecd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55920/0 (socket says 172.21.15.59:55920) 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 -- 172.21.15.59:0/605309647 learned_addr learned my addr 172.21.15.59:0/605309647 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecde00640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed006bf10 0x7efed014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efece800640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed00b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 -- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 msgr2=0x7efed00b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed00b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 -- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed006bf10 msgr2=0x7efed014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed006bf10 0x7efed014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 -- 172.21.15.59:0/605309647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed00c9100 con 0x7efed015c700 2024-04-24T06:40:53.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.901+0000 7efecd400640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c7f0 tx=0x7efebc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.902+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc015070 con 0x7efed015c700 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.902+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc007820 con 0x7efed015c700 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.902+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc007bd0 con 0x7efed015c700 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.902+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed0156d70 con 0x7efed015c700 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.902+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed0157330 con 0x7efed015c700 2024-04-24T06:40:53.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.903+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed0156b90 con 0x7efed015c700 2024-04-24T06:40:53.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.904+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efebc018070 con 0x7efed015c700 2024-04-24T06:40:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.905+0000 7efec6000640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 0x7efeb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.905+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7efebc08bd60 con 0x7efed015c700 2024-04-24T06:40:53.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.906+0000 7efecde00640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 0x7efeb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:53.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.906+0000 7efecde00640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 0x7efeb006edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efec0034000 tx=0x7efec002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:53.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:53.910+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efed0156b90 con 0x7efed015c700 2024-04-24T06:40:54.044 INFO:tasks.ceph.mds.b.smithi066.stderr:2024-04-24T06:40:54.042+0000 7fa229800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:40:54.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.154+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efed004fd90 con 0x7efed015c700 2024-04-24T06:40:54.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.155+0000 7efec6000640 1 -- 172.21.15.59:0/605309647 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5182 (secure 0 0 0) 0x7efebc05a0f0 con 0x7efed015c700 2024-04-24T06:40:54.156 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:54.156 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:40:50.638009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9228":{"gid":9228,"name":"b","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/2441853137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":2441853137},{"type":"v1","addr":"172.21.15.66:6835","nonce":2441853137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9228,"qdb_cluster":[9228]},"id":8}]} 2024-04-24T06:40:54.156 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 100 2024-04-24T06:40:54.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.157+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 msgr2=0x7efeb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.157+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 0x7efeb006edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efec0034000 tx=0x7efec002e000 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.157+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 msgr2=0x7efed0157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.157+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7efebc00c7f0 tx=0x7efebc00ccb0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 shutdown_connections 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efeb006c8f0 0x7efeb006edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efed0160d50 0x7efed00b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed015c700 0x7efed0157850 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 --2- 172.21.15.59:0/605309647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed006bf10 0x7efed014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 >> 172.21.15.59:0/605309647 conn(0x7efed009aa00 msgr2=0x7efed00cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 shutdown_connections 2024-04-24T06:40:54.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.158+0000 7efecf200640 1 -- 172.21.15.59:0/605309647 wait complete. 2024-04-24T06:40:54.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:40:50.638009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9228}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9228': {'gid': 9228, 'name': 'b', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/2441853137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 2441853137}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 2441853137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9228, 'qdb_cluster': [9228]} 2024-04-24T06:40:54.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:40:54.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:40:54.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.797+0000 7fba1de00640 1 Processor -- start 2024-04-24T06:40:54.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.797+0000 7fba1de00640 1 -- start start 2024-04-24T06:40:54.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba1de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba1813f830 0x7fba180ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba1de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba180cd450 0x7fba1806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba1de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba1815d180 con 0x7fba1815cc10 2024-04-24T06:40:54.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba1de00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fba1815d460 con 0x7fba180cd450 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba1de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fba1815d740 con 0x7fba1813f830 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.799+0000 7fba17400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba180cd450 0x7fba1806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba180cd450 0x7fba1806bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55922/0 (socket says 172.21.15.59:55922) 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 -- 172.21.15.59:0/3911964846 learned_addr learned my addr 172.21.15.59:0/3911964846 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba16a00640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba1813f830 0x7fba180ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 -- 172.21.15.59:0/3911964846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba1813f830 msgr2=0x7fba180ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba1813f830 0x7fba180ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 -- 172.21.15.59:0/3911964846 >> v1:172.21.15.66:6789/0 conn(0x7fba1815cc10 legacy=0x7fba180ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.800+0000 7fba17400640 1 -- 172.21.15.59:0/3911964846 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1815da20 con 0x7fba180cd450 2024-04-24T06:40:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.801+0000 7fba17400640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba180cd450 0x7fba1806bed0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320da82df502ddf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.801+0000 7fba16000640 1 -- 172.21.15.59:0/3911964846 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10036070 con 0x7fba180cd450 2024-04-24T06:40:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba16000640 1 -- 172.21.15.59:0/3911964846 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba10031600 con 0x7fba180cd450 2024-04-24T06:40:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba16000640 1 -- 172.21.15.59:0/3911964846 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100319d0 con 0x7fba180cd450 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 -- 172.21.15.59:0/3911964846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba180cd450 msgr2=0x7fba1806bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba180cd450 0x7fba1806bed0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002a6c0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 -- 172.21.15.59:0/3911964846 shutdown_connections 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba180cd450 0x7fba1806bed0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 --2- 172.21.15.59:0/3911964846 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba1813f830 0x7fba180ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 -- 172.21.15.59:0/3911964846 >> 172.21.15.59:0/3911964846 conn(0x7fba180d0c00 msgr2=0x7fba180d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 -- 172.21.15.59:0/3911964846 shutdown_connections 2024-04-24T06:40:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.802+0000 7fba1de00640 1 -- 172.21.15.59:0/3911964846 wait complete. 2024-04-24T06:40:54.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.804+0000 7fba1de00640 1 Processor -- start 2024-04-24T06:40:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.804+0000 7fba1de00640 1 -- start start 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba180cd450 0x7fba18139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba1815cc10 0x7fba1814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba1806c410 con 0x7fba1813f830 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba180cdb70 con 0x7fba1815cc10 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba1de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fba1806c590 con 0x7fba180cd450 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55928/0 (socket says 172.21.15.59:55928) 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.805+0000 7fba0fe00640 1 -- 172.21.15.59:0/1311377160 learned_addr learned my addr 172.21.15.59:0/1311377160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba17400640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba1815cc10 0x7fba1814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba16a00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba180cd450 0x7fba18139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 -- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba180cd450 msgr2=0x7fba18139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba180cd450 0x7fba18139090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 -- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba1815cc10 msgr2=0x7fba1814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba1815cc10 0x7fba1814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 -- 172.21.15.59:0/1311377160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba181378e0 con 0x7fba1813f830 2024-04-24T06:40:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b3f0 tx=0x7fba0000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0f400640 1 -- 172.21.15.59:0/1311377160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00015020 con 0x7fba1813f830 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.806+0000 7fba0f400640 1 -- 172.21.15.59:0/1311377160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba00011830 con 0x7fba1813f830 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.807+0000 7fba0f400640 1 -- 172.21.15.59:0/1311377160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00011c20 con 0x7fba1813f830 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.807+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba18136b20 con 0x7fba1813f830 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.807+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba181370e0 con 0x7fba1813f830 2024-04-24T06:40:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.809+0000 7fba0f400640 1 -- 172.21.15.59:0/1311377160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba00018020 con 0x7fba1813f830 2024-04-24T06:40:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.811+0000 7fba0f400640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.811+0000 7fba0f400640 1 -- 172.21.15.59:0/1311377160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fba0008b760 con 0x7fba1813f830 2024-04-24T06:40:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.811+0000 7fba16a00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.811+0000 7fba20dac740 1 Processor -- start 2024-04-24T06:40:54.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.811+0000 7fba20dac740 1 -- start start 2024-04-24T06:40:54.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba16a00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94de8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e18190 0x555f94decfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f94d52b10 con 0x555f94e188d0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555f94cf92b0 con 0x555f94e357b0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.812+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555f94d4ead0 con 0x555f94e18190 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55942/0 (socket says 172.21.15.59:55942) 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 -- 172.21.15.59:0/4060105793 learned_addr learned my addr 172.21.15.59:0/4060105793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba16a00640 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94de8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 -- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e18190 msgr2=0x555f94decfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e18190 0x555f94decfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 -- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 msgr2=0x555f94de8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94de8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 -- 172.21.15.59:0/4060105793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f94a652d0 con 0x555f94e188d0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba16a00640 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94de8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba0fe00640 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fba00089bc0 tx=0x7fba00059cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131e9f02b238937c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.813+0000 7fba07400640 1 -- 172.21.15.59:0/4060105793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00088070 con 0x555f94e188d0 2024-04-24T06:40:54.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba07400640 1 -- 172.21.15.59:0/4060105793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba00011dc0 con 0x555f94e188d0 2024-04-24T06:40:54.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba07400640 1 -- 172.21.15.59:0/4060105793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00071070 con 0x555f94e188d0 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 -- 172.21.15.59:0/4060105793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 msgr2=0x555f94e17c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fba00089bc0 tx=0x7fba00059cc0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 -- 172.21.15.59:0/4060105793 shutdown_connections 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e18190 0x555f94decfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e17c50 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 --2- 172.21.15.59:0/4060105793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94de8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 -- 172.21.15.59:0/4060105793 >> 172.21.15.59:0/4060105793 conn(0x555f94dea3b0 msgr2=0x555f94e32c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 -- 172.21.15.59:0/4060105793 shutdown_connections 2024-04-24T06:40:54.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.814+0000 7fba20dac740 1 -- 172.21.15.59:0/4060105793 wait complete. 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.815+0000 7fba20dac740 1 -- start start 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.815+0000 7fba20dac740 1 Processor -- start 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94e29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94e19750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f94dec560 con 0x555f94e357b0 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555f94d0c590 con 0x555f94e188d0 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555f94deb950 con 0x555f94e34550 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.816+0000 7fba20dac740 10 client.0 _ll_register_callbacks cb 0x555f94cef3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55944/0 (socket says 172.21.15.59:55944) 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 -- 172.21.15.59:0/3958380585 learned_addr learned my addr 172.21.15.59:0/3958380585 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba16a00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94e29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 -- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 msgr2=0x555f94e19750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94e19750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 -- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 msgr2=0x555f94e29ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94e29ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 -- 172.21.15.59:0/3958380585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f94b2f6b0 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fba0fe00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b3c0 tx=0x7fba00009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9596813941a42c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fb9fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00088040 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.817+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fba00055af0 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00011dc0 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fba0007e050 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 10 client.9357 fetch_fsmap learned FSMap version 100 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x555f94e33d50 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba00011dc0 con 0x555f94e357b0 2024-04-24T06:40:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fba00081020 con 0x555f94e357b0 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 10 client.9357 fetch_fsmap finished waiting for FSMap version 100 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 10 client.9357 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 20 client.9357 populate_metadata read hostname 'smithi059' 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e4ec90 0x555f94e51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 4 client.9357 mds_command: new command op to 9228 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.818+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 --> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f94d60530 con 0x555f94e4ec90 2024-04-24T06:40:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.819+0000 7fba16a00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e4ec90 0x555f94e51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.820+0000 7fba16a00640 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e4ec90 0x555f94e51080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea94d837f9aeeaf2 server_cookie=ee67cd2007fc71ee in_seq=0 out_seq=0 2024-04-24T06:40:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.820+0000 7fb9fbe00640 10 client.9357 ms_handle_connect on v2:172.21.15.66:6833/2441853137 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fb9fbe00640 1 -- 172.21.15.59:0/3958380585 <== mds.0 v2:172.21.15.66:6833/2441853137 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f94d60530 con 0x555f94e4ec90 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fb9fbe00640 10 client.9357 handle_command_reply: tid=0 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 client.9357 shutdown 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 msgr2=0x555f94e1b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b3c0 tx=0x7fba00009300 comp rx=0 tx=0).stop 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 shutdown_connections 2024-04-24T06:40:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e4ec90 0x555f94e51080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94e19750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e357b0 0x555f94e1b400 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 --2- 172.21.15.59:0/3958380585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94e29ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fb9fbe00640 0 client.9357 ms_handle_reset on v2:172.21.15.66:6833/2441853137 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 >> 172.21.15.59:0/3958380585 conn(0x555f94e18190 msgr2=0x555f94e36070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 shutdown_connections 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.823+0000 7fba20dac740 1 -- 172.21.15.59:0/3958380585 wait complete. 2024-04-24T06:40:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.824+0000 7fba20dac740 20 client.9357 trim_cache size 0 max 16384 2024-04-24T06:40:54.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 Processor -- start 2024-04-24T06:40:54.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 -- start start 2024-04-24T06:40:54.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94de38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94e4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba181fffa0 con 0x555f94e188d0 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba1806fff0 con 0x555f94e357b0 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.844+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fba181ff950 con 0x555f94e34550 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55954/0 (socket says 172.21.15.59:55954) 2024-04-24T06:40:54.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 -- 172.21.15.59:0/4090758191 learned_addr learned my addr 172.21.15.59:0/4090758191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba17400640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94e4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba0fe00640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94de38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 -- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 msgr2=0x555f94de38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94de38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 -- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 msgr2=0x555f94e4be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94e4be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba16a00640 1 -- 172.21.15.59:0/4090758191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f94e29e40 con 0x555f94e188d0 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.845+0000 7fba17400640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94e4be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba16a00640 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x555f94deba40 tx=0x7fba08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28bf862bc94408c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba06a00640 1 -- 172.21.15.59:0/4090758191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0802c070 con 0x555f94e188d0 2024-04-24T06:40:54.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba06a00640 1 -- 172.21.15.59:0/4090758191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba08015890 con 0x555f94e188d0 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba06a00640 1 -- 172.21.15.59:0/4090758191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800f040 con 0x555f94e188d0 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba20dac740 1 -- 172.21.15.59:0/4090758191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 msgr2=0x555f94e4b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.846+0000 7fba20dac740 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x555f94deba40 tx=0x7fba08014000 comp rx=0 tx=0).stop 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 -- 172.21.15.59:0/4090758191 shutdown_connections 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e357b0 0x555f94e4be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e34550 0x555f94de38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 --2- 172.21.15.59:0/4090758191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e188d0 0x555f94e4b3a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 -- 172.21.15.59:0/4090758191 >> 172.21.15.59:0/4090758191 conn(0x555f94e18190 msgr2=0x555f94debd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 -- 172.21.15.59:0/4090758191 shutdown_connections 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 -- 172.21.15.59:0/4090758191 wait complete. 2024-04-24T06:40:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.847+0000 7fba20dac740 1 -- start start 2024-04-24T06:40:54.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.848+0000 7fba20dac740 1 Processor -- start 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94ead020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e357b0 0x555f94e364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f94dec560 con 0x555f94e34550 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555f94d0c590 con 0x555f94e188d0 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555f94e2bb30 con 0x555f94e357b0 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba20dac740 10 client.0 _ll_register_callbacks cb 0x7fba00055780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55958/0 (socket says 172.21.15.59:55958) 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 -- 172.21.15.59:0/1954435583 learned_addr learned my addr 172.21.15.59:0/1954435583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:54.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 -- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e357b0 msgr2=0x555f94e364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e357b0 0x555f94e364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 -- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 msgr2=0x555f94ead020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94ead020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.849+0000 7fba0fe00640 1 -- 172.21.15.59:0/1954435583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f94ded3a0 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fba0fe00640 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fba0000bdb0 tx=0x7fba00059380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd1d8a6ec877a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00088040 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fba000551d0 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba08015890 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fba00071040 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fba20dac740 10 client.9363 fetch_fsmap learned FSMap version 100 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x555f94e568c0 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba08015890 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.850+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fba00081050 con 0x555f94e34550 2024-04-24T06:40:54.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 10 client.9363 fetch_fsmap finished waiting for FSMap version 100 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 10 client.9363 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 20 client.9363 populate_metadata read hostname 'smithi059' 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e10a70 0x555f94e55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 4 client.9363 mds_command: new command op to 9228 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 --> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] -- command(tid 0: {"prefix": "status"}) -- 0x555f94d5a3a0 con 0x555f94e10a70 2024-04-24T06:40:54.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.851+0000 7fba16a00640 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e10a70 0x555f94e55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.852+0000 7fba16a00640 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e10a70 0x555f94e55840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d844c83bb681746 server_cookie=fdc48af66df02531 in_seq=0 out_seq=0 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.852+0000 7fb9fbe00640 10 client.9363 ms_handle_connect on v2:172.21.15.66:6833/2441853137 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fb9fbe00640 1 -- 172.21.15.59:0/1954435583 <== mds.0 v2:172.21.15.66:6833/2441853137 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x555f94d5a3a0 con 0x555f94e10a70 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fb9fbe00640 10 client.9363 handle_command_reply: tid=0 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 client.9363 shutdown 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 msgr2=0x555f94e35fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fba0000bdb0 tx=0x7fba00059380 comp rx=0 tx=0).stop 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 shutdown_connections 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555f94e10a70 0x555f94e55840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555f94e357b0 0x555f94e364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f94e34550 0x555f94e35fb0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fb9fbe00640 0 client.9363 ms_handle_reset on v2:172.21.15.66:6833/2441853137 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 --2- 172.21.15.59:0/1954435583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f94e188d0 0x555f94ead020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 >> 172.21.15.59:0/1954435583 conn(0x555f94e18190 msgr2=0x555f94e4b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.853+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 shutdown_connections 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba20dac740 1 -- 172.21.15.59:0/1954435583 wait complete. 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba20dac740 20 client.9363 trim_cache size 0 max 16384 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9228, 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7410997300000002, 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 100, 2024-04-24T06:40:54.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 110, 2024-04-24T06:40:54.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 110, 2024-04-24T06:40:54.858 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.126293411000001 2024-04-24T06:40:54.858 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:40:54.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 msgr2=0x7fb9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08008040 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 msgr2=0x7fba1814ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b3f0 tx=0x7fba0000b8b0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 shutdown_connections 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba1815cc10 0x7fba1814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1813f830 0x7fba1814ec60 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 --2- 172.21.15.59:0/1311377160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba180cd450 0x7fba18139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.854+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 >> 172.21.15.59:0/1311377160 conn(0x7fba180d0c00 msgr2=0x7fba180b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.855+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 shutdown_connections 2024-04-24T06:40:54.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:54.855+0000 7fba1de00640 1 -- 172.21.15.59:0/1311377160 wait complete. 2024-04-24T06:40:54.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:40:55.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.070+0000 7fe037200640 1 Processor -- start 2024-04-24T06:40:55.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.070+0000 7fe037200640 1 -- start start 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe037200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe0380cd050 0x7fe03806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe037200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0380b3080 0x7fe0380cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe037200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03815f160 con 0x7fe03815ece0 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe037200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe0380ced20 con 0x7fe0380b3080 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe037200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe03815fcd0 con 0x7fe0380cd050 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe02fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe03815ece0 0x7fe0380cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58352/0 (socket says 172.21.15.59:58352) 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe02fe00640 1 -- 172.21.15.59:0/4050963185 learned_addr learned my addr 172.21.15.59:0/4050963185 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:55.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.072+0000 7fe036800640 1 --2- 172.21.15.59:0/4050963185 >> v2:172.21.15.59:3300/0 conn(0x7fe0380b3080 0x7fe0380cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe035e00640 1 --2- 172.21.15.59:0/4050963185 >> v2:172.21.15.66:3300/0 conn(0x7fe0380cd050 0x7fe03806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe035400640 1 -- 172.21.15.59:0/4050963185 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1448556036 0 0) 0x7fe03815f160 con 0x7fe03815ece0 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe035400640 1 -- 172.21.15.59:0/4050963185 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe01c003520 con 0x7fe03815ece0 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe036800640 1 -- 172.21.15.59:0/4050963185 >> v2:172.21.15.66:3300/0 conn(0x7fe0380cd050 msgr2=0x7fe03806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe036800640 1 --2- 172.21.15.59:0/4050963185 >> v2:172.21.15.66:3300/0 conn(0x7fe0380cd050 0x7fe03806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe036800640 1 -- 172.21.15.59:0/4050963185 >> v1:172.21.15.59:6789/0 conn(0x7fe03815ece0 legacy=0x7fe0380cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.073+0000 7fe036800640 1 -- 172.21.15.59:0/4050963185 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03815f430 con 0x7fe0380b3080 2024-04-24T06:40:55.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.074+0000 7fe036800640 1 --2- 172.21.15.59:0/4050963185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe0380cf470 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe03806e080 tx=0x7fe03002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe796ba5b1caf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:55.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.074+0000 7fe035400640 1 -- 172.21.15.59:0/4050963185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030007740 con 0x7fe0380b3080 2024-04-24T06:40:55.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.074+0000 7fe035400640 1 -- 172.21.15.59:0/4050963185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0300078e0 con 0x7fe0380b3080 2024-04-24T06:40:55.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.074+0000 7fe035400640 1 -- 172.21.15.59:0/4050963185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030007cb0 con 0x7fe0380b3080 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 -- 172.21.15.59:0/4050963185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 msgr2=0x7fe0380cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 --2- 172.21.15.59:0/4050963185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe0380cf470 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe03806e080 tx=0x7fe03002a5f0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 -- 172.21.15.59:0/4050963185 shutdown_connections 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 --2- 172.21.15.59:0/4050963185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe0380cf470 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 --2- 172.21.15.59:0/4050963185 >> v2:172.21.15.66:3300/0 conn(0x7fe0380cd050 0x7fe03806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 -- 172.21.15.59:0/4050963185 >> 172.21.15.59:0/4050963185 conn(0x7fe0380b21a0 msgr2=0x7fe0380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 -- 172.21.15.59:0/4050963185 shutdown_connections 2024-04-24T06:40:55.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.075+0000 7fe037200640 1 -- 172.21.15.59:0/4050963185 wait complete. 2024-04-24T06:40:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.077+0000 7fe037200640 1 Processor -- start 2024-04-24T06:40:55.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.077+0000 7fe037200640 1 -- start start 2024-04-24T06:40:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0380cd050 0x7fe0380c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe03815ece0 0x7fe03815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380cef50 con 0x7fe0380b3080 2024-04-24T06:40:55.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe03815f9f0 con 0x7fe0380cd050 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe037200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe038159110 con 0x7fe03815ece0 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe035e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe035e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55986/0 (socket says 172.21.15.59:55986) 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.078+0000 7fe035e00640 1 -- 172.21.15.59:0/1196793536 learned_addr learned my addr 172.21.15.59:0/1196793536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe02fe00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0380cd050 0x7fe0380c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 -- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe03815ece0 msgr2=0x7fe03815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe03815ece0 0x7fe03815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 -- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0380cd050 msgr2=0x7fe0380c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0380cd050 0x7fe0380c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 -- 172.21.15.59:0/1196793536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe038158140 con 0x7fe0380b3080 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe035e00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe02800d800 tx=0x7fe02800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800b6c0 con 0x7fe0380b3080 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe028019070 con 0x7fe0380b3080 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800ba80 con 0x7fe0380b3080 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0380737f0 con 0x7fe0380b3080 2024-04-24T06:40:55.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.079+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0380b07a0 con 0x7fe0380b3080 2024-04-24T06:40:55.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.081+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03804fd90 con 0x7fe0380b3080 2024-04-24T06:40:55.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.082+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe02800bc20 con 0x7fe0380b3080 2024-04-24T06:40:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.083+0000 7fe02e000640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 0x7fe01406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:55.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.083+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fe02808bf60 con 0x7fe0380b3080 2024-04-24T06:40:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.083+0000 7fe02fe00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 0x7fe01406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:55.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.084+0000 7fe02fe00640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 0x7fe01406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe0381f7680 tx=0x7fe020009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:55.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.087+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe038158140 con 0x7fe0380b3080 2024-04-24T06:40:55.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.338+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe03809e140 con 0x7fe0380b3080 2024-04-24T06:40:55.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.339+0000 7fe02e000640 1 -- 172.21.15.59:0/1196793536 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7fe02801c360 con 0x7fe0380b3080 2024-04-24T06:40:55.340 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:40:55.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 msgr2=0x7fe01406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 0x7fe01406edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe0381f7680 tx=0x7fe020009040 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 msgr2=0x7fe038157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe02800d800 tx=0x7fe02800dcc0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 shutdown_connections 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe01406c8f0 0x7fe01406edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe03815ece0 0x7fe03815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0380cd050 0x7fe0380c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 --2- 172.21.15.59:0/1196793536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0380b3080 0x7fe038157c00 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.341+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 >> 172.21.15.59:0/1196793536 conn(0x7fe0380b21a0 msgr2=0x7fe0380ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.342+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 shutdown_connections 2024-04-24T06:40:55.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55.342+0000 7fe037200640 1 -- 172.21.15.59:0/1196793536 wait complete. 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:40:55.363 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:40:55.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:55.364 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:40:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41889sec preferred_lft 41889sec 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:40:55.420 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:40:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:40:55.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:55.422 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:40:55.505 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:40:55.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:55.510 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:40:55.567 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:40:55.567 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:40:55.567 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:40:55.567 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:40:55.567 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:40:55.568 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:40:55.568 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:40:55.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:40:55.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:40:55.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:40:55.569 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:40:55.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:40:55.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:55.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:55.796 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:40:55.846 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:40:55.846 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:40:55.847 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:40:55.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:55.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:40:55.937 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:40:55.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:40:55.937 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:55.993 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:40:55.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:40:55.994 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:56.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:40:56.145 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:40:56.198 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:40:56.199 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:40:56.200 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:40:56.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:56.201 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:40:56.246 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:40:56.246 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:40:56.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:40:56.372 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:40:56.417 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:40:56.418 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:40:56.418 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:40:56.418 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:40:56.419 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:40:56.420 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:40:56.421 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:40:56.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:56.422 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:40:56.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:40:56.481+0000 7f73a9414480 -1 init, newargv = 0x56113afec5f0 newargc=15 2024-04-24T06:40:56.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75046]: starting ceph client 2024-04-24T06:40:56.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75046]: starting fuse 2024-04-24T06:40:56.508 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:40:56.508 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:40:56.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:40:56.508 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:40:56.508 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:40:56.509 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:40:56.510 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:56.710 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.75046.asok 2024-04-24T06:40:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:56.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75046.asok 2024-04-24T06:40:56.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:56.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75046.asok status 2024-04-24T06:40:56.965 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:40:56.965 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "75046", 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9378, 2024-04-24T06:40:56.966 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9378 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4100145761 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:40:56.967 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4100145761 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9378 192.168.0.1:0/4100145761", 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/4100145761", 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 100, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 110, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:40:56.968 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:40:56.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:56.980 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:40:56.981 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:57.033 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:40:57.033 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:57.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:57.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:40:57.123 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:57.123 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:57.123 DEBUG:teuthology.orchestra.run.smithi059:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:40:57.123 DEBUG:teuthology.orchestra.run.smithi059:> parent = os.path.join(mount_point, "parent") 2024-04-24T06:40:57.123 DEBUG:teuthology.orchestra.run.smithi059:> os.mkdir(parent) 2024-04-24T06:40:57.124 DEBUG:teuthology.orchestra.run.smithi059:> flushed = os.path.join(parent, "flushed") 2024-04-24T06:40:57.124 DEBUG:teuthology.orchestra.run.smithi059:> os.mkdir(flushed) 2024-04-24T06:40:57.124 DEBUG:teuthology.orchestra.run.smithi059:> for f in ["alpha", "bravo", "charlie"]: 2024-04-24T06:40:57.124 DEBUG:teuthology.orchestra.run.smithi059:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-24T06:40:57.124 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:57.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:57.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:40:57.339 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-24T06:40:57.476 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/bravo 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-24T06:40:57.478 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:57.626 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:40:57.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-24T06:40:57.640 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:57.833 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:40:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-24T06:40:57.848 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:58.042 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:40:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-24T06:40:58.057 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:58.252 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-24T06:40:58.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-24T06:40:58.267 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:58.464 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-24T06:40:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-24T06:40:58.479 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:40:58.673 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-24T06:40:58.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:40:58.689 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:40:58.689 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:40:58.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.001+0000 7efca8200640 1 Processor -- start 2024-04-24T06:40:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.001+0000 7efca8200640 1 -- start start 2024-04-24T06:40:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca8200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efca015e9d0 0x7efca015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca8200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca015edb0 con 0x7efca00d1670 2024-04-24T06:40:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca8200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca00ba630 con 0x7efca00d42c0 2024-04-24T06:40:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca8200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efca00d3e30 con 0x7efca015e9d0 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efca015e9d0 0x7efca015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca6200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efca00d1670 0x7efca00d2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35562/0 (socket says 172.21.15.59:35562) 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.002+0000 7efca6200640 1 -- 172.21.15.59:0/264282461 learned_addr learned my addr 172.21.15.59:0/264282461 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.003+0000 7efca5800640 1 -- 172.21.15.59:0/264282461 >> v1:172.21.15.66:6789/0 conn(0x7efca00d42c0 legacy=0x7efca00d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.003+0000 7efca5800640 1 -- 172.21.15.59:0/264282461 >> v1:172.21.15.59:6789/0 conn(0x7efca00d1670 legacy=0x7efca00d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.003+0000 7efca5800640 1 -- 172.21.15.59:0/264282461 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca00cf7b0 con 0x7efca015e9d0 2024-04-24T06:40:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.003+0000 7efc9fe00640 1 -- 172.21.15.59:0/264282461 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433721086 0 0) 0x7efca015edb0 con 0x7efca00d1670 2024-04-24T06:40:59.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.004+0000 7efca5800640 1 --2- 172.21.15.59:0/264282461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca015e9d0 0x7efca015f9e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efc940097a0 tx=0x7efc9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47ab9e1789bec98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efc9fe00640 1 -- 172.21.15.59:0/264282461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94037070 con 0x7efca015e9d0 2024-04-24T06:40:59.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efc9fe00640 1 -- 172.21.15.59:0/264282461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc94032440 con 0x7efca015e9d0 2024-04-24T06:40:59.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efc9fe00640 1 -- 172.21.15.59:0/264282461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94032810 con 0x7efca015e9d0 2024-04-24T06:40:59.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 -- 172.21.15.59:0/264282461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca015e9d0 msgr2=0x7efca015f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 --2- 172.21.15.59:0/264282461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca015e9d0 0x7efca015f9e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7efc940097a0 tx=0x7efc9402a1e0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 -- 172.21.15.59:0/264282461 shutdown_connections 2024-04-24T06:40:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 --2- 172.21.15.59:0/264282461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca015e9d0 0x7efca015f9e0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 -- 172.21.15.59:0/264282461 >> 172.21.15.59:0/264282461 conn(0x7efca006cbd0 msgr2=0x7efca006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 -- 172.21.15.59:0/264282461 shutdown_connections 2024-04-24T06:40:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.005+0000 7efca8200640 1 -- 172.21.15.59:0/264282461 wait complete. 2024-04-24T06:40:59.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.007+0000 7efca8200640 1 Processor -- start 2024-04-24T06:40:59.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.007+0000 7efca8200640 1 -- start start 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efca00d1670 0x7efca00b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efca015e9d0 0x7efca0151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efca00cfd70 con 0x7efca00d42c0 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efca015ff20 con 0x7efca00d1670 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.008+0000 7efca8200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7efca00ba0e0 con 0x7efca015e9d0 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49088/0 (socket says 172.21.15.59:49088) 2024-04-24T06:40:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 -- 172.21.15.59:0/2103724739 learned_addr learned my addr 172.21.15.59:0/2103724739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca5800640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efca00d1670 0x7efca00b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca6200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efca015e9d0 0x7efca0151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 -- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efca015e9d0 msgr2=0x7efca0151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efca015e9d0 0x7efca0151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 -- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efca00d1670 msgr2=0x7efca00b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efca00d1670 0x7efca00b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 -- 172.21.15.59:0/2103724739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca00b5820 con 0x7efca00d42c0 2024-04-24T06:40:59.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.009+0000 7efca4e00640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c430 tx=0x7efc9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.010+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90011790 con 0x7efca00d42c0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.010+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc90017070 con 0x7efca00d42c0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.010+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90011bb0 con 0x7efca00d42c0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.010+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca00c8860 con 0x7efca00d42c0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.010+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca00ccdd0 con 0x7efca00d42c0 2024-04-24T06:40:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.011+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca00cc8e0 con 0x7efca00d42c0 2024-04-24T06:40:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.012+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc9001c460 con 0x7efca00d42c0 2024-04-24T06:40:59.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.013+0000 7efc9d600640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 0x7efc8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.013+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7efc9008bb10 con 0x7efca00d42c0 2024-04-24T06:40:59.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.013+0000 7efca5800640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 0x7efc8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.014+0000 7efca5800640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 0x7efc8406edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efc94034000 tx=0x7efc9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.018+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc90059ea0 con 0x7efca00d42c0 2024-04-24T06:40:59.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.221+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efca0069190 con 0x7efca00d42c0 2024-04-24T06:40:59.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.224+0000 7efc9d600640 1 -- 172.21.15.59:0/2103724739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16543 (secure 0 0 0) 0x7efc900597a0 con 0x7efca00d42c0 2024-04-24T06:40:59.226 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:40:59.226 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:49.106564+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 msgr2=0x7efc8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 0x7efc8406edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efc94034000 tx=0x7efc9402e000 comp rx=0 tx=0).stop 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 msgr2=0x7efca00caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c430 tx=0x7efc9000c8f0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 shutdown_connections 2024-04-24T06:40:59.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7efc8406c8f0 0x7efc8406edb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7efca015e9d0 0x7efca0151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca00d42c0 0x7efca00caf40 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 --2- 172.21.15.59:0/2103724739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efca00d1670 0x7efca00b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 >> 172.21.15.59:0/2103724739 conn(0x7efca006cbd0 msgr2=0x7efca00b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 shutdown_connections 2024-04-24T06:40:59.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.227+0000 7efca8200640 1 -- 172.21.15.59:0/2103724739 wait complete. 2024-04-24T06:40:59.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:59.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:40:59.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.465+0000 7fdb7b200640 1 Processor -- start 2024-04-24T06:40:59.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.465+0000 7fdb7b200640 1 -- start start 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fdb7c155280 0x7fdb7c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb7c072d20 0x7fdb7c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7c06efe0 con 0x7fdb7c06e740 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c06f2c0 con 0x7fdb7c072d20 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c06f5a0 con 0x7fdb7c155280 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb7c072d20 0x7fdb7c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb7c072d20 0x7fdb7c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49094/0 (socket says 172.21.15.59:49094) 2024-04-24T06:40:59.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.467+0000 7fdb7a800640 1 -- 172.21.15.59:0/1814754993 learned_addr learned my addr 172.21.15.59:0/1814754993 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:59.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.468+0000 7fdb7a800640 1 -- 172.21.15.59:0/1814754993 >> v2:172.21.15.66:3300/0 conn(0x7fdb7c155280 msgr2=0x7fdb7c15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:59.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.468+0000 7fdb7a800640 1 --2- 172.21.15.59:0/1814754993 >> v2:172.21.15.66:3300/0 conn(0x7fdb7c155280 0x7fdb7c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.468+0000 7fdb7a800640 1 -- 172.21.15.59:0/1814754993 >> v1:172.21.15.66:6789/0 conn(0x7fdb7c06e740 legacy=0x7fdb7c072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:59.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.468+0000 7fdb7a800640 1 -- 172.21.15.59:0/1814754993 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c06f880 con 0x7fdb7c072d20 2024-04-24T06:40:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.469+0000 7fdb7a800640 1 --2- 172.21.15.59:0/1814754993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b3190 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0d12c0 tx=0x7fdb7402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca55dd0c6f4d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.469+0000 7fdb78a00640 1 -- 172.21.15.59:0/1814754993 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74036070 con 0x7fdb7c072d20 2024-04-24T06:40:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.469+0000 7fdb78a00640 1 -- 172.21.15.59:0/1814754993 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb74031600 con 0x7fdb7c072d20 2024-04-24T06:40:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.469+0000 7fdb78a00640 1 -- 172.21.15.59:0/1814754993 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb740319d0 con 0x7fdb7c072d20 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 -- 172.21.15.59:0/1814754993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 msgr2=0x7fdb7c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1814754993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b3190 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0d12c0 tx=0x7fdb7402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 -- 172.21.15.59:0/1814754993 shutdown_connections 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1814754993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b3190 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1814754993 >> v2:172.21.15.66:3300/0 conn(0x7fdb7c155280 0x7fdb7c15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 -- 172.21.15.59:0/1814754993 >> 172.21.15.59:0/1814754993 conn(0x7fdb7c06bdf0 msgr2=0x7fdb7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 -- 172.21.15.59:0/1814754993 shutdown_connections 2024-04-24T06:40:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.470+0000 7fdb7b200640 1 -- 172.21.15.59:0/1814754993 wait complete. 2024-04-24T06:40:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.472+0000 7fdb7b200640 1 Processor -- start 2024-04-24T06:40:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.472+0000 7fdb7b200640 1 -- start start 2024-04-24T06:40:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c06e740 0x7fdb7c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb7c155280 0x7fdb7c0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c06ed00 con 0x7fdb7c072d20 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c06eb80 con 0x7fdb7c06e740 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7c0d0600 con 0x7fdb7c155280 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb79400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb79400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49098/0 (socket says 172.21.15.59:49098) 2024-04-24T06:40:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb79400640 1 -- 172.21.15.59:0/1363130381 learned_addr learned my addr 172.21.15.59:0/1363130381 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb79e00640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c06e740 0x7fdb7c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.473+0000 7fdb7a800640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb7c155280 0x7fdb7c0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 -- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb7c155280 msgr2=0x7fdb7c0b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb7c155280 0x7fdb7c0b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 -- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c06e740 msgr2=0x7fdb7c14dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c06e740 0x7fdb7c14dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 -- 172.21.15.59:0/1363130381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c0ccbc0 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb79400640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c410 tx=0x7fdb6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68017070 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb680035f0 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb680039e0 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.474+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c0ccea0 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.475+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c0cd460 con 0x7fdb7c072d20 2024-04-24T06:40:59.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.476+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7c04fd90 con 0x7fdb7c072d20 2024-04-24T06:40:59.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.477+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb6801a070 con 0x7fdb7c072d20 2024-04-24T06:40:59.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.478+0000 7fdb72000640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 0x7fdb5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.478+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fdb6808afe0 con 0x7fdb7c072d20 2024-04-24T06:40:59.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.479+0000 7fdb79e00640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 0x7fdb5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.479+0000 7fdb79e00640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 0x7fdb5c06edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b810 tx=0x7fdb6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.483+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb68059420 con 0x7fdb7c072d20 2024-04-24T06:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.694+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb7c0cc5c0 con 0x7fdb7c072d20 2024-04-24T06:40:59.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.695+0000 7fdb72000640 1 -- 172.21.15.59:0/1363130381 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fdb68058d20 con 0x7fdb7c072d20 2024-04-24T06:40:59.695 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.697+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 msgr2=0x7fdb5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.697+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 0x7fdb5c06edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b810 tx=0x7fdb6c008040 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.697+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 msgr2=0x7fdb7c0b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c410 tx=0x7fdb6800c8d0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 shutdown_connections 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdb5c06c8f0 0x7fdb5c06edb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdb7c155280 0x7fdb7c0b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb7c072d20 0x7fdb7c0b0c90 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 --2- 172.21.15.59:0/1363130381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c06e740 0x7fdb7c14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 >> 172.21.15.59:0/1363130381 conn(0x7fdb7c06bdf0 msgr2=0x7fdb7c15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 shutdown_connections 2024-04-24T06:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.698+0000 7fdb7b200640 1 -- 172.21.15.59:0/1363130381 wait complete. 2024-04-24T06:40:59.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:40:59.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.932+0000 7fd6dca00640 1 Processor -- start 2024-04-24T06:40:59.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.932+0000 7fd6dca00640 1 -- start start 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.934+0000 7fd6dca00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6d8162c10 0x7fd6d80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.934+0000 7fd6dca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.934+0000 7fd6dca00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d80d0fd0 con 0x7fd6d80cfba0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.934+0000 7fd6dca00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d80b35d0 con 0x7fd6d8162c10 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.934+0000 7fd6dca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d80b38b0 con 0x7fd6d80b4670 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.935+0000 7fd6cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.935+0000 7fd6d5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6d8162c10 0x7fd6d80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.935+0000 7fd6cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:33798/0 (socket says 172.21.15.59:33798) 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.935+0000 7fd6d5e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd6d8162c10 0x7fd6d80d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:49408/0 (socket says 172.21.15.59:49408) 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.935+0000 7fd6cd400640 1 -- 172.21.15.59:0/883867270 learned_addr learned my addr 172.21.15.59:0/883867270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.936+0000 7fd6d5e00640 1 -- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 msgr2=0x7fd6d8071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:40:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.936+0000 7fd6d5e00640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.936+0000 7fd6d5e00640 1 -- 172.21.15.59:0/883867270 >> v1:172.21.15.66:6789/0 conn(0x7fd6d80cfba0 legacy=0x7fd6d8166840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.936+0000 7fd6d5e00640 1 -- 172.21.15.59:0/883867270 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80b3b90 con 0x7fd6d8162c10 2024-04-24T06:40:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.936+0000 7fd6cd400640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.938+0000 7fd6d5e00640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d8162c10 0x7fd6d80d0a90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2dfab5209b8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.938+0000 7fd6d5400640 1 -- 172.21.15.59:0/883867270 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8037070 con 0x7fd6d8162c10 2024-04-24T06:40:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.938+0000 7fd6d5400640 1 -- 172.21.15.59:0/883867270 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8032440 con 0x7fd6d8162c10 2024-04-24T06:40:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.938+0000 7fd6d5400640 1 -- 172.21.15.59:0/883867270 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8032810 con 0x7fd6d8162c10 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 -- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d8162c10 msgr2=0x7fd6d80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d8162c10 0x7fd6d80d0a90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802a180 comp rx=0 tx=0).stop 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 -- 172.21.15.59:0/883867270 shutdown_connections 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 --2- 172.21.15.59:0/883867270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d8162c10 0x7fd6d80d0a90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 -- 172.21.15.59:0/883867270 >> 172.21.15.59:0/883867270 conn(0x7fd6d8067d90 msgr2=0x7fd6d806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 -- 172.21.15.59:0/883867270 shutdown_connections 2024-04-24T06:40:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.939+0000 7fd6dca00640 1 -- 172.21.15.59:0/883867270 wait complete. 2024-04-24T06:40:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.941+0000 7fd6dca00640 1 Processor -- start 2024-04-24T06:40:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.941+0000 7fd6dca00640 1 -- start start 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 0x7fd6d81ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8162ff0 con 0x7fd6d8162c10 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80b4150 con 0x7fd6d80cfba0 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6dca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d8163170 con 0x7fd6d80b4670 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49110/0 (socket says 172.21.15.59:49110) 2024-04-24T06:40:59.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.942+0000 7fd6d6800640 1 -- 172.21.15.59:0/1653436160 learned_addr learned my addr 172.21.15.59:0/1653436160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:40:59.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6cd400640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 0x7fd6d81ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d5e00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 -- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 msgr2=0x7fd6d80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d80cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 -- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 msgr2=0x7fd6d81ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 0x7fd6d81ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 -- 172.21.15.59:0/1653436160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d8162140 con 0x7fd6d8162c10 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6cd400640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 0x7fd6d81ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6d6800640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c730 tx=0x7fd6d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0016020 con 0x7fd6d8162c10 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0006d50 con 0x7fd6d8162c10 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003bb0 con 0x7fd6d8162c10 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.943+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d815eab0 con 0x7fd6d8162c10 2024-04-24T06:40:59.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.944+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d815c9a0 con 0x7fd6d8162c10 2024-04-24T06:40:59.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.945+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d8068a80 con 0x7fd6d8162c10 2024-04-24T06:40:59.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.946+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6d000a7f0 con 0x7fd6d8162c10 2024-04-24T06:40:59.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.947+0000 7fd6cea00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 0x7fd6b406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.947+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fd6d008b6f0 con 0x7fd6d8162c10 2024-04-24T06:40:59.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.947+0000 7fd6d5e00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 0x7fd6b406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.948+0000 7fd6d5e00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 0x7fd6b406ee00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:40:59.952+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6d0020020 con 0x7fd6d8162c10 2024-04-24T06:41:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.154+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6d80d0d50 con 0x7fd6d8162c10 2024-04-24T06:41:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.155+0000 7fd6cea00640 1 -- 172.21.15.59:0/1653436160 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fd6d80d0fd0 con 0x7fd6d8162c10 2024-04-24T06:41:00.156 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 msgr2=0x7fd6b406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 0x7fd6b406ee00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).stop 2024-04-24T06:41:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 msgr2=0x7fd6d81ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c730 tx=0x7fd6d000cbf0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 shutdown_connections 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd6b406c940 0x7fd6b406ee00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d8162c10 0x7fd6d81ef960 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6d80cfba0 0x7fd6d81ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 --2- 172.21.15.59:0/1653436160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd6d80b4670 0x7fd6d80cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 >> 172.21.15.59:0/1653436160 conn(0x7fd6d8067d90 msgr2=0x7fd6d815d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 shutdown_connections 2024-04-24T06:41:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.158+0000 7fd6dca00640 1 -- 172.21.15.59:0/1653436160 wait complete. 2024-04-24T06:41:00.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:00.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.394+0000 7f4ce8a00640 1 Processor -- start 2024-04-24T06:41:00.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.395+0000 7f4ce8a00640 1 -- start start 2024-04-24T06:41:00.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.396+0000 7f4ce8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 0x7f4ce40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.396+0000 7f4ce8a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ce40d13e0 0x7f4ce40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.396+0000 7f4ce8a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce40b5da0 con 0x7f4ce40b3600 2024-04-24T06:41:00.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.396+0000 7f4ce8a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce415f100 con 0x7f4ce40d13e0 2024-04-24T06:41:00.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.396+0000 7f4ce8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40d07d0 con 0x7f4ce415ed20 2024-04-24T06:41:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.397+0000 7f4ce2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 0x7f4ce40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.397+0000 7f4ce3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ce40d13e0 0x7f4ce40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.397+0000 7f4ce2800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 0x7f4ce40d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:33814/0 (socket says 172.21.15.59:33814) 2024-04-24T06:41:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.397+0000 7f4ce2800640 1 -- 172.21.15.59:0/51742305 learned_addr learned my addr 172.21.15.59:0/51742305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.398+0000 7f4ce3200640 1 -- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 msgr2=0x7f4ce40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.398+0000 7f4ce3200640 1 --2- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 0x7f4ce40d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.398+0000 7f4ce3200640 1 -- 172.21.15.59:0/51742305 >> v1:172.21.15.66:6789/0 conn(0x7f4ce40b3600 legacy=0x7f4ce40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.398+0000 7f4ce3200640 1 -- 172.21.15.59:0/51742305 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce40d0a20 con 0x7f4ce40d13e0 2024-04-24T06:41:00.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.399+0000 7f4ce3200640 1 --2- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce40d13e0 0x7f4ce40d0290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009d90 tx=0x7f4cdc02a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e75c079af9282e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.400+0000 7f4ce1400640 1 -- 172.21.15.59:0/51742305 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007740 con 0x7f4ce40d13e0 2024-04-24T06:41:00.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.400+0000 7f4ce1400640 1 -- 172.21.15.59:0/51742305 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0078e0 con 0x7f4ce40d13e0 2024-04-24T06:41:00.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.400+0000 7f4ce1400640 1 -- 172.21.15.59:0/51742305 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007cb0 con 0x7f4ce40d13e0 2024-04-24T06:41:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 -- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce40d13e0 msgr2=0x7f4ce40d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce40d13e0 0x7f4ce40d0290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009d90 tx=0x7f4cdc02a540 comp rx=0 tx=0).stop 2024-04-24T06:41:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 -- 172.21.15.59:0/51742305 shutdown_connections 2024-04-24T06:41:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce40d13e0 0x7f4ce40d0290 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/51742305 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce415ed20 0x7f4ce40d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.401+0000 7f4ce8a00640 1 -- 172.21.15.59:0/51742305 >> 172.21.15.59:0/51742305 conn(0x7f4ce408bf90 msgr2=0x7f4ce40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.402+0000 7f4ce8a00640 1 -- 172.21.15.59:0/51742305 shutdown_connections 2024-04-24T06:41:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.402+0000 7f4ce8a00640 1 -- 172.21.15.59:0/51742305 wait complete. 2024-04-24T06:41:00.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.404+0000 7f4ce8a00640 1 Processor -- start 2024-04-24T06:41:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.404+0000 7f4ce8a00640 1 -- start start 2024-04-24T06:41:00.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce40d13e0 0x7f4ce415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 0x7f4ce40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40d3490 con 0x7f4ce40b3600 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40d31b0 con 0x7f4ce415ed20 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.405+0000 7f4ce8a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40b4620 con 0x7f4ce40d13e0 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49116/0 (socket says 172.21.15.59:49116) 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 -- 172.21.15.59:0/881180669 learned_addr learned my addr 172.21.15.59:0/881180669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce1e00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce40d13e0 0x7f4ce415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce3200640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 0x7f4ce40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 -- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce40d13e0 msgr2=0x7f4ce415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce40d13e0 0x7f4ce415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 -- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 msgr2=0x7f4ce40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 0x7f4ce40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce2800640 1 -- 172.21.15.59:0/881180669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce414e4b0 con 0x7f4ce40b3600 2024-04-24T06:41:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.406+0000 7f4ce3200640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 0x7f4ce40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4ce2800640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e870 tx=0x7f4cd400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4017070 con 0x7f4ce40b3600 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd400c8e0 con 0x7f4ce40b3600 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400ccd0 con 0x7f4ce40b3600 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce414e750 con 0x7f4ce40b3600 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.407+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce414ed10 con 0x7f4ce40b3600 2024-04-24T06:41:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.408+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce404fd90 con 0x7f4ce40b3600 2024-04-24T06:41:00.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.410+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cd401a070 con 0x7f4ce40b3600 2024-04-24T06:41:00.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.411+0000 7f4cdaa00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.411+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4cd408bdf0 con 0x7f4ce40b3600 2024-04-24T06:41:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.411+0000 7f4ce1e00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.412+0000 7f4ce1e00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40b1d80 tx=0x7f4cd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.418+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cd405a2d0 con 0x7f4ce40b3600 2024-04-24T06:41:00.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.621+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ce406cc90 con 0x7f4ce40b3600 2024-04-24T06:41:00.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.622+0000 7f4cdaa00640 1 -- 172.21.15.59:0/881180669 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f4cd4059bd0 con 0x7f4ce40b3600 2024-04-24T06:41:00.623 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 msgr2=0x7f4cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40b1d80 tx=0x7f4cd0009040 comp rx=0 tx=0).stop 2024-04-24T06:41:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 msgr2=0x7f4ce41f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e870 tx=0x7f4cd400ed30 comp rx=0 tx=0).stop 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 shutdown_connections 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ce415ed20 0x7f4ce40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ce40d13e0 0x7f4ce415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 --2- 172.21.15.59:0/881180669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce40b3600 0x7f4ce41f40f0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.624+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 >> 172.21.15.59:0/881180669 conn(0x7f4ce408bf90 msgr2=0x7f4ce40b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.625+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 shutdown_connections 2024-04-24T06:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.625+0000 7f4ce8a00640 1 -- 172.21.15.59:0/881180669 wait complete. 2024-04-24T06:41:00.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:00.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.856+0000 7fa3d6a00640 1 Processor -- start 2024-04-24T06:41:00.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.857+0000 7fa3d6a00640 1 -- start start 2024-04-24T06:41:00.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.858+0000 7fa3d6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa3d015c670 0x7fa3d00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.858+0000 7fa3d6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3d00d0cd0 0x7fa3d00b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.858+0000 7fa3d6a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d00b4cf0 con 0x7fa3d00b4230 2024-04-24T06:41:00.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.858+0000 7fa3d6a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d015ca50 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.858+0000 7fa3d6a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d00ccb80 con 0x7fa3d015c670 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3d00d0cd0 0x7fa3d00b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3d00d0cd0 0x7fa3d00b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49124/0 (socket says 172.21.15.59:49124) 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 -- 172.21.15.59:0/2726972064 learned_addr learned my addr 172.21.15.59:0/2726972064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d5600640 1 --2- 172.21.15.59:0/2726972064 >> v2:172.21.15.66:3300/0 conn(0x7fa3d015c670 0x7fa3d00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 -- 172.21.15.59:0/2726972064 >> v2:172.21.15.66:3300/0 conn(0x7fa3d015c670 msgr2=0x7fa3d00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:00.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 --2- 172.21.15.59:0/2726972064 >> v2:172.21.15.66:3300/0 conn(0x7fa3d015c670 0x7fa3d00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 -- 172.21.15.59:0/2726972064 >> v1:172.21.15.66:6789/0 conn(0x7fa3d00b4230 legacy=0x7fa3d00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.859+0000 7fa3d6000640 1 -- 172.21.15.59:0/2726972064 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00ccdd0 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.860+0000 7fa3d6000640 1 --2- 172.21.15.59:0/2726972064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00d0cd0 0x7fa3d00b4690 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00d1270 tx=0x7fa3c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a357bb5c1c3ef155 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.861+0000 7fa3cfe00640 1 -- 172.21.15.59:0/2726972064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8036070 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.861+0000 7fa3cfe00640 1 -- 172.21.15.59:0/2726972064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c8031600 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.861+0000 7fa3cfe00640 1 -- 172.21.15.59:0/2726972064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80319d0 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.861+0000 7fa3d6a00640 1 -- 172.21.15.59:0/2726972064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00d0cd0 msgr2=0x7fa3d00b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.861+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/2726972064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00d0cd0 0x7fa3d00b4690 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00d1270 tx=0x7fa3c802a6f0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 -- 172.21.15.59:0/2726972064 shutdown_connections 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/2726972064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00d0cd0 0x7fa3d00b4690 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/2726972064 >> v2:172.21.15.66:3300/0 conn(0x7fa3d015c670 0x7fa3d00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 -- 172.21.15.59:0/2726972064 >> 172.21.15.59:0/2726972064 conn(0x7fa3d006bdf0 msgr2=0x7fa3d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 -- 172.21.15.59:0/2726972064 shutdown_connections 2024-04-24T06:41:00.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.862+0000 7fa3d6a00640 1 -- 172.21.15.59:0/2726972064 wait complete. 2024-04-24T06:41:00.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.864+0000 7fa3d6a00640 1 Processor -- start 2024-04-24T06:41:00.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.864+0000 7fa3d6a00640 1 -- start start 2024-04-24T06:41:00.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 0x7fa3d0073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d015c670 0x7fa3d015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00b3e60 con 0x7fa3d00b4230 2024-04-24T06:41:00.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00cd390 con 0x7fa3d015c670 2024-04-24T06:41:00.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d00b3fe0 con 0x7fa3d00d0cd0 2024-04-24T06:41:00.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49136/0 (socket says 172.21.15.59:49136) 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d5600640 1 -- 172.21.15.59:0/380107847 learned_addr learned my addr 172.21.15.59:0/380107847 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d4c00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 0x7fa3d0073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.865+0000 7fa3d6000640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d015c670 0x7fa3d015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 -- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 msgr2=0x7fa3d0073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 0x7fa3d0073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 -- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d015c670 msgr2=0x7fa3d015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d015c670 0x7fa3d015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 -- 172.21.15.59:0/380107847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00ca850 con 0x7fa3d00b4230 2024-04-24T06:41:00.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d4c00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 0x7fa3d0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d5600640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400d620 tx=0x7fa3c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400b620 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c4019070 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400ba00 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.866+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d00c97b0 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.867+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d00c9d70 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.868+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d004fd90 con 0x7fa3d00b4230 2024-04-24T06:41:00.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.869+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3c400bba0 con 0x7fa3d00b4230 2024-04-24T06:41:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.870+0000 7fa3cd600640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 0x7fa3b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.870+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c408bfe0 con 0x7fa3d00b4230 2024-04-24T06:41:00.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.871+0000 7fa3d4c00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 0x7fa3b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:00.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.871+0000 7fa3d4c00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 0x7fa3b006edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa3d015f630 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:00.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:00.875+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3c405a3b0 con 0x7fa3d00b4230 2024-04-24T06:41:01.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.098+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa3d015d470 con 0x7fa3d00b4230 2024-04-24T06:41:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.099+0000 7fa3cd600640 1 -- 172.21.15.59:0/380107847 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7fa3c4059cb0 con 0x7fa3d00b4230 2024-04-24T06:41:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 msgr2=0x7fa3b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 0x7fa3b006edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa3d015f630 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 msgr2=0x7fa3d014e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400d620 tx=0x7fa3c400dae0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 shutdown_connections 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa3b006c8f0 0x7fa3b006edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d015c670 0x7fa3d015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa3d00d0cd0 0x7fa3d0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 --2- 172.21.15.59:0/380107847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00b4230 0x7fa3d014e6b0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 >> 172.21.15.59:0/380107847 conn(0x7fa3d006bdf0 msgr2=0x7fa3d01557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 shutdown_connections 2024-04-24T06:41:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.102+0000 7fa3d6a00640 1 -- 172.21.15.59:0/380107847 wait complete. 2024-04-24T06:41:01.123 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:01.123 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:01.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.391+0000 7f3d73800640 1 Processor -- start 2024-04-24T06:41:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.391+0000 7f3d73800640 1 -- start start 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.392+0000 7f3d73800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c1644b0 0x7f3d6c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.392+0000 7f3d73800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b4550 0x7f3d6c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d73800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c160d40 con 0x7f3d6c160960 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d73800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c164270 con 0x7f3d6c1644b0 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d73800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c166de0 con 0x7f3d6c0b4550 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c1644b0 0x7f3d6c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c1644b0 0x7f3d6c163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49146/0 (socket says 172.21.15.59:49146) 2024-04-24T06:41:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 -- 172.21.15.59:0/1002284476 learned_addr learned my addr 172.21.15.59:0/1002284476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d71800640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b4550 0x7f3d6c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 -- 172.21.15.59:0/1002284476 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b4550 msgr2=0x7f3d6c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b4550 0x7f3d6c06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 -- 172.21.15.59:0/1002284476 >> v1:172.21.15.66:6789/0 conn(0x7f3d6c160960 legacy=0x7f3d6c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.393+0000 7f3d63e00640 1 -- 172.21.15.59:0/1002284476 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1670c0 con 0x7f3d6c1644b0 2024-04-24T06:41:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.395+0000 7f3d63e00640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c163d30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009870 tx=0x7f3d5402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76d68e8e2534de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.395+0000 7f3d6be00640 1 -- 172.21.15.59:0/1002284476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54037070 con 0x7f3d6c1644b0 2024-04-24T06:41:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.395+0000 7f3d6be00640 1 -- 172.21.15.59:0/1002284476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d54032440 con 0x7f3d6c1644b0 2024-04-24T06:41:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.395+0000 7f3d6be00640 1 -- 172.21.15.59:0/1002284476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54032810 con 0x7f3d6c1644b0 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 -- 172.21.15.59:0/1002284476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 msgr2=0x7f3d6c163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c163d30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009870 tx=0x7f3d5402a210 comp rx=0 tx=0).stop 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 -- 172.21.15.59:0/1002284476 shutdown_connections 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c0b4550 0x7f3d6c06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 --2- 172.21.15.59:0/1002284476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c163d30 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 -- 172.21.15.59:0/1002284476 >> 172.21.15.59:0/1002284476 conn(0x7f3d6c067ea0 msgr2=0x7f3d6c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 -- 172.21.15.59:0/1002284476 shutdown_connections 2024-04-24T06:41:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.396+0000 7f3d73800640 1 -- 172.21.15.59:0/1002284476 wait complete. 2024-04-24T06:41:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.398+0000 7f3d73800640 1 Processor -- start 2024-04-24T06:41:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.398+0000 7f3d73800640 1 -- start start 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.399+0000 7f3d73800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4550 0x7f3d6c161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.399+0000 7f3d73800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c160960 0x7f3d6c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.399+0000 7f3d73800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.399+0000 7f3d73800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1668e0 con 0x7f3d6c1644b0 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.399+0000 7f3d73800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c167680 con 0x7f3d6c0b4550 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d73800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c166a60 con 0x7f3d6c160960 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49152/0 (socket says 172.21.15.59:49152) 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 -- 172.21.15.59:0/794619378 learned_addr learned my addr 172.21.15.59:0/794619378 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d70e00640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4550 0x7f3d6c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d63e00640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c160960 0x7f3d6c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 -- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c160960 msgr2=0x7f3d6c150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c160960 0x7f3d6c150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 -- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4550 msgr2=0x7f3d6c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4550 0x7f3d6c161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.400+0000 7f3d71800640 1 -- 172.21.15.59:0/794619378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c14e190 con 0x7f3d6c1644b0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d71800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3d64007e30 tx=0x7f3d6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64014620 con 0x7f3d6c1644b0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d64019070 con 0x7f3d6c1644b0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c14e470 con 0x7f3d6c1644b0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64014a00 con 0x7f3d6c1644b0 2024-04-24T06:41:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.401+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c161b50 con 0x7f3d6c1644b0 2024-04-24T06:41:01.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.403+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c068a80 con 0x7f3d6c1644b0 2024-04-24T06:41:01.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.404+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d64014ba0 con 0x7f3d6c1644b0 2024-04-24T06:41:01.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.405+0000 7f3d69600640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 0x7f3d4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.405+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f3d6408c4b0 con 0x7f3d6c1644b0 2024-04-24T06:41:01.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.406+0000 7f3d70e00640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 0x7f3d4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.406+0000 7f3d70e00640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 0x7f3d4806ede0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c004ea0 tx=0x7f3d5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.410+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d6405a840 con 0x7f3d6c1644b0 2024-04-24T06:41:01.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.619+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d6c06e260 con 0x7f3d6c1644b0 2024-04-24T06:41:01.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.621+0000 7f3d69600640 1 -- 172.21.15.59:0/794619378 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16543 (secure 0 0 0) 0x7f3d6405a140 con 0x7f3d6c1644b0 2024-04-24T06:41:01.623 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:01.623 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:49.106564+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 msgr2=0x7f3d4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 0x7f3d4806ede0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c004ea0 tx=0x7f3d5c009040 comp rx=0 tx=0).stop 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 msgr2=0x7f3d6c150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3d64007e30 tx=0x7f3d6400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 shutdown_connections 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3d4806c920 0x7f3d4806ede0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c1644b0 0x7f3d6c150b90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3d6c160960 0x7f3d6c150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 --2- 172.21.15.59:0/794619378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d6c0b4550 0x7f3d6c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 >> 172.21.15.59:0/794619378 conn(0x7f3d6c067ea0 msgr2=0x7f3d6c0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:01.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 shutdown_connections 2024-04-24T06:41:01.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.624+0000 7f3d73800640 1 -- 172.21.15.59:0/794619378 wait complete. 2024-04-24T06:41:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:01.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.851+0000 7fbfa9a00640 1 Processor -- start 2024-04-24T06:41:01.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.852+0000 7fbfa9a00640 1 -- start start 2024-04-24T06:41:01.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfa40d0600 0x7fbfa40d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa9a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa4163320 con 0x7fbfa415d300 2024-04-24T06:41:01.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa9a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa40720d0 con 0x7fbfa4071c70 2024-04-24T06:41:01.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa40d0f90 con 0x7fbfa40d0600 2024-04-24T06:41:01.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbfa415d300 0x7fbfa4167180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35564/0 (socket says 172.21.15.59:35564) 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.853+0000 7fbfa3e00640 1 -- 172.21.15.59:0/3679275133 learned_addr learned my addr 172.21.15.59:0/3679275133 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa9000640 1 --2- 172.21.15.59:0/3679275133 >> v2:172.21.15.59:3300/0 conn(0x7fbfa40d0600 0x7fbfa40d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa2a00640 1 -- 172.21.15.59:0/3679275133 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908553961 0 0) 0x7fbfa4163320 con 0x7fbfa415d300 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa2a00640 1 -- 172.21.15.59:0/3679275133 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf88003520 con 0x7fbfa415d300 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa9000640 1 -- 172.21.15.59:0/3679275133 >> v1:172.21.15.66:6789/0 conn(0x7fbfa4071c70 legacy=0x7fbfa4163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa9000640 1 -- 172.21.15.59:0/3679275133 >> v1:172.21.15.59:6789/0 conn(0x7fbfa415d300 legacy=0x7fbfa4167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.854+0000 7fbfa9000640 1 -- 172.21.15.59:0/3679275133 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa4072380 con 0x7fbfa40d0600 2024-04-24T06:41:01.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.855+0000 7fbfa9000640 1 --2- 172.21.15.59:0/3679275133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa40d0600 0x7fbfa40d0a50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d90 tx=0x7fbf9c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8022227481adfdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa2a00640 1 -- 172.21.15.59:0/3679275133 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c007740 con 0x7fbfa40d0600 2024-04-24T06:41:01.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa2a00640 1 -- 172.21.15.59:0/3679275133 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf9c0078e0 con 0x7fbfa40d0600 2024-04-24T06:41:01.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa2a00640 1 -- 172.21.15.59:0/3679275133 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c007cb0 con 0x7fbfa40d0600 2024-04-24T06:41:01.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa9a00640 1 -- 172.21.15.59:0/3679275133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa40d0600 msgr2=0x7fbfa40d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/3679275133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa40d0600 0x7fbfa40d0a50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d90 tx=0x7fbf9c02a6f0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa9a00640 1 -- 172.21.15.59:0/3679275133 shutdown_connections 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/3679275133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa40d0600 0x7fbfa40d0a50 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.856+0000 7fbfa9a00640 1 -- 172.21.15.59:0/3679275133 >> 172.21.15.59:0/3679275133 conn(0x7fbfa406bdf0 msgr2=0x7fbfa406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.857+0000 7fbfa9a00640 1 -- 172.21.15.59:0/3679275133 shutdown_connections 2024-04-24T06:41:01.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.857+0000 7fbfa9a00640 1 -- 172.21.15.59:0/3679275133 wait complete. 2024-04-24T06:41:01.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.858+0000 7fbfa9a00640 1 Processor -- start 2024-04-24T06:41:01.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.858+0000 7fbfa9a00640 1 -- start start 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 0x7fbfa4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 0x7fbfa4154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4072940 con 0x7fbfa4071c70 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4163650 con 0x7fbfa415d300 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.859+0000 7fbfa9a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa406e1e0 con 0x7fbfa40d0600 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49184/0 (socket says 172.21.15.59:49184) 2024-04-24T06:41:01.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 -- 172.21.15.59:0/2153883555 learned_addr learned my addr 172.21.15.59:0/2153883555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa9000640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 0x7fbfa4154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3400640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 0x7fbfa4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 -- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 msgr2=0x7fbfa4159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 0x7fbfa4159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 -- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 msgr2=0x7fbfa4154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 0x7fbfa4154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 -- 172.21.15.59:0/2153883555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa4154ba0 con 0x7fbfa4071c70 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3400640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 0x7fbfa4159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa9000640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 0x7fbfa4154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.860+0000 7fbfa3e00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c440 tx=0x7fbf9400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.861+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94017070 con 0x7fbfa4071c70 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.861+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf94005ce0 con 0x7fbfa4071c70 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.861+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf940072b0 con 0x7fbfa4071c70 2024-04-24T06:41:01.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.861+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa4154e80 con 0x7fbfa4071c70 2024-04-24T06:41:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.861+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa4155440 con 0x7fbfa4071c70 2024-04-24T06:41:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.862+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa404fd90 con 0x7fbfa4071c70 2024-04-24T06:41:01.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.863+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf9401a070 con 0x7fbfa4071c70 2024-04-24T06:41:01.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.864+0000 7fbf9be00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 0x7fbf8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:01.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.864+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fbf9408af40 con 0x7fbfa4071c70 2024-04-24T06:41:01.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.864+0000 7fbfa3400640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 0x7fbf8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:01.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.865+0000 7fbfa3400640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 0x7fbf8406edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4162a10 tx=0x7fbf90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:01.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:01.869+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf94059390 con 0x7fbfa4071c70 2024-04-24T06:41:02.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.071+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfa40cff50 con 0x7fbfa4071c70 2024-04-24T06:41:02.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.072+0000 7fbf9be00640 1 -- 172.21.15.59:0/2153883555 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fbf94058c90 con 0x7fbfa4071c70 2024-04-24T06:41:02.074 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 msgr2=0x7fbf8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 0x7fbf8406edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4162a10 tx=0x7fbf90009040 comp rx=0 tx=0).stop 2024-04-24T06:41:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 msgr2=0x7fbfa4155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400c440 tx=0x7fbf9400c900 comp rx=0 tx=0).stop 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 shutdown_connections 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf8406c8f0 0x7fbf8406edb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfa415d300 0x7fbfa4154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbfa40d0600 0x7fbfa4159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 --2- 172.21.15.59:0/2153883555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfa4071c70 0x7fbfa4155960 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 >> 172.21.15.59:0/2153883555 conn(0x7fbfa406bdf0 msgr2=0x7fbfa41ebf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 shutdown_connections 2024-04-24T06:41:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.075+0000 7fbfa9a00640 1 -- 172.21.15.59:0/2153883555 wait complete. 2024-04-24T06:41:02.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.312+0000 7fc741000640 1 Processor -- start 2024-04-24T06:41:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.312+0000 7fc741000640 1 -- start start 2024-04-24T06:41:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.313+0000 7fc741000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.313+0000 7fc741000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73c15cab0 con 0x7fc73c15d810 2024-04-24T06:41:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.313+0000 7fc741000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73c160e40 con 0x7fc73c0d0d10 2024-04-24T06:41:02.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.313+0000 7fc741000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc73c15cd30 con 0x7fc73c1567e0 2024-04-24T06:41:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.314+0000 7fc73aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc73c15d810 0x7fc73c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35576/0 (socket says 172.21.15.59:35576) 2024-04-24T06:41:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.314+0000 7fc73aa00640 1 -- 172.21.15.59:0/3605893928 learned_addr learned my addr 172.21.15.59:0/3605893928 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.314+0000 7fc73b400640 1 --2- 172.21.15.59:0/3605893928 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.315+0000 7fc73a000640 1 -- 172.21.15.59:0/3605893928 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505880899 0 0) 0x7fc728001110 con 0x7fc73c15d810 2024-04-24T06:41:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.315+0000 7fc73a000640 1 -- 172.21.15.59:0/3605893928 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc720003520 con 0x7fc73c15d810 2024-04-24T06:41:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.315+0000 7fc73b400640 1 -- 172.21.15.59:0/3605893928 >> v1:172.21.15.66:6789/0 conn(0x7fc73c0d0d10 legacy=0x7fc73c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.315+0000 7fc73b400640 1 -- 172.21.15.59:0/3605893928 >> v1:172.21.15.59:6789/0 conn(0x7fc73c15d810 legacy=0x7fc73c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.315+0000 7fc73b400640 1 -- 172.21.15.59:0/3605893928 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c15d010 con 0x7fc73c1567e0 2024-04-24T06:41:02.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.316+0000 7fc73b400640 1 --2- 172.21.15.59:0/3605893928 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c156bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0097d0 tx=0x7fc72c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a081d36cd4d0a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.317+0000 7fc73a000640 1 -- 172.21.15.59:0/3605893928 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c037070 con 0x7fc73c1567e0 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.317+0000 7fc73a000640 1 -- 172.21.15.59:0/3605893928 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c032440 con 0x7fc73c1567e0 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.317+0000 7fc73a000640 1 -- 172.21.15.59:0/3605893928 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c032810 con 0x7fc73c1567e0 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 -- 172.21.15.59:0/3605893928 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 msgr2=0x7fc73c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 --2- 172.21.15.59:0/3605893928 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c156bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0097d0 tx=0x7fc72c02a210 comp rx=0 tx=0).stop 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 -- 172.21.15.59:0/3605893928 shutdown_connections 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 --2- 172.21.15.59:0/3605893928 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c156bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 -- 172.21.15.59:0/3605893928 >> 172.21.15.59:0/3605893928 conn(0x7fc73c06bdf0 msgr2=0x7fc73c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 -- 172.21.15.59:0/3605893928 shutdown_connections 2024-04-24T06:41:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.318+0000 7fc741000640 1 -- 172.21.15.59:0/3605893928 wait complete. 2024-04-24T06:41:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.320+0000 7fc741000640 1 Processor -- start 2024-04-24T06:41:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.320+0000 7fc741000640 1 -- start start 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c15d810 0x7fc73c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1609a0 con 0x7fc73c0d0d10 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c160820 con 0x7fc73c15d810 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc741000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc73c160b20 con 0x7fc73c1567e0 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc73b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc73b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49190/0 (socket says 172.21.15.59:49190) 2024-04-24T06:41:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.321+0000 7fc73b400640 1 -- 172.21.15.59:0/1624234881 learned_addr learned my addr 172.21.15.59:0/1624234881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:02.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 -- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 msgr2=0x7fc73c14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 -- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c15d810 msgr2=0x7fc73c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c15d810 0x7fc73c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 -- 172.21.15.59:0/1624234881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c14ca70 con 0x7fc73c0d0d10 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc73b400640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0098f0 tx=0x7fc72c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c037050 con 0x7fc73c0d0d10 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c043070 con 0x7fc73c0d0d10 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c03cb30 con 0x7fc73c0d0d10 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.322+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c14cd10 con 0x7fc73c0d0d10 2024-04-24T06:41:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.323+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c14d2d0 con 0x7fc73c0d0d10 2024-04-24T06:41:02.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.324+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73c04fd90 con 0x7fc73c0d0d10 2024-04-24T06:41:02.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.325+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc72c03ccd0 con 0x7fc73c0d0d10 2024-04-24T06:41:02.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.326+0000 7fc733400640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.326+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fc72c0ad380 con 0x7fc73c0d0d10 2024-04-24T06:41:02.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.326+0000 7fc73aa00640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.327+0000 7fc73aa00640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc728000f60 tx=0x7fc728001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.330+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc72c03d3b0 con 0x7fc73c0d0d10 2024-04-24T06:41:02.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.536+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc73c148b70 con 0x7fc73c0d0d10 2024-04-24T06:41:02.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.538+0000 7fc733400640 1 -- 172.21.15.59:0/1624234881 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fc72c07b790 con 0x7fc73c0d0d10 2024-04-24T06:41:02.540 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:02.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 msgr2=0x7fc71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc728000f60 tx=0x7fc728001110 comp rx=0 tx=0).stop 2024-04-24T06:41:02.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 msgr2=0x7fc73c06f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0098f0 tx=0x7fc72c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 shutdown_connections 2024-04-24T06:41:02.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc73c15d810 0x7fc73c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc73c1567e0 0x7fc73c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 --2- 172.21.15.59:0/1624234881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc73c0d0d10 0x7fc73c06f880 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 >> 172.21.15.59:0/1624234881 conn(0x7fc73c06bdf0 msgr2=0x7fc73c15bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 shutdown_connections 2024-04-24T06:41:02.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.541+0000 7fc741000640 1 -- 172.21.15.59:0/1624234881 wait complete. 2024-04-24T06:41:02.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.773+0000 7fde47e00640 1 Processor -- start 2024-04-24T06:41:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.774+0000 7fde47e00640 1 -- start start 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde47e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fde40072480 0x7fde4006f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde47e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde400cfc20 con 0x7fde40145e10 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde47e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde400cff00 con 0x7fde400d0cc0 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde47e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fde400d01e0 con 0x7fde40072480 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde45400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fde40145e10 0x7fde400cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35590/0 (socket says 172.21.15.59:35590) 2024-04-24T06:41:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.775+0000 7fde45400640 1 -- 172.21.15.59:0/3652417349 learned_addr learned my addr 172.21.15.59:0/3652417349 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde45e00640 1 --2- 172.21.15.59:0/3652417349 >> v2:172.21.15.66:3300/0 conn(0x7fde40072480 0x7fde4006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572040541 0 0) 0x7fde400cfc20 con 0x7fde40145e10 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde28003520 con 0x7fde40145e10 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 8170471 0 0) 0x7fde28003520 con 0x7fde40145e10 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 >> v2:172.21.15.66:3300/0 conn(0x7fde40072480 msgr2=0x7fde4006f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 --2- 172.21.15.59:0/3652417349 >> v2:172.21.15.66:3300/0 conn(0x7fde40072480 0x7fde4006f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 >> v1:172.21.15.66:6789/0 conn(0x7fde400d0cc0 legacy=0x7fde40071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde400d04c0 con 0x7fde40145e10 2024-04-24T06:41:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fde34001670 con 0x7fde40145e10 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.776+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 >> v1:172.21.15.59:6789/0 conn(0x7fde40145e10 legacy=0x7fde400cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde280039b0 0x7fde28003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde400d01e0 con 0x7fde280051c0 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde2800c0f0 con 0x7fde28004800 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fde2800c2f0 con 0x7fde280039b0 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45400640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde280039b0 0x7fde28003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde44a00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 -- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde280039b0 msgr2=0x7fde28003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde280039b0 0x7fde28003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 -- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28004800 msgr2=0x7fde28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde45e00640 1 -- 172.21.15.59:0/3652417349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28003520 con 0x7fde280051c0 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.777+0000 7fde44a00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde28004800 0x7fde28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.778+0000 7fde45e00640 1 -- 172.21.15.59:0/3652417349 reap_dead start 2024-04-24T06:41:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.778+0000 7fde45e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fde4006f910 tx=0x7fde28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaed5f50e08919e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38033070 con 0x7fde280051c0 2024-04-24T06:41:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde38003b90 con 0x7fde280051c0 2024-04-24T06:41:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde3fe00640 1 -- 172.21.15.59:0/3652417349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38009be0 con 0x7fde280051c0 2024-04-24T06:41:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 -- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 msgr2=0x7fde2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fde4006f910 tx=0x7fde28005830 comp rx=0 tx=0).stop 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 -- 172.21.15.59:0/3652417349 shutdown_connections 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 --2- 172.21.15.59:0/3652417349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde2800b9c0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 -- 172.21.15.59:0/3652417349 >> 172.21.15.59:0/3652417349 conn(0x7fde40067d90 msgr2=0x7fde4006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 -- 172.21.15.59:0/3652417349 shutdown_connections 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.779+0000 7fde47e00640 1 -- 172.21.15.59:0/3652417349 wait complete. 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.781+0000 7fde47e00640 1 Processor -- start 2024-04-24T06:41:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.781+0000 7fde47e00640 1 -- start start 2024-04-24T06:41:02.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 0x7fde400b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4014f150 0x7fde4014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde401461f0 con 0x7fde280051c0 2024-04-24T06:41:02.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde4006f130 con 0x7fde4014f150 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.782+0000 7fde47e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fde4013e760 con 0x7fde400ae1d0 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49204/0 (socket says 172.21.15.59:49204) 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 -- 172.21.15.59:0/3205413270 learned_addr learned my addr 172.21.15.59:0/3205413270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde44a00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 0x7fde400b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4014f150 0x7fde4014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 -- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 msgr2=0x7fde400b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 0x7fde400b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 -- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4014f150 msgr2=0x7fde4014f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4014f150 0x7fde4014f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 -- 172.21.15.59:0/3205413270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4014d260 con 0x7fde280051c0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde44a00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 0x7fde400b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.783+0000 7fde45400640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d7a0 tx=0x7fde3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.784+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34019070 con 0x7fde280051c0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.784+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde340074e0 con 0x7fde280051c0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.784+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34004a20 con 0x7fde280051c0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.784+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4014d540 con 0x7fde280051c0 2024-04-24T06:41:02.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.784+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4014db00 con 0x7fde280051c0 2024-04-24T06:41:02.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.785+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde40068a80 con 0x7fde280051c0 2024-04-24T06:41:02.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.786+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde3401c070 con 0x7fde280051c0 2024-04-24T06:41:02.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.788+0000 7fde3d600640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 0x7fde2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.788+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fde3408b710 con 0x7fde280051c0 2024-04-24T06:41:02.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.788+0000 7fde44a00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 0x7fde2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.789+0000 7fde44a00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 0x7fde2006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde400ae5b0 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.793+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde34059a10 con 0x7fde280051c0 2024-04-24T06:41:03.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.998+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde400cda50 con 0x7fde280051c0 2024-04-24T06:41:03.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:02.999+0000 7fde3d600640 1 -- 172.21.15.59:0/3205413270 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7fde34059310 con 0x7fde280051c0 2024-04-24T06:41:03.001 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 msgr2=0x7fde2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 0x7fde2006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde400ae5b0 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 msgr2=0x7fde400b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d7a0 tx=0x7fde3400dc60 comp rx=0 tx=0).stop 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 shutdown_connections 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fde2006c8f0 0x7fde2006edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde4014f150 0x7fde4014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fde400ae1d0 0x7fde400b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 --2- 172.21.15.59:0/3205413270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde280051c0 0x7fde400b0bc0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 >> 172.21.15.59:0/3205413270 conn(0x7fde40067d90 msgr2=0x7fde4006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 shutdown_connections 2024-04-24T06:41:03.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.002+0000 7fde47e00640 1 -- 172.21.15.59:0/3205413270 wait complete. 2024-04-24T06:41:03.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:03.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.239+0000 7faf93e00640 1 Processor -- start 2024-04-24T06:41:03.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.239+0000 7faf93e00640 1 -- start start 2024-04-24T06:41:03.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.240+0000 7faf93e00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7faf940d0c10 0x7faf940cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.240+0000 7faf93e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf940af010 0x7faf940af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.240+0000 7faf93e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf941676a0 con 0x7faf940d0830 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.240+0000 7faf93e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faf94166b20 con 0x7faf940af010 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.240+0000 7faf93e00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7faf94166e00 con 0x7faf940d0c10 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf940af010 0x7faf940af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf940af010 0x7faf940af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49216/0 (socket says 172.21.15.59:49216) 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 -- 172.21.15.59:0/719342081 learned_addr learned my addr 172.21.15.59:0/719342081 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf8a000640 1 --2- 172.21.15.59:0/719342081 >> v2:172.21.15.66:3300/0 conn(0x7faf940d0c10 0x7faf940cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 -- 172.21.15.59:0/719342081 >> v2:172.21.15.66:3300/0 conn(0x7faf940d0c10 msgr2=0x7faf940cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 --2- 172.21.15.59:0/719342081 >> v2:172.21.15.66:3300/0 conn(0x7faf940d0c10 0x7faf940cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 -- 172.21.15.59:0/719342081 >> v1:172.21.15.66:6789/0 conn(0x7faf940d0830 legacy=0x7faf94163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.241+0000 7faf93400640 1 -- 172.21.15.59:0/719342081 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf941670e0 con 0x7faf940af010 2024-04-24T06:41:03.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.242+0000 7faf93400640 1 --2- 172.21.15.59:0/719342081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940af010 0x7faf940af450 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faf940d1270 tx=0x7faf8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61f7ab180e5b7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.242+0000 7faf92000640 1 -- 172.21.15.59:0/719342081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c037070 con 0x7faf940af010 2024-04-24T06:41:03.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.242+0000 7faf92000640 1 -- 172.21.15.59:0/719342081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8c032440 con 0x7faf940af010 2024-04-24T06:41:03.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.242+0000 7faf92000640 1 -- 172.21.15.59:0/719342081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c032810 con 0x7faf940af010 2024-04-24T06:41:03.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 -- 172.21.15.59:0/719342081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940af010 msgr2=0x7faf940af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 --2- 172.21.15.59:0/719342081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940af010 0x7faf940af450 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faf940d1270 tx=0x7faf8c02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 -- 172.21.15.59:0/719342081 shutdown_connections 2024-04-24T06:41:03.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 --2- 172.21.15.59:0/719342081 >> v2:172.21.15.66:3300/0 conn(0x7faf940d0c10 0x7faf940cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 --2- 172.21.15.59:0/719342081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940af010 0x7faf940af450 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 -- 172.21.15.59:0/719342081 >> 172.21.15.59:0/719342081 conn(0x7faf9406bdf0 msgr2=0x7faf9406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 -- 172.21.15.59:0/719342081 shutdown_connections 2024-04-24T06:41:03.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.243+0000 7faf93e00640 1 -- 172.21.15.59:0/719342081 wait complete. 2024-04-24T06:41:03.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.245+0000 7faf93e00640 1 Processor -- start 2024-04-24T06:41:03.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.245+0000 7faf93e00640 1 -- start start 2024-04-24T06:41:03.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf940af010 0x7faf9409dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 0x7faf94162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf940b45f0 con 0x7faf940d0830 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faf940af990 con 0x7faf940d0c10 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf93e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faf94166740 con 0x7faf940af010 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.246+0000 7faf8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49228/0 (socket says 172.21.15.59:49228) 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 -- 172.21.15.59:0/2848416873 learned_addr learned my addr 172.21.15.59:0/2848416873 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf93400640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 0x7faf94162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 -- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf940af010 msgr2=0x7faf9409dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf940af010 0x7faf9409dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 -- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 msgr2=0x7faf94162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 0x7faf94162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 -- 172.21.15.59:0/2848416873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9409a7e0 con 0x7faf940d0830 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf93400640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 0x7faf94162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8a000640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ccf0 tx=0x7faf7c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.247+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c003990 con 0x7faf940d0830 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.248+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c003b30 con 0x7faf940d0830 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.248+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c004030 con 0x7faf940d0830 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.248+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9409aac0 con 0x7faf940d0830 2024-04-24T06:41:03.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.248+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9409b080 con 0x7faf940d0830 2024-04-24T06:41:03.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.249+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9404fd90 con 0x7faf940d0830 2024-04-24T06:41:03.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.250+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf7c0041d0 con 0x7faf940d0830 2024-04-24T06:41:03.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.251+0000 7faf8b400640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 0x7faf7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.251+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7faf7c08c0a0 con 0x7faf940d0830 2024-04-24T06:41:03.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.251+0000 7faf92a00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 0x7faf7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.252+0000 7faf92a00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 0x7faf7006ede0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faf84009650 tx=0x7faf84006ba0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.256+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf7c014070 con 0x7faf940d0830 2024-04-24T06:41:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.475+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf94099de0 con 0x7faf940d0830 2024-04-24T06:41:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.476+0000 7faf8b400640 1 -- 172.21.15.59:0/2848416873 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7faf7c05a430 con 0x7faf940d0830 2024-04-24T06:41:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:41:03.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.478+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 msgr2=0x7faf7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 0x7faf7006ede0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faf84009650 tx=0x7faf84006ba0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 msgr2=0x7faf9409a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ccf0 tx=0x7faf7c00a5c0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 shutdown_connections 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf7006c920 0x7faf7006ede0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf940d0c10 0x7faf94162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf940d0830 0x7faf9409a070 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 --2- 172.21.15.59:0/2848416873 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf940af010 0x7faf9409dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 >> 172.21.15.59:0/2848416873 conn(0x7faf9406bdf0 msgr2=0x7faf940b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 shutdown_connections 2024-04-24T06:41:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.479+0000 7faf93e00640 1 -- 172.21.15.59:0/2848416873 wait complete. 2024-04-24T06:41:03.499 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:41:03.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:03.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:03.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75046]: fuse finished with error 0 and tester_r 0 2024-04-24T06:41:03.587 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:41:03.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:03.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:03.669 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:41:03.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-24T06:41:03.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.945+0000 7f4eee800640 1 Processor -- start 2024-04-24T06:41:03.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.945+0000 7f4eee800640 1 -- start start 2024-04-24T06:41:03.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eee800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f4ee815ed20 0x7f4ee80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eee800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ee80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eee800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee80b5da0 con 0x7f4ee80b3600 2024-04-24T06:41:03.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eee800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee815f100 con 0x7f4ee815ed20 2024-04-24T06:41:03.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eee800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee80d07d0 con 0x7f4ee80d13e0 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eeca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ee80b3600 0x7f4ee80b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35604/0 (socket says 172.21.15.59:35604) 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eeca00640 1 -- 172.21.15.59:0/2088533917 learned_addr learned my addr 172.21.15.59:0/2088533917 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eede00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ee80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.947+0000 7f4eed400640 1 --2- 172.21.15.59:0/2088533917 >> v2:172.21.15.66:3300/0 conn(0x7f4ee815ed20 0x7f4ee80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734394625 0 0) 0x7f4ee80b5da0 con 0x7f4ee80b3600 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ed0003520 con 0x7f4ee80b3600 2024-04-24T06:41:03.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2208239086 0 0) 0x7f4ed0003520 con 0x7f4ee80b3600 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 msgr2=0x7f4ee80d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ee80d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 >> v2:172.21.15.66:3300/0 conn(0x7f4ee815ed20 msgr2=0x7f4ee80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 --2- 172.21.15.59:0/2088533917 >> v2:172.21.15.66:3300/0 conn(0x7f4ee815ed20 0x7f4ee80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee80d0a20 con 0x7f4ee80b3600 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4ed4001670 con 0x7f4ee80b3600 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 >> v1:172.21.15.59:6789/0 conn(0x7f4ee80b3600 legacy=0x7f4ee80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed00039b0 0x7f4ed0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ed00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80d07d0 con 0x7f4ed0004800 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed000ba70 con 0x7f4ed00039b0 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.948+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed000bc70 con 0x7f4ee80d13e0 2024-04-24T06:41:03.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eed400640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed00039b0 0x7f4ed0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eede00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ed00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 -- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 msgr2=0x7f4ed00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ed00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 -- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed00039b0 msgr2=0x7f4ed0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed00039b0 0x7f4ed0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eeca00640 1 -- 172.21.15.59:0/2088533917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0003520 con 0x7f4ed0004800 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.949+0000 7f4eede00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee80d13e0 0x7f4ed00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.950+0000 7f4eeca00640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001eb0 tx=0x7f4ed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec5e33ae87caa56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.950+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4007ca0 con 0x7f4ed0004800 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.950+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed4005ad0 con 0x7f4ed0004800 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.950+0000 7f4ee7e00640 1 -- 172.21.15.59:0/2088533917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4007070 con 0x7f4ed0004800 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 -- 172.21.15.59:0/2088533917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed0004800 msgr2=0x7f4ed0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 --2- 172.21.15.59:0/2088533917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001eb0 tx=0x7f4ed0005830 comp rx=0 tx=0).stop 2024-04-24T06:41:03.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eede00640 1 -- 172.21.15.59:0/2088533917 reap_dead start 2024-04-24T06:41:03.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 -- 172.21.15.59:0/2088533917 shutdown_connections 2024-04-24T06:41:03.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 -- 172.21.15.59:0/2088533917 >> 172.21.15.59:0/2088533917 conn(0x7f4ee808bf90 msgr2=0x7f4ee80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 -- 172.21.15.59:0/2088533917 shutdown_connections 2024-04-24T06:41:03.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.951+0000 7f4eee800640 1 -- 172.21.15.59:0/2088533917 wait complete. 2024-04-24T06:41:03.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.953+0000 7f4eee800640 1 Processor -- start 2024-04-24T06:41:03.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.953+0000 7f4eee800640 1 -- start start 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 0x7f4ee80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ee80cdea0 0x7f4ee80cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80d3490 con 0x7f4ee80cbda0 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80d31b0 con 0x7f4ee80cdea0 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eee800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee80d0290 con 0x7f4ee81557b0 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.954+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49240/0 (socket says 172.21.15.59:49240) 2024-04-24T06:41:03.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 -- 172.21.15.59:0/732865550 learned_addr learned my addr 172.21.15.59:0/732865550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eed400640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 0x7f4ee80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eeca00640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ee80cdea0 0x7f4ee80cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 -- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 msgr2=0x7f4ee80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 0x7f4ee80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 -- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ee80cdea0 msgr2=0x7f4ee80cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ee80cdea0 0x7f4ee80cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 -- 172.21.15.59:0/732865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee8155b90 con 0x7f4ee80cbda0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eed400640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 0x7f4ee80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.955+0000 7f4eede00640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80d1c80 tx=0x7f4ee0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.956+0000 7f4ee5600640 1 -- 172.21.15.59:0/732865550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000f040 con 0x7f4ee80cbda0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.956+0000 7f4ee5600640 1 -- 172.21.15.59:0/732865550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee000e8f0 con 0x7f4ee80cbda0 2024-04-24T06:41:03.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.956+0000 7f4ee5600640 1 -- 172.21.15.59:0/732865550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000ece0 con 0x7f4ee80cbda0 2024-04-24T06:41:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.956+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee80c89a0 con 0x7f4ee80cbda0 2024-04-24T06:41:03.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.956+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee80c8f60 con 0x7f4ee80cbda0 2024-04-24T06:41:03.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.958+0000 7f4ee5600640 1 -- 172.21.15.59:0/732865550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ee001c400 con 0x7f4ee80cbda0 2024-04-24T06:41:03.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.959+0000 7f4ee5600640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 0x7f4ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.959+0000 7f4ee5600640 1 -- 172.21.15.59:0/732865550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4ee008be80 con 0x7f4ee80cbda0 2024-04-24T06:41:03.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.959+0000 7f4eed400640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 0x7f4ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.960+0000 7f4ef2d88740 1 Processor -- start 2024-04-24T06:41:03.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.960+0000 7f4eed400640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 0x7f4ec806edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0074e0 tx=0x7f4edc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.960+0000 7f4ef2d88740 1 -- start start 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555aa351b1f0 con 0x555aa35e17e0 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555aa34c1350 con 0x555aa35fee00 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555aa35171b0 con 0x555aa35e1f20 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49254/0 (socket says 172.21.15.59:49254) 2024-04-24T06:41:03.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.961+0000 7f4eede00640 1 -- 172.21.15.59:0/1994629268 learned_addr learned my addr 172.21.15.59:0/1994629268 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eed400640 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 -- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 msgr2=0x555aa35e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 -- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 msgr2=0x555aa35b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 -- 172.21.15.59:0/1994629268 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa3515f50 con 0x555aa35e17e0 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eed400640 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4eede00640 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0001eb0 tx=0x7f4ee0006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71414dd527b63afc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4ed9c00640 1 -- 172.21.15.59:0/1994629268 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0062780 con 0x555aa35e17e0 2024-04-24T06:41:03.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4ed9c00640 1 -- 172.21.15.59:0/1994629268 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee0057f50 con 0x555aa35e17e0 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.962+0000 7f4ed9c00640 1 -- 172.21.15.59:0/1994629268 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00562c0 con 0x555aa35e17e0 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1994629268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 msgr2=0x555aa35b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0001eb0 tx=0x7f4ee0006d50 comp rx=0 tx=0).stop 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1994629268 shutdown_connections 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e17e0 0x555aa35b65f0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1994629268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1994629268 >> 172.21.15.59:0/1994629268 conn(0x555aa35b3a00 msgr2=0x555aa35fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1994629268 shutdown_connections 2024-04-24T06:41:03.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.963+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1994629268 wait complete. 2024-04-24T06:41:03.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.964+0000 7f4ef2d88740 1 -- start start 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.964+0000 7f4ef2d88740 1 Processor -- start 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555aa35b5bb0 con 0x555aa35e1f20 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555aa34d4690 con 0x555aa35fee00 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555aa35b4fa0 con 0x555aa35cd930 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.965+0000 7f4ef2d88740 10 client.0 _ll_register_callbacks cb 0x555aa3510650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49270/0 (socket says 172.21.15.59:49270) 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 -- 172.21.15.59:0/4045674718 learned_addr learned my addr 172.21.15.59:0/4045674718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eeca00640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 -- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 msgr2=0x555aa35cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eede00640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 -- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 msgr2=0x555aa35cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 -- 172.21.15.59:0/4045674718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa34d3920 con 0x555aa35e1f20 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eeca00640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4eed400640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4edc016ee0 tx=0x7f4edc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26c603ee89787f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.966+0000 7f4ece000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc017e10 con 0x555aa35e1f20 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4edc0155c0 con 0x555aa35e1f20 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee0057f50 con 0x555aa35e1f20 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f4edc0157c0 con 0x555aa35e1f20 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 10 client.9450 fetch_fsmap learned FSMap version 100 2024-04-24T06:41:03.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x555aa32f77d0 con 0x555aa35e1f20 2024-04-24T06:41:03.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee0057f50 con 0x555aa35e1f20 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f4edc033020 con 0x555aa35e1f20 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 10 client.9450 fetch_fsmap finished waiting for FSMap version 100 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 10 client.9450 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.967+0000 7f4ef2d88740 20 client.9450 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.968+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35fd8f0 0x555aa3619d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.968+0000 7f4ef2d88740 4 client.9450 mds_command: new command op to 9228 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.968+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 --> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa3528c10 con 0x555aa35fd8f0 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.968+0000 7f4eeca00640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35fd8f0 0x555aa3619d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.969+0000 7f4eeca00640 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35fd8f0 0x555aa3619d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2bc5aba0ce332f server_cookie=5996e4d79cba10ef in_seq=0 out_seq=0 2024-04-24T06:41:03.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.969+0000 7f4ece000640 10 client.9450 ms_handle_connect on v2:172.21.15.66:6833/2441853137 2024-04-24T06:41:03.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.971+0000 7f4ece000640 1 -- 172.21.15.59:0/4045674718 <== mds.0 v2:172.21.15.66:6833/2441853137 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555aa3528c10 con 0x555aa35fd8f0 2024-04-24T06:41:03.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.971+0000 7f4ece000640 10 client.9450 handle_command_reply: tid=0 2024-04-24T06:41:03.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.971+0000 7f4ef2d88740 1 client.9450 shutdown 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 msgr2=0x555aa35d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4edc016ee0 tx=0x7f4edc004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 shutdown_connections 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35fd8f0 0x555aa3619d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35fee00 0x555aa35cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4045674718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35e1f20 0x555aa35d23a0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 >> 172.21.15.59:0/4045674718 conn(0x555aa35e17e0 msgr2=0x555aa35ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ece000640 0 client.9450 ms_handle_reset on v2:172.21.15.66:6833/2441853137 2024-04-24T06:41:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 shutdown_connections 2024-04-24T06:41:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4045674718 wait complete. 2024-04-24T06:41:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.972+0000 7f4ef2d88740 20 client.9450 trim_cache size 0 max 16384 2024-04-24T06:41:03.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.989+0000 7f4ef2d88740 1 Processor -- start 2024-04-24T06:41:03.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.989+0000 7f4ef2d88740 1 -- start start 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35e1f20 0x555aa35b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee82017b0 con 0x555aa35fee00 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8200840 con 0x555aa35e1f20 2024-04-24T06:41:03.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee80ca920 con 0x555aa35cd930 2024-04-24T06:41:03.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49286/0 (socket says 172.21.15.59:49286) 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 -- 172.21.15.59:0/1505396863 learned_addr learned my addr 172.21.15.59:0/1505396863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eeca00640 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35e1f20 0x555aa35b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 -- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 msgr2=0x555aa35e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eed400640 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35e06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 -- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35e1f20 msgr2=0x555aa35b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35e1f20 0x555aa35b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.990+0000 7f4eede00640 1 -- 172.21.15.59:0/1505396863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa35cd760 con 0x555aa35fee00 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.991+0000 7f4eede00640 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0059950 tx=0x7f4ee005d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5527ad046b02aebf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.991+0000 7f4ed9200640 1 -- 172.21.15.59:0/1505396863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0057f50 con 0x555aa35fee00 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.991+0000 7f4ed9200640 1 -- 172.21.15.59:0/1505396863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee0072040 con 0x555aa35fee00 2024-04-24T06:41:03.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.991+0000 7f4ed9200640 1 -- 172.21.15.59:0/1505396863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00622f0 con 0x555aa35fee00 2024-04-24T06:41:03.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1505396863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 msgr2=0x555aa35f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:03.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0059950 tx=0x7f4ee005d000 comp rx=0 tx=0).stop 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1505396863 shutdown_connections 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35f82a0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35e1f20 0x555aa35b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/1505396863 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35cd930 0x555aa35e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1505396863 >> 172.21.15.59:0/1505396863 conn(0x555aa35e17e0 msgr2=0x555aa35b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1505396863 shutdown_connections 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- 172.21.15.59:0/1505396863 wait complete. 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.992+0000 7f4ef2d88740 1 -- start start 2024-04-24T06:41:03.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.993+0000 7f4ef2d88740 1 Processor -- start 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35cd930 0x555aa35b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555aa35ad3a0 con 0x555aa35fee00 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee82001f0 con 0x555aa35cd930 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x555aa35fc0a0 con 0x555aa35e1f20 2024-04-24T06:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ef2d88740 10 client.0 _ll_register_callbacks cb 0x7f4edc009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49290/0 (socket says 172.21.15.59:49290) 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 -- 172.21.15.59:0/4076051519 learned_addr learned my addr 172.21.15.59:0/4076051519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 -- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 msgr2=0x555aa35dda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 -- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35cd930 msgr2=0x555aa35b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35cd930 0x555aa35b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 -- 172.21.15.59:0/4076051519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa322d3f0 con 0x555aa35fee00 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4eede00640 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0059950 tx=0x7f4ee005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2126ecea80b1b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.994+0000 7f4ece000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0057f50 con 0x555aa35fee00 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4ee007ab70 con 0x555aa35fee00 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee0072040 con 0x555aa35fee00 2024-04-24T06:41:03.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f4ee0081050 con 0x555aa35fee00 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 10 client.9456 fetch_fsmap learned FSMap version 100 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x555aa31ed0f0 con 0x555aa35fee00 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee0072040 con 0x555aa35fee00 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f4ee00590a0 con 0x555aa35fee00 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 10 client.9456 fetch_fsmap finished waiting for FSMap version 100 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 10 client.9456 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 20 client.9456 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35ace60 0x555aa35b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 4 client.9456 mds_command: new command op to 9228 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T06:41:03.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.995+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 --> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555aa3522a80 con 0x555aa35ace60 2024-04-24T06:41:04.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.996+0000 7f4eed400640 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35ace60 0x555aa35b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.997+0000 7f4eed400640 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35ace60 0x555aa35b6060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3d7a89cb7ac8ee server_cookie=2e322f2288a0fa28 in_seq=0 out_seq=0 2024-04-24T06:41:04.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:03.997+0000 7f4ece000640 10 client.9456 ms_handle_connect on v2:172.21.15.66:6833/2441853137 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ece000640 1 -- 172.21.15.59:0/4076051519 <== mds.0 v2:172.21.15.66:6833/2441853137 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555aa3522a80 con 0x555aa35ace60 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ece000640 10 client.9456 handle_command_reply: tid=0 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 client.9456 shutdown 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 msgr2=0x555aa35b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0059950 tx=0x7f4ee005b000 comp rx=0 tx=0).stop 2024-04-24T06:41:04.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 shutdown_connections 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:6833/2441853137,v1:172.21.15.66:6835/2441853137] conn(0x555aa35ace60 0x555aa35b6060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555aa35fee00 0x555aa35b5b20 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x555aa35e1f20 0x555aa35dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 --2- 172.21.15.59:0/4076051519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555aa35cd930 0x555aa35b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ece000640 0 client.9456 ms_handle_reset on v2:172.21.15.66:6833/2441853137 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.015+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 >> 172.21.15.59:0/4076051519 conn(0x555aa35e17e0 msgr2=0x555aa35ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 shutdown_connections 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4ef2d88740 1 -- 172.21.15.59:0/4076051519 wait complete. 2024-04-24T06:41:04.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4ef2d88740 20 client.9456 trim_cache size 0 max 16384 2024-04-24T06:41:04.019 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "message": "", 2024-04-24T06:41:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0 2024-04-24T06:41:04.019 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 msgr2=0x7f4ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 0x7f4ec806edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0074e0 tx=0x7f4edc00b040 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 msgr2=0x7f4ee80b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.016+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80d1c80 tx=0x7f4ee0010c20 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 shutdown_connections 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4ec806c8f0 0x7f4ec806edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee80cbda0 0x7f4ee80b1750 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ee80cdea0 0x7f4ee80cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 --2- 172.21.15.59:0/732865550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4ee81557b0 0x7f4ee80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 >> 172.21.15.59:0/732865550 conn(0x7f4ee808bf90 msgr2=0x7f4ee80b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 shutdown_connections 2024-04-24T06:41:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.017+0000 7f4eee800640 1 -- 172.21.15.59:0/732865550 wait complete. 2024-04-24T06:41:04.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:41:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.232+0000 7f83a1600640 1 Processor -- start 2024-04-24T06:41:04.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.233+0000 7f83a1600640 1 -- start start 2024-04-24T06:41:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.234+0000 7f83a1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.234+0000 7f83a1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839c0b4550 0x7f839c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.234+0000 7f83a1600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c160d40 con 0x7f839c160960 2024-04-24T06:41:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.234+0000 7f83a1600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f839c164270 con 0x7f839c0b4550 2024-04-24T06:41:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.234+0000 7f83a1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f839c166de0 con 0x7f839c1644b0 2024-04-24T06:41:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.235+0000 7f839b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839c0b4550 0x7f839c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.235+0000 7f839b400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f839c0b4550 0x7f839c06e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:49590/0 (socket says 172.21.15.59:49590) 2024-04-24T06:41:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.235+0000 7f839b400640 1 -- 172.21.15.59:0/1173413276 learned_addr learned my addr 172.21.15.59:0/1173413276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.235+0000 7f839a000640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.236+0000 7f839b400640 1 -- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 msgr2=0x7f839c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.236+0000 7f839b400640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.236+0000 7f839b400640 1 -- 172.21.15.59:0/1173413276 >> v1:172.21.15.66:6789/0 conn(0x7f839c160960 legacy=0x7f839c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.236+0000 7f839b400640 1 -- 172.21.15.59:0/1173413276 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1670c0 con 0x7f839c0b4550 2024-04-24T06:41:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.238+0000 7f839b400640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c0b4550 0x7f839c06e8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8394009870 tx=0x7f839402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2256fb1e725456 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.238+0000 7f8399600640 1 -- 172.21.15.59:0/1173413276 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394037070 con 0x7f839c0b4550 2024-04-24T06:41:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f8399600640 1 -- 172.21.15.59:0/1173413276 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8394032440 con 0x7f839c0b4550 2024-04-24T06:41:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f8399600640 1 -- 172.21.15.59:0/1173413276 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394032810 con 0x7f839c0b4550 2024-04-24T06:41:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 -- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c0b4550 msgr2=0x7f839c06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c0b4550 0x7f839c06e8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8394009870 tx=0x7f839402a270 comp rx=0 tx=0).stop 2024-04-24T06:41:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 -- 172.21.15.59:0/1173413276 shutdown_connections 2024-04-24T06:41:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c0b4550 0x7f839c06e8d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 --2- 172.21.15.59:0/1173413276 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 -- 172.21.15.59:0/1173413276 >> 172.21.15.59:0/1173413276 conn(0x7f839c067ea0 msgr2=0x7f839c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:04.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.239+0000 7f83a1600640 1 -- 172.21.15.59:0/1173413276 shutdown_connections 2024-04-24T06:41:04.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.240+0000 7f83a1600640 1 -- 172.21.15.59:0/1173413276 wait complete. 2024-04-24T06:41:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.241+0000 7f83a1600640 1 Processor -- start 2024-04-24T06:41:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.241+0000 7f83a1600640 1 -- start start 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c160960 0x7f839c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1668e0 con 0x7f839c0b4550 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f839c167680 con 0x7f839c160960 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.242+0000 7f83a1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f839c166a60 con 0x7f839c1644b0 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.243+0000 7f839aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.243+0000 7f839aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49294/0 (socket says 172.21.15.59:49294) 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.243+0000 7f839aa00640 1 -- 172.21.15.59:0/1555431283 learned_addr learned my addr 172.21.15.59:0/1555431283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.243+0000 7f839a000640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c160960 0x7f839c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.243+0000 7f839b400640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 -- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 msgr2=0x7f839c0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 -- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c160960 msgr2=0x7f839c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c160960 0x7f839c0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 -- 172.21.15.59:0/1555431283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0a99a0 con 0x7f839c0b4550 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.244+0000 7f839aa00640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f838400c400 tx=0x7f838400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.245+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83840052d0 con 0x7f839c0b4550 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.245+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384005490 con 0x7f839c0b4550 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.245+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c0a9c40 con 0x7f839c0b4550 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.245+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384007260 con 0x7f839c0b4550 2024-04-24T06:41:04.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.245+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c0aa200 con 0x7f839c0b4550 2024-04-24T06:41:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.246+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c068a80 con 0x7f839c0b4550 2024-04-24T06:41:04.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.248+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8384007400 con 0x7f839c0b4550 2024-04-24T06:41:04.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.249+0000 7f8392a00640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.249+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f838408b0d0 con 0x7f839c0b4550 2024-04-24T06:41:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.249+0000 7f839a000640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.250+0000 7f839a000640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 0x7f837806edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f839c15dda0 tx=0x7f838c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.253+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8384058e10 con 0x7f839c0b4550 2024-04-24T06:41:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.501+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f839c06e260 con 0x7f839c0b4550 2024-04-24T06:41:04.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.502+0000 7f8392a00640 1 -- 172.21.15.59:0/1555431283 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f838405d110 con 0x7f839c0b4550 2024-04-24T06:41:04.504 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.504+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 msgr2=0x7f837806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.504+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 0x7f837806edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f839c15dda0 tx=0x7f838c00b040 comp rx=0 tx=0).stop 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 msgr2=0x7f839c0aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f838400c400 tx=0x7f838400c8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 shutdown_connections 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f839c1644b0 0x7f839c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f839c160960 0x7f839c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 --2- 172.21.15.59:0/1555431283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0b4550 0x7f839c0aace0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 >> 172.21.15.59:0/1555431283 conn(0x7f839c067ea0 msgr2=0x7f839c0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 shutdown_connections 2024-04-24T06:41:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04.505+0000 7f83a1600640 1 -- 172.21.15.59:0/1555431283 wait complete. 2024-04-24T06:41:04.527 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:41:04.527 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:41:04.527 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:41:04.527 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:41:04.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:04.528 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:41:04.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:41:04.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:41:04.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:04.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:04.528 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:41:04.580 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:41:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:41:04.580 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:41:04.581 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41879sec preferred_lft 41879sec 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:04.582 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.583 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:04.584 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:04.584 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:41:04.584 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:04.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:04.584 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:41:04.663 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:41:04.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:04.664 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:41:04.719 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:41:04.719 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:41:04.720 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:41:04.720 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:41:04.720 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:41:04.720 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:41:04.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:41:04.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:04.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:04.943 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:41:04.991 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:41:04.991 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:41:04.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:04.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:04.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:41:05.077 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:41:05.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:05.077 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:05.134 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:05.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:05.135 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:05.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:05.265 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:05.323 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:05.324 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:05.326 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:05.326 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:05.326 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:05.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:05.327 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:05.383 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:41:05.383 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:41:05.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:05.469 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:05.521 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:05.522 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:05.523 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:05.524 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:05.524 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:05.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:05.525 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:05.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:41:05.579+0000 7fda2818d480 -1 init, newargv = 0x557a369fb6d0 newargc=15 2024-04-24T06:41:05.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76159]: starting ceph client 2024-04-24T06:41:05.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76159]: starting fuse 2024-04-24T06:41:06.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:06.767 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:06.828 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:06.829 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:06.830 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:06.831 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:06.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:06.832 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:06.889 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:41:06.890 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:41:06.890 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:41:06.891 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:07.071 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.76159.asok 2024-04-24T06:41:07.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:07.091 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76159.asok 2024-04-24T06:41:07.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:07.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76159.asok status 2024-04-24T06:41:07.339 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "76159", 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:41:07.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9462, 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9462 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:07.341 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3072622959 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3072622959 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9462 192.168.0.1:0/3072622959", 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3072622959", 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 100, 2024-04-24T06:41:07.342 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 110, 2024-04-24T06:41:07.343 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:41:07.343 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:41:07.343 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:41:07.343 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:07.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:07.356 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:41:07.356 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:07.414 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:41:07.414 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:07.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:07.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:07.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:07.504 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-24T06:41:07.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:07.580 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-24T06:41:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:8+0 records in 2024-04-24T06:41:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:8+0 records out 2024-04-24T06:41:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.169728 s, 49.4 MB/s 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-24T06:41:07.814 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:07.953 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-24T06:41:07.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:07.968 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:07.969 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:07.969 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:07.969 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:07.969 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-24T06:41:07.969 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:08.164 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-24T06:41:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:08.179 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:08.179 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:08.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:08.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.487+0000 7f5a8fe00640 1 Processor -- start 2024-04-24T06:41:08.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.487+0000 7f5a8fe00640 1 -- start start 2024-04-24T06:41:08.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8fe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5a900b5ff0 0x7f5a900b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a900d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a900cf030 con 0x7f5a9015e7c0 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8fe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5a900b7000 con 0x7f5a900b5ff0 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a900cf620 con 0x7f5a900d0d90 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5a9015e7c0 0x7f5a900d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35618/0 (socket says 172.21.15.59:35618) 2024-04-24T06:41:08.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8ea00640 1 -- 172.21.15.59:0/2327258894 learned_addr learned my addr 172.21.15.59:0/2327258894 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a87e00640 1 --2- 172.21.15.59:0/2327258894 >> v2:172.21.15.66:3300/0 conn(0x7f5a900b5ff0 0x7f5a900b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a900d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537806542 0 0) 0x7f5a80001110 con 0x7f5a9015e7c0 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.489+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a74003520 con 0x7f5a9015e7c0 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2723338295 0 0) 0x7f5a74003520 con 0x7f5a9015e7c0 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 msgr2=0x7f5a900d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a900d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 >> v2:172.21.15.66:3300/0 conn(0x7f5a900b5ff0 msgr2=0x7f5a900b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 --2- 172.21.15.59:0/2327258894 >> v2:172.21.15.66:3300/0 conn(0x7f5a900b5ff0 0x7f5a900b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a900b6a40 con 0x7f5a9015e7c0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5a80001840 con 0x7f5a9015e7c0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 >> v1:172.21.15.59:6789/0 conn(0x7f5a9015e7c0 legacy=0x7f5a900d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74003fe0 0x7f5a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900b7000 con 0x7f5a74004d40 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900cf620 con 0x7f5a74003fe0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7400bac0 con 0x7f5a900d0d90 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.490+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a87e00640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74003fe0 0x7f5a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8ea00640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8f400640 1 -- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 msgr2=0x7f5a740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900d0d90 0x7f5a740039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8f400640 1 -- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74003fe0 msgr2=0x7f5a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74003fe0 0x7f5a74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a8f400640 1 -- 172.21.15.59:0/2327258894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a74003520 con 0x7f5a74004d40 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.491+0000 7f5a87e00640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a74003fe0 0x7f5a74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.492+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a740051c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5a88007360 tx=0x7f5a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a402acf0c7b231 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.492+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88009be0 con 0x7f5a74004d40 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.492+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a88036600 con 0x7f5a74004d40 2024-04-24T06:41:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.492+0000 7f5a8e000640 1 -- 172.21.15.59:0/2327258894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880369d0 con 0x7f5a74004d40 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2327258894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 msgr2=0x7f5a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2327258894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a740051c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5a88007360 tx=0x7f5a74005830 comp rx=0 tx=0).stop 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8f400640 1 -- 172.21.15.59:0/2327258894 reap_dead start 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2327258894 shutdown_connections 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2327258894 >> 172.21.15.59:0/2327258894 conn(0x7f5a9009d920 msgr2=0x7f5a9008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2327258894 shutdown_connections 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.493+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2327258894 wait complete. 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.495+0000 7f5a8fe00640 1 Processor -- start 2024-04-24T06:41:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.495+0000 7f5a8fe00640 1 -- start start 2024-04-24T06:41:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900cc8d0 0x7f5a900b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a90152810 0x7f5a901483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900d2800 con 0x7f5a74004d40 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900cf2b0 con 0x7f5a90152810 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5a900d1a80 con 0x7f5a900cc8d0 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49304/0 (socket says 172.21.15.59:49304) 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.496+0000 7f5a8ea00640 1 -- 172.21.15.59:0/2502485704 learned_addr learned my addr 172.21.15.59:0/2502485704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 -- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900cc8d0 msgr2=0x7f5a900b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8f400640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a90152810 0x7f5a901483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900cc8d0 0x7f5a900b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 -- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a90152810 msgr2=0x7f5a901483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a90152810 0x7f5a901483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 -- 172.21.15.59:0/2502485704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a900ce430 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8ea00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000d8b0 tx=0x7f5a8000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80019070 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a800074e0 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800049e0 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.497+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a900cce50 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.498+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a900cd410 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.499+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9004fd90 con 0x7f5a74004d40 2024-04-24T06:41:08.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.500+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a80004b80 con 0x7f5a74004d40 2024-04-24T06:41:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.501+0000 7f5a86a00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 0x7f5a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.501+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f5a8008b710 con 0x7f5a74004d40 2024-04-24T06:41:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.501+0000 7f5a87e00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 0x7f5a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.502+0000 7f5a87e00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 0x7f5a6c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5a900b7660 tx=0x7f5a780093d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.506+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a900cf030 con 0x7f5a74004d40 2024-04-24T06:41:08.665 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:41:08.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:41:08.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.714+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a900d1eb0 con 0x7f5a74004d40 2024-04-24T06:41:08.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.716+0000 7f5a86a00640 1 -- 172.21.15.59:0/2502485704 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16543 (secure 0 0 0) 0x7f5a80059a10 con 0x7f5a74004d40 2024-04-24T06:41:08.718 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:08.718 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:49.106564+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:08.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 msgr2=0x7f5a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 0x7f5a6c06edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5a900b7660 tx=0x7f5a780093d0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 msgr2=0x7f5a900cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000d8b0 tx=0x7f5a8000dd70 comp rx=0 tx=0).stop 2024-04-24T06:41:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 shutdown_connections 2024-04-24T06:41:08.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5a6c06c8f0 0x7f5a6c06edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a90152810 0x7f5a901483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5a900cc8d0 0x7f5a900b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 --2- 172.21.15.59:0/2502485704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a74004d40 0x7f5a900cdef0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 >> 172.21.15.59:0/2502485704 conn(0x7f5a9009d920 msgr2=0x7f5a900b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 shutdown_connections 2024-04-24T06:41:08.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.718+0000 7f5a8fe00640 1 -- 172.21.15.59:0/2502485704 wait complete. 2024-04-24T06:41:08.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:08.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:08.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.931+0000 7f063e800640 1 Processor -- start 2024-04-24T06:41:08.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.932+0000 7f063e800640 1 -- start start 2024-04-24T06:41:08.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.933+0000 7f063e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f063815ed20 0x7f06380d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.933+0000 7f063e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f06380b3600 0x7f06380d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.933+0000 7f063e800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06380b5da0 con 0x7f06380b3f10 2024-04-24T06:41:08.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.933+0000 7f063e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f063815f100 con 0x7f06380b3600 2024-04-24T06:41:08.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.933+0000 7f063e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f06380d0a20 con 0x7f063815ed20 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f063de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f06380b3f10 0x7f06380b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47676/0 (socket says 172.21.15.59:47676) 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f063de00640 1 -- 172.21.15.59:0/2634929548 learned_addr learned my addr 172.21.15.59:0/2634929548 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f063815ed20 0x7f06380d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f063ca00640 1 --2- 172.21.15.59:0/2634929548 >> v2:172.21.15.66:3300/0 conn(0x7f06380b3600 0x7f06380d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2940436171 0 0) 0x7f06380b5da0 con 0x7f06380b3f10 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.934+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0620003520 con 0x7f06380b3f10 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3562831702 0 0) 0x7f0620003520 con 0x7f06380b3f10 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f063815ed20 msgr2=0x7f06380d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f063815ed20 0x7f06380d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 >> v2:172.21.15.66:3300/0 conn(0x7f06380b3600 msgr2=0x7f06380d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 --2- 172.21.15.59:0/2634929548 >> v2:172.21.15.66:3300/0 conn(0x7f06380b3600 0x7f06380d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06380d2ed0 con 0x7f06380b3f10 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f06300019f0 con 0x7f06380b3f10 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 >> v1:172.21.15.59:6789/0 conn(0x7f06380b3f10 legacy=0x7f06380b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06200039b0 0x7f0620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0620004800 0x7f0620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0620005230 0x7f062000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06380d0a20 con 0x7f06200039b0 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f062000c0f0 con 0x7f0620004800 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f062000c2f0 con 0x7f0620005230 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f063815ed20 0x7f06380d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06200039b0 0x7f0620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.935+0000 7f063ca00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0620004800 0x7f0620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063de00640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0620005230 0x7f062000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063d400640 1 -- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0620005230 msgr2=0x7f062000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0620005230 0x7f062000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063d400640 1 -- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0620004800 msgr2=0x7f0620004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0620004800 0x7f0620004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.936+0000 7f063d400640 1 -- 172.21.15.59:0/2634929548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620003520 con 0x7f06200039b0 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f063d400640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06200039b0 0x7f0620003dd0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f062c006fd0 tx=0x7f0620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe554c86ab08d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c037070 con 0x7f06200039b0 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c02aa40 con 0x7f06200039b0 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f0637e00640 1 -- 172.21.15.59:0/2634929548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00f040 con 0x7f06200039b0 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f063e800640 1 -- 172.21.15.59:0/2634929548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06200039b0 msgr2=0x7f0620003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.937+0000 7f063e800640 1 --2- 172.21.15.59:0/2634929548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06200039b0 0x7f0620003dd0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f062c006fd0 tx=0x7f0620005830 comp rx=0 tx=0).stop 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.938+0000 7f063de00640 1 -- 172.21.15.59:0/2634929548 reap_dead start 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.938+0000 7f063e800640 1 -- 172.21.15.59:0/2634929548 shutdown_connections 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.938+0000 7f063e800640 1 -- 172.21.15.59:0/2634929548 >> 172.21.15.59:0/2634929548 conn(0x7f063808bf90 msgr2=0x7f06380d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.938+0000 7f063e800640 1 -- 172.21.15.59:0/2634929548 shutdown_connections 2024-04-24T06:41:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.938+0000 7f063e800640 1 -- 172.21.15.59:0/2634929548 wait complete. 2024-04-24T06:41:08.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.940+0000 7f063e800640 1 Processor -- start 2024-04-24T06:41:08.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.940+0000 7f063e800640 1 -- start start 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06200039b0 0x7f06381f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06381f4910 0x7f063806ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06380d0ea0 con 0x7f063815c990 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06380d3490 con 0x7f06200039b0 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f06380d1020 con 0x7f06381f4910 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49550/0 (socket says 172.21.15.59:49550) 2024-04-24T06:41:08.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.941+0000 7f063ca00640 1 -- 172.21.15.59:0/1120342327 learned_addr learned my addr 172.21.15.59:0/1120342327 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063d400640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06200039b0 0x7f06381f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 -- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06381f4910 msgr2=0x7f063806ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06381f4910 0x7f063806ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 -- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06200039b0 msgr2=0x7f06381f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06200039b0 0x7f06381f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 -- 172.21.15.59:0/1120342327 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06381f4ec0 con 0x7f063815c990 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063ca00640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f062400f630 tx=0x7f062400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624019020 con 0x7f063815c990 2024-04-24T06:41:08.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624014870 con 0x7f063815c990 2024-04-24T06:41:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624014c20 con 0x7f063815c990 2024-04-24T06:41:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.942+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06381f51a0 con 0x7f063815c990 2024-04-24T06:41:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.943+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381f7ca0 con 0x7f063815c990 2024-04-24T06:41:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.944+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063804fd90 con 0x7f063815c990 2024-04-24T06:41:08.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.945+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062401c020 con 0x7f063815c990 2024-04-24T06:41:08.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.946+0000 7f0635600640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:08.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.946+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f062408c2a0 con 0x7f063815c990 2024-04-24T06:41:08.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.946+0000 7f063d400640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:08.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.947+0000 7f063d400640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 0x7f061806ede0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f062c034000 tx=0x7f062c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:08.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:08.951+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0624023020 con 0x7f063815c990 2024-04-24T06:41:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.152+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06380b53c0 con 0x7f063815c990 2024-04-24T06:41:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.153+0000 7f0635600640 1 -- 172.21.15.59:0/1120342327 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f062405a6c0 con 0x7f063815c990 2024-04-24T06:41:09.155 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.155+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 msgr2=0x7f061806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.155+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 0x7f061806ede0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f062c034000 tx=0x7f062c02e000 comp rx=0 tx=0).stop 2024-04-24T06:41:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.155+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 msgr2=0x7f06381f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.155+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f062400f630 tx=0x7f062400faf0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 shutdown_connections 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f06381f4910 0x7f063806ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f063815c990 0x7f06381f43d0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 --2- 172.21.15.59:0/1120342327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06200039b0 0x7f06381f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 >> 172.21.15.59:0/1120342327 conn(0x7f063808bf90 msgr2=0x7f06380cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 shutdown_connections 2024-04-24T06:41:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.156+0000 7f063e800640 1 -- 172.21.15.59:0/1120342327 wait complete. 2024-04-24T06:41:09.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:09.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.392+0000 7f75d5600640 1 Processor -- start 2024-04-24T06:41:09.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.392+0000 7f75d5600640 1 -- start start 2024-04-24T06:41:09.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.393+0000 7f75d5600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f75d01567e0 0x7f75d0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d5600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75d00d0d10 0x7f75d0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d5600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d015cab0 con 0x7f75d015d810 2024-04-24T06:41:09.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d5600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f75d0160e40 con 0x7f75d00d0d10 2024-04-24T06:41:09.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d5600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f75d015cd30 con 0x7f75d01567e0 2024-04-24T06:41:09.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d4c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75d00d0d10 0x7f75d0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d4c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75d00d0d10 0x7f75d0160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49554/0 (socket says 172.21.15.59:49554) 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75d4c00640 1 -- 172.21.15.59:0/3934968633 learned_addr learned my addr 172.21.15.59:0/3934968633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.394+0000 7f75cfe00640 1 --2- 172.21.15.59:0/3934968633 >> v2:172.21.15.66:3300/0 conn(0x7f75d01567e0 0x7f75d0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.395+0000 7f75d4c00640 1 -- 172.21.15.59:0/3934968633 >> v2:172.21.15.66:3300/0 conn(0x7f75d01567e0 msgr2=0x7f75d0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.395+0000 7f75d4c00640 1 --2- 172.21.15.59:0/3934968633 >> v2:172.21.15.66:3300/0 conn(0x7f75d01567e0 0x7f75d0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.395+0000 7f75d4c00640 1 -- 172.21.15.59:0/3934968633 >> v1:172.21.15.66:6789/0 conn(0x7f75d015d810 legacy=0x7f75d00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.395+0000 7f75d4c00640 1 -- 172.21.15.59:0/3934968633 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d015d010 con 0x7f75d00d0d10 2024-04-24T06:41:09.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.396+0000 7f75d4c00640 1 --2- 172.21.15.59:0/3934968633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d0160110 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f75d00b6360 tx=0x7f75c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc29e92e56730124 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75cf400640 1 -- 172.21.15.59:0/3934968633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8036070 con 0x7f75d00d0d10 2024-04-24T06:41:09.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75cf400640 1 -- 172.21.15.59:0/3934968633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8031600 con 0x7f75d00d0d10 2024-04-24T06:41:09.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75cf400640 1 -- 172.21.15.59:0/3934968633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80319d0 con 0x7f75d00d0d10 2024-04-24T06:41:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 -- 172.21.15.59:0/3934968633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 msgr2=0x7f75d0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 --2- 172.21.15.59:0/3934968633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d0160110 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f75d00b6360 tx=0x7f75c802a730 comp rx=0 tx=0).stop 2024-04-24T06:41:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 -- 172.21.15.59:0/3934968633 shutdown_connections 2024-04-24T06:41:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 --2- 172.21.15.59:0/3934968633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d0160110 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 --2- 172.21.15.59:0/3934968633 >> v2:172.21.15.66:3300/0 conn(0x7f75d01567e0 0x7f75d0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.397+0000 7f75d5600640 1 -- 172.21.15.59:0/3934968633 >> 172.21.15.59:0/3934968633 conn(0x7f75d006bdf0 msgr2=0x7f75d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.398+0000 7f75d5600640 1 -- 172.21.15.59:0/3934968633 shutdown_connections 2024-04-24T06:41:09.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.398+0000 7f75d5600640 1 -- 172.21.15.59:0/3934968633 wait complete. 2024-04-24T06:41:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.399+0000 7f75d5600640 1 Processor -- start 2024-04-24T06:41:09.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 -- start start 2024-04-24T06:41:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75d01567e0 0x7f75d01f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 0x7f75d00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01607d0 con 0x7f75d00d0d10 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0160650 con 0x7f75d01567e0 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.400+0000 7f75d5600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f75d0160950 con 0x7f75d015d810 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49570/0 (socket says 172.21.15.59:49570) 2024-04-24T06:41:09.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 -- 172.21.15.59:0/926069319 learned_addr learned my addr 172.21.15.59:0/926069319 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75c7e00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75d01567e0 0x7f75d01f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75d4c00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 0x7f75d00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 -- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 msgr2=0x7f75d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 0x7f75d00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 -- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75d01567e0 msgr2=0x7f75d01f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75d01567e0 0x7f75d01f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75cfe00640 1 -- 172.21.15.59:0/926069319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d006fa50 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.401+0000 7f75d4c00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 0x7f75d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75cfe00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d830 tx=0x7f75c000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000b6c0 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c0019070 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000ba80 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01f5cb0 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.402+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d00ceaa0 con 0x7f75d00d0d10 2024-04-24T06:41:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.404+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d004fd90 con 0x7f75d00d0d10 2024-04-24T06:41:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.404+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75c001e460 con 0x7f75d00d0d10 2024-04-24T06:41:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.405+0000 7f75ccc00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 0x7f75ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.406+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f75c008c2a0 con 0x7f75d00d0d10 2024-04-24T06:41:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.406+0000 7f75c7e00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 0x7f75ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.407+0000 7f75c7e00640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 0x7f75ac06ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f75d0154b70 tx=0x7f75b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.410+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75c005a630 con 0x7f75d00d0d10 2024-04-24T06:41:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.614+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75d015a1c0 con 0x7f75d00d0d10 2024-04-24T06:41:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.615+0000 7f75ccc00640 1 -- 172.21.15.59:0/926069319 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f75c0059f30 con 0x7f75d00d0d10 2024-04-24T06:41:09.617 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 msgr2=0x7f75ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 0x7f75ac06ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f75d0154b70 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 msgr2=0x7f75d01f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d830 tx=0x7f75c000dcf0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 shutdown_connections 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f75ac06c920 0x7f75ac06ede0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f75d015d810 0x7f75d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75d01567e0 0x7f75d01f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 --2- 172.21.15.59:0/926069319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d00d0d10 0x7f75d01f7f40 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 >> 172.21.15.59:0/926069319 conn(0x7f75d006bdf0 msgr2=0x7f75d015a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 shutdown_connections 2024-04-24T06:41:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.618+0000 7f75d5600640 1 -- 172.21.15.59:0/926069319 wait complete. 2024-04-24T06:41:09.640 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.850+0000 7f3736600640 1 Processor -- start 2024-04-24T06:41:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.850+0000 7f3736600640 1 -- start start 2024-04-24T06:41:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f3736600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f37301567e0 0x7f3730156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f3736600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f373015d810 0x7f37300d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f3736600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373015cab0 con 0x7f37300d0b40 2024-04-24T06:41:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f3736600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f37300d0f90 con 0x7f373015d810 2024-04-24T06:41:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f3736600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3730160e40 con 0x7f37301567e0 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f372fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f373015d810 0x7f37300d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f372fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f373015d810 0x7f37300d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49576/0 (socket says 172.21.15.59:49576) 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.852+0000 7f372fe00640 1 -- 172.21.15.59:0/735262121 learned_addr learned my addr 172.21.15.59:0/735262121 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.853+0000 7f3735200640 1 --2- 172.21.15.59:0/735262121 >> v2:172.21.15.66:3300/0 conn(0x7f37301567e0 0x7f3730156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.853+0000 7f372fe00640 1 -- 172.21.15.59:0/735262121 >> v2:172.21.15.66:3300/0 conn(0x7f37301567e0 msgr2=0x7f3730156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.853+0000 7f372fe00640 1 --2- 172.21.15.59:0/735262121 >> v2:172.21.15.66:3300/0 conn(0x7f37301567e0 0x7f3730156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.853+0000 7f372fe00640 1 -- 172.21.15.59:0/735262121 >> v1:172.21.15.66:6789/0 conn(0x7f37300d0b40 legacy=0x7f3730160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.853+0000 7f372fe00640 1 -- 172.21.15.59:0/735262121 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373015cd30 con 0x7f373015d810 2024-04-24T06:41:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.854+0000 7f372fe00640 1 --2- 172.21.15.59:0/735262121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015d810 0x7f37300d0600 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f37300b6360 tx=0x7f371c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58e217d963ca461 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.854+0000 7f372f400640 1 -- 172.21.15.59:0/735262121 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c037070 con 0x7f373015d810 2024-04-24T06:41:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f372f400640 1 -- 172.21.15.59:0/735262121 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c032440 con 0x7f373015d810 2024-04-24T06:41:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f372f400640 1 -- 172.21.15.59:0/735262121 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c032810 con 0x7f373015d810 2024-04-24T06:41:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 -- 172.21.15.59:0/735262121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015d810 msgr2=0x7f37300d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 --2- 172.21.15.59:0/735262121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015d810 0x7f37300d0600 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f37300b6360 tx=0x7f371c02a230 comp rx=0 tx=0).stop 2024-04-24T06:41:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 -- 172.21.15.59:0/735262121 shutdown_connections 2024-04-24T06:41:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 --2- 172.21.15.59:0/735262121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f373015d810 0x7f37300d0600 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 --2- 172.21.15.59:0/735262121 >> v2:172.21.15.66:3300/0 conn(0x7f37301567e0 0x7f3730156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.855+0000 7f3736600640 1 -- 172.21.15.59:0/735262121 >> 172.21.15.59:0/735262121 conn(0x7f373006bdf0 msgr2=0x7f373006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.856+0000 7f3736600640 1 -- 172.21.15.59:0/735262121 shutdown_connections 2024-04-24T06:41:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.856+0000 7f3736600640 1 -- 172.21.15.59:0/735262121 wait complete. 2024-04-24T06:41:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.857+0000 7f3736600640 1 Processor -- start 2024-04-24T06:41:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.858+0000 7f3736600640 1 -- start start 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.858+0000 7f3736600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.858+0000 7f3736600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37301567e0 0x7f373014f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3736600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 0x7f373014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3736600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3730160820 con 0x7f37300d0b40 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3736600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f373015d2f0 con 0x7f373015d810 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3736600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f37301609a0 con 0x7f37301567e0 2024-04-24T06:41:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49590/0 (socket says 172.21.15.59:49590) 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 -- 172.21.15.59:0/172664524 learned_addr learned my addr 172.21.15.59:0/172664524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f372fe00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37301567e0 0x7f373014f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735c00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 0x7f373014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 -- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37301567e0 msgr2=0x7f373014f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37301567e0 0x7f373014f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 -- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 msgr2=0x7f373014f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 0x7f373014f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735200640 1 -- 172.21.15.59:0/172664524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301f48c0 con 0x7f37300d0b40 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.859+0000 7f3735c00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 0x7f373014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f3735200640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f372400d830 tx=0x7f372400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400b6c0 con 0x7f37300d0b40 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724019070 con 0x7f37300d0b40 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3730072700 con 0x7f37300d0b40 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400ba80 con 0x7f37300d0b40 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.860+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373009ddd0 con 0x7f37300d0b40 2024-04-24T06:41:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.861+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37300720e0 con 0x7f37300d0b40 2024-04-24T06:41:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.862+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f372401e460 con 0x7f37300d0b40 2024-04-24T06:41:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.863+0000 7f372cc00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.863+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f372408c230 con 0x7f37300d0b40 2024-04-24T06:41:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.864+0000 7f372fe00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.864+0000 7f372fe00640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f371c034000 tx=0x7f371c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:09.868+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f372405a650 con 0x7f37300d0b40 2024-04-24T06:41:10.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.070+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f373004fd90 con 0x7f37300d0b40 2024-04-24T06:41:10.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.071+0000 7f372cc00640 1 -- 172.21.15.59:0/172664524 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f3724059f50 con 0x7f37300d0b40 2024-04-24T06:41:10.073 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 msgr2=0x7f371006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f371c034000 tx=0x7f371c02e000 comp rx=0 tx=0).stop 2024-04-24T06:41:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 msgr2=0x7f37301f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f372400d830 tx=0x7f372400dcf0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 shutdown_connections 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f373015d810 0x7f373014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f37301567e0 0x7f373014f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 --2- 172.21.15.59:0/172664524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37300d0b40 0x7f37301f4380 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.074+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 >> 172.21.15.59:0/172664524 conn(0x7f373006bdf0 msgr2=0x7f3730099400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.075+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 shutdown_connections 2024-04-24T06:41:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.075+0000 7f3736600640 1 -- 172.21.15.59:0/172664524 wait complete. 2024-04-24T06:41:10.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.306+0000 7fe97d600640 1 Processor -- start 2024-04-24T06:41:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.306+0000 7fe97d600640 1 -- start start 2024-04-24T06:41:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe97d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe9781416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe97d600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe97815c250 0x7fe9781555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe97d600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97815ba70 con 0x7fe97815f060 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe97d600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe978141c20 con 0x7fe97815c250 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe97d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe97815bdc0 con 0x7fe9780d05c0 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe977400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe97815f060 0x7fe9780d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47682/0 (socket says 172.21.15.59:47682) 2024-04-24T06:41:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe977400640 1 -- 172.21.15.59:0/1799863823 learned_addr learned my addr 172.21.15.59:0/1799863823 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.308+0000 7fe977e00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe9781416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe97cc00640 1 --2- 172.21.15.59:0/1799863823 >> v2:172.21.15.66:3300/0 conn(0x7fe97815c250 0x7fe9781555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869900053 0 0) 0x7fe97815ba70 con 0x7fe97815f060 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe95c003520 con 0x7fe97815f060 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2111719979 0 0) 0x7fe95c003520 con 0x7fe97815f060 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 msgr2=0x7fe9781416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe9781416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 >> v2:172.21.15.66:3300/0 conn(0x7fe97815c250 msgr2=0x7fe9781555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 --2- 172.21.15.59:0/1799863823 >> v2:172.21.15.66:3300/0 conn(0x7fe97815c250 0x7fe9781555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97815f800 con 0x7fe97815f060 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe964001670 con 0x7fe97815f060 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 >> v1:172.21.15.59:6789/0 conn(0x7fe97815f060 legacy=0x7fe9780d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c004800 0x7fe95c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe95c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe978141c20 con 0x7fe95c0039b0 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe97815bdc0 con 0x7fe95c004800 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.309+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe95c00b9c0 con 0x7fe9780d05c0 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977400640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c004800 0x7fe95c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe97cc00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe95c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 -- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 msgr2=0x7fe95c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe95c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 -- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c004800 msgr2=0x7fe95c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c004800 0x7fe95c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe977e00640 1 -- 172.21.15.59:0/1799863823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c003520 con 0x7fe95c0039b0 2024-04-24T06:41:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.310+0000 7fe97cc00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe9780d05c0 0x7fe95c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.311+0000 7fe977e00640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe968009ac0 tx=0x7fe95c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aae9418ac4007a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.311+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968036070 con 0x7fe95c0039b0 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968031400 con 0x7fe95c0039b0 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe976a00640 1 -- 172.21.15.59:0/1799863823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968009690 con 0x7fe95c0039b0 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 -- 172.21.15.59:0/1799863823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe95c0039b0 msgr2=0x7fe95c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 --2- 172.21.15.59:0/1799863823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe968009ac0 tx=0x7fe95c005830 comp rx=0 tx=0).stop 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97cc00640 1 -- 172.21.15.59:0/1799863823 reap_dead start 2024-04-24T06:41:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 -- 172.21.15.59:0/1799863823 shutdown_connections 2024-04-24T06:41:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 -- 172.21.15.59:0/1799863823 >> 172.21.15.59:0/1799863823 conn(0x7fe97806bdf0 msgr2=0x7fe97806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 -- 172.21.15.59:0/1799863823 shutdown_connections 2024-04-24T06:41:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.312+0000 7fe97d600640 1 -- 172.21.15.59:0/1799863823 wait complete. 2024-04-24T06:41:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.314+0000 7fe97d600640 1 Processor -- start 2024-04-24T06:41:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.314+0000 7fe97d600640 1 -- start start 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c0039b0 0x7fe97815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97809cf50 0x7fe9781f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe97815f480 con 0x7fe97809cad0 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe97815fdc0 con 0x7fe95c0039b0 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.315+0000 7fe97d600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe9781416e0 con 0x7fe97809cf50 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49610/0 (socket says 172.21.15.59:49610) 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 -- 172.21.15.59:0/2902253986 learned_addr learned my addr 172.21.15.59:0/2902253986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe97cc00640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97809cf50 0x7fe9781f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977e00640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c0039b0 0x7fe97815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 -- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97809cf50 msgr2=0x7fe9781f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97809cf50 0x7fe9781f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 -- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c0039b0 msgr2=0x7fe97815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c0039b0 0x7fe97815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 -- 172.21.15.59:0/2902253986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9781524b0 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.316+0000 7fe977400640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e780 tx=0x7fe96400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.317+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ee90 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.317+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964018070 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.317+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ca90 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.317+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe978152790 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.317+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97809ab10 con 0x7fe97809cad0 2024-04-24T06:41:10.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.318+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97804fd90 con 0x7fe97809cad0 2024-04-24T06:41:10.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.319+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe96400cc30 con 0x7fe97809cad0 2024-04-24T06:41:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.320+0000 7fe96fe00640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 0x7fe95806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.320+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fe96408ba60 con 0x7fe97809cad0 2024-04-24T06:41:10.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.321+0000 7fe977e00640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 0x7fe95806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.321+0000 7fe977e00640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 0x7fe95806ef50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe968033000 tx=0x7fe968008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.326+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe964059df0 con 0x7fe97809cad0 2024-04-24T06:41:10.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.547+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe97809ae20 con 0x7fe97809cad0 2024-04-24T06:41:10.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.548+0000 7fe96fe00640 1 -- 172.21.15.59:0/2902253986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7fe9640596f0 con 0x7fe97809cad0 2024-04-24T06:41:10.550 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:41:10.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.550+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 msgr2=0x7fe95806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.550+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 0x7fe95806ef50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe968033000 tx=0x7fe968008030 comp rx=0 tx=0).stop 2024-04-24T06:41:10.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.550+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 msgr2=0x7fe9781f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.550+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e780 tx=0x7fe96400ec40 comp rx=0 tx=0).stop 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 shutdown_connections 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe95806ca90 0x7fe95806ef50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe97809cf50 0x7fe9781f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97809cad0 0x7fe9781f8790 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 --2- 172.21.15.59:0/2902253986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe95c0039b0 0x7fe97815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 >> 172.21.15.59:0/2902253986 conn(0x7fe97806bdf0 msgr2=0x7fe97814eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 shutdown_connections 2024-04-24T06:41:10.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.551+0000 7fe97d600640 1 -- 172.21.15.59:0/2902253986 wait complete. 2024-04-24T06:41:10.571 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:10.572 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:10.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:10.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.839+0000 7fb73cc00640 1 Processor -- start 2024-04-24T06:41:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.839+0000 7fb73cc00640 1 -- start start 2024-04-24T06:41:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.840+0000 7fb73cc00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb7380b5ff0 0x7fb7380b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.840+0000 7fb73cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb7380d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.840+0000 7fb73cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7380cf030 con 0x7fb73815e7c0 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.840+0000 7fb73cc00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb7380b7000 con 0x7fb7380b5ff0 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.840+0000 7fb73cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7380cf620 con 0x7fb7380d0d90 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb737400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb73815e7c0 0x7fb7380d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47696/0 (socket says 172.21.15.59:47696) 2024-04-24T06:41:10.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb737400640 1 -- 172.21.15.59:0/753582311 learned_addr learned my addr 172.21.15.59:0/753582311 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb736a00640 1 --2- 172.21.15.59:0/753582311 >> v2:172.21.15.66:3300/0 conn(0x7fb7380b5ff0 0x7fb7380b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb737e00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb7380d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745883376 0 0) 0x7fb720001110 con 0x7fb73815e7c0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.841+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb71c003520 con 0x7fb73815e7c0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 731978047 0 0) 0x7fb71c003520 con 0x7fb73815e7c0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 msgr2=0x7fb7380d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb7380d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 >> v2:172.21.15.66:3300/0 conn(0x7fb7380b5ff0 msgr2=0x7fb7380b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 --2- 172.21.15.59:0/753582311 >> v2:172.21.15.66:3300/0 conn(0x7fb7380b5ff0 0x7fb7380b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7380b6a40 con 0x7fb73815e7c0 2024-04-24T06:41:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fb720001840 con 0x7fb73815e7c0 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 >> v1:172.21.15.59:6789/0 conn(0x7fb73815e7c0 legacy=0x7fb7380d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c0039b0 0x7fb71c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb71c004800 0x7fb71c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb71c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380b7000 con 0x7fb71c004800 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380cf620 con 0x7fb71c0039b0 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c00b9c0 con 0x7fb7380d0d90 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.842+0000 7fb736a00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb71c004800 0x7fb71c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb737e00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb71c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb737400640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c0039b0 0x7fb71c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb736a00640 1 -- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 msgr2=0x7fb71c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb736a00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380d0d90 0x7fb71c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb736a00640 1 -- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c0039b0 msgr2=0x7fb71c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:10.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb736a00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c0039b0 0x7fb71c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.843+0000 7fb736a00640 1 -- 172.21.15.59:0/753582311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c003520 con 0x7fb71c004800 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.844+0000 7fb736a00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb71c004800 0x7fb71c004c80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fc0 tx=0x7fb71c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d3faac7c8e6e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.844+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728009c10 con 0x7fb71c004800 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb728036070 con 0x7fb71c004800 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb736000640 1 -- 172.21.15.59:0/753582311 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72802d070 con 0x7fb71c004800 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 -- 172.21.15.59:0/753582311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb71c004800 msgr2=0x7fb71c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 --2- 172.21.15.59:0/753582311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb71c004800 0x7fb71c004c80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb728009fc0 tx=0x7fb71c005830 comp rx=0 tx=0).stop 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb737e00640 1 -- 172.21.15.59:0/753582311 reap_dead start 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 -- 172.21.15.59:0/753582311 shutdown_connections 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 -- 172.21.15.59:0/753582311 >> 172.21.15.59:0/753582311 conn(0x7fb73809d920 msgr2=0x7fb73808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 -- 172.21.15.59:0/753582311 shutdown_connections 2024-04-24T06:41:10.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.845+0000 7fb73cc00640 1 -- 172.21.15.59:0/753582311 wait complete. 2024-04-24T06:41:10.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.847+0000 7fb73cc00640 1 Processor -- start 2024-04-24T06:41:10.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.847+0000 7fb73cc00640 1 -- start start 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c004800 0x7fb7380b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380cd070 0x7fb7381529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380d2800 con 0x7fb73815b700 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380cf2b0 con 0x7fb71c004800 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.848+0000 7fb73cc00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb7380d1a80 con 0x7fb7380cd070 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49620/0 (socket says 172.21.15.59:49620) 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 -- 172.21.15.59:0/698271973 learned_addr learned my addr 172.21.15.59:0/698271973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737400640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c004800 0x7fb7380b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb736a00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380cd070 0x7fb7381529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 -- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380cd070 msgr2=0x7fb7381529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380cd070 0x7fb7381529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 -- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c004800 msgr2=0x7fb7380b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c004800 0x7fb7380b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.849+0000 7fb737e00640 1 -- 172.21.15.59:0/698271973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb738154460 con 0x7fb73815b700 2024-04-24T06:41:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb737e00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb7380a24f0 tx=0x7fb73000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730004440 con 0x7fb73815b700 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb730019070 con 0x7fb73815b700 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730003850 con 0x7fb73815b700 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb738152f00 con 0x7fb73815b700 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.850+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7380b8dd0 con 0x7fb73815b700 2024-04-24T06:41:10.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.851+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73815c0c0 con 0x7fb73815b700 2024-04-24T06:41:10.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.853+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb730012070 con 0x7fb73815b700 2024-04-24T06:41:10.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.853+0000 7fb72f400640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:10.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.854+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fb73008be00 con 0x7fb73815b700 2024-04-24T06:41:10.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.854+0000 7fb737400640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:10.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.855+0000 7fb737400640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 0x7fb71406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb720001050 tx=0x7fb720009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:10.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:10.857+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb73005a310 con 0x7fb73815b700 2024-04-24T06:41:11.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.062+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb73804fd90 con 0x7fb73815b700 2024-04-24T06:41:11.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.064+0000 7fb72f400640 1 -- 172.21.15.59:0/698271973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16543 (secure 0 0 0) 0x7fb730059c10 con 0x7fb73815b700 2024-04-24T06:41:11.066 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:11.066 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:40:49.106564+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 msgr2=0x7fb71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 0x7fb71406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb720001050 tx=0x7fb720009040 comp rx=0 tx=0).stop 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 msgr2=0x7fb73815bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb7380a24f0 tx=0x7fb73000f8a0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 shutdown_connections 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb73815b700 0x7fb73815bb80 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb7380cd070 0x7fb7381529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 --2- 172.21.15.59:0/698271973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb71c004800 0x7fb7380b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 >> 172.21.15.59:0/698271973 conn(0x7fb73809d920 msgr2=0x7fb7380d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 shutdown_connections 2024-04-24T06:41:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.067+0000 7fb73cc00640 1 -- 172.21.15.59:0/698271973 wait complete. 2024-04-24T06:41:11.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:11.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.303+0000 7f50ace00640 1 Processor -- start 2024-04-24T06:41:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.303+0000 7f50ace00640 1 -- start start 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.304+0000 7f50ace00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50a8155a80 0x7f50a806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.304+0000 7f50ace00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f50a80af2a0 0x7f50a80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.304+0000 7f50ace00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a815ef20 con 0x7f50a80d0600 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.304+0000 7f50ace00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f50a815f1c0 con 0x7f50a8155a80 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.304+0000 7f50ace00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f50a815f4a0 con 0x7f50a80af2a0 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50a80d0600 0x7f50a80b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47712/0 (socket says 172.21.15.59:47712) 2024-04-24T06:41:11.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a6a00640 1 -- 172.21.15.59:0/280316891 learned_addr learned my addr 172.21.15.59:0/280316891 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7400640 1 --2- 172.21.15.59:0/280316891 >> v2:172.21.15.59:3300/0 conn(0x7f50a8155a80 0x7f50a806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7e00640 1 --2- 172.21.15.59:0/280316891 >> v2:172.21.15.66:3300/0 conn(0x7f50a80af2a0 0x7f50a80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a6000640 1 -- 172.21.15.59:0/280316891 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726535284 0 0) 0x7f50a815ef20 con 0x7f50a80d0600 2024-04-24T06:41:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a6000640 1 -- 172.21.15.59:0/280316891 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f508c003520 con 0x7f50a80d0600 2024-04-24T06:41:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7400640 1 -- 172.21.15.59:0/280316891 >> v2:172.21.15.66:3300/0 conn(0x7f50a80af2a0 msgr2=0x7f50a80d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7400640 1 --2- 172.21.15.59:0/280316891 >> v2:172.21.15.66:3300/0 conn(0x7f50a80af2a0 0x7f50a80d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7400640 1 -- 172.21.15.59:0/280316891 >> v1:172.21.15.59:6789/0 conn(0x7f50a80d0600 legacy=0x7f50a80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.305+0000 7f50a7400640 1 -- 172.21.15.59:0/280316891 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a815f780 con 0x7f50a8155a80 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.306+0000 7f50a7400640 1 --2- 172.21.15.59:0/280316891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8155a80 0x7f50a806f9c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f50a806df60 tx=0x7f509802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7e0ed44808cfe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50a6000640 1 -- 172.21.15.59:0/280316891 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098037070 con 0x7f50a8155a80 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50a6000640 1 -- 172.21.15.59:0/280316891 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5098032440 con 0x7f50a8155a80 2024-04-24T06:41:11.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50a6000640 1 -- 172.21.15.59:0/280316891 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098032810 con 0x7f50a8155a80 2024-04-24T06:41:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 -- 172.21.15.59:0/280316891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8155a80 msgr2=0x7f50a806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 --2- 172.21.15.59:0/280316891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8155a80 0x7f50a806f9c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f50a806df60 tx=0x7f509802a1a0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 -- 172.21.15.59:0/280316891 shutdown_connections 2024-04-24T06:41:11.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 --2- 172.21.15.59:0/280316891 >> v2:172.21.15.66:3300/0 conn(0x7f50a80af2a0 0x7f50a80d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 --2- 172.21.15.59:0/280316891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a8155a80 0x7f50a806f9c0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.307+0000 7f50ace00640 1 -- 172.21.15.59:0/280316891 >> 172.21.15.59:0/280316891 conn(0x7f50a806bdf0 msgr2=0x7f50a806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:11.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.308+0000 7f50ace00640 1 -- 172.21.15.59:0/280316891 shutdown_connections 2024-04-24T06:41:11.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.308+0000 7f50ace00640 1 -- 172.21.15.59:0/280316891 wait complete. 2024-04-24T06:41:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.309+0000 7f50ace00640 1 Processor -- start 2024-04-24T06:41:11.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.310+0000 7f50ace00640 1 -- start start 2024-04-24T06:41:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.310+0000 7f50ace00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50a80af2a0 0x7f50a8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50ace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50ace00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 0x7f50a81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50ace00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50a806ff00 con 0x7f50a80d0600 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50ace00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50a815fd40 con 0x7f50a8155a80 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50ace00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f50a806ef80 con 0x7f50a80af2a0 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49644/0 (socket says 172.21.15.59:49644) 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 -- 172.21.15.59:0/4166799271 learned_addr learned my addr 172.21.15.59:0/4166799271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a7400640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50a80af2a0 0x7f50a8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a7e00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 0x7f50a81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 -- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50a80af2a0 msgr2=0x7f50a8148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50a80af2a0 0x7f50a8148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 -- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 msgr2=0x7f50a81502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 0x7f50a81502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a6a00640 1 -- 172.21.15.59:0/4166799271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a80978b0 con 0x7f50a80d0600 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.311+0000 7f50a7e00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 0x7f50a81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:11.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f50a6a00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f509400c460 tx=0x7f509400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094017070 con 0x7f50a80d0600 2024-04-24T06:41:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5094005ce0 con 0x7f50a80d0600 2024-04-24T06:41:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094007290 con 0x7f50a80d0600 2024-04-24T06:41:11.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a80b2a90 con 0x7f50a80d0600 2024-04-24T06:41:11.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.312+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a8099b40 con 0x7f50a80d0600 2024-04-24T06:41:11.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.313+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a80ad4f0 con 0x7f50a80d0600 2024-04-24T06:41:11.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.314+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f509401a070 con 0x7f50a80d0600 2024-04-24T06:41:11.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.316+0000 7f509f400640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 0x7f508806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.316+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f509408af00 con 0x7f50a80d0600 2024-04-24T06:41:11.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.316+0000 7f50a7400640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 0x7f508806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.317+0000 7f50a7400640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 0x7f508806edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5098034000 tx=0x7f509802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.321+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50a80ad4f0 con 0x7f50a80d0600 2024-04-24T06:41:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.527+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50a804fd90 con 0x7f50a80d0600 2024-04-24T06:41:11.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.528+0000 7f509f400640 1 -- 172.21.15.59:0/4166799271 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f5094059320 con 0x7f50a80d0600 2024-04-24T06:41:11.530 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:11.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.530+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 msgr2=0x7f508806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.530+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 0x7f508806edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5098034000 tx=0x7f509802e000 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 msgr2=0x7f50a8097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f509400c460 tx=0x7f509400c920 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 shutdown_connections 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f508806c8f0 0x7f508806edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a8155a80 0x7f50a81502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50a80d0600 0x7f50a8097370 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 --2- 172.21.15.59:0/4166799271 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f50a80af2a0 0x7f50a8148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 >> 172.21.15.59:0/4166799271 conn(0x7f50a806bdf0 msgr2=0x7f50a8071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 shutdown_connections 2024-04-24T06:41:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.531+0000 7f50ace00640 1 -- 172.21.15.59:0/4166799271 wait complete. 2024-04-24T06:41:11.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:11.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.765+0000 7f67cd400640 1 Processor -- start 2024-04-24T06:41:11.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.765+0000 7f67cd400640 1 -- start start 2024-04-24T06:41:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.766+0000 7f67cd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67c815e770 0x7f67c815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.766+0000 7f67cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67cd400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c815fce0 con 0x7f67c80d1500 2024-04-24T06:41:11.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67cd400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f67c80d3350 con 0x7f67c815e770 2024-04-24T06:41:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67cd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f67c80d1950 con 0x7f67c80d21c0 2024-04-24T06:41:11.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67c7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67c815e770 0x7f67c815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67c7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67c815e770 0x7f67c815f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49658/0 (socket says 172.21.15.59:49658) 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67c7e00640 1 -- 172.21.15.59:0/3760229655 learned_addr learned my addr 172.21.15.59:0/3760229655 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67bf400640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.767+0000 7f67c7e00640 1 -- 172.21.15.59:0/3760229655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 msgr2=0x7f67c80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.768+0000 7f67c7e00640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.768+0000 7f67c7e00640 1 -- 172.21.15.59:0/3760229655 >> v1:172.21.15.66:6789/0 conn(0x7f67c80d1500 legacy=0x7f67c8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.768+0000 7f67c7e00640 1 -- 172.21.15.59:0/3760229655 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c80d37e0 con 0x7f67c815e770 2024-04-24T06:41:11.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.768+0000 7f67bf400640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.769+0000 7f67c7e00640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c815e770 0x7f67c815f7a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef7c7e8feefdc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.769+0000 7f67c7400640 1 -- 172.21.15.59:0/3760229655 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0037070 con 0x7f67c815e770 2024-04-24T06:41:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.769+0000 7f67c7400640 1 -- 172.21.15.59:0/3760229655 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b0032440 con 0x7f67c815e770 2024-04-24T06:41:11.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.769+0000 7f67c7400640 1 -- 172.21.15.59:0/3760229655 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0032810 con 0x7f67c815e770 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 -- 172.21.15.59:0/3760229655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c815e770 msgr2=0x7f67c815f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c815e770 0x7f67c815f7a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002a210 comp rx=0 tx=0).stop 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 -- 172.21.15.59:0/3760229655 shutdown_connections 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 --2- 172.21.15.59:0/3760229655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c815e770 0x7f67c815f7a0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 -- 172.21.15.59:0/3760229655 >> 172.21.15.59:0/3760229655 conn(0x7f67c808bf90 msgr2=0x7f67c80d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 -- 172.21.15.59:0/3760229655 shutdown_connections 2024-04-24T06:41:11.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.770+0000 7f67cd400640 1 -- 172.21.15.59:0/3760229655 wait complete. 2024-04-24T06:41:11.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.772+0000 7f67cd400640 1 Processor -- start 2024-04-24T06:41:11.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.772+0000 7f67cd400640 1 -- start start 2024-04-24T06:41:11.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c815e770 0x7f67c80c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80d2d00 con 0x7f67c80d1500 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80d2a20 con 0x7f67c815e770 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67cd400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f67c80d2fe0 con 0x7f67c80d21c0 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49672/0 (socket says 172.21.15.59:49672) 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.773+0000 7f67c7e00640 1 -- 172.21.15.59:0/2151760991 learned_addr learned my addr 172.21.15.59:0/2151760991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 -- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 msgr2=0x7f67c80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 -- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c815e770 msgr2=0x7f67c80c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c815e770 0x7f67c80c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:11.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 -- 172.21.15.59:0/2151760991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c80ca6c0 con 0x7f67c80d1500 2024-04-24T06:41:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c7e00640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009990 tx=0x7f67b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.774+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0037040 con 0x7f67c80d1500 2024-04-24T06:41:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.775+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b0042070 con 0x7f67c80d1500 2024-04-24T06:41:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.775+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0030a40 con 0x7f67c80d1500 2024-04-24T06:41:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.775+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c80ce4c0 con 0x7f67c80d1500 2024-04-24T06:41:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.775+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c80cac10 con 0x7f67c80d1500 2024-04-24T06:41:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.776+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c80ce900 con 0x7f67c80d1500 2024-04-24T06:41:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.777+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67b0030be0 con 0x7f67c80d1500 2024-04-24T06:41:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.778+0000 7f67c4c00640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 0x7f67a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.778+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f67b00ac6b0 con 0x7f67c80d1500 2024-04-24T06:41:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.778+0000 7f67bf400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 0x7f67a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.779+0000 7f67bf400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 0x7f67a406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f67c80b5770 tx=0x7f67b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:11.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.782+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67c80ce900 con 0x7f67c80d1500 2024-04-24T06:41:11.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.983+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67c804fd90 con 0x7f67c80d1500 2024-04-24T06:41:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.984+0000 7f67c4c00640 1 -- 172.21.15.59:0/2151760991 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f67b0045360 con 0x7f67c80d1500 2024-04-24T06:41:12.198 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 msgr2=0x7f67a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 0x7f67a406edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f67c80b5770 tx=0x7f67b8008040 comp rx=0 tx=0).stop 2024-04-24T06:41:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 msgr2=0x7f67c814eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009990 tx=0x7f67b00046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 shutdown_connections 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f67a406c8f0 0x7f67a406edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c815e770 0x7f67c80c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f67c80d21c0 0x7f67c80cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 --2- 172.21.15.59:0/2151760991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c80d1500 0x7f67c814eb30 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 >> 172.21.15.59:0/2151760991 conn(0x7f67c808bf90 msgr2=0x7f67c80b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 shutdown_connections 2024-04-24T06:41:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:11.987+0000 7f67cd400640 1 -- 172.21.15.59:0/2151760991 wait complete. 2024-04-24T06:41:12.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:12.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.421+0000 7f803d400640 1 Processor -- start 2024-04-24T06:41:12.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.421+0000 7f803d400640 1 -- start start 2024-04-24T06:41:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.422+0000 7f803d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f80381567e0 0x7f8038156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f803d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803815cab0 con 0x7f80380d0d10 2024-04-24T06:41:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f803d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8038160e40 con 0x7f803815d810 2024-04-24T06:41:12.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f803d400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f803815cd30 con 0x7f80381567e0 2024-04-24T06:41:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f803ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f80380d0d10 0x7f8038160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47718/0 (socket says 172.21.15.59:47718) 2024-04-24T06:41:12.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f803ca00640 1 -- 172.21.15.59:0/3609174371 learned_addr learned my addr 172.21.15.59:0/3609174371 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:12.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f8037e00640 1 --2- 172.21.15.59:0/3609174371 >> v2:172.21.15.66:3300/0 conn(0x7f80381567e0 0x7f8038156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2136355236 0 0) 0x7f8030001490 con 0x7f80380d0d10 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.423+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f801c003520 con 0x7f80380d0d10 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3759768116 0 0) 0x7f801c003520 con 0x7f80380d0d10 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 >> v2:172.21.15.66:3300/0 conn(0x7f80381567e0 msgr2=0x7f8038156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 --2- 172.21.15.59:0/3609174371 >> v2:172.21.15.66:3300/0 conn(0x7f80381567e0 0x7f8038156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 >> v1:172.21.15.66:6789/0 conn(0x7f803815d810 legacy=0x7f80380d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803815d010 con 0x7f80380d0d10 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8030001bc0 con 0x7f80380d0d10 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 >> v1:172.21.15.59:6789/0 conn(0x7f80380d0d10 legacy=0x7f8038160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 0x7f801c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f801c004800 0x7f801c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c005210 0x7f801c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f803815cd30 con 0x7f801c003a00 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f801c010570 con 0x7f801c004800 2024-04-24T06:41:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f801c010770 con 0x7f801c005210 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.424+0000 7f8037e00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 0x7f801c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037400640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f801c004800 0x7f801c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037e00640 1 -- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c005210 msgr2=0x7f801c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037e00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c005210 0x7f801c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037e00640 1 -- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f801c004800 msgr2=0x7f801c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037e00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f801c004800 0x7f801c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037e00640 1 -- 172.21.15.59:0/3609174371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c003520 con 0x7f801c003a00 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f803ca00640 1 -- 172.21.15.59:0/3609174371 reap_dead start 2024-04-24T06:41:12.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.425+0000 7f8037400640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f801c004800 0x7f801c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:12.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f8037e00640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 0x7f801c003e20 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f802800ba30 tx=0x7f801c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc194f7df301d644 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028030070 con 0x7f801c003a00 2024-04-24T06:41:12.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802802ade0 con 0x7f801c003a00 2024-04-24T06:41:12.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f8036a00640 1 -- 172.21.15.59:0/3609174371 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800b600 con 0x7f801c003a00 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f803d400640 1 -- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 msgr2=0x7f801c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.426+0000 7f803d400640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 0x7f801c003e20 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f802800ba30 tx=0x7f801c005830 comp rx=0 tx=0).stop 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.427+0000 7f803d400640 1 -- 172.21.15.59:0/3609174371 shutdown_connections 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.427+0000 7f803d400640 1 --2- 172.21.15.59:0/3609174371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f801c003a00 0x7f801c003e20 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.427+0000 7f803d400640 1 -- 172.21.15.59:0/3609174371 >> 172.21.15.59:0/3609174371 conn(0x7f803806bdf0 msgr2=0x7f803806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.427+0000 7f803d400640 1 -- 172.21.15.59:0/3609174371 shutdown_connections 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.427+0000 7f803d400640 1 -- 172.21.15.59:0/3609174371 wait complete. 2024-04-24T06:41:12.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.428+0000 7f803d400640 1 Processor -- start 2024-04-24T06:41:12.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 -- start start 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c003a00 0x7f8038159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8038152150 0x7f80381f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8038156d40 con 0x7f80381525b0 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8038156bc0 con 0x7f8038152150 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.429+0000 7f803d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8038156ec0 con 0x7f801c003a00 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49684/0 (socket says 172.21.15.59:49684) 2024-04-24T06:41:12.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 -- 172.21.15.59:0/2985533822 learned_addr learned my addr 172.21.15.59:0/2985533822 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f8037e00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c003a00 0x7f8038159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f8037400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8038152150 0x7f80381f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 -- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c003a00 msgr2=0x7f8038159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c003a00 0x7f8038159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 -- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8038152150 msgr2=0x7f80381f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8038152150 0x7f80381f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.430+0000 7f803ca00640 1 -- 172.21.15.59:0/2985533822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80380b2000 con 0x7f80381525b0 2024-04-24T06:41:12.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f803ca00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8030001370 tx=0x7f803000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803000a610 con 0x7f80381525b0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803000a7d0 con 0x7f80381525b0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803000abc0 con 0x7f80381525b0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80380b22a0 con 0x7f80381525b0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.431+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80380b2860 con 0x7f80381525b0 2024-04-24T06:41:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.432+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8038068290 con 0x7f80381525b0 2024-04-24T06:41:12.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.433+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8030011070 con 0x7f80381525b0 2024-04-24T06:41:12.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.434+0000 7f802fe00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 0x7f801806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.434+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f803008b150 con 0x7f80381525b0 2024-04-24T06:41:12.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.434+0000 7f8037e00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 0x7f801806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.435+0000 7f8037e00640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 0x7f801806ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f802800ba00 tx=0x7f8028030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.439+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8038068290 con 0x7f80381525b0 2024-04-24T06:41:12.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.637+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f803804fd90 con 0x7f80381525b0 2024-04-24T06:41:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.638+0000 7f802fe00640 1 -- 172.21.15.59:0/2985533822 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f803001b030 con 0x7f80381525b0 2024-04-24T06:41:12.640 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:12.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.640+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 msgr2=0x7f801806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.640+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 0x7f801806ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f802800ba00 tx=0x7f8028030000 comp rx=0 tx=0).stop 2024-04-24T06:41:12.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.640+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 msgr2=0x7f8038099310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.640+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8030001370 tx=0x7f803000ccd0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 shutdown_connections 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f801806c920 0x7f801806ede0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80381525b0 0x7f8038099310 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8038152150 0x7f80381f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 --2- 172.21.15.59:0/2985533822 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f801c003a00 0x7f8038159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 >> 172.21.15.59:0/2985533822 conn(0x7f803806bdf0 msgr2=0x7f803815b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 shutdown_connections 2024-04-24T06:41:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.641+0000 7f803d400640 1 -- 172.21.15.59:0/2985533822 wait complete. 2024-04-24T06:41:12.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:12.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.870+0000 7f5dbf200640 1 Processor -- start 2024-04-24T06:41:12.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.870+0000 7f5dbf200640 1 -- start start 2024-04-24T06:41:12.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.871+0000 7f5dbf200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f5dc015c670 0x7f5dc00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.871+0000 7f5dbf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc00b4cf0 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.871+0000 7f5dbf200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc00ccdd0 con 0x7f5dc00b4230 2024-04-24T06:41:12.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.871+0000 7f5dbf200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc015ca50 con 0x7f5dc015c670 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.872+0000 7f5dbe800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5dc00d0cd0 0x7f5dc006ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47728/0 (socket says 172.21.15.59:47728) 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.872+0000 7f5dbe800640 1 -- 172.21.15.59:0/3801325005 learned_addr learned my addr 172.21.15.59:0/3801325005 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.872+0000 7f5dbde00640 1 --2- 172.21.15.59:0/3801325005 >> v2:172.21.15.66:3300/0 conn(0x7f5dc015c670 0x7f5dc00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.872+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876945930 0 0) 0x7f5dc00b4cf0 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.872+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5da4003520 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333257540 0 0) 0x7f5da4003520 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 >> v2:172.21.15.66:3300/0 conn(0x7f5dc015c670 msgr2=0x7f5dc00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 --2- 172.21.15.59:0/3801325005 >> v2:172.21.15.66:3300/0 conn(0x7f5dc015c670 0x7f5dc00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 >> v1:172.21.15.66:6789/0 conn(0x7f5dc00b4230 legacy=0x7f5dc00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00cd0b0 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5db8001670 con 0x7f5dc00d0cd0 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 >> v1:172.21.15.59:6789/0 conn(0x7f5dc00d0cd0 legacy=0x7f5dc006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5da40039b0 0x7f5da4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da4004800 0x7f5da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 0x7f5da400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5da400c0e0 con 0x7f5da40051c0 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5da400c290 con 0x7f5da4004800 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5da400c490 con 0x7f5da40039b0 2024-04-24T06:41:12.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.873+0000 7f5dbe800640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 0x7f5da400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5db5400640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da4004800 0x7f5da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbde00640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5da40039b0 0x7f5da4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbe800640 1 -- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5da40039b0 msgr2=0x7f5da4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbe800640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5da40039b0 0x7f5da4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbe800640 1 -- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da4004800 msgr2=0x7f5da4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbe800640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da4004800 0x7f5da4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.874+0000 7f5dbe800640 1 -- 172.21.15.59:0/3801325005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da4003520 con 0x7f5da40051c0 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.875+0000 7f5dbe800640 1 -- 172.21.15.59:0/3801325005 reap_dead start 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.875+0000 7f5dbe800640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 0x7f5da400b9c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c240 tx=0x7f5da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6831ce269ae4fdc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8007cf0 con 0x7f5da40051c0 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db80058d0 con 0x7f5da40051c0 2024-04-24T06:41:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbd400640 1 -- 172.21.15.59:0/3801325005 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8005ce0 con 0x7f5da40051c0 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbf200640 1 -- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 msgr2=0x7f5da400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbf200640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 0x7f5da400b9c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c240 tx=0x7f5da4005830 comp rx=0 tx=0).stop 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbf200640 1 -- 172.21.15.59:0/3801325005 shutdown_connections 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbf200640 1 --2- 172.21.15.59:0/3801325005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40051c0 0x7f5da400b9c0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.876+0000 7f5dbf200640 1 -- 172.21.15.59:0/3801325005 >> 172.21.15.59:0/3801325005 conn(0x7f5dc006bdf0 msgr2=0x7f5dc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.877+0000 7f5dbf200640 1 -- 172.21.15.59:0/3801325005 shutdown_connections 2024-04-24T06:41:12.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.877+0000 7f5dbf200640 1 -- 172.21.15.59:0/3801325005 wait complete. 2024-04-24T06:41:12.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.878+0000 7f5dbf200640 1 Processor -- start 2024-04-24T06:41:12.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 -- start start 2024-04-24T06:41:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da40051c0 0x7f5dc01f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5dc01552c0 0x7f5dc0155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00b3e60 con 0x7f5dc00b1f90 2024-04-24T06:41:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00ccb80 con 0x7f5da40051c0 2024-04-24T06:41:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.879+0000 7f5dbf200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00b3fe0 con 0x7f5dc01552c0 2024-04-24T06:41:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49706/0 (socket says 172.21.15.59:49706) 2024-04-24T06:41:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 -- 172.21.15.59:0/2370386935 learned_addr learned my addr 172.21.15.59:0/2370386935 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5db5400640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5dc01552c0 0x7f5dc0155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbde00640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da40051c0 0x7f5dc01f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 -- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5dc01552c0 msgr2=0x7f5dc0155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5dc01552c0 0x7f5dc0155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 -- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da40051c0 msgr2=0x7f5dc01f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da40051c0 0x7f5dc01f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.880+0000 7f5dbe800640 1 -- 172.21.15.59:0/2370386935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00b2980 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5dbe800640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c360 tx=0x7f5dc00b4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8038040 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db8041070 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80049e0 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc0155c80 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.881+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01561f0 con 0x7f5dc00b1f90 2024-04-24T06:41:12.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.882+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc004fd90 con 0x7f5dc00b1f90 2024-04-24T06:41:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.883+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5db8004b80 con 0x7f5dc00b1f90 2024-04-24T06:41:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.884+0000 7f5db6a00640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 0x7f5d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.884+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f5db80ab730 con 0x7f5dc00b1f90 2024-04-24T06:41:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.885+0000 7f5dbde00640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 0x7f5d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.885+0000 7f5dbde00640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 0x7f5d9c06edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5dc009a220 tx=0x7f5db0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:12.889+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5db8079b50 con 0x7f5dc00b1f90 2024-04-24T06:41:13.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.105+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5dc00cc460 con 0x7f5dc00b1f90 2024-04-24T06:41:13.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.106+0000 7f5db6a00640 1 -- 172.21.15.59:0/2370386935 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7f5db8079450 con 0x7f5dc00b1f90 2024-04-24T06:41:13.107 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-24T06:41:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.108+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 msgr2=0x7f5d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.108+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 0x7f5d9c06edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5dc009a220 tx=0x7f5db0008040 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.108+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 msgr2=0x7f5dc00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.108+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c360 tx=0x7f5dc00b4ec0 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 shutdown_connections 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5d9c06c8f0 0x7f5d9c06edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc00b1f90 0x7f5dc00b2440 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5dc01552c0 0x7f5dc0155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 --2- 172.21.15.59:0/2370386935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da40051c0 0x7f5dc01f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 >> 172.21.15.59:0/2370386935 conn(0x7f5dc006bdf0 msgr2=0x7f5dc015b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 shutdown_connections 2024-04-24T06:41:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:13.109+0000 7f5dbf200640 1 -- 172.21.15.59:0/2370386935 wait complete. 2024-04-24T06:41:13.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:41:13.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:13.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:13.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76159]: fuse finished with error 0 and tester_r 0 2024-04-24T06:41:13.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:41:19.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:19.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:19.323 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:41:19.385 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:19.385 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:19.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:41:19.442 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi059.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:19.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:19.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:19.443 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:41:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-24T06:41:19.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:41:19.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:41:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.720+0000 7f1614800640 1 Processor -- start 2024-04-24T06:41:19.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.721+0000 7f1614800640 1 -- start start 2024-04-24T06:41:19.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.722+0000 7f1614800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c0b44c0 0x7f160c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.722+0000 7f1614800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f160c162de0 0x7f160c06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.722+0000 7f1614800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160c1632e0 con 0x7f160c161e60 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.722+0000 7f1614800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f160c0aefd0 con 0x7f160c162de0 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.722+0000 7f1614800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0adcd0 con 0x7f160c0b44c0 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1612800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f160c162de0 0x7f160c06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1612800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f160c162de0 0x7f160c06ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56226/0 (socket says 172.21.15.59:56226) 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1612800640 1 -- 172.21.15.59:0/3892730956 learned_addr learned my addr 172.21.15.59:0/3892730956 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1611400640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c0b44c0 0x7f160c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1612800640 1 -- 172.21.15.59:0/3892730956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c0b44c0 msgr2=0x7f160c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.723+0000 7f1612800640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c0b44c0 0x7f160c0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.724+0000 7f1612800640 1 -- 172.21.15.59:0/3892730956 >> v1:172.21.15.66:6789/0 conn(0x7f160c161e60 legacy=0x7f160c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.724+0000 7f1612800640 1 -- 172.21.15.59:0/3892730956 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c0adfb0 con 0x7f160c162de0 2024-04-24T06:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.725+0000 7f1612800640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c162de0 0x7f160c06ddf0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471e849cacafba49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.725+0000 7f1610a00640 1 -- 172.21.15.59:0/3892730956 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608036070 con 0x7f160c162de0 2024-04-24T06:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.725+0000 7f1610a00640 1 -- 172.21.15.59:0/3892730956 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1608031600 con 0x7f160c162de0 2024-04-24T06:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.725+0000 7f1610a00640 1 -- 172.21.15.59:0/3892730956 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080319d0 con 0x7f160c162de0 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 -- 172.21.15.59:0/3892730956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c162de0 msgr2=0x7f160c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c162de0 0x7f160c06ddf0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1608009e50 tx=0x7f160802a680 comp rx=0 tx=0).stop 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 -- 172.21.15.59:0/3892730956 shutdown_connections 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c0b44c0 0x7f160c0aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 --2- 172.21.15.59:0/3892730956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c162de0 0x7f160c06ddf0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 -- 172.21.15.59:0/3892730956 >> 172.21.15.59:0/3892730956 conn(0x7f160c0b2130 msgr2=0x7f160c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 -- 172.21.15.59:0/3892730956 shutdown_connections 2024-04-24T06:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.726+0000 7f1614800640 1 -- 172.21.15.59:0/3892730956 wait complete. 2024-04-24T06:41:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.728+0000 7f1614800640 1 Processor -- start 2024-04-24T06:41:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.728+0000 7f1614800640 1 -- start start 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c0b44c0 0x7f160c149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c162de0 0x7f160c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0af480 con 0x7f160c161e60 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0ae570 con 0x7f160c0b44c0 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1614800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0af600 con 0x7f160c162de0 2024-04-24T06:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56240/0 (socket says 172.21.15.59:56240) 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611400640 1 -- 172.21.15.59:0/1488910079 learned_addr learned my addr 172.21.15.59:0/1488910079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1612800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c162de0 0x7f160c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611400640 1 -- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c162de0 msgr2=0x7f160c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611e00640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c0b44c0 0x7f160c149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.729+0000 7f1611400640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c162de0 0x7f160c15c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1611400640 1 -- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c0b44c0 msgr2=0x7f160c149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1611400640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c0b44c0 0x7f160c149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1611400640 1 -- 172.21.15.59:0/1488910079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c158950 con 0x7f160c161e60 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1611400640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00d5d0 tx=0x7f15fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0125f0 con 0x7f160c161e60 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc019070 con 0x7f160c161e60 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c158c30 con 0x7f160c161e60 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0129f0 con 0x7f160c161e60 2024-04-24T06:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.730+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1591f0 con 0x7f160c161e60 2024-04-24T06:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.731+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c068a80 con 0x7f160c161e60 2024-04-24T06:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.732+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15fc012b90 con 0x7f160c161e60 2024-04-24T06:41:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.733+0000 7f1606000640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 0x7f15f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.734+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f15fc08bf60 con 0x7f160c161e60 2024-04-24T06:41:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.734+0000 7f1611e00640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 0x7f15f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:19.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.735+0000 7f1611e00640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 0x7f15f006ede0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1600001500 tx=0x7f1600009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:19.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.738+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15fc05a380 con 0x7f160c161e60 2024-04-24T06:41:19.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.987+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f160c1585a0 con 0x7f160c161e60 2024-04-24T06:41:19.990 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:41:19.988+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:41:19.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.991+0000 7f1606000640 1 -- 172.21.15.59:0/1488910079 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f15fc059c80 con 0x7f160c161e60 2024-04-24T06:41:19.993 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 msgr2=0x7f15f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 0x7f15f006ede0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1600001500 tx=0x7f1600009040 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 msgr2=0x7f160c160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00d5d0 tx=0x7f15fc00da90 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 shutdown_connections 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f15f006c920 0x7f15f006ede0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f160c162de0 0x7f160c15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f160c161e60 0x7f160c160580 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.995+0000 7f1614800640 1 --2- 172.21.15.59:0/1488910079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f160c0b44c0 0x7f160c149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.996+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 >> 172.21.15.59:0/1488910079 conn(0x7f160c0b2130 msgr2=0x7f160c06eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:19.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.996+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 shutdown_connections 2024-04-24T06:41:19.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:19.996+0000 7f1614800640 1 -- 172.21.15.59:0/1488910079 wait complete. 2024-04-24T06:41:20.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:20.025 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:20.038 INFO:tasks.ceph.mds.b.smithi066.stderr: -8115> 2024-04-24T06:40:54.042+0000 7fa229800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:41:20.132 INFO:tasks.ceph.mds.b.smithi066.stdout:starting mds.b at 2024-04-24T06:41:20.169 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-24T06:41:20.170 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:20.171 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-24T06:41:20.172 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:20.173 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.175 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:20.176 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:20.177 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:20.178 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-24T06:41:20.179 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-24T06:41:20.180 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = False 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:20.181 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:20.182 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:20.183 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.184 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:20.185 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-24T06:41:20.186 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-04-24T06:41:20.187 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:20.188 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.189 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.190 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-24T06:41:20.191 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-24T06:41:20.192 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-24T06:41:20.193 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-04-24T06:41:20.194 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-04-24T06:41:20.195 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-04-24T06:41:20.195 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = False 2024-04-24T06:41:20.195 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:20.195 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.195 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:20.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:20.349 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-24T06:41:20.349 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-24T06:41:20.349 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:20.349 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.350 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.351 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-24T06:41:20.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-04-24T06:41:20.353 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:20.354 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-04-24T06:41:20.355 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-04-24T06:41:20.356 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-24T06:41:20.357 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:20.358 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:20.359 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:20.360 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:20.361 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = False 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = False 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-24T06:41:20.362 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.363 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:20.364 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-24T06:41:20.365 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.366 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-24T06:41:20.367 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-24T06:41:20.368 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:20.369 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-24T06:41:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:20.371 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = False 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = False 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:20.386 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-24T06:41:20.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.498+0000 7f32fa1e0f40 1 Processor -- start 2024-04-24T06:41:20.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.499+0000 7f32fa1e0f40 1 -- start start 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.499+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.499+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d4e7850 0x55b59d4e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b59d466750 con 0x55b59d44b500 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b59d3c2cc0 con 0x55b59d4e7470 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b59d4f15d0 con 0x55b59d4e7850 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56258/0 (socket says 172.21.15.59:56258) 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f7400640 1 -- 172.21.15.59:0/2699936824 learned_addr learned my addr 172.21.15.59:0/2699936824 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f7e00640 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d4e7850 0x55b59d4e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f6a00640 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.500+0000 7f32f7400640 1 -- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d4e7850 msgr2=0x55b59d4e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f7400640 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d4e7850 0x55b59d4e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f7400640 1 -- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 msgr2=0x55b59d4df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f7400640 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f7400640 1 -- 172.21.15.59:0/2699936824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59d4dec50 con 0x55b59d44b500 2024-04-24T06:41:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f7400640 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f32e800a500 tx=0x7f32e800a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668da2609f0a02d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f5600640 1 -- 172.21.15.59:0/2699936824 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e80115f0 con 0x55b59d44b500 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f5600640 1 -- 172.21.15.59:0/2699936824 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e8018070 con 0x55b59d44b500 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.501+0000 7f32f5600640 1 -- 172.21.15.59:0/2699936824 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e80119b0 con 0x55b59d44b500 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/2699936824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 msgr2=0x55b59d4ddb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f32e800a500 tx=0x7f32e800a9c0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/2699936824 shutdown_connections 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d4e7850 0x55b59d4e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/2699936824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d44b500 0x55b59d4ddb50 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/2699936824 >> 172.21.15.59:0/2699936824 conn(0x55b59d467290 msgr2=0x55b59d4e5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/2699936824 shutdown_connections 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.502+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/2699936824 wait complete. 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.503+0000 7f32fa1e0f40 1 Processor -- start 2024-04-24T06:41:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.503+0000 7f32fa1e0f40 1 -- start start 2024-04-24T06:41:20.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d44b500 0x55b59d4e3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 0x55b59d4d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b59d447130 con 0x55b59d4e7850 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b59d447570 con 0x55b59d4e7470 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32fa1e0f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55b59d44ade0 con 0x55b59d44b500 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 0x55b59d4d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 0x55b59d4d9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56266/0 (socket says 172.21.15.59:56266) 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32f7e00640 1 -- 172.21.15.59:0/3302698337 learned_addr learned my addr 172.21.15.59:0/3302698337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.504+0000 7f32f6a00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7400640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d44b500 0x55b59d4e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 -- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d44b500 msgr2=0x55b59d4e3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55b59d44b500 0x55b59d4e3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 -- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 msgr2=0x55b59d4d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b59d4e7470 0x55b59d4d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 -- 172.21.15.59:0/3302698337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59d41d9a0 con 0x55b59d4e7850 2024-04-24T06:41:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.505+0000 7f32f7e00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 0x55b59d4d9c30 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f32f0045ed0 tx=0x7f32f0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.506+0000 7f32efe00640 1 -- 172.21.15.59:0/3302698337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0054b70 con 0x55b59d4e7850 2024-04-24T06:41:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.506+0000 7f32efe00640 1 -- 172.21.15.59:0/3302698337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f0054d10 con 0x55b59d4e7850 2024-04-24T06:41:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.506+0000 7f32efe00640 1 -- 172.21.15.59:0/3302698337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0068650 con 0x55b59d4e7850 2024-04-24T06:41:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.506+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b59d41dc40 con 0x55b59d4e7850 2024-04-24T06:41:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.506+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59d4e1e80 con 0x55b59d4e7850 2024-04-24T06:41:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.508+0000 7f32efe00640 1 -- 172.21.15.59:0/3302698337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32f0071440 con 0x55b59d4e7850 2024-04-24T06:41:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.509+0000 7f32efe00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32d806b300 0x7f32d806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.509+0000 7f32efe00640 1 -- 172.21.15.59:0/3302698337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f32f00cf280 con 0x55b59d4e7850 2024-04-24T06:41:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.509+0000 7f32f7400640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32d806b300 0x7f32d806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.509+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55b59d4e3d70 0x55b59d4db9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.509+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55b59d4dbef0 con 0x55b59d4e3d70 2024-04-24T06:41:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.510+0000 7f32f6a00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55b59d4e3d70 0x55b59d4db9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.510+0000 7f32f7400640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32d806b300 0x7f32d806d7c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000f80 tx=0x7f32e8011000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.510+0000 7f32f6a00640 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55b59d4e3d70 0x55b59d4db9b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32f6a00640 1 -- 172.21.15.59:0/3302698337 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v111'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f32e400c6a0 con 0x55b59d4e3d70 2024-04-24T06:41:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55b59d4e3d70 msgr2=0x55b59d4db9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55b59d4e3d70 0x55b59d4db9b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32d806b300 msgr2=0x7f32d806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f32d806b300 0x7f32d806d7c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000f80 tx=0x7f32e8011000 comp rx=0 tx=0).stop 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 msgr2=0x55b59d4d9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 --2- 172.21.15.59:0/3302698337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b59d4e7850 0x55b59d4d9c30 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f32f0045ed0 tx=0x7f32f0002fb0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32f7e00640 1 -- 172.21.15.59:0/3302698337 reap_dead start 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 shutdown_connections 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 >> 172.21.15.59:0/3302698337 conn(0x55b59d467290 msgr2=0x55b59d4480d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 shutdown_connections 2024-04-24T06:41:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.512+0000 7f32fa1e0f40 1 -- 172.21.15.59:0/3302698337 wait complete. 2024-04-24T06:41:20.519 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:41:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.725+0000 7f0f6a200640 1 Processor -- start 2024-04-24T06:41:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.725+0000 7f0f6a200640 1 -- start start 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f6a200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 0x7f0f640d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f6a200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0f640d13e0 0x7f0f640d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f6a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640b5da0 con 0x7f0f640b3600 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f6a200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6415f100 con 0x7f0f640d13e0 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f6a200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0f640d07d0 con 0x7f0f6415ed20 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f69800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0f640d13e0 0x7f0f640d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f63e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0f640b3600 0x7f0f640b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44240/0 (socket says 172.21.15.59:44240) 2024-04-24T06:41:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.727+0000 7f0f63e00640 1 -- 172.21.15.59:0/2407558789 learned_addr learned my addr 172.21.15.59:0/2407558789 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f68e00640 1 --2- 172.21.15.59:0/2407558789 >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 0x7f0f640d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f69800640 1 -- 172.21.15.59:0/2407558789 >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 msgr2=0x7f0f640d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f69800640 1 --2- 172.21.15.59:0/2407558789 >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 0x7f0f640d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f69800640 1 -- 172.21.15.59:0/2407558789 >> v1:172.21.15.59:6789/0 conn(0x7f0f640b3600 legacy=0x7f0f640b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f63400640 1 -- 172.21.15.59:0/2407558789 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031563922 0 0) 0x7f0f640b5da0 con 0x7f0f640b3600 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f69800640 1 -- 172.21.15.59:0/2407558789 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f640d0a20 con 0x7f0f640d13e0 2024-04-24T06:41:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.728+0000 7f0f68e00640 1 --2- 172.21.15.59:0/2407558789 >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 0x7f0f640d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.729+0000 7f0f69800640 1 --2- 172.21.15.59:0/2407558789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640d13e0 0x7f0f640d0290 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f6408e2a0 tx=0x7f0f5c02a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca8f90480491e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.729+0000 7f0f63400640 1 -- 172.21.15.59:0/2407558789 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c007740 con 0x7f0f640d13e0 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.729+0000 7f0f63400640 1 -- 172.21.15.59:0/2407558789 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5c0078e0 con 0x7f0f640d13e0 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.729+0000 7f0f63400640 1 -- 172.21.15.59:0/2407558789 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c007cb0 con 0x7f0f640d13e0 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 -- 172.21.15.59:0/2407558789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640d13e0 msgr2=0x7f0f640d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2407558789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640d13e0 0x7f0f640d0290 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f6408e2a0 tx=0x7f0f5c02a550 comp rx=0 tx=0).stop 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 -- 172.21.15.59:0/2407558789 shutdown_connections 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2407558789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640d13e0 0x7f0f640d0290 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2407558789 >> v2:172.21.15.66:3300/0 conn(0x7f0f6415ed20 0x7f0f640d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 -- 172.21.15.59:0/2407558789 >> 172.21.15.59:0/2407558789 conn(0x7f0f6408bf90 msgr2=0x7f0f640d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:20.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 -- 172.21.15.59:0/2407558789 shutdown_connections 2024-04-24T06:41:20.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.730+0000 7f0f6a200640 1 -- 172.21.15.59:0/2407558789 wait complete. 2024-04-24T06:41:20.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.732+0000 7f0f6a200640 1 Processor -- start 2024-04-24T06:41:20.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.732+0000 7f0f6a200640 1 -- start start 2024-04-24T06:41:20.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f640d13e0 0x7f0f640ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f6415ed20 0x7f0f64158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640d3490 con 0x7f0f640b3600 2024-04-24T06:41:20.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640d31b0 con 0x7f0f6415ed20 2024-04-24T06:41:20.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.733+0000 7f0f6a200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640b4620 con 0x7f0f640d13e0 2024-04-24T06:41:20.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56280/0 (socket says 172.21.15.59:56280) 2024-04-24T06:41:20.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 -- 172.21.15.59:0/2478562230 learned_addr learned my addr 172.21.15.59:0/2478562230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 -- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f640d13e0 msgr2=0x7f0f640ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f640d13e0 0x7f0f640ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 -- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f6415ed20 msgr2=0x7f0f64158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f6415ed20 0x7f0f64158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 -- 172.21.15.59:0/2478562230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f640ca930 con 0x7f0f640b3600 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.734+0000 7f0f68e00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c820 tx=0x7f0f4c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.735+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c015070 con 0x7f0f640b3600 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.735+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c007a30 con 0x7f0f640b3600 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.735+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c007de0 con 0x7f0f640b3600 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.735+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f64159100 con 0x7f0f640b3600 2024-04-24T06:41:20.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.735+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f640c8190 con 0x7f0f640b3600 2024-04-24T06:41:20.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.736+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6404fd90 con 0x7f0f640b3600 2024-04-24T06:41:20.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.737+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f4c018070 con 0x7f0f640b3600 2024-04-24T06:41:20.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.739+0000 7f0f60c00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 0x7f0f4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.739+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f0f4c08ba20 con 0x7f0f640b3600 2024-04-24T06:41:20.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.739+0000 7f0f63e00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 0x7f0f4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.740+0000 7f0f63e00640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 0x7f0f4006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007fd0 tx=0x7f0f54007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.744+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f4c059de0 con 0x7f0f640b3600 2024-04-24T06:41:20.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:20.997+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f0f64074ef0 con 0x7f0f640b3600 2024-04-24T06:41:22.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.011+0000 7f0f60c00640 1 -- 172.21.15.59:0/2478562230 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) ==== 162+0+0 (secure 0 0 0) 0x7f0f4c0596e0 con 0x7f0f640b3600 2024-04-24T06:41:22.014 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:41:22.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.015+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 msgr2=0x7f0f4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.015+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 0x7f0f4006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007fd0 tx=0x7f0f54007920 comp rx=0 tx=0).stop 2024-04-24T06:41:22.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.015+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 msgr2=0x7f0f641f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.015+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c820 tx=0x7f0f4c00cce0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 shutdown_connections 2024-04-24T06:41:22.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0f4006c920 0x7f0f4006ede0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f6415ed20 0x7f0f64158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0f640d13e0 0x7f0f640ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 --2- 172.21.15.59:0/2478562230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f640b3600 0x7f0f641f80d0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 >> 172.21.15.59:0/2478562230 conn(0x7f0f6408bf90 msgr2=0x7f0f6415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:22.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 shutdown_connections 2024-04-24T06:41:22.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.016+0000 7f0f6a200640 1 -- 172.21.15.59:0/2478562230 wait complete. 2024-04-24T06:41:22.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:22.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.245+0000 7f924de00640 1 Processor -- start 2024-04-24T06:41:22.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.246+0000 7f924de00640 1 -- start start 2024-04-24T06:41:22.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.247+0000 7f924de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.247+0000 7f924de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 0x7f92480d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.247+0000 7f924de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92480cf030 con 0x7f924815e7c0 2024-04-24T06:41:22.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.247+0000 7f924de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f92480b7000 con 0x7f92480d0d90 2024-04-24T06:41:22.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.247+0000 7f924de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92480cf620 con 0x7f92480b5ff0 2024-04-24T06:41:22.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.248+0000 7f9247e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.248+0000 7f924d400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 0x7f92480d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.248+0000 7f9247e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:40526/0 (socket says 172.21.15.59:40526) 2024-04-24T06:41:22.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.248+0000 7f9247e00640 1 -- 172.21.15.59:0/3701658256 learned_addr learned my addr 172.21.15.59:0/3701658256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.249+0000 7f9247e00640 1 -- 172.21.15.59:0/3701658256 >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 msgr2=0x7f92480d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.249+0000 7f9247e00640 1 --2- 172.21.15.59:0/3701658256 >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 0x7f92480d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.249+0000 7f9247e00640 1 -- 172.21.15.59:0/3701658256 >> v1:172.21.15.66:6789/0 conn(0x7f924815e7c0 legacy=0x7f92480d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.249+0000 7f9247e00640 1 -- 172.21.15.59:0/3701658256 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480b6a40 con 0x7f92480b5ff0 2024-04-24T06:41:22.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.249+0000 7f924d400640 1 --2- 172.21.15.59:0/3701658256 >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 0x7f92480d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:22.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.250+0000 7f9247e00640 1 --2- 172.21.15.59:0/3701658256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9234009870 tx=0x7f923402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b0ea55c30f7ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:22.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.251+0000 7f9247400640 1 -- 172.21.15.59:0/3701658256 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234037070 con 0x7f92480b5ff0 2024-04-24T06:41:22.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.251+0000 7f9247400640 1 -- 172.21.15.59:0/3701658256 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234032440 con 0x7f92480b5ff0 2024-04-24T06:41:22.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.251+0000 7f9247400640 1 -- 172.21.15.59:0/3701658256 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234032810 con 0x7f92480b5ff0 2024-04-24T06:41:22.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 -- 172.21.15.59:0/3701658256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 msgr2=0x7f92480b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 --2- 172.21.15.59:0/3701658256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9234009870 tx=0x7f923402a250 comp rx=0 tx=0).stop 2024-04-24T06:41:22.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 -- 172.21.15.59:0/3701658256 shutdown_connections 2024-04-24T06:41:22.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 --2- 172.21.15.59:0/3701658256 >> v2:172.21.15.66:3300/0 conn(0x7f92480d0d90 0x7f92480d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 --2- 172.21.15.59:0/3701658256 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f92480b6450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 -- 172.21.15.59:0/3701658256 >> 172.21.15.59:0/3701658256 conn(0x7f924809d920 msgr2=0x7f924808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:22.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 -- 172.21.15.59:0/3701658256 shutdown_connections 2024-04-24T06:41:22.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.252+0000 7f924de00640 1 -- 172.21.15.59:0/3701658256 wait complete. 2024-04-24T06:41:22.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.254+0000 7f924de00640 1 Processor -- start 2024-04-24T06:41:22.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.254+0000 7f924de00640 1 -- start start 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f924814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92480d0d90 0x7f92480c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f92480d2800 con 0x7f924815e7c0 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92480cf2b0 con 0x7f92480d0d90 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.255+0000 7f924de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f92480d1a80 con 0x7f92480b5ff0 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56296/0 (socket says 172.21.15.59:56296) 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 -- 172.21.15.59:0/1914583752 learned_addr learned my addr 172.21.15.59:0/1914583752 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924ca00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f924814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f9247e00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92480d0d90 0x7f92480c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 -- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 msgr2=0x7f924814f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f924814f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 -- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92480d0d90 msgr2=0x7f92480c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92480d0d90 0x7f92480c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924d400640 1 -- 172.21.15.59:0/1914583752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92480b3360 con 0x7f924815e7c0 2024-04-24T06:41:22.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.256+0000 7f924ca00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f924814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f924d400640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f924000efc0 tx=0x7f9240009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240018030 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924000cde0 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240003bb0 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92480b1340 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.257+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92480b1900 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.258+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924804fd90 con 0x7f924815e7c0 2024-04-24T06:41:22.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.259+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9240005d10 con 0x7f924815e7c0 2024-04-24T06:41:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.260+0000 7f9244c00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.260+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f924008c6b0 con 0x7f924815e7c0 2024-04-24T06:41:22.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.260+0000 7f924ca00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.261+0000 7f924ca00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9238007fd0 tx=0x7f9238009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:22.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.264+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9240022070 con 0x7f924815e7c0 2024-04-24T06:41:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.509+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92480d1eb0 con 0x7f924815e7c0 2024-04-24T06:41:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.510+0000 7f9244c00640 1 -- 172.21.15.59:0/1914583752 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5173 (secure 0 0 0) 0x7f9240010040 con 0x7f924815e7c0 2024-04-24T06:41:22.512 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:22.512 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:22.017983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"a","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-24T06:41:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 104 2024-04-24T06:41:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 msgr2=0x7f922806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9238007fd0 tx=0x7f9238009040 comp rx=0 tx=0).stop 2024-04-24T06:41:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 msgr2=0x7f92480cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f924000efc0 tx=0x7f9240009f90 comp rx=0 tx=0).stop 2024-04-24T06:41:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 shutdown_connections 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f922806c8f0 0x7f922806edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924815e7c0 0x7f92480cdb60 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92480d0d90 0x7f92480c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 --2- 172.21.15.59:0/1914583752 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f92480b5ff0 0x7f924814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 >> 172.21.15.59:0/1914583752 conn(0x7f924809d920 msgr2=0x7f92480d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 shutdown_connections 2024-04-24T06:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:22.513+0000 7f924de00640 1 -- 172.21.15.59:0/1914583752 wait complete. 2024-04-24T06:41:22.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:22.017983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9222}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9222': {'gid': 9222, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/3446399276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3446399276}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3446399276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:41:22.939 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9222:up:replay 2024-04-24T06:41:23.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:24.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.163+0000 7f60c0e00640 1 Processor -- start 2024-04-24T06:41:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.163+0000 7f60c0e00640 1 -- start start 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60c0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60c0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60bc0d0d90 0x7f60bc0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60c0e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60bc0cf030 con 0x7f60bc15e7c0 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60c0e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0b7000 con 0x7f60bc0d0d90 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60c0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0cf620 con 0x7f60bc0b5ff0 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.164+0000 7f60bbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60bc0d0d90 0x7f60bc0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f60bc0d0d90 0x7f60bc0d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56310/0 (socket says 172.21.15.59:56310) 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 -- 172.21.15.59:0/665390875 learned_addr learned my addr 172.21.15.59:0/665390875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60baa00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 -- 172.21.15.59:0/665390875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 msgr2=0x7f60bc0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 -- 172.21.15.59:0/665390875 >> v1:172.21.15.66:6789/0 conn(0x7f60bc15e7c0 legacy=0x7f60bc0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60bbe00640 1 -- 172.21.15.59:0/665390875 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0b6a40 con 0x7f60bc0d0d90 2024-04-24T06:41:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.165+0000 7f60baa00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc0b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.166+0000 7f60bbe00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0d1220 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009870 tx=0x7f60b402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331946bdb48264a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.166+0000 7f60ba000640 1 -- 172.21.15.59:0/665390875 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4037070 con 0x7f60bc0d0d90 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.166+0000 7f60ba000640 1 -- 172.21.15.59:0/665390875 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b4032440 con 0x7f60bc0d0d90 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.166+0000 7f60ba000640 1 -- 172.21.15.59:0/665390875 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4032810 con 0x7f60bc0d0d90 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 -- 172.21.15.59:0/665390875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 msgr2=0x7f60bc0d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0d1220 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009870 tx=0x7f60b402a200 comp rx=0 tx=0).stop 2024-04-24T06:41:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 -- 172.21.15.59:0/665390875 shutdown_connections 2024-04-24T06:41:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc0b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 --2- 172.21.15.59:0/665390875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0d1220 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 -- 172.21.15.59:0/665390875 >> 172.21.15.59:0/665390875 conn(0x7f60bc09d920 msgr2=0x7f60bc08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 -- 172.21.15.59:0/665390875 shutdown_connections 2024-04-24T06:41:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.167+0000 7f60c0e00640 1 -- 172.21.15.59:0/665390875 wait complete. 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.169+0000 7f60c0e00640 1 Processor -- start 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.169+0000 7f60c0e00640 1 -- start start 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60bc15e7c0 0x7f60bc0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0d2800 con 0x7f60bc0d0d90 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0cf2b0 con 0x7f60bc15e7c0 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60c0e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0d1a80 con 0x7f60bc0b5ff0 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60baa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56322/0 (socket says 172.21.15.59:56322) 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.170+0000 7f60baa00640 1 -- 172.21.15.59:0/2053806445 learned_addr learned my addr 172.21.15.59:0/2053806445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 -- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 msgr2=0x7f60bc1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60bb400640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60bbe00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60bc15e7c0 0x7f60bc0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc1f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 -- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60bc15e7c0 msgr2=0x7f60bc0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:24.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60bc15e7c0 0x7f60bc0cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 -- 172.21.15.59:0/2053806445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0c9210 con 0x7f60bc0d0d90 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60bb400640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60baa00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f60a400cf60 tx=0x7f60a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.171+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a401a070 con 0x7f60bc0d0d90 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.172+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a400d840 con 0x7f60bc0d0d90 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.172+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a400dc10 con 0x7f60bc0d0d90 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.172+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc0c94f0 con 0x7f60bc0d0d90 2024-04-24T06:41:24.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.172+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc0cc4f0 con 0x7f60bc0d0d90 2024-04-24T06:41:24.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.173+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc04fd90 con 0x7f60bc0d0d90 2024-04-24T06:41:24.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.174+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60a401d070 con 0x7f60bc0d0d90 2024-04-24T06:41:24.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.175+0000 7f60b3400640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 0x7f609806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:24.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.175+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f60a408c3c0 con 0x7f60bc0d0d90 2024-04-24T06:41:24.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.175+0000 7f60bb400640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 0x7f609806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:24.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.176+0000 7f60bb400640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 0x7f609806edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b2ff0 tx=0x7f60ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:24.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.180+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60a405a130 con 0x7f60bc0d0d90 2024-04-24T06:41:24.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.428+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60bc162e60 con 0x7f60bc0d0d90 2024-04-24T06:41:24.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.430+0000 7f60b3400640 1 -- 172.21.15.59:0/2053806445 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5174 (secure 0 0 0) 0x7f60a405e430 con 0x7f60bc0d0d90 2024-04-24T06:41:24.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:24.432 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:23.030369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"a","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-24T06:41:24.432 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 106 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.432+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 msgr2=0x7f609806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.432+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 0x7f609806edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b2ff0 tx=0x7f60ac008040 comp rx=0 tx=0).stop 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.432+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 msgr2=0x7f60bc0c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.432+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f60a400cf60 tx=0x7f60a400cf90 comp rx=0 tx=0).stop 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 shutdown_connections 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f609806c8f0 0x7f609806edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60bc15e7c0 0x7f60bc0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60bc0d0d90 0x7f60bc0c8a70 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 --2- 172.21.15.59:0/2053806445 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f60bc0b5ff0 0x7f60bc1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:24.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 >> 172.21.15.59:0/2053806445 conn(0x7f60bc09d920 msgr2=0x7f60bc0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:24.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 shutdown_connections 2024-04-24T06:41:24.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:24.433+0000 7f60c0e00640 1 -- 172.21.15.59:0/2053806445 wait complete. 2024-04-24T06:41:24.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:23.030369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9222}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9222': {'gid': 9222, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:rejoin', 'state_seq': 12, 'addr': '172.21.15.59:6837/3446399276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3446399276}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3446399276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:41:24.858 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9222:up:rejoin 2024-04-24T06:41:25.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:26.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.083+0000 7f4055000640 1 Processor -- start 2024-04-24T06:41:26.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.083+0000 7f4055000640 1 -- start start 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4055000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f405015ae90 0x7f40500cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4055000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f405006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4055000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f405015b270 con 0x7f405006f350 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4055000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f40500b3080 con 0x7f405015ae90 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4055000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40500b4930 con 0x7f405006b9d0 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4046a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f405006f350 0x7f4050071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44252/0 (socket says 172.21.15.59:44252) 2024-04-24T06:41:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f4046a00640 1 -- 172.21.15.59:0/4067951570 learned_addr learned my addr 172.21.15.59:0/4067951570 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:26.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f404fe00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f405006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.085+0000 7f404f400640 1 --2- 172.21.15.59:0/4067951570 >> v2:172.21.15.66:3300/0 conn(0x7f405015ae90 0x7f40500cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526775804 0 0) 0x7f405015b270 con 0x7f405006f350 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4034003520 con 0x7f405006f350 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3007441091 0 0) 0x7f4034003520 con 0x7f405006f350 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 msgr2=0x7f405006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f405006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 >> v2:172.21.15.66:3300/0 conn(0x7f405015ae90 msgr2=0x7f40500cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 --2- 172.21.15.59:0/4067951570 >> v2:172.21.15.66:3300/0 conn(0x7f405015ae90 0x7f40500cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40500b4c10 con 0x7f405006f350 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f4038001670 con 0x7f405006f350 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 >> v1:172.21.15.59:6789/0 conn(0x7f405006f350 legacy=0x7f4050071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40340039b0 0x7f4034003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4034004800 0x7f4034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f40340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b3080 con 0x7f4034004800 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b4930 con 0x7f40340039b0 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.086+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f403400b9c0 con 0x7f405006b9d0 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4034004800 0x7f4034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f404fe00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f40340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f404f400640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40340039b0 0x7f4034003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 -- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 msgr2=0x7f40340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f405006b9d0 0x7f40340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 -- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40340039b0 msgr2=0x7f4034003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40340039b0 0x7f4034003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.087+0000 7f4046a00640 1 -- 172.21.15.59:0/4067951570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4034003520 con 0x7f4034004800 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.088+0000 7f4046a00640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4034004800 0x7f4034004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f403800c160 tx=0x7f4034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2eda6eb353584b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038007c80 con 0x7f4034004800 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038005ad0 con 0x7f4034004800 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f404ea00640 1 -- 172.21.15.59:0/4067951570 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038007070 con 0x7f4034004800 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 -- 172.21.15.59:0/4067951570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4034004800 msgr2=0x7f4034004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 --2- 172.21.15.59:0/4067951570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4034004800 0x7f4034004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f403800c160 tx=0x7f4034005830 comp rx=0 tx=0).stop 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f404fe00640 1 -- 172.21.15.59:0/4067951570 reap_dead start 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 -- 172.21.15.59:0/4067951570 shutdown_connections 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 -- 172.21.15.59:0/4067951570 >> 172.21.15.59:0/4067951570 conn(0x7f40500b21a0 msgr2=0x7f40500b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 -- 172.21.15.59:0/4067951570 shutdown_connections 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.089+0000 7f4055000640 1 -- 172.21.15.59:0/4067951570 wait complete. 2024-04-24T06:41:26.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.091+0000 7f4055000640 1 Processor -- start 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.091+0000 7f4055000640 1 -- start start 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4034004800 0x7f405015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405015d2f0 0x7f40501eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40500722f0 con 0x7f40501e8230 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40500b32d0 con 0x7f405015d2f0 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.092+0000 7f4055000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f40500b45c0 con 0x7f4034004800 2024-04-24T06:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56332/0 (socket says 172.21.15.59:56332) 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 -- 172.21.15.59:0/836576801 learned_addr learned my addr 172.21.15.59:0/836576801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404f400640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4034004800 0x7f405015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f4046a00640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405015d2f0 0x7f40501eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 -- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4034004800 msgr2=0x7f405015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4034004800 0x7f405015cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 -- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405015d2f0 msgr2=0x7f40501eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405015d2f0 0x7f40501eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.093+0000 7f404fe00640 1 -- 172.21.15.59:0/836576801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405015a600 con 0x7f40501e8230 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f404fe00640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f40500cca50 tx=0x7f404800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404801c070 con 0x7f40501e8230 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4048017400 con 0x7f40501e8230 2024-04-24T06:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40480177f0 con 0x7f40501e8230 2024-04-24T06:41:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501f8a00 con 0x7f40501e8230 2024-04-24T06:41:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.094+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501f8fc0 con 0x7f40501e8230 2024-04-24T06:41:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.095+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405004fd90 con 0x7f40501e8230 2024-04-24T06:41:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.096+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f404801f070 con 0x7f40501e8230 2024-04-24T06:41:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.097+0000 7f4047e00640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 0x7f402c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.097+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f404808bc00 con 0x7f40501e8230 2024-04-24T06:41:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.097+0000 7f404f400640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 0x7f402c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.098+0000 7f404f400640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 0x7f402c06ede0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f404000bc10 tx=0x7f4040008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.102+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4048012070 con 0x7f40501e8230 2024-04-24T06:41:26.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.348+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40500af760 con 0x7f40501e8230 2024-04-24T06:41:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.349+0000 7f4047e00640 1 -- 172.21.15.59:0/836576801 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5181 (secure 0 0 0) 0x7f40480598e0 con 0x7f40501e8230 2024-04-24T06:41:26.351 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:26.352 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:25.036625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":8}]} 2024-04-24T06:41:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 107 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.352+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 msgr2=0x7f402c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.352+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 0x7f402c06ede0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f404000bc10 tx=0x7f4040008040 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.352+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 msgr2=0x7f40501eafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.352+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f40500cca50 tx=0x7f404800f4a0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 shutdown_connections 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f402c06c920 0x7f402c06ede0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40501e8230 0x7f40501eafc0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f405015d2f0 0x7f40501eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 --2- 172.21.15.59:0/836576801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4034004800 0x7f405015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 >> 172.21.15.59:0/836576801 conn(0x7f40500b21a0 msgr2=0x7f4050072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 shutdown_connections 2024-04-24T06:41:26.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.353+0000 7f4055000640 1 -- 172.21.15.59:0/836576801 wait complete. 2024-04-24T06:41:26.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:25.036625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9222}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9222': {'gid': 9222, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.59:6837/3446399276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3446399276}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3446399276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9222, 'qdb_cluster': [9222]} 2024-04-24T06:41:26.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:41:26.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T06:41:26.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.995+0000 7f0abb200640 1 Processor -- start 2024-04-24T06:41:26.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.995+0000 7f0abb200640 1 -- start start 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0abb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0abc0af4b0 0x7f0abc15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0abb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0af890 0x7f0abc144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0abb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0abc15f3f0 con 0x7f0abc0b3010 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0abb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0abc144f60 con 0x7f0abc0af4b0 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0abb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc15f6d0 con 0x7f0abc0af890 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0aba800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0abc0b3010 0x7f0abc0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44266/0 (socket says 172.21.15.59:44266) 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0aba800640 1 -- 172.21.15.59:0/521087594 learned_addr learned my addr 172.21.15.59:0/521087594 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.997+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/521087594 >> v2:172.21.15.59:3300/0 conn(0x7f0abc0af4b0 0x7f0abc15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0af890 0x7f0abc144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab9e00640 1 -- 172.21.15.59:0/521087594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0af890 msgr2=0x7f0abc144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0af890 0x7f0abc144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab9400640 1 -- 172.21.15.59:0/521087594 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13476709 0 0) 0x7f0abc15f3f0 con 0x7f0abc0b3010 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab9e00640 1 -- 172.21.15.59:0/521087594 >> v1:172.21.15.59:6789/0 conn(0x7f0abc0b3010 legacy=0x7f0abc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:26.998+0000 7f0ab9e00640 1 -- 172.21.15.59:0/521087594 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc15f9b0 con 0x7f0abc0af4b0 2024-04-24T06:41:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc15ed50 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0829d1e3c2b7d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0ab9400640 1 -- 172.21.15.59:0/521087594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac037070 con 0x7f0abc0af4b0 2024-04-24T06:41:27.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0ab9400640 1 -- 172.21.15.59:0/521087594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac032440 con 0x7f0abc0af4b0 2024-04-24T06:41:27.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0ab9400640 1 -- 172.21.15.59:0/521087594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac032810 con 0x7f0abc0af4b0 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0abb200640 1 -- 172.21.15.59:0/521087594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 msgr2=0x7f0abc15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.000+0000 7f0abb200640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc15ed50 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02a180 comp rx=0 tx=0).stop 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 -- 172.21.15.59:0/521087594 shutdown_connections 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0af890 0x7f0abc144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 --2- 172.21.15.59:0/521087594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc15ed50 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 -- 172.21.15.59:0/521087594 >> 172.21.15.59:0/521087594 conn(0x7f0abc06bdf0 msgr2=0x7f0abc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 -- 172.21.15.59:0/521087594 shutdown_connections 2024-04-24T06:41:27.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.001+0000 7f0abb200640 1 -- 172.21.15.59:0/521087594 wait complete. 2024-04-24T06:41:27.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.003+0000 7f0abb200640 1 Processor -- start 2024-04-24T06:41:27.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.003+0000 7f0abb200640 1 -- start start 2024-04-24T06:41:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0b3010 0x7f0abc15d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc0aeaf0 0x7f0abc145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc0d0e50 con 0x7f0abc0af4b0 2024-04-24T06:41:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc0d0cd0 con 0x7f0abc0aeaf0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0abb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc0b34c0 con 0x7f0abc0b3010 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0ab9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0ab9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56356/0 (socket says 172.21.15.59:56356) 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.004+0000 7f0ab9e00640 1 -- 172.21.15.59:0/3292812864 learned_addr learned my addr 172.21.15.59:0/3292812864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 -- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0b3010 msgr2=0x7f0abc15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0b3010 0x7f0abc15d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 -- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc0aeaf0 msgr2=0x7f0abc145c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc0aeaf0 0x7f0abc145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 -- 172.21.15.59:0/3292812864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc1f4bd0 con 0x7f0abc0af4b0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab2000640 1 -- 172.21.15.59:0/3292812864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac037050 con 0x7f0abc0af4b0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab2000640 1 -- 172.21.15.59:0/3292812864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac038040 con 0x7f0abc0af4b0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0ab2000640 1 -- 172.21.15.59:0/3292812864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac004470 con 0x7f0abc0af4b0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.005+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc1f4eb0 con 0x7f0abc0af4b0 2024-04-24T06:41:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.006+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1f5470 con 0x7f0abc0af4b0 2024-04-24T06:41:27.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.009+0000 7f0ab2000640 1 -- 172.21.15.59:0/3292812864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0aac030900 con 0x7f0abc0af4b0 2024-04-24T06:41:27.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.009+0000 7f0ab2000640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 0x7f0a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.010+0000 7f0ab2000640 1 -- 172.21.15.59:0/3292812864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f0aac0ac750 con 0x7f0abc0af4b0 2024-04-24T06:41:27.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.010+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 0x7f0a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.010+0000 7f0ac3a33740 1 Processor -- start 2024-04-24T06:41:27.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 -- start start 2024-04-24T06:41:27.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 0x7f0a9806edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0abc15dc50 tx=0x7f0aa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950360 0x562ca1925170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ca188ab10 con 0x562ca196d980 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562ca18312b0 con 0x562ca1950360 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.011+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562ca1886ad0 con 0x562ca1950aa0 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56358/0 (socket says 172.21.15.59:56358) 2024-04-24T06:41:27.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 -- 172.21.15.59:0/1500654409 learned_addr learned my addr 172.21.15.59:0/1500654409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 -- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 msgr2=0x562ca194fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 -- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950360 msgr2=0x562ca1925170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950360 0x562ca1925170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 -- 172.21.15.59:0/1500654409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca159d2d0 con 0x562ca196d980 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.012+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0aab50 tx=0x7f0aac0767e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3d7133f0b41877 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0aaa600640 1 -- 172.21.15.59:0/1500654409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac07a410 con 0x562ca196d980 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0aaa600640 1 -- 172.21.15.59:0/1500654409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac07e7c0 con 0x562ca196d980 2024-04-24T06:41:27.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0aaa600640 1 -- 172.21.15.59:0/1500654409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac082ff0 con 0x562ca196d980 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 -- 172.21.15.59:0/1500654409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 msgr2=0x562ca1920310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0aab50 tx=0x7f0aac0767e0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 -- 172.21.15.59:0/1500654409 shutdown_connections 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950360 0x562ca1925170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/1500654409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca1920310 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.013+0000 7f0ac3a33740 1 -- 172.21.15.59:0/1500654409 >> 172.21.15.59:0/1500654409 conn(0x562ca1922580 msgr2=0x562ca196ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.014+0000 7f0ac3a33740 1 -- 172.21.15.59:0/1500654409 shutdown_connections 2024-04-24T06:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.014+0000 7f0ac3a33740 1 -- 172.21.15.59:0/1500654409 wait complete. 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.014+0000 7f0ac3a33740 1 -- start start 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.014+0000 7f0ac3a33740 1 Processor -- start 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca196d980 0x562ca1925740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ca188ab10 con 0x562ca1940660 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562ca1844590 con 0x562ca196d980 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.015+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562ca1924730 con 0x562ca1950aa0 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0ac3a33740 10 client.0 _ll_register_callbacks cb 0x562ca18273a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56372/0 (socket says 172.21.15.59:56372) 2024-04-24T06:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 -- 172.21.15.59:0/892052408 learned_addr learned my addr 172.21.15.59:0/892052408 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca196d980 0x562ca1925740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 -- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 msgr2=0x562ca194fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 -- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca196d980 msgr2=0x562ca1925740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca196d980 0x562ca1925740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.016+0000 7f0aba800640 1 -- 172.21.15.59:0/892052408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca16676b0 con 0x562ca1940660 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0aba800640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c190 tx=0x7f0ab400c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac98bc600c9686c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0a9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40070d0 con 0x562ca1940660 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f0ab4015b30 con 0x562ca1940660 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac07e7c0 con 0x562ca1940660 2024-04-24T06:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4015d30 con 0x562ca1940660 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0ac3a33740 10 client.9585 fetch_fsmap learned FSMap version 107 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.017+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x562ca1951ec0 con 0x562ca1940660 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac07e7c0 con 0x562ca1940660 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab401c050 con 0x562ca1940660 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 10 client.9585 fetch_fsmap finished waiting for FSMap version 107 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 10 client.9585 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 20 client.9585 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca194e480 0x562ca193e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 4 client.9585 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 --> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca1898530 con 0x562ca194e480 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.018+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca194e480 0x562ca193e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.019+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca194e480 0x562ca193e890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ee6209ff16be0 server_cookie=bf8e96b356583c11 in_seq=0 out_seq=0 2024-04-24T06:41:27.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.019+0000 7f0a9ea00640 10 client.9585 ms_handle_connect on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:27.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.019+0000 7f0a9ea00640 1 -- 172.21.15.59:0/892052408 <== mds.0 v2:172.21.15.59:6835/3446399276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ca1898530 con 0x562ca194e480 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.019+0000 7f0a9ea00640 10 client.9585 handle_command_reply: tid=0 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 client.9585 shutdown 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 msgr2=0x562ca1940ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c190 tx=0x7f0ab400c650 comp rx=0 tx=0).stop 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 shutdown_connections 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca194e480 0x562ca193e890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca1940660 0x562ca1940ae0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca196d980 0x562ca1925740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/892052408 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca194fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 >> 172.21.15.59:0/892052408 conn(0x562ca1950360 msgr2=0x562ca196d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0a9ea00640 0 client.9585 ms_handle_reset on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 shutdown_connections 2024-04-24T06:41:27.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 1 -- 172.21.15.59:0/892052408 wait complete. 2024-04-24T06:41:27.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.020+0000 7f0ac3a33740 20 client.9585 trim_cache size 0 max 16384 2024-04-24T06:41:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.040+0000 7f0ac3a33740 1 Processor -- start 2024-04-24T06:41:27.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 -- start start 2024-04-24T06:41:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1940660 0x562ca1923b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca1924060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc0ae140 con 0x562ca196d980 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc06f780 con 0x562ca1940660 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.041+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc0adaf0 con 0x562ca1950aa0 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56388/0 (socket says 172.21.15.59:56388) 2024-04-24T06:41:27.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 -- 172.21.15.59:0/377555802 learned_addr learned my addr 172.21.15.59:0/377555802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 -- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 msgr2=0x562ca1924060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca1924060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 -- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1940660 msgr2=0x562ca1923b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1940660 0x562ca1923b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.042+0000 7f0aba800640 1 -- 172.21.15.59:0/377555802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca1927560 con 0x562ca196d980 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0aba800640 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4015ef0 tx=0x7f0ab4006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79485a7615364cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0aa9c00640 1 -- 172.21.15.59:0/377555802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40072d0 con 0x562ca196d980 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0aa9c00640 1 -- 172.21.15.59:0/377555802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b070 con 0x562ca196d980 2024-04-24T06:41:27.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0aa9c00640 1 -- 172.21.15.59:0/377555802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4007670 con 0x562ca196d980 2024-04-24T06:41:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0ac3a33740 1 -- 172.21.15.59:0/377555802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 msgr2=0x562ca196ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.043+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4015ef0 tx=0x7f0ab4006000 comp rx=0 tx=0).stop 2024-04-24T06:41:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 -- 172.21.15.59:0/377555802 shutdown_connections 2024-04-24T06:41:27.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca196ac20 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1950aa0 0x562ca1924060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/377555802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1940660 0x562ca1923b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 -- 172.21.15.59:0/377555802 >> 172.21.15.59:0/377555802 conn(0x562ca1950360 msgr2=0x562ca194faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 -- 172.21.15.59:0/377555802 shutdown_connections 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 -- 172.21.15.59:0/377555802 wait complete. 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.044+0000 7f0ac3a33740 1 -- start start 2024-04-24T06:41:27.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.045+0000 7f0ac3a33740 1 Processor -- start 2024-04-24T06:41:27.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1940660 0x562ca196c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 0x562ca19e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ca194e300 con 0x562ca196d980 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562ca19521a0 con 0x562ca1950aa0 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x562ca194e950 con 0x562ca1940660 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ac3a33740 10 client.0 _ll_register_callbacks cb 0x7f0ab4001ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56402/0 (socket says 172.21.15.59:56402) 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 -- 172.21.15.59:0/3995953723 learned_addr learned my addr 172.21.15.59:0/3995953723 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 0x562ca19e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1940660 0x562ca196c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 -- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1940660 msgr2=0x562ca196c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1940660 0x562ca196c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 -- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 msgr2=0x562ca19e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 0x562ca19e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0aba800640 1 -- 172.21.15.59:0/3995953723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca1920840 con 0x562ca196d980 2024-04-24T06:41:27.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.046+0000 7f0ab3e00640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 0x562ca19e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0aba800640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007fd0 tx=0x7f0ab4001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f29d9b14ca33b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0a9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40074e0 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f0ab4007680 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab400b070 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4039050 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0ac3a33740 10 client.9591 fetch_fsmap learned FSMap version 107 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.047+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x562ca19e5bf0 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b070 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab403c020 con 0x562ca196d980 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 10 client.9591 fetch_fsmap finished waiting for FSMap version 107 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 10 client.9591 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 20 client.9591 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:27.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca198d9c0 0x562ca194aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 4 client.9591 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 --> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] -- command(tid 0: {"prefix": "status"}) -- 0x562ca18923a0 con 0x562ca198d9c0 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.048+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca198d9c0 0x562ca194aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ab9e00640 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca198d9c0 0x562ca194aa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7e20b77b372250 server_cookie=cb0b5d1781ef500e in_seq=0 out_seq=0 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0a9ea00640 10 client.9591 ms_handle_connect on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0a9ea00640 1 -- 172.21.15.59:0/3995953723 <== mds.0 v2:172.21.15.59:6835/3446399276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x562ca18923a0 con 0x562ca198d9c0 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0a9ea00640 10 client.9591 handle_command_reply: tid=0 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 client.9591 shutdown 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 msgr2=0x562ca19e5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007fd0 tx=0x7f0ab4001400 comp rx=0 tx=0).stop 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 shutdown_connections 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x562ca198d9c0 0x562ca194aa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ca196d980 0x562ca19e5590 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562ca1950aa0 0x562ca19e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 --2- 172.21.15.59:0/3995953723 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x562ca1940660 0x562ca196c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 >> 172.21.15.59:0/3995953723 conn(0x562ca1950360 msgr2=0x562ca191f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0a9ea00640 0 client.9591 ms_handle_reset on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.049+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 shutdown_connections 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0ac3a33740 1 -- 172.21.15.59:0/3995953723 wait complete. 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0ac3a33740 20 client.9591 trim_cache size 0 max 16384 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:27.055 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9222, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0282699170000003, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 107, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 111, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 111, 2024-04-24T06:41:27.056 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 44.336393495999999 2024-04-24T06:41:27.057 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:27.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 msgr2=0x7f0a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 0x7f0a9806edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0abc15dc50 tx=0x7f0aa4008040 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 msgr2=0x7f0abc0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac0046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 shutdown_connections 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0a9806c8f0 0x7f0a9806edb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc0aeaf0 0x7f0abc145c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0abc0b3010 0x7f0abc15d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 --2- 172.21.15.59:0/3292812864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0abc0af4b0 0x7f0abc0cd620 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.050+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 >> 172.21.15.59:0/3292812864 conn(0x7f0abc06bdf0 msgr2=0x7f0abc0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.051+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 shutdown_connections 2024-04-24T06:41:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.051+0000 7f0abb200640 1 -- 172.21.15.59:0/3292812864 wait complete. 2024-04-24T06:41:27.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:41:27.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.266+0000 7feba1600640 1 Processor -- start 2024-04-24T06:41:27.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.266+0000 7feba1600640 1 -- start start 2024-04-24T06:41:27.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7feb9c15e7c0 0x7feb9c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feb9c0d2630 0x7feb9c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c0b7000 con 0x7feb9c0d2630 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c0cf620 con 0x7feb9c15e7c0 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feba1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0cf030 con 0x7feb9c0b6530 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feb9b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feb9c0d2630 0x7feb9c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feb9b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feb9c0d2630 0x7feb9c0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56404/0 (socket says 172.21.15.59:56404) 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.268+0000 7feb9b400640 1 -- 172.21.15.59:0/797170448 learned_addr learned my addr 172.21.15.59:0/797170448 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.269+0000 7feb9b400640 1 -- 172.21.15.59:0/797170448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 msgr2=0x7feb9c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.269+0000 7feb9b400640 1 --2- 172.21.15.59:0/797170448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.269+0000 7feb9b400640 1 -- 172.21.15.59:0/797170448 >> v2:172.21.15.66:3300/0 conn(0x7feb9c15e7c0 msgr2=0x7feb9c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.269+0000 7feb9b400640 1 --2- 172.21.15.59:0/797170448 >> v2:172.21.15.66:3300/0 conn(0x7feb9c15e7c0 0x7feb9c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.269+0000 7feb9b400640 1 -- 172.21.15.59:0/797170448 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0b6a40 con 0x7feb9c0d2630 2024-04-24T06:41:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.270+0000 7feb9b400640 1 --2- 172.21.15.59:0/797170448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c0b5ff0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7feb9c08c6d0 tx=0x7feb8402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57584087e8899f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.270+0000 7feb9aa00640 1 -- 172.21.15.59:0/797170448 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84037070 con 0x7feb9c0d2630 2024-04-24T06:41:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.270+0000 7feb9aa00640 1 -- 172.21.15.59:0/797170448 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb84032440 con 0x7feb9c0d2630 2024-04-24T06:41:27.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.270+0000 7feb9aa00640 1 -- 172.21.15.59:0/797170448 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84032810 con 0x7feb9c0d2630 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 -- 172.21.15.59:0/797170448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 msgr2=0x7feb9c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 --2- 172.21.15.59:0/797170448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c0b5ff0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7feb9c08c6d0 tx=0x7feb8402a1d0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 -- 172.21.15.59:0/797170448 shutdown_connections 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 --2- 172.21.15.59:0/797170448 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 --2- 172.21.15.59:0/797170448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c0b5ff0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 --2- 172.21.15.59:0/797170448 >> v2:172.21.15.66:3300/0 conn(0x7feb9c15e7c0 0x7feb9c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 -- 172.21.15.59:0/797170448 >> 172.21.15.59:0/797170448 conn(0x7feb9c09d920 msgr2=0x7feb9c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 -- 172.21.15.59:0/797170448 shutdown_connections 2024-04-24T06:41:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.271+0000 7feba1600640 1 -- 172.21.15.59:0/797170448 wait complete. 2024-04-24T06:41:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.273+0000 7feba1600640 1 Processor -- start 2024-04-24T06:41:27.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.273+0000 7feba1600640 1 -- start start 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb9c15e7c0 0x7feb9c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0d1a80 con 0x7feb9c0d2630 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0cf2b0 con 0x7feb9c15e7c0 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feba1600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0d1c00 con 0x7feb9c0b6530 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feb9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feb9b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56412/0 (socket says 172.21.15.59:56412) 2024-04-24T06:41:27.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.274+0000 7feb9b400640 1 -- 172.21.15.59:0/1394459405 learned_addr learned my addr 172.21.15.59:0/1394459405 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 -- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 msgr2=0x7feb9c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 -- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb9c15e7c0 msgr2=0x7feb9c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feba0c00640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb9c15e7c0 0x7feb9c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb9c15e7c0 0x7feb9c14b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 -- 172.21.15.59:0/1394459405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0b11a0 con 0x7feb9c0d2630 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb9b400640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7feb8402a210 tx=0x7feb840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84041070 con 0x7feb9c0d2630 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8403c870 con 0x7feb9c0d2630 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84037040 con 0x7feb9c0d2630 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.275+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c0b3060 con 0x7feb9c0d2630 2024-04-24T06:41:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.276+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c0b1440 con 0x7feb9c0d2630 2024-04-24T06:41:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.277+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c04fd90 con 0x7feb9c0d2630 2024-04-24T06:41:27.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.278+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb84044070 con 0x7feb9c0d2630 2024-04-24T06:41:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.279+0000 7feb93e00640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 0x7feb7806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.279+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7feb840ac6d0 con 0x7feb9c0d2630 2024-04-24T06:41:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.279+0000 7feb9be00640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 0x7feb7806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.280+0000 7feb9be00640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 0x7feb7806ee30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ab90 tx=0x7feb8c009250 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.283+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb8407ab10 con 0x7feb9c0d2630 2024-04-24T06:41:27.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.530+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feb9c0d2d10 con 0x7feb9c0d2630 2024-04-24T06:41:27.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.531+0000 7feb93e00640 1 -- 172.21.15.59:0/1394459405 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7feb8407a410 con 0x7feb9c0d2630 2024-04-24T06:41:27.533 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 msgr2=0x7feb7806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 0x7feb7806ee30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ab90 tx=0x7feb8c009250 comp rx=0 tx=0).stop 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 msgr2=0x7feb9c1f8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7feb8402a210 tx=0x7feb840046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 shutdown_connections 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7feb7806c970 0x7feb7806ee30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb9c15e7c0 0x7feb9c14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb9c0d2630 0x7feb9c1f8210 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 --2- 172.21.15.59:0/1394459405 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7feb9c0b6530 0x7feb9c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 >> 172.21.15.59:0/1394459405 conn(0x7feb9c09d920 msgr2=0x7feb9c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 shutdown_connections 2024-04-24T06:41:27.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27.534+0000 7feba1600640 1 -- 172.21.15.59:0/1394459405 wait complete. 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:41:27.556 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:27.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:27.557 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:41:27.607 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:41:27.607 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:41:27.607 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:41:27.608 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41856sec preferred_lft 41856sec 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:41:27.609 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:27.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:27.610 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:41:27.691 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:41:27.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:27.696 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:41:27.751 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:41:27.751 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:41:27.751 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:41:27.752 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:41:27.752 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:41:27.753 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:41:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:41:27.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:27.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:27.977 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:41:28.028 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:41:28.028 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:41:28.029 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:28.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:28.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:41:28.107 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:41:28.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:28.107 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:28.156 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:28.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:28.157 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:28.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:28.339 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:28.400 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:28.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:28.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:28.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:28.403 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:28.455 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:41:28.455 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:41:28.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:28.532 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:28.575 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:28.575 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:28.575 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:28.575 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:28.575 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:28.576 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:28.577 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:28.578 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:28.578 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:28.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:28.579 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:28.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:41:28.643+0000 7f4623ad2480 -1 init, newargv = 0x5631aeaa38d0 newargc=15 2024-04-24T06:41:28.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77614]: starting ceph client 2024-04-24T06:41:28.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77614]: starting fuse 2024-04-24T06:41:28.663 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:41:28.664 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:41:28.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:41:28.664 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:41:28.664 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:41:28.664 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:41:28.665 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:41:28.666 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:28.667 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:41:28.667 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:28.860 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.77614.asok 2024-04-24T06:41:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:28.877 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77614.asok 2024-04-24T06:41:28.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:28.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77614.asok status 2024-04-24T06:41:29.041 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:41:29.039+0000 7f3a95200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "77614", 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:41:29.117 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9603, 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9603 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2130038545 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:29.118 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2130038545 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9603 192.168.0.1:0/2130038545", 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2130038545", 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 107, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 112, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:41:29.119 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:29.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:29.132 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:41:29.132 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:29.187 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:41:29.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:29.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:29.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:29.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:29.277 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-24T06:41:29.369 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-24T06:41:29.369 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-24T06:41:29.369 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-24T06:41:29.370 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-24T06:41:29.370 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-24T06:41:29.370 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed 2024-04-24T06:41:29.370 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed/jfile 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-24T06:41:29.372 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:29.517 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:41:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-24T06:41:29.532 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:29.726 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:41:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-24T06:41:29.740 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:29.933 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:41:29.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-24T06:41:29.948 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:30.144 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-24T06:41:30.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:30.159 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:30.159 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:30.160 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:30.160 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:30.160 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-24T06:41:30.160 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:30.361 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-24T06:41:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:30.376 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:30.377 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:30.377 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:30.377 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:30.377 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-24T06:41:30.377 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:30.574 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-24T06:41:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:30.588 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:30.588 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:30.588 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:30.589 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:30.589 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-24T06:41:30.589 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:30.783 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-24T06:41:30.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:30.798 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:30.798 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:30.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:31.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.110+0000 7f9116000640 1 Processor -- start 2024-04-24T06:41:31.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.111+0000 7f9116000640 1 -- start start 2024-04-24T06:41:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.112+0000 7f9116000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f91100b3820 0x7f911009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.112+0000 7f9116000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f91101625c0 0x7f911009af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.112+0000 7f9116000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91100cd2b0 con 0x7f911009ba00 2024-04-24T06:41:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.112+0000 7f9116000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f91100d06e0 con 0x7f91100b3820 2024-04-24T06:41:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.112+0000 7f9116000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f91100cd550 con 0x7f91101625c0 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f910fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f911009ba00 0x7f911015e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60906/0 (socket says 172.21.15.59:60906) 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f910fe00640 1 -- 172.21.15.59:0/3155923890 learned_addr learned my addr 172.21.15.59:0/3155923890 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9114c00640 1 --2- 172.21.15.59:0/3155923890 >> v2:172.21.15.59:3300/0 conn(0x7f91100b3820 0x7f911009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9115600640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f91101625c0 0x7f911009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9114c00640 1 -- 172.21.15.59:0/3155923890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f91101625c0 msgr2=0x7f911009af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9114c00640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f91101625c0 0x7f911009af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f910f400640 1 -- 172.21.15.59:0/3155923890 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486400461 0 0) 0x7f91100cd2b0 con 0x7f911009ba00 2024-04-24T06:41:31.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9114c00640 1 -- 172.21.15.59:0/3155923890 >> v1:172.21.15.59:6789/0 conn(0x7f911009ba00 legacy=0x7f911015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.113+0000 7f9114c00640 1 -- 172.21.15.59:0/3155923890 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91100cd830 con 0x7f91100b3820 2024-04-24T06:41:31.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f9114c00640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91100b3820 0x7f911009aa00 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f91040097a0 tx=0x7f910402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939d6a84b9053c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f910f400640 1 -- 172.21.15.59:0/3155923890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104037070 con 0x7f91100b3820 2024-04-24T06:41:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f910f400640 1 -- 172.21.15.59:0/3155923890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910402ae50 con 0x7f91100b3820 2024-04-24T06:41:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f910f400640 1 -- 172.21.15.59:0/3155923890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104032660 con 0x7f91100b3820 2024-04-24T06:41:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f9116000640 1 -- 172.21.15.59:0/3155923890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91100b3820 msgr2=0x7f911009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.115+0000 7f9116000640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91100b3820 0x7f911009aa00 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f91040097a0 tx=0x7f910402a100 comp rx=0 tx=0).stop 2024-04-24T06:41:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 -- 172.21.15.59:0/3155923890 shutdown_connections 2024-04-24T06:41:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f91101625c0 0x7f911009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 --2- 172.21.15.59:0/3155923890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91100b3820 0x7f911009aa00 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 -- 172.21.15.59:0/3155923890 >> 172.21.15.59:0/3155923890 conn(0x7f91100b3ef0 msgr2=0x7f91100b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 -- 172.21.15.59:0/3155923890 shutdown_connections 2024-04-24T06:41:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.116+0000 7f9116000640 1 -- 172.21.15.59:0/3155923890 wait complete. 2024-04-24T06:41:31.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.117+0000 7f9116000640 1 Processor -- start 2024-04-24T06:41:31.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.118+0000 7f9116000640 1 -- start start 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f911009ba00 0x7f91101f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91100b3820 0x7f91100cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91100ce0d0 con 0x7f91101625c0 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91100cddf0 con 0x7f91100b3820 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9116000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f91100ce5f0 con 0x7f911009ba00 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9115600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9115600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52512/0 (socket says 172.21.15.59:52512) 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9115600640 1 -- 172.21.15.59:0/2776335906 learned_addr learned my addr 172.21.15.59:0/2776335906 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f910fe00640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91100b3820 0x7f91100cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.119+0000 7f9115600640 1 -- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f911009ba00 msgr2=0x7f91101f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9115600640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f911009ba00 0x7f91101f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9115600640 1 -- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91100b3820 msgr2=0x7f91100cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9115600640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91100b3820 0x7f91100cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9115600640 1 -- 172.21.15.59:0/2776335906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91101541e0 con 0x7f91101625c0 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9115600640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f9108007e60 tx=0x7f910800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108014420 con 0x7f91101625c0 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9108019070 con 0x7f91101625c0 2024-04-24T06:41:31.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108014820 con 0x7f91101625c0 2024-04-24T06:41:31.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91101544c0 con 0x7f91101625c0 2024-04-24T06:41:31.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.120+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9110154a80 con 0x7f91101625c0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.121+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911004fd90 con 0x7f91101625c0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.123+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91080149c0 con 0x7f91101625c0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.123+0000 7f910cc00640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 0x7f90f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.124+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f910808c450 con 0x7f91101625c0 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.124+0000 7f9114c00640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 0x7f90f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.124+0000 7f9114c00640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 0x7f90f006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9104034000 tx=0x7f910402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.128+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f910805a7e0 con 0x7f91101625c0 2024-04-24T06:41:31.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.341+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91100cffd0 con 0x7f91101625c0 2024-04-24T06:41:31.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.343+0000 7f910cc00640 1 -- 172.21.15.59:0/2776335906 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16826 (secure 0 0 0) 0x7f910805a0e0 con 0x7f91101625c0 2024-04-24T06:41:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:31.346 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:29.062723+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 msgr2=0x7f90f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 0x7f90f006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9104034000 tx=0x7f910402c000 comp rx=0 tx=0).stop 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 msgr2=0x7f91100cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f9108007e60 tx=0x7f910800e4b0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 shutdown_connections 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f90f006c8f0 0x7f90f006edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91101625c0 0x7f91100cb800 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91100b3820 0x7f91100cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 --2- 172.21.15.59:0/2776335906 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f911009ba00 0x7f91101f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 >> 172.21.15.59:0/2776335906 conn(0x7f91100b3ef0 msgr2=0x7f9110070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 shutdown_connections 2024-04-24T06:41:31.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.346+0000 7f9116000640 1 -- 172.21.15.59:0/2776335906 wait complete. 2024-04-24T06:41:31.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:31.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:31.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.578+0000 7fa95d000640 1 Processor -- start 2024-04-24T06:41:31.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.579+0000 7fa95d000640 1 -- start start 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa95d000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa9580d0d10 0x7fa958156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa95d000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95815cab0 con 0x7fa95815d810 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa95d000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa958160e40 con 0x7fa9581567e0 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa95d000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa95815cd30 con 0x7fa9580d0d10 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa957e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa95815d810 0x7fa958160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60920/0 (socket says 172.21.15.59:60920) 2024-04-24T06:41:31.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.580+0000 7fa957e00640 1 -- 172.21.15.59:0/3255275628 learned_addr learned my addr 172.21.15.59:0/3255275628 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa94fe00640 1 --2- 172.21.15.59:0/3255275628 >> v2:172.21.15.66:3300/0 conn(0x7fa9580d0d10 0x7fa958156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061343867 0 0) 0x7fa950001490 con 0x7fa95815d810 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa93c003520 con 0x7fa95815d810 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2190297165 0 0) 0x7fa93c003520 con 0x7fa95815d810 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 >> v2:172.21.15.66:3300/0 conn(0x7fa9580d0d10 msgr2=0x7fa958156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 --2- 172.21.15.59:0/3255275628 >> v2:172.21.15.66:3300/0 conn(0x7fa9580d0d10 0x7fa958156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 >> v1:172.21.15.66:6789/0 conn(0x7fa9581567e0 legacy=0x7fa9580d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95815d010 con 0x7fa95815d810 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.581+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa950001bc0 con 0x7fa95815d810 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 >> v1:172.21.15.59:6789/0 conn(0x7fa95815d810 legacy=0x7fa958160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa93c003a00 0x7fa93c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa93c004800 0x7fa93c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa93c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa95815cd30 con 0x7fa93c0051c0 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c00c0c0 con 0x7fa93c004800 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c00c2c0 con 0x7fa93c003a00 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa957e00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa93c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa94fe00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa93c004800 0x7fa93c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa957400640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa93c003a00 0x7fa93c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa957e00640 1 -- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa93c003a00 msgr2=0x7fa93c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:31.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa957e00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa93c003a00 0x7fa93c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.582+0000 7fa957e00640 1 -- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa93c004800 msgr2=0x7fa93c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.583+0000 7fa957e00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa93c004800 0x7fa93c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.583+0000 7fa957e00640 1 -- 172.21.15.59:0/3255275628 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c003520 con 0x7fa93c0051c0 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.583+0000 7fa957e00640 1 -- 172.21.15.59:0/3255275628 reap_dead start 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa957e00640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa93c00b9c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fa95000a930 tx=0x7fa93c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97faefd857478585 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000a580 con 0x7fa93c0051c0 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95002f070 con 0x7fa93c0051c0 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa956a00640 1 -- 172.21.15.59:0/3255275628 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95002bd90 con 0x7fa93c0051c0 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa95d000640 1 -- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 msgr2=0x7fa93c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.584+0000 7fa95d000640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa93c00b9c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fa95000a930 tx=0x7fa93c005830 comp rx=0 tx=0).stop 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.585+0000 7fa95d000640 1 -- 172.21.15.59:0/3255275628 shutdown_connections 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.585+0000 7fa95d000640 1 --2- 172.21.15.59:0/3255275628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa93c00b9c0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.585+0000 7fa95d000640 1 -- 172.21.15.59:0/3255275628 >> 172.21.15.59:0/3255275628 conn(0x7fa95806bdf0 msgr2=0x7fa95806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.585+0000 7fa95d000640 1 -- 172.21.15.59:0/3255275628 shutdown_connections 2024-04-24T06:41:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.585+0000 7fa95d000640 1 -- 172.21.15.59:0/3255275628 wait complete. 2024-04-24T06:41:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.587+0000 7fa95d000640 1 Processor -- start 2024-04-24T06:41:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.587+0000 7fa95d000640 1 -- start start 2024-04-24T06:41:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa958153c70 0x7fa9581540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa95806eec0 0x7fa95806f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa958156d40 con 0x7fa93c0051c0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa958156bc0 con 0x7fa958153c70 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa95d000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa958156ec0 con 0x7fa95806eec0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa957400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa957400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52520/0 (socket says 172.21.15.59:52520) 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.588+0000 7fa957400640 1 -- 172.21.15.59:0/1420163710 learned_addr learned my addr 172.21.15.59:0/1420163710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa958153c70 0x7fa9581540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 -- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa95806eec0 msgr2=0x7fa95806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa95806eec0 0x7fa95806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 -- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa958153c70 msgr2=0x7fa9581540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa958153c70 0x7fa9581540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 -- 172.21.15.59:0/1420163710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9581e7f50 con 0x7fa93c0051c0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa957400640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e500 tx=0x7fa94800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480052e0 con 0x7fa93c0051c0 2024-04-24T06:41:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa948018070 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948003810 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.589+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9581e8230 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.590+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581e87f0 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.591+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95804fd90 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.592+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9480039b0 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.593+0000 7fa94f400640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 0x7fa93406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.593+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fa94808b8a0 con 0x7fa93c0051c0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.593+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 0x7fa93406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.594+0000 7fa94fe00640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 0x7fa93406ede0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa95815bc90 tx=0x7fa940008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.597+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa948059bb0 con 0x7fa93c0051c0 2024-04-24T06:41:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.799+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa95806c480 con 0x7fa93c0051c0 2024-04-24T06:41:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.800+0000 7fa94f400640 1 -- 172.21.15.59:0/1420163710 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fa9480594b0 con 0x7fa93c0051c0 2024-04-24T06:41:31.802 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 msgr2=0x7fa93406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 0x7fa93406ede0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa95815bc90 tx=0x7fa940008040 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 msgr2=0x7fa958099ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e500 tx=0x7fa94800e9c0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 shutdown_connections 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa93406c920 0x7fa93406ede0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa95806eec0 0x7fa95806f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa958153c70 0x7fa9581540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 --2- 172.21.15.59:0/1420163710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa93c0051c0 0x7fa958099ae0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 >> 172.21.15.59:0/1420163710 conn(0x7fa95806bdf0 msgr2=0x7fa95815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 shutdown_connections 2024-04-24T06:41:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:31.803+0000 7fa95d000640 1 -- 172.21.15.59:0/1420163710 wait complete. 2024-04-24T06:41:31.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.040+0000 7f0d31a00640 1 Processor -- start 2024-04-24T06:41:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.040+0000 7f0d31a00640 1 -- start start 2024-04-24T06:41:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.041+0000 7f0d31a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d2c1567e0 0x7f0d2c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.041+0000 7f0d31a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f0d2c15d810 0x7f0d2c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.041+0000 7f0d31a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2c15cab0 con 0x7f0d2c0d0b40 2024-04-24T06:41:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.041+0000 7f0d31a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c0d0f90 con 0x7f0d2c1567e0 2024-04-24T06:41:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.041+0000 7f0d31a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c160e40 con 0x7f0d2c15d810 2024-04-24T06:41:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d2c1567e0 0x7f0d2c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d2c1567e0 0x7f0d2c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52522/0 (socket says 172.21.15.59:52522) 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 -- 172.21.15.59:0/1049642700 learned_addr learned my addr 172.21.15.59:0/1049642700 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2b400640 1 --2- 172.21.15.59:0/1049642700 >> v2:172.21.15.66:3300/0 conn(0x7f0d2c15d810 0x7f0d2c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 -- 172.21.15.59:0/1049642700 >> v2:172.21.15.66:3300/0 conn(0x7f0d2c15d810 msgr2=0x7f0d2c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 --2- 172.21.15.59:0/1049642700 >> v2:172.21.15.66:3300/0 conn(0x7f0d2c15d810 0x7f0d2c0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 -- 172.21.15.59:0/1049642700 >> v1:172.21.15.66:6789/0 conn(0x7f0d2c0d0b40 legacy=0x7f0d2c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.042+0000 7f0d2be00640 1 -- 172.21.15.59:0/1049642700 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c15cd30 con 0x7f0d2c1567e0 2024-04-24T06:41:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.044+0000 7f0d2be00640 1 --2- 172.21.15.59:0/1049642700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c156bc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c074140 tx=0x7f0d1402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98778fe9450473 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.044+0000 7f0d2aa00640 1 -- 172.21.15.59:0/1049642700 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14037070 con 0x7f0d2c1567e0 2024-04-24T06:41:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.044+0000 7f0d2aa00640 1 -- 172.21.15.59:0/1049642700 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d14032440 con 0x7f0d2c1567e0 2024-04-24T06:41:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.044+0000 7f0d2aa00640 1 -- 172.21.15.59:0/1049642700 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14032810 con 0x7f0d2c1567e0 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 -- 172.21.15.59:0/1049642700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 msgr2=0x7f0d2c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 --2- 172.21.15.59:0/1049642700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c156bc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c074140 tx=0x7f0d1402a230 comp rx=0 tx=0).stop 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 -- 172.21.15.59:0/1049642700 shutdown_connections 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 --2- 172.21.15.59:0/1049642700 >> v2:172.21.15.66:3300/0 conn(0x7f0d2c15d810 0x7f0d2c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 --2- 172.21.15.59:0/1049642700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c156bc0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 -- 172.21.15.59:0/1049642700 >> 172.21.15.59:0/1049642700 conn(0x7f0d2c06bdf0 msgr2=0x7f0d2c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 -- 172.21.15.59:0/1049642700 shutdown_connections 2024-04-24T06:41:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.045+0000 7f0d31a00640 1 -- 172.21.15.59:0/1049642700 wait complete. 2024-04-24T06:41:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.047+0000 7f0d31a00640 1 Processor -- start 2024-04-24T06:41:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.047+0000 7f0d31a00640 1 -- start start 2024-04-24T06:41:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d2c0d0b40 0x7f0d2c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 0x7f0d2c1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c160820 con 0x7f0d2c1567e0 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c15d2f0 con 0x7f0d2c15d810 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.048+0000 7f0d31a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2c1609a0 con 0x7f0d2c0d0b40 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52532/0 (socket says 172.21.15.59:52532) 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 -- 172.21.15.59:0/4176190188 learned_addr learned my addr 172.21.15.59:0/4176190188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2be00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d2c0d0b40 0x7f0d2c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d31000640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 0x7f0d2c1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 -- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d2c0d0b40 msgr2=0x7f0d2c09dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d2c0d0b40 0x7f0d2c09dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 -- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 msgr2=0x7f0d2c1f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 0x7f0d2c1f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d2b400640 1 -- 172.21.15.59:0/4176190188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c159970 con 0x7f0d2c1567e0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.049+0000 7f0d31000640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 0x7f0d2c1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d2b400640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d7d0 tx=0x7f0d1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c004030 con 0x7f0d2c1567e0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1c0041d0 con 0x7f0d2c1567e0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c159c50 con 0x7f0d2c1567e0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00b6c0 con 0x7f0d2c1567e0 2024-04-24T06:41:32.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.050+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c15a210 con 0x7f0d2c1567e0 2024-04-24T06:41:32.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.051+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c04fd90 con 0x7f0d2c1567e0 2024-04-24T06:41:32.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.052+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d1c00b860 con 0x7f0d2c1567e0 2024-04-24T06:41:32.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.053+0000 7f0d23e00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 0x7f0d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.053+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f0d1c08bec0 con 0x7f0d2c1567e0 2024-04-24T06:41:32.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.053+0000 7f0d2be00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 0x7f0d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.054+0000 7f0d2be00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 0x7f0d0806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0d14034000 tx=0x7f0d1402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.057+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d1c01c360 con 0x7f0d2c1567e0 2024-04-24T06:41:32.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.261+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d2c0b5150 con 0x7f0d2c1567e0 2024-04-24T06:41:32.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.262+0000 7f0d23e00640 1 -- 172.21.15.59:0/4176190188 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f0d1c05a250 con 0x7f0d2c1567e0 2024-04-24T06:41:32.264 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:32.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 msgr2=0x7f0d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 0x7f0d0806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0d14034000 tx=0x7f0d1402e000 comp rx=0 tx=0).stop 2024-04-24T06:41:32.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 msgr2=0x7f0d2c14f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d7d0 tx=0x7f0d1c00dc90 comp rx=0 tx=0).stop 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 shutdown_connections 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f0d0806c8f0 0x7f0d0806edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d2c15d810 0x7f0d2c1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d2c1567e0 0x7f0d2c14f290 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 --2- 172.21.15.59:0/4176190188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0d2c0d0b40 0x7f0d2c09dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 >> 172.21.15.59:0/4176190188 conn(0x7f0d2c06bdf0 msgr2=0x7f0d2c15ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 shutdown_connections 2024-04-24T06:41:32.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.265+0000 7f0d31a00640 1 -- 172.21.15.59:0/4176190188 wait complete. 2024-04-24T06:41:32.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:32.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.496+0000 7fbc53e00640 1 Processor -- start 2024-04-24T06:41:32.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.496+0000 7fbc53e00640 1 -- start start 2024-04-24T06:41:32.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc5415ed20 0x7fbc540d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc540b3600 0x7fbc540d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc540b5da0 con 0x7fbc540b3f10 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5415f100 con 0x7fbc540b3600 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc540d0a20 con 0x7fbc5415ed20 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc540b3f10 0x7fbc540b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60936/0 (socket says 172.21.15.59:60936) 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc53400640 1 -- 172.21.15.59:0/504788543 learned_addr learned my addr 172.21.15.59:0/504788543 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.498+0000 7fbc52000640 1 --2- 172.21.15.59:0/504788543 >> v2:172.21.15.59:3300/0 conn(0x7fbc540b3600 0x7fbc540d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc52a00640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc5415ed20 0x7fbc540d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc51600640 1 -- 172.21.15.59:0/504788543 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491945964 0 0) 0x7fbc540b5da0 con 0x7fbc540b3f10 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc51600640 1 -- 172.21.15.59:0/504788543 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc38003520 con 0x7fbc540b3f10 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc52000640 1 -- 172.21.15.59:0/504788543 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc5415ed20 msgr2=0x7fbc540d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc52000640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc5415ed20 0x7fbc540d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc52000640 1 -- 172.21.15.59:0/504788543 >> v1:172.21.15.59:6789/0 conn(0x7fbc540b3f10 legacy=0x7fbc540b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.499+0000 7fbc52000640 1 -- 172.21.15.59:0/504788543 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc540d2ed0 con 0x7fbc540b3600 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.500+0000 7fbc52000640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc540d13e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fbc400097a0 tx=0x7fbc4002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e37cdda4ab84a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.500+0000 7fbc51600640 1 -- 172.21.15.59:0/504788543 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40037070 con 0x7fbc540b3600 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.500+0000 7fbc51600640 1 -- 172.21.15.59:0/504788543 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4002adb0 con 0x7fbc540b3600 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.500+0000 7fbc51600640 1 -- 172.21.15.59:0/504788543 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40032660 con 0x7fbc540b3600 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 -- 172.21.15.59:0/504788543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 msgr2=0x7fbc540d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc540d13e0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fbc400097a0 tx=0x7fbc4002a060 comp rx=0 tx=0).stop 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 -- 172.21.15.59:0/504788543 shutdown_connections 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc540d13e0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 --2- 172.21.15.59:0/504788543 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc5415ed20 0x7fbc540d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 -- 172.21.15.59:0/504788543 >> 172.21.15.59:0/504788543 conn(0x7fbc5408bf90 msgr2=0x7fbc540d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 -- 172.21.15.59:0/504788543 shutdown_connections 2024-04-24T06:41:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.501+0000 7fbc53e00640 1 -- 172.21.15.59:0/504788543 wait complete. 2024-04-24T06:41:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.503+0000 7fbc53e00640 1 Processor -- start 2024-04-24T06:41:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.503+0000 7fbc53e00640 1 -- start start 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc540b3f10 0x7fbc54159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc5415ed20 0x7fbc5415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc540d0290 con 0x7fbc540b3600 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc540d3490 con 0x7fbc5415ed20 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc53e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbc540d0410 con 0x7fbc540b3f10 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc52a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc52a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52558/0 (socket says 172.21.15.59:52558) 2024-04-24T06:41:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.504+0000 7fbc52a00640 1 -- 172.21.15.59:0/4119455748 learned_addr learned my addr 172.21.15.59:0/4119455748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52000640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc540b3f10 0x7fbc54159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 -- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc540b3f10 msgr2=0x7fbc54159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc540b3f10 0x7fbc54159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc53400640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc5415ed20 0x7fbc5415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 -- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc5415ed20 msgr2=0x7fbc5415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc5415ed20 0x7fbc5415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 -- 172.21.15.59:0/4119455748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc541e64e0 con 0x7fbc540b3600 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc52a00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fbc44007e70 tx=0x7fbc4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44014620 con 0x7fbc540b3600 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc44019070 con 0x7fbc540b3600 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44014a00 con 0x7fbc540b3600 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.505+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc541e67c0 con 0x7fbc540b3600 2024-04-24T06:41:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.506+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541e6d80 con 0x7fbc540b3600 2024-04-24T06:41:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.507+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5404fd90 con 0x7fbc540b3600 2024-04-24T06:41:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.508+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc44014ba0 con 0x7fbc540b3600 2024-04-24T06:41:32.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.509+0000 7fbc4aa00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 0x7fbc3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.509+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fbc4408c460 con 0x7fbc540b3600 2024-04-24T06:41:32.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.509+0000 7fbc52000640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 0x7fbc3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.510+0000 7fbc52000640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 0x7fbc3406ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc40034000 tx=0x7fbc4002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.513+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc4405a7f0 con 0x7fbc540b3600 2024-04-24T06:41:32.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.721+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc540b4f20 con 0x7fbc540b3600 2024-04-24T06:41:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.722+0000 7fbc4aa00640 1 -- 172.21.15.59:0/4119455748 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fbc4405a0f0 con 0x7fbc540b3600 2024-04-24T06:41:32.724 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 msgr2=0x7fbc3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 0x7fbc3406ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc40034000 tx=0x7fbc4002e000 comp rx=0 tx=0).stop 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 msgr2=0x7fbc54155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fbc44007e70 tx=0x7fbc4400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 shutdown_connections 2024-04-24T06:41:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbc3406c920 0x7fbc3406ede0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc5415ed20 0x7fbc5415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbc540b3f10 0x7fbc54159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 --2- 172.21.15.59:0/4119455748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc540b3600 0x7fbc54155c00 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.725+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 >> 172.21.15.59:0/4119455748 conn(0x7fbc5408bf90 msgr2=0x7fbc541e9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.726+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 shutdown_connections 2024-04-24T06:41:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.726+0000 7fbc53e00640 1 -- 172.21.15.59:0/4119455748 wait complete. 2024-04-24T06:41:32.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.959+0000 7fbb3da00640 1 Processor -- start 2024-04-24T06:41:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.959+0000 7fbb3da00640 1 -- start start 2024-04-24T06:41:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb380d0f80 0x7fbb380cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb3815f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb380cf930 con 0x7fbb380d1360 2024-04-24T06:41:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3da00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3815fdc0 con 0x7fbb380d0f80 2024-04-24T06:41:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380cfc10 con 0x7fbb3815ea40 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb37e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb380d0f80 0x7fbb380cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3d000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbb380d1360 0x7fbb38162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60948/0 (socket says 172.21.15.59:60948) 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.961+0000 7fbb3d000640 1 -- 172.21.15.59:0/1100387999 learned_addr learned my addr 172.21.15.59:0/1100387999 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.962+0000 7fbb2fe00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb3815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.962+0000 7fbb37e00640 1 -- 172.21.15.59:0/1100387999 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 msgr2=0x7fbb3815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.962+0000 7fbb37e00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb3815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.962+0000 7fbb37e00640 1 -- 172.21.15.59:0/1100387999 >> v1:172.21.15.59:6789/0 conn(0x7fbb380d1360 legacy=0x7fbb38162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.962+0000 7fbb37e00640 1 -- 172.21.15.59:0/1100387999 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380cfef0 con 0x7fbb380d0f80 2024-04-24T06:41:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.963+0000 7fbb37e00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380cf250 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fbb280097a0 tx=0x7fbb2802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e191442f89e5bdbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.963+0000 7fbb37400640 1 -- 172.21.15.59:0/1100387999 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28037070 con 0x7fbb380d0f80 2024-04-24T06:41:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb37400640 1 -- 172.21.15.59:0/1100387999 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2802ae40 con 0x7fbb380d0f80 2024-04-24T06:41:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb37400640 1 -- 172.21.15.59:0/1100387999 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28032660 con 0x7fbb380d0f80 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 -- 172.21.15.59:0/1100387999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 msgr2=0x7fbb380cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380cf250 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fbb280097a0 tx=0x7fbb2802a0f0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 -- 172.21.15.59:0/1100387999 shutdown_connections 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb3815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 --2- 172.21.15.59:0/1100387999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380cf250 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.964+0000 7fbb3da00640 1 -- 172.21.15.59:0/1100387999 >> 172.21.15.59:0/1100387999 conn(0x7fbb3808bf90 msgr2=0x7fbb380d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.965+0000 7fbb3da00640 1 -- 172.21.15.59:0/1100387999 shutdown_connections 2024-04-24T06:41:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.965+0000 7fbb3da00640 1 -- 172.21.15.59:0/1100387999 wait complete. 2024-04-24T06:41:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.967+0000 7fbb3da00640 1 Processor -- start 2024-04-24T06:41:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.967+0000 7fbb3da00640 1 -- start start 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380d1360 0x7fbb380b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb380b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380d17e0 con 0x7fbb380d0f80 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380d04b0 con 0x7fbb380d1360 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb3da00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380d3320 con 0x7fbb3815ea40 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.968+0000 7fbb37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52576/0 (socket says 172.21.15.59:52576) 2024-04-24T06:41:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 -- 172.21.15.59:0/2555211483 learned_addr learned my addr 172.21.15.59:0/2555211483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 -- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 msgr2=0x7fbb380b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb380b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 -- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380d1360 msgr2=0x7fbb380b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380d1360 0x7fbb380b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 -- 172.21.15.59:0/2555211483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380b5b20 con 0x7fbb380d0f80 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.969+0000 7fbb37e00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fbb28035040 tx=0x7fbb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.970+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28041070 con 0x7fbb380d0f80 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.970+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2803c670 con 0x7fbb380d0f80 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.970+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28037040 con 0x7fbb380d0f80 2024-04-24T06:41:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.970+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb380c8240 con 0x7fbb380d0f80 2024-04-24T06:41:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.970+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb380c8800 con 0x7fbb380d0f80 2024-04-24T06:41:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.971+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3804fd90 con 0x7fbb380d0f80 2024-04-24T06:41:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.972+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb2803a070 con 0x7fbb380d0f80 2024-04-24T06:41:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.973+0000 7fbb34c00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.973+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fbb280ac0a0 con 0x7fbb380d0f80 2024-04-24T06:41:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.974+0000 7fbb2fe00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.975+0000 7fbb2fe00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 0x7fbb1406edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbb381f7de0 tx=0x7fbb20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:32.978+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb2807a310 con 0x7fbb380d0f80 2024-04-24T06:41:33.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.199+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb380c89d0 con 0x7fbb380d0f80 2024-04-24T06:41:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.200+0000 7fbb34c00640 1 -- 172.21.15.59:0/2555211483 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v113) ==== 80+0+122 (secure 0 0 0) 0x7fbb28079c10 con 0x7fbb380d0f80 2024-04-24T06:41:33.202 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:33.202 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.202+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 msgr2=0x7fbb1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.202+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 0x7fbb1406edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbb381f7de0 tx=0x7fbb20008040 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 msgr2=0x7fbb380b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fbb28035040 tx=0x7fbb280046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 shutdown_connections 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb3815ea40 0x7fbb380b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380d1360 0x7fbb380b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 --2- 172.21.15.59:0/2555211483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb380d0f80 0x7fbb380b47b0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 >> 172.21.15.59:0/2555211483 conn(0x7fbb3808bf90 msgr2=0x7fbb38161980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 shutdown_connections 2024-04-24T06:41:33.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.203+0000 7fbb3da00640 1 -- 172.21.15.59:0/2555211483 wait complete. 2024-04-24T06:41:33.225 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:33.225 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:33.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:33.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.530+0000 7fa216200640 1 Processor -- start 2024-04-24T06:41:33.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.530+0000 7fa216200640 1 -- start start 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa210162df0 0x7fa21006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa2100cfb90 0x7fa2100b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2100b3f60 0x7fa2100c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa21006eaf0 con 0x7fa2100b3f60 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa2100b3590 con 0x7fa2100cfb90 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa216200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa21006ed40 con 0x7fa210162df0 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.532+0000 7fa215800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2100b3f60 0x7fa2100c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2100b3f60 0x7fa2100c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52580/0 (socket says 172.21.15.59:52580) 2024-04-24T06:41:33.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 -- 172.21.15.59:0/961677400 learned_addr learned my addr 172.21.15.59:0/961677400 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:33.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa214e00640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa210162df0 0x7fa21006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa20fe00640 1 --2- 172.21.15.59:0/961677400 >> v2:172.21.15.66:3300/0 conn(0x7fa2100cfb90 0x7fa2100b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 -- 172.21.15.59:0/961677400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa210162df0 msgr2=0x7fa21006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa210162df0 0x7fa21006f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 -- 172.21.15.59:0/961677400 >> v2:172.21.15.66:3300/0 conn(0x7fa2100cfb90 msgr2=0x7fa2100b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 --2- 172.21.15.59:0/961677400 >> v2:172.21.15.66:3300/0 conn(0x7fa2100cfb90 0x7fa2100b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.533+0000 7fa215800640 1 -- 172.21.15.59:0/961677400 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100b3220 con 0x7fa2100b3f60 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.534+0000 7fa215800640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2100b3f60 0x7fa2100c31a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa2100b2cc0 tx=0x7fa20802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b03a7708de1f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa20f400640 1 -- 172.21.15.59:0/961677400 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208036070 con 0x7fa2100b3f60 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa20f400640 1 -- 172.21.15.59:0/961677400 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa208031600 con 0x7fa2100b3f60 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa20f400640 1 -- 172.21.15.59:0/961677400 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080319d0 con 0x7fa2100b3f60 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 -- 172.21.15.59:0/961677400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2100b3f60 msgr2=0x7fa2100c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2100b3f60 0x7fa2100c31a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa2100b2cc0 tx=0x7fa20802a6a0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 -- 172.21.15.59:0/961677400 shutdown_connections 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2100b3f60 0x7fa2100c31a0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 --2- 172.21.15.59:0/961677400 >> v2:172.21.15.66:3300/0 conn(0x7fa2100cfb90 0x7fa2100b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 --2- 172.21.15.59:0/961677400 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa210162df0 0x7fa21006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 -- 172.21.15.59:0/961677400 >> 172.21.15.59:0/961677400 conn(0x7fa2100b21a0 msgr2=0x7fa2100b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.535+0000 7fa216200640 1 -- 172.21.15.59:0/961677400 shutdown_connections 2024-04-24T06:41:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.536+0000 7fa216200640 1 -- 172.21.15.59:0/961677400 wait complete. 2024-04-24T06:41:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.537+0000 7fa216200640 1 Processor -- start 2024-04-24T06:41:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.537+0000 7fa216200640 1 -- start start 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2100b3f60 0x7fa2100af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 0x7fa2100c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100c3860 con 0x7fa210162df0 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100c36e0 con 0x7fa2100b3f60 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.538+0000 7fa216200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa2100c39e0 con 0x7fa2100cfb90 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52596/0 (socket says 172.21.15.59:52596) 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 -- 172.21.15.59:0/351051740 learned_addr learned my addr 172.21.15.59:0/351051740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:33.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa20fe00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 0x7fa2100c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa214e00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2100b3f60 0x7fa2100af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 -- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 msgr2=0x7fa2100c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 0x7fa2100c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 -- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2100b3f60 msgr2=0x7fa2100af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2100b3f60 0x7fa2100af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 -- 172.21.15.59:0/351051740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100ca2a0 con 0x7fa210162df0 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa20fe00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 0x7fa2100c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.539+0000 7fa215800640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa208009f40 tx=0x7fa208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.540+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20803f020 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.540+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa208037040 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.540+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208036070 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.540+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2100afd60 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.540+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2100b0320 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.541+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21004fd90 con 0x7fa210162df0 2024-04-24T06:41:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.542+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa208042020 con 0x7fa210162df0 2024-04-24T06:41:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.543+0000 7fa20cc00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 0x7fa1f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.543+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fa2080ac180 con 0x7fa210162df0 2024-04-24T06:41:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.543+0000 7fa214e00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 0x7fa1f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.544+0000 7fa214e00640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 0x7fa1f006edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa204006fd0 tx=0x7fa204008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:33.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.548+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa208049030 con 0x7fa210162df0 2024-04-24T06:41:33.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.755+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2100ca7e0 con 0x7fa210162df0 2024-04-24T06:41:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.758+0000 7fa20cc00640 1 -- 172.21.15.59:0/351051740 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16826 (secure 0 0 0) 0x7fa20807a480 con 0x7fa210162df0 2024-04-24T06:41:33.761 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:33.761 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:29.062723+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.761+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 msgr2=0x7fa1f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.761+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 0x7fa1f006edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa204006fd0 tx=0x7fa204008040 comp rx=0 tx=0).stop 2024-04-24T06:41:33.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.761+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 msgr2=0x7fa2101fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.761+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa208009f40 tx=0x7fa208004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 shutdown_connections 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1f006c8f0 0x7fa1f006edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa210162df0 0x7fa2101fb440 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa2100cfb90 0x7fa2100c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 --2- 172.21.15.59:0/351051740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2100b3f60 0x7fa2100af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 >> 172.21.15.59:0/351051740 conn(0x7fa2100b21a0 msgr2=0x7fa2100d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 shutdown_connections 2024-04-24T06:41:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:33.762+0000 7fa216200640 1 -- 172.21.15.59:0/351051740 wait complete. 2024-04-24T06:41:33.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:33.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:34.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.001+0000 7f2b79a00640 1 Processor -- start 2024-04-24T06:41:34.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.001+0000 7f2b79a00640 1 -- start start 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.003+0000 7f2b79a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b74145e10 0x7f2b7406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.003+0000 7f2b79a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.003+0000 7f2b79a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b740cfc20 con 0x7f2b740d0cc0 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.003+0000 7f2b79a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740cff00 con 0x7f2b74145e10 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.003+0000 7f2b79a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740d01e0 con 0x7f2b74071d70 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b72a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2b74145e10 0x7f2b7406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b73400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b72000640 1 --1- >> v1:172.21.15.66:6789/0 conn(0x7f2b740d0cc0 0x7f2b740cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.66:6789/0 says I am v1:172.21.15.59:35170/0 (socket says 172.21.15.59:35170) 2024-04-24T06:41:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b72000640 1 -- 172.21.15.59:0/3292414524 learned_addr learned my addr 172.21.15.59:0/3292414524 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b73400640 1 -- 172.21.15.59:0/3292414524 >> v2:172.21.15.66:3300/0 conn(0x7f2b74145e10 msgr2=0x7f2b7406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b73400640 1 --2- 172.21.15.59:0/3292414524 >> v2:172.21.15.66:3300/0 conn(0x7f2b74145e10 0x7f2b7406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.004+0000 7f2b73400640 1 -- 172.21.15.59:0/3292414524 >> v1:172.21.15.66:6789/0 conn(0x7f2b740d0cc0 legacy=0x7f2b740cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.005+0000 7f2b73400640 1 -- 172.21.15.59:0/3292414524 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740d04c0 con 0x7f2b74071d70 2024-04-24T06:41:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.006+0000 7f2b73400640 1 --2- 172.21.15.59:0/3292414524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74072200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009d90 tx=0x7f2b6c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85aee4f123ebe40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.006+0000 7f2b71600640 1 -- 172.21.15.59:0/3292414524 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c007740 con 0x7f2b74071d70 2024-04-24T06:41:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b71600640 1 -- 172.21.15.59:0/3292414524 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c0078e0 con 0x7f2b74071d70 2024-04-24T06:41:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b71600640 1 -- 172.21.15.59:0/3292414524 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c007cb0 con 0x7f2b74071d70 2024-04-24T06:41:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b79a00640 1 -- 172.21.15.59:0/3292414524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 msgr2=0x7f2b74072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3292414524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74072200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009d90 tx=0x7f2b6c02a660 comp rx=0 tx=0).stop 2024-04-24T06:41:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b79a00640 1 -- 172.21.15.59:0/3292414524 shutdown_connections 2024-04-24T06:41:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3292414524 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74072200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.007+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3292414524 >> v2:172.21.15.66:3300/0 conn(0x7f2b74145e10 0x7f2b7406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.008+0000 7f2b79a00640 1 -- 172.21.15.59:0/3292414524 >> 172.21.15.59:0/3292414524 conn(0x7f2b74067d90 msgr2=0x7f2b7406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.008+0000 7f2b79a00640 1 -- 172.21.15.59:0/3292414524 shutdown_connections 2024-04-24T06:41:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.008+0000 7f2b79a00640 1 -- 172.21.15.59:0/3292414524 wait complete. 2024-04-24T06:41:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.010+0000 7f2b79a00640 1 Processor -- start 2024-04-24T06:41:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.010+0000 7f2b79a00640 1 -- start start 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b740d0cc0 0x7f2b740ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741461f0 con 0x7f2b74145e10 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7415cdd0 con 0x7f2b740d0cc0 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.011+0000 7f2b79a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740b1660 con 0x7f2b74071d70 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52600/0 (socket says 172.21.15.59:52600) 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 -- 172.21.15.59:0/3640878823 learned_addr learned my addr 172.21.15.59:0/3640878823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:34.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b72a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b72000640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b740d0cc0 0x7f2b740ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 -- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 msgr2=0x7f2b74131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74131a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 -- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b740d0cc0 msgr2=0x7f2b740ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b740d0cc0 0x7f2b740ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.012+0000 7f2b73400640 1 -- 172.21.15.59:0/3640878823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740a9600 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b73400640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009eb0 tx=0x7f2b6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c040070 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c031bc0 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c009450 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b740b1380 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.013+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b740a9b80 con 0x7f2b74145e10 2024-04-24T06:41:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.014+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7406b9d0 con 0x7f2b74145e10 2024-04-24T06:41:34.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.016+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b6c039040 con 0x7f2b74145e10 2024-04-24T06:41:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.017+0000 7f2b6aa00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.017+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f2b6c0ad3a0 con 0x7f2b74145e10 2024-04-24T06:41:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.017+0000 7f2b72a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.018+0000 7f2b72a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009aa0 tx=0x7f2b64009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.021+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b7406b9d0 con 0x7f2b74145e10 2024-04-24T06:41:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.227+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b74145670 con 0x7f2b74145e10 2024-04-24T06:41:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.228+0000 7f2b6aa00640 1 -- 172.21.15.59:0/3640878823 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f2b74145670 con 0x7f2b74145e10 2024-04-24T06:41:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 msgr2=0x7f2b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009aa0 tx=0x7f2b64009190 comp rx=0 tx=0).stop 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 msgr2=0x7f2b740ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009eb0 tx=0x7f2b6c004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 shutdown_connections 2024-04-24T06:41:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b5406c8f0 0x7f2b5406edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b74145e10 0x7f2b740ae730 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b740d0cc0 0x7f2b740ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 --2- 172.21.15.59:0/3640878823 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2b74071d70 0x7f2b74131a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.231+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 >> 172.21.15.59:0/3640878823 conn(0x7f2b74067d90 msgr2=0x7f2b741449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.232+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 shutdown_connections 2024-04-24T06:41:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.232+0000 7f2b79a00640 1 -- 172.21.15.59:0/3640878823 wait complete. 2024-04-24T06:41:34.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.464+0000 7f9bb5800640 1 Processor -- start 2024-04-24T06:41:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.464+0000 7f9bb5800640 1 -- start start 2024-04-24T06:41:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.465+0000 7f9bb5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bb015ed20 0x7f9bb00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.465+0000 7f9bb5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.465+0000 7f9bb5800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9bb00b3f10 0x7f9bb00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bb5800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb015f100 con 0x7f9bb015ed20 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bb5800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb00b5da0 con 0x7f9bb00b3f10 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bb5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00d0a20 con 0x7f9bb00b3600 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bb015ed20 0x7f9bb00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bb015ed20 0x7f9bb00d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52602/0 (socket says 172.21.15.59:52602) 2024-04-24T06:41:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 -- 172.21.15.59:0/3805122048 learned_addr learned my addr 172.21.15.59:0/3805122048 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 -- 172.21.15.59:0/3805122048 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 msgr2=0x7f9bb00d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3805122048 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bafe00640 1 -- 172.21.15.59:0/3805122048 >> v2:172.21.15.66:3300/0 conn(0x7f9bb00b3f10 msgr2=0x7f9bb00b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.466+0000 7f9bb4e00640 1 --2- 172.21.15.59:0/3805122048 >> v2:172.21.15.66:3300/0 conn(0x7f9bb00b3f10 0x7f9bb00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.467+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3805122048 >> v2:172.21.15.66:3300/0 conn(0x7f9bb00b3f10 0x7f9bb00b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.467+0000 7f9bafe00640 1 -- 172.21.15.59:0/3805122048 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00d2ed0 con 0x7f9bb015ed20 2024-04-24T06:41:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.468+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3805122048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015ed20 0x7f9bb00d0ea0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9bb008c430 tx=0x7f9ba002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4a738306a5003 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.468+0000 7f9baea00640 1 -- 172.21.15.59:0/3805122048 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0037070 con 0x7f9bb015ed20 2024-04-24T06:41:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.468+0000 7f9baea00640 1 -- 172.21.15.59:0/3805122048 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba002ae00 con 0x7f9bb015ed20 2024-04-24T06:41:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.468+0000 7f9baea00640 1 -- 172.21.15.59:0/3805122048 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0032660 con 0x7f9bb015ed20 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 -- 172.21.15.59:0/3805122048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015ed20 msgr2=0x7f9bb00d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3805122048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015ed20 0x7f9bb00d0ea0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9bb008c430 tx=0x7f9ba002a0b0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 -- 172.21.15.59:0/3805122048 shutdown_connections 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3805122048 >> v2:172.21.15.66:3300/0 conn(0x7f9bb00b3f10 0x7f9bb00b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3805122048 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3805122048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015ed20 0x7f9bb00d0ea0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 -- 172.21.15.59:0/3805122048 >> 172.21.15.59:0/3805122048 conn(0x7f9bb008bf90 msgr2=0x7f9bb00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 -- 172.21.15.59:0/3805122048 shutdown_connections 2024-04-24T06:41:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.469+0000 7f9bb5800640 1 -- 172.21.15.59:0/3805122048 wait complete. 2024-04-24T06:41:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.472+0000 7f9bb5800640 1 Processor -- start 2024-04-24T06:41:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.472+0000 7f9bb5800640 1 -- start start 2024-04-24T06:41:34.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bb015ed20 0x7f9bb00a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00b48e0 con 0x7f9bb00b3f10 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00d3490 con 0x7f9bb015ed20 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9bb5800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00b4a60 con 0x7f9bb00b3600 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.473+0000 7f9baf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52608/0 (socket says 172.21.15.59:52608) 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 -- 172.21.15.59:0/3728858086 learned_addr learned my addr 172.21.15.59:0/3728858086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9bb4e00640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bb015ed20 0x7f9bb00a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 -- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 msgr2=0x7f9bb00a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 -- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bb015ed20 msgr2=0x7f9bb00a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bb015ed20 0x7f9bb00a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.474+0000 7f9baf400640 1 -- 172.21.15.59:0/3728858086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00a1860 con 0x7f9bb00b3f10 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9baf400640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800b4a0 tx=0x7f9b9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98014070 con 0x7f9bb00b3f10 2024-04-24T06:41:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b98010610 con 0x7f9bb00b3f10 2024-04-24T06:41:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98010a00 con 0x7f9bb00b3f10 2024-04-24T06:41:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb00a1b40 con 0x7f9bb00b3f10 2024-04-24T06:41:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.475+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb00a2100 con 0x7f9bb00b3f10 2024-04-24T06:41:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.476+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb004fd90 con 0x7f9bb00b3f10 2024-04-24T06:41:34.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.478+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b98017070 con 0x7f9bb00b3f10 2024-04-24T06:41:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.479+0000 7f9ba7e00640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.479+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f9b9808b5d0 con 0x7f9bb00b3f10 2024-04-24T06:41:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.479+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.480+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0034000 tx=0x7f9ba002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.486+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b98059310 con 0x7f9bb00b3f10 2024-04-24T06:41:34.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.688+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9bb00d0250 con 0x7f9bb00b3f10 2024-04-24T06:41:34.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.689+0000 7f9ba7e00640 1 -- 172.21.15.59:0/3728858086 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f9b9805d610 con 0x7f9bb00b3f10 2024-04-24T06:41:34.691 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.691+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 msgr2=0x7f9b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.691+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0034000 tx=0x7f9ba002e000 comp rx=0 tx=0).stop 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.691+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 msgr2=0x7f9bb00a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.691+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800b4a0 tx=0x7f9b9800b960 comp rx=0 tx=0).stop 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.691+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 shutdown_connections 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bb015ed20 0x7f9bb00a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00b3f10 0x7f9bb00a0950 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 --2- 172.21.15.59:0/3728858086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bb00b3600 0x7f9bb00a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 >> 172.21.15.59:0/3728858086 conn(0x7f9bb008bf90 msgr2=0x7f9bb00b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 shutdown_connections 2024-04-24T06:41:34.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.692+0000 7f9bb5800640 1 -- 172.21.15.59:0/3728858086 wait complete. 2024-04-24T06:41:34.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:34.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.924+0000 7fd1ff000640 1 Processor -- start 2024-04-24T06:41:34.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.924+0000 7fd1ff000640 1 -- start start 2024-04-24T06:41:34.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.925+0000 7fd1ff000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1ff000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1f80d0350 0x7fd1f80d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1ff000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f815bf60 con 0x7fd1f80b56f0 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1ff000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd1f815c240 con 0x7fd1f80d0350 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1ff000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f815c520 con 0x7fd1f806fa50 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1f7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd1f80b56f0 0x7fd1f806f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60950/0 (socket says 172.21.15.59:60950) 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1f7e00640 1 -- 172.21.15.59:0/2215900390 learned_addr learned my addr 172.21.15.59:0/2215900390 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1fd000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1f80d0350 0x7fd1f80d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1f7400640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.926+0000 7fd1f6a00640 1 -- 172.21.15.59:0/2215900390 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245507739 0 0) 0x7fd1f815bf60 con 0x7fd1f80b56f0 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.927+0000 7fd1f6a00640 1 -- 172.21.15.59:0/2215900390 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1dc003520 con 0x7fd1f80b56f0 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.927+0000 7fd1fd000640 1 -- 172.21.15.59:0/2215900390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 msgr2=0x7fd1f806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.927+0000 7fd1fd000640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.927+0000 7fd1fd000640 1 -- 172.21.15.59:0/2215900390 >> v1:172.21.15.59:6789/0 conn(0x7fd1f80b56f0 legacy=0x7fd1f806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.927+0000 7fd1fd000640 1 -- 172.21.15.59:0/2215900390 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f815c800 con 0x7fd1f80d0350 2024-04-24T06:41:34.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.928+0000 7fd1fd000640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f80d0800 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097a0 tx=0x7fd1f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7ccc02ac7bf693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.928+0000 7fd1f6a00640 1 -- 172.21.15.59:0/2215900390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0037070 con 0x7fd1f80d0350 2024-04-24T06:41:34.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.928+0000 7fd1f6a00640 1 -- 172.21.15.59:0/2215900390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0032440 con 0x7fd1f80d0350 2024-04-24T06:41:34.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.928+0000 7fd1f6a00640 1 -- 172.21.15.59:0/2215900390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0032810 con 0x7fd1f80d0350 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 -- 172.21.15.59:0/2215900390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 msgr2=0x7fd1f80d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f80d0800 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097a0 tx=0x7fd1f002a180 comp rx=0 tx=0).stop 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 -- 172.21.15.59:0/2215900390 shutdown_connections 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f80d0800 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 --2- 172.21.15.59:0/2215900390 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 -- 172.21.15.59:0/2215900390 >> 172.21.15.59:0/2215900390 conn(0x7fd1f8067d90 msgr2=0x7fd1f806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 -- 172.21.15.59:0/2215900390 shutdown_connections 2024-04-24T06:41:34.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.929+0000 7fd1ff000640 1 -- 172.21.15.59:0/2215900390 wait complete. 2024-04-24T06:41:34.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.931+0000 7fd1ff000640 1 Processor -- start 2024-04-24T06:41:34.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.931+0000 7fd1ff000640 1 -- start start 2024-04-24T06:41:34.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f8151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80b56f0 0x7fd1f8149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f80b3710 con 0x7fd1f80d0350 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f815bc10 con 0x7fd1f80b56f0 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1ff000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f80b3890 con 0x7fd1f806fa50 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1fd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1fd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52628/0 (socket says 172.21.15.59:52628) 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.932+0000 7fd1fd000640 1 -- 172.21.15.59:0/3397311751 learned_addr learned my addr 172.21.15.59:0/3397311751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1f7e00640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f8151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1f7400640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80b56f0 0x7fd1f8149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 -- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 msgr2=0x7fd1f8151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f8151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 -- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80b56f0 msgr2=0x7fd1f8149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80b56f0 0x7fd1f8149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 -- 172.21.15.59:0/3397311751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f815a770 con 0x7fd1f80d0350 2024-04-24T06:41:34.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1f7e00640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f8151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.933+0000 7fd1fd000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00098c0 tx=0x7fd1f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.934+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0041070 con 0x7fd1f80d0350 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.934+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f003c870 con 0x7fd1f80d0350 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.934+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0037040 con 0x7fd1f80d0350 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.934+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f815aa10 con 0x7fd1f80d0350 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.934+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f815afd0 con 0x7fd1f80d0350 2024-04-24T06:41:34.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.935+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8068a80 con 0x7fd1f80d0350 2024-04-24T06:41:34.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.936+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1f0044070 con 0x7fd1f80d0350 2024-04-24T06:41:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.937+0000 7fd1efe00640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 0x7fd1d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.938+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fd1f00ac5d0 con 0x7fd1f80d0350 2024-04-24T06:41:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.938+0000 7fd1f7e00640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 0x7fd1d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.939+0000 7fd1f7e00640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 0x7fd1d806edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8005c00 tx=0x7fd1e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:34.943+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1f007a960 con 0x7fd1f80d0350 2024-04-24T06:41:35.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.150+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1f806ef20 con 0x7fd1f80d0350 2024-04-24T06:41:35.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.151+0000 7fd1efe00640 1 -- 172.21.15.59:0/3397311751 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fd1f007a260 con 0x7fd1f80d0350 2024-04-24T06:41:35.154 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:35.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 msgr2=0x7fd1d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 0x7fd1d806edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8005c00 tx=0x7fd1e8009040 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 msgr2=0x7fd1f81492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00098c0 tx=0x7fd1f00046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 shutdown_connections 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd1d806c8f0 0x7fd1d806edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1f80d0350 0x7fd1f81492f0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1f80b56f0 0x7fd1f8149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 --2- 172.21.15.59:0/3397311751 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1f806fa50 0x7fd1f8151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 >> 172.21.15.59:0/3397311751 conn(0x7fd1f8067d90 msgr2=0x7fd1f80ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 shutdown_connections 2024-04-24T06:41:35.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.154+0000 7fd1ff000640 1 -- 172.21.15.59:0/3397311751 wait complete. 2024-04-24T06:41:35.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.385+0000 7f8b92400640 1 Processor -- start 2024-04-24T06:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.386+0000 7f8b92400640 1 -- start start 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b92400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b8c15f060 0x7f8b8c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b92400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c15ba70 con 0x7f8b8c1416e0 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b92400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c15c960 con 0x7f8b8c0d05c0 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b92400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c15bdc0 con 0x7f8b8c15f060 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b8be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8b8c1416e0 0x7f8b8c15c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60956/0 (socket says 172.21.15.59:60956) 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b8be00640 1 -- 172.21.15.59:0/2279440292 learned_addr learned my addr 172.21.15.59:0/2279440292 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.387+0000 7f8b91a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b8c15f060 0x7f8b8c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.388+0000 7f8b8b400640 1 -- 172.21.15.59:0/2279440292 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158636653 0 0) 0x7f8b8c15ba70 con 0x7f8b8c1416e0 2024-04-24T06:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.388+0000 7f8b8b400640 1 -- 172.21.15.59:0/2279440292 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b70003520 con 0x7f8b8c1416e0 2024-04-24T06:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.388+0000 7f8b91a00640 1 -- 172.21.15.59:0/2279440292 >> v1:172.21.15.66:6789/0 conn(0x7f8b8c0d05c0 legacy=0x7f8b8c0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.388+0000 7f8b91a00640 1 -- 172.21.15.59:0/2279440292 >> v1:172.21.15.59:6789/0 conn(0x7f8b8c1416e0 legacy=0x7f8b8c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.388+0000 7f8b91a00640 1 -- 172.21.15.59:0/2279440292 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c15f800 con 0x7f8b8c15f060 2024-04-24T06:41:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.389+0000 7f8b91a00640 1 --2- 172.21.15.59:0/2279440292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c15f060 0x7f8b8c1555d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c15bf40 tx=0x7f8b8402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ed1fc397cce43d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.390+0000 7f8b8b400640 1 -- 172.21.15.59:0/2279440292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84037070 con 0x7f8b8c15f060 2024-04-24T06:41:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.390+0000 7f8b8b400640 1 -- 172.21.15.59:0/2279440292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b84032440 con 0x7f8b8c15f060 2024-04-24T06:41:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.390+0000 7f8b8b400640 1 -- 172.21.15.59:0/2279440292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b840327f0 con 0x7f8b8c15f060 2024-04-24T06:41:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.390+0000 7f8b92400640 1 -- 172.21.15.59:0/2279440292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c15f060 msgr2=0x7f8b8c1555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.390+0000 7f8b92400640 1 --2- 172.21.15.59:0/2279440292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c15f060 0x7f8b8c1555d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c15bf40 tx=0x7f8b8402a250 comp rx=0 tx=0).stop 2024-04-24T06:41:35.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.391+0000 7f8b92400640 1 -- 172.21.15.59:0/2279440292 shutdown_connections 2024-04-24T06:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.391+0000 7f8b92400640 1 --2- 172.21.15.59:0/2279440292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c15f060 0x7f8b8c1555d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.391+0000 7f8b92400640 1 -- 172.21.15.59:0/2279440292 >> 172.21.15.59:0/2279440292 conn(0x7f8b8c06bdf0 msgr2=0x7f8b8c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.391+0000 7f8b92400640 1 -- 172.21.15.59:0/2279440292 shutdown_connections 2024-04-24T06:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.391+0000 7f8b92400640 1 -- 172.21.15.59:0/2279440292 wait complete. 2024-04-24T06:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.393+0000 7f8b92400640 1 Processor -- start 2024-04-24T06:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.393+0000 7f8b92400640 1 -- start start 2024-04-24T06:41:35.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b8c0d05c0 0x7f8b8c09a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b8c15f060 0x7f8b8c0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c141b70 con 0x7f8b8c1416e0 2024-04-24T06:41:35.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c15fdc0 con 0x7f8b8c15f060 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b92400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c15f4f0 con 0x7f8b8c0d05c0 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b8be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b8be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52654/0 (socket says 172.21.15.59:52654) 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b8be00640 1 -- 172.21.15.59:0/3757645367 learned_addr learned my addr 172.21.15.59:0/3757645367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.394+0000 7f8b91a00640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b8c15f060 0x7f8b8c0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 -- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b8c0d05c0 msgr2=0x7f8b8c09a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b8c0d05c0 0x7f8b8c09a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 -- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b8c15f060 msgr2=0x7f8b8c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b8c15f060 0x7f8b8c0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 -- 172.21.15.59:0/3757645367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c09a800 con 0x7f8b8c1416e0 2024-04-24T06:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b8be00640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c530 tx=0x7f8b7800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017070 con 0x7f8b8c1416e0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78005ce0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780072b0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c09aae0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.395+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c09b0a0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.397+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c04fd90 con 0x7f8b8c1416e0 2024-04-24T06:41:35.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.398+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b7801a070 con 0x7f8b8c1416e0 2024-04-24T06:41:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.399+0000 7f8b88c00640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.399+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f8b7808b090 con 0x7f8b8c1416e0 2024-04-24T06:41:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.399+0000 7f8b91000640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.400+0000 7f8b91000640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0012e0 tx=0x7f8b7c007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.404+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b780594e0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.624+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b8c09e130 con 0x7f8b8c1416e0 2024-04-24T06:41:35.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.625+0000 7f8b88c00640 1 -- 172.21.15.59:0/3757645367 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v113) ==== 80+0+122 (secure 0 0 0) 0x7f8b78058de0 con 0x7f8b8c1416e0 2024-04-24T06:41:35.627 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:35.627 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:35.627 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-24T06:41:35.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.627+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 msgr2=0x7f8b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.627+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0012e0 tx=0x7f8b7c007d30 comp rx=0 tx=0).stop 2024-04-24T06:41:35.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.627+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 msgr2=0x7f8b8c09dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.627+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c530 tx=0x7f8b7800c9f0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 shutdown_connections 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b8c15f060 0x7f8b8c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b8c1416e0 0x7f8b8c09dbf0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 --2- 172.21.15.59:0/3757645367 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8b8c0d05c0 0x7f8b8c09a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 >> 172.21.15.59:0/3757645367 conn(0x7f8b8c06bdf0 msgr2=0x7f8b8c06f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 shutdown_connections 2024-04-24T06:41:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:35.628+0000 7f8b92400640 1 -- 172.21.15.59:0/3757645367 wait complete. 2024-04-24T06:41:35.649 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:41:35.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:35.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:35.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77614]: fuse finished with error 0 and tester_r 0 2024-04-24T06:41:35.736 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:41:38.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:41:38.812 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:41:41.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:41.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:41.839 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:41:41.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.113+0000 7f02b6a00640 1 Processor -- start 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.114+0000 7f02b6a00640 1 -- start start 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f02b015e7c0 0x7f02b00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b00cf030 con 0x7f02b00d2630 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f02b00d1740 con 0x7f02b015e7c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f02b00b7000 con 0x7f02b00d0d90 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b5600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f02b015e7c0 0x7f02b00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:33920/0 (socket says 172.21.15.59:33920) 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.115+0000 7f02b6000640 1 -- 172.21.15.59:0/2569089437 learned_addr learned my addr 172.21.15.59:0/2569089437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.116+0000 7f02b6000640 1 -- 172.21.15.59:0/2569089437 >> v2:172.21.15.66:3300/0 conn(0x7f02b015e7c0 msgr2=0x7f02b00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.116+0000 7f02b6000640 1 --2- 172.21.15.59:0/2569089437 >> v2:172.21.15.66:3300/0 conn(0x7f02b015e7c0 0x7f02b00d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.116+0000 7f02b6000640 1 -- 172.21.15.59:0/2569089437 >> v1:172.21.15.66:6789/0 conn(0x7f02b00d2630 legacy=0x7f02b00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.116+0000 7f02b6000640 1 -- 172.21.15.59:0/2569089437 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b00b6a40 con 0x7f02b00d0d90 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.117+0000 7f02b6000640 1 --2- 172.21.15.59:0/2569089437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e50 tx=0x7f02a802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2f32d8caad8535 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.118+0000 7f02afe00640 1 -- 172.21.15.59:0/2569089437 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8036070 con 0x7f02b00d0d90 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.118+0000 7f02afe00640 1 -- 172.21.15.59:0/2569089437 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a8031600 con 0x7f02b00d0d90 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.118+0000 7f02afe00640 1 -- 172.21.15.59:0/2569089437 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80319d0 con 0x7f02b00d0d90 2024-04-24T06:41:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.118+0000 7f02b6a00640 1 -- 172.21.15.59:0/2569089437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 msgr2=0x7f02b00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.118+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2569089437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e50 tx=0x7f02a802a710 comp rx=0 tx=0).stop 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 -- 172.21.15.59:0/2569089437 shutdown_connections 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2569089437 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d0d90 0x7f02b00d1200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2569089437 >> v2:172.21.15.66:3300/0 conn(0x7f02b015e7c0 0x7f02b00d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 -- 172.21.15.59:0/2569089437 >> 172.21.15.59:0/2569089437 conn(0x7f02b009d920 msgr2=0x7f02b008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 -- 172.21.15.59:0/2569089437 shutdown_connections 2024-04-24T06:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.119+0000 7f02b6a00640 1 -- 172.21.15.59:0/2569089437 wait complete. 2024-04-24T06:41:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.121+0000 7f02b6a00640 1 Processor -- start 2024-04-24T06:41:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.121+0000 7f02b6a00640 1 -- start start 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02b00d0d90 0x7f02b00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d2630 0x7f02b00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02b00cf2b0 con 0x7f02b015e7c0 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02b00cf620 con 0x7f02b00d0d90 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f02b00b6700 con 0x7f02b00d2630 2024-04-24T06:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46400/0 (socket says 172.21.15.59:46400) 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b6000640 1 -- 172.21.15.59:0/2606544802 learned_addr learned my addr 172.21.15.59:0/2606544802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b5600640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02b00d0d90 0x7f02b00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.123+0000 7f02b4c00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d2630 0x7f02b00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 -- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d2630 msgr2=0x7f02b00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d2630 0x7f02b00cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 -- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02b00d0d90 msgr2=0x7f02b00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02b00d0d90 0x7f02b00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 -- 172.21.15.59:0/2606544802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b01582a0 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6000640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e50 tx=0x7f02a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8036050 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a8042070 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8009510 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.124+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b00ca530 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.125+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b00caaf0 con 0x7f02b015e7c0 2024-04-24T06:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.126+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b004fd90 con 0x7f02b015e7c0 2024-04-24T06:41:42.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.127+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02a8007bf0 con 0x7f02b015e7c0 2024-04-24T06:41:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.127+0000 7f02ad600640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.128+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f02a80acae0 con 0x7f02b015e7c0 2024-04-24T06:41:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.128+0000 7f02b5600640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.128+0000 7f02b5600640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 0x7f029006edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009a30 tx=0x7f02a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.136+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02a807ad50 con 0x7f02b015e7c0 2024-04-24T06:41:42.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.385+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02b00d1eb0 con 0x7f02b015e7c0 2024-04-24T06:41:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.387+0000 7f02ad600640 1 -- 172.21.15.59:0/2606544802 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5181 (secure 0 0 0) 0x7f02a807a650 con 0x7f02b015e7c0 2024-04-24T06:41:42.388 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:42.388 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:25.036625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":13,"addr":"172.21.15.59:6837/3446399276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3446399276},{"type":"v1","addr":"172.21.15.59:6837","nonce":3446399276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":8}]} 2024-04-24T06:41:42.388 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 107 2024-04-24T06:41:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 msgr2=0x7f029006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 0x7f029006edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009a30 tx=0x7f02a4008040 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 msgr2=0x7f02b00c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e50 tx=0x7f02a8004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 shutdown_connections 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f029006c8f0 0x7f029006edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b015e7c0 0x7f02b00c96b0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02b00d2630 0x7f02b00cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 --2- 172.21.15.59:0/2606544802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02b00d0d90 0x7f02b00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.390+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 >> 172.21.15.59:0/2606544802 conn(0x7f02b009d920 msgr2=0x7f02b00d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.391+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 shutdown_connections 2024-04-24T06:41:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.391+0000 7f02b6a00640 1 -- 172.21.15.59:0/2606544802 wait complete. 2024-04-24T06:41:42.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-24T06:41:42.647 INFO:teuthology.orchestra.run.smithi059.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-24T06:41:42.648 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-24T06:41:42.648 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-24T06:41:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.759+0000 7ffa7ac34f40 1 Processor -- start 2024-04-24T06:41:42.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.759+0000 7ffa7ac34f40 1 -- start start 2024-04-24T06:41:42.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.759+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab254330 0x5577ab2eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.759+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577ab26f750 con 0x5577ab2f0850 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577ab1cbcc0 con 0x5577ab254330 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5577ab2fa7e0 con 0x5577ab2f0470 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46414/0 (socket says 172.21.15.59:46414) 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 -- 172.21.15.59:0/2316481215 learned_addr learned my addr 172.21.15.59:0/2316481215 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa77400640 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa77e00640 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab254330 0x5577ab2eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 -- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 msgr2=0x5577ab2533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 -- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab254330 msgr2=0x5577ab2eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.760+0000 7ffa78800640 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab254330 0x5577ab2eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.761+0000 7ffa78800640 1 -- 172.21.15.59:0/2316481215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ab2527c0 con 0x5577ab2f0850 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.761+0000 7ffa78800640 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7ffa70001ce0 tx=0x7ffa7000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eefb92ffd117609 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.761+0000 7ffa76000640 1 -- 172.21.15.59:0/2316481215 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000bd20 con 0x5577ab2f0850 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.761+0000 7ffa76000640 1 -- 172.21.15.59:0/2316481215 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7000f400 con 0x5577ab2f0850 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.761+0000 7ffa76000640 1 -- 172.21.15.59:0/2316481215 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000f7f0 con 0x5577ab2f0850 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/2316481215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 msgr2=0x5577ab2fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7ffa70001ce0 tx=0x7ffa7000b5b0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/2316481215 shutdown_connections 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab2f0850 0x5577ab2fc240 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/2316481215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab254330 0x5577ab2eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/2316481215 >> 172.21.15.59:0/2316481215 conn(0x5577ab270290 msgr2=0x5577ab24fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/2316481215 shutdown_connections 2024-04-24T06:41:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.762+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/2316481215 wait complete. 2024-04-24T06:41:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.763+0000 7ffa7ac34f40 1 Processor -- start 2024-04-24T06:41:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.763+0000 7ffa7ac34f40 1 -- start start 2024-04-24T06:41:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 0x5577ab225750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab2f0850 0x5577ab252990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5577ab26cb40 con 0x5577ab254330 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5577ab2538e0 con 0x5577ab2f0850 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa7ac34f40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x5577ab2ef710 con 0x5577ab2f0470 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 0x5577ab225750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 0x5577ab225750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46428/0 (socket says 172.21.15.59:46428) 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.764+0000 7ffa77e00640 1 -- 172.21.15.59:0/612430357 learned_addr learned my addr 172.21.15.59:0/612430357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 -- 172.21.15.59:0/612430357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 msgr2=0x5577ab2eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5577ab2f0470 0x5577ab2eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 -- 172.21.15.59:0/612430357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab2f0850 msgr2=0x5577ab252990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5577ab2f0850 0x5577ab252990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 -- 172.21.15.59:0/612430357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ab2ebc90 con 0x5577ab254330 2024-04-24T06:41:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa77e00640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 0x5577ab225750 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800f6a0 tx=0x7ffa6800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa74c00640 1 -- 172.21.15.59:0/612430357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68019020 con 0x5577ab254330 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa74c00640 1 -- 172.21.15.59:0/612430357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6800d7b0 con 0x5577ab254330 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa74c00640 1 -- 172.21.15.59:0/612430357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800db80 con 0x5577ab254330 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.765+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577ab223920 con 0x5577ab254330 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.766+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ab223ee0 con 0x5577ab254330 2024-04-24T06:41:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.768+0000 7ffa74c00640 1 -- 172.21.15.59:0/612430357 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa6801c020 con 0x5577ab254330 2024-04-24T06:41:42.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.768+0000 7ffa74c00640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa5806b300 0x7ffa5806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.769+0000 7ffa74c00640 1 -- 172.21.15.59:0/612430357 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7ffa6808b130 con 0x5577ab254330 2024-04-24T06:41:42.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.769+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5577ab24f270 0x5577ab26a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.769+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5577ab26abb0 con 0x5577ab24f270 2024-04-24T06:41:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.769+0000 7ffa77400640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa5806b300 0x7ffa5806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.769+0000 7ffa78800640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5577ab24f270 0x5577ab26a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.770+0000 7ffa77400640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa5806b300 0x7ffa5806d7c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffa64057000 tx=0x7ffa64051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.770+0000 7ffa78800640 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5577ab24f270 0x5577ab26a670 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.770+0000 7ffa78800640 1 -- 172.21.15.59:0/612430357 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7ffa7000f5b0 con 0x5577ab24f270 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5577ab24f270 msgr2=0x5577ab26a670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x5577ab24f270 0x5577ab26a670 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa5806b300 msgr2=0x7ffa5806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffa5806b300 0x7ffa5806d7c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffa64057000 tx=0x7ffa64051000 comp rx=0 tx=0).stop 2024-04-24T06:41:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 msgr2=0x5577ab225750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 --2- 172.21.15.59:0/612430357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5577ab254330 0x5577ab225750 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800f6a0 tx=0x7ffa6800fb60 comp rx=0 tx=0).stop 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa78800640 1 -- 172.21.15.59:0/612430357 reap_dead start 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 shutdown_connections 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 >> 172.21.15.59:0/612430357 conn(0x5577ab270290 msgr2=0x5577ab24fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 shutdown_connections 2024-04-24T06:41:42.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.771+0000 7ffa7ac34f40 1 -- 172.21.15.59:0/612430357 wait complete. 2024-04-24T06:41:42.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:41:42.777 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-24T06:41:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.867+0000 7fb1dfb1bf40 1 Processor -- start 2024-04-24T06:41:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 -- start start 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 0x56440293f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 0x564402a2f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644029549c0 con 0x564402a11ac0 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5644028fbcc0 con 0x564402a2cad0 2024-04-24T06:41:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.868+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564402952b20 con 0x5644029521d0 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46430/0 (socket says 172.21.15.59:46430) 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 -- 172.21.15.59:0/618583671 learned_addr learned my addr 172.21.15.59:0/618583671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1dc200640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 0x56440293f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1dcc00640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 0x564402a2f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 -- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 msgr2=0x56440293f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 0x56440293f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 -- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 msgr2=0x564402a2f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 0x564402a2f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1d3800640 1 -- 172.21.15.59:0/618583671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564402a30110 con 0x564402a11ac0 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1dc200640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 0x56440293f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.869+0000 7fb1dcc00640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 0x564402a2f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1d3800640 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009780 tx=0x7fb1c4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ab4cbaebcb4ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1dae00640 1 -- 172.21.15.59:0/618583671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4008870 con 0x564402a11ac0 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1dae00640 1 -- 172.21.15.59:0/618583671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c4018070 con 0x564402a11ac0 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1dae00640 1 -- 172.21.15.59:0/618583671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4008c30 con 0x564402a11ac0 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/618583671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 msgr2=0x564402a2f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.870+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009780 tx=0x7fb1c4005a00 comp rx=0 tx=0).stop 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/618583671 shutdown_connections 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564402a2cad0 0x564402a2f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a11ac0 0x564402a2f3b0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/618583671 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x5644029521d0 0x56440293f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/618583671 >> 172.21.15.59:0/618583671 conn(0x56440299ecc0 msgr2=0x56440299da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/618583671 shutdown_connections 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.871+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/618583671 wait complete. 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.872+0000 7fb1dfb1bf40 1 Processor -- start 2024-04-24T06:41:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.872+0000 7fb1dfb1bf40 1 -- start start 2024-04-24T06:41:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644029521d0 0x564402987350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564402a11ac0 0x564402a0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 0x564402a100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564402981a00 con 0x564402a2cad0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564402a2e2a0 con 0x5644029521d0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dfb1bf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564402a2e5a0 con 0x564402a11ac0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 0x564402a100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 0x564402a100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46442/0 (socket says 172.21.15.59:46442) 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dcc00640 1 -- 172.21.15.59:0/590420805 learned_addr learned my addr 172.21.15.59:0/590420805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1d3800640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564402a11ac0 0x564402a0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.873+0000 7fb1dc200640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644029521d0 0x564402987350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 -- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564402a11ac0 msgr2=0x564402a0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564402a11ac0 0x564402a0d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 -- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644029521d0 msgr2=0x564402987350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644029521d0 0x564402987350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 -- 172.21.15.59:0/590420805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564402a20c60 con 0x564402a2cad0 2024-04-24T06:41:42.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dcc00640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 0x564402a100f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400a4f0 tx=0x7fb1d4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1d9a00640 1 -- 172.21.15.59:0/590420805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4064020 con 0x564402a2cad0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1d9a00640 1 -- 172.21.15.59:0/590420805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d4054a20 con 0x564402a2cad0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1d9a00640 1 -- 172.21.15.59:0/590420805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4059030 con 0x564402a2cad0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564402986870 con 0x564402a2cad0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.874+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564402a0dd10 con 0x564402a2cad0 2024-04-24T06:41:42.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.876+0000 7fb1d9a00640 1 -- 172.21.15.59:0/590420805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1d4067020 con 0x564402a2cad0 2024-04-24T06:41:42.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.877+0000 7fb1d9a00640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1b806b300 0x7fb1b806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.877+0000 7fb1d9a00640 1 -- 172.21.15.59:0/590420805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fb1d40ce300 con 0x564402a2cad0 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.878+0000 7fb1dc200640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1b806b300 0x7fb1b806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.878+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x564402a110f0 0x564402a210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.878+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x564402a21600 con 0x564402a110f0 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.878+0000 7fb1d3800640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x564402a110f0 0x564402a210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.878+0000 7fb1dc200640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1b806b300 0x7fb1b806d7c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0012e0 tx=0x7fb1cc006170 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1d3800640 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x564402a110f0 0x564402a210c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1d3800640 1 -- 172.21.15.59:0/590420805 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb1c4008890 con 0x564402a110f0 2024-04-24T06:41:42.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x564402a110f0 msgr2=0x564402a210c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x564402a110f0 0x564402a210c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1b806b300 msgr2=0x7fb1b806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb1b806b300 0x7fb1b806d7c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0012e0 tx=0x7fb1cc006170 comp rx=0 tx=0).stop 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 msgr2=0x564402a100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.879+0000 7fb1dfb1bf40 1 --2- 172.21.15.59:0/590420805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564402a2cad0 0x564402a100f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400a4f0 tx=0x7fb1d4002fb0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.880+0000 7fb1dcc00640 1 -- 172.21.15.59:0/590420805 reap_dead start 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.880+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 shutdown_connections 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.880+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 >> 172.21.15.59:0/590420805 conn(0x56440299ecc0 msgr2=0x56440299da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.880+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 shutdown_connections 2024-04-24T06:41:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.880+0000 7fb1dfb1bf40 1 -- 172.21.15.59:0/590420805 wait complete. 2024-04-24T06:41:42.885 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-24T06:41:42.885 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-24T06:41:42.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.972+0000 7f2b5198af40 1 Processor -- start 2024-04-24T06:41:42.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.972+0000 7f2b5198af40 1 -- start start 2024-04-24T06:41:42.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 0x55f5d084c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08c0ac0 0x55f5d084cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5d08039c0 con 0x55f5d08dbad0 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5d07aacc0 con 0x55f5d08011d0 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b5198af40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55f5d0801b20 con 0x55f5d08c0ac0 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46468/0 (socket says 172.21.15.59:46468) 2024-04-24T06:41:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 -- 172.21.15.59:0/3288660466 learned_addr learned my addr 172.21.15.59:0/3288660466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4ec00640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 0x55f5d084c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 -- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08c0ac0 msgr2=0x55f5d084cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4e200640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08c0ac0 0x55f5d084cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08c0ac0 0x55f5d084cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 -- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 msgr2=0x55f5d084c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 0x55f5d084c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4f600640 1 -- 172.21.15.59:0/3288660466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d08327f0 con 0x55f5d08dbad0 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.973+0000 7f2b4ec00640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 0x55f5d084c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.974+0000 7f2b4f600640 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800a4f0 tx=0x7f2b48051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94604c0f2ac40d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.974+0000 7f2b4ce00640 1 -- 172.21.15.59:0/3288660466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48056070 con 0x55f5d08dbad0 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.974+0000 7f2b4ce00640 1 -- 172.21.15.59:0/3288660466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48052ab0 con 0x55f5d08dbad0 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.974+0000 7f2b4ce00640 1 -- 172.21.15.59:0/3288660466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48052e60 con 0x55f5d08dbad0 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 -- 172.21.15.59:0/3288660466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 msgr2=0x55f5d084fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800a4f0 tx=0x7f2b48051000 comp rx=0 tx=0).stop 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 -- 172.21.15.59:0/3288660466 shutdown_connections 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d084fad0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08c0ac0 0x55f5d084cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 --2- 172.21.15.59:0/3288660466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08011d0 0x55f5d084c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 -- 172.21.15.59:0/3288660466 >> 172.21.15.59:0/3288660466 conn(0x55f5d084dcc0 msgr2=0x55f5d0830a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 -- 172.21.15.59:0/3288660466 shutdown_connections 2024-04-24T06:41:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.975+0000 7f2b5198af40 1 -- 172.21.15.59:0/3288660466 wait complete. 2024-04-24T06:41:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.976+0000 7f2b5198af40 1 Processor -- start 2024-04-24T06:41:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.976+0000 7f2b5198af40 1 -- start start 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08011d0 0x55f5d0835430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08c0ac0 0x55f5d08bc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d08d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5d08d6f50 con 0x55f5d08dbad0 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f5d08d6d00 con 0x55f5d08c0ac0 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b5198af40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55f5d08316a0 con 0x55f5d08011d0 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d08d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d08d9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46482/0 (socket says 172.21.15.59:46482) 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 -- 172.21.15.59:0/1959177170 learned_addr learned my addr 172.21.15.59:0/1959177170 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4e200640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08c0ac0 0x55f5d08bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 -- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08011d0 msgr2=0x55f5d0835430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55f5d08011d0 0x55f5d0835430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 -- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08c0ac0 msgr2=0x55f5d08bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5d08c0ac0 0x55f5d08bc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.977+0000 7f2b4f600640 1 -- 172.21.15.59:0/1959177170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5d08bcda0 con 0x55f5d08dbad0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b4f600640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d08d9ac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f2b48025dc0 tx=0x7f2b48002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b47400640 1 -- 172.21.15.59:0/1959177170 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48056040 con 0x55f5d08dbad0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b47400640 1 -- 172.21.15.59:0/1959177170 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48072070 con 0x55f5d08dbad0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b47400640 1 -- 172.21.15.59:0/1959177170 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48065440 con 0x55f5d08dbad0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f5d08c4070 con 0x55f5d08dbad0 2024-04-24T06:41:42.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.978+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5d08c3d90 con 0x55f5d08dbad0 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.980+0000 7f2b47400640 1 -- 172.21.15.59:0/1959177170 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b48059020 con 0x55f5d08dbad0 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.981+0000 7f2b47400640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b3006b2d0 0x7f2b3006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.981+0000 7f2b47400640 1 -- 172.21.15.59:0/1959177170 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f2b480cfa10 con 0x55f5d08dbad0 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.982+0000 7f2b4ec00640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b3006b2d0 0x7f2b3006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.982+0000 7f2b5198af40 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55f5d08da000 0x55f5d08c1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.982+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55f5d08c1680 con 0x55f5d08da000 2024-04-24T06:41:42.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.982+0000 7f2b4ec00640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b3006b2d0 0x7f2b3006d790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000a530 tx=0x7f2b400059f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.982+0000 7f2b4e200640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55f5d08da000 0x55f5d08c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.983+0000 7f2b4e200640 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55f5d08da000 0x55f5d08c1140 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.984+0000 7f2b4e200640 1 -- 172.21.15.59:0/1959177170 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f2b3c00c570 con 0x55f5d08da000 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55f5d08da000 msgr2=0x55f5d08c1140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55f5d08da000 0x55f5d08c1140 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b3006b2d0 msgr2=0x7f2b3006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2b3006b2d0 0x7f2b3006d790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000a530 tx=0x7f2b400059f0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 msgr2=0x55f5d08d9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 --2- 172.21.15.59:0/1959177170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5d08dbad0 0x55f5d08d9ac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f2b48025dc0 tx=0x7f2b48002fb0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b4f600640 1 -- 172.21.15.59:0/1959177170 reap_dead start 2024-04-24T06:41:42.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 shutdown_connections 2024-04-24T06:41:42.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 >> 172.21.15.59:0/1959177170 conn(0x55f5d084dcc0 msgr2=0x55f5d08309e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 shutdown_connections 2024-04-24T06:41:42.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:42.985+0000 7f2b5198af40 1 -- 172.21.15.59:0/1959177170 wait complete. 2024-04-24T06:41:42.990 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-24T06:41:42.991 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-24T06:41:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.087+0000 7f12a731cf40 1 Processor -- start 2024-04-24T06:41:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.087+0000 7f12a731cf40 1 -- start start 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059d13240 0x560059dbf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560059dbf8f0 0x560059d30ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560059c8bcc0 con 0x7f12880030f0 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560059c8fd00 con 0x560059dbf8f0 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a731cf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x560059ce49c0 con 0x560059d13240 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46506/0 (socket says 172.21.15.59:46506) 2024-04-24T06:41:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.088+0000 7f12a3a00640 1 -- 172.21.15.59:0/2835455093 learned_addr learned my addr 172.21.15.59:0/2835455093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a4400640 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560059dbf8f0 0x560059d30ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 -- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059d13240 msgr2=0x560059dbf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059d13240 0x560059dbf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 -- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560059dbf8f0 msgr2=0x560059d30ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560059dbf8f0 0x560059d30ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 -- 172.21.15.59:0/2835455093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560059d11d30 con 0x7f12880030f0 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a3a00640 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1294006860 tx=0x7f1294006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defa8582a6c01614 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a1c00640 1 -- 172.21.15.59:0/2835455093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940062e0 con 0x7f12880030f0 2024-04-24T06:41:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a1c00640 1 -- 172.21.15.59:0/2835455093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1294006e50 con 0x7f12880030f0 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.089+0000 7f12a1c00640 1 -- 172.21.15.59:0/2835455093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129400cb30 con 0x7f12880030f0 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 -- 172.21.15.59:0/2835455093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 msgr2=0x560059d9d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1294006860 tx=0x7f1294006d20 comp rx=0 tx=0).stop 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 -- 172.21.15.59:0/2835455093 shutdown_connections 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560059dbf8f0 0x560059d30ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059d13240 0x560059dbf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 --2- 172.21.15.59:0/2835455093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12880030f0 0x560059d9d980 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 -- 172.21.15.59:0/2835455093 >> 172.21.15.59:0/2835455093 conn(0x560059d2ecc0 msgr2=0x560059d2dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 -- 172.21.15.59:0/2835455093 shutdown_connections 2024-04-24T06:41:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.090+0000 7f12a731cf40 1 -- 172.21.15.59:0/2835455093 wait complete. 2024-04-24T06:41:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.091+0000 7f12a731cf40 1 Processor -- start 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.091+0000 7f12a731cf40 1 -- start start 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 0x560059db6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059dbf8f0 0x560059da0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12880030f0 0x560059dbaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560059d122b0 con 0x560059d13240 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560059db80d0 con 0x7f12880030f0 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a731cf40 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x560059d11fd0 con 0x560059dbf8f0 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 0x560059db6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 0x560059db6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46514/0 (socket says 172.21.15.59:46514) 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.092+0000 7f12a3a00640 1 -- 172.21.15.59:0/1298373466 learned_addr learned my addr 172.21.15.59:0/1298373466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a4400640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12880030f0 0x560059dbaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3000640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059dbf8f0 0x560059da0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 -- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059dbf8f0 msgr2=0x560059da0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x560059dbf8f0 0x560059da0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 -- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12880030f0 msgr2=0x560059dbaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12880030f0 0x560059dbaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 -- 172.21.15.59:0/1298373466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560059d17170 con 0x560059d13240 2024-04-24T06:41:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.093+0000 7f12a3a00640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 0x560059db6d10 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x560059dbc4e0 tx=0x7f1294001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.094+0000 7f129be00640 1 -- 172.21.15.59:0/1298373466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294023020 con 0x560059d13240 2024-04-24T06:41:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.094+0000 7f129be00640 1 -- 172.21.15.59:0/1298373466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1294003c10 con 0x560059d13240 2024-04-24T06:41:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.094+0000 7f129be00640 1 -- 172.21.15.59:0/1298373466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940062e0 con 0x560059d13240 2024-04-24T06:41:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.094+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560059db7630 con 0x560059d13240 2024-04-24T06:41:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.094+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560059db7250 con 0x560059d13240 2024-04-24T06:41:43.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.096+0000 7f129be00640 1 -- 172.21.15.59:0/1298373466 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1294026020 con 0x560059d13240 2024-04-24T06:41:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.097+0000 7f129be00640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f128406b2d0 0x7f128406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.097+0000 7f129be00640 1 -- 172.21.15.59:0/1298373466 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f129408ce70 con 0x560059d13240 2024-04-24T06:41:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.098+0000 7f12a3000640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f128406b2d0 0x7f128406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.098+0000 7f12a731cf40 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x560059dbba90 0x560059e57330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.098+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x560059d2e470 con 0x560059dbba90 2024-04-24T06:41:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.099+0000 7f12a4400640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x560059dbba90 0x560059e57330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.099+0000 7f12a3000640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f128406b2d0 0x7f128406d790 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x560059ceb400 tx=0x7f1290051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.100+0000 7f12a4400640 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x560059dbba90 0x560059e57330 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a4400640 1 -- 172.21.15.59:0/1298373466 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f129c00e040 con 0x560059dbba90 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x560059dbba90 msgr2=0x560059e57330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x560059dbba90 0x560059e57330 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f128406b2d0 msgr2=0x7f128406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f128406b2d0 0x7f128406d790 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x560059ceb400 tx=0x7f1290051000 comp rx=0 tx=0).stop 2024-04-24T06:41:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 msgr2=0x560059db6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 --2- 172.21.15.59:0/1298373466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560059d13240 0x560059db6d10 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x560059dbc4e0 tx=0x7f1294001cf0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a4400640 1 -- 172.21.15.59:0/1298373466 reap_dead start 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 shutdown_connections 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.101+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 >> 172.21.15.59:0/1298373466 conn(0x560059d2ecc0 msgr2=0x560059d9f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.102+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 shutdown_connections 2024-04-24T06:41:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.102+0000 7f12a731cf40 1 -- 172.21.15.59:0/1298373466 wait complete. 2024-04-24T06:41:43.107 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-24T06:41:43.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-24T06:41:43.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.318+0000 7f7371000640 1 Processor -- start 2024-04-24T06:41:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.318+0000 7f7371000640 1 -- start start 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f7371000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f736c15ed20 0x7f736c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f7371000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c0d13e0 0x7f736c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f7371000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736c0b5da0 con 0x7f736c0b3600 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f7371000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f736c15f100 con 0x7f736c15ed20 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f7371000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f736c0d07d0 con 0x7f736c0d13e0 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f736b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f736c15ed20 0x7f736c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f736b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f736c15ed20 0x7f736c0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46516/0 (socket says 172.21.15.59:46516) 2024-04-24T06:41:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.319+0000 7f736b400640 1 -- 172.21.15.59:0/2828522712 learned_addr learned my addr 172.21.15.59:0/2828522712 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.320+0000 7f736b400640 1 -- 172.21.15.59:0/2828522712 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c0d13e0 msgr2=0x7f736c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.320+0000 7f736b400640 1 --2- 172.21.15.59:0/2828522712 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c0d13e0 0x7f736c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.320+0000 7f736b400640 1 -- 172.21.15.59:0/2828522712 >> v1:172.21.15.66:6789/0 conn(0x7f736c0b3600 legacy=0x7f736c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.320+0000 7f736b400640 1 -- 172.21.15.59:0/2828522712 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c0d0a20 con 0x7f736c15ed20 2024-04-24T06:41:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.321+0000 7f736b400640 1 --2- 172.21.15.59:0/2828522712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c15ed20 0x7f736c0d0ea0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f735c009e50 tx=0x7f735c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc2d67ec417cc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.321+0000 7f736a000640 1 -- 172.21.15.59:0/2828522712 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c036070 con 0x7f736c15ed20 2024-04-24T06:41:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.321+0000 7f736a000640 1 -- 172.21.15.59:0/2828522712 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735c031600 con 0x7f736c15ed20 2024-04-24T06:41:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.321+0000 7f736a000640 1 -- 172.21.15.59:0/2828522712 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c0319d0 con 0x7f736c15ed20 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 -- 172.21.15.59:0/2828522712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c15ed20 msgr2=0x7f736c0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 --2- 172.21.15.59:0/2828522712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c15ed20 0x7f736c0d0ea0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f735c009e50 tx=0x7f735c02a5b0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 -- 172.21.15.59:0/2828522712 shutdown_connections 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 --2- 172.21.15.59:0/2828522712 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c0d13e0 0x7f736c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 --2- 172.21.15.59:0/2828522712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c15ed20 0x7f736c0d0ea0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 -- 172.21.15.59:0/2828522712 >> 172.21.15.59:0/2828522712 conn(0x7f736c08bf90 msgr2=0x7f736c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 -- 172.21.15.59:0/2828522712 shutdown_connections 2024-04-24T06:41:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.322+0000 7f7371000640 1 -- 172.21.15.59:0/2828522712 wait complete. 2024-04-24T06:41:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.324+0000 7f7371000640 1 Processor -- start 2024-04-24T06:41:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.324+0000 7f7371000640 1 -- start start 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f736c0d13e0 0x7f736c0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c15ed20 0x7f736c0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f736c0d3490 con 0x7f736c0b3600 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f736c0d31b0 con 0x7f736c0d13e0 2024-04-24T06:41:43.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f7371000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f736c0b4620 con 0x7f736c15ed20 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f736b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f736b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46518/0 (socket says 172.21.15.59:46518) 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.325+0000 7f736b400640 1 -- 172.21.15.59:0/3490774473 learned_addr learned my addr 172.21.15.59:0/3490774473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 -- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c15ed20 msgr2=0x7f736c0b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c15ed20 0x7f736c0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 -- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f736c0d13e0 msgr2=0x7f736c0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f736c0d13e0 0x7f736c0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 -- 172.21.15.59:0/3490774473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1560e0 con 0x7f736c0b3600 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f736b400640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f735c009e50 tx=0x7f735c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f7363400640 1 -- 172.21.15.59:0/3490774473 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c041020 con 0x7f736c0b3600 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f7363400640 1 -- 172.21.15.59:0/3490774473 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735c007800 con 0x7f736c0b3600 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f7363400640 1 -- 172.21.15.59:0/3490774473 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c036040 con 0x7f736c0b3600 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c0b7ca0 con 0x7f736c0b3600 2024-04-24T06:41:43.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.326+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c1525a0 con 0x7f736c0b3600 2024-04-24T06:41:43.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.329+0000 7f7363400640 1 -- 172.21.15.59:0/3490774473 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f735c044020 con 0x7f736c0b3600 2024-04-24T06:41:43.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.329+0000 7f7363400640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 0x7f734806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.330+0000 7f7363400640 1 -- 172.21.15.59:0/3490774473 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f735c0aca40 con 0x7f736c0b3600 2024-04-24T06:41:43.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.330+0000 7f736aa00640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 0x7f734806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.330+0000 7f73755ae740 1 Processor -- start 2024-04-24T06:41:43.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.330+0000 7f73755ae740 1 -- start start 2024-04-24T06:41:43.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f736aa00640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 0x7f734806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f736c1f9040 tx=0x7f7354007990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04f63790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564e04ecd1f0 con 0x564e04f937e0 2024-04-24T06:41:43.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564e04e73350 con 0x564e04fb0e00 2024-04-24T06:41:43.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.331+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564e04ec91b0 con 0x564e04f93f20 2024-04-24T06:41:43.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46528/0 (socket says 172.21.15.59:46528) 2024-04-24T06:41:43.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 -- 172.21.15.59:0/3382060188 learned_addr learned my addr 172.21.15.59:0/3382060188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736aa00640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736b400640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04f63790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 -- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 msgr2=0x564e04f932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f932a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 -- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 msgr2=0x564e04f63790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04f63790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 -- 172.21.15.59:0/3382060188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e04ec7f50 con 0x564e04f937e0 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736b400640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04f63790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.332+0000 7f736be00640 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f73640097d0 tx=0x7f7364009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca8bb2fc0932f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.333+0000 7f735be00640 1 -- 172.21.15.59:0/3382060188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364015020 con 0x564e04f937e0 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.333+0000 7f735be00640 1 -- 172.21.15.59:0/3382060188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736400bad0 con 0x564e04f937e0 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.333+0000 7f735be00640 1 -- 172.21.15.59:0/3382060188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364011420 con 0x564e04f937e0 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.333+0000 7f73755ae740 1 -- 172.21.15.59:0/3382060188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 msgr2=0x564e04f685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.333+0000 7f73755ae740 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f73640097d0 tx=0x7f7364009c90 comp rx=0 tx=0).stop 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 -- 172.21.15.59:0/3382060188 shutdown_connections 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04f937e0 0x564e04f685f0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 --2- 172.21.15.59:0/3382060188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04f63790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 -- 172.21.15.59:0/3382060188 >> 172.21.15.59:0/3382060188 conn(0x564e04f65a00 msgr2=0x564e04fae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 -- 172.21.15.59:0/3382060188 shutdown_connections 2024-04-24T06:41:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 -- 172.21.15.59:0/3382060188 wait complete. 2024-04-24T06:41:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.334+0000 7f73755ae740 1 -- start start 2024-04-24T06:41:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.335+0000 7f73755ae740 1 Processor -- start 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f80bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564e04f67bb0 con 0x564e04fafc90 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564e04e86690 con 0x564e04fb0e00 2024-04-24T06:41:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564e04f66fa0 con 0x564e04f93f20 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f73755ae740 10 client.0 _ll_register_callbacks cb 0x564e04ec2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46536/0 (socket says 172.21.15.59:46536) 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 -- 172.21.15.59:0/1128554208 learned_addr learned my addr 172.21.15.59:0/1128554208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 -- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 msgr2=0x564e04f80bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f80bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 -- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 msgr2=0x564e04fc1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.336+0000 7f736be00640 1 -- 172.21.15.59:0/1128554208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e04e85920 con 0x564e04fafc90 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f736be00640 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f7364009cd0 tx=0x7f7364004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64cf8be958d3158 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f7358c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736401a830 con 0x564e04fafc90 2024-04-24T06:41:43.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f736401a9d0 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736400bad0 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f736402e050 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f73755ae740 10 client.9720 fetch_fsmap learned FSMap version 107 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.337+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x564e04ca97d0 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736400bad0 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f7364031020 con 0x564e04fafc90 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 10 client.9720 fetch_fsmap finished waiting for FSMap version 107 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 10 client.9720 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 20 client.9720 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95460 0x564e04f958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 4 client.9720 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 --> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e04edac10 con 0x564e04f95460 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.338+0000 7f736b400640 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95460 0x564e04f958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.339+0000 7f736b400640 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95460 0x564e04f958d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a749df04a87c7d96 server_cookie=74b5f372d82f408 in_seq=0 out_seq=0 2024-04-24T06:41:43.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.339+0000 7f7358c00640 10 client.9720 ms_handle_connect on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.339+0000 7f7358c00640 1 -- 172.21.15.59:0/1128554208 <== mds.0 v2:172.21.15.59:6835/3446399276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e04edac10 con 0x564e04f95460 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.339+0000 7f7358c00640 10 client.9720 handle_command_reply: tid=0 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.339+0000 7f73755ae740 1 client.9720 shutdown 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 msgr2=0x564e04fb0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f7364009cd0 tx=0x7f7364004840 comp rx=0 tx=0).stop 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 shutdown_connections 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95460 0x564e04f958d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fb0110 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 --2- 172.21.15.59:0/1128554208 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f7358c00640 0 client.9720 ms_handle_reset on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:43.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 >> 172.21.15.59:0/1128554208 conn(0x564e04f937e0 msgr2=0x564e04fae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 shutdown_connections 2024-04-24T06:41:43.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 1 -- 172.21.15.59:0/1128554208 wait complete. 2024-04-24T06:41:43.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.340+0000 7f73755ae740 20 client.9720 trim_cache size 0 max 16384 2024-04-24T06:41:43.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.357+0000 7f73755ae740 1 Processor -- start 2024-04-24T06:41:43.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.357+0000 7f73755ae740 1 -- start start 2024-04-24T06:41:43.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.357+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f68f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.357+0000 7f73755ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f73755ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f736c201070 con 0x564e04fafc90 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f736c0cc4d0 con 0x564e04fb0e00 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f736c0cb830 con 0x564e04f93f20 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46548/0 (socket says 172.21.15.59:46548) 2024-04-24T06:41:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 -- 172.21.15.59:0/3090636792 learned_addr learned my addr 172.21.15.59:0/3090636792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 -- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 msgr2=0x564e04f68f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f68f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 -- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 msgr2=0x564e04fc0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.358+0000 7f736aa00640 1 -- 172.21.15.59:0/3090636792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e04bdf3f0 con 0x564e04fafc90 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f736aa00640 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f7354005410 tx=0x7f7354001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986fbb713d39ab37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f735b400640 1 -- 172.21.15.59:0/3090636792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354012850 con 0x564e04fafc90 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f735b400640 1 -- 172.21.15.59:0/3090636792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7354012a10 con 0x564e04fafc90 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f735b400640 1 -- 172.21.15.59:0/3090636792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354012e00 con 0x564e04fafc90 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f73755ae740 1 -- 172.21.15.59:0/3090636792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 msgr2=0x564e04fc09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.359+0000 7f73755ae740 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f7354005410 tx=0x7f7354001090 comp rx=0 tx=0).stop 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 -- 172.21.15.59:0/3090636792 shutdown_connections 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04fb0e00 0x564e04fc0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04fc09e0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 --2- 172.21.15.59:0/3090636792 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04f93f20 0x564e04f68f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 -- 172.21.15.59:0/3090636792 >> 172.21.15.59:0/3090636792 conn(0x564e04f937e0 msgr2=0x564e04f809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 -- 172.21.15.59:0/3090636792 shutdown_connections 2024-04-24T06:41:43.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 -- 172.21.15.59:0/3090636792 wait complete. 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.360+0000 7f73755ae740 1 -- start start 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.361+0000 7f73755ae740 1 Processor -- start 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04f93f20 0x564e04f962d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04fb0e00 0x564e04fd0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f736c0cbe80 con 0x564e04fafc90 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564e04fd1d30 con 0x564e04f93f20 2024-04-24T06:41:43.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x564e04fc1020 con 0x564e04fb0e00 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f73755ae740 10 client.0 _ll_register_callbacks cb 0x7f736400ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46562/0 (socket says 172.21.15.59:46562) 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 -- 172.21.15.59:0/3592534120 learned_addr learned my addr 172.21.15.59:0/3592534120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 -- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04fb0e00 msgr2=0x564e04fd0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04fb0e00 0x564e04fd0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.362+0000 7f736aa00640 1 -- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04f93f20 msgr2=0x564e04f962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f736aa00640 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04f93f20 0x564e04f962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f736aa00640 1 -- 172.21.15.59:0/3592534120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e04b9f0f0 con 0x564e04fafc90 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f736aa00640 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f7354007d40 tx=0x7f7354011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0ab03051411732 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f7358c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354021660 con 0x564e04fafc90 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7f7354021800 con 0x564e04fafc90 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7354012a10 con 0x564e04fafc90 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f7354033050 con 0x564e04fafc90 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f73755ae740 10 client.9726 fetch_fsmap learned FSMap version 107 2024-04-24T06:41:43.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.363+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x564e04e92810 con 0x564e04fafc90 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7354012a10 con 0x564e04fafc90 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f7354036020 con 0x564e04fafc90 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 10 client.9726 fetch_fsmap finished waiting for FSMap version 107 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 10 client.9726 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 20 client.9726 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95960 0x564e04fd1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 4 client.9726 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 --> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e04ed4a80 con 0x564e04f95960 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.364+0000 7f736b400640 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95960 0x564e04fd1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.365+0000 7f736b400640 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95960 0x564e04fd1300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2b8fef2d445a5 server_cookie=390c7f0f62a266cc in_seq=0 out_seq=0 2024-04-24T06:41:43.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.365+0000 7f7358c00640 10 client.9726 ms_handle_connect on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:43.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f7358c00640 1 -- 172.21.15.59:0/3592534120 <== mds.0 v2:172.21.15.59:6835/3446399276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e04ed4a80 con 0x564e04f95960 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f7358c00640 10 client.9726 handle_command_reply: tid=0 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 client.9726 shutdown 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 msgr2=0x564e04f96810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f7354007d40 tx=0x7f7354011000 comp rx=0 tx=0).stop 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 shutdown_connections 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:6835/3446399276,v1:172.21.15.59:6837/3446399276] conn(0x564e04f95960 0x564e04fd1300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x564e04fb0e00 0x564e04fd0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564e04fafc90 0x564e04f96810 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 --2- 172.21.15.59:0/3592534120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564e04f93f20 0x564e04f962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.382+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 >> 172.21.15.59:0/3592534120 conn(0x564e04f937e0 msgr2=0x564e04f915f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.383+0000 7f7358c00640 0 client.9726 ms_handle_reset on v2:172.21.15.59:6835/3446399276 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.383+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 shutdown_connections 2024-04-24T06:41:43.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.383+0000 7f73755ae740 1 -- 172.21.15.59:0/3592534120 wait complete. 2024-04-24T06:41:43.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.383+0000 7f73755ae740 20 client.9726 trim_cache size 0 max 16384 2024-04-24T06:41:43.385 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "message": "", 2024-04-24T06:41:43.385 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0 2024-04-24T06:41:43.385 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 msgr2=0x7f734806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 0x7f734806edb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f736c1f9040 tx=0x7f7354007990 comp rx=0 tx=0).stop 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 msgr2=0x7f736c152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f735c009e50 tx=0x7f735c004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 shutdown_connections 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f734806c8f0 0x7f734806edb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f736c15ed20 0x7f736c0b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f736c0d13e0 0x7f736c0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 --2- 172.21.15.59:0/3490774473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f736c0b3600 0x7f736c152da0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 >> 172.21.15.59:0/3490774473 conn(0x7f736c08bf90 msgr2=0x7f736c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 shutdown_connections 2024-04-24T06:41:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.384+0000 7f7371000640 1 -- 172.21.15.59:0/3490774473 wait complete. 2024-04-24T06:41:43.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:41:43.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.598+0000 7fb0d1c00640 1 Processor -- start 2024-04-24T06:41:43.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.598+0000 7fb0d1c00640 1 -- start start 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.599+0000 7fb0d1c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fb0cc0af890 0x7fb0cc15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.599+0000 7fb0d1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0cc144a20 0x7fb0cc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.599+0000 7fb0d1c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0cc0b3720 con 0x7fb0cc0af4b0 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0d1c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc144ed0 con 0x7fb0cc144a20 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0d1c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc15f3f0 con 0x7fb0cc0af890 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0d1200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0cc144a20 0x7fb0cc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0d1200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0cc144a20 0x7fb0cc0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46574/0 (socket says 172.21.15.59:46574) 2024-04-24T06:41:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0d1200640 1 -- 172.21.15.59:0/1891632323 learned_addr learned my addr 172.21.15.59:0/1891632323 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.600+0000 7fb0c3400640 1 --2- 172.21.15.59:0/1891632323 >> v2:172.21.15.66:3300/0 conn(0x7fb0cc0af890 0x7fb0cc15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.601+0000 7fb0d1200640 1 -- 172.21.15.59:0/1891632323 >> v2:172.21.15.66:3300/0 conn(0x7fb0cc0af890 msgr2=0x7fb0cc15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.601+0000 7fb0d1200640 1 --2- 172.21.15.59:0/1891632323 >> v2:172.21.15.66:3300/0 conn(0x7fb0cc0af890 0x7fb0cc15ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.601+0000 7fb0d1200640 1 -- 172.21.15.59:0/1891632323 >> v1:172.21.15.66:6789/0 conn(0x7fb0cc0af4b0 legacy=0x7fb0cc0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.601+0000 7fb0d1200640 1 -- 172.21.15.59:0/1891632323 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc15f6d0 con 0x7fb0cc144a20 2024-04-24T06:41:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.602+0000 7fb0d1200640 1 --2- 172.21.15.59:0/1891632323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0d05c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0cdc00 tx=0x7fb0c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed4db6075cb8553 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.602+0000 7fb0cb400640 1 -- 172.21.15.59:0/1891632323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4036070 con 0x7fb0cc144a20 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.602+0000 7fb0cb400640 1 -- 172.21.15.59:0/1891632323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c4031600 con 0x7fb0cc144a20 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.602+0000 7fb0cb400640 1 -- 172.21.15.59:0/1891632323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40319d0 con 0x7fb0cc144a20 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 -- 172.21.15.59:0/1891632323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc144a20 msgr2=0x7fb0cc0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/1891632323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0d05c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0cdc00 tx=0x7fb0c402a6f0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 -- 172.21.15.59:0/1891632323 shutdown_connections 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/1891632323 >> v2:172.21.15.66:3300/0 conn(0x7fb0cc0af890 0x7fb0cc15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/1891632323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0d05c0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 -- 172.21.15.59:0/1891632323 >> 172.21.15.59:0/1891632323 conn(0x7fb0cc06bdf0 msgr2=0x7fb0cc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 -- 172.21.15.59:0/1891632323 shutdown_connections 2024-04-24T06:41:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.603+0000 7fb0d1c00640 1 -- 172.21.15.59:0/1891632323 wait complete. 2024-04-24T06:41:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.605+0000 7fb0d1c00640 1 Processor -- start 2024-04-24T06:41:43.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.605+0000 7fb0d1c00640 1 -- start start 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0cc0af4b0 0x7fb0cc0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0d0b00 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc15fc90 con 0x7fb0cc144a20 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb0cc0d0c80 con 0x7fb0cc0af4b0 2024-04-24T06:41:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46580/0 (socket says 172.21.15.59:46580) 2024-04-24T06:41:43.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.606+0000 7fb0d1200640 1 -- 172.21.15.59:0/2827118538 learned_addr learned my addr 172.21.15.59:0/2827118538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0c3400640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 -- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0cc0af4b0 msgr2=0x7fb0cc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0cc0af4b0 0x7fb0cc0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 -- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 msgr2=0x7fb0cc0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 -- 172.21.15.59:0/2827118538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc0ccd00 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0c3400640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.607+0000 7fb0d1200640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009f40 tx=0x7fb0c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.608+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4036050 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.608+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c4042070 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.608+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc0ccf90 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.608+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4009510 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.608+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc160220 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.609+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc04fd90 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.610+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0c4007bd0 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.611+0000 7fb0c8c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 0x7fb0a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.611+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5853+0+0 (secure 0 0 0) 0x7fb0c40acb10 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.611+0000 7fb0cbe00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 0x7fb0a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.612+0000 7fb0cbe00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 0x7fb0a806ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc003cc0 tx=0x7fb0bc0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.615+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0c4045360 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.863+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb0cc0cedb0 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.865 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:41:43.864+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:41:43.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.868+0000 7fb0c8c00640 1 -- 172.21.15.59:0/2827118538 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7fb0c407aea0 con 0x7fb0cc1f56f0 2024-04-24T06:41:43.869 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:41:43.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.872+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 msgr2=0x7fb0a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.872+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 0x7fb0a806ede0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc003cc0 tx=0x7fb0bc0079b0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.872+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 msgr2=0x7fb0cc160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.872+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009f40 tx=0x7fb0c4004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 shutdown_connections 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb0a806c920 0x7fb0a806ede0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0cc1f56f0 0x7fb0cc160a20 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc144a20 0x7fb0cc0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 --2- 172.21.15.59:0/2827118538 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb0cc0af4b0 0x7fb0cc0cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.873+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 >> 172.21.15.59:0/2827118538 conn(0x7fb0cc06bdf0 msgr2=0x7fb0cc06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.874+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 shutdown_connections 2024-04-24T06:41:43.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.874+0000 7fb0d1c00640 1 -- 172.21.15.59:0/2827118538 wait complete. 2024-04-24T06:41:43.888 INFO:tasks.ceph.mds.a.smithi059.stderr: -2126> 2024-04-24T06:41:29.039+0000 7f3a95200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:41:43.901 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-24T06:41:43.942 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:41:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.992+0000 7f52a3e61280 1 -- start start 2024-04-24T06:41:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.992+0000 7f52a3e61280 1 Processor -- start 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6827c9c00 0x55a682939080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683560800 0x55a682938b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6000 con 0x55a683560000 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6180 con 0x55a6827c9c00 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a6835c6300 con 0x55a683560800 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46622/0 (socket says 172.21.15.59:46622) 2024-04-24T06:41:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.993+0000 7f52a2000640 1 -- 172.21.15.59:0/2034179420 learned_addr learned my addr 172.21.15.59:0/2034179420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 -- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683560800 msgr2=0x55a682938b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683560800 0x55a682938b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 -- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6827c9c00 msgr2=0x55a682939080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6827c9c00 0x55a682939080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 -- 172.21.15.59:0/2034179420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6827fc780 con 0x55a683560000 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a2000640 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x55a6835e91a0 tx=0x55a6835626c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e018111d19adcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f529ee00640 1 -- 172.21.15.59:0/2034179420 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827ee680 con 0x55a683560000 2024-04-24T06:41:43.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f529ee00640 1 -- 172.21.15.59:0/2034179420 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6827f4e00 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f529ee00640 1 -- 172.21.15.59:0/2034179420 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827ee9c0 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6827fd680 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a6827fc000 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.994+0000 7f52a3e61280 4 waiting for MDS map... 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.995+0000 7f529ee00640 1 -- 172.21.15.59:0/2034179420 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a683684400 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.995+0000 7f529ee00640 1 -- 172.21.15.59:0/2034179420 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x55a682823180 con 0x55a683560000 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.995+0000 7f52a3e61280 4 Got MDS map 108 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.995+0000 7f52a3e61280 10 main: JournalTool::main 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.995+0000 7f52a3e61280 4 main: JournalTool: connecting to RADOS... 2024-04-24T06:41:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.996+0000 7f52a3e61280 1 Processor -- start 2024-04-24T06:41:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.996+0000 7f52a3e61280 1 -- start start 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.996+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a3e61280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6480 con 0x55a683693800 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6600 con 0x55a683693400 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a6835c6780 con 0x55a683693c00 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46634/0 (socket says 172.21.15.59:46634) 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 -- 172.21.15.59:0/3211674756 learned_addr learned my addr 172.21.15.59:0/3211674756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 -- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693c00 msgr2=0x55a682939b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 -- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693400 msgr2=0x55a682938000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 -- 172.21.15.59:0/3211674756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6827fc960 con 0x55a683693800 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.997+0000 7f52a2000640 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x55a6835e95c0 tx=0x55a683562660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323725fcbc69572 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f529c600640 1 -- 172.21.15.59:0/3211674756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827eeb60 con 0x55a683693800 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f529c600640 1 -- 172.21.15.59:0/3211674756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6827f4c40 con 0x55a683693800 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f529c600640 1 -- 172.21.15.59:0/3211674756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827ee820 con 0x55a683693800 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 -- 172.21.15.59:0/3211674756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 msgr2=0x55a682939600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x55a6835e95c0 tx=0x55a683562660 comp rx=0 tx=0).stop 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 -- 172.21.15.59:0/3211674756 shutdown_connections 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3211674756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 -- 172.21.15.59:0/3211674756 >> 172.21.15.59:0/3211674756 conn(0x55a683560400 msgr2=0x55a68354d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 -- 172.21.15.59:0/3211674756 shutdown_connections 2024-04-24T06:41:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.998+0000 7f52a3e61280 1 -- 172.21.15.59:0/3211674756 wait complete. 2024-04-24T06:41:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.999+0000 7f52a3e61280 1 Processor -- start 2024-04-24T06:41:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:43.999+0000 7f52a3e61280 1 -- start start 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6a80 con 0x55a683693400 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c6900 con 0x55a683693c00 2024-04-24T06:41:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a6835c7500 con 0x55a683693800 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46636/0 (socket says 172.21.15.59:46636) 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.000+0000 7f52a2a00640 1 -- 172.21.15.59:0/3133081775 learned_addr learned my addr 172.21.15.59:0/3133081775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 -- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693800 msgr2=0x55a682939600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 -- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693c00 msgr2=0x55a682939b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 -- 172.21.15.59:0/3133081775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6827fc780 con 0x55a683693400 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a2a00640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x55a68369b6e0 tx=0x55a68369ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f529a800640 1 -- 172.21.15.59:0/3133081775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827eeb60 con 0x55a683693400 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f529a800640 1 -- 172.21.15.59:0/3133081775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6827f41c0 con 0x55a683693400 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f529a800640 1 -- 172.21.15.59:0/3133081775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6827ee820 con 0x55a683693400 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a6827fc3c0 con 0x55a683693400 2024-04-24T06:41:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6827fd860 con 0x55a683693400 2024-04-24T06:41:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.001+0000 7f52a3e61280 4 main: JournalTool: resolving pool 16 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.003+0000 7f529a800640 1 -- 172.21.15.59:0/3133081775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55a6836f2c00 con 0x55a683693400 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.004+0000 7f529a800640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 0x55a68293a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.004+0000 7f529a800640 1 -- 172.21.15.59:0/3133081775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a6836f0e00 con 0x55a683693400 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.004+0000 7f52a3e61280 4 main: JournalTool: creating IoCtx.. 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.004+0000 7f52a3e61280 4 main: Executing for rank 0 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.004+0000 7f52a2000640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 0x55a68293a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.005+0000 7f52a2000640 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 0x55a68293a100 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55a6835e95c0 tx=0x55a683563f80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.005+0000 7f52a3e61280 1 -- start start 2024-04-24T06:41:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.005+0000 7f52a3e61280 1 Processor -- start 2024-04-24T06:41:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 0x55a68293a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683705400 0x55a68293ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683705800 0x55a68293b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c7800 con 0x55a683705000 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a6835c7980 con 0x55a683705800 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a3e61280 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a6835c7b00 con 0x55a683705400 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 0x55a68293a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 0x55a68293a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46648/0 (socket says 172.21.15.59:46648) 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.006+0000 7f52a2a00640 1 -- 172.21.15.59:0/85556417 learned_addr learned my addr 172.21.15.59:0/85556417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683705400 msgr2=0x55a68293ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683705400 0x55a68293ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683705800 msgr2=0x55a68293b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683705800 0x55a68293b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6827fda40 con 0x55a683705000 2024-04-24T06:41:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a2a00640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 0x55a68293a680 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x55a683737740 tx=0x55a68369f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9701f3a3a6765f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f5296c00640 1 -- 172.21.15.59:0/85556417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a682acd380 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f5296c00640 1 -- 172.21.15.59:0/85556417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6827f48c0 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f5296c00640 1 -- 172.21.15.59:0/85556417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a682acd520 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6827fd4a0 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a6827fc780 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f52a3e61280 4 waiting for MDS map... 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.007+0000 7f5296c00640 1 -- 172.21.15.59:0/85556417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a682b98400 con 0x55a683705000 2024-04-24T06:41:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.008+0000 7f5296c00640 1 -- 172.21.15.59:0/85556417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x55a682823500 con 0x55a683705000 2024-04-24T06:41:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.008+0000 7f52a3e61280 4 Got MDS map 108 2024-04-24T06:41:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.008+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a683705c00 0x55a68293b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.008+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682b9b400 con 0x55a683705c00 2024-04-24T06:41:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.008+0000 7f52a2000640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a683705c00 0x55a68293b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.009+0000 7f52a2000640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a683705c00 0x55a68293b700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=276d4bb4d355eb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.016+0000 7f52a2000640 1 -- 172.21.15.59:0/85556417 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v114'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a683552000 con 0x55a683705c00 2024-04-24T06:41:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.016+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682b9b800 con 0x55a683705c00 2024-04-24T06:41:44.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a2000640 1 -- 172.21.15.59:0/85556417 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v114'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a683552000 con 0x55a683705c00 2024-04-24T06:41:44.018 INFO:teuthology.orchestra.run.smithi059.stdout:writing EResetJournal entry 2024-04-24T06:41:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682b9bc00 con 0x55a683705c00 2024-04-24T06:41:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a682bb0000 0x55a682bb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bb0400 con 0x55a682bb0000 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a682bb0800 0x55a682bb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bb0c00 con 0x55a682bb0800 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bb1000 con 0x55a682bb0000 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a682bb1400 0x55a682bb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bb1800 con 0x55a682bb1400 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a682bb1c00 0x55a682bb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.018+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bbe000 con 0x55a682bb1c00 2024-04-24T06:41:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.019+0000 7f52a1600640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a682bb0000 0x55a682bb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.019+0000 7f52a2a00640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a682bb0800 0x55a682bb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.019+0000 7f52a2000640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a682bb1400 0x55a682bb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.019+0000 7f52a1600640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a682bb1c00 0x55a682bb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.020+0000 7f52a2a00640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a682bb0800 0x55a682bb6580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=6fb9c22cda10e2db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.020+0000 7f52a1600640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a682bb0000 0x55a682bb6000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=daba882cb19ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.020+0000 7f52a2000640 1 -- 172.21.15.59:0/85556417 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(3 200.00000001 [delete] v114'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a683552000 con 0x55a683705c00 2024-04-24T06:41:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.020+0000 7f52a2000640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a682bb1400 0x55a682bb6b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=270c2b208396c43f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.021+0000 7f52a1600640 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a682bb1c00 0x55a682bb7080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=4c074350b18cfb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.023+0000 7f52a2a00640 1 -- 172.21.15.59:0/85556417 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(5 200.00000003 [delete] v114'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a683552500 con 0x55a682bb0800 2024-04-24T06:41:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.024+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(4 200.00000002 [delete] v114'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a683552780 con 0x55a682bb0000 2024-04-24T06:41:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.024+0000 7f5294e00640 1 -- 172.21.15.59:0/85556417 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a682bb0400 con 0x55a683705c00 2024-04-24T06:41:44.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.024+0000 7f52a2000640 1 -- 172.21.15.59:0/85556417 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(7 200.00000005 [delete] v114'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a683552000 con 0x55a682bb1400 2024-04-24T06:41:44.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.025+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(6 200.00000004 [delete] v114'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a683552780 con 0x55a682bb0000 2024-04-24T06:41:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.026+0000 7f52a2000640 1 -- 172.21.15.59:0/85556417 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v114'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a683552000 con 0x55a683705c00 2024-04-24T06:41:44.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.032+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(8 200.00000006 [delete] v114'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a683552780 con 0x55a682bb1c00 2024-04-24T06:41:44.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.032+0000 7f52a3e61280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a683705c00 msgr2=0x55a68293b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a683705c00 0x55a68293b700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a682bb0800 msgr2=0x55a682bb6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a682bb0800 0x55a682bb6580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a682bb1400 msgr2=0x55a682bb6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a682bb1400 0x55a682bb6b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a682bb0000 msgr2=0x55a682bb6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a682bb0000 0x55a682bb6000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a682bb1c00 msgr2=0x55a682bb7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a682bb1c00 0x55a682bb7080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 reap_dead start 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 reap_dead start 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 msgr2=0x55a68293a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/85556417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683705000 0x55a68293a680 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x55a683737740 tx=0x55a68369f620 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a1600640 1 -- 172.21.15.59:0/85556417 reap_dead start 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 shutdown_connections 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 >> 172.21.15.59:0/85556417 conn(0x55a683704c00 msgr2=0x55a68354c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 shutdown_connections 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/85556417 wait complete. 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 msgr2=0x55a68293a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 0x55a68293a100 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55a6835e95c0 tx=0x55a683563f80 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 msgr2=0x55a682938000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.033+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x55a68369b6e0 tx=0x55a68369ecc0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 shutdown_connections 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a683704800 0x55a68293a100 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a683693c00 0x55a682939b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683693800 0x55a682939600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/3133081775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683693400 0x55a682938000 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 >> 172.21.15.59:0/3133081775 conn(0x55a683560400 msgr2=0x55a68354d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 shutdown_connections 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/3133081775 wait complete. 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 msgr2=0x55a682938580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x55a6835e91a0 tx=0x55a6835626c0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 shutdown_connections 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a683560800 0x55a682938b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a6827c9c00 0x55a682939080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 --2- 172.21.15.59:0/2034179420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a683560000 0x55a682938580 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.034+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 >> 172.21.15.59:0/2034179420 conn(0x55a6827c9000 msgr2=0x55a68354e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.035+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 shutdown_connections 2024-04-24T06:41:44.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.035+0000 7f52a3e61280 1 -- 172.21.15.59:0/2034179420 wait complete. 2024-04-24T06:41:44.041 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.139435, result: writing EResetJournal entry 2024-04-24T06:41:44.041 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-24T06:41:44.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.189+0000 7f6d7a527740 1 -- start start 2024-04-24T06:41:44.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.189+0000 7f6d7a527740 1 Processor -- start 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1616f3c00 0x55a1624c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162490400 0x55a1624c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a162502000 con 0x55a162490800 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a162502180 con 0x55a1616f3c00 2024-04-24T06:41:44.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.190+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a162502300 con 0x55a162490400 2024-04-24T06:41:44.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46672/0 (socket says 172.21.15.59:46672) 2024-04-24T06:41:44.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 -- 172.21.15.59:0/3401519869 learned_addr learned my addr 172.21.15.59:0/3401519869 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1616f3c00 0x55a1624c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d78600640 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162490400 0x55a1624c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 -- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162490400 msgr2=0x55a1624c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162490400 0x55a1624c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 -- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1616f3c00 msgr2=0x55a1624c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1616f3c00 0x55a1624c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 -- 172.21.15.59:0/3401519869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16171f680 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.191+0000 7f6d77200640 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x55a16251fc20 tx=0x55a162496000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72bc74bb9a638b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d74a00640 1 -- 172.21.15.59:0/3401519869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a1617189c0 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d74a00640 1 -- 172.21.15.59:0/3401519869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a16173c540 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d74a00640 1 -- 172.21.15.59:0/3401519869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a161718b60 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16171f2c0 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a16171eb40 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d7a527740 4 waiting for MDS map... 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d74a00640 1 -- 172.21.15.59:0/3401519869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a1625ca600 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d74a00640 1 -- 172.21.15.59:0/3401519869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x55a16174b180 con 0x55a162490800 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.192+0000 7f6d7a527740 4 Got MDS map 108 2024-04-24T06:41:44.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.193+0000 7f6d7a527740 4 datascan.main: Using metadata pool output 2024-04-24T06:41:44.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.193+0000 7f6d7a527740 4 datascan.main: connecting to RADOS... 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.193+0000 7f6d7a527740 1 Processor -- start 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 -- start start 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628000 0x55a1624c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a162502480 con 0x55a162628800 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a162502600 con 0x55a162628400 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.194+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a162502780 con 0x55a162628000 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46688/0 (socket says 172.21.15.59:46688) 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 -- 172.21.15.59:0/2174325037 learned_addr learned my addr 172.21.15.59:0/2174325037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77c00640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d78600640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628000 0x55a1624c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 -- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628000 msgr2=0x55a1624c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628000 0x55a1624c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 -- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 msgr2=0x55a1624c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77200640 1 -- 172.21.15.59:0/2174325037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16171e3c0 con 0x55a162628800 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.195+0000 7f6d77c00640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d77200640 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x55a1625df050 tx=0x55a162492ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037e1c5c8b1f4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d72200640 1 -- 172.21.15.59:0/2174325037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a161718820 con 0x55a162628800 2024-04-24T06:41:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d72200640 1 -- 172.21.15.59:0/2174325037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a16173cc40 con 0x55a162628800 2024-04-24T06:41:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d72200640 1 -- 172.21.15.59:0/2174325037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a1617189c0 con 0x55a162628800 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 -- 172.21.15.59:0/2174325037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 msgr2=0x55a1624c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x55a1625df050 tx=0x55a162492ae0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 -- 172.21.15.59:0/2174325037 shutdown_connections 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628800 0x55a1624c8100 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 --2- 172.21.15.59:0/2174325037 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628000 0x55a1624c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.196+0000 7f6d7a527740 1 -- 172.21.15.59:0/2174325037 >> 172.21.15.59:0/2174325037 conn(0x55a1625c5c00 msgr2=0x55a1617bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.197+0000 7f6d7a527740 1 -- 172.21.15.59:0/2174325037 shutdown_connections 2024-04-24T06:41:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.197+0000 7f6d7a527740 1 -- 172.21.15.59:0/2174325037 wait complete. 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.197+0000 7f6d7a527740 1 Processor -- start 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.197+0000 7f6d7a527740 1 -- start start 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 0x55a1624c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628800 0x55a1624c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a162503e00 con 0x55a162628000 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a162503c80 con 0x55a162628400 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.198+0000 7f6d7a527740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a162503b00 con 0x55a162628800 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 0x55a1624c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 0x55a1624c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46704/0 (socket says 172.21.15.59:46704) 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 learned_addr learned my addr 172.21.15.59:0/3447743366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628800 0x55a1624c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628800 msgr2=0x55a1624c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162628800 0x55a1624c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 msgr2=0x55a1624c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a162628400 0x55a1624c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16171f680 con 0x55a162628000 2024-04-24T06:41:44.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.199+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 0x55a1624c9180 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x55a1625dfda0 tx=0x55a1625e2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d70400640 1 -- 172.21.15.59:0/3447743366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a161718820 con 0x55a162628000 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d70400640 1 -- 172.21.15.59:0/3447743366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a16173d340 con 0x55a162628000 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d70400640 1 -- 172.21.15.59:0/3447743366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a1617189c0 con 0x55a162628000 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a16171f2c0 con 0x55a162628000 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16171fa40 con 0x55a162628000 2024-04-24T06:41:44.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.200+0000 7f6d7a527740 4 datascan.init: resolving metadata pool 16 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d70400640 1 -- 172.21.15.59:0/3447743366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55a162636c00 con 0x55a162628000 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d70400640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a161a6c000 0x55a1624c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d70400640 1 -- 172.21.15.59:0/3447743366 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a162640800 con 0x55a162628000 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d7a527740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d7a527740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d7a527740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a162628c00 0x55a1624c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629000 con 0x55a162628c00 2024-04-24T06:41:44.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.202+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a161a6c000 0x55a1624c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.203+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a162628c00 0x55a1624c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.203+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a161a6c000 0x55a1624c9700 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55a162494720 tx=0x55a162493f80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.204+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a162628c00 0x55a1624c8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.205+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(1 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.205+0000 7f6d71800640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a161a6c400 0x55a161a84000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.205+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6c800 con 0x55a161a6c400 2024-04-24T06:41:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.206+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a161a6c400 0x55a161a84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.206+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a161a6c400 0x55a161a84000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.207+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(2 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.207+0000 7f6d72200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a161a6d000 0x55a161a84580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.207+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d400 con 0x55a161a6d000 2024-04-24T06:41:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.207+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a161a6d000 0x55a161a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.208+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a161a6d000 0x55a161a84580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.208+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(3 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.208+0000 7f6d71800640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a161a6cc00 0x55a161a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.208+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6cc00 2024-04-24T06:41:44.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.209+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a161a6cc00 0x55a161a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.209+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a161a6cc00 0x55a161a84b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.210+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(4 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.210+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d800 con 0x55a162628c00 2024-04-24T06:41:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.211+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(5 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.211+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6d000 2024-04-24T06:41:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.211+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(6 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.211+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d800 con 0x55a161a6cc00 2024-04-24T06:41:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.211+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(7 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.212+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629000 con 0x55a162628c00 2024-04-24T06:41:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 3 ==== osd_op_reply(8 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a000 con 0x55a161a6cc00 2024-04-24T06:41:44.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(9 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a1625c5000 con 0x55a161a6d000 2024-04-24T06:41:44.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(10 [pgnls start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.213+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a400 con 0x55a161a6d000 2024-04-24T06:41:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.214+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(11 [pgnls start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.214+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a1625c5400 con 0x55a161a6cc00 2024-04-24T06:41:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.214+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(12 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.214+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a800 con 0x55a162628c00 2024-04-24T06:41:44.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.215+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 4 ==== osd_op_reply(13 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.215+0000 7f6d71800640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a1625c5000 0x55a161a85080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.215+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d400 con 0x55a1625c5000 2024-04-24T06:41:44.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.216+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a1625c5000 0x55a161a85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.218+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a1625c5000 0x55a161a85080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.219+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(14 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.219+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9ac00 con 0x55a161a6c400 2024-04-24T06:41:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.219+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(15 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.219+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6cc00 2024-04-24T06:41:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.220+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 5 ==== osd_op_reply(16 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.220+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d800 con 0x55a1625c5000 2024-04-24T06:41:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.221+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(17 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.221+0000 7f6d71800640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a162491c00 0x55a161a85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.221+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162491400 con 0x55a162491c00 2024-04-24T06:41:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.222+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a162491c00 0x55a161a85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.223+0000 7f6d77c00640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a162491c00 0x55a161a85600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.224+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(18 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.224+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6d000 2024-04-24T06:41:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.225+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 5 ==== osd_op_reply(19 [pgnls start_epoch 114 out=81b] v113'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.225+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a162629400 con 0x55a161a6d000 2024-04-24T06:41:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.226+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.226+0000 7f6d7a527740 10 datascan.operator(): handling object 1099511627778.0 2024-04-24T06:41:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.226+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a000 con 0x55a161a6d000 2024-04-24T06:41:44.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.229+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 7 ==== osd_op_reply(21 10000000002.00000000 [call] v114'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.229+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a800 con 0x55a162628c00 2024-04-24T06:41:44.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.230+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 5 ==== osd_op_reply(22 [pgnls start_epoch 114 out=49b] v110'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.230+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629000 con 0x55a162628c00 2024-04-24T06:41:44.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.231+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 6 ==== osd_op_reply(23 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.231+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b800 con 0x55a161a6d000 2024-04-24T06:41:44.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.231+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 8 ==== osd_op_reply(24 [pgnls start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.232+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629c00 con 0x55a162491c00 2024-04-24T06:41:44.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.233+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(25 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.233+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9bc00 con 0x55a161a6c400 2024-04-24T06:41:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.233+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(26 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.233+0000 7f6d72200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a1625c5400 0x55a161a86680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.233+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a400 con 0x55a1625c5400 2024-04-24T06:41:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.234+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a1625c5400 0x55a161a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.235+0000 7f6d77200640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a1625c5400 0x55a161a86680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.236+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(27 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.236+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aac000 con 0x55a161a6cc00 2024-04-24T06:41:44.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.237+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 6 ==== osd_op_reply(28 [pgnls start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.237+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a000 con 0x55a162491c00 2024-04-24T06:41:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.238+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(29 [pgnls start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.238+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aac400 con 0x55a161a6cc00 2024-04-24T06:41:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.239+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 7 ==== osd_op_reply(30 [pgnls start_epoch 114 out=81b] v110'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.239+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9ac00 con 0x55a161a6cc00 2024-04-24T06:41:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.239+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 8 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.239+0000 7f6d7a527740 10 datascan.operator(): handling object 1099511628279.0 2024-04-24T06:41:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.239+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b000 con 0x55a161a6cc00 2024-04-24T06:41:44.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.242+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 9 ==== osd_op_reply(32 100000001f7.00000000 [call] v114'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.242+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a161a9b400 0x55a1624c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.242+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6c800 con 0x55a161a9b400 2024-04-24T06:41:44.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.242+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a161a9b400 0x55a1624c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.243+0000 7f6d78600640 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a161a9b400 0x55a1624c7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.243+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(33 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a9b400 2024-04-24T06:41:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.243+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aac800 con 0x55a1625c5400 2024-04-24T06:41:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.244+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(34 [pgnls start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.244+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b800 con 0x55a161a6c400 2024-04-24T06:41:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.245+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(35 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.245+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aacc00 con 0x55a1625c5000 2024-04-24T06:41:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.246+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(36 [pgnls start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.246+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629400 con 0x55a161a6cc00 2024-04-24T06:41:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.247+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 10 ==== osd_op_reply(37 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.247+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aad000 con 0x55a161a6cc00 2024-04-24T06:41:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.247+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 11 ==== osd_op_reply(38 [pgnls start_epoch 114 out=49b] v110'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.247+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6d000 2024-04-24T06:41:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.248+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 9 ==== osd_op_reply(39 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.248+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aad400 con 0x55a161a6c400 2024-04-24T06:41:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.249+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(40 [pgnls start_epoch 114 out=81b] v110'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.249+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d400 con 0x55a161a6c400 2024-04-24T06:41:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.249+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.249+0000 7f6d7a527740 10 datascan.operator(): handling object 1099511627780.1 2024-04-24T06:41:44.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.249+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9bc00 con 0x55a1625c5400 2024-04-24T06:41:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.253+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(42 10000000004.00000000 [call] v114'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.253+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b800 con 0x55a161a6d000 2024-04-24T06:41:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.254+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 10 ==== osd_op_reply(43 [pgnls start_epoch 114 out=49b] v110'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.254+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aad800 con 0x55a1625c5000 2024-04-24T06:41:44.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.255+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(44 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.255+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162491400 con 0x55a162491c00 2024-04-24T06:41:44.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.256+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(45 [pgnls start_epoch 114 out=81b] v113'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.256+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a800 con 0x55a162491c00 2024-04-24T06:41:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.258+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.258+0000 7f6d7a527740 10 datascan.operator(): handling object 1099511628280.0 2024-04-24T06:41:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.258+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a400 con 0x55a162491c00 2024-04-24T06:41:44.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.263+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v114'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.263+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9ac00 con 0x55a162491c00 2024-04-24T06:41:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.264+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 7 ==== osd_op_reply(48 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.264+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a000 con 0x55a1625c5400 2024-04-24T06:41:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.265+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(49 [pgnls start_epoch 114 out=81b] v114'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.265+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a162629000 con 0x55a1625c5400 2024-04-24T06:41:44.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.266+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.266+0000 7f6d7a527740 10 datascan.operator(): handling object 1099511627780.0 2024-04-24T06:41:44.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.266+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d800 con 0x55a1625c5400 2024-04-24T06:41:44.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.270+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 6 ==== osd_op_reply(51 10000000004.00000000 [call] v114'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.270+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6c800 con 0x55a161a6c400 2024-04-24T06:41:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.271+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(52 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.271+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b800 con 0x55a162491c00 2024-04-24T06:41:44.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.272+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.5 v2:172.21.15.66:6816/79236266 8 ==== osd_op_reply(53 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a162491c00 2024-04-24T06:41:44.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.272+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aacc00 con 0x55a1625c5000 2024-04-24T06:41:44.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.273+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(54 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.273+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a800 con 0x55a162628c00 2024-04-24T06:41:44.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.275+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 7 ==== osd_op_reply(55 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.275+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aac000 con 0x55a162628c00 2024-04-24T06:41:44.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.276+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 8 ==== osd_op_reply(56 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.276+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a400 con 0x55a162628c00 2024-04-24T06:41:44.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.278+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 9 ==== osd_op_reply(57 [pgnls start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.278+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aac400 con 0x55a162628c00 2024-04-24T06:41:44.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.279+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 10 ==== osd_op_reply(58 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.279+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162491400 con 0x55a1625c5000 2024-04-24T06:41:44.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.280+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 6 ==== osd_op_reply(59 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.280+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aadc00 con 0x55a1625c5000 2024-04-24T06:41:44.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.281+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 7 ==== osd_op_reply(60 [pgnls start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.282+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6dc00 con 0x55a161a6d000 2024-04-24T06:41:44.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.282+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 <== osd.3 v2:172.21.15.59:6824/2800980920 11 ==== osd_op_reply(61 [pgnls start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480780 con 0x55a161a6d000 2024-04-24T06:41:44.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.283+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9b000 con 0x55a162628c00 2024-04-24T06:41:44.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.284+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.7 v2:172.21.15.66:6824/607700871 11 ==== osd_op_reply(62 [pgnls start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a162628c00 2024-04-24T06:41:44.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.284+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629c00 con 0x55a161a6cc00 2024-04-24T06:41:44.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.285+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 12 ==== osd_op_reply(63 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.285+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d400 con 0x55a1625c5000 2024-04-24T06:41:44.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.286+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 8 ==== osd_op_reply(64 [pgnls start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.286+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9bc00 con 0x55a1625c5000 2024-04-24T06:41:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.287+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 9 ==== osd_op_reply(65 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.287+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629400 con 0x55a161a9b400 2024-04-24T06:41:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.289+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(66 [pgnls start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a9b400 2024-04-24T06:41:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.289+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a000 con 0x55a161a6cc00 2024-04-24T06:41:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.290+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 13 ==== osd_op_reply(67 [pgnls start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.290+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a162629000 con 0x55a1625c5000 2024-04-24T06:41:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.291+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 10 ==== osd_op_reply(68 [pgnls start_epoch 114 out=49b] v110'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.291+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a9a800 con 0x55a1625c5400 2024-04-24T06:41:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.292+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 7 ==== osd_op_reply(69 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.292+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aad400 con 0x55a161a6cc00 2024-04-24T06:41:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.293+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 14 ==== osd_op_reply(70 [pgnls start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.293+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6c800 con 0x55a161a6c400 2024-04-24T06:41:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.295+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(71 [pgnls start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a161a6c400 2024-04-24T06:41:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.295+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aad800 con 0x55a1625c5000 2024-04-24T06:41:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.296+0000 7f6d78600640 1 -- 172.21.15.59:0/3447743366 <== osd.4 v2:172.21.15.66:6800/4254676376 11 ==== osd_op_reply(72 [pgnls start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480500 con 0x55a1625c5000 2024-04-24T06:41:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.296+0000 7f6d72200640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161a6d800 con 0x55a1625c5400 2024-04-24T06:41:44.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.297+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.6 v2:172.21.15.66:6808/280071528 8 ==== osd_op_reply(73 [pgnls start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a1625c5400 2024-04-24T06:41:44.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.297+0000 7f6d71800640 1 -- 172.21.15.59:0/3447743366 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a161aacc00 con 0x55a161a6cc00 2024-04-24T06:41:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d77200640 1 -- 172.21.15.59:0/3447743366 <== osd.0 v2:172.21.15.59:6808/2713443937 15 ==== osd_op_reply(74 [pgnls start_epoch 114 out=49b] v110'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a162480000 con 0x55a161a6cc00 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a161a6cc00 msgr2=0x55a161a84b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a161a6cc00 0x55a161a84b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a161a6c400 msgr2=0x55a161a84000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a161a6c400 0x55a161a84000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a161a9b400 msgr2=0x55a1624c7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a161a9b400 0x55a1624c7b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a161a6d000 msgr2=0x55a161a84580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a161a6d000 0x55a161a84580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.298+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a1625c5000 msgr2=0x55a161a85080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a1625c5000 0x55a161a85080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 reap_dead start 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 reap_dead start 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a162491c00 msgr2=0x55a161a85600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a162491c00 0x55a161a85600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a1625c5400 msgr2=0x55a161a86680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a1625c5400 0x55a161a86680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a162628c00 msgr2=0x55a1624c8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a162628c00 0x55a1624c8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a161a6c000 msgr2=0x55a1624c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a161a6c000 0x55a1624c9700 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55a162494720 tx=0x55a162493f80 comp rx=0 tx=0).stop 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 msgr2=0x55a1624c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3447743366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162628000 0x55a1624c9180 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x55a1625dfda0 tx=0x55a1625e2e40 comp rx=0 tx=0).stop 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.299+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 reap_dead start 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.300+0000 7f6d77c00640 1 -- 172.21.15.59:0/3447743366 reap_dead start 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.300+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 shutdown_connections 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.300+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 >> 172.21.15.59:0/3447743366 conn(0x55a1625c5c00 msgr2=0x55a1617bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.300+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 shutdown_connections 2024-04-24T06:41:44.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.300+0000 7f6d7a527740 1 -- 172.21.15.59:0/3447743366 wait complete. 2024-04-24T06:41:44.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.301+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 msgr2=0x55a1624c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.301+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x55a16251fc20 tx=0x55a162496000 comp rx=0 tx=0).stop 2024-04-24T06:41:44.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.301+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 shutdown_connections 2024-04-24T06:41:44.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.301+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a162490400 0x55a1624c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.301+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a162490800 0x55a1624c6b00 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.302+0000 7f6d7a527740 1 --2- 172.21.15.59:0/3401519869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a1616f3c00 0x55a1624c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.302+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 >> 172.21.15.59:0/3401519869 conn(0x55a1616f3000 msgr2=0x55a1617c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.302+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 shutdown_connections 2024-04-24T06:41:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.302+0000 7f6d7a527740 1 -- 172.21.15.59:0/3401519869 wait complete. 2024-04-24T06:41:44.313 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.271962, result: 2024-04-24T06:41:44.313 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-24T06:41:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.422+0000 7fdf38e2c740 1 -- start start 2024-04-24T06:41:44.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.422+0000 7fdf38e2c740 1 Processor -- start 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ec240c00 0x55a0ed014000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 0x55a0ed014b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a0ecfb0600 con 0x55a0ec241800 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a0ecfb0480 con 0x55a0ec241c00 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a0ecfb0300 con 0x55a0ec240c00 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46720/0 (socket says 172.21.15.59:46720) 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.423+0000 7fdf35600640 1 -- 172.21.15.59:0/3771135560 learned_addr learned my addr 172.21.15.59:0/3771135560 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf36a00640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 0x55a0ed014b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf36000640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ec240c00 0x55a0ed014000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf35600640 1 -- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ec240c00 msgr2=0x55a0ed014000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf35600640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ec240c00 0x55a0ed014000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf35600640 1 -- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 msgr2=0x55a0ed014b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf35600640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 0x55a0ed014b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf35600640 1 -- 172.21.15.59:0/3771135560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ec26d680 con 0x55a0ec241800 2024-04-24T06:41:44.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.424+0000 7fdf36a00640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 0x55a0ed014b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf35600640 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x55a0ed059860 tx=0x55a0ecfe4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b7948da237de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf32e00640 1 -- 172.21.15.59:0/3771135560 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec266680 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf32e00640 1 -- 172.21.15.59:0/3771135560 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0ec28a1c0 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf32e00640 1 -- 172.21.15.59:0/3771135560 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec266b60 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ec26d4a0 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a0ec26da40 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf38e2c740 4 waiting for MDS map... 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.425+0000 7fdf32e00640 1 -- 172.21.15.59:0/3771135560 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a0ed116600 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf32e00640 1 -- 172.21.15.59:0/3771135560 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x55a0ec299180 con 0x55a0ec241800 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf38e2c740 4 Got MDS map 108 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf38e2c740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf38e2c740 4 datascan.main: Using metadata pool output 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf38e2c740 4 datascan.main: connecting to RADOS... 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.426+0000 7fdf38e2c740 1 Processor -- start 2024-04-24T06:41:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 -- start start 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a0ecfb0180 con 0x55a0ed119c00 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a0ec2a0480 con 0x55a0ed119800 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.427+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a0ec2a0900 con 0x55a0ed119400 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46730/0 (socket says 172.21.15.59:46730) 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 -- 172.21.15.59:0/1166703206 learned_addr learned my addr 172.21.15.59:0/1166703206 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf36a00640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf36000640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 -- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 msgr2=0x55a0ed015080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 -- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119800 msgr2=0x55a0ed015600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 -- 172.21.15.59:0/1166703206 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ec26d860 con 0x55a0ed119c00 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf36a00640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.428+0000 7fdf35600640 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x55a0ed12ac90 tx=0x55a0ecfe1ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7830b32fa9fd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf30600640 1 -- 172.21.15.59:0/1166703206 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec266820 con 0x55a0ed119c00 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf30600640 1 -- 172.21.15.59:0/1166703206 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0ec28ac40 con 0x55a0ed119c00 2024-04-24T06:41:44.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf30600640 1 -- 172.21.15.59:0/1166703206 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec266680 con 0x55a0ed119c00 2024-04-24T06:41:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 -- 172.21.15.59:0/1166703206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 msgr2=0x55a0ed015b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x55a0ed12ac90 tx=0x55a0ecfe1ce0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 -- 172.21.15.59:0/1166703206 shutdown_connections 2024-04-24T06:41:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/1166703206 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 -- 172.21.15.59:0/1166703206 >> 172.21.15.59:0/1166703206 conn(0x55a0ed119000 msgr2=0x55a0ec30c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 -- 172.21.15.59:0/1166703206 shutdown_connections 2024-04-24T06:41:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.429+0000 7fdf38e2c740 1 -- 172.21.15.59:0/1166703206 wait complete. 2024-04-24T06:41:44.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.430+0000 7fdf38e2c740 1 Processor -- start 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.430+0000 7fdf38e2c740 1 -- start start 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a0ecfb1380 con 0x55a0ed119c00 2024-04-24T06:41:44.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a0ecfb1500 con 0x55a0ed119400 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf38e2c740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55a0ecfb0180 con 0x55a0ed119800 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf35600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46738/0 (socket says 172.21.15.59:46738) 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.431+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 learned_addr learned my addr 172.21.15.59:0/734388493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119800 msgr2=0x55a0ed015600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ed119800 0x55a0ed015600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119400 msgr2=0x55a0ed015080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ed119400 0x55a0ed015080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ec26d680 con 0x55a0ed119c00 2024-04-24T06:41:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x55a0ed12bf50 tx=0x55a0ed12ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf2e800640 1 -- 172.21.15.59:0/734388493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec266b60 con 0x55a0ed119c00 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf2e800640 1 -- 172.21.15.59:0/734388493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0ec28aa80 con 0x55a0ed119c00 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf2e800640 1 -- 172.21.15.59:0/734388493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0ec2664e0 con 0x55a0ed119c00 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.432+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a0ec26d4a0 con 0x55a0ed119c00 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.433+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ec26d2c0 con 0x55a0ed119c00 2024-04-24T06:41:44.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.433+0000 7fdf38e2c740 4 datascan.init: resolving metadata pool 16 2024-04-24T06:41:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.434+0000 7fdf2e800640 1 -- 172.21.15.59:0/734388493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55a0ed10cc00 con 0x55a0ed119c00 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf2e800640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 0x55a0ed016100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf2e800640 1 -- 172.21.15.59:0/734388493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55a0ed192600 con 0x55a0ed119c00 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf38e2c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf38e2c740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf38e2c740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a0ed181400 0x55a0ed016680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ed181800 con 0x55a0ed181400 2024-04-24T06:41:44.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.435+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 0x55a0ed016100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.436+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a0ed181400 0x55a0ed016680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.436+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 0x55a0ed016100 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55a0ecfe2150 tx=0x55a0ecfe0ae0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.436+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a0ed181400 0x55a0ed016680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.437+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv39 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.437+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a0ed181c00 0x55a0ed016c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.437+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ed181c00 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.437+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a0ed181c00 0x55a0ed016c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.438+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a0ed181c00 0x55a0ed016c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.439+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.439+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a0ec5de400 0x55a0ed017180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.439+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de800 con 0x55a0ec5de400 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.440+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a0ec5de400 0x55a0ed017180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.441+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a0ec5de400 0x55a0ed017180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.442+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.442+0000 7fdf2fc00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a0ed181800 0x55a0ec5f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.442+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de800 con 0x55a0ed181800 2024-04-24T06:41:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.442+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a0ed181800 0x55a0ec5f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.443+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a0ed181800 0x55a0ec5f0000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.443+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.443+0000 7fdf30600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a0ec5dfc00 0x55a0ec5f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.444+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.444+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a0ec5dfc00 0x55a0ec5f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.444+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a0ec5dfc00 0x55a0ec5f0580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.445+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.445+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df800 con 0x55a0ed181400 2024-04-24T06:41:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.445+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.445+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ec5de400 2024-04-24T06:41:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.446+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.447+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4800 con 0x55a0ec5dfc00 2024-04-24T06:41:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.447+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.447+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ed181400 2024-04-24T06:41:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.447+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.448+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4c00 con 0x55a0ec5de400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.449+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.449+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ecfdf400 con 0x55a0ed181400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.449+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.449+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ecfdf000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ed181400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.450+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4800 con 0x55a0ec5de400 2024-04-24T06:41:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.452+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.452+0000 7fdf2fc00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a0ec5e4000 0x55a0ec5f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.452+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df800 con 0x55a0ec5e4000 2024-04-24T06:41:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.452+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a0ec5e4000 0x55a0ec5f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.454+0000 7fdf36000640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a0ec5e4000 0x55a0ec5f0b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.455+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.455+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5400 con 0x55a0ed181800 2024-04-24T06:41:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.455+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.455+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ed181400 2024-04-24T06:41:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.456+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.456+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5800 con 0x55a0ec5e4000 2024-04-24T06:41:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.457+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.457+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4c00 con 0x55a0ed181c00 2024-04-24T06:41:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.458+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.458+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 114 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5c00 con 0x55a0ec5dfc00 2024-04-24T06:41:44.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.458+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 114 out=81b] v114'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.459+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5dec00 con 0x55a0ec5de400 2024-04-24T06:41:44.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.460+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.460+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc000 con 0x55a0ed181800 2024-04-24T06:41:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.460+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.460+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ec5dfc00 2024-04-24T06:41:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.461+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.461+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc400 con 0x55a0ed181400 2024-04-24T06:41:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.461+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.462+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4800 con 0x55a0ec5de400 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.463+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.463+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc800 con 0x55a0ec5dfc00 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.463+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.463+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ecfdf400 con 0x55a0ed181400 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.464+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.464+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fcc00 con 0x55a0ec5de400 2024-04-24T06:41:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.465+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.465+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5000 con 0x55a0ed181400 2024-04-24T06:41:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.466+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.466+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.466+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.466+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4c00 con 0x55a0ec5dfc00 2024-04-24T06:41:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.467+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.467+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc800 con 0x55a0ed181400 2024-04-24T06:41:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.468+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.468+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5c00 con 0x55a0ec5de400 2024-04-24T06:41:44.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.469+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.469+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc400 con 0x55a0ec5e4000 2024-04-24T06:41:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.470+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.470+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ed181800 2024-04-24T06:41:44.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.470+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.470+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd400 con 0x55a0ed181400 2024-04-24T06:41:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.471+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.471+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5800 con 0x55a0ec5e4000 2024-04-24T06:41:44.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.472+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.472+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df800 con 0x55a0ed181c00 2024-04-24T06:41:44.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.473+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.473+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ecfdf000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.474+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 114 out=49b] v114'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.474+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ec5de400 2024-04-24T06:41:44.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.475+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 114 out=49b] v110'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.475+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5400 con 0x55a0ec5de400 2024-04-24T06:41:44.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.476+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.476+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5dec00 con 0x55a0ec5dfc00 2024-04-24T06:41:44.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.477+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.477+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ed181c00 2024-04-24T06:41:44.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.478+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.478+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc000 con 0x55a0ed181800 2024-04-24T06:41:44.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.479+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.479+0000 7fdf2fc00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a0ec5e4800 0x55a0ec5f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.479+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ecfdf400 con 0x55a0ec5e4800 2024-04-24T06:41:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.480+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a0ec5e4800 0x55a0ec5f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.481+0000 7fdf35600640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a0ec5e4800 0x55a0ec5f1080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.482+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ec5e4800 2024-04-24T06:41:44.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.482+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fcc00 con 0x55a0ed181400 2024-04-24T06:41:44.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.483+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 12 ==== osd_op_reply(47 [pgnls-filter start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.483+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de800 con 0x55a0ed181c00 2024-04-24T06:41:44.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.484+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.484+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc800 con 0x55a0ed181400 2024-04-24T06:41:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.485+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 13 ==== osd_op_reply(49 [pgnls-filter start_epoch 114 out=81b] v114'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.485+0000 7fdf38e2c740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-24T06:41:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.485+0000 7fdf38e2c740 10 datascan.operator(): handling object 100000001f7.0 2024-04-24T06:41:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.485+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df400 con 0x55a0ed181400 2024-04-24T06:41:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.486+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 14 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.486+0000 7fdf38e2c740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-24T06:41:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.486+0000 7fdf38e2c740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-24T06:41:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.486+0000 7fdf38e2c740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-24T06:41:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.486+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5800 con 0x55a0ec5dfc00 2024-04-24T06:41:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.487+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.487+0000 7fdf38e2c740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-24T06:41:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.487+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a0ecfdf000 0x55a0ed017700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.487+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc400 con 0x55a0ecfdf000 2024-04-24T06:41:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.488+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a0ecfdf000 0x55a0ed017700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.489+0000 7fdf36a00640 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a0ecfdf000 0x55a0ed017700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.489+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ecfdf000 2024-04-24T06:41:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.489+0000 7fdf38e2c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-24T06:41:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-24T06:41:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 20 datascan.get_frag_of: frag is * 2024-04-24T06:41:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd400 con 0x55a0ec5dfc00 2024-04-24T06:41:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-24T06:41:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-24T06:41:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.490+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fcc00 con 0x55a0ec5dfc00 2024-04-24T06:41:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.491+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.491+0000 7fdf38e2c740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-24T06:41:44.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.491+0000 7fdf38e2c740 20 datascan.read_dentry: bravo_head not found in result 2024-04-24T06:41:44.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.491+0000 7fdf38e2c740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-24T06:41:44.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.491+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc800 con 0x55a0ec5dfc00 2024-04-24T06:41:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.494+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v114'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.494+0000 7fdf38e2c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-24T06:41:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.494+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd000 con 0x55a0ecfdf000 2024-04-24T06:41:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.495+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ecfdf000 2024-04-24T06:41:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.495+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd800 con 0x55a0ec5e4800 2024-04-24T06:41:44.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.496+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ec5e4800 2024-04-24T06:41:44.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.496+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ed180c00 con 0x55a0ed181800 2024-04-24T06:41:44.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.497+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.497+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fdc00 con 0x55a0ec5e4000 2024-04-24T06:41:44.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.498+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.498+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ed180800 con 0x55a0ed181400 2024-04-24T06:41:44.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.499+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.499+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5000 con 0x55a0ed181400 2024-04-24T06:41:44.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.500+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 16 ==== osd_op_reply(61 [pgnls-filter start_epoch 114 out=49b] v110'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.500+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.501+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.501+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5c00 con 0x55a0ed181800 2024-04-24T06:41:44.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.502+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 114 out=49b] v110'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.503+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df800 con 0x55a0ec5dfc00 2024-04-24T06:41:44.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.503+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 114 out=49b] v110'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.503+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ec5e4000 2024-04-24T06:41:44.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.505+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.505+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5dec00 con 0x55a0ed181c00 2024-04-24T06:41:44.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.506+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(66 [pgnls-filter start_epoch 114 out=49b] v114'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.506+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df800 con 0x55a0ed181c00 2024-04-24T06:41:44.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.507+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.507+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de800 con 0x55a0ec5e4800 2024-04-24T06:41:44.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.509+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 114 out=49b] v114'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ec5e4800 2024-04-24T06:41:44.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.509+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc400 con 0x55a0ed181800 2024-04-24T06:41:44.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.510+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.510+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5400 con 0x55a0ed181c00 2024-04-24T06:41:44.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.511+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.5 v2:172.21.15.66:6816/79236266 8 ==== osd_op_reply(70 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ed181c00 2024-04-24T06:41:44.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.511+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd400 con 0x55a0ec5e4000 2024-04-24T06:41:44.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.513+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.513+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4c00 con 0x55a0ec5de400 2024-04-24T06:41:44.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.514+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.514+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fcc00 con 0x55a0ec5de400 2024-04-24T06:41:44.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.515+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.515+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4400 con 0x55a0ec5de400 2024-04-24T06:41:44.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.516+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.516+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc800 con 0x55a0ec5de400 2024-04-24T06:41:44.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.518+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.518+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df400 con 0x55a0ec5e4000 2024-04-24T06:41:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.519+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.519+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd000 con 0x55a0ec5e4000 2024-04-24T06:41:44.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.520+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 114 out=49b] v110'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.521+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5df000 con 0x55a0ec5dfc00 2024-04-24T06:41:44.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.521+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.3 v2:172.21.15.59:6824/2800980920 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 114 out=49b] v110'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ec5dfc00 2024-04-24T06:41:44.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.522+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5800 con 0x55a0ec5de400 2024-04-24T06:41:44.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.523+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.7 v2:172.21.15.66:6824/607700871 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 114 out=49b] v110'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5de400 2024-04-24T06:41:44.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.523+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de000 con 0x55a0ed181400 2024-04-24T06:41:44.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.524+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 17 ==== osd_op_reply(80 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.524+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fdc00 con 0x55a0ec5e4000 2024-04-24T06:41:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.525+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.525+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5000 con 0x55a0ec5e4000 2024-04-24T06:41:44.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.527+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.527+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd400 con 0x55a0ecfdf000 2024-04-24T06:41:44.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.528+0000 7fdf36a00640 1 -- 172.21.15.59:0/734388493 <== osd.2 v2:172.21.15.59:6809/1388377934 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 114 out=49b] v110'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce500 con 0x55a0ecfdf000 2024-04-24T06:41:44.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.528+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e4c00 con 0x55a0ed181400 2024-04-24T06:41:44.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.529+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 18 ==== osd_op_reply(84 [pgnls-filter start_epoch 114 out=49b] v110'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.529+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc000 con 0x55a0ec5e4000 2024-04-24T06:41:44.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.530+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 114 out=49b] v110'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.530+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5e5c00 con 0x55a0ec5e4800 2024-04-24T06:41:44.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.531+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ec5e4800 2024-04-24T06:41:44.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.531+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fd800 con 0x55a0ed181400 2024-04-24T06:41:44.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.533+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 19 ==== osd_op_reply(87 [pgnls-filter start_epoch 114 out=49b] v110'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.533+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ed180800 con 0x55a0ed181800 2024-04-24T06:41:44.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.534+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.1 v2:172.21.15.59:6800/2375715406 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 114 out=49b] v110'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ed181800 2024-04-24T06:41:44.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.534+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5fc400 con 0x55a0ec5e4000 2024-04-24T06:41:44.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.535+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.4 v2:172.21.15.66:6800/4254676376 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 114 out=49b] v110'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ec5e4000 2024-04-24T06:41:44.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.535+0000 7fdf2fc00640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ed180c00 con 0x55a0ec5e4800 2024-04-24T06:41:44.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.536+0000 7fdf35600640 1 -- 172.21.15.59:0/734388493 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 114 out=49b] v110'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce780 con 0x55a0ec5e4800 2024-04-24T06:41:44.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.536+0000 7fdf30600640 1 -- 172.21.15.59:0/734388493 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 114 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55a0ec5de800 con 0x55a0ed181400 2024-04-24T06:41:44.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.537+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 <== osd.0 v2:172.21.15.59:6808/2713443937 20 ==== osd_op_reply(91 [pgnls-filter start_epoch 114 out=49b] v110'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0ecfce000 con 0x55a0ed181400 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a0ed181400 msgr2=0x55a0ed016680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55a0ed181400 0x55a0ed016680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a0ed181800 msgr2=0x55a0ec5f0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55a0ed181800 0x55a0ec5f0000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a0ecfdf000 msgr2=0x55a0ed017700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55a0ecfdf000 0x55a0ed017700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a0ec5dfc00 msgr2=0x55a0ec5f0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55a0ec5dfc00 0x55a0ec5f0580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a0ec5e4000 msgr2=0x55a0ec5f0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55a0ec5e4000 0x55a0ec5f0b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a0ed181c00 msgr2=0x55a0ed016c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55a0ed181c00 0x55a0ed016c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a0ec5e4800 msgr2=0x55a0ec5f1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55a0ec5e4800 0x55a0ec5f1080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a0ec5de400 msgr2=0x55a0ed017180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55a0ec5de400 0x55a0ed017180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 reap_dead start 2024-04-24T06:41:44.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 reap_dead start 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 reap_dead start 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf36000640 1 -- 172.21.15.59:0/734388493 reap_dead start 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 msgr2=0x55a0ed016100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 0x55a0ed016100 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55a0ecfe2150 tx=0x55a0ecfe0ae0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 msgr2=0x55a0ed015b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.538+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x55a0ed12bf50 tx=0x55a0ed12ec00 comp rx=0 tx=0).stop 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 shutdown_connections 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55a0ed181000 0x55a0ed016100 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/734388493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ed119c00 0x55a0ed015b80 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 >> 172.21.15.59:0/734388493 conn(0x55a0ed119000 msgr2=0x55a0ec30c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 shutdown_connections 2024-04-24T06:41:44.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.539+0000 7fdf38e2c740 1 -- 172.21.15.59:0/734388493 wait complete. 2024-04-24T06:41:44.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.540+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 msgr2=0x55a0ed014580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.540+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x55a0ed059860 tx=0x55a0ecfe4120 comp rx=0 tx=0).stop 2024-04-24T06:41:44.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 shutdown_connections 2024-04-24T06:41:44.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a0ec241c00 0x55a0ed014b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a0ec241800 0x55a0ed014580 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 --2- 172.21.15.59:0/3771135560 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55a0ec240c00 0x55a0ed014000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 >> 172.21.15.59:0/3771135560 conn(0x55a0ec241000 msgr2=0x55a0ec30d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 shutdown_connections 2024-04-24T06:41:44.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.541+0000 7fdf38e2c740 1 -- 172.21.15.59:0/3771135560 wait complete. 2024-04-24T06:41:44.550 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.236971, result: 2024-04-24T06:41:44.550 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-24T06:41:44.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 -- start start 2024-04-24T06:41:44.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 Processor -- start 2024-04-24T06:41:44.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144800 0x55dff317b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 0x55dff317ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.666+0000 7f3638c30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dff3182000 con 0x55dff3145400 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff3182180 con 0x55dff3145c00 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55dff3182300 con 0x55dff3144800 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46768/0 (socket says 172.21.15.59:46768) 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 -- 172.21.15.59:0/2760764849 learned_addr learned my addr 172.21.15.59:0/2760764849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3635e00640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144800 0x55dff317b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3635400640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 0x55dff317ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 -- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144800 msgr2=0x55dff317b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144800 0x55dff317b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 -- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 msgr2=0x55dff317ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 0x55dff317ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.667+0000 7f3636800640 1 -- 172.21.15.59:0/2760764849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff23d3680 con 0x55dff3145400 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3635400640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 0x55dff317ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3636800640 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x55dff31cd9b0 tx=0x55dff3147f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d70ad51ab7eee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3632c00640 1 -- 172.21.15.59:0/2760764849 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc9c0 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3632c00640 1 -- 172.21.15.59:0/2760764849 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55dff23f0c40 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3632c00640 1 -- 172.21.15.59:0/2760764849 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc680 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff23d3860 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55dff23d30e0 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.668+0000 7f3638c30740 4 waiting for MDS map... 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.669+0000 7f3632c00640 1 -- 172.21.15.59:0/2760764849 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55dff3184200 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.669+0000 7f3632c00640 1 -- 172.21.15.59:0/2760764849 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x55dff23ff180 con 0x55dff3145400 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.669+0000 7f3638c30740 4 Got MDS map 108 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.669+0000 7f3638c30740 4 datascan.main: Using metadata pool output 2024-04-24T06:41:44.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.669+0000 7f3638c30740 4 datascan.main: connecting to RADOS... 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.670+0000 7f3638c30740 1 Processor -- start 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.670+0000 7f3638c30740 1 -- start start 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d8000 0x55dff317c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dff3182480 con 0x55dff3287c00 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff3182600 con 0x55dff32d8000 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55dff3182780 con 0x55dff3144400 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46784/0 (socket says 172.21.15.59:46784) 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 -- 172.21.15.59:0/3698303175 learned_addr learned my addr 172.21.15.59:0/3698303175 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 -- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144400 msgr2=0x55dff317b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 -- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d8000 msgr2=0x55dff317c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d8000 0x55dff317c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.671+0000 7f3635e00640 1 -- 172.21.15.59:0/3698303175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff23d34a0 con 0x55dff3287c00 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.672+0000 7f3635e00640 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x55dff328cb10 tx=0x55dff328ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7294e06562ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.672+0000 7f3630400640 1 -- 172.21.15.59:0/3698303175 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc9c0 con 0x55dff3287c00 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.672+0000 7f3630400640 1 -- 172.21.15.59:0/3698303175 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55dff23f01c0 con 0x55dff3287c00 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.672+0000 7f3630400640 1 -- 172.21.15.59:0/3698303175 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc680 con 0x55dff3287c00 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 -- 172.21.15.59:0/3698303175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 msgr2=0x55dff317bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x55dff328cb10 tx=0x55dff328ec00 comp rx=0 tx=0).stop 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 -- 172.21.15.59:0/3698303175 shutdown_connections 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d8000 0x55dff317c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 --2- 172.21.15.59:0/3698303175 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 -- 172.21.15.59:0/3698303175 >> 172.21.15.59:0/3698303175 conn(0x55dff3145800 msgr2=0x55dff2475800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 -- 172.21.15.59:0/3698303175 shutdown_connections 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 -- 172.21.15.59:0/3698303175 wait complete. 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.673+0000 7f3638c30740 1 Processor -- start 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 -- start start 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d9400 0x55dff317c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dff3183e00 con 0x55dff3144400 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55dff3183c80 con 0x55dff32d9400 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.674+0000 7f3638c30740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x55dff3183b00 con 0x55dff3287c00 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46800/0 (socket says 172.21.15.59:46800) 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 learned_addr learned my addr 172.21.15.59:0/3312095917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3287c00 msgr2=0x55dff317bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3287c00 0x55dff317bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d9400 msgr2=0x55dff317c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff32d9400 0x55dff317c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.675+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff23d3680 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x55dff328dda0 tx=0x55dff328ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f362e600640 1 -- 172.21.15.59:0/3312095917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc9c0 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f362e600640 1 -- 172.21.15.59:0/3312095917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55dff23f0a80 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f362e600640 1 -- 172.21.15.59:0/3312095917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dff23cc680 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dff23d32c0 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff23d3a40 con 0x55dff3144400 2024-04-24T06:41:44.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.676+0000 7f3638c30740 4 datascan.init: resolving metadata pool 16 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.678+0000 7f362e600640 1 -- 172.21.15.59:0/3312095917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55dff32e4c00 con 0x55dff3144400 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.678+0000 7f362e600640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 0x55dff317c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.678+0000 7f362e600640 1 -- 172.21.15.59:0/3312095917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x55dff32df000 con 0x55dff3144400 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.678+0000 7f3638c30740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.678+0000 7f3638c30740 4 datascan.main: resolving metadata pool 16 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.679+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55dff32d9000 0x55dff317cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.679+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 0x55dff317c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.679+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8c00 con 0x55dff32d9000 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.679+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55dff32d9000 0x55dff317cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.679+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 0x55dff317c680 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55dff328cb10 tx=0x55dff3146900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.680+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55dff32d9000 0x55dff317cc00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.680+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== osd_op_reply(1 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.680+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55dff32d8c00 0x55dff317a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.680+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f4800 con 0x55dff32d8c00 2024-04-24T06:41:44.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.680+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55dff32d8c00 0x55dff317a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.681+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55dff32d8c00 0x55dff317a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.681+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== osd_op_reply(2 [pgnls start_epoch 114 out=177b] v113'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.681+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f4800 con 0x55dff32d9000 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== osd_op_reply(3 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f4800 con 0x55dff32d9000 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 3 ==== osd_op_reply(4 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55dff32f4800 0x55dff2730000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5000 con 0x55dff32f4800 2024-04-24T06:41:44.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.682+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55dff32f4800 0x55dff2730000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.683+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55dff32f4800 0x55dff2730000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.683+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== osd_op_reply(5 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.683+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5000 con 0x55dff32d9000 2024-04-24T06:41:44.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.684+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 4 ==== osd_op_reply(6 [pgnls start_epoch 114 out=149b] v114'45 uv45 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.684+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55dff32f5000 0x55dff2730580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.684+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5400 con 0x55dff32f5000 2024-04-24T06:41:44.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.685+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55dff32f5000 0x55dff2730580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.686+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55dff32f5000 0x55dff2730580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.687+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== osd_op_reply(7 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.687+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5400 con 0x55dff32d8c00 2024-04-24T06:41:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.688+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== osd_op_reply(8 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.688+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55dff32f5400 0x55dff2731080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.688+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286400 con 0x55dff32f5400 2024-04-24T06:41:44.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.689+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55dff32f5400 0x55dff2731080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.690+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55dff32f5400 0x55dff2731080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.691+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== osd_op_reply(9 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32f5400 2024-04-24T06:41:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.691+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55dff3286400 0x55dff2731600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.691+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff3286400 2024-04-24T06:41:44.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.692+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55dff3286400 0x55dff2731600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.693+0000 7f3635e00640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55dff3286400 0x55dff2731600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.694+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== osd_op_reply(10 [pgnls start_epoch 114 out=81b] v113'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.694+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff32d8c00 2024-04-24T06:41:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.694+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 3 ==== osd_op_reply(11 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.694+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff32f4800 2024-04-24T06:41:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.695+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== osd_op_reply(12 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.695+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff3286400 2024-04-24T06:41:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.696+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== osd_op_reply(13 [pgnls start_epoch 114 out=77b] v113'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.696+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff32f4800 2024-04-24T06:41:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.697+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 3 ==== osd_op_reply(14 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.697+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286800 con 0x55dff32d8c00 2024-04-24T06:41:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.697+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 4 ==== osd_op_reply(15 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.697+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55dff3286800 0x55dff2731b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.697+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286c00 con 0x55dff3286800 2024-04-24T06:41:44.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.698+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55dff3286800 0x55dff2731b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.699+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55dff3286800 0x55dff2731b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.700+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== osd_op_reply(16 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff3286800 2024-04-24T06:41:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.700+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286c00 con 0x55dff32d8c00 2024-04-24T06:41:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.701+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 5 ==== osd_op_reply(17 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.701+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff3286c00 con 0x55dff32f5000 2024-04-24T06:41:44.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.702+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== osd_op_reply(18 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.702+0000 7f3635400640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55dff3286c00 0x55dff2730b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.702+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff3286c00 2024-04-24T06:41:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.702+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55dff3286c00 0x55dff2730b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.703+0000 7f3636800640 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55dff3286c00 0x55dff2730b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.703+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== osd_op_reply(19 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.703+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32f5400 2024-04-24T06:41:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.704+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== osd_op_reply(20 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32f5400 2024-04-24T06:41:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.704+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32d9000 2024-04-24T06:41:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.705+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 5 ==== osd_op_reply(21 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.705+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32d9000 2024-04-24T06:41:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.705+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 6 ==== osd_op_reply(22 [pgnls start_epoch 114 out=97b] v113'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.705+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff3286400 2024-04-24T06:41:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.707+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 3 ==== osd_op_reply(23 [pgnls start_epoch 114 out=74b] v109'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.707+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32d8c00 2024-04-24T06:41:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.707+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 6 ==== osd_op_reply(24 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.707+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32f5000 2024-04-24T06:41:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.708+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 3 ==== osd_op_reply(25 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.708+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff3286c00 2024-04-24T06:41:44.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.709+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== osd_op_reply(26 [pgnls start_epoch 114 out=98b] v113'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.709+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff3286c00 2024-04-24T06:41:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.710+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 3 ==== osd_op_reply(27 [pgnls start_epoch 114 out=76b] v114'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.710+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32f5000 2024-04-24T06:41:44.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.711+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 4 ==== osd_op_reply(28 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.711+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32f4800 2024-04-24T06:41:44.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.711+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 4 ==== osd_op_reply(29 [pgnls start_epoch 114 out=105b] v114'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.712+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32d8c00 2024-04-24T06:41:44.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.712+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 7 ==== osd_op_reply(30 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.712+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32d9000 2024-04-24T06:41:44.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.713+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 7 ==== osd_op_reply(31 [pgnls start_epoch 114 out=73b] v114'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.713+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5800 con 0x55dff32f5000 2024-04-24T06:41:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.714+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 5 ==== osd_op_reply(32 [pgnls start_epoch 114 out=101b] v114'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.714+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-24T06:41:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.714+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8400 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.715+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8000 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.716+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.716+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-24T06:41:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.716+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5c00 con 0x55dff32d8c00 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752000 con 0x55dff32d8c00 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-24T06:41:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.717+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752400 con 0x55dff32d9000 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.718+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv40 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.718+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-24T06:41:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.718+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752800 con 0x55dff32d8c00 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.719+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.719+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.719+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752c00 con 0x55dff3286400 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.720+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.720+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.720+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.720+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-24T06:41:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.720+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2753000 con 0x55dff32d8c00 2024-04-24T06:41:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.721+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.721+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-24T06:41:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.721+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2753400 con 0x55dff32d9000 2024-04-24T06:41:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-24T06:41:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2753800 con 0x55dff32d8c00 2024-04-24T06:41:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.722+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-24T06:41:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-24T06:41:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-24T06:41:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2753c00 con 0x55dff3286c00 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.723+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752000 con 0x55dff32f4800 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.724+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.724+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-24T06:41:44.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.724+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752800 con 0x55dff32f4800 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 20 datascan.scan_links: injected ino detected 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-24T06:41:44.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.725+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32f5c00 con 0x55dff32f5000 2024-04-24T06:41:44.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.726+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.726+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.727+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 10 ==== osd_op_reply(48 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.727+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.728+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 16 ==== osd_op_reply(49 [pgnls start_epoch 114 out=177b] v113'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.728+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.729+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 11 ==== osd_op_reply(50 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.729+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.729+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 12 ==== osd_op_reply(51 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.729+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f4800 2024-04-24T06:41:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.730+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 7 ==== osd_op_reply(52 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.730+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.731+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 13 ==== osd_op_reply(53 [pgnls start_epoch 114 out=149b] v114'45 uv45 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.731+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5000 2024-04-24T06:41:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.732+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 7 ==== osd_op_reply(54 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.732+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.733+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 17 ==== osd_op_reply(55 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.733+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5400 2024-04-24T06:41:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.734+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.4 v2:172.21.15.66:6800/4254676376 3 ==== osd_op_reply(56 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32f5400 2024-04-24T06:41:44.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.734+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286400 2024-04-24T06:41:44.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.735+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 5 ==== osd_op_reply(57 [pgnls start_epoch 114 out=81b] v113'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.735+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.736+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 18 ==== osd_op_reply(58 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.736+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f4800 2024-04-24T06:41:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.737+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 8 ==== osd_op_reply(59 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.737+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286400 2024-04-24T06:41:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.738+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 6 ==== osd_op_reply(60 [pgnls start_epoch 114 out=77b] v113'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.739+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f4800 2024-04-24T06:41:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.739+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 9 ==== osd_op_reply(61 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.739+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.740+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 19 ==== osd_op_reply(62 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.740+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286800 2024-04-24T06:41:44.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.741+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== osd_op_reply(63 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff3286800 2024-04-24T06:41:44.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.741+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.742+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 20 ==== osd_op_reply(64 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.742+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5000 2024-04-24T06:41:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.744+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 8 ==== osd_op_reply(65 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.744+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286c00 2024-04-24T06:41:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.745+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 5 ==== osd_op_reply(66 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.745+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5400 2024-04-24T06:41:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.746+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.4 v2:172.21.15.66:6800/4254676376 4 ==== osd_op_reply(67 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32f5400 2024-04-24T06:41:44.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.746+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.747+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 14 ==== osd_op_reply(68 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.747+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.748+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 15 ==== osd_op_reply(69 [pgnls start_epoch 114 out=97b] v113'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.748+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286400 2024-04-24T06:41:44.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.749+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 7 ==== osd_op_reply(70 [pgnls start_epoch 114 out=74b] v109'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.749+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.750+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 21 ==== osd_op_reply(71 [pgnls start_epoch 114 out=73b] v113'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.750+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5000 2024-04-24T06:41:44.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.751+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 9 ==== osd_op_reply(72 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.751+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286c00 2024-04-24T06:41:44.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.752+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 6 ==== osd_op_reply(73 [pgnls start_epoch 114 out=98b] v113'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.752+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff3286c00 2024-04-24T06:41:44.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.753+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 7 ==== osd_op_reply(74 [pgnls start_epoch 114 out=76b] v114'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.753+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5000 2024-04-24T06:41:44.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.754+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 10 ==== osd_op_reply(75 [pgnls start_epoch 114 out=49b] v114'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.755+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f4800 2024-04-24T06:41:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.755+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 10 ==== osd_op_reply(76 [pgnls start_epoch 114 out=105b] v114'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.756+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d8c00 2024-04-24T06:41:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.756+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 22 ==== osd_op_reply(77 [pgnls start_epoch 114 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.757+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32d9000 2024-04-24T06:41:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.757+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 16 ==== osd_op_reply(78 [pgnls start_epoch 114 out=73b] v114'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.758+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 114 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f5000 2024-04-24T06:41:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.759+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 11 ==== osd_op_reply(79 [pgnls start_epoch 114 out=101b] v114'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.759+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-24T06:41:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.759+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8400 con 0x55dff32d8c00 2024-04-24T06:41:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.760+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.760+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-24T06:41:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.760+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8000 con 0x55dff32d8c00 2024-04-24T06:41:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.761+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.761+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-24T06:41:44.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.761+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff3287400 con 0x55dff32d8c00 2024-04-24T06:41:44.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.762+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.762+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-24T06:41:44.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.762+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff3287000 con 0x55dff32d8c00 2024-04-24T06:41:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.763+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.763+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-24T06:41:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.763+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275c000 con 0x55dff32d8c00 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-24T06:41:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.764+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275c400 con 0x55dff32d9000 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.765+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv40 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.766+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-24T06:41:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.766+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275c800 con 0x55dff32d8c00 2024-04-24T06:41:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.767+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.767+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-24T06:41:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.767+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275cc00 con 0x55dff3286400 2024-04-24T06:41:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.768+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.768+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-24T06:41:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.768+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-24T06:41:44.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.768+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-24T06:41:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.768+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275d000 con 0x55dff32d8c00 2024-04-24T06:41:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.769+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.769+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-24T06:41:44.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.769+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275d400 con 0x55dff32d9000 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.0 v2:172.21.15.59:6808/2713443937 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134000 con 0x55dff32d9000 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-24T06:41:44.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-24T06:41:44.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.770+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275d800 con 0x55dff32d8c00 2024-04-24T06:41:44.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.771+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.2 v2:172.21.15.59:6809/1388377934 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134500 con 0x55dff32d8c00 2024-04-24T06:41:44.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.771+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.771+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.771+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.771+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275dc00 con 0x55dff3286c00 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.773+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.773+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.773+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.773+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-24T06:41:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.773+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff275c400 con 0x55dff32f4800 2024-04-24T06:41:44.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.774+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.774+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-24T06:41:44.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.774+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff3287400 con 0x55dff32f4800 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 20 datascan.scan_links: injected ino detected 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.775+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff3287000 con 0x55dff32f5000 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3635400640 1 -- 172.21.15.59:0/3312095917 <== osd.5 v2:172.21.15.66:6816/79236266 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55dff3134000 con 0x55dff32f5000 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 10 datascan.scan_links: processing 1 injected_inos 2024-04-24T06:41:44.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-24T06:41:44.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-24T06:41:44.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.777+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8800 con 0x55dff32f4800 2024-04-24T06:41:44.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.778+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.778+0000 7f3638c30740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-24T06:41:44.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.778+0000 7f3638c30740 20 datascan.scan_links: first is now 2 2024-04-24T06:41:44.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.778+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8400 con 0x55dff32f4800 2024-04-24T06:41:44.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.782+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.3 v2:172.21.15.59:6824/2800980920 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v114'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55dff3134780 con 0x55dff32f4800 2024-04-24T06:41:44.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.782+0000 7f3638c30740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-24T06:41:44.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.782+0000 7f3638c30740 10 datascan.scan_links: updating inotable 2024-04-24T06:41:44.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.782+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff32d8000 con 0x55dff3286c00 2024-04-24T06:41:44.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.783+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.783+0000 7f3638c30740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-24T06:41:44.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.783+0000 7f3638c30740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-24T06:41:44.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.783+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55dff2753400 con 0x55dff3286c00 2024-04-24T06:41:44.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.788+0000 7f3636800640 1 -- 172.21.15.59:0/3312095917 <== osd.1 v2:172.21.15.59:6800/2375715406 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v114'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55dff3134500 con 0x55dff3286c00 2024-04-24T06:41:44.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.788+0000 7f3638c30740 10 datascan.scan_links: updating snaptable 2024-04-24T06:41:44.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.788+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55dff2752400 con 0x55dff3286400 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 <== osd.6 v2:172.21.15.66:6808/280071528 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55dff3134780 con 0x55dff3286400 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55dff32d9000 msgr2=0x55dff317cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x55dff32d9000 0x55dff317cc00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55dff3286c00 msgr2=0x55dff2730b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x55dff3286c00 0x55dff2730b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55dff32d8c00 msgr2=0x55dff317a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x55dff32d8c00 0x55dff317a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55dff32f4800 msgr2=0x55dff2730000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x55dff32f4800 0x55dff2730000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 reap_dead start 2024-04-24T06:41:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55dff32f5400 msgr2=0x55dff2731080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x55dff32f5400 0x55dff2731080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55dff32f5000 msgr2=0x55dff2730580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x55dff32f5000 0x55dff2730580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55dff3286400 msgr2=0x55dff2731600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x55dff3286400 0x55dff2731600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55dff3286800 msgr2=0x55dff2731b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.790+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x55dff3286800 0x55dff2731b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3635e00640 1 -- 172.21.15.59:0/3312095917 reap_dead start 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 msgr2=0x55dff317c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 0x55dff317c680 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55dff328cb10 tx=0x55dff3146900 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 msgr2=0x55dff317b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x55dff328dda0 tx=0x55dff328ed20 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 shutdown_connections 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x55dff32f4c00 0x55dff317c680 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 --2- 172.21.15.59:0/3312095917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3144400 0x55dff317b600 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.791+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 >> 172.21.15.59:0/3312095917 conn(0x55dff3145800 msgr2=0x55dff2475800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.792+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 shutdown_connections 2024-04-24T06:41:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.792+0000 7f3638c30740 1 -- 172.21.15.59:0/3312095917 wait complete. 2024-04-24T06:41:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 msgr2=0x55dff317a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x55dff31cd9b0 tx=0x55dff3147f20 comp rx=0 tx=0).stop 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 shutdown_connections 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dff3145400 0x55dff317a580 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dff3145c00 0x55dff317ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 --2- 172.21.15.59:0/2760764849 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x55dff3144800 0x55dff317b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.793+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 >> 172.21.15.59:0/2760764849 conn(0x55dff23a7000 msgr2=0x55dff2474000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.794+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 shutdown_connections 2024-04-24T06:41:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:44.794+0000 7f3638c30740 1 -- 172.21.15.59:0/2760764849 wait complete. 2024-04-24T06:41:44.806 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.256102, result: 2024-04-24T06:41:44.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:44.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:44.958 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-24T06:41:44.958 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-24T06:41:44.958 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:44.958 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-24T06:41:44.958 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.959 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:44.960 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.961 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-24T06:41:44.962 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.963 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:44.964 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-24T06:41:44.965 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-24T06:41:44.966 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:44.967 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-24T06:41:44.968 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:44.969 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:44.970 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = False 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-24T06:41:44.971 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:44.972 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:44.974 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:44.975 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-24T06:41:44.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-24T06:41:44.977 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:44.978 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:44.979 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:44.980 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:44.982 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.983 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-04-24T06:41:44.984 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-04-24T06:41:44.985 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = False 2024-04-24T06:41:44.986 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-04-24T06:41:44.986 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-04-24T06:41:44.986 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:44.986 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:44.986 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:44.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:45.006 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:45.147 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.148 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:45.149 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:45.151 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:45.152 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-24T06:41:45.153 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-24T06:41:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-24T06:41:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-24T06:41:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-24T06:41:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-24T06:41:45.154 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-24T06:41:45.155 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-24T06:41:45.156 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:45.157 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-24T06:41:45.158 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-24T06:41:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-24T06:41:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-24T06:41:45.159 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-24T06:41:45.160 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-24T06:41:45.161 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.162 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.163 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T06:41:45.164 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T06:41:45.164 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.164 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-04-24T06:41:45.164 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.164 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-04-24T06:41:45.165 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-04-24T06:41:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T06:41:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T06:41:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-04-24T06:41:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.166 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-04-24T06:41:45.167 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.59,172.21.15.66,[v2:172.21.15.66:3301,v1:172.21.15.66:6790]" 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = fce72cba-ae3c-4ba2-abdc-e927f443d3c7 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.168 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-04-24T06:41:45.169 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-04-24T06:41:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-04-24T06:41:45.171 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-04-24T06:41:45.172 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-04-24T06:41:45.173 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T06:41:45.174 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-04-24T06:41:45.175 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: mon op complaint time = 120 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T06:41:45.176 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 600 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-04-24T06:41:45.177 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds = 20 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: debug mds balancer = 20 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug frag = True 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: mds debug scatterstat = True 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: mds op complaint time = 180 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: mds verify scatter = True 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: osd op complaint time = 180 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: rados mon op timeout = 900 2024-04-24T06:41:45.178 INFO:teuthology.orchestra.run.smithi066.stdout: rados osd op timeout = 900 2024-04-24T06:41:45.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-24T06:41:45.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.406+0000 7f494b200640 1 Processor -- start 2024-04-24T06:41:45.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.407+0000 7f494b200640 1 -- start start 2024-04-24T06:41:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.408+0000 7f494b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c0d0d10 0x7f494c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.408+0000 7f494b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f494c15d810 0x7f494c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.408+0000 7f494b200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494c15cab0 con 0x7f494c1567e0 2024-04-24T06:41:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.408+0000 7f494b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f494c160e40 con 0x7f494c15d810 2024-04-24T06:41:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.408+0000 7f494b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f494c15cd30 con 0x7f494c0d0d10 2024-04-24T06:41:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.409+0000 7f494a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f494c15d810 0x7f494c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.409+0000 7f494a800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f494c15d810 0x7f494c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.59:55048/0 (socket says 172.21.15.59:55048) 2024-04-24T06:41:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.409+0000 7f494a800640 1 -- 172.21.15.59:0/1540246828 learned_addr learned my addr 172.21.15.59:0/1540246828 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.409+0000 7f4941400640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c0d0d10 0x7f494c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.410+0000 7f494a800640 1 -- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c0d0d10 msgr2=0x7f494c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.410+0000 7f494a800640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c0d0d10 0x7f494c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.410+0000 7f494a800640 1 -- 172.21.15.59:0/1540246828 >> v1:172.21.15.66:6789/0 conn(0x7f494c1567e0 legacy=0x7f494c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.410+0000 7f494a800640 1 -- 172.21.15.59:0/1540246828 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c15d010 con 0x7f494c15d810 2024-04-24T06:41:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.410+0000 7f4949400640 1 -- 172.21.15.59:0/1540246828 <== mon.1 v1:172.21.15.66:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789262085 0 0) 0x7f493c001490 con 0x7f494c1567e0 2024-04-24T06:41:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.411+0000 7f494a800640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c15d810 0x7f494c160110 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f494402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20c18960817975d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.412+0000 7f4949400640 1 -- 172.21.15.59:0/1540246828 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944037070 con 0x7f494c15d810 2024-04-24T06:41:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.412+0000 7f4949400640 1 -- 172.21.15.59:0/1540246828 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4944032440 con 0x7f494c15d810 2024-04-24T06:41:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.412+0000 7f4949400640 1 -- 172.21.15.59:0/1540246828 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944032810 con 0x7f494c15d810 2024-04-24T06:41:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.412+0000 7f494b200640 1 -- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c15d810 msgr2=0x7f494c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.412+0000 7f494b200640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c15d810 0x7f494c160110 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f494402a270 comp rx=0 tx=0).stop 2024-04-24T06:41:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 -- 172.21.15.59:0/1540246828 shutdown_connections 2024-04-24T06:41:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c15d810 0x7f494c160110 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 --2- 172.21.15.59:0/1540246828 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c0d0d10 0x7f494c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 -- 172.21.15.59:0/1540246828 >> 172.21.15.59:0/1540246828 conn(0x7f494c06bdf0 msgr2=0x7f494c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 -- 172.21.15.59:0/1540246828 shutdown_connections 2024-04-24T06:41:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.413+0000 7f494b200640 1 -- 172.21.15.59:0/1540246828 wait complete. 2024-04-24T06:41:45.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.415+0000 7f494b200640 1 Processor -- start 2024-04-24T06:41:45.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.415+0000 7f494b200640 1 -- start start 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 0x7f494c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 0x7f494c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f494c1607d0 con 0x7f494c15d810 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f494c160650 con 0x7f494c0d0d10 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f494c160950 con 0x7f494c1567e0 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46806/0 (socket says 172.21.15.59:46806) 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f494a800640 1 -- 172.21.15.59:0/2570937356 learned_addr learned my addr 172.21.15.59:0/2570937356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.416+0000 7f4949e00640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 0x7f494c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4941400640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 0x7f494c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 -- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 msgr2=0x7f494c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 0x7f494c15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 -- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 msgr2=0x7f494c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 0x7f494c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 -- 172.21.15.59:0/2570937356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c15e450 con 0x7f494c15d810 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4941400640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 0x7f494c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4949e00640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 0x7f494c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494a800640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f49440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944041020 con 0x7f494c15d810 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494403d640 con 0x7f494c15d810 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944037040 con 0x7f494c15d810 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.417+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c15e730 con 0x7f494c15d810 2024-04-24T06:41:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.418+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c15ecf0 con 0x7f494c15d810 2024-04-24T06:41:45.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.419+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c068290 con 0x7f494c15d810 2024-04-24T06:41:45.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.420+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4944044020 con 0x7f494c15d810 2024-04-24T06:41:45.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.421+0000 7f4942a00640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:45.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.421+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f49440ac3b0 con 0x7f494c15d810 2024-04-24T06:41:45.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.421+0000 7f4949e00640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:45.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.422+0000 7f4949e00640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 0x7f492806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f493c0012e0 tx=0x7f493c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.426+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f494c068290 con 0x7f494c15d810 2024-04-24T06:41:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.683+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f494c04fd90 con 0x7f494c15d810 2024-04-24T06:41:45.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.885+0000 7f4942a00640 1 -- 172.21.15.59:0/2570937356 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) ==== 162+0+0 (secure 0 0 0) 0x7f494404b030 con 0x7f494c15d810 2024-04-24T06:41:45.886 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-24T06:41:45.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 msgr2=0x7f492806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 0x7f492806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f493c0012e0 tx=0x7f493c009040 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 msgr2=0x7f494c1465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f49440046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 shutdown_connections 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f492806c8f0 0x7f492806edb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f494c15d810 0x7f494c1465e0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.888+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f494c1567e0 0x7f494c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.889+0000 7f494b200640 1 --2- 172.21.15.59:0/2570937356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f494c0d0d10 0x7f494c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.889+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 >> 172.21.15.59:0/2570937356 conn(0x7f494c06bdf0 msgr2=0x7f494c09a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.889+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 shutdown_connections 2024-04-24T06:41:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:45.889+0000 7f494b200640 1 -- 172.21.15.59:0/2570937356 wait complete. 2024-04-24T06:41:45.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:46.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.107+0000 7f8aab200640 1 Processor -- start 2024-04-24T06:41:46.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.107+0000 7f8aab200640 1 -- start start 2024-04-24T06:41:46.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.108+0000 7f8aab200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8aac0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.108+0000 7f8aab200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0b3870 con 0x7f8aac0cd760 2024-04-24T06:41:46.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.108+0000 7f8aab200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0b3b50 con 0x7f8aac06f3f0 2024-04-24T06:41:46.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.108+0000 7f8aab200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac0b41a0 con 0x7f8aac15b450 2024-04-24T06:41:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.109+0000 7f8aaa800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8aac0cd760 0x7f8aac0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39830/0 (socket says 172.21.15.59:39830) 2024-04-24T06:41:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.109+0000 7f8aaa800640 1 -- 172.21.15.59:0/2985810817 learned_addr learned my addr 172.21.15.59:0/2985810817 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.109+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8aac0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.109+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364126030 0 0) 0x7f8aac0b3870 con 0x7f8aac0cd760 2024-04-24T06:41:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.109+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a8c003520 con 0x7f8aac0cd760 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1197722067 0 0) 0x7f8a8c003520 con 0x7f8aac0cd760 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 msgr2=0x7f8aac0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8aac0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 >> v1:172.21.15.66:6789/0 conn(0x7f8aac06f3f0 legacy=0x7f8aac0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0b3e30 con 0x7f8aac0cd760 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f8aa4001670 con 0x7f8aac0cd760 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 >> v1:172.21.15.59:6789/0 conn(0x7f8aac0cd760 legacy=0x7f8aac0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8c0039e0 0x7f8a8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8a8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c004d40 0x7f8a8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0b41a0 con 0x7f8a8c004d40 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c00dc80 con 0x7f8a8c0039e0 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8c00de80 con 0x7f8aac15b450 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aaa800640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c004d40 0x7f8a8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8c0039e0 0x7f8a8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.110+0000 7f8aa9400640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8a8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aaa800640 1 -- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 msgr2=0x7f8a8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aaa800640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac15b450 0x7f8a8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aaa800640 1 -- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8c0039e0 msgr2=0x7f8a8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aaa800640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8c0039e0 0x7f8a8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aaa800640 1 -- 172.21.15.59:0/2985810817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c003520 con 0x7f8a8c004d40 2024-04-24T06:41:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.111+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a8c0039e0 0x7f8a8c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.112+0000 7f8aaa800640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c004d40 0x7f8a8c0051c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c210 tx=0x7f8a8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8232f656b6cbc999 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.112+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4007cf0 con 0x7f8a8c004d40 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.112+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4005af0 con 0x7f8a8c004d40 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.112+0000 7f8aa8a00640 1 -- 172.21.15.59:0/2985810817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4007070 con 0x7f8a8c004d40 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 -- 172.21.15.59:0/2985810817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c004d40 msgr2=0x7f8a8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 --2- 172.21.15.59:0/2985810817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c004d40 0x7f8a8c0051c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c210 tx=0x7f8a8c005830 comp rx=0 tx=0).stop 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aaa800640 1 -- 172.21.15.59:0/2985810817 reap_dead start 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 -- 172.21.15.59:0/2985810817 shutdown_connections 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 -- 172.21.15.59:0/2985810817 >> 172.21.15.59:0/2985810817 conn(0x7f8aac0b21a0 msgr2=0x7f8aac0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 -- 172.21.15.59:0/2985810817 shutdown_connections 2024-04-24T06:41:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.113+0000 7f8aab200640 1 -- 172.21.15.59:0/2985810817 wait complete. 2024-04-24T06:41:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.115+0000 7f8aab200640 1 Processor -- start 2024-04-24T06:41:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.115+0000 7f8aab200640 1 -- start start 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac0cb690 0x7f8aac0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0b0620 0x7f8aac0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0b3080 con 0x7f8aac1f4570 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac0b4ff0 con 0x7f8aac0b0620 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aab200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac0b3200 con 0x7f8aac0cb690 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46824/0 (socket says 172.21.15.59:46824) 2024-04-24T06:41:46.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 -- 172.21.15.59:0/2321519703 learned_addr learned my addr 172.21.15.59:0/2321519703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 -- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac0cb690 msgr2=0x7f8aac0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac0cb690 0x7f8aac0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 -- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0b0620 msgr2=0x7f8aac0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0b0620 0x7f8aac0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.116+0000 7f8aa9e00640 1 -- 172.21.15.59:0/2321519703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0c3e80 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aa9e00640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0108e0 tx=0x7f8a9c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00f040 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00ec20 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01c400 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac0c4160 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.117+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac0c4720 con 0x7f8aac1f4570 2024-04-24T06:41:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.118+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac04fd90 con 0x7f8aac1f4570 2024-04-24T06:41:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.119+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a9c01c5a0 con 0x7f8aac1f4570 2024-04-24T06:41:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.120+0000 7f8aa2000640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.120+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f8a9c08c470 con 0x7f8aac1f4570 2024-04-24T06:41:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.121+0000 7f8aa9400640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:46.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.121+0000 7f8aa9400640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 0x7f8a8806edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8a94007fd0 tx=0x7f8a94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.125+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a9c018030 con 0x7f8aac1f4570 2024-04-24T06:41:46.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.368+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aac159030 con 0x7f8aac1f4570 2024-04-24T06:41:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.369+0000 7f8aa2000640 1 -- 172.21.15.59:0/2321519703 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5173 (secure 0 0 0) 0x7f8a9c05a710 con 0x7f8aac1f4570 2024-04-24T06:41:46.370 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:46.370 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:45.892017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-24T06:41:46.371 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 111 2024-04-24T06:41:46.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 msgr2=0x7f8a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 0x7f8a8806edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8a94007fd0 tx=0x7f8a94009040 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 msgr2=0x7f8aac15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0108e0 tx=0x7f8a9c010da0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 shutdown_connections 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0b0620 0x7f8aac0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8aac0cb690 0x7f8aac0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 --2- 172.21.15.59:0/2321519703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac1f4570 0x7f8aac15c290 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 >> 172.21.15.59:0/2321519703 conn(0x7f8aac0b21a0 msgr2=0x7f8aac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.372+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 shutdown_connections 2024-04-24T06:41:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:46.373+0000 7f8aab200640 1 -- 172.21.15.59:0/2321519703 wait complete. 2024-04-24T06:41:46.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:45.892017+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9738': {'gid': 9738, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/2269052051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2269052051}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2269052051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:41:46.797 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9738:up:replay 2024-04-24T06:41:47.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:48.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.016+0000 7f4187c00640 1 Processor -- start 2024-04-24T06:41:48.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.016+0000 7f4187c00640 1 -- start start 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41800d0810 0x7f41800d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 0x7f41800ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 0x7f41800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f41800d1130 con 0x7f41800d0810 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f41800cbd00 con 0x7f418015ecc0 2024-04-24T06:41:48.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4187c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f41800cbfe0 con 0x7f41800ce250 2024-04-24T06:41:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.017+0000 7f4185200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41800d0810 0x7f41800d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41800d0810 0x7f41800d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46828/0 (socket says 172.21.15.59:46828) 2024-04-24T06:41:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 -- 172.21.15.59:0/2860171421 learned_addr learned my addr 172.21.15.59:0/2860171421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f417fe00640 1 --2- 172.21.15.59:0/2860171421 >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 0x7f41800ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185c00640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 0x7f41800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 -- 172.21.15.59:0/2860171421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 msgr2=0x7f41800ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 0x7f41800ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 -- 172.21.15.59:0/2860171421 >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 msgr2=0x7f41800ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 --2- 172.21.15.59:0/2860171421 >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 0x7f41800ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185200640 1 -- 172.21.15.59:0/2860171421 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41800cc2c0 con 0x7f41800d0810 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f417fe00640 1 --2- 172.21.15.59:0/2860171421 >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 0x7f41800ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.018+0000 7f4185c00640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 0x7f41800ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:48.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.020+0000 7f4185200640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41800d0810 0x7f41800d0bf0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f41800cdfd0 tx=0x7f417002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6293b31c1659ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.020+0000 7f417f400640 1 -- 172.21.15.59:0/2860171421 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170036070 con 0x7f41800d0810 2024-04-24T06:41:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.020+0000 7f417f400640 1 -- 172.21.15.59:0/2860171421 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4170031600 con 0x7f41800d0810 2024-04-24T06:41:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.020+0000 7f417f400640 1 -- 172.21.15.59:0/2860171421 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41700319d0 con 0x7f41800d0810 2024-04-24T06:41:48.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 -- 172.21.15.59:0/2860171421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41800d0810 msgr2=0x7f41800d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41800d0810 0x7f41800d0bf0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f41800cdfd0 tx=0x7f417002a690 comp rx=0 tx=0).stop 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 -- 172.21.15.59:0/2860171421 shutdown_connections 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800ce250 0x7f41800ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 --2- 172.21.15.59:0/2860171421 >> v2:172.21.15.66:3300/0 conn(0x7f418015ecc0 0x7f41800ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 --2- 172.21.15.59:0/2860171421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41800d0810 0x7f41800d0bf0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 -- 172.21.15.59:0/2860171421 >> 172.21.15.59:0/2860171421 conn(0x7f4180073430 msgr2=0x7f418009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 -- 172.21.15.59:0/2860171421 shutdown_connections 2024-04-24T06:41:48.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.021+0000 7f4187c00640 1 -- 172.21.15.59:0/2860171421 wait complete. 2024-04-24T06:41:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.023+0000 7f4187c00640 1 Processor -- start 2024-04-24T06:41:48.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.023+0000 7f4187c00640 1 -- start start 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41800ce250 0x7f418015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800d0810 0x7f4180161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41800cec40 con 0x7f418015ecc0 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41800ae810 con 0x7f41800ce250 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4187c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f41800ccac0 con 0x7f41800d0810 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4185c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4185c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46834/0 (socket says 172.21.15.59:46834) 2024-04-24T06:41:48.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f4185c00640 1 -- 172.21.15.59:0/3835561447 learned_addr learned my addr 172.21.15.59:0/3835561447 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.024+0000 7f417fe00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800d0810 0x7f4180161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185200640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41800ce250 0x7f418015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 -- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800d0810 msgr2=0x7f4180161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800d0810 0x7f4180161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 -- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41800ce250 msgr2=0x7f418015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41800ce250 0x7f418015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 -- 172.21.15.59:0/3835561447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4180160b80 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4185c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f417800c7f0 tx=0x7f417800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178015070 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178007820 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178007c10 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4180160e60 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.025+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4180161700 con 0x7f418015ecc0 2024-04-24T06:41:48.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.027+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4180161030 con 0x7f418015ecc0 2024-04-24T06:41:48.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.028+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4178018070 con 0x7f418015ecc0 2024-04-24T06:41:48.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.029+0000 7f417cc00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:48.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.029+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f417808bd90 con 0x7f418015ecc0 2024-04-24T06:41:48.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.030+0000 7f4185200640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:48.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.031+0000 7f4185200640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 0x7f415c06edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4170033000 tx=0x7f4170009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:48.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.034+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f417805a180 con 0x7f418015ecc0 2024-04-24T06:41:48.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.283+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4180068a80 con 0x7f418015ecc0 2024-04-24T06:41:48.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.284+0000 7f417cc00640 1 -- 172.21.15.59:0/3835561447 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5173 (secure 0 0 0) 0x7f4178059a80 con 0x7f418015ecc0 2024-04-24T06:41:48.287 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:48.287 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:46.905001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-24T06:41:48.287 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.287+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 msgr2=0x7f415c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.287+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 0x7f415c06edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4170033000 tx=0x7f4170009000 comp rx=0 tx=0).stop 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.287+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 msgr2=0x7f41800b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.287+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f417800c7f0 tx=0x7f417800ccb0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 shutdown_connections 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f415c06c8f0 0x7f415c06edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f418015ecc0 0x7f41800b17c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f41800d0810 0x7f4180161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 --2- 172.21.15.59:0/3835561447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41800ce250 0x7f418015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:48.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 >> 172.21.15.59:0/3835561447 conn(0x7f4180073430 msgr2=0x7f418015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:48.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 shutdown_connections 2024-04-24T06:41:48.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:48.288+0000 7f4187c00640 1 -- 172.21.15.59:0/3835561447 wait complete. 2024-04-24T06:41:48.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:46.905001+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9738': {'gid': 9738, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6837/2269052051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2269052051}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2269052051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-24T06:41:48.710 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9738:up:rejoin 2024-04-24T06:41:49.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:49.885 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-24T06:41:49.882+0000 7f2e4a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:41:49.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.928+0000 7ff822c00640 1 Processor -- start 2024-04-24T06:41:49.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.928+0000 7ff822c00640 1 -- start start 2024-04-24T06:41:49.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.929+0000 7ff822c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff81c169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.929+0000 7ff822c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c0b87f0 con 0x7ff81c0b3490 2024-04-24T06:41:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.929+0000 7ff822c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81c0d2f30 con 0x7ff81c15eb60 2024-04-24T06:41:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.929+0000 7ff822c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c0ba7a0 con 0x7ff81c0b99a0 2024-04-24T06:41:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.930+0000 7ff813400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff81c0b3490 0x7ff81c0b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46542/0 (socket says 172.21.15.59:46542) 2024-04-24T06:41:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.930+0000 7ff813400640 1 -- 172.21.15.59:0/3175242806 learned_addr learned my addr 172.21.15.59:0/3175242806 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.930+0000 7ff820c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff81c169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.930+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325406367 0 0) 0x7ff81c0b87f0 con 0x7ff81c0b3490 2024-04-24T06:41:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.930+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff800003520 con 0x7ff81c0b3490 2024-04-24T06:41:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 304068481 0 0) 0x7ff800003520 con 0x7ff81c0b3490 2024-04-24T06:41:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 msgr2=0x7ff81c169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff81c169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 >> v1:172.21.15.66:6789/0 conn(0x7ff81c15eb60 legacy=0x7ff81c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c0ba1e0 con 0x7ff81c0b3490 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7ff804001670 con 0x7ff81c0b3490 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 >> v1:172.21.15.59:6789/0 conn(0x7ff81c0b3490 legacy=0x7ff81c0b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8000039e0 0x7ff800003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff800004d40 0x7ff8000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0ba7a0 con 0x7ff800004d40 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff80000ba70 con 0x7ff8000039e0 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff80000bc70 con 0x7ff81c0b99a0 2024-04-24T06:41:49.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.931+0000 7ff820c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff800004d40 0x7ff8000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff813400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff820c00640 1 -- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 msgr2=0x7ff800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:49.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff81be00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8000039e0 0x7ff800003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff820c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff800004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff820c00640 1 -- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8000039e0 msgr2=0x7ff800003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff820c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8000039e0 0x7ff800003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff820c00640 1 -- 172.21.15.59:0/3175242806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff800003520 con 0x7ff800004d40 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff81be00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8000039e0 0x7ff800003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.932+0000 7ff813400640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c0b99a0 0x7ff800004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.933+0000 7ff820c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff800004d40 0x7ff8000051c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f90 tx=0x7ff800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63849ad41f90eb9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.933+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814009c10 con 0x7ff800004d40 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.933+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814036070 con 0x7ff800004d40 2024-04-24T06:41:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.933+0000 7ff81b400640 1 -- 172.21.15.59:0/3175242806 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814031850 con 0x7ff800004d40 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 -- 172.21.15.59:0/3175242806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff800004d40 msgr2=0x7ff8000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 --2- 172.21.15.59:0/3175242806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff800004d40 0x7ff8000051c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f90 tx=0x7ff800005830 comp rx=0 tx=0).stop 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff820c00640 1 -- 172.21.15.59:0/3175242806 reap_dead start 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 -- 172.21.15.59:0/3175242806 shutdown_connections 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 -- 172.21.15.59:0/3175242806 >> 172.21.15.59:0/3175242806 conn(0x7ff81c067b50 msgr2=0x7ff81c067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 -- 172.21.15.59:0/3175242806 shutdown_connections 2024-04-24T06:41:49.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.934+0000 7ff822c00640 1 -- 172.21.15.59:0/3175242806 wait complete. 2024-04-24T06:41:49.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.936+0000 7ff822c00640 1 Processor -- start 2024-04-24T06:41:49.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.936+0000 7ff822c00640 1 -- start start 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff800004d40 0x7ff81c14bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c161460 0x7ff81c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0b8d70 con 0x7ff81c0ca500 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c0b8a90 con 0x7ff800004d40 2024-04-24T06:41:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff822c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c0b3c80 con 0x7ff81c161460 2024-04-24T06:41:49.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.937+0000 7ff813400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41436/0 (socket says 172.21.15.59:41436) 2024-04-24T06:41:49.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 -- 172.21.15.59:0/875802930 learned_addr learned my addr 172.21.15.59:0/875802930 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff81be00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff800004d40 0x7ff81c14bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff820c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c161460 0x7ff81c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 -- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c161460 msgr2=0x7ff81c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c161460 0x7ff81c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 -- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff800004d40 msgr2=0x7ff81c14bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff800004d40 0x7ff81c14bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:49.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 -- 172.21.15.59:0/875802930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c14c150 con 0x7ff81c0ca500 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.938+0000 7ff813400640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d860 tx=0x7ff80400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.939+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804019070 con 0x7ff81c0ca500 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.939+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8040074e0 con 0x7ff81c0ca500 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.939+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040049e0 con 0x7ff81c0ca500 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.939+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1f4810 con 0x7ff81c0ca500 2024-04-24T06:41:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.939+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1f4dd0 con 0x7ff81c0ca500 2024-04-24T06:41:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.940+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c068a80 con 0x7ff81c0ca500 2024-04-24T06:41:49.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.941+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff80401c070 con 0x7ff81c0ca500 2024-04-24T06:41:49.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.942+0000 7ff818c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 0x7ff7f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:49.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.942+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7ff80408b740 con 0x7ff81c0ca500 2024-04-24T06:41:49.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.942+0000 7ff81be00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 0x7ff7f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:49.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.943+0000 7ff81be00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 0x7ff7f806ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001750 tx=0x7ff80c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:49.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:49.947+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff81c14c150 con 0x7ff81c0ca500 2024-04-24T06:41:50.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.193+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff81c0680d0 con 0x7ff81c0ca500 2024-04-24T06:41:50.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.195+0000 7ff818c00640 1 -- 172.21.15.59:0/875802930 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5180 (secure 0 0 0) 0x7ff804059a70 con 0x7ff81c0ca500 2024-04-24T06:41:50.196 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:50.196 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:48.912512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9738,"qdb_cluster":[9738]},"id":8}]} 2024-04-24T06:41:50.196 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 114 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.197+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 msgr2=0x7ff7f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.197+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 0x7ff7f806ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff80c001750 tx=0x7ff80c009040 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.197+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 msgr2=0x7ff81c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.197+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d860 tx=0x7ff80400dd20 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 shutdown_connections 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ff7f806c920 0x7ff7f806ede0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ff81c161460 0x7ff81c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81c0ca500 0x7ff81c0ca980 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 --2- 172.21.15.59:0/875802930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff800004d40 0x7ff81c14bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 >> 172.21.15.59:0/875802930 conn(0x7ff81c067b50 msgr2=0x7ff81c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 shutdown_connections 2024-04-24T06:41:50.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.198+0000 7ff822c00640 1 -- 172.21.15.59:0/875802930 wait complete. 2024-04-24T06:41:50.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T06:40:47.539600+0000', 'modified': '2024-04-24T06:41:48.912512+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9738': {'gid': 9738, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6837/2269052051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2269052051}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2269052051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9738, 'qdb_cluster': [9738]} 2024-04-24T06:41:50.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T06:41:50.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T06:41:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.842+0000 7f59d7200640 1 Processor -- start 2024-04-24T06:41:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.843+0000 7f59d7200640 1 -- start start 2024-04-24T06:41:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d0f80 0x7f59d00cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 0x7f59d015f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59d00d1360 0x7f59d0162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f59d015fdc0 con 0x7f59d00d1360 2024-04-24T06:41:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00cf930 con 0x7f59d015ea40 2024-04-24T06:41:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00cfc10 con 0x7f59d00d0f80 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59d00d1360 0x7f59d0162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59d00d1360 0x7f59d0162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41442/0 (socket says 172.21.15.59:41442) 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.844+0000 7f59d6800640 1 -- 172.21.15.59:0/822868346 learned_addr learned my addr 172.21.15.59:0/822868346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d5400640 1 --2- 172.21.15.59:0/822868346 >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 0x7f59d015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d5e00640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d0f80 0x7f59d00cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d6800640 1 -- 172.21.15.59:0/822868346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d0f80 msgr2=0x7f59d00cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d6800640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d0f80 0x7f59d00cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d6800640 1 -- 172.21.15.59:0/822868346 >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 msgr2=0x7f59d015f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d6800640 1 --2- 172.21.15.59:0/822868346 >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 0x7f59d015f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d6800640 1 -- 172.21.15.59:0/822868346 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00cfef0 con 0x7f59d00d1360 2024-04-24T06:41:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.845+0000 7f59d5400640 1 --2- 172.21.15.59:0/822868346 >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 0x7f59d015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.846+0000 7f59d6800640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d1360 0x7f59d0162370 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f59d0073c00 tx=0x7f59cc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b83b6aa3ac068f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.846+0000 7f59d4a00640 1 -- 172.21.15.59:0/822868346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc036070 con 0x7f59d00d1360 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.846+0000 7f59d4a00640 1 -- 172.21.15.59:0/822868346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc031600 con 0x7f59d00d1360 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.846+0000 7f59d4a00640 1 -- 172.21.15.59:0/822868346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0319d0 con 0x7f59d00d1360 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 -- 172.21.15.59:0/822868346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d1360 msgr2=0x7f59d0162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d1360 0x7f59d0162370 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f59d0073c00 tx=0x7f59cc02a610 comp rx=0 tx=0).stop 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 -- 172.21.15.59:0/822868346 shutdown_connections 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d1360 0x7f59d0162370 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 --2- 172.21.15.59:0/822868346 >> v2:172.21.15.66:3300/0 conn(0x7f59d015ea40 0x7f59d015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 --2- 172.21.15.59:0/822868346 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d0f80 0x7f59d00cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 -- 172.21.15.59:0/822868346 >> 172.21.15.59:0/822868346 conn(0x7f59d008bf90 msgr2=0x7f59d00d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 -- 172.21.15.59:0/822868346 shutdown_connections 2024-04-24T06:41:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.847+0000 7f59d7200640 1 -- 172.21.15.59:0/822868346 wait complete. 2024-04-24T06:41:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.849+0000 7f59d7200640 1 Processor -- start 2024-04-24T06:41:50.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.849+0000 7f59d7200640 1 -- start start 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 0x7f59d0156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 0x7f59d0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00d17c0 con 0x7f59d00d0f80 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00d04b0 con 0x7f59d015ea40 2024-04-24T06:41:50.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d7200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f59d01628b0 con 0x7f59d00d1360 2024-04-24T06:41:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.850+0000 7f59d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41454/0 (socket says 172.21.15.59:41454) 2024-04-24T06:41:50.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 -- 172.21.15.59:0/830660707 learned_addr learned my addr 172.21.15.59:0/830660707 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5400640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 0x7f59d0156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d6800640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 0x7f59d0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 -- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 msgr2=0x7f59d0156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 0x7f59d0156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 -- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 msgr2=0x7f59d0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 0x7f59d0156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 -- 172.21.15.59:0/830660707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00b41d0 con 0x7f59d00d0f80 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d6800640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 0x7f59d0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5400640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 0x7f59d0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.851+0000 7f59d5e00640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c7f0 tx=0x7f59c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.852+0000 7f59ca000640 1 -- 172.21.15.59:0/830660707 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4015070 con 0x7f59d00d0f80 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.852+0000 7f59ca000640 1 -- 172.21.15.59:0/830660707 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4004240 con 0x7f59d00d0f80 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.852+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d00b44b0 con 0x7f59d00d0f80 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.852+0000 7f59ca000640 1 -- 172.21.15.59:0/830660707 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4007620 con 0x7f59d00d0f80 2024-04-24T06:41:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.852+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d00b4a70 con 0x7f59d00d0f80 2024-04-24T06:41:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.855+0000 7f59ca000640 1 -- 172.21.15.59:0/830660707 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59c40079d0 con 0x7f59d00d0f80 2024-04-24T06:41:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.856+0000 7f59ca000640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 0x7f59b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.856+0000 7f59ca000640 1 -- 172.21.15.59:0/830660707 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f59c408bd50 con 0x7f59d00d0f80 2024-04-24T06:41:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.856+0000 7f59d5400640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 0x7f59b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.856+0000 7f59db5fb740 1 Processor -- start 2024-04-24T06:41:50.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 -- start start 2024-04-24T06:41:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59d5400640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 0x7f59b406edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d3aa0 0x559db62d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3360 0x559db62a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559db620db10 con 0x559db62f0980 2024-04-24T06:41:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559db61b42b0 con 0x559db62d3360 2024-04-24T06:41:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.857+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559db6209ad0 con 0x559db62d3aa0 2024-04-24T06:41:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41468/0 (socket says 172.21.15.59:41468) 2024-04-24T06:41:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 -- 172.21.15.59:0/1189690217 learned_addr learned my addr 172.21.15.59:0/1189690217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 -- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d3aa0 msgr2=0x559db62d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d3aa0 0x559db62d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 -- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3360 msgr2=0x559db62a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3360 0x559db62a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 -- 172.21.15.59:0/1189690217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db5f202d0 con 0x559db62f0980 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.858+0000 7f59d5e00640 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f59c4059a00 tx=0x7f59c40595f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daec76a7491573f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59be600640 1 -- 172.21.15.59:0/1189690217 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4088070 con 0x559db62f0980 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59be600640 1 -- 172.21.15.59:0/1189690217 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c408a200 con 0x559db62f0980 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59be600640 1 -- 172.21.15.59:0/1189690217 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c401e040 con 0x559db62f0980 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 -- 172.21.15.59:0/1189690217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 msgr2=0x559db62a3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f59c4059a00 tx=0x7f59c40595f0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 -- 172.21.15.59:0/1189690217 shutdown_connections 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3360 0x559db62a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d3aa0 0x559db62d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 --2- 172.21.15.59:0/1189690217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62a3310 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 -- 172.21.15.59:0/1189690217 >> 172.21.15.59:0/1189690217 conn(0x559db62a5580 msgr2=0x559db62ede20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.859+0000 7f59db5fb740 1 -- 172.21.15.59:0/1189690217 shutdown_connections 2024-04-24T06:41:50.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.860+0000 7f59db5fb740 1 -- 172.21.15.59:0/1189690217 wait complete. 2024-04-24T06:41:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.860+0000 7f59db5fb740 1 -- start start 2024-04-24T06:41:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.860+0000 7f59db5fb740 1 Processor -- start 2024-04-24T06:41:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.861+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db63011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.861+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59db5fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559db62a7730 con 0x559db62d4560 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559db61c7590 con 0x559db62d3aa0 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559db62a6b20 con 0x559db62f0980 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59db5fb740 10 client.0 _ll_register_callbacks cb 0x559db61aa3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41484/0 (socket says 172.21.15.59:41484) 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 -- 172.21.15.59:0/2614246427 learned_addr learned my addr 172.21.15.59:0/2614246427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d5e00640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db63011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d5400640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 -- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 msgr2=0x559db62d4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62d4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 -- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 msgr2=0x559db63011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db63011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.862+0000 7f59d6800640 1 -- 172.21.15.59:0/2614246427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db5fea6b0 con 0x559db62d4560 2024-04-24T06:41:50.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59d6800640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009f10 tx=0x7f59cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee49cbd724a7b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc040030 con 0x559db62d4560 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f59cc031bf0 con 0x559db62d4560 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c408a200 con 0x559db62d4560 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f59cc04c050 con 0x559db62d4560 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.863+0000 7f59db5fb740 10 client.9825 fetch_fsmap learned FSMap version 114 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x559db62d5920 con 0x559db62d4560 2024-04-24T06:41:50.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c408a200 con 0x559db62d4560 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 114) ==== 3494+0+0 (secure 0 0 0) 0x7f59cc04f020 con 0x559db62d4560 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 10 client.9825 fetch_fsmap finished waiting for FSMap version 114 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 10 client.9825 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 20 client.9825 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db62e52c0 0x559db62fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 4 client.9825 mds_command: new command op to 9738 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.864+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 --> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db621b530 con 0x559db62e52c0 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.865+0000 7f59d5e00640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db62e52c0 0x559db62fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.865+0000 7f59d5e00640 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db62e52c0 0x559db62fecd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce77ceccfd7c2cf server_cookie=4eae81c88adc74c3 in_seq=0 out_seq=0 2024-04-24T06:41:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.865+0000 7f59b2a00640 10 client.9825 ms_handle_connect on v2:172.21.15.59:6835/2269052051 2024-04-24T06:41:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59b2a00640 1 -- 172.21.15.59:0/2614246427 <== mds.0 v2:172.21.15.59:6835/2269052051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559db621b530 con 0x559db62e52c0 2024-04-24T06:41:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59b2a00640 10 client.9825 handle_command_reply: tid=0 2024-04-24T06:41:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 client.9825 shutdown 2024-04-24T06:41:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 msgr2=0x559db62d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009f10 tx=0x7f59cc004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 shutdown_connections 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db62e52c0 0x559db62fecd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62d6410 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 --2- 172.21.15.59:0/2614246427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db63011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 >> 172.21.15.59:0/2614246427 conn(0x559db62d3360 msgr2=0x559db62eff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.866+0000 7f59b2a00640 0 client.9825 ms_handle_reset on v2:172.21.15.59:6835/2269052051 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.867+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 shutdown_connections 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.867+0000 7f59db5fb740 1 -- 172.21.15.59:0/2614246427 wait complete. 2024-04-24T06:41:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.867+0000 7f59db5fb740 20 client.9825 trim_cache size 0 max 16384 2024-04-24T06:41:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.887+0000 7f59db5fb740 1 Processor -- start 2024-04-24T06:41:50.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.887+0000 7f59db5fb740 1 -- start start 2024-04-24T06:41:50.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db62fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d4560 0x559db62fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01fff20 con 0x559db62f0980 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01ff2b0 con 0x559db62d3aa0 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f59d01ff8d0 con 0x559db62d4560 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.888+0000 7f59d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41492/0 (socket says 172.21.15.59:41492) 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 -- 172.21.15.59:0/3132944283 learned_addr learned my addr 172.21.15.59:0/3132944283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d5e00640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db62fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d5400640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d4560 0x559db62fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 -- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d4560 msgr2=0x559db62fdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d4560 0x559db62fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 -- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 msgr2=0x559db62fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db62fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 -- 172.21.15.59:0/3132944283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db62a66b0 con 0x559db62f0980 2024-04-24T06:41:50.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d5e00640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db62fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.889+0000 7f59d6800640 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f59cc031ce0 tx=0x7f59cc037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a98f09973557c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59bdc00640 1 -- 172.21.15.59:0/3132944283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc055070 con 0x559db62f0980 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59bdc00640 1 -- 172.21.15.59:0/3132944283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc045750 con 0x559db62f0980 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59bdc00640 1 -- 172.21.15.59:0/3132944283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc040040 con 0x559db62f0980 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 -- 172.21.15.59:0/3132944283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 msgr2=0x559db62edc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f59cc031ce0 tx=0x7f59cc037000 comp rx=0 tx=0).stop 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 -- 172.21.15.59:0/3132944283 shutdown_connections 2024-04-24T06:41:50.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62f0980 0x559db62edc20 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62d4560 0x559db62fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 --2- 172.21.15.59:0/3132944283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db62fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 -- 172.21.15.59:0/3132944283 >> 172.21.15.59:0/3132944283 conn(0x559db62d3360 msgr2=0x559db62f13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 -- 172.21.15.59:0/3132944283 shutdown_connections 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.890+0000 7f59db5fb740 1 -- 172.21.15.59:0/3132944283 wait complete. 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.891+0000 7f59db5fb740 1 -- start start 2024-04-24T06:41:50.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.891+0000 7f59db5fb740 1 Processor -- start 2024-04-24T06:41:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db6368350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559db62e5140 con 0x559db62d4560 2024-04-24T06:41:50.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559db62ff440 con 0x559db62d3aa0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x559db62e5790 con 0x559db62f0980 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.892+0000 7f59db5fb740 10 client.0 _ll_register_callbacks cb 0x7f59cc0317c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41508/0 (socket says 172.21.15.59:41508) 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 -- 172.21.15.59:0/46938771 learned_addr learned my addr 172.21.15.59:0/46938771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5e00640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db6368350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 -- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 msgr2=0x559db62cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 -- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 msgr2=0x559db6368350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db6368350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 -- 172.21.15.59:0/46938771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db6311d60 con 0x559db62d4560 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59d5400640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f59d015ee40 tx=0x7f59c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf363f300944a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.893+0000 7f59b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-24T06:41:50.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0023030 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f59c0014cb0 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc045750 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f59c002c050 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59db5fb740 10 client.9831 fetch_fsmap learned FSMap version 114 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x559db63688f0 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59cc045750 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 114) ==== 3494+0+0 (secure 0 0 0) 0x7f59c002f020 con 0x559db62d4560 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.894+0000 7f59db5fb740 10 client.9831 fetch_fsmap finished waiting for FSMap version 114 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59db5fb740 10 client.9831 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59db5fb740 20 client.9831 populate_metadata read hostname 'smithi059' 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db6310ce0 0x559db63110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59db5fb740 4 client.9831 mds_command: new command op to 9738 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 --> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] -- command(tid 0: {"prefix": "status"}) -- 0x559db62153a0 con 0x559db6310ce0 2024-04-24T06:41:50.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59d5e00640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db6310ce0 0x559db63110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59d5e00640 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db6310ce0 0x559db63110c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc60ef8a0ed935f server_cookie=64555bb92e9ddda3 in_seq=0 out_seq=0 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.895+0000 7f59b2a00640 10 client.9831 ms_handle_connect on v2:172.21.15.59:6835/2269052051 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59b2a00640 1 -- 172.21.15.59:0/46938771 <== mds.0 v2:172.21.15.59:6835/2269052051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x559db62153a0 con 0x559db6310ce0 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59b2a00640 10 client.9831 handle_command_reply: tid=0 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 client.9831 shutdown 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 msgr2=0x559db62cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f59d015ee40 tx=0x7f59c00046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 shutdown_connections 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:6835/2269052051,v1:172.21.15.59:6837/2269052051] conn(0x559db6310ce0 0x559db63110c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x559db62f0980 0x559db62cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559db62d4560 0x559db62cf430 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 --2- 172.21.15.59:0/46938771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559db62d3aa0 0x559db6368350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59b2a00640 0 client.9831 ms_handle_reset on v2:172.21.15.59:6835/2269052051 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 >> 172.21.15.59:0/46938771 conn(0x559db62d3360 msgr2=0x559db62f1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 shutdown_connections 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 1 -- 172.21.15.59:0/46938771 wait complete. 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.896+0000 7f59db5fb740 20 client.9831 trim_cache size 0 max 16384 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "fce72cba-ae3c-4ba2-abdc-e927f443d3c7", 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9738, 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-24T06:41:50.900 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 4.9988503260000003, 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 114, 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 114, 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 114, 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 6.9503886440000002 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:50.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 msgr2=0x7f59b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 0x7f59b406edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0008040 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 msgr2=0x7f59d0161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c7f0 tx=0x7f59c400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 shutdown_connections 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f59b406c8f0 0x7f59b406edb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59d015ea40 0x7f59d0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f59d00d1360 0x7f59d0156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 --2- 172.21.15.59:0/830660707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59d00d0f80 0x7f59d0161560 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 >> 172.21.15.59:0/830660707 conn(0x7f59d008bf90 msgr2=0x7f59d00d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 shutdown_connections 2024-04-24T06:41:50.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:50.897+0000 7f59d7200640 1 -- 172.21.15.59:0/830660707 wait complete. 2024-04-24T06:41:50.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T06:41:51.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.133+0000 7fad8b200640 1 Processor -- start 2024-04-24T06:41:51.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.133+0000 7fad8b200640 1 -- start start 2024-04-24T06:41:51.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.134+0000 7fad8b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad8c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad8b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fad8c0d0d90 0x7fad8c0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad8b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8c0cf030 con 0x7fad8c0d2630 2024-04-24T06:41:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad8b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fad8c0b7000 con 0x7fad8c0d0d90 2024-04-24T06:41:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad8b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0cf620 con 0x7fad8c15e7c0 2024-04-24T06:41:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad81400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fad8c0d2630 0x7fad8c0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46548/0 (socket says 172.21.15.59:46548) 2024-04-24T06:41:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad81400640 1 -- 172.21.15.59:0/3753720499 learned_addr learned my addr 172.21.15.59:0/3753720499 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829921770 0 0) 0x7fad74001110 con 0x7fad8c0d2630 2024-04-24T06:41:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad70003520 con 0x7fad8c0d2630 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad8a800640 1 --2- 172.21.15.59:0/3753720499 >> v2:172.21.15.66:3300/0 conn(0x7fad8c0d0d90 0x7fad8c0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.135+0000 7fad89e00640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad8c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2299101696 0 0) 0x7fad70003520 con 0x7fad8c0d2630 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 msgr2=0x7fad8c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad8c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 >> v2:172.21.15.66:3300/0 conn(0x7fad8c0d0d90 msgr2=0x7fad8c0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 --2- 172.21.15.59:0/3753720499 >> v2:172.21.15.66:3300/0 conn(0x7fad8c0d0d90 0x7fad8c0d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c0b6a40 con 0x7fad8c0d2630 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fad74001840 con 0x7fad8c0d2630 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 >> v1:172.21.15.59:6789/0 conn(0x7fad8c0d2630 legacy=0x7fad8c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad70004d40 0x7fad700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0b7000 con 0x7fad70004d40 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0cf620 con 0x7fad700039b0 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fad7000baf0 con 0x7fad8c15e7c0 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.136+0000 7fad8a800640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad70004d40 0x7fad700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad89e00640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad81400640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad8a800640 1 -- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 msgr2=0x7fad70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad8a800640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c15e7c0 0x7fad70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad8a800640 1 -- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700039b0 msgr2=0x7fad70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad8a800640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.137+0000 7fad8a800640 1 -- 172.21.15.59:0/3753720499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad70003520 con 0x7fad70004d40 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.138+0000 7fad8a800640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad70004d40 0x7fad700051c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fad84007360 tx=0x7fad70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e60580094e6f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.138+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84009be0 con 0x7fad70004d40 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.138+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8402e620 con 0x7fad70004d40 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.138+0000 7fad89400640 1 -- 172.21.15.59:0/3753720499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8402e9f0 con 0x7fad70004d40 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 -- 172.21.15.59:0/3753720499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad70004d40 msgr2=0x7fad700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 --2- 172.21.15.59:0/3753720499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad70004d40 0x7fad700051c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fad84007360 tx=0x7fad70005830 comp rx=0 tx=0).stop 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8a800640 1 -- 172.21.15.59:0/3753720499 reap_dead start 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 -- 172.21.15.59:0/3753720499 shutdown_connections 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 -- 172.21.15.59:0/3753720499 >> 172.21.15.59:0/3753720499 conn(0x7fad8c09d920 msgr2=0x7fad8c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 -- 172.21.15.59:0/3753720499 shutdown_connections 2024-04-24T06:41:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.139+0000 7fad8b200640 1 -- 172.21.15.59:0/3753720499 wait complete. 2024-04-24T06:41:51.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.141+0000 7fad8b200640 1 Processor -- start 2024-04-24T06:41:51.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.141+0000 7fad8b200640 1 -- start start 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70004d40 0x7fad8c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c0b2c30 0x7fad8c162370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0b6700 con 0x7fad8c157630 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0cf2b0 con 0x7fad70004d40 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0d1a80 con 0x7fad8c0b2c30 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41524/0 (socket says 172.21.15.59:41524) 2024-04-24T06:41:51.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.142+0000 7fad8a800640 1 -- 172.21.15.59:0/1160261049 learned_addr learned my addr 172.21.15.59:0/1160261049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:51.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 -- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c0b2c30 msgr2=0x7fad8c162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c0b2c30 0x7fad8c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 -- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70004d40 msgr2=0x7fad8c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70004d40 0x7fad8c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 -- 172.21.15.59:0/1160261049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c1e8b80 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad8a800640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad84036630 tx=0x7fad84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84046070 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad840368a0 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.143+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84009be0 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.144+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c1e8e20 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.144+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c1e93e0 con 0x7fad8c157630 2024-04-24T06:41:51.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.145+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c04fd90 con 0x7fad8c157630 2024-04-24T06:41:51.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.146+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad84049070 con 0x7fad8c157630 2024-04-24T06:41:51.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.147+0000 7fad82a00640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 0x7fad6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.147+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fad840ac290 con 0x7fad8c157630 2024-04-24T06:41:51.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.147+0000 7fad89e00640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 0x7fad6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.148+0000 7fad89e00640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 0x7fad6806ede0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006fd0 tx=0x7fad7c00d280 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:51.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.151+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad8403d070 con 0x7fad8c157630 2024-04-24T06:41:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.398+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fad8c0d1eb0 con 0x7fad8c157630 2024-04-24T06:41:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.399+0000 7fad82a00640 1 -- 172.21.15.59:0/1160261049 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) ==== 53+0+73 (secure 0 0 0) 0x7fad84032030 con 0x7fad8c157630 2024-04-24T06:41:51.400 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 msgr2=0x7fad6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 0x7fad6806ede0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006fd0 tx=0x7fad7c00d280 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 msgr2=0x7fad8c1628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad84036630 tx=0x7fad84004ab0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 shutdown_connections 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fad6806c920 0x7fad6806ede0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad8c157630 0x7fad8c1628b0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fad8c0b2c30 0x7fad8c162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.401+0000 7fad8b200640 1 --2- 172.21.15.59:0/1160261049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70004d40 0x7fad8c15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.402+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 >> 172.21.15.59:0/1160261049 conn(0x7fad8c09d920 msgr2=0x7fad8c160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.402+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 shutdown_connections 2024-04-24T06:41:51.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51.402+0000 7fad8b200640 1 -- 172.21.15.59:0/1160261049 wait complete. 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:41:51.423 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:51.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:51.424 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:51.475 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41833sec preferred_lft 41833sec 2024-04-24T06:41:51.476 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::dc26:c0ff:feba:2b/64 scope link 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 66:a9:f1:8a:24:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:51.477 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64a9:f1ff:fe8a:2432/64 scope link 2024-04-24T06:41:51.478 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.478 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T06:41:51.478 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 1a:27:cd:51:11:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:51.478 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::1827:cdff:fe51:11b6/64 scope link 2024-04-24T06:41:51.478 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:41:51.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:51.479 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:41:51.558 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:41:51.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:51.562 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-24T06:41:51.617 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-24T06:41:51.617 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:41:51.617 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:41:51.617 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:41:51.617 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:41:51.618 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:41:51.618 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-24T06:41:51.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:41:51.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:41:51.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:41:51.619 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-24T06:41:51.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:41:51.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:51.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:51.838 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:41:51.889 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:41:51.889 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:41:51.890 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:51.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:51.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-24T06:41:51.972 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T06:41:51.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:51.972 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:52.026 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:41:52.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:41:52.027 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:52.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:52.171 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:52.228 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:52.229 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:52.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:52.231 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:52.284 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T06:41:52.284 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T06:41:52.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-24T06:41:52.358 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T06:41:52.399 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-24T06:41:52.400 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T06:41:52.401 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:52.402 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:41:52.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:52.403 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-24T06:41:52.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-24T06:41:52.456+0000 7f8a609ee480 -1 init, newargv = 0x55ac62c455c0 newargc=15 2024-04-24T06:41:52.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79490]: starting ceph client 2024-04-24T06:41:52.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79490]: starting fuse 2024-04-24T06:41:52.487 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-24T06:41:52.487 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-24T06:41:52.488 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-24T06:41:52.489 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:52.692 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.79490.asok 2024-04-24T06:41:52.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:52.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79490.asok 2024-04-24T06:41:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:52.709 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79490.asok status 2024-04-24T06:41:52.942 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:41:52.942 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "79490", 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9843, 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-24T06:41:52.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9843 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2083637132 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2083637132 2024-04-24T06:41:52.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9843 192.168.0.1:0/2083637132", 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2083637132", 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 114, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 114, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-24T06:41:52.945 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:41:52.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:52.958 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-24T06:41:52.958 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:53.012 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-24T06:41:53.012 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:53.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:53.013 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:53.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:53.105 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-24T06:41:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-24T06:41:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-24T06:41:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-24T06:41:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-24T06:41:53.188 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-24T06:41:53.189 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/bravo 2024-04-24T06:41:53.189 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed 2024-04-24T06:41:53.189 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed/jfile 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-24T06:41:53.190 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:53.334 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-24T06:41:53.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:53.350 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:53.350 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:53.350 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:53.351 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:53.351 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-24T06:41:53.351 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:53.546 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-24T06:41:53.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:53.561 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:53.562 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:53.562 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:53.562 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:53.562 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-24T06:41:53.562 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:53.764 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-24T06:41:53.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:53.779 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:53.780 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:53.780 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:53.780 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:53.780 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-24T06:41:53.780 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:53.978 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-24T06:41:53.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-24T06:41:53.992 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:54.189 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-24T06:41:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:54.203 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:54.204 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:54.204 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:54.204 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:54.204 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-24T06:41:54.204 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:54.399 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-24T06:41:54.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-24T06:41:54.414 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:54.612 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-24T06:41:54.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:54.626 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-24T06:41:54.627 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-24T06:41:54.627 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-24T06:41:54.627 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-24T06:41:54.627 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-24T06:41:54.627 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-24T06:41:54.822 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-24T06:41:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:41:54.837 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T06:41:54.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:41:54.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T06:41:55.112 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T06:41:55.113 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:41:55.113 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:55.113 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:55.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:55.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.391+0000 7f274d400640 1 Processor -- start 2024-04-24T06:41:55.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.392+0000 7f274d400640 1 -- start start 2024-04-24T06:41:55.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.393+0000 7f274d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f274815e7c0 0x7f27480d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.393+0000 7f274d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f27480d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.393+0000 7f274d400640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27480cf030 con 0x7f27480d2630 2024-04-24T06:41:55.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.393+0000 7f274d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f27480d1740 con 0x7f274815e7c0 2024-04-24T06:41:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.393+0000 7f274d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27480b7000 con 0x7f27480d0d90 2024-04-24T06:41:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f274815e7c0 0x7f27480d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f274815e7c0 0x7f27480d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41526/0 (socket says 172.21.15.59:41526) 2024-04-24T06:41:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 -- 172.21.15.59:0/2835415551 learned_addr learned my addr 172.21.15.59:0/2835415551 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f274ca00640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f27480d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 -- 172.21.15.59:0/2835415551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 msgr2=0x7f27480d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f27480d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 -- 172.21.15.59:0/2835415551 >> v1:172.21.15.66:6789/0 conn(0x7f27480d2630 legacy=0x7f27480b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f2747e00640 1 -- 172.21.15.59:0/2835415551 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27480b6a40 con 0x7f274815e7c0 2024-04-24T06:41:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.394+0000 7f274ca00640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f27480d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.395+0000 7f2747e00640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f274815e7c0 0x7f27480d20f0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2738009870 tx=0x7f273802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ec025f59866a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f2746a00640 1 -- 172.21.15.59:0/2835415551 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738037070 con 0x7f274815e7c0 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f2746a00640 1 -- 172.21.15.59:0/2835415551 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2738032440 con 0x7f274815e7c0 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f2746a00640 1 -- 172.21.15.59:0/2835415551 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738032810 con 0x7f274815e7c0 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 -- 172.21.15.59:0/2835415551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f274815e7c0 msgr2=0x7f27480d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f274815e7c0 0x7f27480d20f0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2738009870 tx=0x7f273802a200 comp rx=0 tx=0).stop 2024-04-24T06:41:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 -- 172.21.15.59:0/2835415551 shutdown_connections 2024-04-24T06:41:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f27480d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 --2- 172.21.15.59:0/2835415551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f274815e7c0 0x7f27480d20f0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 -- 172.21.15.59:0/2835415551 >> 172.21.15.59:0/2835415551 conn(0x7f274809d920 msgr2=0x7f274808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 -- 172.21.15.59:0/2835415551 shutdown_connections 2024-04-24T06:41:55.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.396+0000 7f274d400640 1 -- 172.21.15.59:0/2835415551 wait complete. 2024-04-24T06:41:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.398+0000 7f274d400640 1 Processor -- start 2024-04-24T06:41:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.398+0000 7f274d400640 1 -- start start 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f274814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 0x7f27480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27480cf2b0 con 0x7f27480d2630 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27480cf620 con 0x7f274815e7c0 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f274d400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f27480b6700 con 0x7f27480d0d90 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.399+0000 7f2747400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41530/0 (socket says 172.21.15.59:41530) 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 -- 172.21.15.59:0/1365977487 learned_addr learned my addr 172.21.15.59:0/1365977487 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747e00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f274814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f274ca00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 0x7f27480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 -- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 msgr2=0x7f274814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f274814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 -- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 msgr2=0x7f27480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 0x7f27480ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 -- 172.21.15.59:0/1365977487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27480cd760 con 0x7f27480d2630 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f274ca00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 0x7f27480ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:55.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.400+0000 7f2747400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f273400c400 tx=0x7f273400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.401+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734011790 con 0x7f27480d2630 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.401+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2734017070 con 0x7f27480d2630 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.401+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734011bb0 con 0x7f27480d2630 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.401+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27480cda40 con 0x7f27480d2630 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.401+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27480ce000 con 0x7f27480d2630 2024-04-24T06:41:55.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.402+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27480ce3e0 con 0x7f27480d2630 2024-04-24T06:41:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.403+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f273401c460 con 0x7f27480d2630 2024-04-24T06:41:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.404+0000 7f273fe00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.405+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f273408ba10 con 0x7f27480d2630 2024-04-24T06:41:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.405+0000 7f2747e00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.405+0000 7f2747e00640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.410+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27480ce3e0 con 0x7f27480d2630 2024-04-24T06:41:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.617+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f274804fd90 con 0x7f27480d2630 2024-04-24T06:41:55.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.619+0000 7f273fe00640 1 -- 172.21.15.59:0/1365977487 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16956 (secure 0 0 0) 0x7f2734059de0 con 0x7f27480d2630 2024-04-24T06:41:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:55.621 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:43.864670+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.59:6835/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:55.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 msgr2=0x7f272806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 msgr2=0x7f274814b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f273400c400 tx=0x7f273400c8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 shutdown_connections 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274815e7c0 0x7f27480ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.622+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27480d2630 0x7f274814b120 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.623+0000 7f274d400640 1 --2- 172.21.15.59:0/1365977487 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f27480d0d90 0x7f274814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.623+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 >> 172.21.15.59:0/1365977487 conn(0x7f274809d920 msgr2=0x7f2748161140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.623+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 shutdown_connections 2024-04-24T06:41:55.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.623+0000 7f274d400640 1 -- 172.21.15.59:0/1365977487 wait complete. 2024-04-24T06:41:55.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:55.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:55.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.857+0000 7fef4f800640 1 Processor -- start 2024-04-24T06:41:55.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.858+0000 7fef4f800640 1 -- start start 2024-04-24T06:41:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.859+0000 7fef4f800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fef4815cb20 0x7fef480cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.859+0000 7fef4f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef48163770 0x7fef480d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.859+0000 7fef4f800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef480d01c0 con 0x7fef48160180 2024-04-24T06:41:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.859+0000 7fef4f800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fef4815cf00 con 0x7fef4815cb20 2024-04-24T06:41:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.859+0000 7fef4f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fef4815f6a0 con 0x7fef48163770 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fef4815cb20 0x7fef480cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fef4815cb20 0x7fef480cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41544/0 (socket says 172.21.15.59:41544) 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 -- 172.21.15.59:0/2782442507 learned_addr learned my addr 172.21.15.59:0/2782442507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef47e00640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef48163770 0x7fef480d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 -- 172.21.15.59:0/2782442507 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef48163770 msgr2=0x7fef480d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef48163770 0x7fef480d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 -- 172.21.15.59:0/2782442507 >> v1:172.21.15.66:6789/0 conn(0x7fef48160180 legacy=0x7fef480b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.860+0000 7fef4ce00640 1 -- 172.21.15.59:0/2782442507 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4815f980 con 0x7fef4815cb20 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.861+0000 7fef4ce00640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4815cb20 0x7fef480cf840 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009870 tx=0x7fef3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9bb32ee7b39907 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef47400640 1 -- 172.21.15.59:0/2782442507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c037070 con 0x7fef4815cb20 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef47400640 1 -- 172.21.15.59:0/2782442507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c032440 con 0x7fef4815cb20 2024-04-24T06:41:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef47400640 1 -- 172.21.15.59:0/2782442507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c032810 con 0x7fef4815cb20 2024-04-24T06:41:55.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 -- 172.21.15.59:0/2782442507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4815cb20 msgr2=0x7fef480cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4815cb20 0x7fef480cf840 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009870 tx=0x7fef3c02a180 comp rx=0 tx=0).stop 2024-04-24T06:41:55.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 -- 172.21.15.59:0/2782442507 shutdown_connections 2024-04-24T06:41:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef48163770 0x7fef480d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 --2- 172.21.15.59:0/2782442507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef4815cb20 0x7fef480cf840 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 -- 172.21.15.59:0/2782442507 >> 172.21.15.59:0/2782442507 conn(0x7fef4806cc70 msgr2=0x7fef4806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 -- 172.21.15.59:0/2782442507 shutdown_connections 2024-04-24T06:41:55.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.862+0000 7fef4f800640 1 -- 172.21.15.59:0/2782442507 wait complete. 2024-04-24T06:41:55.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.864+0000 7fef4f800640 1 Processor -- start 2024-04-24T06:41:55.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.864+0000 7fef4f800640 1 -- start start 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef4815cb20 0x7fef480c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef48160180 0x7fef480c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef480d1780 con 0x7fef48163770 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef480d1900 con 0x7fef48160180 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.865+0000 7fef4f800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fef480d1a80 con 0x7fef4815cb20 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41550/0 (socket says 172.21.15.59:41550) 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 -- 172.21.15.59:0/3702422935 learned_addr learned my addr 172.21.15.59:0/3702422935 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:55.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef47e00640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef48160180 0x7fef480c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 -- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef4815cb20 msgr2=0x7fef480c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4ce00640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef4815cb20 0x7fef480c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef4815cb20 0x7fef480c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 -- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef48160180 msgr2=0x7fef480c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef48160180 0x7fef480c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 -- 172.21.15.59:0/3702422935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef481f8520 con 0x7fef48163770 2024-04-24T06:41:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.866+0000 7fef4d800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c780 tx=0x7fef4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.867+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40004b60 con 0x7fef48163770 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.867+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef40004d00 con 0x7fef48163770 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.867+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000aa90 con 0x7fef48163770 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.867+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481f8800 con 0x7fef48163770 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.867+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481f8dc0 con 0x7fef48163770 2024-04-24T06:41:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.868+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef48068a80 con 0x7fef48163770 2024-04-24T06:41:55.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.869+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef4000ac30 con 0x7fef48163770 2024-04-24T06:41:55.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.870+0000 7fef44c00640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.870+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fef4008bf60 con 0x7fef48163770 2024-04-24T06:41:55.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.870+0000 7fef4ce00640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.871+0000 7fef4ce00640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 0x7fef2806ede0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:55.875+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef40020070 con 0x7fef48163770 2024-04-24T06:41:56.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.081+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef480d29c0 con 0x7fef48163770 2024-04-24T06:41:56.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.082+0000 7fef44c00640 1 -- 172.21.15.59:0/3702422935 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7fef4005a200 con 0x7fef48163770 2024-04-24T06:41:56.083 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:56.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 msgr2=0x7fef2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 0x7fef2806ede0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02e000 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 msgr2=0x7fef4815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c780 tx=0x7fef4000cc40 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 shutdown_connections 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef48163770 0x7fef4815db10 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef48160180 0x7fef480c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 --2- 172.21.15.59:0/3702422935 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fef4815cb20 0x7fef480c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 >> 172.21.15.59:0/3702422935 conn(0x7fef4806cc70 msgr2=0x7fef4815acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:56.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 shutdown_connections 2024-04-24T06:41:56.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.085+0000 7fef4f800640 1 -- 172.21.15.59:0/3702422935 wait complete. 2024-04-24T06:41:56.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.323+0000 7f464f600640 1 Processor -- start 2024-04-24T06:41:56.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.323+0000 7f464f600640 1 -- start start 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464f600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f464815e960 0x7f46480d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464f600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4648162c80 0x7f46480d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464f600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46481631c0 con 0x7f46480d1720 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464f600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f46480d1b80 con 0x7f4648162c80 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464f600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f46480d29b0 con 0x7f464815e960 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f4647e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f46480d1720 0x7f4648162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46556/0 (socket says 172.21.15.59:46556) 2024-04-24T06:41:56.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f464d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4648162c80 0x7f46480d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.325+0000 7f4647e00640 1 -- 172.21.15.59:0/2029650556 learned_addr learned my addr 172.21.15.59:0/2029650556 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f464cc00640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f464815e960 0x7f46480d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f4647400640 1 -- 172.21.15.59:0/2029650556 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979085151 0 0) 0x7f46481631c0 con 0x7f46480d1720 2024-04-24T06:41:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f4647400640 1 -- 172.21.15.59:0/2029650556 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4630003520 con 0x7f46480d1720 2024-04-24T06:41:56.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f464d600640 1 -- 172.21.15.59:0/2029650556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f464815e960 msgr2=0x7f46480d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:56.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f464d600640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f464815e960 0x7f46480d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f464d600640 1 -- 172.21.15.59:0/2029650556 >> v1:172.21.15.59:6789/0 conn(0x7f46480d1720 legacy=0x7f4648162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.326+0000 7f464d600640 1 -- 172.21.15.59:0/2029650556 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46480d2c90 con 0x7f4648162c80 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.327+0000 7f464d600640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f46480d3490 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c956654268cdabbc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f4647400640 1 -- 172.21.15.59:0/2029650556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640031600 con 0x7f4648162c80 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f4647400640 1 -- 172.21.15.59:0/2029650556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46400317a0 con 0x7f4648162c80 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f4647400640 1 -- 172.21.15.59:0/2029650556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640031b50 con 0x7f4648162c80 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f464f600640 1 -- 172.21.15.59:0/2029650556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 msgr2=0x7f46480d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f464f600640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f46480d3490 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a5d0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f464f600640 1 -- 172.21.15.59:0/2029650556 shutdown_connections 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f464f600640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f46480d3490 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.328+0000 7f464f600640 1 --2- 172.21.15.59:0/2029650556 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f464815e960 0x7f46480d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.329+0000 7f464f600640 1 -- 172.21.15.59:0/2029650556 >> 172.21.15.59:0/2029650556 conn(0x7f4648067760 msgr2=0x7f4648067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.329+0000 7f464f600640 1 -- 172.21.15.59:0/2029650556 shutdown_connections 2024-04-24T06:41:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.329+0000 7f464f600640 1 -- 172.21.15.59:0/2029650556 wait complete. 2024-04-24T06:41:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.330+0000 7f464f600640 1 Processor -- start 2024-04-24T06:41:56.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.331+0000 7f464f600640 1 -- start start 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.331+0000 7f464f600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46480d1720 0x7f46480b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.331+0000 7f464f600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815e960 0x7f46480cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464f600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f464815ed40 con 0x7f4648162c80 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464f600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46480b7930 con 0x7f464815e960 2024-04-24T06:41:56.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464f600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f464815eec0 con 0x7f46480d1720 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41570/0 (socket says 172.21.15.59:41570) 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 -- 172.21.15.59:0/4242629898 learned_addr learned my addr 172.21.15.59:0/4242629898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f4647e00640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815e960 0x7f46480cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464cc00640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46480d1720 0x7f46480b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 -- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46480d1720 msgr2=0x7f46480b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46480d1720 0x7f46480b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 -- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815e960 msgr2=0x7f46480cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815e960 0x7f46480cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.332+0000 7f464d600640 1 -- 172.21.15.59:0/4242629898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46480cbb20 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f464d600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f4640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640009450 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640031b50 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640003e10 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46480cbe00 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.333+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46481619c0 con 0x7f4648162c80 2024-04-24T06:41:56.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.334+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4648068a80 con 0x7f4648162c80 2024-04-24T06:41:56.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.335+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46400330d0 con 0x7f4648162c80 2024-04-24T06:41:56.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.337+0000 7f4644c00640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 0x7f462806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.337+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f46400accc0 con 0x7f4648162c80 2024-04-24T06:41:56.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.337+0000 7f464cc00640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 0x7f462806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.338+0000 7f464cc00640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 0x7f462806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f463c009a10 tx=0x7f463c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.342+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46400b3080 con 0x7f4648162c80 2024-04-24T06:41:56.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.545+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f46480d2430 con 0x7f4648162c80 2024-04-24T06:41:56.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.546+0000 7f4644c00640 1 -- 172.21.15.59:0/4242629898 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7f464003f030 con 0x7f4648162c80 2024-04-24T06:41:56.547 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:56.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 msgr2=0x7f462806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 0x7f462806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f463c009a10 tx=0x7f463c008040 comp rx=0 tx=0).stop 2024-04-24T06:41:56.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 msgr2=0x7f4648161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f4640004a40 comp rx=0 tx=0).stop 2024-04-24T06:41:56.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 shutdown_connections 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f462806c8f0 0x7f462806edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4648162c80 0x7f4648161480 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f464815e960 0x7f46480cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 --2- 172.21.15.59:0/4242629898 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f46480d1720 0x7f46480b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 >> 172.21.15.59:0/4242629898 conn(0x7f4648067760 msgr2=0x7f46480b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.549+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 shutdown_connections 2024-04-24T06:41:56.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.550+0000 7f464f600640 1 -- 172.21.15.59:0/4242629898 wait complete. 2024-04-24T06:41:56.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:41:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.782+0000 7fd14a000640 1 Processor -- start 2024-04-24T06:41:56.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.782+0000 7fd14a000640 1 -- start start 2024-04-24T06:41:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd14a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1440d0600 0x7fd1440ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd14a000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fd14406dcb0 0x7fd144162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd14a000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd144163240 con 0x7fd1440d09e0 2024-04-24T06:41:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd14a000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd144072a20 con 0x7fd1440d0600 2024-04-24T06:41:56.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd14a000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fd144071e10 con 0x7fd14406dcb0 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd148c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1440d0600 0x7fd1440ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd148c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1440d0600 0x7fd1440ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41580/0 (socket says 172.21.15.59:41580) 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd148c00640 1 -- 172.21.15.59:0/1681658324 learned_addr learned my addr 172.21.15.59:0/1681658324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd148c00640 1 -- 172.21.15.59:0/1681658324 >> v2:172.21.15.66:3300/0 conn(0x7fd14406dcb0 msgr2=0x7fd144162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.784+0000 7fd148c00640 1 --2- 172.21.15.59:0/1681658324 >> v2:172.21.15.66:3300/0 conn(0x7fd14406dcb0 0x7fd144162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.785+0000 7fd148c00640 1 -- 172.21.15.59:0/1681658324 >> v1:172.21.15.66:6789/0 conn(0x7fd1440d09e0 legacy=0x7fd14406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:56.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.785+0000 7fd148c00640 1 -- 172.21.15.59:0/1681658324 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1440720f0 con 0x7fd1440d0600 2024-04-24T06:41:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.786+0000 7fd148c00640 1 --2- 172.21.15.59:0/1681658324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d0600 0x7fd1440ad170 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd1440d11c0 tx=0x7fd13402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598f14e21b0d5370 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.786+0000 7fd143400640 1 -- 172.21.15.59:0/1681658324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134037070 con 0x7fd1440d0600 2024-04-24T06:41:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.786+0000 7fd143400640 1 -- 172.21.15.59:0/1681658324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd134032440 con 0x7fd1440d0600 2024-04-24T06:41:56.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.786+0000 7fd143400640 1 -- 172.21.15.59:0/1681658324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134032810 con 0x7fd1440d0600 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 -- 172.21.15.59:0/1681658324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d0600 msgr2=0x7fd1440ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 --2- 172.21.15.59:0/1681658324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d0600 0x7fd1440ad170 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd1440d11c0 tx=0x7fd13402a230 comp rx=0 tx=0).stop 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 -- 172.21.15.59:0/1681658324 shutdown_connections 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 --2- 172.21.15.59:0/1681658324 >> v2:172.21.15.66:3300/0 conn(0x7fd14406dcb0 0x7fd144162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 --2- 172.21.15.59:0/1681658324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d0600 0x7fd1440ad170 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 -- 172.21.15.59:0/1681658324 >> 172.21.15.59:0/1681658324 conn(0x7fd14406bdf0 msgr2=0x7fd14406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 -- 172.21.15.59:0/1681658324 shutdown_connections 2024-04-24T06:41:56.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.787+0000 7fd14a000640 1 -- 172.21.15.59:0/1681658324 wait complete. 2024-04-24T06:41:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.789+0000 7fd14a000640 1 Processor -- start 2024-04-24T06:41:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.789+0000 7fd14a000640 1 -- start start 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd14406dcb0 0x7fd144149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1440d0600 0x7fd14414a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd14415eaf0 con 0x7fd1440d09e0 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440726b0 con 0x7fd14406dcb0 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.790+0000 7fd14a000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd1440d0e90 con 0x7fd1440d0600 2024-04-24T06:41:56.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41588/0 (socket says 172.21.15.59:41588) 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 -- 172.21.15.59:0/2993376918 learned_addr learned my addr 172.21.15.59:0/2993376918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd148c00640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd14406dcb0 0x7fd144149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd143e00640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1440d0600 0x7fd14414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 -- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1440d0600 msgr2=0x7fd14414a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1440d0600 0x7fd14414a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 -- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd14406dcb0 msgr2=0x7fd144149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd14406dcb0 0x7fd144149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.791+0000 7fd149600640 1 -- 172.21.15.59:0/2993376918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd144156e90 con 0x7fd1440d09e0 2024-04-24T06:41:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd149600640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b990 tx=0x7fd13c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c019070 con 0x7fd1440d09e0 2024-04-24T06:41:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c00cba0 con 0x7fd1440d09e0 2024-04-24T06:41:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c014440 con 0x7fd1440d09e0 2024-04-24T06:41:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14415c680 con 0x7fd1440d09e0 2024-04-24T06:41:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.792+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1440cb000 con 0x7fd1440d09e0 2024-04-24T06:41:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.794+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14404fd90 con 0x7fd1440d09e0 2024-04-24T06:41:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.795+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd13c01c070 con 0x7fd1440d09e0 2024-04-24T06:41:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.796+0000 7fd140c00640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 0x7fd12006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.796+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fd13c08b930 con 0x7fd1440d09e0 2024-04-24T06:41:56.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.796+0000 7fd148c00640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 0x7fd12006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:56.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.797+0000 7fd148c00640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 0x7fd12006edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd134034000 tx=0x7fd13402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:56.801+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd13c059d00 con 0x7fd1440d09e0 2024-04-24T06:41:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.007+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1440cff50 con 0x7fd1440d09e0 2024-04-24T06:41:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.008+0000 7fd140c00640 1 -- 172.21.15.59:0/2993376918 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7fd13c059600 con 0x7fd1440d09e0 2024-04-24T06:41:57.009 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 msgr2=0x7fd12006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 0x7fd12006edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd134034000 tx=0x7fd13402e000 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 msgr2=0x7fd14415bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b990 tx=0x7fd13c00be50 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 shutdown_connections 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd12006c8f0 0x7fd12006edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1440d09e0 0x7fd14415bb10 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd1440d0600 0x7fd14414a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.010+0000 7fd14a000640 1 --2- 172.21.15.59:0/2993376918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd14406dcb0 0x7fd144149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.011+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 >> 172.21.15.59:0/2993376918 conn(0x7fd14406bdf0 msgr2=0x7fd1440cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:57.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.011+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 shutdown_connections 2024-04-24T06:41:57.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.011+0000 7fd14a000640 1 -- 172.21.15.59:0/2993376918 wait complete. 2024-04-24T06:41:57.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:41:57.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.244+0000 7f76a4200640 1 Processor -- start 2024-04-24T06:41:57.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.244+0000 7f76a4200640 1 -- start start 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f769c161e60 0x7f769c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c162de0 0x7f769c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f769c0afa80 0x7f769c0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f769c0aefd0 con 0x7f769c161e60 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f769c1632e0 con 0x7f769c0afa80 2024-04-24T06:41:57.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.246+0000 7f76a4200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0adcd0 con 0x7f769c162de0 2024-04-24T06:41:57.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f769c161e60 0x7f769c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f769c161e60 0x7f769c0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41604/0 (socket says 172.21.15.59:41604) 2024-04-24T06:41:57.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 -- 172.21.15.59:0/1440571430 learned_addr learned my addr 172.21.15.59:0/1440571430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a0e00640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c162de0 0x7f769c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a2200640 1 --2- 172.21.15.59:0/1440571430 >> v2:172.21.15.66:3300/0 conn(0x7f769c0afa80 0x7f769c0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 -- 172.21.15.59:0/1440571430 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c162de0 msgr2=0x7f769c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c162de0 0x7f769c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 -- 172.21.15.59:0/1440571430 >> v2:172.21.15.66:3300/0 conn(0x7f769c0afa80 msgr2=0x7f769c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 --2- 172.21.15.59:0/1440571430 >> v2:172.21.15.66:3300/0 conn(0x7f769c0afa80 0x7f769c0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.247+0000 7f76a1800640 1 -- 172.21.15.59:0/1440571430 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c0adfb0 con 0x7f769c161e60 2024-04-24T06:41:57.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.248+0000 7f76a1800640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c161e60 0x7f769c0aea90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f769c075670 tx=0x7f769002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c48d59492611ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f769be00640 1 -- 172.21.15.59:0/1440571430 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690037070 con 0x7f769c161e60 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f769be00640 1 -- 172.21.15.59:0/1440571430 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7690032440 con 0x7f769c161e60 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f769be00640 1 -- 172.21.15.59:0/1440571430 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690032810 con 0x7f769c161e60 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 -- 172.21.15.59:0/1440571430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c161e60 msgr2=0x7f769c0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c161e60 0x7f769c0aea90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f769c075670 tx=0x7f769002a140 comp rx=0 tx=0).stop 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 -- 172.21.15.59:0/1440571430 shutdown_connections 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 --2- 172.21.15.59:0/1440571430 >> v2:172.21.15.66:3300/0 conn(0x7f769c0afa80 0x7f769c0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c162de0 0x7f769c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 --2- 172.21.15.59:0/1440571430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c161e60 0x7f769c0aea90 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.249+0000 7f76a4200640 1 -- 172.21.15.59:0/1440571430 >> 172.21.15.59:0/1440571430 conn(0x7f769c0b2130 msgr2=0x7f769c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.250+0000 7f76a4200640 1 -- 172.21.15.59:0/1440571430 shutdown_connections 2024-04-24T06:41:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.250+0000 7f76a4200640 1 -- 172.21.15.59:0/1440571430 wait complete. 2024-04-24T06:41:57.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.251+0000 7f76a4200640 1 Processor -- start 2024-04-24T06:41:57.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.252+0000 7f76a4200640 1 -- start start 2024-04-24T06:41:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769c0afa80 0x7f769c1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c161e60 0x7f769c1f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0af480 con 0x7f769c162de0 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0ae570 con 0x7f769c0afa80 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a4200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0af600 con 0x7f769c161e60 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41606/0 (socket says 172.21.15.59:41606) 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 -- 172.21.15.59:0/3299421135 learned_addr learned my addr 172.21.15.59:0/3299421135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a1800640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769c0afa80 0x7f769c1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a0e00640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c161e60 0x7f769c1f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 -- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c161e60 msgr2=0x7f769c1f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c161e60 0x7f769c1f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 -- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769c0afa80 msgr2=0x7f769c1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769c0afa80 0x7f769c1f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.253+0000 7f76a2200640 1 -- 172.21.15.59:0/3299421135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c148c80 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f76a2200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f7694007e60 tx=0x7f769400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694014420 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7694019070 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694014820 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1f4060 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.254+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1489a0 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.255+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c068a80 con 0x7f769c162de0 2024-04-24T06:41:57.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.256+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76940149c0 con 0x7f769c162de0 2024-04-24T06:41:57.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.258+0000 7f7699600640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 0x7f767c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:57.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.258+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f769408c480 con 0x7f769c162de0 2024-04-24T06:41:57.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.258+0000 7f76a1800640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 0x7f767c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:57.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.259+0000 7f76a1800640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 0x7f767c06ede0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7690034000 tx=0x7f769002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:57.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.262+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f769405a7b0 con 0x7f769c162de0 2024-04-24T06:41:57.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.480+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f769c06cd70 con 0x7f769c162de0 2024-04-24T06:41:57.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.481+0000 7f7699600640 1 -- 172.21.15.59:0/3299421135 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v114) ==== 80+0+244 (secure 0 0 0) 0x7f769405a0b0 con 0x7f769c162de0 2024-04-24T06:41:57.482 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/3446399276 2024-04-25T06:41:43.864619+0000 2024-04-24T06:41:57.482 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6835/3446399276 2024-04-25T06:41:43.864619+0000 2024-04-24T06:41:57.482 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6835/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:57.482 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.66:6833/2441853137 2024-04-25T06:41:19.989739+0000 2024-04-24T06:41:57.482 INFO:teuthology.orchestra.run.smithi059.stderr:listed 4 entries 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 msgr2=0x7f767c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 0x7f767c06ede0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7690034000 tx=0x7f769002e000 comp rx=0 tx=0).stop 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 msgr2=0x7f769c147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f7694007e60 tx=0x7f769400e4b0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 shutdown_connections 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f767c06c920 0x7f767c06ede0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f769c162de0 0x7f769c147d50 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f769c161e60 0x7f769c1f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 --2- 172.21.15.59:0/3299421135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769c0afa80 0x7f769c1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:57.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 >> 172.21.15.59:0/3299421135 conn(0x7f769c0b2130 msgr2=0x7f769c0d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:57.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 shutdown_connections 2024-04-24T06:41:57.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:57.484+0000 7f76a4200640 1 -- 172.21.15.59:0/3299421135 wait complete. 2024-04-24T06:41:57.505 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-24T06:41:57.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:57.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T06:41:57.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79490]: fuse finished with error 0 and tester_r 0 2024-04-24T06:41:57.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:57.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:57.590 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:41:57.647 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:41:57.647 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:41:57.647 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:57.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:41:57.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:41:57.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:57.710 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T06:41:57.766 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-24T06:41:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:41:57.767 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:41:57.768 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:41:57.768 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:57.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:41:57.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:41:57.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:57.829 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T06:41:57.887 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-24T06:41:57.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:41:57.888 INFO:tasks.cephfs.mount:Terminating background process 2024-04-24T06:41:57.888 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:41:57.888 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:41:57.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:41:57.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:41:57.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:41:57.953 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T06:41:58.009 INFO:teuthology.orchestra.run.smithi066.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-24T06:41:58.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:41:58.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:58.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.231+0000 7fc88ee00640 1 Processor -- start 2024-04-24T06:41:58.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.232+0000 7fc88ee00640 1 -- start start 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc88815b450 0x7fc8880b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8880cd760 0x7fc8880ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8880b3870 con 0x7fc88806f3f0 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8880b41a0 con 0x7fc88815b450 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc8880b3b50 con 0x7fc8880cd760 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc88815b450 0x7fc8880b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.233+0000 7fc88da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc88815b450 0x7fc8880b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41614/0 (socket says 172.21.15.59:41614) 2024-04-24T06:41:58.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88da00640 1 -- 172.21.15.59:0/1201519994 learned_addr learned my addr 172.21.15.59:0/1201519994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:58.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88e400640 1 --2- 172.21.15.59:0/1201519994 >> v2:172.21.15.66:3300/0 conn(0x7fc8880cd760 0x7fc8880ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88da00640 1 -- 172.21.15.59:0/1201519994 >> v2:172.21.15.66:3300/0 conn(0x7fc8880cd760 msgr2=0x7fc8880ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:58.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88da00640 1 --2- 172.21.15.59:0/1201519994 >> v2:172.21.15.66:3300/0 conn(0x7fc8880cd760 0x7fc8880ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88da00640 1 -- 172.21.15.59:0/1201519994 >> v1:172.21.15.66:6789/0 conn(0x7fc88806f3f0 legacy=0x7fc8880cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:58.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.234+0000 7fc88da00640 1 -- 172.21.15.59:0/1201519994 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8880b3e30 con 0x7fc88815b450 2024-04-24T06:41:58.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.235+0000 7fc88da00640 1 --2- 172.21.15.59:0/1201519994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc88815b450 0x7fc8880b4630 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fc8880af020 tx=0x7fc87c02c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0db54a73bc49fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.235+0000 7fc887e00640 1 -- 172.21.15.59:0/1201519994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c038070 con 0x7fc88815b450 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.235+0000 7fc887e00640 1 -- 172.21.15.59:0/1201519994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c031710 con 0x7fc88815b450 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc887e00640 1 -- 172.21.15.59:0/1201519994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c031ae0 con 0x7fc88815b450 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 -- 172.21.15.59:0/1201519994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc88815b450 msgr2=0x7fc8880b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 --2- 172.21.15.59:0/1201519994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc88815b450 0x7fc8880b4630 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fc8880af020 tx=0x7fc87c02c4d0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 -- 172.21.15.59:0/1201519994 shutdown_connections 2024-04-24T06:41:58.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 --2- 172.21.15.59:0/1201519994 >> v2:172.21.15.66:3300/0 conn(0x7fc8880cd760 0x7fc8880ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 --2- 172.21.15.59:0/1201519994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc88815b450 0x7fc8880b4630 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 -- 172.21.15.59:0/1201519994 >> 172.21.15.59:0/1201519994 conn(0x7fc8880b21a0 msgr2=0x7fc8880b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:58.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 -- 172.21.15.59:0/1201519994 shutdown_connections 2024-04-24T06:41:58.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.236+0000 7fc88ee00640 1 -- 172.21.15.59:0/1201519994 wait complete. 2024-04-24T06:41:58.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.238+0000 7fc88ee00640 1 Processor -- start 2024-04-24T06:41:58.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.238+0000 7fc88ee00640 1 -- start start 2024-04-24T06:41:58.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc88806f3f0 0x7fc8880c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc88815b450 0x7fc8880722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880b4ff0 con 0x7fc8880cd760 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc88815b830 con 0x7fc88815b450 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.239+0000 7fc88ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8880b3080 con 0x7fc88806f3f0 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41628/0 (socket says 172.21.15.59:41628) 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 -- 172.21.15.59:0/788717499 learned_addr learned my addr 172.21.15.59:0/788717499 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88da00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc88806f3f0 0x7fc8880c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88e400640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc88815b450 0x7fc8880722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 -- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc88806f3f0 msgr2=0x7fc8880c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc88806f3f0 0x7fc8880c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 -- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc88815b450 msgr2=0x7fc8880722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc88815b450 0x7fc8880722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.240+0000 7fc88d000640 1 -- 172.21.15.59:0/788717499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88815b9b0 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc88d000640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c410 tx=0x7fc87400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874017070 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8740035f0 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874003a20 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88809dc50 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.241+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881f45f0 con 0x7fc8880cd760 2024-04-24T06:41:58.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.242+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc888068290 con 0x7fc8880cd760 2024-04-24T06:41:58.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.243+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc874003bc0 con 0x7fc8880cd760 2024-04-24T06:41:58.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.244+0000 7fc885600640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 0x7fc86806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.244+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fc87408ae80 con 0x7fc8880cd760 2024-04-24T06:41:58.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.245+0000 7fc88da00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 0x7fc86806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.245+0000 7fc88da00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 0x7fc86806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc87c035000 tx=0x7fc87c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.249+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc888068290 con 0x7fc8880cd760 2024-04-24T06:41:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.495+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88804fd90 con 0x7fc8880cd760 2024-04-24T06:41:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.497+0000 7fc885600640 1 -- 172.21.15.59:0/788717499 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5180 (secure 0 0 0) 0x7fc8740010c0 con 0x7fc8880cd760 2024-04-24T06:41:58.498 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:58.498 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:48.912512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9738,"qdb_cluster":[9738]},"id":8}]} 2024-04-24T06:41:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 114 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.499+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 msgr2=0x7fc86806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.499+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 0x7fc86806edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc87c035000 tx=0x7fc87c008030 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 msgr2=0x7fc888158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c410 tx=0x7fc87400c8d0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 shutdown_connections 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc86806c8f0 0x7fc86806edb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc88815b450 0x7fc8880722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8880cd760 0x7fc888158100 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 --2- 172.21.15.59:0/788717499 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc88806f3f0 0x7fc8880c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 >> 172.21.15.59:0/788717499 conn(0x7fc8880b21a0 msgr2=0x7fc8880cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 shutdown_connections 2024-04-24T06:41:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.500+0000 7fc88ee00640 1 -- 172.21.15.59:0/788717499 wait complete. 2024-04-24T06:41:58.519 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:58.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.728+0000 7f2d5e800640 1 Processor -- start 2024-04-24T06:41:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.728+0000 7f2d5e800640 1 -- start start 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.729+0000 7f2d5e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d58162fb0 0x7f2d581673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.729+0000 7f2d5e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d580d1a60 0x7f2d58163390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5e800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f2d581638d0 0x7f2d580c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2d58164420 con 0x7f2d58162fb0 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5e800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f2d58163e60 con 0x7f2d581638d0 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2d58164140 con 0x7f2d580d1a60 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d58162fb0 0x7f2d581673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d58162fb0 0x7f2d581673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41642/0 (socket says 172.21.15.59:41642) 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 -- 172.21.15.59:0/4238411386 learned_addr learned my addr 172.21.15.59:0/4238411386 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5ca00640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d580d1a60 0x7f2d58163390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 -- 172.21.15.59:0/4238411386 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d580d1a60 msgr2=0x7f2d58163390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d580d1a60 0x7f2d58163390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 -- 172.21.15.59:0/4238411386 >> v2:172.21.15.66:3300/0 conn(0x7f2d581638d0 msgr2=0x7f2d580c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 --2- 172.21.15.59:0/4238411386 >> v2:172.21.15.66:3300/0 conn(0x7f2d581638d0 0x7f2d580c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.730+0000 7f2d5d400640 1 -- 172.21.15.59:0/4238411386 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d58166bf0 con 0x7f2d58162fb0 2024-04-24T06:41:58.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.731+0000 7f2d5d400640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d58162fb0 0x7f2d581673f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009870 tx=0x7f2d4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b67c73946f69c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.732+0000 7f2d57e00640 1 -- 172.21.15.59:0/4238411386 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037070 con 0x7f2d58162fb0 2024-04-24T06:41:58.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.732+0000 7f2d57e00640 1 -- 172.21.15.59:0/4238411386 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c032440 con 0x7f2d58162fb0 2024-04-24T06:41:58.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.732+0000 7f2d57e00640 1 -- 172.21.15.59:0/4238411386 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c032810 con 0x7f2d58162fb0 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.732+0000 7f2d5e800640 1 -- 172.21.15.59:0/4238411386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d58162fb0 msgr2=0x7f2d581673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.732+0000 7f2d5e800640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d58162fb0 0x7f2d581673f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009870 tx=0x7f2d4c02a180 comp rx=0 tx=0).stop 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 -- 172.21.15.59:0/4238411386 shutdown_connections 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 --2- 172.21.15.59:0/4238411386 >> v2:172.21.15.66:3300/0 conn(0x7f2d581638d0 0x7f2d580c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d580d1a60 0x7f2d58163390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 --2- 172.21.15.59:0/4238411386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d58162fb0 0x7f2d581673f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 -- 172.21.15.59:0/4238411386 >> 172.21.15.59:0/4238411386 conn(0x7f2d5809aa00 msgr2=0x7f2d580d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 -- 172.21.15.59:0/4238411386 shutdown_connections 2024-04-24T06:41:58.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.733+0000 7f2d5e800640 1 -- 172.21.15.59:0/4238411386 wait complete. 2024-04-24T06:41:58.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.735+0000 7f2d5e800640 1 Processor -- start 2024-04-24T06:41:58.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.735+0000 7f2d5e800640 1 -- start start 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d58162fb0 0x7f2d580c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d581638d0 0x7f2d580c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580c47f0 con 0x7f2d580d1a60 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58167930 con 0x7f2d581638d0 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5e800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f2d580c4970 con 0x7f2d58162fb0 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41656/0 (socket says 172.21.15.59:41656) 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.736+0000 7f2d5d400640 1 -- 172.21.15.59:0/3392052459 learned_addr learned my addr 172.21.15.59:0/3392052459 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 -- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d58162fb0 msgr2=0x7f2d580c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d58162fb0 0x7f2d580c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 -- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d581638d0 msgr2=0x7f2d580c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d581638d0 0x7f2d580c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 -- 172.21.15.59:0/3392052459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580c7b90 con 0x7f2d580d1a60 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5d400640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037040 con 0x7f2d580d1a60 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c042070 con 0x7f2d580d1a60 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c030bb0 con 0x7f2d580d1a60 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.737+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d580af160 con 0x7f2d580d1a60 2024-04-24T06:41:58.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.738+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d580af720 con 0x7f2d580d1a60 2024-04-24T06:41:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.739+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5804fd90 con 0x7f2d580d1a60 2024-04-24T06:41:58.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.740+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d4c047460 con 0x7f2d580d1a60 2024-04-24T06:41:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.741+0000 7f2d55600640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.741+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f2d4c0ac930 con 0x7f2d580d1a60 2024-04-24T06:41:58.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.741+0000 7f2d5ca00640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:58.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.742+0000 7f2d5ca00640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2d581fc260 tx=0x7f2d48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:58.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.745+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d4c07abd0 con 0x7f2d580d1a60 2024-04-24T06:41:58.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.952+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d5806e8a0 con 0x7f2d580d1a60 2024-04-24T06:41:58.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.955+0000 7f2d55600640 1 -- 172.21.15.59:0/3392052459 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16956 (secure 0 0 0) 0x7f2d4c07a4d0 con 0x7f2d580d1a60 2024-04-24T06:41:58.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:41:58.957 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:43.864670+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.59:6835/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:58.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 msgr2=0x7f2d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2d581fc260 tx=0x7f2d48008040 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 msgr2=0x7f2d580ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0046c0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 shutdown_connections 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d581638d0 0x7f2d580c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f2d58162fb0 0x7f2d580c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 --2- 172.21.15.59:0/3392052459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d580d1a60 0x7f2d580ce3e0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:58.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 >> 172.21.15.59:0/3392052459 conn(0x7f2d5809aa00 msgr2=0x7f2d5806fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 shutdown_connections 2024-04-24T06:41:58.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:58.959+0000 7f2d5e800640 1 -- 172.21.15.59:0/3392052459 wait complete. 2024-04-24T06:41:58.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:28:33.226060+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:40:45.397925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:40:46.528123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:58.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:59.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.192+0000 7fba43e00640 1 Processor -- start 2024-04-24T06:41:59.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.192+0000 7fba43e00640 1 -- start start 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.193+0000 7fba43e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba4415ed20 0x7fba440d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3600 0x7fba440d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba440b5da0 con 0x7fba440b3f10 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fba4415f100 con 0x7fba4415ed20 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fba440d0a20 con 0x7fba440b3600 2024-04-24T06:41:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba42a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba4415ed20 0x7fba440d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fba440b3f10 0x7fba440b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45692/0 (socket says 172.21.15.59:45692) 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba43400640 1 -- 172.21.15.59:0/1261587727 learned_addr learned my addr 172.21.15.59:0/1261587727 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba42a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba4415ed20 0x7fba440d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45198/0 (socket says 172.21.15.59:45198) 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba42000640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3600 0x7fba440d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba42a00640 1 -- 172.21.15.59:0/1261587727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3600 msgr2=0x7fba440d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.194+0000 7fba42a00640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3600 0x7fba440d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.195+0000 7fba42a00640 1 -- 172.21.15.59:0/1261587727 >> v1:172.21.15.59:6789/0 conn(0x7fba440b3f10 legacy=0x7fba440b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.195+0000 7fba42a00640 1 -- 172.21.15.59:0/1261587727 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440d2ed0 con 0x7fba4415ed20 2024-04-24T06:41:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.196+0000 7fba42a00640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba440d0ea0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fba340097a0 tx=0x7fba3402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f0516f7b60a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.196+0000 7fba41600640 1 -- 172.21.15.59:0/1261587727 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34037070 con 0x7fba4415ed20 2024-04-24T06:41:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.196+0000 7fba41600640 1 -- 172.21.15.59:0/1261587727 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3402adb0 con 0x7fba4415ed20 2024-04-24T06:41:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.196+0000 7fba41600640 1 -- 172.21.15.59:0/1261587727 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34032660 con 0x7fba4415ed20 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 -- 172.21.15.59:0/1261587727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 msgr2=0x7fba440d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba440d0ea0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fba340097a0 tx=0x7fba3402a060 comp rx=0 tx=0).stop 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 -- 172.21.15.59:0/1261587727 shutdown_connections 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3600 0x7fba440d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 --2- 172.21.15.59:0/1261587727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba440d0ea0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 -- 172.21.15.59:0/1261587727 >> 172.21.15.59:0/1261587727 conn(0x7fba4408bf90 msgr2=0x7fba440d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 -- 172.21.15.59:0/1261587727 shutdown_connections 2024-04-24T06:41:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.197+0000 7fba43e00640 1 -- 172.21.15.59:0/1261587727 wait complete. 2024-04-24T06:41:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.199+0000 7fba43e00640 1 Processor -- start 2024-04-24T06:41:59.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.199+0000 7fba43e00640 1 -- start start 2024-04-24T06:41:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba440b3600 0x7fba440c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 0x7fba441f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba440d0290 con 0x7fba4415ed20 2024-04-24T06:41:59.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fba440d3490 con 0x7fba440b3600 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fba440d0410 con 0x7fba440b3f10 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45212/0 (socket says 172.21.15.59:45212) 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.201+0000 7fba43400640 1 -- 172.21.15.59:0/2309359622 learned_addr learned my addr 172.21.15.59:0/2309359622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba42a00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba440b3600 0x7fba440c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba42000640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 0x7fba441f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 -- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 msgr2=0x7fba441f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 0x7fba441f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 -- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba440b3600 msgr2=0x7fba440c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba440b3600 0x7fba440c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 -- 172.21.15.59:0/2309359622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440a0f40 con 0x7fba4415ed20 2024-04-24T06:41:59.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba42000640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 0x7fba441f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba43400640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00cbf0 tx=0x7fba3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.202+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c017070 con 0x7fba4415ed20 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.203+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3c00ad40 con 0x7fba4415ed20 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.203+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba440a1220 con 0x7fba4415ed20 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.203+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c012600 con 0x7fba4415ed20 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.203+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba440a17e0 con 0x7fba4415ed20 2024-04-24T06:41:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.204+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4404fd90 con 0x7fba4415ed20 2024-04-24T06:41:59.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.205+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba3c01a070 con 0x7fba4415ed20 2024-04-24T06:41:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.207+0000 7fba3aa00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 0x7fba2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.207+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fba3c08b8b0 con 0x7fba4415ed20 2024-04-24T06:41:59.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.207+0000 7fba42a00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 0x7fba2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.208+0000 7fba42a00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 0x7fba2006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba34034000 tx=0x7fba3402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.211+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba3c059590 con 0x7fba4415ed20 2024-04-24T06:41:59.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.415+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba440b1190 con 0x7fba4415ed20 2024-04-24T06:41:59.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.416+0000 7fba3aa00640 1 -- 172.21.15.59:0/2309359622 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7fba3c05d890 con 0x7fba4415ed20 2024-04-24T06:41:59.417 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-24T06:41:59.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.418+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 msgr2=0x7fba2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.418+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 0x7fba2006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba34034000 tx=0x7fba3402e000 comp rx=0 tx=0).stop 2024-04-24T06:41:59.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.418+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 msgr2=0x7fba44151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.418+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00cbf0 tx=0x7fba3c00a430 comp rx=0 tx=0).stop 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 shutdown_connections 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fba2006c8f0 0x7fba2006edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba4415ed20 0x7fba44151490 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fba440b3f10 0x7fba441f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 --2- 172.21.15.59:0/2309359622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba440b3600 0x7fba440c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 >> 172.21.15.59:0/2309359622 conn(0x7fba4408bf90 msgr2=0x7fba440b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 shutdown_connections 2024-04-24T06:41:59.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.419+0000 7fba43e00640 1 -- 172.21.15.59:0/2309359622 wait complete. 2024-04-24T06:41:59.438 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:41:59.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.655+0000 7f028ee00640 1 Processor -- start 2024-04-24T06:41:59.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.656+0000 7f028ee00640 1 -- start start 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.657+0000 7f028ee00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f028815e920 0x7f02880cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.657+0000 7f028ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02880ce0d0 con 0x7f028806c0e0 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.657+0000 7f028ee00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02880ce390 con 0x7f02880cdc40 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.657+0000 7f028ee00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f02880b0bb0 con 0x7f028815e920 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.658+0000 7f028e400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f028806c0e0 0x7f0288168660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45706/0 (socket says 172.21.15.59:45706) 2024-04-24T06:41:59.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.658+0000 7f028e400640 1 -- 172.21.15.59:0/1121486091 learned_addr learned my addr 172.21.15.59:0/1121486091 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.658+0000 7f028da00640 1 --2- 172.21.15.59:0/1121486091 >> v2:172.21.15.66:3300/0 conn(0x7f028815e920 0x7f02880cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.658+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1129655242 0 0) 0x7f02880ce0d0 con 0x7f028806c0e0 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.658+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0270003520 con 0x7f028806c0e0 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2922010407 0 0) 0x7f0270003520 con 0x7f028806c0e0 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 >> v2:172.21.15.66:3300/0 conn(0x7f028815e920 msgr2=0x7f02880cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 --2- 172.21.15.59:0/1121486091 >> v2:172.21.15.66:3300/0 conn(0x7f028815e920 0x7f02880cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 >> v1:172.21.15.66:6789/0 conn(0x7f02880cdc40 legacy=0x7f028806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880ce670 con 0x7f028806c0e0 2024-04-24T06:41:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f0280001a10 con 0x7f028806c0e0 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 >> v1:172.21.15.59:6789/0 conn(0x7f028806c0e0 legacy=0x7f0288168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 0x7f0270003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0270004800 0x7f0270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02700052e0 0x7f027000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f027000c1b0 con 0x7f0270003a00 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f027000c360 con 0x7f02700052e0 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f027000c560 con 0x7f0270004800 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.659+0000 7f028da00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 0x7f0270003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028e400640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02700052e0 0x7f027000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028d000640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0270004800 0x7f0270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028da00640 1 -- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0270004800 msgr2=0x7f0270004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028da00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f0270004800 0x7f0270004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028da00640 1 -- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02700052e0 msgr2=0x7f027000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028da00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02700052e0 0x7f027000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028da00640 1 -- 172.21.15.59:0/1121486091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0270003520 con 0x7f0270003a00 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028e400640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02700052e0 0x7f027000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.660+0000 7f028e400640 1 -- 172.21.15.59:0/1121486091 reap_dead start 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.661+0000 7f028da00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 0x7f0270003e20 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f027c009ac0 tx=0x7f0270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b7bbfda04fcc3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.661+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c036070 con 0x7f0270003a00 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027c031400 con 0x7f0270003a00 2024-04-24T06:41:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f0287e00640 1 -- 172.21.15.59:0/1121486091 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c009690 con 0x7f0270003a00 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 -- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 msgr2=0x7f0270003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 0x7f0270003e20 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f027c009ac0 tx=0x7f0270005830 comp rx=0 tx=0).stop 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 -- 172.21.15.59:0/1121486091 shutdown_connections 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 --2- 172.21.15.59:0/1121486091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0270003a00 0x7f0270003e20 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 -- 172.21.15.59:0/1121486091 >> 172.21.15.59:0/1121486091 conn(0x7f02880b2f10 msgr2=0x7f02880b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 -- 172.21.15.59:0/1121486091 shutdown_connections 2024-04-24T06:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.662+0000 7f028ee00640 1 -- 172.21.15.59:0/1121486091 wait complete. 2024-04-24T06:41:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.664+0000 7f028ee00640 1 Processor -- start 2024-04-24T06:41:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.664+0000 7f028ee00640 1 -- start start 2024-04-24T06:41:59.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 0x7f028809eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02880c6aa0 0x7f02880c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f028815ed00 con 0x7f02880c5540 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02880cec30 con 0x7f0270003a00 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.665+0000 7f028ee00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f028815ee80 con 0x7f02880c6aa0 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45230/0 (socket says 172.21.15.59:45230) 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 -- 172.21.15.59:0/1178600979 learned_addr learned my addr 172.21.15.59:0/1178600979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028da00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 0x7f028809eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 -- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02880c6aa0 msgr2=0x7f02880c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02880c6aa0 0x7f02880c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 -- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 msgr2=0x7f028809eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 0x7f028809eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 -- 172.21.15.59:0/1178600979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880c5ce0 con 0x7f02880c5540 2024-04-24T06:41:59.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028da00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 0x7f028809eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.666+0000 7f028d000640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f027400d4f0 tx=0x7f027400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.667+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274005480 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.667+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0274017070 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.667+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274003810 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.667+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02880c5fc0 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.667+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02880c6580 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.668+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028804fd90 con 0x7f02880c5540 2024-04-24T06:41:59.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.669+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02740039b0 con 0x7f02880c5540 2024-04-24T06:41:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.671+0000 7f0285600640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 0x7f026806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.671+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f027408b940 con 0x7f02880c5540 2024-04-24T06:41:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.671+0000 7f028da00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 0x7f026806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.671+0000 7f028da00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 0x7f026806ede0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f027c033000 tx=0x7f027c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.675+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0274059c70 con 0x7f02880c5540 2024-04-24T06:41:59.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.880+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f028806bb70 con 0x7f02880c5540 2024-04-24T06:41:59.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.881+0000 7f0285600640 1 -- 172.21.15.59:0/1178600979 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7f0274059570 con 0x7f02880c5540 2024-04-24T06:41:59.882 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.883+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 msgr2=0x7f026806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.883+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 0x7f026806ede0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f027c033000 tx=0x7f027c008030 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.883+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 msgr2=0x7f02880b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.883+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f027400d4f0 tx=0x7f027400d9b0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 shutdown_connections 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f026806c920 0x7f026806ede0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f02880c6aa0 0x7f02880c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02880c5540 0x7f02880b4250 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 --2- 172.21.15.59:0/1178600979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0270003a00 0x7f028809eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 >> 172.21.15.59:0/1178600979 conn(0x7f02880b2f10 msgr2=0x7f028806ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 shutdown_connections 2024-04-24T06:41:59.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:41:59.884+0000 7f028ee00640 1 -- 172.21.15.59:0/1178600979 wait complete. 2024-04-24T06:41:59.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:42:00.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.122+0000 7fb297e00640 1 Processor -- start 2024-04-24T06:42:00.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.122+0000 7fb297e00640 1 -- start start 2024-04-24T06:42:00.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.123+0000 7fb297e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb2980d15b0 0x7fb2980cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.123+0000 7fb297e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2980b5da0 con 0x7fb2980b4f20 2024-04-24T06:42:00.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.123+0000 7fb297e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2980d0a20 con 0x7fb29815ed20 2024-04-24T06:42:00.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.123+0000 7fb297e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb29815f100 con 0x7fb2980d15b0 2024-04-24T06:42:00.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb296000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb2980d15b0 0x7fb2980cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb297400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb2980b4f20 0x7fb2980cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45722/0 (socket says 172.21.15.59:45722) 2024-04-24T06:42:00.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb296000640 1 --2- 172.21.15.59:0/1463011458 >> v2:172.21.15.59:3300/0 conn(0x7fb2980d15b0 0x7fb2980cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45244/0 (socket says 172.21.15.59:45244) 2024-04-24T06:42:00.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb297400640 1 -- 172.21.15.59:0/1463011458 learned_addr learned my addr 172.21.15.59:0/1463011458 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:00.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb296000640 1 -- 172.21.15.59:0/1463011458 >> v1:172.21.15.66:6789/0 conn(0x7fb29815ed20 legacy=0x7fb2980d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:00.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb296000640 1 -- 172.21.15.59:0/1463011458 >> v1:172.21.15.59:6789/0 conn(0x7fb2980b4f20 legacy=0x7fb2980cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.124+0000 7fb296000640 1 -- 172.21.15.59:0/1463011458 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980d2ed0 con 0x7fb2980d15b0 2024-04-24T06:42:00.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.126+0000 7fb296000640 1 --2- 172.21.15.59:0/1463011458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb2980cf280 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7422b72155c0d38e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.126+0000 7fb295600640 1 -- 172.21.15.59:0/1463011458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280037070 con 0x7fb2980d15b0 2024-04-24T06:42:00.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.126+0000 7fb295600640 1 -- 172.21.15.59:0/1463011458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28002adc0 con 0x7fb2980d15b0 2024-04-24T06:42:00.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.126+0000 7fb295600640 1 -- 172.21.15.59:0/1463011458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280032660 con 0x7fb2980d15b0 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 -- 172.21.15.59:0/1463011458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 msgr2=0x7fb2980cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 --2- 172.21.15.59:0/1463011458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb2980cf280 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a0b0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 -- 172.21.15.59:0/1463011458 shutdown_connections 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 --2- 172.21.15.59:0/1463011458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb2980cf280 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 -- 172.21.15.59:0/1463011458 >> 172.21.15.59:0/1463011458 conn(0x7fb29808bf90 msgr2=0x7fb2980d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 -- 172.21.15.59:0/1463011458 shutdown_connections 2024-04-24T06:42:00.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.127+0000 7fb297e00640 1 -- 172.21.15.59:0/1463011458 wait complete. 2024-04-24T06:42:00.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.129+0000 7fb297e00640 1 Processor -- start 2024-04-24T06:42:00.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.129+0000 7fb297e00640 1 -- start start 2024-04-24T06:42:00.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2980b4f20 0x7fb2980c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb29815ed20 0x7fb2980ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980b3600 con 0x7fb2980d15b0 2024-04-24T06:42:00.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d3490 con 0x7fb2980b4f20 2024-04-24T06:42:00.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.130+0000 7fb297e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980b3780 con 0x7fb29815ed20 2024-04-24T06:42:00.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45260/0 (socket says 172.21.15.59:45260) 2024-04-24T06:42:00.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 -- 172.21.15.59:0/2640746638 learned_addr learned my addr 172.21.15.59:0/2640746638 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 -- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb29815ed20 msgr2=0x7fb2980ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb29815ed20 0x7fb2980ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 -- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2980b4f20 msgr2=0x7fb2980c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2980b4f20 0x7fb2980c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 -- 172.21.15.59:0/2640746638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980c86d0 con 0x7fb2980d15b0 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.131+0000 7fb296000640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fb28002a6a0 tx=0x7fb2800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.132+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280037040 con 0x7fb2980d15b0 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.132+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb280042070 con 0x7fb2980d15b0 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.132+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2980c89b0 con 0x7fb2980d15b0 2024-04-24T06:42:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.132+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28003b870 con 0x7fb2980d15b0 2024-04-24T06:42:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.132+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2980c8f70 con 0x7fb2980d15b0 2024-04-24T06:42:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.133+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2980b5810 con 0x7fb2980d15b0 2024-04-24T06:42:00.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.134+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb28003ba10 con 0x7fb2980d15b0 2024-04-24T06:42:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.135+0000 7fb28ea00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.135+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fb2800ac390 con 0x7fb2980d15b0 2024-04-24T06:42:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.136+0000 7fb296a00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.136+0000 7fb296a00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 0x7fb27406edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb288006fd0 tx=0x7fb288006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.140+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2980b5810 con 0x7fb2980d15b0 2024-04-24T06:42:00.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.345+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2980ce8d0 con 0x7fb2980d15b0 2024-04-24T06:42:00.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.346+0000 7fb28ea00640 1 -- 172.21.15.59:0/2640746638 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7fb280030e20 con 0x7fb2980d15b0 2024-04-24T06:42:00.347 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-24T06:42:00.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.348+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 msgr2=0x7fb27406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.348+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 0x7fb27406edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb288006fd0 tx=0x7fb288006bc0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.348+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 msgr2=0x7fb298156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.348+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fb28002a6a0 tx=0x7fb2800046c0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 shutdown_connections 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fb27406c8f0 0x7fb27406edb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fb29815ed20 0x7fb2980ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2980d15b0 0x7fb298156d30 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 --2- 172.21.15.59:0/2640746638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2980b4f20 0x7fb2980c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 >> 172.21.15.59:0/2640746638 conn(0x7fb29808bf90 msgr2=0x7fb2980cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:00.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 shutdown_connections 2024-04-24T06:42:00.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.349+0000 7fb297e00640 1 -- 172.21.15.59:0/2640746638 wait complete. 2024-04-24T06:42:00.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:42:00.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.577+0000 7f3a3de00640 1 Processor -- start 2024-04-24T06:42:00.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.577+0000 7f3a3de00640 1 -- start start 2024-04-24T06:42:00.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.578+0000 7f3a3de00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3a380cd760 0x7f3a380b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.578+0000 7f3a3de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a380b4690 0x7f3a380cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.578+0000 7f3a3de00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a38160210 con 0x7f3a3815c3d0 2024-04-24T06:42:00.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.578+0000 7f3a3de00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3a381604f0 con 0x7f3a380b4690 2024-04-24T06:42:00.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.578+0000 7f3a3de00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3a381607d0 con 0x7f3a380cd760 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a380b4690 0x7f3a380cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a380b4690 0x7f3a380cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45270/0 (socket says 172.21.15.59:45270) 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 -- 172.21.15.59:0/597171477 learned_addr learned my addr 172.21.15.59:0/597171477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a2fe00640 1 --2- 172.21.15.59:0/597171477 >> v2:172.21.15.66:3300/0 conn(0x7f3a380cd760 0x7f3a380b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 -- 172.21.15.59:0/597171477 >> v2:172.21.15.66:3300/0 conn(0x7f3a380cd760 msgr2=0x7f3a380b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 --2- 172.21.15.59:0/597171477 >> v2:172.21.15.66:3300/0 conn(0x7f3a380cd760 0x7f3a380b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 -- 172.21.15.59:0/597171477 >> v1:172.21.15.66:6789/0 conn(0x7f3a3815c3d0 legacy=0x7f3a380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:00.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.579+0000 7f3a3d400640 1 -- 172.21.15.59:0/597171477 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a38160ab0 con 0x7f3a380b4690 2024-04-24T06:42:00.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.580+0000 7f3a3d400640 1 --2- 172.21.15.59:0/597171477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a380b4690 0x7f3a380cb7d0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806e620 tx=0x7f3a3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c67144b3530482d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.580+0000 7f3a37e00640 1 -- 172.21.15.59:0/597171477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30036070 con 0x7f3a380b4690 2024-04-24T06:42:00.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a37e00640 1 -- 172.21.15.59:0/597171477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a30031600 con 0x7f3a380b4690 2024-04-24T06:42:00.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a37e00640 1 -- 172.21.15.59:0/597171477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300319d0 con 0x7f3a380b4690 2024-04-24T06:42:00.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 -- 172.21.15.59:0/597171477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a380b4690 msgr2=0x7f3a380cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 --2- 172.21.15.59:0/597171477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a380b4690 0x7f3a380cb7d0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3a3806e620 tx=0x7f3a3002a6f0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 -- 172.21.15.59:0/597171477 shutdown_connections 2024-04-24T06:42:00.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 --2- 172.21.15.59:0/597171477 >> v2:172.21.15.66:3300/0 conn(0x7f3a380cd760 0x7f3a380b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 --2- 172.21.15.59:0/597171477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a380b4690 0x7f3a380cb7d0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 -- 172.21.15.59:0/597171477 >> 172.21.15.59:0/597171477 conn(0x7f3a380b21a0 msgr2=0x7f3a380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:00.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 -- 172.21.15.59:0/597171477 shutdown_connections 2024-04-24T06:42:00.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.581+0000 7f3a3de00640 1 -- 172.21.15.59:0/597171477 wait complete. 2024-04-24T06:42:00.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.583+0000 7f3a3de00640 1 Processor -- start 2024-04-24T06:42:00.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.583+0000 7f3a3de00640 1 -- start start 2024-04-24T06:42:00.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a380b4690 0x7f3a380c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a380cd760 0x7f3a3815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3815c7b0 con 0x7f3a3815c3d0 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3a380b4ae0 con 0x7f3a380b4690 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3de00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3815c930 con 0x7f3a380cd760 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45280/0 (socket says 172.21.15.59:45280) 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.584+0000 7f3a3d400640 1 -- 172.21.15.59:0/3807613569 learned_addr learned my addr 172.21.15.59:0/3807613569 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 -- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a380cd760 msgr2=0x7f3a3815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a380cd760 0x7f3a3815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 -- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a380b4690 msgr2=0x7f3a380c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a380b4690 0x7f3a380c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 -- 172.21.15.59:0/3807613569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a380c74d0 con 0x7f3a3815c3d0 2024-04-24T06:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3d400640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009f40 tx=0x7f3a30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30036050 con 0x7f3a3815c3d0 2024-04-24T06:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a30042070 con 0x7f3a3815c3d0 2024-04-24T06:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30009510 con 0x7f3a3815c3d0 2024-04-24T06:42:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.585+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a380c77b0 con 0x7f3a3815c3d0 2024-04-24T06:42:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.586+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a380c6200 con 0x7f3a3815c3d0 2024-04-24T06:42:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.587+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3804fd90 con 0x7f3a3815c3d0 2024-04-24T06:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.588+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a30007bd0 con 0x7f3a3815c3d0 2024-04-24T06:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.589+0000 7f3a35600640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 0x7f3a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.589+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f3a300acb60 con 0x7f3a3815c3d0 2024-04-24T06:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.589+0000 7f3a3ca00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 0x7f3a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.590+0000 7f3a3ca00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 0x7f3a1406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.593+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a30045360 con 0x7f3a3815c3d0 2024-04-24T06:42:00.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.832+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a380b3c40 con 0x7f3a3815c3d0 2024-04-24T06:42:00.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.834+0000 7f3a35600640 1 -- 172.21.15.59:0/3807613569 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5180 (secure 0 0 0) 0x7f3a3007aea0 con 0x7f3a3815c3d0 2024-04-24T06:42:00.834 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:00.834 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:48.912512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9738,"qdb_cluster":[9738]},"id":8}]} 2024-04-24T06:42:00.835 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 114 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 msgr2=0x7f3a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 0x7f3a1406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006bc0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 msgr2=0x7f3a38073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009f40 tx=0x7f3a30004a40 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 shutdown_connections 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3a1406c8f0 0x7f3a1406edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a3815c3d0 0x7f3a38073040 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3a380cd760 0x7f3a3815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 --2- 172.21.15.59:0/3807613569 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a380b4690 0x7f3a380c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:00.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 >> 172.21.15.59:0/3807613569 conn(0x7f3a380b21a0 msgr2=0x7f3a380d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:00.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.836+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 shutdown_connections 2024-04-24T06:42:00.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:00.837+0000 7f3a3de00640 1 -- 172.21.15.59:0/3807613569 wait complete. 2024-04-24T06:42:00.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:42:01.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.055+0000 7fc8f0200640 1 Processor -- start 2024-04-24T06:42:01.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.055+0000 7fc8f0200640 1 -- start start 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e815c5e0 0x7fc8e80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80af920 0x7fc8e80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc8e80d06e0 0x7fc8e8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e80afd80 con 0x7fc8e815c5e0 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e815fa20 con 0x7fc8e80d06e0 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.057+0000 7fc8f0200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e815f0f0 con 0x7fc8e80af920 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e815c5e0 0x7fc8e80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e815c5e0 0x7fc8e80af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45292/0 (socket says 172.21.15.59:45292) 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 -- 172.21.15.59:0/3080163375 learned_addr learned my addr 172.21.15.59:0/3080163375 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 -- 172.21.15.59:0/3080163375 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80af920 msgr2=0x7fc8e80d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 --2- 172.21.15.59:0/3080163375 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80af920 0x7fc8e80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 -- 172.21.15.59:0/3080163375 >> v2:172.21.15.66:3300/0 conn(0x7fc8e80d06e0 msgr2=0x7fc8e8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:01.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 --2- 172.21.15.59:0/3080163375 >> v2:172.21.15.66:3300/0 conn(0x7fc8e80d06e0 0x7fc8e8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.058+0000 7fc8ed800640 1 -- 172.21.15.59:0/3080163375 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e815f3d0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.059+0000 7fc8ed800640 1 --2- 172.21.15.59:0/3080163375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80af3e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fc8e806df00 tx=0x7fc8dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323ec170a32aafe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3080163375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc037070 con 0x7fc8e815c5e0 2024-04-24T06:42:01.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3080163375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc032440 con 0x7fc8e815c5e0 2024-04-24T06:42:01.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3080163375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc032810 con 0x7fc8e815c5e0 2024-04-24T06:42:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 -- 172.21.15.59:0/3080163375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 msgr2=0x7fc8e80af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 --2- 172.21.15.59:0/3080163375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80af3e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fc8e806df00 tx=0x7fc8dc02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 -- 172.21.15.59:0/3080163375 shutdown_connections 2024-04-24T06:42:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 --2- 172.21.15.59:0/3080163375 >> v2:172.21.15.66:3300/0 conn(0x7fc8e80d06e0 0x7fc8e8071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 --2- 172.21.15.59:0/3080163375 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80af920 0x7fc8e80d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 --2- 172.21.15.59:0/3080163375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80af3e0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 -- 172.21.15.59:0/3080163375 >> 172.21.15.59:0/3080163375 conn(0x7fc8e8067ea0 msgr2=0x7fc8e806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:01.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 -- 172.21.15.59:0/3080163375 shutdown_connections 2024-04-24T06:42:01.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.060+0000 7fc8f0200640 1 -- 172.21.15.59:0/3080163375 wait complete. 2024-04-24T06:42:01.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.062+0000 7fc8f0200640 1 Processor -- start 2024-04-24T06:42:01.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.062+0000 7fc8f0200640 1 -- start start 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e80af920 0x7fc8e8153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80d06e0 0x7fc8e80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e80d0cf0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e80d0b70 con 0x7fc8e80af920 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8f0200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e806e480 con 0x7fc8e80d06e0 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8ee200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8ee200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45296/0 (socket says 172.21.15.59:45296) 2024-04-24T06:42:01.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.063+0000 7fc8ee200640 1 -- 172.21.15.59:0/2612314648 learned_addr learned my addr 172.21.15.59:0/2612314648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ed800640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e80af920 0x7fc8e8153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 -- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80d06e0 msgr2=0x7fc8e80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80d06e0 0x7fc8e80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 -- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e80af920 msgr2=0x7fc8e8153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e80af920 0x7fc8e8153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 -- 172.21.15.59:0/2612314648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e81544a0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8ee200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000b9a0 tx=0x7fc8e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0019070 con 0x7fc8e815c5e0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e000cbc0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.064+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e80b1a70 con 0x7fc8e815c5e0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.065+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0014440 con 0x7fc8e815c5e0 2024-04-24T06:42:01.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.065+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e80b0cb0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.066+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8068a80 con 0x7fc8e815c5e0 2024-04-24T06:42:01.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.067+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8e001c070 con 0x7fc8e815c5e0 2024-04-24T06:42:01.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.068+0000 7fc8e5600640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 0x7fc8c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.068+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7fc8e008b970 con 0x7fc8e815c5e0 2024-04-24T06:42:01.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.068+0000 7fc8ed800640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 0x7fc8c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.069+0000 7fc8ed800640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 0x7fc8c806ede0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc034000 tx=0x7fc8dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.072+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8e81544a0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.275+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8e80cc870 con 0x7fc8e815c5e0 2024-04-24T06:42:01.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.278+0000 7fc8e5600640 1 -- 172.21.15.59:0/2612314648 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16956 (secure 0 0 0) 0x7fc8e0059ca0 con 0x7fc8e815c5e0 2024-04-24T06:42:01.279 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:01.279 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:43.864670+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.59:6835/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 msgr2=0x7fc8c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 0x7fc8c806ede0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc034000 tx=0x7fc8dc02e000 comp rx=0 tx=0).stop 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 msgr2=0x7fc8e80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000b9a0 tx=0x7fc8e000be60 comp rx=0 tx=0).stop 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 shutdown_connections 2024-04-24T06:42:01.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc8c806c920 0x7fc8c806ede0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e815c5e0 0x7fc8e80cd470 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc8e80d06e0 0x7fc8e80aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 --2- 172.21.15.59:0/2612314648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8e80af920 0x7fc8e8153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 >> 172.21.15.59:0/2612314648 conn(0x7fc8e8067ea0 msgr2=0x7fc8e8162d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.280+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 shutdown_connections 2024-04-24T06:42:01.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.281+0000 7fc8f0200640 1 -- 172.21.15.59:0/2612314648 wait complete. 2024-04-24T06:42:01.300 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:42:01.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:42:01.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.505+0000 7f04fca00640 1 Processor -- start 2024-04-24T06:42:01.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.505+0000 7f04fca00640 1 -- start start 2024-04-24T06:42:01.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.506+0000 7f04fca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04f8160910 0x7f04f8160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.506+0000 7f04fca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f80cd520 con 0x7f04f80cd020 2024-04-24T06:42:01.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.506+0000 7f04fca00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f80cd800 con 0x7f04f81454e0 2024-04-24T06:42:01.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.506+0000 7f04fca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f04f80cddc0 con 0x7f04f8160910 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04f8160910 0x7f04f8160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04f8160910 0x7f04f8160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45302/0 (socket says 172.21.15.59:45302) 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 -- 172.21.15.59:0/2316473115 learned_addr learned my addr 172.21.15.59:0/2316473115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 -- 172.21.15.59:0/2316473115 >> v1:172.21.15.66:6789/0 conn(0x7f04f81454e0 legacy=0x7f04f8145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 -- 172.21.15.59:0/2316473115 >> v1:172.21.15.59:6789/0 conn(0x7f04f80cd020 legacy=0x7f04f80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.507+0000 7f04f6800640 1 -- 172.21.15.59:0/2316473115 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f80cdae0 con 0x7f04f8160910 2024-04-24T06:42:01.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.508+0000 7f04f6800640 1 --2- 172.21.15.59:0/2316473115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f8160910 0x7f04f8160d80 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009870 tx=0x7f04f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee25b96224fa00f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04f5400640 1 -- 172.21.15.59:0/2316473115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0037070 con 0x7f04f8160910 2024-04-24T06:42:01.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04f5400640 1 -- 172.21.15.59:0/2316473115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0032440 con 0x7f04f8160910 2024-04-24T06:42:01.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04f5400640 1 -- 172.21.15.59:0/2316473115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0032810 con 0x7f04f8160910 2024-04-24T06:42:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 -- 172.21.15.59:0/2316473115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f8160910 msgr2=0x7f04f8160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 --2- 172.21.15.59:0/2316473115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f8160910 0x7f04f8160d80 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009870 tx=0x7f04f002a210 comp rx=0 tx=0).stop 2024-04-24T06:42:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 -- 172.21.15.59:0/2316473115 shutdown_connections 2024-04-24T06:42:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 --2- 172.21.15.59:0/2316473115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f8160910 0x7f04f8160d80 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 -- 172.21.15.59:0/2316473115 >> 172.21.15.59:0/2316473115 conn(0x7f04f8067ea0 msgr2=0x7f04f806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:01.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 -- 172.21.15.59:0/2316473115 shutdown_connections 2024-04-24T06:42:01.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.509+0000 7f04fca00640 1 -- 172.21.15.59:0/2316473115 wait complete. 2024-04-24T06:42:01.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.511+0000 7f04fca00640 1 Processor -- start 2024-04-24T06:42:01.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.511+0000 7f04fca00640 1 -- start start 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 0x7f04f81f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 0x7f04f815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80b4010 con 0x7f04f81454e0 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81462c0 con 0x7f04f8160910 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.512+0000 7f04fca00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f04f813f550 con 0x7f04f80cd020 2024-04-24T06:42:01.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45312/0 (socket says 172.21.15.59:45312) 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 -- 172.21.15.59:0/748276498 learned_addr learned my addr 172.21.15.59:0/748276498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04f6800640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 0x7f04f815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 -- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 msgr2=0x7f04f81f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04f5e00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 0x7f04f81f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 0x7f04f81f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 -- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 msgr2=0x7f04f815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 0x7f04f815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04ed400640 1 -- 172.21.15.59:0/748276498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f815e5a0 con 0x7f04f81454e0 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04f6800640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 0x7f04f815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.513+0000 7f04f5e00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 0x7f04f81f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:01.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04ed400640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c400 tx=0x7f04e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00052d0 con 0x7f04f81454e0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e0005490 con 0x7f04f81454e0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f813f7a0 con 0x7f04f81454e0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0007260 con 0x7f04f81454e0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.514+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f813fd60 con 0x7f04f81454e0 2024-04-24T06:42:01.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.515+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8068a80 con 0x7f04f81454e0 2024-04-24T06:42:01.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.516+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04e0007400 con 0x7f04f81454e0 2024-04-24T06:42:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.517+0000 7f04eea00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 0x7f04d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.517+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f04e008b270 con 0x7f04f81454e0 2024-04-24T06:42:01.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.518+0000 7f04f5e00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 0x7f04d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.518+0000 7f04f5e00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 0x7f04d406edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04f815c1e0 tx=0x7f04e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.522+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04e0059510 con 0x7f04f81454e0 2024-04-24T06:42:01.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.768+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04f8160030 con 0x7f04f81454e0 2024-04-24T06:42:01.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.770+0000 7f04eea00640 1 -- 172.21.15.59:0/748276498 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5180 (secure 0 0 0) 0x7f04e0058e10 con 0x7f04f81454e0 2024-04-24T06:42:01.771 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:01.771 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:48.912512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9738,"qdb_cluster":[9738]},"id":8}]} 2024-04-24T06:42:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 114 2024-04-24T06:42:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 msgr2=0x7f04d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 0x7f04d406edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f04f815c1e0 tx=0x7f04e8009040 comp rx=0 tx=0).stop 2024-04-24T06:42:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 msgr2=0x7f04f813eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c400 tx=0x7f04e000c8c0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 shutdown_connections 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f04d406c8f0 0x7f04d406edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f8160910 0x7f04f815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.772+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04f81454e0 0x7f04f813eda0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.773+0000 7f04fca00640 1 --2- 172.21.15.59:0/748276498 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f04f80cd020 0x7f04f81f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.773+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 >> 172.21.15.59:0/748276498 conn(0x7f04f8067ea0 msgr2=0x7f04f806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.773+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 shutdown_connections 2024-04-24T06:42:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.773+0000 7f04fca00640 1 -- 172.21.15.59:0/748276498 wait complete. 2024-04-24T06:42:01.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:42:01.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.984+0000 7f3f07400640 1 Processor -- start 2024-04-24T06:42:01.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.984+0000 7f3f07400640 1 -- start start 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 0x7f3f000af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f000d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3f000d06e0 0x7f3f00071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000afd80 con 0x7f3f000d06e0 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f3f0015fa20 con 0x7f3f0015c5e0 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.985+0000 7f3f07400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0015f0f0 con 0x7f3f000af920 2024-04-24T06:42:01.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3f000d06e0 0x7f3f00071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3f000d06e0 0x7f3f00071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45316/0 (socket says 172.21.15.59:45316) 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 -- 172.21.15.59:0/413140609 learned_addr learned my addr 172.21.15.59:0/413140609 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 -- 172.21.15.59:0/413140609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 msgr2=0x7f3f000d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f04a00640 1 --2- 172.21.15.59:0/413140609 >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 0x7f3f000af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 --2- 172.21.15.59:0/413140609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f000d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 -- 172.21.15.59:0/413140609 >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 msgr2=0x7f3f000af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 --2- 172.21.15.59:0/413140609 >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 0x7f3f000af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f05400640 1 -- 172.21.15.59:0/413140609 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0015f3d0 con 0x7f3f000d06e0 2024-04-24T06:42:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.986+0000 7f3f04a00640 1 --2- 172.21.15.59:0/413140609 >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 0x7f3f000af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:01.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.987+0000 7f3f05400640 1 --2- 172.21.15.59:0/413140609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00071de0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f3f00072e60 tx=0x7f3ef802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d7fc0c6c6b5346 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3eff400640 1 -- 172.21.15.59:0/413140609 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8037070 con 0x7f3f000d06e0 2024-04-24T06:42:01.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3eff400640 1 -- 172.21.15.59:0/413140609 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef8032440 con 0x7f3f000d06e0 2024-04-24T06:42:01.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3eff400640 1 -- 172.21.15.59:0/413140609 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8032810 con 0x7f3f000d06e0 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 -- 172.21.15.59:0/413140609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 msgr2=0x7f3f00071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 --2- 172.21.15.59:0/413140609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00071de0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f3f00072e60 tx=0x7f3ef802a1e0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 -- 172.21.15.59:0/413140609 shutdown_connections 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 --2- 172.21.15.59:0/413140609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00071de0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 --2- 172.21.15.59:0/413140609 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f000d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 --2- 172.21.15.59:0/413140609 >> v2:172.21.15.66:3300/0 conn(0x7f3f0015c5e0 0x7f3f000af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.988+0000 7f3f07400640 1 -- 172.21.15.59:0/413140609 >> 172.21.15.59:0/413140609 conn(0x7f3f00067ea0 msgr2=0x7f3f0006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.989+0000 7f3f07400640 1 -- 172.21.15.59:0/413140609 shutdown_connections 2024-04-24T06:42:01.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.989+0000 7f3f07400640 1 -- 172.21.15.59:0/413140609 wait complete. 2024-04-24T06:42:01.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.990+0000 7f3f07400640 1 Processor -- start 2024-04-24T06:42:01.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.990+0000 7f3f07400640 1 -- start start 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f001404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0015c5e0 0x7f3f001572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d0cf0 con 0x7f3f000d06e0 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d0b70 con 0x7f3f0015c5e0 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.991+0000 7f3f07400640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0006e480 con 0x7f3f000af920 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45332/0 (socket says 172.21.15.59:45332) 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 -- 172.21.15.59:0/900319695 learned_addr learned my addr 172.21.15.59:0/900319695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3f05400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0015c5e0 0x7f3f001572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 -- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 msgr2=0x7f3f001404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f001404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 -- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0015c5e0 msgr2=0x7f3f001572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:01.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0015c5e0 0x7f3f001572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:01.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 -- 172.21.15.59:0/900319695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001e9c60 con 0x7f3f000d06e0 2024-04-24T06:42:01.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.992+0000 7f3effe00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800b4a0 tx=0x7f3ee800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.993+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8014070 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.993+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8010610 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.993+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8010a00 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.993+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001562b0 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.993+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f00156870 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.994+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00068a80 con 0x7f3f000d06e0 2024-04-24T06:42:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.995+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ee8017070 con 0x7f3f000d06e0 2024-04-24T06:42:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.996+0000 7f3efcc00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.996+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f3ee808b690 con 0x7f3f000d06e0 2024-04-24T06:42:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.996+0000 7f3f04a00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:01.997+0000 7f3f04a00640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 0x7f3edc06edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0008e00 tx=0x7f3ef0008330 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.001+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ee80599d0 con 0x7f3f000d06e0 2024-04-24T06:42:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.245+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f000ccb20 con 0x7f3f000d06e0 2024-04-24T06:42:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.246+0000 7f3efcc00640 1 -- 172.21.15.59:0/900319695 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5180 (secure 0 0 0) 0x7f3ee80592d0 con 0x7f3f000d06e0 2024-04-24T06:42:02.247 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:02.247 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T06:40:47.539600+0000","modified":"2024-04-24T06:41:48.912512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9738":{"gid":9738,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2269052051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2269052051},{"type":"v1","addr":"172.21.15.59:6837","nonce":2269052051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9738,"qdb_cluster":[9738]},"id":8}]} 2024-04-24T06:42:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 114 2024-04-24T06:42:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 msgr2=0x7f3edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 0x7f3edc06edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0008e00 tx=0x7f3ef0008330 comp rx=0 tx=0).stop 2024-04-24T06:42:02.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 msgr2=0x7f3f00156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800b4a0 tx=0x7f3ee800b960 comp rx=0 tx=0).stop 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 shutdown_connections 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f0015c5e0 0x7f3f001572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f000d06e0 0x7f3f00156d80 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.249+0000 7f3f07400640 1 --2- 172.21.15.59:0/900319695 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f3f000af920 0x7f3f001404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.250+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 >> 172.21.15.59:0/900319695 conn(0x7f3f00067ea0 msgr2=0x7f3f001622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.250+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 shutdown_connections 2024-04-24T06:42:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.250+0000 7f3f07400640 1 -- 172.21.15.59:0/900319695 wait complete. 2024-04-24T06:42:02.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:42:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.472+0000 7f7720200640 1 Processor -- start 2024-04-24T06:42:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.472+0000 7f7720200640 1 -- start start 2024-04-24T06:42:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.473+0000 7f7720200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f771815c5e0 0x7f77180af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f7720200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771815f0f0 con 0x7f77180d08b0 2024-04-24T06:42:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f7720200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771815f3d0 con 0x7f77180af920 2024-04-24T06:42:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f7720200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f77180afd80 con 0x7f771815c5e0 2024-04-24T06:42:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f771e200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f77180d08b0 0x7f77181624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45730/0 (socket says 172.21.15.59:45730) 2024-04-24T06:42:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f771e200640 1 -- 172.21.15.59:0/4046160962 learned_addr learned my addr 172.21.15.59:0/4046160962 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:02.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.474+0000 7f771d800640 1 --2- 172.21.15.59:0/4046160962 >> v2:172.21.15.66:3300/0 conn(0x7f771815c5e0 0x7f77180af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68536678 0 0) 0x7f771815f0f0 con 0x7f77180d08b0 2024-04-24T06:42:02.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7700003520 con 0x7f77180d08b0 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2344825557 0 0) 0x7f7700003520 con 0x7f77180d08b0 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 >> v2:172.21.15.66:3300/0 conn(0x7f771815c5e0 msgr2=0x7f77180af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 --2- 172.21.15.59:0/4046160962 >> v2:172.21.15.66:3300/0 conn(0x7f771815c5e0 0x7f77180af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 >> v1:172.21.15.66:6789/0 conn(0x7f77180af920 legacy=0x7f77180d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771815f6b0 con 0x7f77180d08b0 2024-04-24T06:42:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f77100019f0 con 0x7f77180d08b0 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 >> v1:172.21.15.59:6789/0 conn(0x7f77180d08b0 legacy=0x7f77181624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7700003a00 0x7f7700003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7700004800 0x7f7700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 0x7f770000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f770000c090 con 0x7f77000051c0 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f770000c240 con 0x7f7700003a00 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.475+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f770000c440 con 0x7f7700004800 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 0x7f770000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771d800640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7700003a00 0x7f7700003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771ce00640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7700004800 0x7f7700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 -- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7700004800 msgr2=0x7f7700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7700004800 0x7f7700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 -- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7700003a00 msgr2=0x7f7700003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7700003a00 0x7f7700003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771e200640 1 -- 172.21.15.59:0/4046160962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7700003520 con 0x7f77000051c0 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.476+0000 7f771ce00640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7700004800 0x7f7700004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.477+0000 7f771e200640 1 -- 172.21.15.59:0/4046160962 reap_dead start 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.477+0000 7f771e200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 0x7f770000b9c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f771000a790 tx=0x7f7700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff75c70d1cff4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.477+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771000a3b0 con 0x7f77000051c0 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771002f070 con 0x7f77000051c0 2024-04-24T06:42:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7717e00640 1 -- 172.21.15.59:0/4046160962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771002bc00 con 0x7f77000051c0 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 -- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 msgr2=0x7f770000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 0x7f770000b9c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f771000a790 tx=0x7f7700005830 comp rx=0 tx=0).stop 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 -- 172.21.15.59:0/4046160962 shutdown_connections 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 --2- 172.21.15.59:0/4046160962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77000051c0 0x7f770000b9c0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 -- 172.21.15.59:0/4046160962 >> 172.21.15.59:0/4046160962 conn(0x7f7718067ea0 msgr2=0x7f771806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 -- 172.21.15.59:0/4046160962 shutdown_connections 2024-04-24T06:42:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.478+0000 7f7720200640 1 -- 172.21.15.59:0/4046160962 wait complete. 2024-04-24T06:42:02.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.480+0000 7f7720200640 1 Processor -- start 2024-04-24T06:42:02.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.480+0000 7f7720200640 1 -- start start 2024-04-24T06:42:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f77000051c0 0x7f771814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77180b19a0 0x7f77180ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77180af3e0 con 0x7f77181f7a00 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f771815fa20 con 0x7f77180b19a0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f7720200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f77180af560 con 0x7f77000051c0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f771e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f771e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45350/0 (socket says 172.21.15.59:45350) 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.481+0000 7f771e200640 1 -- 172.21.15.59:0/2267180821 learned_addr learned my addr 172.21.15.59:0/2267180821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771ce00640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77180b19a0 0x7f77180ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771d800640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f77000051c0 0x7f771814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 -- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f77000051c0 msgr2=0x7f771814bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f77000051c0 0x7f771814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 -- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77180b19a0 msgr2=0x7f77180ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77180b19a0 0x7f77180ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 -- 172.21.15.59:0/2267180821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180b1e20 con 0x7f77181f7a00 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f771e200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f771000a880 tx=0x7f771002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771002b7a0 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771002b940 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180cd2e0 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.482+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771002bcf0 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.483+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77180cf9a0 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.484+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718068a80 con 0x7f77181f7a00 2024-04-24T06:42:02.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.485+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7710034470 con 0x7f77181f7a00 2024-04-24T06:42:02.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.486+0000 7f7715600640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.486+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f77100abe80 con 0x7f77181f7a00 2024-04-24T06:42:02.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.486+0000 7f771d800640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.487+0000 7f771d800640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 0x7f76fc06edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f770c0098d0 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.490+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77180b1e20 con 0x7f77181f7a00 2024-04-24T06:42:02.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.699+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f771806e260 con 0x7f77181f7a00 2024-04-24T06:42:02.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.702+0000 7f7715600640 1 -- 172.21.15.59:0/2267180821 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16956 (secure 0 0 0) 0x7f771007a120 con 0x7f77181f7a00 2024-04-24T06:42:02.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:02.703 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":114,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:41:43.864670+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:40:45.397925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-24T06:40:46.528123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.59:6835/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:42:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.704+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 msgr2=0x7f76fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.704+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 0x7f76fc06edb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f770c0098d0 tx=0x7f770c008040 comp rx=0 tx=0).stop 2024-04-24T06:42:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 msgr2=0x7f77181f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f771000a880 tx=0x7f771002d070 comp rx=0 tx=0).stop 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 shutdown_connections 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77181f7a00 0x7f77181f7e80 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77180b19a0 0x7f77180ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 --2- 172.21.15.59:0/2267180821 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f77000051c0 0x7f771814bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 >> 172.21.15.59:0/2267180821 conn(0x7f7718067ea0 msgr2=0x7f77180b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 shutdown_connections 2024-04-24T06:42:02.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.705+0000 7f7720200640 1 -- 172.21.15.59:0/2267180821 wait complete. 2024-04-24T06:42:02.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:42:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.938+0000 7f7d6a000640 1 Processor -- start 2024-04-24T06:42:02.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.938+0000 7f7d6a000640 1 -- start start 2024-04-24T06:42:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.939+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d640cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.939+0000 7f7d6a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d640d0cd0 0x7f7d640b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.939+0000 7f7d6a000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d640b4cf0 con 0x7f7d640b4230 2024-04-24T06:42:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.939+0000 7f7d6a000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6415ca50 con 0x7f7d640d0cd0 2024-04-24T06:42:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.939+0000 7f7d6a000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7d640ccb80 con 0x7f7d6415c670 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d640d0cd0 0x7f7d640b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d640d0cd0 0x7f7d640b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45366/0 (socket says 172.21.15.59:45366) 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 -- 172.21.15.59:0/3577516796 learned_addr learned my addr 172.21.15.59:0/3577516796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d68c00640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d640cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 -- 172.21.15.59:0/3577516796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 msgr2=0x7f7d640cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:02.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d640cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 -- 172.21.15.59:0/3577516796 >> v1:172.21.15.66:6789/0 conn(0x7f7d640b4230 legacy=0x7f7d640d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:02.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.940+0000 7f7d69600640 1 -- 172.21.15.59:0/3577516796 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d640ccdd0 con 0x7f7d640d0cd0 2024-04-24T06:42:02.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.941+0000 7f7d69600640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d640b4690 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009e50 tx=0x7f7d5c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cc700a8c6e1778 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d63400640 1 -- 172.21.15.59:0/3577516796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c036070 con 0x7f7d640d0cd0 2024-04-24T06:42:02.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d63400640 1 -- 172.21.15.59:0/3577516796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c031600 con 0x7f7d640d0cd0 2024-04-24T06:42:02.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d63400640 1 -- 172.21.15.59:0/3577516796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0319d0 con 0x7f7d640d0cd0 2024-04-24T06:42:02.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d6a000640 1 -- 172.21.15.59:0/3577516796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 msgr2=0x7f7d640b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d6a000640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d640b4690 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009e50 tx=0x7f7d5c02a6d0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d6a000640 1 -- 172.21.15.59:0/3577516796 shutdown_connections 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d6a000640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d640b4690 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.942+0000 7f7d6a000640 1 --2- 172.21.15.59:0/3577516796 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d640cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.943+0000 7f7d6a000640 1 -- 172.21.15.59:0/3577516796 >> 172.21.15.59:0/3577516796 conn(0x7f7d6406bdf0 msgr2=0x7f7d6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.943+0000 7f7d6a000640 1 -- 172.21.15.59:0/3577516796 shutdown_connections 2024-04-24T06:42:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.943+0000 7f7d6a000640 1 -- 172.21.15.59:0/3577516796 wait complete. 2024-04-24T06:42:02.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.944+0000 7f7d6a000640 1 Processor -- start 2024-04-24T06:42:02.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.944+0000 7f7d6a000640 1 -- start start 2024-04-24T06:42:02.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d640b4230 0x7f7d64151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d641f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640b3e60 con 0x7f7d640d0cd0 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7d640cd390 con 0x7f7d640b4230 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.945+0000 7f7d6a000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7d640b3fe0 con 0x7f7d6415c670 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45380/0 (socket says 172.21.15.59:45380) 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 -- 172.21.15.59:0/451052749 learned_addr learned my addr 172.21.15.59:0/451052749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d69600640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d641f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d68c00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d640b4230 0x7f7d64151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 -- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 msgr2=0x7f7d641f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d641f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 -- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d640b4230 msgr2=0x7f7d64151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d640b4230 0x7f7d64151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.946+0000 7f7d63e00640 1 -- 172.21.15.59:0/451052749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6414a060 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d63e00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c430 tx=0x7f7d5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011790 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54017070 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011b70 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d64152440 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.947+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d64152a00 con 0x7f7d640d0cd0 2024-04-24T06:42:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.948+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6404fd90 con 0x7f7d640d0cd0 2024-04-24T06:42:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.949+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d54011d10 con 0x7f7d640d0cd0 2024-04-24T06:42:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.950+0000 7f7d60c00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 0x7f7d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.950+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5939+0+0 (secure 0 0 0) 0x7f7d5408bad0 con 0x7f7d640d0cd0 2024-04-24T06:42:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.951+0000 7f7d68c00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 0x7f7d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:02.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.951+0000 7f7d68c00640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 0x7f7d4806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:02.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:02.954+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d54059e00 con 0x7f7d640d0cd0 2024-04-24T06:42:03.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.197+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d640ceab0 con 0x7f7d640d0cd0 2024-04-24T06:42:03.199 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-24T06:42:03.198+0000 7f648d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:42:03.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.202+0000 7f7d60c00640 1 -- 172.21.15.59:0/451052749 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7f7d54059700 con 0x7f7d640d0cd0 2024-04-24T06:42:03.203 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:42:03.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 msgr2=0x7f7d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 0x7f7d4806edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58008040 comp rx=0 tx=0).stop 2024-04-24T06:42:03.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 msgr2=0x7f7d64153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c430 tx=0x7f7d5400c8f0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 shutdown_connections 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7d4806c8f0 0x7f7d4806edb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7d6415c670 0x7f7d641f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d640d0cd0 0x7f7d64153ec0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 --2- 172.21.15.59:0/451052749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d640b4230 0x7f7d64151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.205+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 >> 172.21.15.59:0/451052749 conn(0x7f7d6406bdf0 msgr2=0x7f7d64072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.206+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 shutdown_connections 2024-04-24T06:42:03.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.206+0000 7f7d6a000640 1 -- 172.21.15.59:0/451052749 wait complete. 2024-04-24T06:42:03.211 INFO:tasks.ceph.mds.a.smithi059.stderr: -1543> 2024-04-24T06:41:49.882+0000 7f2e4a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:42:03.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:42:03.259 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-24T06:42:03.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.408+0000 7fddc5a00640 1 Processor -- start 2024-04-24T06:42:03.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.409+0000 7fddc5a00640 1 -- start start 2024-04-24T06:42:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.410+0000 7fddc5a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fddc00724d0 0x7fddc0162240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.410+0000 7fddc5a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.410+0000 7fddc5a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc00728b0 con 0x7fddc006fb50 2024-04-24T06:42:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.410+0000 7fddc5a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fddc00b40a0 con 0x7fddc00724d0 2024-04-24T06:42:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.410+0000 7fddc5a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fddc0162a70 con 0x7fddc00d05d0 2024-04-24T06:42:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.411+0000 7fddc5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.411+0000 7fddc5000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:45468/0 (socket says 172.21.15.59:45468) 2024-04-24T06:42:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.411+0000 7fddc5000640 1 -- 172.21.15.59:0/3828941076 learned_addr learned my addr 172.21.15.59:0/3828941076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.411+0000 7fddbfe00640 1 --2- 172.21.15.59:0/3828941076 >> v2:172.21.15.66:3300/0 conn(0x7fddc00724d0 0x7fddc0162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.412+0000 7fddc5000640 1 -- 172.21.15.59:0/3828941076 >> v2:172.21.15.66:3300/0 conn(0x7fddc00724d0 msgr2=0x7fddc0162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.412+0000 7fddc5000640 1 --2- 172.21.15.59:0/3828941076 >> v2:172.21.15.66:3300/0 conn(0x7fddc00724d0 0x7fddc0162240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.412+0000 7fddc5000640 1 -- 172.21.15.59:0/3828941076 >> v1:172.21.15.66:6789/0 conn(0x7fddc006fb50 legacy=0x7fddc00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.412+0000 7fddc5000640 1 -- 172.21.15.59:0/3828941076 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc0162d50 con 0x7fddc00d05d0 2024-04-24T06:42:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.414+0000 7fddc5000640 1 --2- 172.21.15.59:0/3828941076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009e80 tx=0x7fddb802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da72502a24145c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.414+0000 7fddbea00640 1 -- 172.21.15.59:0/3828941076 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8036070 con 0x7fddc00d05d0 2024-04-24T06:42:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.414+0000 7fddbea00640 1 -- 172.21.15.59:0/3828941076 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8031600 con 0x7fddc00d05d0 2024-04-24T06:42:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.414+0000 7fddbea00640 1 -- 172.21.15.59:0/3828941076 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80319d0 con 0x7fddc00d05d0 2024-04-24T06:42:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 -- 172.21.15.59:0/3828941076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 msgr2=0x7fddc00d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 --2- 172.21.15.59:0/3828941076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009e80 tx=0x7fddb802a640 comp rx=0 tx=0).stop 2024-04-24T06:42:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 -- 172.21.15.59:0/3828941076 shutdown_connections 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 --2- 172.21.15.59:0/3828941076 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00d05d0 0x7fddc00d0a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 --2- 172.21.15.59:0/3828941076 >> v2:172.21.15.66:3300/0 conn(0x7fddc00724d0 0x7fddc0162240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 -- 172.21.15.59:0/3828941076 >> 172.21.15.59:0/3828941076 conn(0x7fddc006bdf0 msgr2=0x7fddc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 -- 172.21.15.59:0/3828941076 shutdown_connections 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.415+0000 7fddc5a00640 1 -- 172.21.15.59:0/3828941076 wait complete. 2024-04-24T06:42:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.417+0000 7fddc5a00640 1 Processor -- start 2024-04-24T06:42:03.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.417+0000 7fddc5a00640 1 -- start start 2024-04-24T06:42:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc006fb50 0x7fddc0149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00724d0 0x7fddc009ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddc00b3470 con 0x7fddc00d05d0 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fddc00b3190 con 0x7fddc006fb50 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fddc0162780 con 0x7fddc00724d0 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45414/0 (socket says 172.21.15.59:45414) 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.418+0000 7fddc5000640 1 -- 172.21.15.59:0/279089643 learned_addr learned my addr 172.21.15.59:0/279089643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddbfe00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc006fb50 0x7fddc0149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddbf400640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00724d0 0x7fddc009ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 -- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00724d0 msgr2=0x7fddc009ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00724d0 0x7fddc009ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 -- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc006fb50 msgr2=0x7fddc0149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc006fb50 0x7fddc0149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 -- 172.21.15.59:0/279089643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc009d510 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.419+0000 7fddc5000640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fddb8034040 tx=0x7fddb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.420+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8036040 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.420+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8003990 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.420+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009510 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.420+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc009d7f0 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.420+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc009ddb0 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.421+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc015b880 con 0x7fddc00d05d0 2024-04-24T06:42:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.422+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddb80077a0 con 0x7fddc00d05d0 2024-04-24T06:42:03.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.423+0000 7fddb7e00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 0x7fdd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:03.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.423+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6025+0+0 (secure 0 0 0) 0x7fddb80acec0 con 0x7fddc00d05d0 2024-04-24T06:42:03.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.423+0000 7fddbfe00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 0x7fdd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:03.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.424+0000 7fddbfe00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 0x7fdd9c06edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fddc009bdc0 tx=0x7fddb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:03.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.428+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddc015b880 con 0x7fddc00d05d0 2024-04-24T06:42:03.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:03.677+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fddc004fd90 con 0x7fddc00d05d0 2024-04-24T06:42:04.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.206+0000 7fddb7e00640 1 -- 172.21.15.59:0/279089643 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) ==== 104+0+0 (secure 0 0 0) 0x7fddb807b110 con 0x7fddc00d05d0 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 msgr2=0x7fdd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 0x7fdd9c06edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fddc009bdc0 tx=0x7fddb0008040 comp rx=0 tx=0).stop 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 msgr2=0x7fddc00ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fddb8034040 tx=0x7fddb8004a40 comp rx=0 tx=0).stop 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 shutdown_connections 2024-04-24T06:42:04.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdd9c06c8f0 0x7fdd9c06edb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc00d05d0 0x7fddc00ae6a0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fddc00724d0 0x7fddc009ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 --2- 172.21.15.59:0/279089643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc006fb50 0x7fddc0149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 >> 172.21.15.59:0/279089643 conn(0x7fddc006bdf0 msgr2=0x7fddc006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.209+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 shutdown_connections 2024-04-24T06:42:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.210+0000 7fddc5a00640 1 -- 172.21.15.59:0/279089643 wait complete. 2024-04-24T06:42:04.236 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:42:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.507+0000 7fd034e00640 1 Processor -- start 2024-04-24T06:42:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.507+0000 7fd034e00640 1 -- start start 2024-04-24T06:42:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd034e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd0300cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd034e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd03006cbb0 0x7fd03006d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd034e00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300cf070 con 0x7fd0300d4210 2024-04-24T06:42:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd034e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd0300cf2f0 con 0x7fd03006cbb0 2024-04-24T06:42:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd034e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300d1c60 con 0x7fd03015e8e0 2024-04-24T06:42:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd02fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd03006cbb0 0x7fd03006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd02fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd03006cbb0 0x7fd03006d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45420/0 (socket says 172.21.15.59:45420) 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd02fe00640 1 -- 172.21.15.59:0/256664464 learned_addr learned my addr 172.21.15.59:0/256664464 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd02fe00640 1 -- 172.21.15.59:0/256664464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 msgr2=0x7fd0300cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.509+0000 7fd02fe00640 1 --2- 172.21.15.59:0/256664464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd0300cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.510+0000 7fd02fe00640 1 -- 172.21.15.59:0/256664464 >> v1:172.21.15.66:6789/0 conn(0x7fd0300d4210 legacy=0x7fd0300d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.510+0000 7fd02fe00640 1 -- 172.21.15.59:0/256664464 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0300cf5b0 con 0x7fd03006cbb0 2024-04-24T06:42:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.511+0000 7fd02fe00640 1 --2- 172.21.15.59:0/256664464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd03006cbb0 0x7fd03006d040 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fd028009e80 tx=0x7fd02802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9152946bc3d48bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:04.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.511+0000 7fd02e000640 1 -- 172.21.15.59:0/256664464 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028036070 con 0x7fd03006cbb0 2024-04-24T06:42:04.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.511+0000 7fd02e000640 1 -- 172.21.15.59:0/256664464 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028031600 con 0x7fd03006cbb0 2024-04-24T06:42:04.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.511+0000 7fd02e000640 1 -- 172.21.15.59:0/256664464 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280319d0 con 0x7fd03006cbb0 2024-04-24T06:42:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 -- 172.21.15.59:0/256664464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd03006cbb0 msgr2=0x7fd03006d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 --2- 172.21.15.59:0/256664464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd03006cbb0 0x7fd03006d040 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fd028009e80 tx=0x7fd02802a660 comp rx=0 tx=0).stop 2024-04-24T06:42:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 -- 172.21.15.59:0/256664464 shutdown_connections 2024-04-24T06:42:04.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 --2- 172.21.15.59:0/256664464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd03006cbb0 0x7fd03006d040 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 --2- 172.21.15.59:0/256664464 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd0300cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 -- 172.21.15.59:0/256664464 >> 172.21.15.59:0/256664464 conn(0x7fd03008e060 msgr2=0x7fd03009d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:04.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 -- 172.21.15.59:0/256664464 shutdown_connections 2024-04-24T06:42:04.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.512+0000 7fd034e00640 1 -- 172.21.15.59:0/256664464 wait complete. 2024-04-24T06:42:04.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.514+0000 7fd034e00640 1 Processor -- start 2024-04-24T06:42:04.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.514+0000 7fd034e00640 1 -- start start 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 0x7fd03009e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd03009fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd03015ecc0 con 0x7fd0300d4210 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd03006d580 con 0x7fd03006cbb0 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd034e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300d02f0 con 0x7fd03015e8e0 2024-04-24T06:42:04.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd02ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd02ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45422/0 (socket says 172.21.15.59:45422) 2024-04-24T06:42:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd02ea00640 1 -- 172.21.15.59:0/464474102 learned_addr learned my addr 172.21.15.59:0/464474102 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.515+0000 7fd02fe00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd03009fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02f400640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 0x7fd03009e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:04.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 -- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 msgr2=0x7fd03009fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd03009fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 -- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 msgr2=0x7fd03009e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 0x7fd03009e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 -- 172.21.15.59:0/464474102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03014eea0 con 0x7fd0300d4210 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02f400640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 0x7fd03009e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd02ea00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c430 tx=0x7fd01c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c017070 con 0x7fd0300d4210 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.516+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01c003640 con 0x7fd0300d4210 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.517+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c003a30 con 0x7fd0300d4210 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.517+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03014f180 con 0x7fd0300d4210 2024-04-24T06:42:04.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.517+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03014f740 con 0x7fd0300d4210 2024-04-24T06:42:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.518+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03004fd90 con 0x7fd0300d4210 2024-04-24T06:42:04.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.519+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd01c003bd0 con 0x7fd0300d4210 2024-04-24T06:42:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.520+0000 7fd027400640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.520+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6025+0+0 (secure 0 0 0) 0x7fd01c08aff0 con 0x7fd0300d4210 2024-04-24T06:42:04.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.521+0000 7fd02f400640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:04.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.521+0000 7fd02f400640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 0x7fd01006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd020009890 tx=0x7fd020008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:04.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.525+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd01c059240 con 0x7fd0300d4210 2024-04-24T06:42:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:04.729+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0300d3b00 con 0x7fd0300d4210 2024-04-24T06:42:05.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.221+0000 7fd027400640 1 -- 172.21.15.59:0/464474102 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) ==== 181+0+0 (secure 0 0 0) 0x7fd01c058b40 con 0x7fd0300d4210 2024-04-24T06:42:05.222 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:42:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.224+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 msgr2=0x7fd01006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.224+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 0x7fd01006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd020009890 tx=0x7fd020008040 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.224+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 msgr2=0x7fd0300a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.224+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c430 tx=0x7fd01c00c8f0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 shutdown_connections 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd01006c8f0 0x7fd01006edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd03015e8e0 0x7fd03009fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0300d4210 0x7fd0300a1c70 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 --2- 172.21.15.59:0/464474102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd03006cbb0 0x7fd03009e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 >> 172.21.15.59:0/464474102 conn(0x7fd03008e060 msgr2=0x7fd03006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 shutdown_connections 2024-04-24T06:42:05.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.225+0000 7fd034e00640 1 -- 172.21.15.59:0/464474102 wait complete. 2024-04-24T06:42:05.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:42:05.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.465+0000 7fe347200640 1 Processor -- start 2024-04-24T06:42:05.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.465+0000 7fe347200640 1 -- start start 2024-04-24T06:42:05.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.466+0000 7fe347200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe340161bc0 0x7fe340072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe347200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34006f7c0 con 0x7fe3400cd110 2024-04-24T06:42:05.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe347200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34006faa0 con 0x7fe34006f050 2024-04-24T06:42:05.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe347200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe34006fd80 con 0x7fe340161bc0 2024-04-24T06:42:05.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe346800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe3400cd110 0x7fe3400ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45734/0 (socket says 172.21.15.59:45734) 2024-04-24T06:42:05.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe346800640 1 -- 172.21.15.59:0/2937583194 learned_addr learned my addr 172.21.15.59:0/2937583194 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.467+0000 7fe345e00640 1 --2- 172.21.15.59:0/2937583194 >> v2:172.21.15.66:3300/0 conn(0x7fe340161bc0 0x7fe340072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435488532 0 0) 0x7fe34006f7c0 con 0x7fe3400cd110 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe328003520 con 0x7fe3400cd110 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2203736089 0 0) 0x7fe328003520 con 0x7fe3400cd110 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 >> v2:172.21.15.66:3300/0 conn(0x7fe340161bc0 msgr2=0x7fe340072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 --2- 172.21.15.59:0/2937583194 >> v2:172.21.15.66:3300/0 conn(0x7fe340161bc0 0x7fe340072fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 >> v1:172.21.15.66:6789/0 conn(0x7fe34006f050 legacy=0x7fe3400cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400ce320 con 0x7fe3400cd110 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fe33c0019f0 con 0x7fe3400cd110 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 >> v1:172.21.15.59:6789/0 conn(0x7fe3400cd110 legacy=0x7fe3400ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe34006fd80 con 0x7fe328004800 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe32800c060 con 0x7fe3280051c0 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe32800c260 con 0x7fe3280039b0 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.468+0000 7fe345400640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345400640 1 -- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3280039b0 msgr2=0x7fe328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345e00640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe346800640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345400640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345400640 1 -- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3280051c0 msgr2=0x7fe32800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:05.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345400640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe345400640 1 -- 172.21.15.59:0/2937583194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe328003520 con 0x7fe328004800 2024-04-24T06:42:05.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.469+0000 7fe346800640 1 -- 172.21.15.59:0/2937583194 reap_dead start 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.470+0000 7fe345400640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 0x7fe328004c80 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe330001750 tx=0x7fe328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8d594013ad2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.470+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330001370 con 0x7fe328004800 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.470+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3300052d0 con 0x7fe328004800 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.470+0000 7fe344a00640 1 -- 172.21.15.59:0/2937583194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33002abc0 con 0x7fe328004800 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 -- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 msgr2=0x7fe328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 0x7fe328004c80 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fe330001750 tx=0x7fe328005830 comp rx=0 tx=0).stop 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 -- 172.21.15.59:0/2937583194 shutdown_connections 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 --2- 172.21.15.59:0/2937583194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 -- 172.21.15.59:0/2937583194 >> 172.21.15.59:0/2937583194 conn(0x7fe3400b1f90 msgr2=0x7fe3400b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:05.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 -- 172.21.15.59:0/2937583194 shutdown_connections 2024-04-24T06:42:05.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.471+0000 7fe347200640 1 -- 172.21.15.59:0/2937583194 wait complete. 2024-04-24T06:42:05.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.473+0000 7fe347200640 1 Processor -- start 2024-04-24T06:42:05.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.473+0000 7fe347200640 1 -- start start 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe328004800 0x7fe3400ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 0x7fe34015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe340072fd0 con 0x7fe3401f8c20 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe340161fa0 con 0x7fe34014cc90 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe347200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe340073150 con 0x7fe328004800 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.474+0000 7fe345400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45436/0 (socket says 172.21.15.59:45436) 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 -- 172.21.15.59:0/1572478209 learned_addr learned my addr 172.21.15.59:0/1572478209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:05.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe346800640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 0x7fe34015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345e00640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe328004800 0x7fe3400ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 -- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe328004800 msgr2=0x7fe3400ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe328004800 0x7fe3400ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 -- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 msgr2=0x7fe34015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 0x7fe34015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 -- 172.21.15.59:0/1572478209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400cb000 con 0x7fe3401f8c20 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe346800640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 0x7fe34015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:05.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.475+0000 7fe345400640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fe33002a570 tx=0x7fe330001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.476+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330037040 con 0x7fe3401f8c20 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.476+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330040070 con 0x7fe3401f8c20 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.476+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330007a30 con 0x7fe3401f8c20 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.476+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3400cb2e0 con 0x7fe3401f8c20 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.476+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3400cb8a0 con 0x7fe3401f8c20 2024-04-24T06:42:05.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.477+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34004fd90 con 0x7fe3401f8c20 2024-04-24T06:42:05.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.478+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3300052d0 con 0x7fe3401f8c20 2024-04-24T06:42:05.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.479+0000 7fe33a000640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 0x7fe32406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:05.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.479+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5587+0+0 (secure 0 0 0) 0x7fe3300ab820 con 0x7fe3401f8c20 2024-04-24T06:42:05.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.479+0000 7fe345e00640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 0x7fe32406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:05.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.480+0000 7fe345e00640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 0x7fe32406ede0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe334009990 tx=0x7fe334008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:05.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.484+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe330079b90 con 0x7fe3401f8c20 2024-04-24T06:42:05.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:05.688+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe3400cefe0 con 0x7fe3401f8c20 2024-04-24T06:42:06.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.232+0000 7fe33a000640 1 -- 172.21.15.59:0/1572478209 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) ==== 169+0+0 (secure 0 0 0) 0x7fe330079490 con 0x7fe3401f8c20 2024-04-24T06:42:06.233 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-24T06:42:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.236+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 msgr2=0x7fe32406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.236+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 0x7fe32406ede0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe334009990 tx=0x7fe334008040 comp rx=0 tx=0).stop 2024-04-24T06:42:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.236+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 msgr2=0x7fe34014b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.236+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fe33002a570 tx=0x7fe330001120 comp rx=0 tx=0).stop 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 shutdown_connections 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe32406c920 0x7fe32406ede0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe34014cc90 0x7fe34015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3401f8c20 0x7fe34014b860 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 --2- 172.21.15.59:0/1572478209 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe328004800 0x7fe3400ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 >> 172.21.15.59:0/1572478209 conn(0x7fe3400b1f90 msgr2=0x7fe3400cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 shutdown_connections 2024-04-24T06:42:06.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.237+0000 7fe347200640 1 -- 172.21.15.59:0/1572478209 wait complete. 2024-04-24T06:42:06.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-24T06:42:06.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.470+0000 7f4aa8c00640 1 Processor -- start 2024-04-24T06:42:06.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.470+0000 7f4aa8c00640 1 -- start start 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aa406f5f0 0x7f4aa406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa8c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa41668b0 con 0x7f4aa4162a30 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa8c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa4166b90 con 0x7f4aa40d0ce0 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa8c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa40b5500 con 0x7f4aa406f5f0 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aa406f5f0 0x7f4aa406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.472+0000 7f4aa3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4aa4162a30 0x7f4aa40d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45738/0 (socket says 172.21.15.59:45738) 2024-04-24T06:42:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.473+0000 7f4aa3400640 1 -- 172.21.15.59:0/981331146 learned_addr learned my addr 172.21.15.59:0/981331146 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.473+0000 7f4aa3e00640 1 -- 172.21.15.59:0/981331146 >> v1:172.21.15.66:6789/0 conn(0x7f4aa40d0ce0 legacy=0x7f4aa406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.473+0000 7f4aa3e00640 1 -- 172.21.15.59:0/981331146 >> v1:172.21.15.59:6789/0 conn(0x7f4aa4162a30 legacy=0x7f4aa40d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.473+0000 7f4aa3e00640 1 -- 172.21.15.59:0/981331146 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa4166e70 con 0x7f4aa406f5f0 2024-04-24T06:42:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.473+0000 7f4aa2a00640 1 -- 172.21.15.59:0/981331146 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875780033 0 0) 0x7f4aa41668b0 con 0x7f4aa4162a30 2024-04-24T06:42:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.474+0000 7f4aa3e00640 1 --2- 172.21.15.59:0/981331146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa406ea40 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009d90 tx=0x7f4a9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e816f4c0d5fb927 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.474+0000 7f4aa2a00640 1 -- 172.21.15.59:0/981331146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007740 con 0x7f4aa406f5f0 2024-04-24T06:42:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa2a00640 1 -- 172.21.15.59:0/981331146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c0078e0 con 0x7f4aa406f5f0 2024-04-24T06:42:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa2a00640 1 -- 172.21.15.59:0/981331146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007cb0 con 0x7f4aa406f5f0 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 -- 172.21.15.59:0/981331146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 msgr2=0x7f4aa406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/981331146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa406ea40 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009d90 tx=0x7f4a9c02a6a0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 -- 172.21.15.59:0/981331146 shutdown_connections 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/981331146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa406ea40 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 -- 172.21.15.59:0/981331146 >> 172.21.15.59:0/981331146 conn(0x7f4aa406bdf0 msgr2=0x7f4aa406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:06.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.475+0000 7f4aa8c00640 1 -- 172.21.15.59:0/981331146 shutdown_connections 2024-04-24T06:42:06.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.476+0000 7f4aa8c00640 1 -- 172.21.15.59:0/981331146 wait complete. 2024-04-24T06:42:06.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.477+0000 7f4aa8c00640 1 Processor -- start 2024-04-24T06:42:06.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.477+0000 7f4aa8c00640 1 -- start start 2024-04-24T06:42:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aa40d0ce0 0x7f4aa415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aa4162a30 0x7f4aa40b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa40ad170 con 0x7f4aa406f5f0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4166660 con 0x7f4aa40d0ce0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.478+0000 7f4aa8c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa4162e10 con 0x7f4aa4162a30 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45466/0 (socket says 172.21.15.59:45466) 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 -- 172.21.15.59:0/3101685094 learned_addr learned my addr 172.21.15.59:0/3101685094 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4a9be00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aa40d0ce0 0x7f4aa415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3e00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aa4162a30 0x7f4aa40b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 -- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aa4162a30 msgr2=0x7f4aa40b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aa4162a30 0x7f4aa40b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 -- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aa40d0ce0 msgr2=0x7f4aa415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aa40d0ce0 0x7f4aa415c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 -- 172.21.15.59:0/3101685094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa40b0c80 con 0x7f4aa406f5f0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.479+0000 7f4aa3400640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c460 tx=0x7f4a9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.480+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94018070 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.480+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94007500 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.480+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004a20 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.480+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa40b0f20 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.480+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa40b14e0 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.481+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa404fd90 con 0x7f4aa406f5f0 2024-04-24T06:42:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.482+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a9401b070 con 0x7f4aa406f5f0 2024-04-24T06:42:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.483+0000 7f4a9b400640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.483+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f4a9408abc0 con 0x7f4aa406f5f0 2024-04-24T06:42:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.483+0000 7f4a9be00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.484+0000 7f4a9be00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 0x7f4a8006edb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4aa409e290 tx=0x7f4a8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.487+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a94059230 con 0x7f4aa406f5f0 2024-04-24T06:42:06.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.694+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4aa40b4820 con 0x7f4aa406f5f0 2024-04-24T06:42:06.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.700+0000 7f4a9b400640 1 -- 172.21.15.59:0/3101685094 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f4a94058b30 con 0x7f4aa406f5f0 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 msgr2=0x7f4a8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 0x7f4a8006edb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4aa409e290 tx=0x7f4a8c009040 comp rx=0 tx=0).stop 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 msgr2=0x7f4aa40b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c460 tx=0x7f4a9400c920 comp rx=0 tx=0).stop 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 shutdown_connections 2024-04-24T06:42:06.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f4aa4162a30 0x7f4aa40b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4aa40d0ce0 0x7f4aa415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 --2- 172.21.15.59:0/3101685094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa406f5f0 0x7f4aa40b1d70 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 >> 172.21.15.59:0/3101685094 conn(0x7f4aa406bdf0 msgr2=0x7f4aa415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 shutdown_connections 2024-04-24T06:42:06.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.704+0000 7f4aa8c00640 1 -- 172.21.15.59:0/3101685094 wait complete. 2024-04-24T06:42:06.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-24T06:42:06.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.939+0000 7fa1e5a00640 1 Processor -- start 2024-04-24T06:42:06.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.940+0000 7fa1e5a00640 1 -- start start 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.941+0000 7fa1e5a00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa1e00b4130 0x7fa1e006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.941+0000 7fa1e5a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e006ed40 con 0x7fa1e0162df0 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.941+0000 7fa1e5a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e00b3220 con 0x7fa1e00cfb90 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.941+0000 7fa1e5a00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e006eaf0 con 0x7fa1e00b4130 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.942+0000 7fa1dfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1e0162df0 0x7fa1e00b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45752/0 (socket says 172.21.15.59:45752) 2024-04-24T06:42:06.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.942+0000 7fa1dfe00640 1 -- 172.21.15.59:0/2844305644 learned_addr learned my addr 172.21.15.59:0/2844305644 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:06.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.942+0000 7fa1d7400640 1 --2- 172.21.15.59:0/2844305644 >> v2:172.21.15.66:3300/0 conn(0x7fa1e00b4130 0x7fa1e006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.942+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989384053 0 0) 0x7fa1e006ed40 con 0x7fa1e0162df0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.942+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1c4003520 con 0x7fa1e0162df0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2121214218 0 0) 0x7fa1c4003520 con 0x7fa1e0162df0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 >> v2:172.21.15.66:3300/0 conn(0x7fa1e00b4130 msgr2=0x7fa1e006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 --2- 172.21.15.59:0/2844305644 >> v2:172.21.15.66:3300/0 conn(0x7fa1e00b4130 0x7fa1e006f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 >> v1:172.21.15.66:6789/0 conn(0x7fa1e00cfb90 legacy=0x7fa1e00c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00af4c0 con 0x7fa1e0162df0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa1d0001670 con 0x7fa1e0162df0 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1d7400640 1 --2- 172.21.15.59:0/2844305644 >> v2:172.21.15.66:3300/0 conn(0x7fa1e00b4130 0x7fa1e006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 >> v1:172.21.15.59:6789/0 conn(0x7fa1e0162df0 legacy=0x7fa1e00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 0x7fa1c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4004800 0x7fa1c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c40051c0 0x7fa1c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e006eaf0 con 0x7fa1c4003a00 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c400c0c0 con 0x7fa1c4004800 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c400c2c0 con 0x7fa1c40051c0 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.943+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 0x7fa1c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1d7400640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4004800 0x7fa1c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1e5000640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c40051c0 0x7fa1c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1dfe00640 1 -- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c40051c0 msgr2=0x7fa1c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1c40051c0 0x7fa1c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1dfe00640 1 -- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4004800 msgr2=0x7fa1c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4004800 0x7fa1c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1dfe00640 1 -- 172.21.15.59:0/2844305644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c4003520 con 0x7fa1c4003a00 2024-04-24T06:42:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.944+0000 7fa1e5000640 1 -- 172.21.15.59:0/2844305644 reap_dead start 2024-04-24T06:42:06.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.945+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 0x7fa1c4003e20 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c190 tx=0x7fa1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857e724c4b0c4271 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.945+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0007c60 con 0x7fa1c4003a00 2024-04-24T06:42:06.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.945+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d0005ad0 con 0x7fa1c4003a00 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.945+0000 7fa1df400640 1 -- 172.21.15.59:0/2844305644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0007070 con 0x7fa1c4003a00 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 -- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 msgr2=0x7fa1c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 0x7fa1c4003e20 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c190 tx=0x7fa1c4005830 comp rx=0 tx=0).stop 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 -- 172.21.15.59:0/2844305644 shutdown_connections 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/2844305644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1c4003a00 0x7fa1c4003e20 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 -- 172.21.15.59:0/2844305644 >> 172.21.15.59:0/2844305644 conn(0x7fa1e00b21a0 msgr2=0x7fa1e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 -- 172.21.15.59:0/2844305644 shutdown_connections 2024-04-24T06:42:06.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.946+0000 7fa1e5a00640 1 -- 172.21.15.59:0/2844305644 wait complete. 2024-04-24T06:42:06.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.948+0000 7fa1e5a00640 1 Processor -- start 2024-04-24T06:42:06.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.948+0000 7fa1e5a00640 1 -- start start 2024-04-24T06:42:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4003a00 0x7fa1e0158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1e00c7280 0x7fa1e00c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e006fa10 con 0x7fa1e00ae180 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e006f890 con 0x7fa1c4003a00 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.949+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00b3590 con 0x7fa1e00c7280 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45490/0 (socket says 172.21.15.59:45490) 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 -- 172.21.15.59:0/1686730449 learned_addr learned my addr 172.21.15.59:0/1686730449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1d7400640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1e00c7280 0x7fa1e00c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4003a00 0x7fa1e0158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 -- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1e00c7280 msgr2=0x7fa1e00c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1e00c7280 0x7fa1e00c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 -- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4003a00 msgr2=0x7fa1e0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4003a00 0x7fa1e0158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.950+0000 7fa1e5000640 1 -- 172.21.15.59:0/1686730449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e0163ad0 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1e5000640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa1e006d960 tx=0x7fa1d800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003970 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8018070 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003d70 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e00c7db0 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.951+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e0163d70 con 0x7fa1e00ae180 2024-04-24T06:42:06.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.952+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e004fd90 con 0x7fa1e00ae180 2024-04-24T06:42:06.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.954+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1d80050d0 con 0x7fa1e00ae180 2024-04-24T06:42:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.954+0000 7fa1dcc00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 0x7fa1bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.955+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fa1d808b600 con 0x7fa1e00ae180 2024-04-24T06:42:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.955+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 0x7fa1bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.956+0000 7fa1dfe00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 0x7fa1bc06edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c160 tx=0x7fa1d0034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:06.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:06.960+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1d8059be0 con 0x7fa1e00ae180 2024-04-24T06:42:07.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:07.255+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fa1e00b2970 con 0x7fa1e00ae180 2024-04-24T06:42:08.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.245+0000 7fa1dcc00640 1 -- 172.21.15.59:0/1686730449 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v635) ==== 144+0+0 (secure 0 0 0) 0x7fa1d80594e0 con 0x7fa1e00ae180 2024-04-24T06:42:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.249+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 msgr2=0x7fa1bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.249+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 0x7fa1bc06edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c160 tx=0x7fa1d0034040 comp rx=0 tx=0).stop 2024-04-24T06:42:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.249+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 msgr2=0x7fa1e00ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.249+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa1e006d960 tx=0x7fa1d800eb90 comp rx=0 tx=0).stop 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 shutdown_connections 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa1bc06c8f0 0x7fa1bc06edb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1e00ae180 0x7fa1e00ae600 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa1e00c7280 0x7fa1e00c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 --2- 172.21.15.59:0/1686730449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1c4003a00 0x7fa1e0158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 >> 172.21.15.59:0/1686730449 conn(0x7fa1e00b21a0 msgr2=0x7fa1e00cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 shutdown_connections 2024-04-24T06:42:08.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:08.250+0000 7fa1e5a00640 1 -- 172.21.15.59:0/1686730449 wait complete. 2024-04-24T06:42:08.276 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-24T06:42:08.276 INFO:tasks.cephfs_test_runner: 2024-04-24T06:42:08.276 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-24T06:42:08.276 INFO:tasks.cephfs_test_runner:Ran 7 tests in 764.107s 2024-04-24T06:42:08.277 INFO:tasks.cephfs_test_runner: 2024-04-24T06:42:08.277 INFO:tasks.cephfs_test_runner:OK 2024-04-24T06:42:08.277 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-24T06:42:08.287 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T06:42:08.295 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T06:42:08.296 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:42:08.296 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:42:08.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:42:08.355 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-24T06:42:08.355 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:42:08.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-24T06:42:08.410 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:42:08.410 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:42:08.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:42:08.473 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-24T06:42:08.473 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:42:08.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-24T06:42:08.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.533 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-24T06:42:08.594 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:42:08.594 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T06:42:08.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:42:08.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T06:42:08.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.769 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-24T06:42:08.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.922 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-04-24T06:42:08.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:08.944 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T06:42:08.983 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-24T06:42:08.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:08.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T06:42:09.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:09.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T06:42:09.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:42:09.174 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-04-24T06:42:09.300 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T06:42:09.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T06:42:09.310 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:42:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.531+0000 7fc1b3200640 1 Processor -- start 2024-04-24T06:42:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.532+0000 7fc1b3200640 1 -- start start 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b40cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1b4160920 0x7fc1b40cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc1b40cd8b0 0x7fc1b40cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b40d07f0 con 0x7fc1b4160920 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b40d0ad0 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1b3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b40d0db0 con 0x7fc1b4160540 2024-04-24T06:42:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1b4160920 0x7fc1b40cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1b4160920 0x7fc1b40cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47266/0 (socket says 172.21.15.59:47266) 2024-04-24T06:42:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.533+0000 7fc1abe00640 1 -- 172.21.15.59:0/3385579160 learned_addr learned my addr 172.21.15.59:0/3385579160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1b1e00640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b40cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1abe00640 1 -- 172.21.15.59:0/3385579160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 msgr2=0x7fc1b40cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1abe00640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b40cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1b2800640 1 --2- 172.21.15.59:0/3385579160 >> v2:172.21.15.66:3300/0 conn(0x7fc1b40cd8b0 0x7fc1b40cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1abe00640 1 -- 172.21.15.59:0/3385579160 >> v2:172.21.15.66:3300/0 conn(0x7fc1b40cd8b0 msgr2=0x7fc1b40cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1abe00640 1 --2- 172.21.15.59:0/3385579160 >> v2:172.21.15.66:3300/0 conn(0x7fc1b40cd8b0 0x7fc1b40cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1abe00640 1 -- 172.21.15.59:0/3385579160 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b40d1090 con 0x7fc1b4160920 2024-04-24T06:42:09.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.534+0000 7fc1b1e00640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b40cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:09.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.535+0000 7fc1abe00640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b4160920 0x7fc1b40cd370 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4073da0 tx=0x7fc19c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1f98cd6053b474 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:09.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.535+0000 7fc1b1400640 1 -- 172.21.15.59:0/3385579160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c037070 con 0x7fc1b4160920 2024-04-24T06:42:09.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.535+0000 7fc1b1400640 1 -- 172.21.15.59:0/3385579160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c032440 con 0x7fc1b4160920 2024-04-24T06:42:09.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.535+0000 7fc1b1400640 1 -- 172.21.15.59:0/3385579160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c032810 con 0x7fc1b4160920 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 -- 172.21.15.59:0/3385579160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b4160920 msgr2=0x7fc1b40cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b4160920 0x7fc1b40cd370 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4073da0 tx=0x7fc19c02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 -- 172.21.15.59:0/3385579160 shutdown_connections 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 --2- 172.21.15.59:0/3385579160 >> v2:172.21.15.66:3300/0 conn(0x7fc1b40cd8b0 0x7fc1b40cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b4160920 0x7fc1b40cd370 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 --2- 172.21.15.59:0/3385579160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b40cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 -- 172.21.15.59:0/3385579160 >> 172.21.15.59:0/3385579160 conn(0x7fc1b40b3ee0 msgr2=0x7fc1b40b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 -- 172.21.15.59:0/3385579160 shutdown_connections 2024-04-24T06:42:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.536+0000 7fc1b3200640 1 -- 172.21.15.59:0/3385579160 wait complete. 2024-04-24T06:42:09.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.538+0000 7fc1b3200640 1 Processor -- start 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.538+0000 7fc1b3200640 1 -- start start 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 0x7fc1b4071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b40d0500 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b40d1370 con 0x7fc1b4160920 2024-04-24T06:42:09.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b3200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b40ce200 con 0x7fc1b4160540 2024-04-24T06:42:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47280/0 (socket says 172.21.15.59:47280) 2024-04-24T06:42:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.539+0000 7fc1b1e00640 1 -- 172.21.15.59:0/4120816677 learned_addr learned my addr 172.21.15.59:0/4120816677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b2800640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 0x7fc1b4071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1abe00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 -- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 msgr2=0x7fc1b4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b4070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 -- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 msgr2=0x7fc1b4071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 0x7fc1b4071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 -- 172.21.15.59:0/4120816677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b414e620 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b2800640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 0x7fc1b4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1abe00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1b1e00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400c7f0 tx=0x7fc1a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.540+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4015070 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.541+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a4007820 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.541+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b414e900 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.541+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4007bd0 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.541+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b414eec0 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.542+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b404fd90 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.543+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1a4018070 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.544+0000 7fc1aa000640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 0x7fc19006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.544+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fc1a408ba80 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.544+0000 7fc1abe00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 0x7fc19006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.545+0000 7fc1abe00640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 0x7fc19006edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034000 tx=0x7fc19c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:09.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.548+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1a408de20 con 0x7fc1b40cd8b0 2024-04-24T06:42:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.752+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1b40b4df0 con 0x7fc19006c8f0 2024-04-24T06:42:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.754+0000 7fc1aa000640 1 -- 172.21.15.59:0/4120816677 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7fc1b40b4df0 con 0x7fc19006c8f0 2024-04-24T06:42:09.755 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:09.756 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:42:09.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 msgr2=0x7fc19006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 0x7fc19006edb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034000 tx=0x7fc19c02e000 comp rx=0 tx=0).stop 2024-04-24T06:42:09.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 msgr2=0x7fc1b40c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400c7f0 tx=0x7fc1a400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 shutdown_connections 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc19006c8f0 0x7fc19006edb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1b4160920 0x7fc1b4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc1b4160540 0x7fc1b4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 --2- 172.21.15.59:0/4120816677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1b40cd8b0 0x7fc1b40c88f0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 >> 172.21.15.59:0/4120816677 conn(0x7fc1b40b3ee0 msgr2=0x7fc1b409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 shutdown_connections 2024-04-24T06:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.757+0000 7fc1b3200640 1 -- 172.21.15.59:0/4120816677 wait complete. 2024-04-24T06:42:09.778 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-04-24T06:42:09.474367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262028,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712692,"statfs":{"total":773094113280,"available":772825796608,"internally_reserved":0,"allocated":12886016,"data_stored":6273944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-263,"num_write":-181,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.269001"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-24T06:42:07.377917+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:42:07.377917+0000","last_peered":"2024-04-24T06:42:07.377917+0000","last_clean":"2024-04-24T06:42:07.377917+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:42:07.377917+0000","last_undegraded":"2024-04-24T06:42:07.377917+0000","last_fullsized":"2024-04-24T06:42:07.377917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:42:09.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:42:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.986+0000 7f9e0ea00640 1 Processor -- start 2024-04-24T06:42:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.987+0000 7f9e0ea00640 1 -- start start 2024-04-24T06:42:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.988+0000 7f9e0ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e0815e7c0 0x7f9e080d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.988+0000 7f9e0ea00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9e080d2630 0x7f9e080b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.988+0000 7f9e0ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e080cf030 con 0x7f9e080b6530 2024-04-24T06:42:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.988+0000 7f9e0ea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9e080b7000 con 0x7f9e0815e7c0 2024-04-24T06:42:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.988+0000 7f9e0ea00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9e080cf620 con 0x7f9e080d2630 2024-04-24T06:42:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e0815e7c0 0x7f9e080d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9e080b6530 0x7f9e080d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39262/0 (socket says 172.21.15.59:39262) 2024-04-24T06:42:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0e000640 1 -- 172.21.15.59:0/2054459717 learned_addr learned my addr 172.21.15.59:0/2054459717 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0cc00640 1 --2- 172.21.15.59:0/2054459717 >> v2:172.21.15.66:3300/0 conn(0x7f9e080d2630 0x7f9e080b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0d600640 1 -- 172.21.15.59:0/2054459717 >> v2:172.21.15.66:3300/0 conn(0x7f9e080d2630 msgr2=0x7f9e080b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0d600640 1 --2- 172.21.15.59:0/2054459717 >> v2:172.21.15.66:3300/0 conn(0x7f9e080d2630 0x7f9e080b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0d600640 1 -- 172.21.15.59:0/2054459717 >> v1:172.21.15.59:6789/0 conn(0x7f9e080b6530 legacy=0x7f9e080d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.989+0000 7f9e0d600640 1 -- 172.21.15.59:0/2054459717 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e080b6a40 con 0x7f9e0815e7c0 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.990+0000 7f9e0d600640 1 --2- 172.21.15.59:0/2054459717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e0815e7c0 0x7f9e080d20f0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f9e080d55c0 tx=0x7f9dfc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fcc34308fc639b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e07e00640 1 -- 172.21.15.59:0/2054459717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc037070 con 0x7f9e0815e7c0 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e07e00640 1 -- 172.21.15.59:0/2054459717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc032440 con 0x7f9e0815e7c0 2024-04-24T06:42:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e07e00640 1 -- 172.21.15.59:0/2054459717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc032810 con 0x7f9e0815e7c0 2024-04-24T06:42:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 -- 172.21.15.59:0/2054459717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e0815e7c0 msgr2=0x7f9e080d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/2054459717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e0815e7c0 0x7f9e080d20f0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f9e080d55c0 tx=0x7f9dfc02a1a0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 -- 172.21.15.59:0/2054459717 shutdown_connections 2024-04-24T06:42:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/2054459717 >> v2:172.21.15.66:3300/0 conn(0x7f9e080d2630 0x7f9e080b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/2054459717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e0815e7c0 0x7f9e080d20f0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.991+0000 7f9e0ea00640 1 -- 172.21.15.59:0/2054459717 >> 172.21.15.59:0/2054459717 conn(0x7f9e0809d920 msgr2=0x7f9e0808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.992+0000 7f9e0ea00640 1 -- 172.21.15.59:0/2054459717 shutdown_connections 2024-04-24T06:42:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.992+0000 7f9e0ea00640 1 -- 172.21.15.59:0/2054459717 wait complete. 2024-04-24T06:42:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.993+0000 7f9e0ea00640 1 Processor -- start 2024-04-24T06:42:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.993+0000 7f9e0ea00640 1 -- start start 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e080d2630 0x7f9e08148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e0815e7c0 0x7f9e080cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080d1a80 con 0x7f9e080b6530 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e080cf2b0 con 0x7f9e0815e7c0 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.994+0000 7f9e0ea00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9e080d1c00 con 0x7f9e080d2630 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47306/0 (socket says 172.21.15.59:47306) 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 -- 172.21.15.59:0/1765290133 learned_addr learned my addr 172.21.15.59:0/1765290133 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 -- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e080d2630 msgr2=0x7f9e08148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0cc00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e080d2630 0x7f9e08148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:09.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e080d2630 0x7f9e08148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 -- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e0815e7c0 msgr2=0x7f9e080cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:09.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e0815e7c0 0x7f9e080cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:09.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 -- 172.21.15.59:0/1765290133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0814bcf0 con 0x7f9e080b6530 2024-04-24T06:42:09.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.995+0000 7f9e0d600640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.996+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc037040 con 0x7f9e080b6530 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.996+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc042070 con 0x7f9e080b6530 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.996+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc030bd0 con 0x7f9e080b6530 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.996+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0814bf70 con 0x7f9e080b6530 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.996+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0814c530 con 0x7f9e080b6530 2024-04-24T06:42:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.997+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08069480 con 0x7f9e080b6530 2024-04-24T06:42:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.998+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dfc047460 con 0x7f9e080b6530 2024-04-24T06:42:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.999+0000 7f9e05600640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 0x7f9de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.999+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f9dfc0ac540 con 0x7f9e080b6530 2024-04-24T06:42:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:09.999+0000 7f9e0cc00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 0x7f9de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.000+0000 7f9e0cc00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 0x7f9de806edb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9e081f7a30 tx=0x7f9df4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.004+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e08069480 con 0x7f9e080b6530 2024-04-24T06:42:10.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.207+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e0804fd90 con 0x7f9de806c8f0 2024-04-24T06:42:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.209+0000 7f9e05600640 1 -- 172.21.15.59:0/1765290133 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f9e0804fd90 con 0x7f9de806c8f0 2024-04-24T06:42:10.210 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:10.210 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:42:10.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.211+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 msgr2=0x7f9de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.211+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 0x7f9de806edb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9e081f7a30 tx=0x7f9df4008040 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.211+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 msgr2=0x7f9e0814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.211+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 shutdown_connections 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9de806c8f0 0x7f9de806edb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e0815e7c0 0x7f9e080cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9e080d2630 0x7f9e08148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 --2- 172.21.15.59:0/1765290133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080b6530 0x7f9e0814b7b0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 >> 172.21.15.59:0/1765290133 conn(0x7f9e0809d920 msgr2=0x7f9e080b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 shutdown_connections 2024-04-24T06:42:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.212+0000 7f9e0ea00640 1 -- 172.21.15.59:0/1765290133 wait complete. 2024-04-24T06:42:10.232 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-04-24T06:42:09.474367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262028,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712692,"statfs":{"total":773094113280,"available":772825796608,"internally_reserved":0,"allocated":12886016,"data_stored":6273944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-263,"num_write":-181,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.269001"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-24T06:42:07.377917+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:42:07.377917+0000","last_peered":"2024-04-24T06:42:07.377917+0000","last_clean":"2024-04-24T06:42:07.377917+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:42:07.377917+0000","last_undegraded":"2024-04-24T06:42:07.377917+0000","last_fullsized":"2024-04-24T06:42:07.377917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:42:10.233 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T06:42:10.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:42:10.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.434+0000 7f7eaec00640 1 Processor -- start 2024-04-24T06:42:10.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.434+0000 7f7eaec00640 1 -- start start 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.435+0000 7f7eaec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.435+0000 7f7eaec00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7ea80b2e20 0x7f7ea80af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.435+0000 7f7eaec00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea80b35f0 con 0x7f7ea80d09e0 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.435+0000 7f7eaec00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea80cd2d0 con 0x7f7ea80b2e20 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.435+0000 7f7eaec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea80b3890 con 0x7f7ea815c9d0 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.436+0000 7f7ead800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.436+0000 7f7eace00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f7ea80b2e20 0x7f7ea80af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.436+0000 7f7ead800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3301/0 says I am v2:172.21.15.59:46346/0 (socket says 172.21.15.59:46346) 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.436+0000 7f7ead800640 1 -- 172.21.15.59:0/1586242033 learned_addr learned my addr 172.21.15.59:0/1586242033 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.437+0000 7f7ead800640 1 -- 172.21.15.59:0/1586242033 >> v2:172.21.15.66:3300/0 conn(0x7f7ea80b2e20 msgr2=0x7f7ea80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.437+0000 7f7ead800640 1 --2- 172.21.15.59:0/1586242033 >> v2:172.21.15.66:3300/0 conn(0x7f7ea80b2e20 0x7f7ea80af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.437+0000 7f7ead800640 1 -- 172.21.15.59:0/1586242033 >> v1:172.21.15.66:6789/0 conn(0x7f7ea80d09e0 legacy=0x7f7ea80d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.437+0000 7f7ead800640 1 -- 172.21.15.59:0/1586242033 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea80b3b70 con 0x7f7ea815c9d0 2024-04-24T06:42:10.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.438+0000 7f7ead800640 1 --2- 172.21.15.59:0/1586242033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009e80 tx=0x7f7e9802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd965c39fed8696 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.438+0000 7f7ea7e00640 1 -- 172.21.15.59:0/1586242033 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98036070 con 0x7f7ea815c9d0 2024-04-24T06:42:10.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.438+0000 7f7ea7e00640 1 -- 172.21.15.59:0/1586242033 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98031600 con 0x7f7ea815c9d0 2024-04-24T06:42:10.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.438+0000 7f7ea7e00640 1 -- 172.21.15.59:0/1586242033 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980319d0 con 0x7f7ea815c9d0 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 -- 172.21.15.59:0/1586242033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 msgr2=0x7f7ea809b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 --2- 172.21.15.59:0/1586242033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009e80 tx=0x7f7e9802a750 comp rx=0 tx=0).stop 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 -- 172.21.15.59:0/1586242033 shutdown_connections 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 --2- 172.21.15.59:0/1586242033 >> v2:172.21.15.66:3300/0 conn(0x7f7ea80b2e20 0x7f7ea80af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 --2- 172.21.15.59:0/1586242033 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea815c9d0 0x7f7ea809b410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 -- 172.21.15.59:0/1586242033 >> 172.21.15.59:0/1586242033 conn(0x7f7ea80b3ef0 msgr2=0x7f7ea80b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:10.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 -- 172.21.15.59:0/1586242033 shutdown_connections 2024-04-24T06:42:10.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.439+0000 7f7eaec00640 1 -- 172.21.15.59:0/1586242033 wait complete. 2024-04-24T06:42:10.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.441+0000 7f7eaec00640 1 Processor -- start 2024-04-24T06:42:10.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.441+0000 7f7eaec00640 1 -- start start 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea80b2e20 0x7f7ea81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ea815c9d0 0x7f7ea80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80b3280 con 0x7f7ea80d09e0 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80ccef0 con 0x7f7ea815c9d0 2024-04-24T06:42:10.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eaec00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea815cdb0 con 0x7f7ea80b2e20 2024-04-24T06:42:10.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47318/0 (socket says 172.21.15.59:47318) 2024-04-24T06:42:10.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.442+0000 7f7eace00640 1 -- 172.21.15.59:0/2870023685 learned_addr learned my addr 172.21.15.59:0/2870023685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eae200640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ea815c9d0 0x7f7ea80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7ead800640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea80b2e20 0x7f7ea81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 -- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea80b2e20 msgr2=0x7f7ea81548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea80b2e20 0x7f7ea81548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 -- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ea815c9d0 msgr2=0x7f7ea80b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ea815c9d0 0x7f7ea80b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 -- 172.21.15.59:0/2870023685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea8154e00 con 0x7f7ea80d09e0 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.443+0000 7f7eace00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9e0 tx=0x7f7e9400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.444+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.444+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400c970 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.444+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cd60 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.444+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea80cafa0 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.444+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea80cb560 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.445+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea804fd90 con 0x7f7ea80d09e0 2024-04-24T06:42:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.446+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e9401a070 con 0x7f7ea80d09e0 2024-04-24T06:42:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.447+0000 7f7ea5600640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.447+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f7e9408b910 con 0x7f7ea80d09e0 2024-04-24T06:42:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.447+0000 7f7ead800640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.448+0000 7f7ead800640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 0x7f7e8806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7e98033000 tx=0x7f7e98009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.451+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ea8154e00 con 0x7f7ea80d09e0 2024-04-24T06:42:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.660+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ea80d07c0 con 0x7f7e8806c8f0 2024-04-24T06:42:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.661+0000 7f7ea5600640 1 -- 172.21.15.59:0/2870023685 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f7ea80d07c0 con 0x7f7e8806c8f0 2024-04-24T06:42:10.662 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:10.663 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 msgr2=0x7f7e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 0x7f7e8806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7e98033000 tx=0x7f7e98009000 comp rx=0 tx=0).stop 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 msgr2=0x7f7ea80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9e0 tx=0x7f7e9400eea0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 shutdown_connections 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7e8806c8f0 0x7f7e8806edb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ea815c9d0 0x7f7ea80b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea80d09e0 0x7f7ea80cc040 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 --2- 172.21.15.59:0/2870023685 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ea80b2e20 0x7f7ea81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 >> 172.21.15.59:0/2870023685 conn(0x7f7ea80b3ef0 msgr2=0x7f7ea806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 shutdown_connections 2024-04-24T06:42:10.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.665+0000 7f7eaec00640 1 -- 172.21.15.59:0/2870023685 wait complete. 2024-04-24T06:42:10.685 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-04-24T06:42:09.474367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262028,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712692,"statfs":{"total":773094113280,"available":772825796608,"internally_reserved":0,"allocated":12886016,"data_stored":6273944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-263,"num_write":-181,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.269001"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-24T06:42:07.377917+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:42:07.377917+0000","last_peered":"2024-04-24T06:42:07.377917+0000","last_clean":"2024-04-24T06:42:07.377917+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:42:07.377917+0000","last_undegraded":"2024-04-24T06:42:07.377917+0000","last_fullsized":"2024-04-24T06:42:07.377917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:42:10.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:42:10.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.873+0000 7fe696800640 1 Processor -- start 2024-04-24T06:42:10.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.873+0000 7fe696800640 1 -- start start 2024-04-24T06:42:10.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.875+0000 7fe696800640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 0x7fe6900ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.875+0000 7fe696800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6900b2950 0x7fe6900cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.875+0000 7fe696800640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6900d0740 con 0x7fe6900cf400 2024-04-24T06:42:10.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.875+0000 7fe696800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900cf870 con 0x7fe6900b2950 2024-04-24T06:42:10.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.875+0000 7fe696800640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900b2da0 con 0x7fe69015eaa0 2024-04-24T06:42:10.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6900b2950 0x7fe6900cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6900b2950 0x7fe6900cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47322/0 (socket says 172.21.15.59:47322) 2024-04-24T06:42:10.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 -- 172.21.15.59:0/376338725 learned_addr learned my addr 172.21.15.59:0/376338725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:10.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68f400640 1 --2- 172.21.15.59:0/376338725 >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 0x7fe6900ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 -- 172.21.15.59:0/376338725 >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 msgr2=0x7fe6900ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 --2- 172.21.15.59:0/376338725 >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 0x7fe6900ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 -- 172.21.15.59:0/376338725 >> v1:172.21.15.66:6789/0 conn(0x7fe6900cf400 legacy=0x7fe6900b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68fe00640 1 -- 172.21.15.59:0/376338725 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6900b2fc0 con 0x7fe6900b2950 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.876+0000 7fe68f400640 1 --2- 172.21.15.59:0/376338725 >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 0x7fe6900ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:10.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.877+0000 7fe68fe00640 1 --2- 172.21.15.59:0/376338725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6900b2950 0x7fe6900cdcf0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe6900ccb90 tx=0x7fe68802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190fee95e5b214b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.877+0000 7fe68e000640 1 -- 172.21.15.59:0/376338725 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688037070 con 0x7fe6900b2950 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe68e000640 1 -- 172.21.15.59:0/376338725 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe688032440 con 0x7fe6900b2950 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe68e000640 1 -- 172.21.15.59:0/376338725 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688032810 con 0x7fe6900b2950 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 -- 172.21.15.59:0/376338725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6900b2950 msgr2=0x7fe6900cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 --2- 172.21.15.59:0/376338725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6900b2950 0x7fe6900cdcf0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe6900ccb90 tx=0x7fe68802a2c0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 -- 172.21.15.59:0/376338725 shutdown_connections 2024-04-24T06:42:10.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 --2- 172.21.15.59:0/376338725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6900b2950 0x7fe6900cdcf0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 --2- 172.21.15.59:0/376338725 >> v2:172.21.15.66:3300/0 conn(0x7fe69015eaa0 0x7fe6900ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 -- 172.21.15.59:0/376338725 >> 172.21.15.59:0/376338725 conn(0x7fe6900d25b0 msgr2=0x7fe6900d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:10.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 -- 172.21.15.59:0/376338725 shutdown_connections 2024-04-24T06:42:10.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.878+0000 7fe696800640 1 -- 172.21.15.59:0/376338725 wait complete. 2024-04-24T06:42:10.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.880+0000 7fe696800640 1 Processor -- start 2024-04-24T06:42:10.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.880+0000 7fe696800640 1 -- start start 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900b2950 0x7fe69015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6900cf400 0x7fe690160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6900d09e0 con 0x7fe69015eaa0 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe69015ee80 con 0x7fe6900b2950 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe696800640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fe6900d0cc0 con 0x7fe6900cf400 2024-04-24T06:42:10.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.881+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47334/0 (socket says 172.21.15.59:47334) 2024-04-24T06:42:10.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 -- 172.21.15.59:0/117173007 learned_addr learned my addr 172.21.15.59:0/117173007 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68f400640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900b2950 0x7fe69015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 -- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6900cf400 msgr2=0x7fe690160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68ea00640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6900cf400 0x7fe690160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6900cf400 0x7fe690160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 -- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900b2950 msgr2=0x7fe69015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900b2950 0x7fe69015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 -- 172.21.15.59:0/117173007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe690160270 con 0x7fe69015eaa0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.882+0000 7fe68fe00640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097f0 tx=0x7fe6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.883+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688041020 con 0x7fe69015eaa0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.883+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe688030800 con 0x7fe69015eaa0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.883+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688037040 con 0x7fe69015eaa0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.883+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6900cc3e0 con 0x7fe69015eaa0 2024-04-24T06:42:10.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.883+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69015f530 con 0x7fe69015eaa0 2024-04-24T06:42:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.884+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690069190 con 0x7fe69015eaa0 2024-04-24T06:42:10.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.885+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe688044020 con 0x7fe69015eaa0 2024-04-24T06:42:10.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.886+0000 7fe687400640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 0x7fe67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:10.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.886+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fe6880ac290 con 0x7fe69015eaa0 2024-04-24T06:42:10.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.886+0000 7fe68f400640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 0x7fe67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:10.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.887+0000 7fe68f400640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 0x7fe67006edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe680007510 tx=0x7fe6800074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:10.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:10.890+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe68804b030 con 0x7fe69015eaa0 2024-04-24T06:42:11.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.098+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69015f000 con 0x7fe69015eaa0 2024-04-24T06:42:11.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.100+0000 7fe687400640 1 -- 172.21.15.59:0/117173007 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+13206 (secure 0 0 0) 0x7fe68807a900 con 0x7fe69015eaa0 2024-04-24T06:42:11.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:11.102 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":117,"fsid":"fce72cba-ae3c-4ba2-abdc-e927f443d3c7","created":"2024-04-24T06:28:19.922180+0000","modified":"2024-04-24T06:42:06.221731+0000","last_up_change":"2024-04-24T06:28:32.559400+0000","last_in_change":"2024-04-24T06:28:23.443041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:28:33.226060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9dfda38-de77-4e28-9c07-9d8651a35103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6810","nonce":2713443937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6814","nonce":2713443937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6819","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6821","nonce":2713443937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2713443937},{"type":"v1","addr":"172.21.15.59:6817","nonce":2713443937}]},"public_addr":"172.21.15.59:6810/2713443937","cluster_addr":"172.21.15.59:6814/2713443937","heartbeat_back_addr":"172.21.15.59:6821/2713443937","heartbeat_front_addr":"172.21.15.59:6817/2713443937","state":["exists","up"]},{"osd":1,"uuid":"2cdfd9f0-875c-4896-8b64-f84e435ed544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6801","nonce":2375715406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6803","nonce":2375715406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6807","nonce":2375715406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":2375715406},{"type":"v1","addr":"172.21.15.59:6805","nonce":2375715406}]},"public_addr":"172.21.15.59:6801/2375715406","cluster_addr":"172.21.15.59:6803/2375715406","heartbeat_back_addr":"172.21.15.59:6807/2375715406","heartbeat_front_addr":"172.21.15.59:6805/2375715406","state":["exists","up"]},{"osd":2,"uuid":"35f4ea9b-5f4e-456f-b819-b9121469d316","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6811","nonce":1388377934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6815","nonce":1388377934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6823","nonce":1388377934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1388377934},{"type":"v1","addr":"172.21.15.59:6820","nonce":1388377934}]},"public_addr":"172.21.15.59:6811/1388377934","cluster_addr":"172.21.15.59:6815/1388377934","heartbeat_back_addr":"172.21.15.59:6823/1388377934","heartbeat_front_addr":"172.21.15.59:6820/1388377934","state":["exists","up"]},{"osd":3,"uuid":"e444ebd4-f529-486c-803d-a536f20d5b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6825","nonce":2800980920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6827","nonce":2800980920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6831","nonce":2800980920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2800980920},{"type":"v1","addr":"172.21.15.59:6829","nonce":2800980920}]},"public_addr":"172.21.15.59:6825/2800980920","cluster_addr":"172.21.15.59:6827/2800980920","heartbeat_back_addr":"172.21.15.59:6831/2800980920","heartbeat_front_addr":"172.21.15.59:6829/2800980920","state":["exists","up"]},{"osd":4,"uuid":"8d9d7e5e-8655-4873-845e-49975cfd2ce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6801","nonce":4254676376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6803","nonce":4254676376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6807","nonce":4254676376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4254676376},{"type":"v1","addr":"172.21.15.66:6805","nonce":4254676376}]},"public_addr":"172.21.15.66:6801/4254676376","cluster_addr":"172.21.15.66:6803/4254676376","heartbeat_back_addr":"172.21.15.66:6807/4254676376","heartbeat_front_addr":"172.21.15.66:6805/4254676376","state":["exists","up"]},{"osd":5,"uuid":"e9c0ab38-c941-4ebb-88cb-2a7b4b157e1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6817","nonce":79236266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6819","nonce":79236266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6823","nonce":79236266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":79236266},{"type":"v1","addr":"172.21.15.66:6821","nonce":79236266}]},"public_addr":"172.21.15.66:6817/79236266","cluster_addr":"172.21.15.66:6819/79236266","heartbeat_back_addr":"172.21.15.66:6823/79236266","heartbeat_front_addr":"172.21.15.66:6821/79236266","state":["exists","up"]},{"osd":6,"uuid":"33f963ab-a2df-4fda-a7e4-34d0116e6efa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6809","nonce":280071528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6811","nonce":280071528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6815","nonce":280071528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":280071528},{"type":"v1","addr":"172.21.15.66:6813","nonce":280071528}]},"public_addr":"172.21.15.66:6809/280071528","cluster_addr":"172.21.15.66:6811/280071528","heartbeat_back_addr":"172.21.15.66:6815/280071528","heartbeat_front_addr":"172.21.15.66:6813/280071528","state":["exists","up"]},{"osd":7,"uuid":"2c312b36-a2ce-426d-a404-5c717fd2bd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6825","nonce":607700871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6827","nonce":607700871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6831","nonce":607700871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":607700871},{"type":"v1","addr":"172.21.15.66:6829","nonce":607700871}]},"public_addr":"172.21.15.66:6825/607700871","cluster_addr":"172.21.15.66:6827/607700871","heartbeat_back_addr":"172.21.15.66:6831/607700871","heartbeat_front_addr":"172.21.15.66:6829/607700871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.795809+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.012178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.307337+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.829185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:29.067588+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.867722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.855527+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:28:28.859196+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2269052051":"2024-04-25T06:42:03.198805+0000","172.21.15.59:6837/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.59:6835/3446399276":"2024-04-25T06:41:43.864619+0000","172.21.15.66:6835/2441853137":"2024-04-25T06:41:19.989739+0000","172.21.15.59:6835/2269052051":"2024-04-25T06:42:03.198805+0000","172.21.15.66:6833/2441853137":"2024-04-25T06:41:19.989739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 msgr2=0x7fe67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 0x7fe67006edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe680007510 tx=0x7fe6800074a0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 msgr2=0x7fe6901f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097f0 tx=0x7fe6880046c0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 shutdown_connections 2024-04-24T06:42:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fe67006c8f0 0x7fe67006edb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe69015eaa0 0x7fe6901f8c70 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fe6900cf400 0x7fe690160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 --2- 172.21.15.59:0/117173007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6900b2950 0x7fe69015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.103+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 >> 172.21.15.59:0/117173007 conn(0x7fe6900d25b0 msgr2=0x7fe6900ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.104+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 shutdown_connections 2024-04-24T06:42:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:11.104+0000 7fe696800640 1 -- 172.21.15.59:0/117173007 wait complete. 2024-04-24T06:42:12.124 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T06:42:12.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.347+0000 7f7edfe00640 1 Processor -- start 2024-04-24T06:42:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.347+0000 7f7edfe00640 1 -- start start 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7ee015f030 0x7f7ee00d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee015f950 con 0x7f7ee00ce670 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee00ceeb0 con 0x7f7ee015f030 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee00cf0e0 con 0x7f7ee015fbd0 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7ee015f030 0x7f7ee00d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7ee015f030 0x7f7ee00d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47348/0 (socket says 172.21.15.59:47348) 2024-04-24T06:42:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.349+0000 7f7edea00640 1 -- 172.21.15.59:0/1355036550 learned_addr learned my addr 172.21.15.59:0/1355036550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7ed6000640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7edea00640 1 -- 172.21.15.59:0/1355036550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 msgr2=0x7f7ee015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7edea00640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7edea00640 1 -- 172.21.15.59:0/1355036550 >> v1:172.21.15.59:6789/0 conn(0x7f7ee00ce670 legacy=0x7f7ee00cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7edea00640 1 -- 172.21.15.59:0/1355036550 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee00cf3c0 con 0x7f7ee015f030 2024-04-24T06:42:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7ed6000640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.350+0000 7f7ede000640 1 -- 172.21.15.59:0/1355036550 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2276645956 0 0) 0x7f7ee015f950 con 0x7f7ee00ce670 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.351+0000 7f7edea00640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee00d1dd0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d150 tx=0x7f7ed002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a27cd8c4db7ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.351+0000 7f7ede000640 1 -- 172.21.15.59:0/1355036550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0039070 con 0x7f7ee015f030 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.351+0000 7f7ede000640 1 -- 172.21.15.59:0/1355036550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0009600 con 0x7f7ee015f030 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.351+0000 7f7ede000640 1 -- 172.21.15.59:0/1355036550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00099d0 con 0x7f7ee015f030 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 -- 172.21.15.59:0/1355036550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 msgr2=0x7f7ee00d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee00d1dd0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d150 tx=0x7f7ed002c890 comp rx=0 tx=0).stop 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 -- 172.21.15.59:0/1355036550 shutdown_connections 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 --2- 172.21.15.59:0/1355036550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee00d1dd0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 -- 172.21.15.59:0/1355036550 >> 172.21.15.59:0/1355036550 conn(0x7f7ee00d12e0 msgr2=0x7f7ee00d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 -- 172.21.15.59:0/1355036550 shutdown_connections 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.352+0000 7f7edfe00640 1 -- 172.21.15.59:0/1355036550 wait complete. 2024-04-24T06:42:12.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.354+0000 7f7edfe00640 1 Processor -- start 2024-04-24T06:42:12.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.354+0000 7f7edfe00640 1 -- start start 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00ce670 0x7f7ee0157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00cec40 con 0x7f7ee015f030 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00ceac0 con 0x7f7ee00ce670 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7edfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee015c700 con 0x7f7ee015fbd0 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47350/0 (socket says 172.21.15.59:47350) 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 -- 172.21.15.59:0/891558590 learned_addr learned my addr 172.21.15.59:0/891558590 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 -- 172.21.15.59:0/891558590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 msgr2=0x7f7ee015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f7ee015fbd0 0x7f7ee015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 -- 172.21.15.59:0/891558590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00ce670 msgr2=0x7f7ee0157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee00ce670 0x7f7ee0157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.355+0000 7f7ed6000640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee01f8110 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7ed6000640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee015a060 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c7f0 tx=0x7f7ec800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8015070 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8007820 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8007bd0 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01f83f0 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.356+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01f89b0 con 0x7f7ee015f030 2024-04-24T06:42:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.357+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ee004fd90 con 0x7f7ee015f030 2024-04-24T06:42:12.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.358+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec8018070 con 0x7f7ee015f030 2024-04-24T06:42:12.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.359+0000 7f7ed7400640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.359+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f7ec808ba20 con 0x7f7ee015f030 2024-04-24T06:42:12.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7ed7400640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7f7ebc0728f0 0x7f7ebc074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ec801de50 con 0x7f7ebc0728f0 2024-04-24T06:42:12.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7edea00640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f7ec80902a0 con 0x7f7ee015f030 2024-04-24T06:42:12.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7edf400640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7f7ebc0728f0 0x7f7ebc074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.360+0000 7f7edea00640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0036000 tx=0x7f7ed0031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.361+0000 7f7edf400640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7f7ebc0728f0 0x7f7ebc074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.361+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== osd.0 v2:172.21.15.59:6808/2713443937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ec801de50 con 0x7f7ebc0728f0 2024-04-24T06:42:12.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.376+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 --> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7ee01f8e00 con 0x7f7ebc0728f0 2024-04-24T06:42:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7ed7400640 1 -- 172.21.15.59:0/891558590 <== osd.0 v2:172.21.15.59:6808/2713443937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7ee01f8e00 con 0x7f7ebc0728f0 2024-04-24T06:42:12.377 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:12.377 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:12.377 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:12.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7f7ebc0728f0 msgr2=0x7f7ebc074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6808/2713443937,v1:172.21.15.59:6810/2713443937] conn(0x7f7ebc0728f0 0x7f7ebc074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7ebc06c8f0 msgr2=0x7f7ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f7ebc06c8f0 0x7f7ebc06edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0036000 tx=0x7f7ed0031000 comp rx=0 tx=0).stop 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 msgr2=0x7f7ee015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 --2- 172.21.15.59:0/891558590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ee015f030 0x7f7ee015a060 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c7f0 tx=0x7f7ec800ccb0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edf400640 1 -- 172.21.15.59:0/891558590 reap_dead start 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 shutdown_connections 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 >> 172.21.15.59:0/891558590 conn(0x7f7ee00d12e0 msgr2=0x7f7ee006ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 shutdown_connections 2024-04-24T06:42:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.377+0000 7f7edfe00640 1 -- 172.21.15.59:0/891558590 wait complete. 2024-04-24T06:42:12.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T06:42:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.600+0000 7f8f34a00640 1 Processor -- start 2024-04-24T06:42:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.600+0000 7f8f34a00640 1 -- start start 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f34a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 0x7f8f3006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f34a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f300cd880 con 0x7f8f30072680 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f34a00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f30071ec0 con 0x7f8f300cf410 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f34a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f300cdb20 con 0x7f8f3006ddf0 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f2e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f30072680 0x7f8f300cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39288/0 (socket says 172.21.15.59:39288) 2024-04-24T06:42:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.602+0000 7f8f2e800640 1 -- 172.21.15.59:0/251453981 learned_addr learned my addr 172.21.15.59:0/251453981 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f27e00640 1 --2- 172.21.15.59:0/251453981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 0x7f8f3006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770802895 0 0) 0x7f8f300cd880 con 0x7f8f30072680 2024-04-24T06:42:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f14003520 con 0x7f8f30072680 2024-04-24T06:42:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f27e00640 1 -- 172.21.15.59:0/251453981 >> v1:172.21.15.66:6789/0 conn(0x7f8f300cf410 legacy=0x7f8f3015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f27e00640 1 -- 172.21.15.59:0/251453981 >> v1:172.21.15.59:6789/0 conn(0x7f8f30072680 legacy=0x7f8f300cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1385814348 0 0) 0x7f8f14003520 con 0x7f8f30072680 2024-04-24T06:42:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.603+0000 7f8f27e00640 1 -- 172.21.15.59:0/251453981 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30072160 con 0x7f8f3006ddf0 2024-04-24T06:42:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.605+0000 7f8f27e00640 1 --2- 172.21.15.59:0/251453981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 0x7f8f3006e250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009800 tx=0x7f8f1802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b675ba1e7c4830aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.605+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18037070 con 0x7f8f3006ddf0 2024-04-24T06:42:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.605+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1802abc0 con 0x7f8f3006ddf0 2024-04-24T06:42:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.605+0000 7f8f2d400640 1 -- 172.21.15.59:0/251453981 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280019f0 con 0x7f8f3006ddf0 2024-04-24T06:42:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 -- 172.21.15.59:0/251453981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 msgr2=0x7f8f3006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 --2- 172.21.15.59:0/251453981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 0x7f8f3006e250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009800 tx=0x7f8f1802a130 comp rx=0 tx=0).stop 2024-04-24T06:42:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 -- 172.21.15.59:0/251453981 shutdown_connections 2024-04-24T06:42:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 --2- 172.21.15.59:0/251453981 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f3006ddf0 0x7f8f3006e250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 -- 172.21.15.59:0/251453981 >> 172.21.15.59:0/251453981 conn(0x7f8f300b2130 msgr2=0x7f8f300b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 -- 172.21.15.59:0/251453981 shutdown_connections 2024-04-24T06:42:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.606+0000 7f8f34a00640 1 -- 172.21.15.59:0/251453981 wait complete. 2024-04-24T06:42:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.608+0000 7f8f34a00640 1 Processor -- start 2024-04-24T06:42:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.608+0000 7f8f34a00640 1 -- start start 2024-04-24T06:42:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f30072680 0x7f8f3015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f300cf410 0x7f8f3015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3015a820 con 0x7f8f3006ddf0 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3015a6a0 con 0x7f8f300cf410 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f34a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f8f300cff00 con 0x7f8f30072680 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f2de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f2de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47366/0 (socket says 172.21.15.59:47366) 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f2de00640 1 -- 172.21.15.59:0/1863345801 learned_addr learned my addr 172.21.15.59:0/1863345801 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f27e00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f30072680 0x7f8f3015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.609+0000 7f8f2e800640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f300cf410 0x7f8f3015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 -- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f30072680 msgr2=0x7f8f3015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f30072680 0x7f8f3015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 -- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f300cf410 msgr2=0x7f8f3015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f300cf410 0x7f8f3015f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 -- 172.21.15.59:0/1863345801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30155370 con 0x7f8f3006ddf0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f2de00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c340 tx=0x7f8f2000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20017070 con 0x7f8f3006ddf0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f200052d0 con 0x7f8f3006ddf0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20003850 con 0x7f8f3006ddf0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f30155650 con 0x7f8f3006ddf0 2024-04-24T06:42:12.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.610+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3015e820 con 0x7f8f3006ddf0 2024-04-24T06:42:12.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.611+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30068a80 con 0x7f8f3006ddf0 2024-04-24T06:42:12.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.613+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f2001a070 con 0x7f8f3006ddf0 2024-04-24T06:42:12.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.613+0000 7f8f26000640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 0x7f8f0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.614+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f8f2008abd0 con 0x7f8f3006ddf0 2024-04-24T06:42:12.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.614+0000 7f8f27e00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 0x7f8f0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.615+0000 7f8f27e00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 0x7f8f0c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8f18034000 tx=0x7f8f1802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.618+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f20059250 con 0x7f8f3006ddf0 2024-04-24T06:42:12.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.819+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8f300700b0 con 0x7f8f0c06c920 2024-04-24T06:42:12.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.820+0000 7f8f26000640 1 -- 172.21.15.59:0/1863345801 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8f300700b0 con 0x7f8f0c06c920 2024-04-24T06:42:12.821 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.822+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 msgr2=0x7f8f0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 0x7f8f0c06ede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8f18034000 tx=0x7f8f1802e000 comp rx=0 tx=0).stop 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 msgr2=0x7f8f30154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c340 tx=0x7f8f2000c800 comp rx=0 tx=0).stop 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 shutdown_connections 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f8f0c06c920 0x7f8f0c06ede0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f300cf410 0x7f8f3015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f8f30072680 0x7f8f3015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 --2- 172.21.15.59:0/1863345801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f3006ddf0 0x7f8f30154bb0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 >> 172.21.15.59:0/1863345801 conn(0x7f8f300b2130 msgr2=0x7f8f3015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 shutdown_connections 2024-04-24T06:42:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:12.823+0000 7f8f34a00640 1 -- 172.21.15.59:0/1863345801 wait complete. 2024-04-24T06:42:12.842 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T06:42:12.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.059+0000 7faf78c00640 1 Processor -- start 2024-04-24T06:42:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.059+0000 7faf78c00640 1 -- start start 2024-04-24T06:42:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.060+0000 7faf78c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740d0fd0 0x7faf740b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.060+0000 7faf78c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf740d13b0 0x7faf740cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.060+0000 7faf78c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740cfc20 con 0x7faf740cf5a0 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.060+0000 7faf78c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faf740d04c0 con 0x7faf740d13b0 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.060+0000 7faf78c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faf740d22e0 con 0x7faf740d0fd0 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf740d13b0 0x7faf740cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf740d13b0 0x7faf740cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47368/0 (socket says 172.21.15.59:47368) 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 -- 172.21.15.59:0/2327808901 learned_addr learned my addr 172.21.15.59:0/2327808901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf73e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faf740cf5a0 0x7faf740b6020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39294/0 (socket says 172.21.15.59:39294) 2024-04-24T06:42:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf73400640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740d0fd0 0x7faf740b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 -- 172.21.15.59:0/2327808901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740d0fd0 msgr2=0x7faf740b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740d0fd0 0x7faf740b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 -- 172.21.15.59:0/2327808901 >> v1:172.21.15.59:6789/0 conn(0x7faf740cf5a0 legacy=0x7faf740b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72a00640 1 -- 172.21.15.59:0/2327808901 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740cff00 con 0x7faf740d13b0 2024-04-24T06:42:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.061+0000 7faf72000640 1 -- 172.21.15.59:0/2327808901 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400343953 0 0) 0x7faf740cfc20 con 0x7faf740cf5a0 2024-04-24T06:42:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf72a00640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf740cf060 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009e50 tx=0x7faf5c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919bf3e864e6db9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf72000640 1 -- 172.21.15.59:0/2327808901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c036070 con 0x7faf740d13b0 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf72000640 1 -- 172.21.15.59:0/2327808901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c031600 con 0x7faf740d13b0 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf72000640 1 -- 172.21.15.59:0/2327808901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0319d0 con 0x7faf740d13b0 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf78c00640 1 -- 172.21.15.59:0/2327808901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 msgr2=0x7faf740cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.063+0000 7faf78c00640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf740cf060 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009e50 tx=0x7faf5c02a680 comp rx=0 tx=0).stop 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 -- 172.21.15.59:0/2327808901 shutdown_connections 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf740cf060 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 --2- 172.21.15.59:0/2327808901 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740d0fd0 0x7faf740b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 -- 172.21.15.59:0/2327808901 >> 172.21.15.59:0/2327808901 conn(0x7faf7408e030 msgr2=0x7faf7409d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 -- 172.21.15.59:0/2327808901 shutdown_connections 2024-04-24T06:42:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.064+0000 7faf78c00640 1 -- 172.21.15.59:0/2327808901 wait complete. 2024-04-24T06:42:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.065+0000 7faf78c00640 1 Processor -- start 2024-04-24T06:42:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.066+0000 7faf78c00640 1 -- start start 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740cf5a0 0x7faf740cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf740d0fd0 0x7faf740b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf7406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf740d1810 con 0x7faf740d13b0 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faf7415f070 con 0x7faf740d0fd0 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf78c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7faf7415ea90 con 0x7faf740cf5a0 2024-04-24T06:42:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf7406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf7406dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47378/0 (socket says 172.21.15.59:47378) 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73e00640 1 -- 172.21.15.59:0/294979561 learned_addr learned my addr 172.21.15.59:0/294979561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73400640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740cf5a0 0x7faf740cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf72a00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf740d0fd0 0x7faf740b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73e00640 1 -- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740cf5a0 msgr2=0x7faf740cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.067+0000 7faf73e00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7faf740cf5a0 0x7faf740cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf73e00640 1 -- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf740d0fd0 msgr2=0x7faf740b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf73e00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf740d0fd0 0x7faf740b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf73e00640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740b9210 con 0x7faf740d13b0 2024-04-24T06:42:13.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf73e00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf7406dff0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c480 tx=0x7faf6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c018070 con 0x7faf740d13b0 2024-04-24T06:42:13.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c007500 con 0x7faf740d13b0 2024-04-24T06:42:13.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf740cbc00 con 0x7faf740d13b0 2024-04-24T06:42:13.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c004a20 con 0x7faf740d13b0 2024-04-24T06:42:13.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.068+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf740cc1c0 con 0x7faf740d13b0 2024-04-24T06:42:13.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.069+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf7404fd90 con 0x7faf740d13b0 2024-04-24T06:42:13.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.071+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf6c01b070 con 0x7faf740d13b0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.071+0000 7faf6b400640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7faf6c08ac20 con 0x7faf740d13b0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf73400640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf5006c8f0 0x7faf5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf6b400640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7faf500728f0 0x7faf50074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf500753c0 con 0x7faf500728f0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7faf6c08f2a0 con 0x7faf740d13b0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf72a00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7faf500728f0 0x7faf50074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.072+0000 7faf73400640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faf64009ad0 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.073+0000 7faf72a00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7faf500728f0 0x7faf50074d10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.074+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== osd.1 v2:172.21.15.59:6800/2375715406 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf500753c0 con 0x7faf500728f0 2024-04-24T06:42:13.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.089+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 --> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faf740b3d20 con 0x7faf500728f0 2024-04-24T06:42:13.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.089+0000 7faf6b400640 1 -- 172.21.15.59:0/294979561 <== osd.1 v2:172.21.15.59:6800/2375715406 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faf740b3d20 con 0x7faf500728f0 2024-04-24T06:42:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:13.090 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:13.090 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:13.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7faf500728f0 msgr2=0x7faf50074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6800/2375715406,v1:172.21.15.59:6801/2375715406] conn(0x7faf500728f0 0x7faf50074d10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf5006c8f0 msgr2=0x7faf5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7faf5006c8f0 0x7faf5006edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faf64009ad0 tx=0x7faf64008040 comp rx=0 tx=0).stop 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 msgr2=0x7faf7406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 --2- 172.21.15.59:0/294979561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf740d13b0 0x7faf7406dff0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c480 tx=0x7faf6c00c940 comp rx=0 tx=0).stop 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf73e00640 1 -- 172.21.15.59:0/294979561 reap_dead start 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 shutdown_connections 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 >> 172.21.15.59:0/294979561 conn(0x7faf7408e030 msgr2=0x7faf740d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 shutdown_connections 2024-04-24T06:42:13.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.090+0000 7faf78c00640 1 -- 172.21.15.59:0/294979561 wait complete. 2024-04-24T06:42:13.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T06:42:13.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.321+0000 7ffb58e00640 1 Processor -- start 2024-04-24T06:42:13.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.321+0000 7ffb58e00640 1 -- start start 2024-04-24T06:42:13.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.322+0000 7ffb58e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54161e60 0x7ffb540aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.322+0000 7ffb58e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffb54162de0 0x7ffb540b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb58e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb541632e0 con 0x7ffb540afa80 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb58e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffb540aefd0 con 0x7ffb54162de0 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb58e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffb540adcd0 con 0x7ffb54161e60 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb51400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffb54162de0 0x7ffb540b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb52800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffb540afa80 0x7ffb5406ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39306/0 (socket says 172.21.15.59:39306) 2024-04-24T06:42:13.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb52800640 1 -- 172.21.15.59:0/113468160 learned_addr learned my addr 172.21.15.59:0/113468160 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb51e00640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54161e60 0x7ffb540aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.323+0000 7ffb51400640 1 -- 172.21.15.59:0/113468160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54161e60 msgr2=0x7ffb540aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.324+0000 7ffb51400640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54161e60 0x7ffb540aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.324+0000 7ffb51400640 1 -- 172.21.15.59:0/113468160 >> v1:172.21.15.59:6789/0 conn(0x7ffb540afa80 legacy=0x7ffb5406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.324+0000 7ffb51400640 1 -- 172.21.15.59:0/113468160 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb540adfb0 con 0x7ffb54162de0 2024-04-24T06:42:13.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.324+0000 7ffb50a00640 1 -- 172.21.15.59:0/113468160 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059608721 0 0) 0x7ffb541632e0 con 0x7ffb540afa80 2024-04-24T06:42:13.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.325+0000 7ffb51400640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54162de0 0x7ffb540b3db0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009d90 tx=0x7ffb3c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe11a59df91701b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.325+0000 7ffb50a00640 1 -- 172.21.15.59:0/113468160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c031600 con 0x7ffb54162de0 2024-04-24T06:42:13.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.325+0000 7ffb50a00640 1 -- 172.21.15.59:0/113468160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0317a0 con 0x7ffb54162de0 2024-04-24T06:42:13.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.325+0000 7ffb50a00640 1 -- 172.21.15.59:0/113468160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c031b70 con 0x7ffb54162de0 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 -- 172.21.15.59:0/113468160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54162de0 msgr2=0x7ffb540b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54162de0 0x7ffb540b3db0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009d90 tx=0x7ffb3c02a5d0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 -- 172.21.15.59:0/113468160 shutdown_connections 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54162de0 0x7ffb540b3db0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 --2- 172.21.15.59:0/113468160 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54161e60 0x7ffb540aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 -- 172.21.15.59:0/113468160 >> 172.21.15.59:0/113468160 conn(0x7ffb540b2130 msgr2=0x7ffb540b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 -- 172.21.15.59:0/113468160 shutdown_connections 2024-04-24T06:42:13.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.326+0000 7ffb58e00640 1 -- 172.21.15.59:0/113468160 wait complete. 2024-04-24T06:42:13.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.328+0000 7ffb58e00640 1 Processor -- start 2024-04-24T06:42:13.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.328+0000 7ffb58e00640 1 -- start start 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb540afa80 0x7ffb540c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54162de0 0x7ffb540d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb540af480 con 0x7ffb54161e60 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffb540ae570 con 0x7ffb540afa80 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb58e00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7ffb540af600 con 0x7ffb54162de0 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb51400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb51400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47396/0 (socket says 172.21.15.59:47396) 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.329+0000 7ffb51400640 1 -- 172.21.15.59:0/1457498612 learned_addr learned my addr 172.21.15.59:0/1457498612 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb52800640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54162de0 0x7ffb540d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb540afa80 0x7ffb540c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 -- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54162de0 msgr2=0x7ffb540d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54162de0 0x7ffb540d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 -- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb540afa80 msgr2=0x7ffb540c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb540afa80 0x7ffb540c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 -- 172.21.15.59:0/1457498612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb540cfb20 con 0x7ffb54161e60 2024-04-24T06:42:13.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.330+0000 7ffb51400640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009d90 tx=0x7ffb3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.331+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c03f070 con 0x7ffb54161e60 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.331+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c030be0 con 0x7ffb54161e60 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.331+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c009450 con 0x7ffb54161e60 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.331+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb540c6af0 con 0x7ffb54161e60 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.331+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb540c70b0 con 0x7ffb54161e60 2024-04-24T06:42:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.332+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb54068290 con 0x7ffb54161e60 2024-04-24T06:42:13.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.333+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb3c031650 con 0x7ffb54161e60 2024-04-24T06:42:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.335+0000 7ffb4a000640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.335+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7ffb3c0ac9e0 con 0x7ffb54161e60 2024-04-24T06:42:13.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.335+0000 7ffb51e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.336+0000 7ffb51e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 0x7ffb3006edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006fd0 tx=0x7ffb44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.339+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb54068290 con 0x7ffb54161e60 2024-04-24T06:42:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.547+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ffb54068a80 con 0x7ffb3006c8f0 2024-04-24T06:42:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.548+0000 7ffb4a000640 1 -- 172.21.15.59:0/1457498612 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb54068a80 con 0x7ffb3006c8f0 2024-04-24T06:42:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 msgr2=0x7ffb3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 0x7ffb3006edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006fd0 tx=0x7ffb44008040 comp rx=0 tx=0).stop 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 msgr2=0x7ffb5415a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009d90 tx=0x7ffb3c004a40 comp rx=0 tx=0).stop 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 shutdown_connections 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7ffb54162de0 0x7ffb540d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb54161e60 0x7ffb5415a610 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 --2- 172.21.15.59:0/1457498612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb540afa80 0x7ffb540c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 >> 172.21.15.59:0/1457498612 conn(0x7ffb540b2130 msgr2=0x7ffb5406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 shutdown_connections 2024-04-24T06:42:13.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.551+0000 7ffb58e00640 1 -- 172.21.15.59:0/1457498612 wait complete. 2024-04-24T06:42:13.571 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T06:42:13.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.791+0000 7fbf83200640 1 Processor -- start 2024-04-24T06:42:13.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.791+0000 7fbf83200640 1 -- start start 2024-04-24T06:42:13.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf83200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbf840d1710 0x7fbf840d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf83200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf840cec40 0x7fbf840d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf83200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf840cfc70 con 0x7fbf8415e990 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf83200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbf840d0230 con 0x7fbf840cec40 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf83200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbf840d21c0 con 0x7fbf840d1710 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf82800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf840cec40 0x7fbf840d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf82800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf840cec40 0x7fbf840d3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47402/0 (socket says 172.21.15.59:47402) 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.793+0000 7fbf82800640 1 -- 172.21.15.59:0/2077757804 learned_addr learned my addr 172.21.15.59:0/2077757804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.794+0000 7fbf81400640 1 --2- 172.21.15.59:0/2077757804 >> v2:172.21.15.66:3300/0 conn(0x7fbf840d1710 0x7fbf840d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.794+0000 7fbf82800640 1 -- 172.21.15.59:0/2077757804 >> v2:172.21.15.66:3300/0 conn(0x7fbf840d1710 msgr2=0x7fbf840d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.794+0000 7fbf82800640 1 --2- 172.21.15.59:0/2077757804 >> v2:172.21.15.66:3300/0 conn(0x7fbf840d1710 0x7fbf840d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.794+0000 7fbf82800640 1 -- 172.21.15.59:0/2077757804 >> v1:172.21.15.66:6789/0 conn(0x7fbf8415e990 legacy=0x7fbf840d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.794+0000 7fbf82800640 1 -- 172.21.15.59:0/2077757804 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf840cff50 con 0x7fbf840cec40 2024-04-24T06:42:13.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.795+0000 7fbf82800640 1 --2- 172.21.15.59:0/2077757804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840cec40 0x7fbf840d3150 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbf840b7180 tx=0x7fbf7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1594afe5d2fde246 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.795+0000 7fbf80a00640 1 -- 172.21.15.59:0/2077757804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c037070 con 0x7fbf840cec40 2024-04-24T06:42:13.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.795+0000 7fbf80a00640 1 -- 172.21.15.59:0/2077757804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c032440 con 0x7fbf840cec40 2024-04-24T06:42:13.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.795+0000 7fbf80a00640 1 -- 172.21.15.59:0/2077757804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c032810 con 0x7fbf840cec40 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 -- 172.21.15.59:0/2077757804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840cec40 msgr2=0x7fbf840d3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 --2- 172.21.15.59:0/2077757804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840cec40 0x7fbf840d3150 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fbf840b7180 tx=0x7fbf7c02a270 comp rx=0 tx=0).stop 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 -- 172.21.15.59:0/2077757804 shutdown_connections 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 --2- 172.21.15.59:0/2077757804 >> v2:172.21.15.66:3300/0 conn(0x7fbf840d1710 0x7fbf840d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 --2- 172.21.15.59:0/2077757804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840cec40 0x7fbf840d3150 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 -- 172.21.15.59:0/2077757804 >> 172.21.15.59:0/2077757804 conn(0x7fbf8408bf90 msgr2=0x7fbf840d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 -- 172.21.15.59:0/2077757804 shutdown_connections 2024-04-24T06:42:13.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.796+0000 7fbf83200640 1 -- 172.21.15.59:0/2077757804 wait complete. 2024-04-24T06:42:13.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.798+0000 7fbf83200640 1 Processor -- start 2024-04-24T06:42:13.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.798+0000 7fbf83200640 1 -- start start 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf840cec40 0x7fbf840cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 0x7fbf841f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf8415e990 0x7fbf840d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8406e1f0 con 0x7fbf840d1710 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840d26f0 con 0x7fbf840cec40 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf83200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbf841f4b00 con 0x7fbf8415e990 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.799+0000 7fbf81400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 0x7fbf841f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 0x7fbf841f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47408/0 (socket says 172.21.15.59:47408) 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 -- 172.21.15.59:0/1696327486 learned_addr learned my addr 172.21.15.59:0/1696327486 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf82800640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf8415e990 0x7fbf840d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81e00640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf840cec40 0x7fbf840cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 -- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf8415e990 msgr2=0x7fbf840d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbf8415e990 0x7fbf840d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 -- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf840cec40 msgr2=0x7fbf840cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf840cec40 0x7fbf840cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf841f4de0 con 0x7fbf840d1710 2024-04-24T06:42:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81e00640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf840cec40 0x7fbf840cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.800+0000 7fbf81400640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 0x7fbf841f4210 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007fa0 tx=0x7fbf6c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.801+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c014620 con 0x7fbf840d1710 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.801+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c019070 con 0x7fbf840d1710 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.801+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c014a00 con 0x7fbf840d1710 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.801+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf841f50c0 con 0x7fbf840d1710 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.801+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf840c83c0 con 0x7fbf840d1710 2024-04-24T06:42:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.802+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf8404fd90 con 0x7fbf840d1710 2024-04-24T06:42:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.803+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf6c014ba0 con 0x7fbf840d1710 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf7a000640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf6006c940 0x7fbf6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fbf6c08c2b0 con 0x7fbf840d1710 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf7a000640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7fbf60072940 0x7fbf60074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf60075410 con 0x7fbf60072940 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7fbf6c093050 con 0x7fbf840d1710 2024-04-24T06:42:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.804+0000 7fbf81e00640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf6006c940 0x7fbf6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.805+0000 7fbf82800640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7fbf60072940 0x7fbf60074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.805+0000 7fbf81e00640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf6006c940 0x7fbf6006ee00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbf8406dea0 tx=0x7fbf74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.805+0000 7fbf82800640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7fbf60072940 0x7fbf60074d60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.806+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== osd.2 v2:172.21.15.59:6809/1388377934 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf60075410 con 0x7fbf60072940 2024-04-24T06:42:13.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.821+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 --> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbf840c79f0 con 0x7fbf60072940 2024-04-24T06:42:13.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf7a000640 1 -- 172.21.15.59:0/1696327486 <== osd.2 v2:172.21.15.59:6809/1388377934 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbf840c79f0 con 0x7fbf60072940 2024-04-24T06:42:13.823 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:13.823 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:13.823 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7fbf60072940 msgr2=0x7fbf60074d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6809/1388377934,v1:172.21.15.59:6811/1388377934] conn(0x7fbf60072940 0x7fbf60074d60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf6006c940 msgr2=0x7fbf6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbf6006c940 0x7fbf6006ee00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbf8406dea0 tx=0x7fbf74009040 comp rx=0 tx=0).stop 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 msgr2=0x7fbf841f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.822+0000 7fbf83200640 1 --2- 172.21.15.59:0/1696327486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf840d1710 0x7fbf841f4210 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007fa0 tx=0x7fbf6c00e570 comp rx=0 tx=0).stop 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.823+0000 7fbf82800640 1 -- 172.21.15.59:0/1696327486 reap_dead start 2024-04-24T06:42:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.823+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 shutdown_connections 2024-04-24T06:42:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.823+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 >> 172.21.15.59:0/1696327486 conn(0x7fbf8408bf90 msgr2=0x7fbf840d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.823+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 shutdown_connections 2024-04-24T06:42:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:13.823+0000 7fbf83200640 1 -- 172.21.15.59:0/1696327486 wait complete. 2024-04-24T06:42:13.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T06:42:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.054+0000 7fbb7be00640 1 Processor -- start 2024-04-24T06:42:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.054+0000 7fbb7be00640 1 -- start start 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c0d05c0 0x7fbb7c1416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fbb7c15f060 0x7fbb7c0d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb7c15c250 0x7fbb7c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c141c20 con 0x7fbb7c15c250 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c15bdc0 con 0x7fbb7c15f060 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c0d0ee0 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb7c15c250 0x7fbb7c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb7c15c250 0x7fbb7c1555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47418/0 (socket says 172.21.15.59:47418) 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7b400640 1 -- 172.21.15.59:0/134231084 learned_addr learned my addr 172.21.15.59:0/134231084 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.056+0000 7fbb7aa00640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c0d05c0 0x7fbb7c1416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7a000640 1 --2- 172.21.15.59:0/134231084 >> v2:172.21.15.66:3300/0 conn(0x7fbb7c15f060 0x7fbb7c0d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7b400640 1 -- 172.21.15.59:0/134231084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c0d05c0 msgr2=0x7fbb7c1416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7b400640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c0d05c0 0x7fbb7c1416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7b400640 1 -- 172.21.15.59:0/134231084 >> v2:172.21.15.66:3300/0 conn(0x7fbb7c15f060 msgr2=0x7fbb7c0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7b400640 1 --2- 172.21.15.59:0/134231084 >> v2:172.21.15.66:3300/0 conn(0x7fbb7c15f060 0x7fbb7c0d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.057+0000 7fbb7b400640 1 -- 172.21.15.59:0/134231084 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c15ba70 con 0x7fbb7c15c250 2024-04-24T06:42:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.058+0000 7fbb7b400640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c15c250 0x7fbb7c1555d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0d0100 tx=0x7fbb7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abf750f74408b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.058+0000 7fbb79600640 1 -- 172.21.15.59:0/134231084 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74036070 con 0x7fbb7c15c250 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.058+0000 7fbb79600640 1 -- 172.21.15.59:0/134231084 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74031600 con 0x7fbb7c15c250 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.058+0000 7fbb79600640 1 -- 172.21.15.59:0/134231084 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb740319d0 con 0x7fbb7c15c250 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 -- 172.21.15.59:0/134231084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c15c250 msgr2=0x7fbb7c1555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c15c250 0x7fbb7c1555d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0d0100 tx=0x7fbb7402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 -- 172.21.15.59:0/134231084 shutdown_connections 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c15c250 0x7fbb7c1555d0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 --2- 172.21.15.59:0/134231084 >> v2:172.21.15.66:3300/0 conn(0x7fbb7c15f060 0x7fbb7c0d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 --2- 172.21.15.59:0/134231084 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c0d05c0 0x7fbb7c1416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 -- 172.21.15.59:0/134231084 >> 172.21.15.59:0/134231084 conn(0x7fbb7c06bdf0 msgr2=0x7fbb7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 -- 172.21.15.59:0/134231084 shutdown_connections 2024-04-24T06:42:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.059+0000 7fbb7be00640 1 -- 172.21.15.59:0/134231084 wait complete. 2024-04-24T06:42:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.061+0000 7fbb7be00640 1 Processor -- start 2024-04-24T06:42:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.061+0000 7fbb7be00640 1 -- start start 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c15c250 0x7fbb7c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 0x7fbb7c154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15fdc0 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15fae0 con 0x7fbb7c15c250 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7be00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c15f4c0 con 0x7fbb7c15f060 2024-04-24T06:42:14.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47424/0 (socket says 172.21.15.59:47424) 2024-04-24T06:42:14.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.062+0000 7fbb7aa00640 1 -- 172.21.15.59:0/3691101870 learned_addr learned my addr 172.21.15.59:0/3691101870 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7a000640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c15c250 0x7fbb7c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7b400640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 0x7fbb7c154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 -- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 msgr2=0x7fbb7c154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 0x7fbb7c154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 -- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c15c250 msgr2=0x7fbb7c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c15c250 0x7fbb7c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 -- 172.21.15.59:0/3691101870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c141f20 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7b400640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 0x7fbb7c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb7aa00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d800 tx=0x7fbb6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.063+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00b6c0 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.064+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c019070 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.064+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c142200 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.064+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00ba80 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.064+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1427c0 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.065+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c04fd90 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.066+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb6c00bc20 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.067+0000 7fbb72a00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 0x7fbb5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.067+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fbb6c08be00 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.067+0000 7fbb7a000640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 0x7fbb5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.068+0000 7fbb7a000640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 0x7fbb5c06edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.071+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb6c01c360 con 0x7fbb7c0d05c0 2024-04-24T06:42:14.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.275+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fbb7c0cfec0 con 0x7fbb5c06c8f0 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.276+0000 7fbb72a00640 1 -- 172.21.15.59:0/3691101870 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbb7c0cfec0 con 0x7fbb5c06c8f0 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 msgr2=0x7fbb5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 0x7fbb5c06edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68008040 comp rx=0 tx=0).stop 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 msgr2=0x7fbb7c0999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d800 tx=0x7fbb6c00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 shutdown_connections 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fbb5c06c8f0 0x7fbb5c06edb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fbb7c15f060 0x7fbb7c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb7c15c250 0x7fbb7c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 --2- 172.21.15.59:0/3691101870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb7c0d05c0 0x7fbb7c0999e0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 >> 172.21.15.59:0/3691101870 conn(0x7fbb7c06bdf0 msgr2=0x7fbb7c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 shutdown_connections 2024-04-24T06:42:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.279+0000 7fbb7be00640 1 -- 172.21.15.59:0/3691101870 wait complete. 2024-04-24T06:42:14.473 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T06:42:14.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.702+0000 7fc5bfe00640 1 Processor -- start 2024-04-24T06:42:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.702+0000 7fc5bfe00640 1 -- start start 2024-04-24T06:42:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.703+0000 7fc5bfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5c015e990 0x7fc5c00d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.703+0000 7fc5bfe00640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc5c00cec40 0x7fc5c00d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.703+0000 7fc5bfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c00cfc70 con 0x7fc5c00d4d00 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.703+0000 7fc5bfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c00d0230 con 0x7fc5c015e990 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.703+0000 7fc5bfe00640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c00d21c0 con 0x7fc5c00cec40 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.704+0000 7fc5bea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5c015e990 0x7fc5c00d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.704+0000 7fc5bea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5c015e990 0x7fc5c00d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47434/0 (socket says 172.21.15.59:47434) 2024-04-24T06:42:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.704+0000 7fc5bea00640 1 -- 172.21.15.59:0/1773375779 learned_addr learned my addr 172.21.15.59:0/1773375779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.704+0000 7fc5be000640 1 --2- 172.21.15.59:0/1773375779 >> v2:172.21.15.66:3300/0 conn(0x7fc5c00cec40 0x7fc5c00d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.705+0000 7fc5bea00640 1 -- 172.21.15.59:0/1773375779 >> v2:172.21.15.66:3300/0 conn(0x7fc5c00cec40 msgr2=0x7fc5c00d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.705+0000 7fc5bea00640 1 --2- 172.21.15.59:0/1773375779 >> v2:172.21.15.66:3300/0 conn(0x7fc5c00cec40 0x7fc5c00d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.705+0000 7fc5bea00640 1 -- 172.21.15.59:0/1773375779 >> v1:172.21.15.59:6789/0 conn(0x7fc5c00d4d00 legacy=0x7fc5c006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.705+0000 7fc5bea00640 1 -- 172.21.15.59:0/1773375779 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c00cff50 con 0x7fc5c015e990 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.706+0000 7fc5bea00640 1 --2- 172.21.15.59:0/1773375779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c015e990 0x7fc5c00d1000 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b7180 tx=0x7fc5b002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6856104f333ad71a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.706+0000 7fc5bd600640 1 -- 172.21.15.59:0/1773375779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0036070 con 0x7fc5c015e990 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.706+0000 7fc5bd600640 1 -- 172.21.15.59:0/1773375779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0031600 con 0x7fc5c015e990 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.706+0000 7fc5bd600640 1 -- 172.21.15.59:0/1773375779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00319d0 con 0x7fc5c015e990 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 -- 172.21.15.59:0/1773375779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c015e990 msgr2=0x7fc5c00d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/1773375779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c015e990 0x7fc5c00d1000 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b7180 tx=0x7fc5b002a730 comp rx=0 tx=0).stop 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 -- 172.21.15.59:0/1773375779 shutdown_connections 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/1773375779 >> v2:172.21.15.66:3300/0 conn(0x7fc5c00cec40 0x7fc5c00d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/1773375779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c015e990 0x7fc5c00d1000 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 -- 172.21.15.59:0/1773375779 >> 172.21.15.59:0/1773375779 conn(0x7fc5c008bf90 msgr2=0x7fc5c00d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 -- 172.21.15.59:0/1773375779 shutdown_connections 2024-04-24T06:42:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.707+0000 7fc5bfe00640 1 -- 172.21.15.59:0/1773375779 wait complete. 2024-04-24T06:42:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.709+0000 7fc5bfe00640 1 Processor -- start 2024-04-24T06:42:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.709+0000 7fc5bfe00640 1 -- start start 2024-04-24T06:42:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5c00cec40 0x7fc5c00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 0x7fc5c00a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5c015e990 0x7fc5c00a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c006e1f0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00d26f0 con 0x7fc5c015e990 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5bfe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c015d3c0 con 0x7fc5c00cec40 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 0x7fc5c00a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 0x7fc5c00a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47440/0 (socket says 172.21.15.59:47440) 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.710+0000 7fc5be000640 1 -- 172.21.15.59:0/3285828680 learned_addr learned my addr 172.21.15.59:0/3285828680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5bea00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5c00cec40 0x7fc5c00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 -- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5c00cec40 msgr2=0x7fc5c00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc5c00cec40 0x7fc5c00cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 -- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5c015e990 msgr2=0x7fc5c00a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5bf400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5c015e990 0x7fc5c00a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5c015e990 0x7fc5c00a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c00ce4c0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5bf400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5c015e990 0x7fc5c00a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5be000640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 0x7fc5c00a1940 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800d800 tx=0x7fc5a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800b6c0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a8019070 con 0x7fc5c00d4d00 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800baa0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.711+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c00c8df0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.712+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c00c9690 con 0x7fc5c00d4d00 2024-04-24T06:42:14.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.713+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5c00c91f0 con 0x7fc5c00d4d00 2024-04-24T06:42:14.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.714+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5a800bc40 con 0x7fc5c00d4d00 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5b7400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc59c06c8f0 0x7fc59c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fc5a808be30 con 0x7fc5c00d4d00 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5b7400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7fc59c0728f0 0x7fc59c074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc59c0753c0 con 0x7fc59c0728f0 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7fc5a808e450 con 0x7fc5c00d4d00 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5bea00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc59c06c8f0 0x7fc59c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.715+0000 7fc5bf400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7fc59c0728f0 0x7fc59c074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.716+0000 7fc5bf400640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7fc59c0728f0 0x7fc59c074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.716+0000 7fc5bea00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc59c06c8f0 0x7fc59c06edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0033000 tx=0x7fc5b0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.717+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== osd.3 v2:172.21.15.59:6824/2800980920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc59c0753c0 con 0x7fc59c0728f0 2024-04-24T06:42:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.732+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 --> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc5c004fd90 con 0x7fc59c0728f0 2024-04-24T06:42:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.732+0000 7fc5b7400640 1 -- 172.21.15.59:0/3285828680 <== osd.3 v2:172.21.15.59:6824/2800980920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc5c004fd90 con 0x7fc59c0728f0 2024-04-24T06:42:14.733 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:14.733 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:14.733 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7fc59c0728f0 msgr2=0x7fc59c074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6824/2800980920,v1:172.21.15.59:6825/2800980920] conn(0x7fc59c0728f0 0x7fc59c074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc59c06c8f0 msgr2=0x7fc59c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc59c06c8f0 0x7fc59c06edb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0033000 tx=0x7fc5b0009000 comp rx=0 tx=0).stop 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 msgr2=0x7fc5c00a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 --2- 172.21.15.59:0/3285828680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c00d4d00 0x7fc5c00a1940 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800d800 tx=0x7fc5a800dcc0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bf400640 1 -- 172.21.15.59:0/3285828680 reap_dead start 2024-04-24T06:42:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 shutdown_connections 2024-04-24T06:42:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 >> 172.21.15.59:0/3285828680 conn(0x7fc5c008bf90 msgr2=0x7fc5c00b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 shutdown_connections 2024-04-24T06:42:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.733+0000 7fc5bfe00640 1 -- 172.21.15.59:0/3285828680 wait complete. 2024-04-24T06:42:14.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T06:42:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.953+0000 7f5c8fe00640 1 Processor -- start 2024-04-24T06:42:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.953+0000 7f5c8fe00640 1 -- start start 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 0x7f5c900cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c901676a0 con 0x7f5c900d0c10 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8fe00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c90166b20 con 0x7f5c900d0830 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5c90166e00 con 0x7f5c900af720 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c86000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5c900d0c10 0x7f5c900af010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39332/0 (socket says 172.21.15.59:39332) 2024-04-24T06:42:14.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c86000640 1 -- 172.21.15.59:0/3733684550 learned_addr learned my addr 172.21.15.59:0/3733684550 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8f400640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 0x7f5c900cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.955+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743369060 0 0) 0x7f5c901676a0 con 0x7f5c900d0c10 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c74003520 con 0x7f5c900d0c10 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2103310156 0 0) 0x7f5c74003520 con 0x7f5c900d0c10 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 msgr2=0x7f5c900cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 0x7f5c900cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 >> v1:172.21.15.66:6789/0 conn(0x7f5c900d0830 legacy=0x7f5c90163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c901670e0 con 0x7f5c900d0c10 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f5c78001670 con 0x7f5c900d0c10 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 >> v1:172.21.15.59:6789/0 conn(0x7f5c900d0c10 legacy=0x7f5c900af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c740039e0 0x7f5c74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 0x7f5c74005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90166e00 con 0x7f5c74004800 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7400bb80 con 0x7f5c740039e0 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7400bd80 con 0x7f5c900af720 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.956+0000 7f5c86000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c8ea00640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c740039e0 0x7f5c74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c86000640 1 -- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 msgr2=0x7f5c74005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c86000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c900af720 0x7f5c74005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c86000640 1 -- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c740039e0 msgr2=0x7f5c74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:14.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c86000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c740039e0 0x7f5c74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.957+0000 7f5c86000640 1 -- 172.21.15.59:0/3733684550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c74003520 con 0x7f5c74004800 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.958+0000 7f5c86000640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c74004c80 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007fd0 tx=0x7f5c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7201ffc7faad13b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78007c20 con 0x7f5c74004800 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c78005af0 con 0x7f5c74004800 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8e000640 1 -- 172.21.15.59:0/3733684550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78007070 con 0x7f5c74004800 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3733684550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 msgr2=0x7f5c74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3733684550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c74004c80 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007fd0 tx=0x7f5c74005830 comp rx=0 tx=0).stop 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8f400640 1 -- 172.21.15.59:0/3733684550 reap_dead start 2024-04-24T06:42:14.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3733684550 shutdown_connections 2024-04-24T06:42:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3733684550 >> 172.21.15.59:0/3733684550 conn(0x7f5c9006bdf0 msgr2=0x7f5c9006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3733684550 shutdown_connections 2024-04-24T06:42:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.959+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3733684550 wait complete. 2024-04-24T06:42:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.961+0000 7f5c8fe00640 1 Processor -- start 2024-04-24T06:42:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.961+0000 7f5c8fe00640 1 -- start start 2024-04-24T06:42:14.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9015e600 0x7f5c901576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 0x7f5c9015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90166740 con 0x7f5c74004800 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900b45f0 con 0x7f5c9015e600 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.962+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f5c901668c0 con 0x7f5c9015aa70 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47456/0 (socket says 172.21.15.59:47456) 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 -- 172.21.15.59:0/3959392299 learned_addr learned my addr 172.21.15.59:0/3959392299 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8f400640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 0x7f5c9015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c86000640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9015e600 0x7f5c901576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 -- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 msgr2=0x7f5c9015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 0x7f5c9015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 -- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9015e600 msgr2=0x7f5c901576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9015e600 0x7f5c901576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8ea00640 1 -- 172.21.15.59:0/3959392299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90164cc0 con 0x7f5c74004800 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.963+0000 7f5c8f400640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 0x7f5c9015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c8ea00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e500 tx=0x7f5c8000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800052e0 con 0x7f5c74004800 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c80018070 con 0x7f5c74004800 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80003810 con 0x7f5c74004800 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c90164f60 con 0x7f5c74004800 2024-04-24T06:42:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.964+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c90165520 con 0x7f5c74004800 2024-04-24T06:42:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.965+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9004fd90 con 0x7f5c74004800 2024-04-24T06:42:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.966+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c800039b0 con 0x7f5c74004800 2024-04-24T06:42:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.967+0000 7f5c87e00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 0x7f5c6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.967+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f5c8008b660 con 0x7f5c74004800 2024-04-24T06:42:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.967+0000 7f5c86000640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 0x7f5c6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.968+0000 7f5c86000640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 0x7f5c6c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5c78032000 tx=0x7f5c78009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:14.972+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c80059c50 con 0x7f5c74004800 2024-04-24T06:42:15.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.178+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f5c900cedb0 con 0x7f5c6c06c8f0 2024-04-24T06:42:15.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.179+0000 7f5c87e00640 1 -- 172.21.15.59:0/3959392299 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5c900cedb0 con 0x7f5c6c06c8f0 2024-04-24T06:42:15.180 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.181+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 msgr2=0x7f5c6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.181+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 0x7f5c6c06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5c78032000 tx=0x7f5c78009030 comp rx=0 tx=0).stop 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.181+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 msgr2=0x7f5c90158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.181+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e500 tx=0x7f5c8000e9c0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 shutdown_connections 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f5c6c06c8f0 0x7f5c6c06edb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f5c9015aa70 0x7f5c9015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9015e600 0x7f5c901576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 --2- 172.21.15.59:0/3959392299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c74004800 0x7f5c90158ec0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 >> 172.21.15.59:0/3959392299 conn(0x7f5c9006bdf0 msgr2=0x7f5c900ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 shutdown_connections 2024-04-24T06:42:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.182+0000 7f5c8fe00640 1 -- 172.21.15.59:0/3959392299 wait complete. 2024-04-24T06:42:15.202 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T06:42:15.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.422+0000 7f7209a00640 1 Processor -- start 2024-04-24T06:42:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.422+0000 7f7209a00640 1 -- start start 2024-04-24T06:42:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.424+0000 7f7209a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.424+0000 7f7209a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7204160100 0x7f7204160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.424+0000 7f7209a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7204157000 con 0x7f7204156c20 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.424+0000 7f7209a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f72040afb00 con 0x7f7204160100 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.424+0000 7f7209a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f720415cc20 con 0x7f720406bdf0 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7209000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7204160100 0x7f7204160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7203e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7204156c20 0x7f72040aef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39338/0 (socket says 172.21.15.59:39338) 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7209000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7204160100 0x7f7204160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47460/0 (socket says 172.21.15.59:47460) 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7203e00640 1 -- 172.21.15.59:0/4218076426 learned_addr learned my addr 172.21.15.59:0/4218076426 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f71fbe00640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7209000640 1 -- 172.21.15.59:0/4218076426 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 msgr2=0x7f720406c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7209000640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7209000640 1 -- 172.21.15.59:0/4218076426 >> v1:172.21.15.59:6789/0 conn(0x7f7204156c20 legacy=0x7f72040aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.426+0000 7f7209000640 1 -- 172.21.15.59:0/4218076426 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720415cf00 con 0x7f7204160100 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.425+0000 7f7203400640 1 -- 172.21.15.59:0/4218076426 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763912482 0 0) 0x7f7204157000 con 0x7f7204156c20 2024-04-24T06:42:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.427+0000 7f7209000640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204160100 0x7f7204160590 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009d90 tx=0x7f71fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d1a1221e0ebbf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.427+0000 7f7203400640 1 -- 172.21.15.59:0/4218076426 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc007740 con 0x7f7204160100 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.427+0000 7f7203400640 1 -- 172.21.15.59:0/4218076426 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc0078e0 con 0x7f7204160100 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.427+0000 7f7203400640 1 -- 172.21.15.59:0/4218076426 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc007cb0 con 0x7f7204160100 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 -- 172.21.15.59:0/4218076426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204160100 msgr2=0x7f7204160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204160100 0x7f7204160590 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009d90 tx=0x7f71fc02a660 comp rx=0 tx=0).stop 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 -- 172.21.15.59:0/4218076426 shutdown_connections 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204160100 0x7f7204160590 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 --2- 172.21.15.59:0/4218076426 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 -- 172.21.15.59:0/4218076426 >> 172.21.15.59:0/4218076426 conn(0x7f72040b2200 msgr2=0x7f72040b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 -- 172.21.15.59:0/4218076426 shutdown_connections 2024-04-24T06:42:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.428+0000 7f7209a00640 1 -- 172.21.15.59:0/4218076426 wait complete. 2024-04-24T06:42:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.430+0000 7f7209a00640 1 Processor -- start 2024-04-24T06:42:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.430+0000 7f7209a00640 1 -- start start 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 0x7f7204153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204160100 0x7f72040c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f720415d9e0 con 0x7f7204156c20 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f720415d4c0 con 0x7f7204160100 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209a00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f7204073ba0 con 0x7f720406bdf0 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 0x7f7204153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 0x7f7204153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47466/0 (socket says 172.21.15.59:47466) 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 -- 172.21.15.59:0/3434568713 learned_addr learned my addr 172.21.15.59:0/3434568713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7203e00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f7209000640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204160100 0x7f72040c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 -- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 msgr2=0x7f720406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f720406bdf0 0x7f720406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.431+0000 7f71fbe00640 1 -- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204160100 msgr2=0x7f72040c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f71fbe00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204160100 0x7f72040c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f71fbe00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72040c99b0 con 0x7f7204156c20 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7209000640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7204160100 0x7f72040c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f71fbe00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 0x7f7204153830 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00d830 tx=0x7f71ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00b6c0 con 0x7f7204156c20 2024-04-24T06:42:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec019070 con 0x7f7204156c20 2024-04-24T06:42:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00ba80 con 0x7f7204156c20 2024-04-24T06:42:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72040c9c80 con 0x7f7204156c20 2024-04-24T06:42:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.432+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72040ca240 con 0x7f7204156c20 2024-04-24T06:42:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.433+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72040ca480 con 0x7f7204156c20 2024-04-24T06:42:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.434+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71ec01e460 con 0x7f7204156c20 2024-04-24T06:42:15.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.435+0000 7f7200c00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71e006c8f0 0x7f71e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f71ec08bff0 con 0x7f7204156c20 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7203e00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71e006c8f0 0x7f71e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7200c00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f71e00728f0 0x7f71e0074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71e00753c0 con 0x7f71e00728f0 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f71ec08e610 con 0x7f7204156c20 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.436+0000 7f7203e00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71e006c8f0 0x7f71e006edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007fd0 tx=0x7f71f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.437+0000 7f7209000640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f71e00728f0 0x7f71e0074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.438+0000 7f7209000640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f71e00728f0 0x7f71e0074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.440+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== osd.4 v2:172.21.15.66:6800/4254676376 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71e00753c0 con 0x7f71e00728f0 2024-04-24T06:42:15.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.455+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 --> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f720406dc40 con 0x7f71e00728f0 2024-04-24T06:42:15.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7200c00640 1 -- 172.21.15.59:0/3434568713 <== osd.4 v2:172.21.15.66:6800/4254676376 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71fc0076f0 con 0x7f71e00728f0 2024-04-24T06:42:15.457 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:15.457 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:15.457 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f71e00728f0 msgr2=0x7f71e0074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.66:6800/4254676376,v1:172.21.15.66:6801/4254676376] conn(0x7f71e00728f0 0x7f71e0074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71e006c8f0 msgr2=0x7f71e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f71e006c8f0 0x7f71e006edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007fd0 tx=0x7f71f4009040 comp rx=0 tx=0).stop 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 msgr2=0x7f7204153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209a00640 1 --2- 172.21.15.59:0/3434568713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7204156c20 0x7f7204153830 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00d830 tx=0x7f71ec00dcf0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.456+0000 7f7209000640 1 -- 172.21.15.59:0/3434568713 reap_dead start 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.457+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 shutdown_connections 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.457+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 >> 172.21.15.59:0/3434568713 conn(0x7f72040b2200 msgr2=0x7f72040b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.457+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 shutdown_connections 2024-04-24T06:42:15.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.457+0000 7f7209a00640 1 -- 172.21.15.59:0/3434568713 wait complete. 2024-04-24T06:42:15.475 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T06:42:15.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.689+0000 7fc449600640 1 Processor -- start 2024-04-24T06:42:15.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.690+0000 7fc449600640 1 -- start start 2024-04-24T06:42:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 0x7fc44415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc44406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc444072680 0x7fc444072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc44415dda0 con 0x7fc444072680 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440cd880 con 0x7fc4440cf410 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.691+0000 7fc449600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc444071ec0 con 0x7fc44406ddf0 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc444072680 0x7fc444072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc444072680 0x7fc444072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47478/0 (socket says 172.21.15.59:47478) 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 -- 172.21.15.59:0/505245913 learned_addr learned my addr 172.21.15.59:0/505245913 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc442a00640 1 --2- 172.21.15.59:0/505245913 >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 0x7fc44415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc442000640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc44406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 -- 172.21.15.59:0/505245913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 msgr2=0x7fc44406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc44406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 -- 172.21.15.59:0/505245913 >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 msgr2=0x7fc44415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 --2- 172.21.15.59:0/505245913 >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 0x7fc44415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc443400640 1 -- 172.21.15.59:0/505245913 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440cdb20 con 0x7fc444072680 2024-04-24T06:42:15.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.692+0000 7fc442a00640 1 --2- 172.21.15.59:0/505245913 >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 0x7fc44415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.693+0000 7fc443400640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444072b10 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009870 tx=0x7fc43c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da77e19c806a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.694+0000 7fc441600640 1 -- 172.21.15.59:0/505245913 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c037070 con 0x7fc444072680 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.694+0000 7fc441600640 1 -- 172.21.15.59:0/505245913 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c032440 con 0x7fc444072680 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.694+0000 7fc441600640 1 -- 172.21.15.59:0/505245913 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c032810 con 0x7fc444072680 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.694+0000 7fc449600640 1 -- 172.21.15.59:0/505245913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 msgr2=0x7fc444072b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.694+0000 7fc449600640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444072b10 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009870 tx=0x7fc43c02a180 comp rx=0 tx=0).stop 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 -- 172.21.15.59:0/505245913 shutdown_connections 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444072b10 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 --2- 172.21.15.59:0/505245913 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc44406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 --2- 172.21.15.59:0/505245913 >> v2:172.21.15.66:3300/0 conn(0x7fc4440cf410 0x7fc44415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 -- 172.21.15.59:0/505245913 >> 172.21.15.59:0/505245913 conn(0x7fc4440b2130 msgr2=0x7fc4440b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:15.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 -- 172.21.15.59:0/505245913 shutdown_connections 2024-04-24T06:42:15.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.695+0000 7fc449600640 1 -- 172.21.15.59:0/505245913 wait complete. 2024-04-24T06:42:15.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.696+0000 7fc449600640 1 Processor -- start 2024-04-24T06:42:15.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.697+0000 7fc449600640 1 -- start start 2024-04-24T06:42:15.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.697+0000 7fc449600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc444153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.697+0000 7fc449600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc449600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 0x7fc444158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc449600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc44415a6a0 con 0x7fc444072680 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc449600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440cde00 con 0x7fc4440cf410 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc449600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fc44415a820 con 0x7fc44406ddf0 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47484/0 (socket says 172.21.15.59:47484) 2024-04-24T06:42:15.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 -- 172.21.15.59:0/267326086 learned_addr learned my addr 172.21.15.59:0/267326086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:15.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc443400640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 0x7fc444158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442a00640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc444153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 -- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 msgr2=0x7fc444153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc444153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 -- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 msgr2=0x7fc444158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 0x7fc444158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc442000640 1 -- 172.21.15.59:0/267326086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440ae320 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.698+0000 7fc443400640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 0x7fc444158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc442000640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00d800 tx=0x7fc42c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c018030 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0096a0 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c009870 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4440ae600 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.699+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4440aebc0 con 0x7fc444072680 2024-04-24T06:42:15.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.700+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc444068a80 con 0x7fc444072680 2024-04-24T06:42:15.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.702+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc42c01b070 con 0x7fc444072680 2024-04-24T06:42:15.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.702+0000 7fc43aa00640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 0x7fc42006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:15.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.703+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fc42c08bc00 con 0x7fc444072680 2024-04-24T06:42:15.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.703+0000 7fc442a00640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 0x7fc42006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:15.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.704+0000 7fc442a00640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 0x7fc42006edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc434008f80 tx=0x7fc4340084d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:15.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.707+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc42c05a1e0 con 0x7fc444072680 2024-04-24T06:42:15.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.912+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc444073eb0 con 0x7fc42006c8f0 2024-04-24T06:42:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.913+0000 7fc43aa00640 1 -- 172.21.15.59:0/267326086 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc444073eb0 con 0x7fc42006c8f0 2024-04-24T06:42:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T06:42:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 msgr2=0x7fc42006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 0x7fc42006edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc434008f80 tx=0x7fc4340084d0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 msgr2=0x7fc444158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00d800 tx=0x7fc42c00dcc0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 shutdown_connections 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fc42006c8f0 0x7fc42006edb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4440cf410 0x7fc444158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc444072680 0x7fc444158510 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 --2- 172.21.15.59:0/267326086 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fc44406ddf0 0x7fc444153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 >> 172.21.15.59:0/267326086 conn(0x7fc4440b2130 msgr2=0x7fc44415df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.916+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 shutdown_connections 2024-04-24T06:42:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:15.917+0000 7fc449600640 1 -- 172.21.15.59:0/267326086 wait complete. 2024-04-24T06:42:15.936 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T06:42:15.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.152+0000 7f9843000640 1 Processor -- start 2024-04-24T06:42:16.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.152+0000 7f9843000640 1 -- start start 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9843000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f983c0cec40 0x7f983c0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9843000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983c0cfc70 con 0x7f983c15e990 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9843000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f983c0cff50 con 0x7f983c0d1710 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9843000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f983c0d0230 con 0x7f983c0cec40 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9842600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f983c0cec40 0x7f983c0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9841c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f983c15e990 0x7f983c0d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39352/0 (socket says 172.21.15.59:39352) 2024-04-24T06:42:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.153+0000 7f9841c00640 1 -- 172.21.15.59:0/320785794 learned_addr learned my addr 172.21.15.59:0/320785794 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:16.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.154+0000 7f9842600640 1 -- 172.21.15.59:0/320785794 >> v1:172.21.15.66:6789/0 conn(0x7f983c0d1710 legacy=0x7f983c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.154+0000 7f9842600640 1 -- 172.21.15.59:0/320785794 >> v1:172.21.15.59:6789/0 conn(0x7f983c15e990 legacy=0x7f983c0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.154+0000 7f9842600640 1 -- 172.21.15.59:0/320785794 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c0d2410 con 0x7f983c0cec40 2024-04-24T06:42:16.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.155+0000 7f9842600640 1 --2- 172.21.15.59:0/320785794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0cec40 0x7f983c0d3320 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9834009d90 tx=0x7f983402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18152c07cba72cab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.155+0000 7f983be00640 1 -- 172.21.15.59:0/320785794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834007740 con 0x7f983c0cec40 2024-04-24T06:42:16.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f983be00640 1 -- 172.21.15.59:0/320785794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98340078e0 con 0x7f983c0cec40 2024-04-24T06:42:16.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f983be00640 1 -- 172.21.15.59:0/320785794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834007cb0 con 0x7f983c0cec40 2024-04-24T06:42:16.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 -- 172.21.15.59:0/320785794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0cec40 msgr2=0x7f983c0d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 --2- 172.21.15.59:0/320785794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0cec40 0x7f983c0d3320 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9834009d90 tx=0x7f983402a6a0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 -- 172.21.15.59:0/320785794 shutdown_connections 2024-04-24T06:42:16.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 --2- 172.21.15.59:0/320785794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0cec40 0x7f983c0d3320 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 -- 172.21.15.59:0/320785794 >> 172.21.15.59:0/320785794 conn(0x7f983c08bf90 msgr2=0x7f983c0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 -- 172.21.15.59:0/320785794 shutdown_connections 2024-04-24T06:42:16.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.156+0000 7f9843000640 1 -- 172.21.15.59:0/320785794 wait complete. 2024-04-24T06:42:16.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.158+0000 7f9843000640 1 Processor -- start 2024-04-24T06:42:16.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.158+0000 7f9843000640 1 -- start start 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.159+0000 7f9843000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983c0cec40 0x7f983c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9843000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 0x7f983c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9843000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f983c15e990 0x7f983c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9843000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f983c06e1f0 con 0x7f983c0d1710 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9843000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f983c0d21c0 con 0x7f983c0cec40 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9843000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f983c1f4350 con 0x7f983c15e990 2024-04-24T06:42:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 0x7f983c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 0x7f983c0ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47506/0 (socket says 172.21.15.59:47506) 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 -- 172.21.15.59:0/3168299106 learned_addr learned my addr 172.21.15.59:0/3168299106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841c00640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983c0cec40 0x7f983c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9842600640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f983c15e990 0x7f983c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 -- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f983c15e990 msgr2=0x7f983c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f983c15e990 0x7f983c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 -- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983c0cec40 msgr2=0x7f983c15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f983c0cec40 0x7f983c15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.160+0000 7f9841200640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c0cb0b0 con 0x7f983c0d1710 2024-04-24T06:42:16.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9841200640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 0x7f983c0ca7d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9828004640 tx=0x7f982800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828011790 con 0x7f983c0d1710 2024-04-24T06:42:16.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9828017070 con 0x7f983c0d1710 2024-04-24T06:42:16.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828011bb0 con 0x7f983c0d1710 2024-04-24T06:42:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983c0cb390 con 0x7f983c0d1710 2024-04-24T06:42:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.161+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983c0b8b20 con 0x7f983c0d1710 2024-04-24T06:42:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.162+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f983c06dd60 con 0x7f983c0d1710 2024-04-24T06:42:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.163+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9828011d50 con 0x7f983c0d1710 2024-04-24T06:42:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.164+0000 7f9839600640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f981c06c8f0 0x7f981c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f982808b4a0 con 0x7f983c0d1710 2024-04-24T06:42:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9839600640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f981c0728f0 0x7f981c074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f982800de50 con 0x7f981c0728f0 2024-04-24T06:42:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f9828092050 con 0x7f983c0d1710 2024-04-24T06:42:16.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9841c00640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f981c06c8f0 0x7f981c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9841c00640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f981c06c8f0 0x7f981c06edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f983c0b4420 tx=0x7f982c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.165+0000 7f9842600640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f981c0728f0 0x7f981c074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.167+0000 7f9842600640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f981c0728f0 0x7f981c074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.169+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== osd.5 v2:172.21.15.66:6816/79236266 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f982800de50 con 0x7f981c0728f0 2024-04-24T06:42:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.184+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 --> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f983c04fd90 con 0x7f981c0728f0 2024-04-24T06:42:16.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9839600640 1 -- 172.21.15.59:0/3168299106 <== osd.5 v2:172.21.15.66:6816/79236266 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f983c04fd90 con 0x7f981c0728f0 2024-04-24T06:42:16.185 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:16.186 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:16.186 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f981c0728f0 msgr2=0x7f981c074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.66:6816/79236266,v1:172.21.15.66:6817/79236266] conn(0x7f981c0728f0 0x7f981c074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f981c06c8f0 msgr2=0x7f981c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f981c06c8f0 0x7f981c06edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f983c0b4420 tx=0x7f982c009040 comp rx=0 tx=0).stop 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 msgr2=0x7f983c0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 --2- 172.21.15.59:0/3168299106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f983c0d1710 0x7f983c0ca7d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9828004640 tx=0x7f982800c8e0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9842600640 1 -- 172.21.15.59:0/3168299106 reap_dead start 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 shutdown_connections 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.185+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 >> 172.21.15.59:0/3168299106 conn(0x7f983c08bf90 msgr2=0x7f983c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.186+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 shutdown_connections 2024-04-24T06:42:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.186+0000 7f9843000640 1 -- 172.21.15.59:0/3168299106 wait complete. 2024-04-24T06:42:16.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T06:42:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.409+0000 7f9bd2200640 1 Processor -- start 2024-04-24T06:42:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.409+0000 7f9bd2200640 1 -- start start 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 0x7f9bcc144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9bcc144f60 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f9bcc15f3f0 con 0x7f9bcc0af890 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd2200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bcc15f6d0 con 0x7f9bcc0b3010 2024-04-24T06:42:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47522/0 (socket says 172.21.15.59:47522) 2024-04-24T06:42:16.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.410+0000 7f9bd0e00640 1 -- 172.21.15.59:0/3627165269 learned_addr learned my addr 172.21.15.59:0/3627165269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bcbe00640 1 --2- 172.21.15.59:0/3627165269 >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 0x7f9bcc144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd0e00640 1 -- 172.21.15.59:0/3627165269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 msgr2=0x7f9bcc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd1800640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc0d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd0e00640 1 -- 172.21.15.59:0/3627165269 >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 msgr2=0x7f9bcc144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/3627165269 >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 0x7f9bcc144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd0e00640 1 -- 172.21.15.59:0/3627165269 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc15f9b0 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bcbe00640 1 --2- 172.21.15.59:0/3627165269 >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 0x7f9bcc144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.411+0000 7f9bd1800640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.412+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0cdc00 tx=0x7f9bbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d51155c175ff8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.412+0000 7f9bcb400640 1 -- 172.21.15.59:0/3627165269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc037070 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bcb400640 1 -- 172.21.15.59:0/3627165269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bbc032440 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bcb400640 1 -- 172.21.15.59:0/3627165269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc032810 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 -- 172.21.15.59:0/3627165269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc0af4b0 msgr2=0x7f9bcc15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0cdc00 tx=0x7f9bbc02a1e0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 -- 172.21.15.59:0/3627165269 shutdown_connections 2024-04-24T06:42:16.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 --2- 172.21.15.59:0/3627165269 >> v2:172.21.15.66:3300/0 conn(0x7f9bcc0af890 0x7f9bcc144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 --2- 172.21.15.59:0/3627165269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc15ed50 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 -- 172.21.15.59:0/3627165269 >> 172.21.15.59:0/3627165269 conn(0x7f9bcc06bdf0 msgr2=0x7f9bcc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 -- 172.21.15.59:0/3627165269 shutdown_connections 2024-04-24T06:42:16.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.413+0000 7f9bd2200640 1 -- 172.21.15.59:0/3627165269 wait complete. 2024-04-24T06:42:16.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.415+0000 7f9bd2200640 1 Processor -- start 2024-04-24T06:42:16.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.415+0000 7f9bd2200640 1 -- start start 2024-04-24T06:42:16.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc06ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0b3640 con 0x7f9bcc06f110 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0b34c0 con 0x7f9bcc0af4b0 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd2200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f9bcc0d0b00 con 0x7f9bcc0b3010 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.416+0000 7f9bd1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47524/0 (socket says 172.21.15.59:47524) 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 -- 172.21.15.59:0/389394931 learned_addr learned my addr 172.21.15.59:0/389394931 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bcbe00640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc06ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 -- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 msgr2=0x7f9bcc06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc06ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 -- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc0af4b0 msgr2=0x7f9bcc0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 -- 172.21.15.59:0/389394931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc09ad50 con 0x7f9bcc06f110 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bd1800640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c7f0 tx=0x7f9bc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.417+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4015020 con 0x7f9bcc06f110 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.418+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc4004230 con 0x7f9bcc06f110 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.418+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bcc09b030 con 0x7f9bcc06f110 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.418+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4007620 con 0x7f9bcc06f110 2024-04-24T06:42:16.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.418+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bcc14df90 con 0x7f9bcc06f110 2024-04-24T06:42:16.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.419+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bcc04fd90 con 0x7f9bcc06f110 2024-04-24T06:42:16.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.420+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bc4018020 con 0x7f9bcc06f110 2024-04-24T06:42:16.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.421+0000 7f9bc8c00640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.421+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f9bc408bb70 con 0x7f9bcc06f110 2024-04-24T06:42:16.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.421+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.422+0000 7f9bd0e00640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 0x7f9bac06ede0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc034000 tx=0x7f9bbc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.425+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bc401f020 con 0x7f9bcc06f110 2024-04-24T06:42:16.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.630+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f9bcc15d5e0 con 0x7f9bac06c920 2024-04-24T06:42:16.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.631+0000 7f9bc8c00640 1 -- 172.21.15.59:0/389394931 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9bbc0327b0 con 0x7f9bac06c920 2024-04-24T06:42:16.632 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T06:42:16.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 msgr2=0x7f9bac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 0x7f9bac06ede0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc034000 tx=0x7f9bbc02e000 comp rx=0 tx=0).stop 2024-04-24T06:42:16.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 msgr2=0x7f9bcc09a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c7f0 tx=0x7f9bc400ccb0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 shutdown_connections 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f9bac06c920 0x7f9bac06ede0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bcc06f110 0x7f9bcc09a170 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f9bcc0b3010 0x7f9bcc06ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 --2- 172.21.15.59:0/389394931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc0af4b0 0x7f9bcc0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 >> 172.21.15.59:0/389394931 conn(0x7f9bcc06bdf0 msgr2=0x7f9bcc1611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 shutdown_connections 2024-04-24T06:42:16.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.634+0000 7f9bd2200640 1 -- 172.21.15.59:0/389394931 wait complete. 2024-04-24T06:42:16.653 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T06:42:16.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:16.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.870+0000 7f58bb200640 1 Processor -- start 2024-04-24T06:42:16.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.871+0000 7f58bb200640 1 -- start start 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.872+0000 7f58bb200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7f58bc15f070 0x7f58bc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.872+0000 7f58bb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58bc06c740 con 0x7f58bc159020 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.872+0000 7f58bb200640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58bc071df0 con 0x7f58bc06c330 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.872+0000 7f58bb200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7f58bc0720d0 con 0x7f58bc15f070 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.873+0000 7f58b9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f58bc159020 0x7f58bc073200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39362/0 (socket says 172.21.15.59:39362) 2024-04-24T06:42:16.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.873+0000 7f58b9e00640 1 -- 172.21.15.59:0/2205906832 learned_addr learned my addr 172.21.15.59:0/2205906832 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.873+0000 7f58b3e00640 1 --2- 172.21.15.59:0/2205906832 >> v2:172.21.15.66:3300/0 conn(0x7f58bc15f070 0x7f58bc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.873+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306948643 0 0) 0x7f58ac001110 con 0x7f58bc159020 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.873+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58a0005730 con 0x7f58bc159020 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4127382712 0 0) 0x7f58a0005730 con 0x7f58bc159020 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 >> v2:172.21.15.66:3300/0 conn(0x7f58bc15f070 msgr2=0x7f58bc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 --2- 172.21.15.59:0/2205906832 >> v2:172.21.15.66:3300/0 conn(0x7f58bc15f070 0x7f58bc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 >> v1:172.21.15.66:6789/0 conn(0x7f58bc06c330 legacy=0x7f58bc15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc0723b0 con 0x7f58bc159020 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7f58ac001840 con 0x7f58bc159020 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 >> v1:172.21.15.59:6789/0 conn(0x7f58bc159020 legacy=0x7f58bc073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58a0005bc0 0x7f58a0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58a0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58a00073d0 0x7f58a000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0720d0 con 0x7f58a0006a10 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58a000e270 con 0x7f58a0005bc0 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f58a000e470 con 0x7f58a00073d0 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b3e00640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58a0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.874+0000 7f58b9e00640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58a0005bc0 0x7f58a0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58b3e00640 1 -- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58a00073d0 msgr2=0x7f58a000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58ba800640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58a00073d0 0x7f58a000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58b3e00640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58a00073d0 0x7f58a000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58b3e00640 1 -- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58a0005bc0 msgr2=0x7f58a0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58b3e00640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58a0005bc0 0x7f58a0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58b3e00640 1 -- 172.21.15.59:0/2205906832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a0005730 con 0x7f58a0006a10 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58ba800640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58a00073d0 0x7f58a000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:16.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.875+0000 7f58ba800640 1 -- 172.21.15.59:0/2205906832 reap_dead start 2024-04-24T06:42:16.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.876+0000 7f58b3e00640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58a0006e90 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009ac0 tx=0x7f58a0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec22f89e27922ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.876+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4036070 con 0x7f58a0006a10 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.876+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a4031400 con 0x7f58a0006a10 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.876+0000 7f58b9400640 1 -- 172.21.15.59:0/2205906832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4009690 con 0x7f58a0006a10 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 -- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 msgr2=0x7f58a0006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58a0006e90 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009ac0 tx=0x7f58a0007a40 comp rx=0 tx=0).stop 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 -- 172.21.15.59:0/2205906832 shutdown_connections 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 --2- 172.21.15.59:0/2205906832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58a0006e90 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 -- 172.21.15.59:0/2205906832 >> 172.21.15.59:0/2205906832 conn(0x7f58bc0b2200 msgr2=0x7f58bc0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 -- 172.21.15.59:0/2205906832 shutdown_connections 2024-04-24T06:42:16.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.877+0000 7f58bb200640 1 -- 172.21.15.59:0/2205906832 wait complete. 2024-04-24T06:42:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.879+0000 7f58bb200640 1 Processor -- start 2024-04-24T06:42:16.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.879+0000 7f58bb200640 1 -- start start 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58bc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc1565e0 0x7f58bc157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58bc0b0ce0 0x7f58bc1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc159400 con 0x7f58a0006a10 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc072f00 con 0x7f58bc1565e0 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58bb200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc072970 con 0x7f58bc0b0ce0 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58bc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58bc15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47548/0 (socket says 172.21.15.59:47548) 2024-04-24T06:42:16.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.880+0000 7f58b9e00640 1 -- 172.21.15.59:0/1032386565 learned_addr learned my addr 172.21.15.59:0/1032386565 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b3e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc1565e0 0x7f58bc157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 -- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58bc0b0ce0 msgr2=0x7f58bc1f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58ba800640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58bc0b0ce0 0x7f58bc1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f58bc0b0ce0 0x7f58bc1f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 -- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc1565e0 msgr2=0x7f58bc157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc1565e0 0x7f58bc157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc155540 con 0x7f58a0006a10 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b3e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58bc1565e0 0x7f58bc157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b9e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58bc15e500 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f58ac007300 tx=0x7f58ac0077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.881+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e950 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.882+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac018070 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.882+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc155820 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.882+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00ed30 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.882+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc155de0 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.882+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58bc04fd90 con 0x7f58a0006a10 2024-04-24T06:42:16.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.884+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58ac026430 con 0x7f58a0006a10 2024-04-24T06:42:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b2000640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f589806c920 0x7f589806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f58ac08c0e0 con 0x7f58a0006a10 2024-04-24T06:42:16.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b3e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f589806c920 0x7f589806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b2000640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f5898072920 0x7f5898074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:16.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58ac024e80 con 0x7f5898072920 2024-04-24T06:42:16.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.885+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f58ac08c490 con 0x7f58a0006a10 2024-04-24T06:42:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.886+0000 7f58b3e00640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f589806c920 0x7f589806ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58a4033000 tx=0x7f58a4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.886+0000 7f58ba800640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f5898072920 0x7f5898074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:16.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.887+0000 7f58ba800640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f5898072920 0x7f5898074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:16.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.889+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== osd.6 v2:172.21.15.66:6808/280071528 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58ac024e80 con 0x7f5898072920 2024-04-24T06:42:16.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.904+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 --> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58bc06ea90 con 0x7f5898072920 2024-04-24T06:42:16.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.905+0000 7f58b2000640 1 -- 172.21.15.59:0/1032386565 <== osd.6 v2:172.21.15.66:6808/280071528 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58bc06ea90 con 0x7f5898072920 2024-04-24T06:42:16.906 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:16.906 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:16.906 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:16.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f5898072920 msgr2=0x7f5898074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.66:6808/280071528,v1:172.21.15.66:6809/280071528] conn(0x7f5898072920 0x7f5898074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f589806c920 msgr2=0x7f589806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f589806c920 0x7f589806ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f58a4033000 tx=0x7f58a4008030 comp rx=0 tx=0).stop 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 msgr2=0x7f58bc15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 --2- 172.21.15.59:0/1032386565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a0006a10 0x7f58bc15e500 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f58ac007300 tx=0x7f58ac0077c0 comp rx=0 tx=0).stop 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58ba800640 1 -- 172.21.15.59:0/1032386565 reap_dead start 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 shutdown_connections 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 >> 172.21.15.59:0/1032386565 conn(0x7f58bc0b2200 msgr2=0x7f58bc1513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 shutdown_connections 2024-04-24T06:42:16.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:16.906+0000 7f58bb200640 1 -- 172.21.15.59:0/1032386565 wait complete. 2024-04-24T06:42:16.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T06:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.126+0000 7f452b200640 1 Processor -- start 2024-04-24T06:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.126+0000 7f452b200640 1 -- start start 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af890 0x7f452415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4524144a20 0x7f45240d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45240b3720 con 0x7f45240af4b0 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4524144ed0 con 0x7f4524144a20 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f452415f3f0 con 0x7f45240af890 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f452a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4524144a20 0x7f45240d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f4529e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f45240af4b0 0x7f45240b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39378/0 (socket says 172.21.15.59:39378) 2024-04-24T06:42:17.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.128+0000 7f4529e00640 1 -- 172.21.15.59:0/2100884698 learned_addr learned my addr 172.21.15.59:0/2100884698 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:17.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f4529400640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af890 0x7f452415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f452a800640 1 -- 172.21.15.59:0/2100884698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af890 msgr2=0x7f452415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f452a800640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af890 0x7f452415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f452a800640 1 -- 172.21.15.59:0/2100884698 >> v1:172.21.15.59:6789/0 conn(0x7f45240af4b0 legacy=0x7f45240b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f452a800640 1 -- 172.21.15.59:0/2100884698 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452415f6d0 con 0x7f4524144a20 2024-04-24T06:42:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.129+0000 7f4528a00640 1 -- 172.21.15.59:0/2100884698 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1424930465 0 0) 0x7f45240b3720 con 0x7f45240af4b0 2024-04-24T06:42:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.130+0000 7f452a800640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f45240d05c0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f4520009d90 tx=0x7f452002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60175768387b26df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.130+0000 7f4528a00640 1 -- 172.21.15.59:0/2100884698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520007740 con 0x7f4524144a20 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.130+0000 7f4528a00640 1 -- 172.21.15.59:0/2100884698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45200078e0 con 0x7f4524144a20 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.130+0000 7f4528a00640 1 -- 172.21.15.59:0/2100884698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520007cb0 con 0x7f4524144a20 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 -- 172.21.15.59:0/2100884698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 msgr2=0x7f45240d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f45240d05c0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f4520009d90 tx=0x7f452002a660 comp rx=0 tx=0).stop 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 -- 172.21.15.59:0/2100884698 shutdown_connections 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f45240d05c0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 --2- 172.21.15.59:0/2100884698 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af890 0x7f452415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 -- 172.21.15.59:0/2100884698 >> 172.21.15.59:0/2100884698 conn(0x7f452406bdf0 msgr2=0x7f452406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 -- 172.21.15.59:0/2100884698 shutdown_connections 2024-04-24T06:42:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.131+0000 7f452b200640 1 -- 172.21.15.59:0/2100884698 wait complete. 2024-04-24T06:42:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.133+0000 7f452b200640 1 Processor -- start 2024-04-24T06:42:17.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.133+0000 7f452b200640 1 -- start start 2024-04-24T06:42:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af4b0 0x7f45241f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452409a170 0x7f452406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45240d0b00 con 0x7f4524144a20 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f452415fc90 con 0x7f452409a170 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f452b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f45240d0c80 con 0x7f45240af4b0 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f4529400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f4529400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47564/0 (socket says 172.21.15.59:47564) 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.134+0000 7f4529400640 1 -- 172.21.15.59:0/3469677391 learned_addr learned my addr 172.21.15.59:0/3469677391 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529e00640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af4b0 0x7f45241f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 -- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af4b0 msgr2=0x7f45241f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af4b0 0x7f45241f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 -- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452409a170 msgr2=0x7f452406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452409a170 0x7f452406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 -- 172.21.15.59:0/3469677391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452409a730 con 0x7f4524144a20 2024-04-24T06:42:17.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f4529400640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f451400d830 tx=0x7f451400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400b6c0 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4514019070 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400ba80 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.135+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452409aa10 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.136+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452409afd0 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.137+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452404fd90 con 0x7f4524144a20 2024-04-24T06:42:17.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.138+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f451401e460 con 0x7f4524144a20 2024-04-24T06:42:17.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.139+0000 7f451e000640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 0x7f450806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.139+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f451408bfe0 con 0x7f4524144a20 2024-04-24T06:42:17.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.139+0000 7f4529e00640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 0x7f450806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.140+0000 7f4529e00640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 0x7f450806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f60 tx=0x7f4518009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.143+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f451405a660 con 0x7f4524144a20 2024-04-24T06:42:17.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.348+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f45240cedb0 con 0x7f450806c8f0 2024-04-24T06:42:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.349+0000 7f451e000640 1 -- 172.21.15.59:0/3469677391 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f45240cedb0 con 0x7f450806c8f0 2024-04-24T06:42:17.350 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 msgr2=0x7f450806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 0x7f450806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f60 tx=0x7f4518009040 comp rx=0 tx=0).stop 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 msgr2=0x7f4524142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f451400d830 tx=0x7f451400dcf0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 shutdown_connections 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f450806c8f0 0x7f450806edb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452409a170 0x7f452406ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4524144a20 0x7f4524142940 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 --2- 172.21.15.59:0/3469677391 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f45240af4b0 0x7f45241f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 >> 172.21.15.59:0/3469677391 conn(0x7f452406bdf0 msgr2=0x7f45240b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 shutdown_connections 2024-04-24T06:42:17.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.352+0000 7f452b200640 1 -- 172.21.15.59:0/3469677391 wait complete. 2024-04-24T06:42:17.371 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T06:42:17.371 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T06:42:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.578+0000 7fd57a600640 1 Processor -- start 2024-04-24T06:42:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.579+0000 7fd57a600640 1 -- start start 2024-04-24T06:42:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.580+0000 7fd57a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 0x7fd5740d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.580+0000 7fd57a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5740d0310 con 0x7fd574161fd0 2024-04-24T06:42:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.580+0000 7fd57a600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5740d05f0 con 0x7fd5740cf5e0 2024-04-24T06:42:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.580+0000 7fd57a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd5740d1a40 con 0x7fd5740d10a0 2024-04-24T06:42:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.581+0000 7fd573400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd574161fd0 0x7fd5740b40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39382/0 (socket says 172.21.15.59:39382) 2024-04-24T06:42:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.581+0000 7fd573400640 1 -- 172.21.15.59:0/2261689068 learned_addr learned my addr 172.21.15.59:0/2261689068 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.581+0000 7fd56be00640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 0x7fd5740d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.581+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570100404 0 0) 0x7fd5740d0310 con 0x7fd574161fd0 2024-04-24T06:42:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.581+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd558003520 con 0x7fd574161fd0 2024-04-24T06:42:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1096806483 0 0) 0x7fd558003520 con 0x7fd574161fd0 2024-04-24T06:42:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 msgr2=0x7fd5740d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 0x7fd5740d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 >> v1:172.21.15.66:6789/0 conn(0x7fd5740cf5e0 legacy=0x7fd5740cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5740d2520 con 0x7fd574161fd0 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd564001670 con 0x7fd574161fd0 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 >> v1:172.21.15.59:6789/0 conn(0x7fd574161fd0 legacy=0x7fd5740b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd558003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd558004800 0x7fd558004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 0x7fd5580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd55800ba70 con 0x7fd5580039e0 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd55800bc20 con 0x7fd558004800 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd55800be20 con 0x7fd5740d10a0 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.582+0000 7fd573400640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd558003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.583+0000 7fd573400640 1 -- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 msgr2=0x7fd5580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.583+0000 7fd573400640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740d10a0 0x7fd5580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.583+0000 7fd573400640 1 -- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd558004800 msgr2=0x7fd558004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.583+0000 7fd573400640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd558004800 0x7fd558004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.583+0000 7fd573400640 1 -- 172.21.15.59:0/2261689068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd558003520 con 0x7fd5580039e0 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd573400640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd558003e00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c160 tx=0x7fd558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d4c60186d7bdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564007c60 con 0x7fd5580039e0 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564005ad0 con 0x7fd5580039e0 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd572a00640 1 -- 172.21.15.59:0/2261689068 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564007070 con 0x7fd5580039e0 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd57a600640 1 -- 172.21.15.59:0/2261689068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 msgr2=0x7fd558003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd57a600640 1 --2- 172.21.15.59:0/2261689068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd558003e00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c160 tx=0x7fd558005830 comp rx=0 tx=0).stop 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.584+0000 7fd573e00640 1 -- 172.21.15.59:0/2261689068 reap_dead start 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.585+0000 7fd57a600640 1 -- 172.21.15.59:0/2261689068 shutdown_connections 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.585+0000 7fd57a600640 1 -- 172.21.15.59:0/2261689068 >> 172.21.15.59:0/2261689068 conn(0x7fd57408bf90 msgr2=0x7fd5740d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.585+0000 7fd57a600640 1 -- 172.21.15.59:0/2261689068 shutdown_connections 2024-04-24T06:42:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.585+0000 7fd57a600640 1 -- 172.21.15.59:0/2261689068 wait complete. 2024-04-24T06:42:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.586+0000 7fd57a600640 1 Processor -- start 2024-04-24T06:42:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 -- start start 2024-04-24T06:42:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd574158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57415f660 0x7fd57415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740c8900 0x7fd5740c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5740d1680 con 0x7fd5580039e0 2024-04-24T06:42:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5740d1500 con 0x7fd57415f660 2024-04-24T06:42:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.587+0000 7fd57a600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd5740d1800 con 0x7fd5740c8900 2024-04-24T06:42:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd574158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd574158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47584/0 (socket says 172.21.15.59:47584) 2024-04-24T06:42:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 -- 172.21.15.59:0/805459253 learned_addr learned my addr 172.21.15.59:0/805459253 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd56be00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57415f660 0x7fd57415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573e00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740c8900 0x7fd5740c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 -- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740c8900 msgr2=0x7fd5740c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd5740c8900 0x7fd5740c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 -- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57415f660 msgr2=0x7fd57415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57415f660 0x7fd57415fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.588+0000 7fd573400640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5740cb920 con 0x7fd5580039e0 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd573400640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd574158d20 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c160 tx=0x7fd564001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564041020 con 0x7fd5580039e0 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5640013b0 con 0x7fd5580039e0 2024-04-24T06:42:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564007070 con 0x7fd5580039e0 2024-04-24T06:42:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5740c92c0 con 0x7fd5580039e0 2024-04-24T06:42:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.589+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5740c9580 con 0x7fd5580039e0 2024-04-24T06:42:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.590+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd574068a80 con 0x7fd5580039e0 2024-04-24T06:42:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.591+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd564044020 con 0x7fd5580039e0 2024-04-24T06:42:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.592+0000 7fd56b400640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd55006c920 0x7fd55006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.592+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fd5640abcc0 con 0x7fd5580039e0 2024-04-24T06:42:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd56b400640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fd550072920 0x7fd550074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd564049e70 con 0x7fd550072920 2024-04-24T06:42:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd56be00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd55006c920 0x7fd55006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7fd5640ae2e0 con 0x7fd5580039e0 2024-04-24T06:42:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd56be00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd55006c920 0x7fd55006ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd574069240 tx=0x7fd55c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.593+0000 7fd573e00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fd550072920 0x7fd550074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.594+0000 7fd573e00640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fd550072920 0x7fd550074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.597+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== osd.7 v2:172.21.15.66:6824/607700871 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd564049e70 con 0x7fd550072920 2024-04-24T06:42:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.611+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 --> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd5740d1ea0 con 0x7fd550072920 2024-04-24T06:42:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.612+0000 7fd56b400640 1 -- 172.21.15.59:0/805459253 <== osd.7 v2:172.21.15.66:6824/607700871 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd5740d1ea0 con 0x7fd550072920 2024-04-24T06:42:17.613 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-24T06:42:17.613 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T06:42:17.614 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fd550072920 msgr2=0x7fd550074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.66:6824/607700871,v1:172.21.15.66:6825/607700871] conn(0x7fd550072920 0x7fd550074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd55006c920 msgr2=0x7fd55006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd55006c920 0x7fd55006ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd574069240 tx=0x7fd55c008040 comp rx=0 tx=0).stop 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 msgr2=0x7fd574158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 --2- 172.21.15.59:0/805459253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5580039e0 0x7fd574158d20 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c160 tx=0x7fd564001970 comp rx=0 tx=0).stop 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd573e00640 1 -- 172.21.15.59:0/805459253 reap_dead start 2024-04-24T06:42:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 shutdown_connections 2024-04-24T06:42:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 >> 172.21.15.59:0/805459253 conn(0x7fd57408bf90 msgr2=0x7fd5740d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.613+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 shutdown_connections 2024-04-24T06:42:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.614+0000 7fd57a600640 1 -- 172.21.15.59:0/805459253 wait complete. 2024-04-24T06:42:17.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T06:42:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.852+0000 7fa0b0000640 1 Processor -- start 2024-04-24T06:42:17.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.852+0000 7fa0b0000640 1 -- start start 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0b0000640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fa0a80cf410 0x7fa0a815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0b0000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 0x7fa0a806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0b0000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a815dda0 con 0x7fa0a8072680 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0b0000640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a80cd880 con 0x7fa0a80cf410 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0b0000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a8071ec0 con 0x7fa0a806ddf0 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0ae000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa0a8072680 0x7fa0a80cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39386/0 (socket says 172.21.15.59:39386) 2024-04-24T06:42:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0ae000640 1 -- 172.21.15.59:0/3671831443 learned_addr learned my addr 172.21.15.59:0/3671831443 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.854+0000 7fa0ad600640 1 --2- 172.21.15.59:0/3671831443 >> v2:172.21.15.66:3300/0 conn(0x7fa0a80cf410 0x7fa0a815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0acc00640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 0x7fa0a806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109800128 0 0) 0x7fa0a0001490 con 0x7fa0a8072680 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa090003520 con 0x7fa0a8072680 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2637893366 0 0) 0x7fa090003520 con 0x7fa0a8072680 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 msgr2=0x7fa0a806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 0x7fa0a806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 >> v2:172.21.15.66:3300/0 conn(0x7fa0a80cf410 msgr2=0x7fa0a815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 --2- 172.21.15.59:0/3671831443 >> v2:172.21.15.66:3300/0 conn(0x7fa0a80cf410 0x7fa0a815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a80cdb20 con 0x7fa0a8072680 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fa0a0001bc0 con 0x7fa0a8072680 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 >> v1:172.21.15.59:6789/0 conn(0x7fa0a8072680 legacy=0x7fa0a80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 0x7fa090004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a80cd880 con 0x7fa0900039b0 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8071ec0 con 0x7fa090004d40 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa09000b9c0 con 0x7fa0a806ddf0 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.855+0000 7fa0ad600640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ad600640 1 -- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 msgr2=0x7fa090004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ad600640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0a806ddf0 0x7fa090004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ae000640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ad600640 1 -- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa090004d40 msgr2=0x7fa0900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ad600640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa090004d40 0x7fa0900051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.856+0000 7fa0ad600640 1 -- 172.21.15.59:0/3671831443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa090003520 con 0x7fa0900039b0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.857+0000 7fa0ad600640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009ac0 tx=0x7fa090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec7d555e7ef708e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.857+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c036070 con 0x7fa0900039b0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.857+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09c031400 con 0x7fa0900039b0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.857+0000 7fa0a7e00640 1 -- 172.21.15.59:0/3671831443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009690 con 0x7fa0900039b0 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 -- 172.21.15.59:0/3671831443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0900039b0 msgr2=0x7fa090003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 --2- 172.21.15.59:0/3671831443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0900039b0 0x7fa090003dd0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009ac0 tx=0x7fa090005830 comp rx=0 tx=0).stop 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0ae000640 1 -- 172.21.15.59:0/3671831443 reap_dead start 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 -- 172.21.15.59:0/3671831443 shutdown_connections 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 -- 172.21.15.59:0/3671831443 >> 172.21.15.59:0/3671831443 conn(0x7fa0a80b2130 msgr2=0x7fa0a80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 -- 172.21.15.59:0/3671831443 shutdown_connections 2024-04-24T06:42:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.858+0000 7fa0b0000640 1 -- 172.21.15.59:0/3671831443 wait complete. 2024-04-24T06:42:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.860+0000 7fa0b0000640 1 Processor -- start 2024-04-24T06:42:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.860+0000 7fa0b0000640 1 -- start start 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0900039b0 0x7fa0a8145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0a814f6b0 0x7fa0a8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a815a6a0 con 0x7fa0a8154a30 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a80cde00 con 0x7fa0a814f6b0 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0b0000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a815a820 con 0x7fa0900039b0 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0acc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0acc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47612/0 (socket says 172.21.15.59:47612) 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.861+0000 7fa0acc00640 1 -- 172.21.15.59:0/1098325139 learned_addr learned my addr 172.21.15.59:0/1098325139 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 -- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0900039b0 msgr2=0x7fa0a8145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0900039b0 0x7fa0a8145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 -- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0a814f6b0 msgr2=0x7fa0a8150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0a814f6b0 0x7fa0a8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 -- 172.21.15.59:0/1098325139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a8150100 con 0x7fa0a8154a30 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0acc00640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fa09400e7c0 tx=0x7fa09400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094017070 con 0x7fa0a8154a30 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09400c8e0 con 0x7fa0a8154a30 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09400ccb0 con 0x7fa0a8154a30 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.862+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a814fe20 con 0x7fa0a8154a30 2024-04-24T06:42:17.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.863+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a8152140 con 0x7fa0a8154a30 2024-04-24T06:42:17.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.864+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a8068a80 con 0x7fa0a8154a30 2024-04-24T06:42:17.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.865+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa09401a070 con 0x7fa0a8154a30 2024-04-24T06:42:17.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.865+0000 7fa0a5600640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 0x7fa08806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.866+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fa09408bab0 con 0x7fa0a8154a30 2024-04-24T06:42:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.866+0000 7fa0ad600640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 0x7fa08806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.867+0000 7fa0ad600640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 0x7fa08806ede0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa09c033000 tx=0x7fa09c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:17.870+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa09405a120 con 0x7fa0a8154a30 2024-04-24T06:42:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.073+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa0a80700b0 con 0x7fa08806c920 2024-04-24T06:42:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.074+0000 7fa0a5600640 1 -- 172.21.15.59:0/1098325139 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa0a80700b0 con 0x7fa08806c920 2024-04-24T06:42:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T06:42:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.076+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 msgr2=0x7fa08806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.076+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 0x7fa08806ede0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa09c033000 tx=0x7fa09c02e000 comp rx=0 tx=0).stop 2024-04-24T06:42:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.076+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 msgr2=0x7fa0a81575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.076+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fa09400e7c0 tx=0x7fa09400ec80 comp rx=0 tx=0).stop 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 shutdown_connections 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fa08806c920 0x7fa08806ede0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0a814f6b0 0x7fa0a8150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0a8154a30 0x7fa0a81575b0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 --2- 172.21.15.59:0/1098325139 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fa0900039b0 0x7fa0a8145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 >> 172.21.15.59:0/1098325139 conn(0x7fa0a80b2130 msgr2=0x7fa0a80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 shutdown_connections 2024-04-24T06:42:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.077+0000 7fa0b0000640 1 -- 172.21.15.59:0/1098325139 wait complete. 2024-04-24T06:42:18.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:42:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.294+0000 7fd779c00640 1 Processor -- start 2024-04-24T06:42:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.294+0000 7fd779c00640 1 -- start start 2024-04-24T06:42:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd779c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 0x7fd7740cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd779c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd774160210 con 0x7fd7740cd590 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd779c00640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7741604f0 con 0x7fd7740b4860 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd779c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd7740cd9f0 con 0x7fd77415c3d0 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd773400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd7740cd590 0x7fd7740b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39388/0 (socket says 172.21.15.59:39388) 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.295+0000 7fd773400640 1 -- 172.21.15.59:0/2991691623 learned_addr learned my addr 172.21.15.59:0/2991691623 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd773e00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 0x7fd7740cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359070103 0 0) 0x7fd774160210 con 0x7fd7740cd590 2024-04-24T06:42:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd754003520 con 0x7fd7740cd590 2024-04-24T06:42:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4036780509 0 0) 0x7fd754003520 con 0x7fd7740cd590 2024-04-24T06:42:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 msgr2=0x7fd7740cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 0x7fd7740cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 >> v1:172.21.15.66:6789/0 conn(0x7fd7740b4860 legacy=0x7fd7740cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7741607d0 con 0x7fd7740cd590 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2936116336 0 0) 0x7fd75c0019f0 con 0x7fd7740cd590 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 >> v1:172.21.15.59:6789/0 conn(0x7fd7740cd590 legacy=0x7fd7740b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 0x7fd7540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754003f60 0x7fd754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd754004d40 0x7fd7540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd75400ba70 con 0x7fd754004d40 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd75400bc20 con 0x7fd754003f60 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.296+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd75400be20 con 0x7fd77415c3d0 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd754004d40 0x7fd7540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd773400640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754003f60 0x7fd754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 -- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 msgr2=0x7fd7540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd77415c3d0 0x7fd7540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 -- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754003f60 msgr2=0x7fd754004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754003f60 0x7fd754004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.297+0000 7fd779200640 1 -- 172.21.15.59:0/2991691623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd754003520 con 0x7fd754004d40 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.298+0000 7fd779200640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd754004d40 0x7fd7540051c0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fd76c001750 tx=0x7fd754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5550d88f88316617 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c001370 con 0x7fd754004d40 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76c0052d0 con 0x7fd754004d40 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd772a00640 1 -- 172.21.15.59:0/2991691623 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c02abe0 con 0x7fd754004d40 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd779c00640 1 -- 172.21.15.59:0/2991691623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd754004d40 msgr2=0x7fd7540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd779c00640 1 --2- 172.21.15.59:0/2991691623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd754004d40 0x7fd7540051c0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fd76c001750 tx=0x7fd754005830 comp rx=0 tx=0).stop 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd779200640 1 -- 172.21.15.59:0/2991691623 reap_dead start 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd779c00640 1 -- 172.21.15.59:0/2991691623 shutdown_connections 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.299+0000 7fd779c00640 1 -- 172.21.15.59:0/2991691623 >> 172.21.15.59:0/2991691623 conn(0x7fd7740b21a0 msgr2=0x7fd7740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.300+0000 7fd779c00640 1 -- 172.21.15.59:0/2991691623 shutdown_connections 2024-04-24T06:42:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.300+0000 7fd779c00640 1 -- 172.21.15.59:0/2991691623 wait complete. 2024-04-24T06:42:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.301+0000 7fd779c00640 1 Processor -- start 2024-04-24T06:42:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.301+0000 7fd779c00640 1 -- start start 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.302+0000 7fd779c00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754004d40 0x7fd77415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.302+0000 7fd779c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.302+0000 7fd779c00640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7740ae1e0 0x7fd7740ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.302+0000 7fd779c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd77415c7b0 con 0x7fd77415dc30 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.302+0000 7fd779c00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd774160d90 con 0x7fd754004d40 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd779c00640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fd77415c930 con 0x7fd7740ae1e0 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47638/0 (socket says 172.21.15.59:47638) 2024-04-24T06:42:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 -- 172.21.15.59:0/2944236691 learned_addr learned my addr 172.21.15.59:0/2944236691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773e00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754004d40 0x7fd77415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 -- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7740ae1e0 msgr2=0x7fd7740ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7740ae1e0 0x7fd7740ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 -- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754004d40 msgr2=0x7fd77415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754004d40 0x7fd77415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.303+0000 7fd773400640 1 -- 172.21.15.59:0/2944236691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77415ac20 con 0x7fd77415dc30 2024-04-24T06:42:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd773400640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fd75c007b60 tx=0x7fd75c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c013420 con 0x7fd77415dc30 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c018070 con 0x7fd77415dc30 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c013820 con 0x7fd77415dc30 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7740c9760 con 0x7fd77415dc30 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.304+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7740c9d20 con 0x7fd77415dc30 2024-04-24T06:42:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.305+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7740aeba0 con 0x7fd77415dc30 2024-04-24T06:42:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.307+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd75c0139c0 con 0x7fd77415dc30 2024-04-24T06:42:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.308+0000 7fd76be00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 0x7fd75006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.308+0000 7fd773e00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 0x7fd75006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.309+0000 7fd773e00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 0x7fd75006ede0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd764004770 tx=0x7fd764008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.309+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fd75c08a870 con 0x7fd77415dc30 2024-04-24T06:42:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.314+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd77415ac20 con 0x7fd77415dc30 2024-04-24T06:42:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.513+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd77404fd90 con 0x7fd75006c920 2024-04-24T06:42:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.515+0000 7fd76be00640 1 -- 172.21.15.59:0/2944236691 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7fd77404fd90 con 0x7fd75006c920 2024-04-24T06:42:18.516 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 msgr2=0x7fd75006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 0x7fd75006ede0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd764004770 tx=0x7fd764008040 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 msgr2=0x7fd774073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fd75c007b60 tx=0x7fd75c00e530 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 shutdown_connections 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fd75006c920 0x7fd75006ede0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fd7740ae1e0 0x7fd7740ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd77415dc30 0x7fd774073510 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 --2- 172.21.15.59:0/2944236691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd754004d40 0x7fd77415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 >> 172.21.15.59:0/2944236691 conn(0x7fd7740b21a0 msgr2=0x7fd774074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 shutdown_connections 2024-04-24T06:42:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:18.518+0000 7fd779c00640 1 -- 172.21.15.59:0/2944236691 wait complete. 2024-04-24T06:42:18.537 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-04-24T06:42:17.476548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262028,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712692,"statfs":{"total":773094113280,"available":772825796608,"internally_reserved":0,"allocated":12886016,"data_stored":6273944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.239525"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":348,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-24T06:42:07.377917+0000","last_change":"2024-04-24T06:28:34.818598+0000","last_active":"2024-04-24T06:42:07.377917+0000","last_peered":"2024-04-24T06:42:07.377917+0000","last_clean":"2024-04-24T06:42:07.377917+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:42:07.377917+0000","last_undegraded":"2024-04-24T06:42:07.377917+0000","last_fullsized":"2024-04-24T06:42:07.377917+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:28:33.590103+0000","last_clean_scrub_stamp":"2024-04-24T06:28:33.590103+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:17.175800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:42:18.539 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T06:28:33.590103+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=6, tm_min=28, tm_sec=33, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=6, tm_min=42, tm_sec=11, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T06:42:18.539 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T06:42:38.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:42:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.764+0000 7f0266600640 1 Processor -- start 2024-04-24T06:42:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.764+0000 7f0266600640 1 -- start start 2024-04-24T06:42:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0266600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f026015c3d0 0x7f02600cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0266600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0260160210 con 0x7f02600b4860 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0266600640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02601604f0 con 0x7f02600cd590 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0266600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02601607d0 con 0x7f026015c3d0 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0265200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f026015c3d0 0x7f02600cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0265c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f02600b4860 0x7f02600cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48374/0 (socket says 172.21.15.59:48374) 2024-04-24T06:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.766+0000 7f0265c00640 1 -- 172.21.15.59:0/1896728336 learned_addr learned my addr 172.21.15.59:0/1896728336 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.767+0000 7f0265200640 1 -- 172.21.15.59:0/1896728336 >> v1:172.21.15.66:6789/0 conn(0x7f02600cd590 legacy=0x7f02600b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.767+0000 7f0265200640 1 -- 172.21.15.59:0/1896728336 >> v1:172.21.15.59:6789/0 conn(0x7f02600b4860 legacy=0x7f02600cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.767+0000 7f0265200640 1 -- 172.21.15.59:0/1896728336 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260160ab0 con 0x7f026015c3d0 2024-04-24T06:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.767+0000 7f025f400640 1 -- 172.21.15.59:0/1896728336 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832980133 0 0) 0x7f0260160210 con 0x7f02600b4860 2024-04-24T06:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.768+0000 7f0265200640 1 --2- 172.21.15.59:0/1896728336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f026015c3d0 0x7f02600cd050 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7eb529df562f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.768+0000 7f025f400640 1 -- 172.21.15.59:0/1896728336 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254037070 con 0x7f026015c3d0 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.768+0000 7f025f400640 1 -- 172.21.15.59:0/1896728336 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0254032440 con 0x7f026015c3d0 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.768+0000 7f025f400640 1 -- 172.21.15.59:0/1896728336 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254032810 con 0x7f026015c3d0 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 -- 172.21.15.59:0/1896728336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f026015c3d0 msgr2=0x7f02600cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 --2- 172.21.15.59:0/1896728336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f026015c3d0 0x7f02600cd050 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402a230 comp rx=0 tx=0).stop 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 -- 172.21.15.59:0/1896728336 shutdown_connections 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 --2- 172.21.15.59:0/1896728336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f026015c3d0 0x7f02600cd050 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 -- 172.21.15.59:0/1896728336 >> 172.21.15.59:0/1896728336 conn(0x7f02600b21a0 msgr2=0x7f02600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 -- 172.21.15.59:0/1896728336 shutdown_connections 2024-04-24T06:42:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.769+0000 7f0266600640 1 -- 172.21.15.59:0/1896728336 wait complete. 2024-04-24T06:42:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.771+0000 7f0266600640 1 Processor -- start 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.771+0000 7f0266600640 1 -- start start 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02600cd590 0x7f026015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 0x7f02601f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f026015c930 con 0x7f02600b4860 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f026015c7b0 con 0x7f02600cd590 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0266600640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7f02600cbee0 con 0x7f026015c3d0 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.772+0000 7f0265200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60680/0 (socket says 172.21.15.59:60680) 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 -- 172.21.15.59:0/3651353659 learned_addr learned my addr 172.21.15.59:0/3651353659 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f025fe00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02600cd590 0x7f026015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265c00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 0x7f02601f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 -- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 msgr2=0x7f02601f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 0x7f02601f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 -- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02600cd590 msgr2=0x7f026015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02600cd590 0x7f026015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 -- 172.21.15.59:0/3651353659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02601f21d0 con 0x7f02600b4860 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265c00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 0x7f02601f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.773+0000 7f0265200640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f02540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.774+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254040020 con 0x7f02600b4860 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.774+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0254038040 con 0x7f02600b4860 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.774+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254037070 con 0x7f02600b4860 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.774+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02600c9dc0 con 0x7f02600b4860 2024-04-24T06:42:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.774+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601f2450 con 0x7f02600b4860 2024-04-24T06:42:38.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.775+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026004fd90 con 0x7f02600b4860 2024-04-24T06:42:38.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.776+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0254043020 con 0x7f02600b4860 2024-04-24T06:42:38.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.778+0000 7f025cc00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 0x7f024006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:38.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.778+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7f02540abfe0 con 0x7f02600b4860 2024-04-24T06:42:38.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.778+0000 7f025fe00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 0x7f024006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:38.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.779+0000 7f025fe00640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 0x7f024006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f026015f5d0 tx=0x7f024c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:38.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.783+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f025404a030 con 0x7f02600b4860 2024-04-24T06:42:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.980+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 --> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02600ca2e0 con 0x7f024006c8f0 2024-04-24T06:42:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.982+0000 7f025cc00640 1 -- 172.21.15.59:0/3651353659 <== mgr.4106 v2:172.21.15.59:6832/4266715280 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13380 (secure 0 0 0) 0x7f024c001c70 con 0x7f024006c8f0 2024-04-24T06:42:38.984 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-24T06:42:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 msgr2=0x7f024006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 0x7f024006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f026015f5d0 tx=0x7f024c009040 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 msgr2=0x7f02601f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f02540046c0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 shutdown_connections 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7f024006c8f0 0x7f024006edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7f026015c3d0 0x7f02601f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02600cd590 0x7f026015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 --2- 172.21.15.59:0/3651353659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02600b4860 0x7f02601f6320 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.985+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 >> 172.21.15.59:0/3651353659 conn(0x7f02600b21a0 msgr2=0x7f0260074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.986+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 shutdown_connections 2024-04-24T06:42:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:38.986+0000 7f0266600640 1 -- 172.21.15.59:0/3651353659 wait complete. 2024-04-24T06:42:39.007 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-04-24T06:42:37.482173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262016,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712704,"statfs":{"total":773094113280,"available":772825808896,"internally_reserved":0,"allocated":12886016,"data_stored":6273944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"1.0","version":"117'119","reported_seq":359,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-24T06:42:18.244338+0000","last_change":"2024-04-24T06:42:18.244436+0000","last_active":"2024-04-24T06:42:18.244338+0000","last_peered":"2024-04-24T06:42:18.244338+0000","last_clean":"2024-04-24T06:42:18.244338+0000","last_became_active":"2024-04-24T06:28:34.810974+0000","last_became_peered":"2024-04-24T06:28:34.810974+0000","last_unstale":"2024-04-24T06:42:18.244338+0000","last_undegraded":"2024-04-24T06:42:18.244338+0000","last_fullsized":"2024-04-24T06:42:18.244338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'119","last_scrub_stamp":"2024-04-24T06:42:18.244138+0000","last_deep_scrub":"117'119","last_deep_scrub_stamp":"2024-04-24T06:42:18.244138+0000","last_clean_scrub_stamp":"2024-04-24T06:42:18.244138+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:13:41.053379+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":636651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2056192,"data_stored":1227019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:42:39.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T06:42:39.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:39.139 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.205+0000 7fcf1b200640 1 Processor -- start 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.205+0000 7fcf1b200640 1 -- start start 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf1b200640 1 --2- >> v2:172.21.15.66:3300/0 conn(0x7fcf1c155a80 0x7fcf1c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0af2a0 0x7fcf1c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf1b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c15ef20 con 0x7fcf1c0d0600 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf1b200640 1 -- --> v2:172.21.15.66:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c15f1c0 con 0x7fcf1c155a80 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf1b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c15f4a0 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf11400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcf1c0d0600 0x7fcf1c0b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49146/0 (socket says 172.21.15.59:49146) 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.207+0000 7fcf11400640 1 -- 172.21.15.59:0/908637779 learned_addr learned my addr 172.21.15.59:0/908637779 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19e00640 1 --2- 172.21.15.59:0/908637779 >> v2:172.21.15.66:3300/0 conn(0x7fcf1c155a80 0x7fcf1c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf1a800640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0af2a0 0x7fcf1c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19400640 1 -- 172.21.15.59:0/908637779 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529930331 0 0) 0x7fcf1c15ef20 con 0x7fcf1c0d0600 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19400640 1 -- 172.21.15.59:0/908637779 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf00003520 con 0x7fcf1c0d0600 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19e00640 1 -- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0af2a0 msgr2=0x7fcf1c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19e00640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0af2a0 0x7fcf1c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19e00640 1 -- 172.21.15.59:0/908637779 >> v1:172.21.15.59:6789/0 conn(0x7fcf1c0d0600 legacy=0x7fcf1c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.208+0000 7fcf19e00640 1 -- 172.21.15.59:0/908637779 --> v2:172.21.15.66:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c15f780 con 0x7fcf1c155a80 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.210+0000 7fcf19e00640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c06f9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d90 tx=0x7fcf0c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72c96c8240fad13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.210+0000 7fcf19400640 1 -- 172.21.15.59:0/908637779 <== mon.1 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007740 con 0x7fcf1c155a80 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.210+0000 7fcf19400640 1 -- 172.21.15.59:0/908637779 <== mon.1 v2:172.21.15.66:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0078e0 con 0x7fcf1c155a80 2024-04-24T06:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.210+0000 7fcf19400640 1 -- 172.21.15.59:0/908637779 <== mon.1 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007cb0 con 0x7fcf1c155a80 2024-04-24T06:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 -- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 msgr2=0x7fcf1c06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c06f9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d90 tx=0x7fcf0c02a660 comp rx=0 tx=0).stop 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 -- 172.21.15.59:0/908637779 shutdown_connections 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0af2a0 0x7fcf1c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 --2- 172.21.15.59:0/908637779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c06f9c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 -- 172.21.15.59:0/908637779 >> 172.21.15.59:0/908637779 conn(0x7fcf1c06bdf0 msgr2=0x7fcf1c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 -- 172.21.15.59:0/908637779 shutdown_connections 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.211+0000 7fcf1b200640 1 -- 172.21.15.59:0/908637779 wait complete. 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.213+0000 7fcf1b200640 1 Processor -- start 2024-04-24T06:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.213+0000 7fcf1b200640 1 -- start start 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 0x7fcf1c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c06ff00 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c15fd40 con 0x7fcf1c155a80 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf1b200640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c06ef80 con 0x7fcf1c0d0600 2024-04-24T06:42:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf19e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf19e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40070/0 (socket says 172.21.15.59:40070) 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf19e00640 1 -- 172.21.15.59:0/1422327739 learned_addr learned my addr 172.21.15.59:0/1422327739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.214+0000 7fcf11400640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 0x7fcf1c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 -- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 msgr2=0x7fcf1c0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf1a800640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 0x7fcf1c0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 -- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 msgr2=0x7fcf1c14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 -- 172.21.15.59:0/1422327739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0ccf30 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf11400640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 0x7fcf1c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf1a800640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf19e00640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d90 tx=0x7fcf0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c03f070 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c031bc0 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c15e440 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.215+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c044460 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.216+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c15d960 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.217+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c04fd90 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.218+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf0c044600 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.219+0000 7fcf12a00640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 0x7fcef806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.219+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fcf0c0acff0 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.219+0000 7fcf11400640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 0x7fcef806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.220+0000 7fcf11400640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 0x7fcef806edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0b00e0 tx=0x7fcf04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.223+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf0c07b680 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.429+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fcf1c06eba0 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.437+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7fcf0c07af80 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.439+0000 7fcf12a00640 1 -- 172.21.15.59:0/1422327739 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcf0c07f330 con 0x7fcf1c0af2a0 2024-04-24T06:42:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.441+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 msgr2=0x7fcef806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.441+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 0x7fcef806edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0b00e0 tx=0x7fcf04009040 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.441+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 msgr2=0x7fcf1c0b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d90 tx=0x7fcf0c004a40 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 shutdown_connections 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fcef806c8f0 0x7fcef806edb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf1c155a80 0x7fcf1c14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fcf1c0d0600 0x7fcf1c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 --2- 172.21.15.59:0/1422327739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1c0af2a0 0x7fcf1c0b0290 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 >> 172.21.15.59:0/1422327739 conn(0x7fcf1c06bdf0 msgr2=0x7fcf1c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 shutdown_connections 2024-04-24T06:42:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.442+0000 7fcf1b200640 1 -- 172.21.15.59:0/1422327739 wait complete. 2024-04-24T06:42:39.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:42:39.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:42:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.667+0000 7fdcf6000640 1 Processor -- start 2024-04-24T06:42:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.667+0000 7fdcf6000640 1 -- start start 2024-04-24T06:42:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.669+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.669+0000 7fdcf6000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00cd9f0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.669+0000 7fdcf6000640 1 -- --> v1:172.21.15.66:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00cdcb0 con 0x7fdcf015e200 2024-04-24T06:42:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.669+0000 7fdcf6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf009b6f0 con 0x7fdcf00d0370 2024-04-24T06:42:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.670+0000 7fdcef400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdcf00d0b70 0x7fdcf00d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49154/0 (socket says 172.21.15.59:49154) 2024-04-24T06:42:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.670+0000 7fdcef400640 1 -- 172.21.15.59:0/4171492709 learned_addr learned my addr 172.21.15.59:0/4171492709 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-24T06:42:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.670+0000 7fdce7e00640 1 --2- 172.21.15.59:0/4171492709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.670+0000 7fdceea00640 1 -- 172.21.15.59:0/4171492709 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589018356 0 0) 0x7fdcf00cd9f0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.671+0000 7fdceea00640 1 -- 172.21.15.59:0/4171492709 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcd4003520 con 0x7fdcf00d0b70 2024-04-24T06:42:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.671+0000 7fdce7e00640 1 -- 172.21.15.59:0/4171492709 >> v1:172.21.15.66:6789/0 conn(0x7fdcf015e200 legacy=0x7fdcf015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.671+0000 7fdce7e00640 1 -- 172.21.15.59:0/4171492709 >> v1:172.21.15.59:6789/0 conn(0x7fdcf00d0b70 legacy=0x7fdcf00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.671+0000 7fdce7e00640 1 -- 172.21.15.59:0/4171492709 --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00cdf90 con 0x7fdcf00d0370 2024-04-24T06:42:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.672+0000 7fdce7e00640 1 --2- 172.21.15.59:0/4171492709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf00cd250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097d0 tx=0x7fdcd802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661c2f94f080b10e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.673+0000 7fdceea00640 1 -- 172.21.15.59:0/4171492709 <== mon.2 v2:172.21.15.66:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037070 con 0x7fdcf00d0370 2024-04-24T06:42:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.673+0000 7fdceea00640 1 -- 172.21.15.59:0/4171492709 <== mon.2 v2:172.21.15.66:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdcd8032440 con 0x7fdcf00d0370 2024-04-24T06:42:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.673+0000 7fdceea00640 1 -- 172.21.15.59:0/4171492709 <== mon.2 v2:172.21.15.66:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8032830 con 0x7fdcf00d0370 2024-04-24T06:42:39.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.673+0000 7fdcf6000640 1 -- 172.21.15.59:0/4171492709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 msgr2=0x7fdcf00cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.673+0000 7fdcf6000640 1 --2- 172.21.15.59:0/4171492709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf00cd250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097d0 tx=0x7fdcd802a180 comp rx=0 tx=0).stop 2024-04-24T06:42:39.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.674+0000 7fdcf6000640 1 -- 172.21.15.59:0/4171492709 shutdown_connections 2024-04-24T06:42:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.674+0000 7fdcf6000640 1 --2- 172.21.15.59:0/4171492709 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf00cd250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.674+0000 7fdcf6000640 1 -- 172.21.15.59:0/4171492709 >> 172.21.15.59:0/4171492709 conn(0x7fdcf00b4080 msgr2=0x7fdcf00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.674+0000 7fdcf6000640 1 -- 172.21.15.59:0/4171492709 shutdown_connections 2024-04-24T06:42:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.674+0000 7fdcf6000640 1 -- 172.21.15.59:0/4171492709 wait complete. 2024-04-24T06:42:39.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.676+0000 7fdcf6000640 1 Processor -- start 2024-04-24T06:42:39.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.676+0000 7fdcf6000640 1 -- start start 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.677+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf015e200 0x7fdcf00cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcf6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00cd790 con 0x7fdcf00d0b70 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcf6000640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00d07b0 con 0x7fdcf015e200 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcf6000640 1 -- --> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf015eda0 con 0x7fdcf00d0370 2024-04-24T06:42:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40082/0 (socket says 172.21.15.59:40082) 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 -- 172.21.15.59:0/2949213193 learned_addr learned my addr 172.21.15.59:0/2949213193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcef400640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdcefe00640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf015e200 0x7fdcf00cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 -- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 msgr2=0x7fdcf014c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf014c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 -- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf015e200 msgr2=0x7fdcf00cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.678+0000 7fdce7e00640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf015e200 0x7fdcf00cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdce7e00640 1 -- 172.21.15.59:0/2949213193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00cc8e0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdce7e00640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8035040 tx=0x7fdcd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8041030 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdcd8030540 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037040 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.679+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf00c72b0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.680+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf00c9900 con 0x7fdcf00d0b70 2024-04-24T06:42:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.681+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0068a80 con 0x7fdcf00d0b70 2024-04-24T06:42:39.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.682+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcd8044070 con 0x7fdcf00d0b70 2024-04-24T06:42:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.683+0000 7fdce7400640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 0x7fdccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.683+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5115+0+0 (secure 0 0 0) 0x7fdcd80ac270 con 0x7fdcf00d0b70 2024-04-24T06:42:39.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.683+0000 7fdcef400640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 0x7fdccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:39.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.684+0000 7fdcef400640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 0x7fdccc06edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdce00011e0 tx=0x7fdce0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:39.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.687+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcd807a7c0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.934+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcf00b3e60 con 0x7fdcf00d0b70 2024-04-24T06:42:39.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.935+0000 7fdce7400640 1 -- 172.21.15.59:0/2949213193 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3860 (secure 0 0 0) 0x7fdcd807a0c0 con 0x7fdcf00d0b70 2024-04-24T06:42:39.937 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-24T06:42:39.937 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/2356304127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2356304127},{"type":"v1","addr":"172.21.15.66:6834","nonce":2356304127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6850,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/72588431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":72588431},{"type":"v1","addr":"172.21.15.66:6835","nonce":72588431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/475578955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":475578955},{"type":"v1","addr":"172.21.15.59:6836","nonce":475578955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/917105371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":917105371},{"type":"v1","addr":"172.21.15.59:6837","nonce":917105371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[]} 2024-04-24T06:42:39.937 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 116 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 msgr2=0x7fdccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 0x7fdccc06edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdce00011e0 tx=0x7fdce0009040 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 msgr2=0x7fdcf014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8035040 tx=0x7fdcd80046c0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 shutdown_connections 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:6832/4266715280,v1:172.21.15.59:6833/4266715280] conn(0x7fdccc06c8f0 0x7fdccc06edb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf015e200 0x7fdcf00cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcf00d0b70 0x7fdcf014cca0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 --2- 172.21.15.59:0/2949213193 >> [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] conn(0x7fdcf00d0370 0x7fdcf014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 >> 172.21.15.59:0/2949213193 conn(0x7fdcf00b4080 msgr2=0x7fdcf015d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 shutdown_connections 2024-04-24T06:42:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-24T06:42:39.938+0000 7fdcf6000640 1 -- 172.21.15.59:0/2949213193 wait complete. 2024-04-24T06:42:39.959 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-24T06:42:39.960 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T06:42:39.960 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T06:42:39.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:39.979 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T06:42:39.979 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T06:42:39.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.041 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T06:42:40.042 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T06:42:40.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.069 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T06:42:40.069 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T06:42:40.069 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.136 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T06:42:40.136 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T06:42:40.137 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T06:42:40.137 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.203 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T06:42:40.203 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T06:42:40.203 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.277 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T06:42:40.277 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T06:42:40.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.360 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T06:42:40.361 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T06:42:40.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.439 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T06:42:40.440 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T06:42:40.440 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.515 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T06:42:40.515 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T06:42:40.515 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.604 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T06:42:40.605 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T06:42:40.605 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.671 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T06:42:40.671 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T06:42:40.671 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.739 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T06:42:40.739 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T06:42:40.740 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T06:42:40.740 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.825 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T06:42:40.825 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T06:42:40.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.897 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T06:42:40.897 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T06:42:40.897 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T06:42:40.897 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.950 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T06:42:40.951 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T06:42:40.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:40.985 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T06:42:40.985 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T06:42:40.985 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T06:42:41.056 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T06:42:41.056 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T06:42:41.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-24T06:42:41.167 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:34:15.258911+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-24T06:42:41.168 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-24T06:42:41.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-24T06:42:41.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-24T06:42:41.345 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-24T06:36:25.066126+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-24T06:42:41.346 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:42:41.346 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T06:42:41.582 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:42:41.583 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T06:42:41.764 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:42:41.764 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T06:42:41.903 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:42:41.903 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T06:42:42.078 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:42:42.078 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T06:42:42.346 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:42:42.346 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T06:42:42.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:42:42.477 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T06:42:42.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:42:42.619 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T06:42:42.797 INFO:tasks.ceph:Archiving mon data... 2024-04-24T06:42:42.799 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/data/mon.a.tgz 2024-04-24T06:42:42.799 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-04-24T06:42:42.859 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.dnXqVYghKc 2024-04-24T06:42:42.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.dnXqVYghKc 2024-04-24T06:42:43.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.dnXqVYghKc 2024-04-24T06:42:43.202 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.dnXqVYghKc is 403KB 2024-04-24T06:42:43.313 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.dnXqVYghKc 2024-04-24T06:42:43.373 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/data/mon.b.tgz 2024-04-24T06:42:43.373 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-04-24T06:42:43.432 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.Px2R6go6Ni 2024-04-24T06:42:43.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Px2R6go6Ni 2024-04-24T06:42:43.627 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.Px2R6go6Ni 2024-04-24T06:42:43.775 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.Px2R6go6Ni is 404KB 2024-04-24T06:42:43.886 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.Px2R6go6Ni 2024-04-24T06:42:43.946 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/data/mon.c.tgz 2024-04-24T06:42:43.946 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-04-24T06:42:44.002 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.xpU2QJ3UYt 2024-04-24T06:42:44.002 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xpU2QJ3UYt 2024-04-24T06:42:44.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.xpU2QJ3UYt 2024-04-24T06:42:44.341 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.xpU2QJ3UYt is 404KB 2024-04-24T06:42:44.412 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.xpU2QJ3UYt 2024-04-24T06:42:44.469 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T06:42:44.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T06:42:44.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T06:42:44.620 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T06:42:44.679 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T06:42:44.679 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T06:42:44.681 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi059/crash 2024-04-24T06:42:44.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T06:42:44.774 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi066/crash 2024-04-24T06:42:44.775 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T06:42:44.866 INFO:tasks.ceph:Shutting down logrotate 2024-04-24T06:42:44.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:44.874 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:44.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:45.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:45.180 INFO:tasks.ceph:Compressing logs... 2024-04-24T06:42:45.231 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T06:42:45.239 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T06:42:45.321 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66780.log 2024-04-24T06:42:45.322 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62000.log 2024-04-24T06:42:45.323 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.66780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72951.log 2024-04-24T06:42:45.324 INFO:teuthology.orchestra.run.smithi059.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.66780.log.gz 2024-04-24T06:42:45.324 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56744.log 2024-04-24T06:42:45.325 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62000.log.gz 2024-04-24T06:42:45.325 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73599.log 2024-04-24T06:42:45.326 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56744.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72951.log.gz 2024-04-24T06:42:45.327 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56744.log.gz 2024-04-24T06:42:45.327 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62712.log 2024-04-24T06:42:45.328 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39552.log 2024-04-24T06:42:45.328 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81053.log 2024-04-24T06:42:45.328 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73599.log.gz 2024-04-24T06:42:45.329 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T06:42:45.330 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70306.log 2024-04-24T06:42:45.330 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62712.log.gz 2024-04-24T06:42:45.330 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81053.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81053.log.gz -5 2024-04-24T06:42:45.330 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70390.log 2024-04-24T06:42:45.331 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.3.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39096.log 2024-04-24T06:42:45.332 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.log: 91.4% -- replaced with /var/log/ceph/ceph-client.3.39552.log.gz 2024-04-24T06:42:45.332 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33506.log 2024-04-24T06:42:45.333 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-04-24T06:42:45.333 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70306.log.gz 2024-04-24T06:42:45.333 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71898.log 2024-04-24T06:42:45.333 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70390.log.gz 2024-04-24T06:42:45.336 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80580.log 2024-04-24T06:42:45.336 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-04-24T06:42:45.336 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-04-24T06:42:45.336 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.71898.log.gz 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.1.39096.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33506.log: 90.8% -- replaced with /var/log/ceph/ceph-client.1.39096.log.gz 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33506.log.gz 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33466.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-24T06:42:45.337 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-24T06:42:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T06:42:45.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T06:42:45.340 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59701.log 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80580.log.gz 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.46919.log -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38402.log 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59701.log.gz 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46751.log 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38402.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38402.log.gz -- 2024-04-24T06:42:45.341 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.47027.log 2024-04-24T06:42:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33346.log 2024-04-24T06:42:45.343 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T06:42:45.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T06:42:45.344 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46751.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46751.log.gz 2024-04-24T06:42:45.344 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-04-24T06:42:45.344 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71810.log 2024-04-24T06:42:45.344 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2024-04-24T06:42:45.344 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-04-24T06:42:45.345 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-04-24T06:42:45.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33346.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T06:42:45.346 INFO:teuthology.orchestra.run.smithi066.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33346.log.gz 2024-04-24T06:42:45.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67643.log 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.71810.log.gz 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56147.log 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69118.log 2024-04-24T06:42:45.347 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67643.log.gz 2024-04-24T06:42:45.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-04-24T06:42:45.349 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56147.log.gz 2024-04-24T06:42:45.349 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58594.log 2024-04-24T06:42:45.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33426.log 2024-04-24T06:42:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71513.log 2024-04-24T06:42:45.350 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69118.log.gz 2024-04-24T06:42:45.350 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33756.log 2024-04-24T06:42:45.351 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58594.log.gz 2024-04-24T06:42:45.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-04-24T06:42:45.352 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-04-24T06:42:45.352 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71513.log.gz 2024-04-24T06:42:45.353 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33756.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33756.log.gz 2024-04-24T06:42:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33426.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33426.log.gz 2024-04-24T06:42:45.355 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79146.log 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45642.log 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61905.log 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79146.log.gz 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45642.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45642.log.gz 2024-04-24T06:42:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61508.log 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48017.log 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61905.log.gz 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56614.log 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61508.log.gz 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48017.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48017.log.gz 2024-04-24T06:42:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77031.log 2024-04-24T06:42:45.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33386.log 2024-04-24T06:42:45.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.33386.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2024-04-24T06:42:45.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33306.log 2024-04-24T06:42:45.362 INFO:teuthology.orchestra.run.smithi066.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T06:42:45.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44540.log 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56614.log.gz 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71471.log 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.77031.log.gz 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59430.log 2024-04-24T06:42:45.364 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44540.log.gz 2024-04-24T06:42:45.365 INFO:teuthology.orchestra.run.smithi066.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33386.log.gz 2024-04-24T06:42:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71471.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71471.log.gz 2024-04-24T06:42:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64176.log 2024-04-24T06:42:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59616.log 2024-04-24T06:42:45.366 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59430.log.gz 2024-04-24T06:42:45.367 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.33306.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33306.log.gz 2024-04-24T06:42:45.367 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-24T06:42:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54293.log 2024-04-24T06:42:45.368 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64176.log.gz 2024-04-24T06:42:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44253.log 2024-04-24T06:42:45.369 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59616.log.gz 2024-04-24T06:42:45.370 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61340.log 2024-04-24T06:42:45.370 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44253.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54293.log.gz 2024-04-24T06:42:45.370 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80707.log 2024-04-24T06:42:45.371 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44253.log.gz 2024-04-24T06:42:45.371 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39632.log 2024-04-24T06:42:45.372 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61340.log.gz 2024-04-24T06:42:45.372 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71597.log 2024-04-24T06:42:45.373 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80707.log.gz 2024-04-24T06:42:45.374 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73413.log 2024-04-24T06:42:45.374 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39632.log.gz 2024-04-24T06:42:45.375 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68021.log 2024-04-24T06:42:45.375 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71597.log.gz 2024-04-24T06:42:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57776.log 2024-04-24T06:42:45.376 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73413.log.gz 2024-04-24T06:42:45.377 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46351.log 2024-04-24T06:42:45.377 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68021.log.gz 2024-04-24T06:42:45.378 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57776.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57776.log.gz 2024-04-24T06:42:45.378 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-24T06:42:45.379 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46309.log 2024-04-24T06:42:45.380 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46351.log.gz 2024-04-24T06:42:45.381 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38774.log 2024-04-24T06:42:45.381 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-04-24T06:42:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42735.log 2024-04-24T06:42:45.382 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2024-04-24T06:42:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2024-04-24T06:42:45.383 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38774.log.gz 2024-04-24T06:42:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56786.log 2024-04-24T06:42:45.384 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42735.log.gz 2024-04-24T06:42:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52223.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2024-04-24T06:42:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46877.log 2024-04-24T06:42:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74408.log 2024-04-24T06:42:45.387 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56786.log.gz 2024-04-24T06:42:45.388 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60067.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46877.log.gz 2024-04-24T06:42:45.389 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74408.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74408.log.gz 2024-04-24T06:42:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79058.log 2024-04-24T06:42:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58002.log 2024-04-24T06:42:45.391 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60067.log.gz 2024-04-24T06:42:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81523.log 2024-04-24T06:42:45.392 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79058.log.gz 2024-04-24T06:42:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70080.log 2024-04-24T06:42:45.393 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58002.log.gz 2024-04-24T06:42:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2024-04-24T06:42:45.394 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81523.log.gz 2024-04-24T06:42:45.395 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61821.log 2024-04-24T06:42:45.395 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70080.log.gz 2024-04-24T06:42:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52265.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-04-24T06:42:45.397 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69033.log 2024-04-24T06:42:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-04-24T06:42:45.398 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61821.log.gz 2024-04-24T06:42:45.399 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60833.log 2024-04-24T06:42:45.399 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69033.log.gz 2024-04-24T06:42:45.400 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73497.log 2024-04-24T06:42:45.400 INFO:teuthology.orchestra.run.smithi059.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-04-24T06:42:45.401 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40255.log 2024-04-24T06:42:45.402 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60833.log.gz 2024-04-24T06:42:45.403 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73981.log 2024-04-24T06:42:45.403 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73497.log.gz 2024-04-24T06:42:45.404 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57818.log 2024-04-24T06:42:45.404 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40255.log.gz 2024-04-24T06:42:45.405 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81008.log 2024-04-24T06:42:45.405 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73981.log.gz 2024-04-24T06:42:45.407 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39960.log 2024-04-24T06:42:45.407 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57818.log.gz 2024-04-24T06:42:45.408 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81008.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81008.log.gz /var/log/ceph/ceph-client.admin.73161.log 2024-04-24T06:42:45.408 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39960.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39960.log.gz 2024-04-24T06:42:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2024-04-24T06:42:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40301.log 2024-04-24T06:42:45.410 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73161.log.gz 2024-04-24T06:42:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79989.log 2024-04-24T06:42:45.412 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2024-04-24T06:42:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75811.log 2024-04-24T06:42:45.413 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40301.log.gz 2024-04-24T06:42:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81103.log 2024-04-24T06:42:45.414 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79989.log.gz 2024-04-24T06:42:45.415 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60133.log 2024-04-24T06:42:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81103.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75811.log.gz 2024-04-24T06:42:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55395.log 2024-04-24T06:42:45.417 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81103.log.gz 2024-04-24T06:42:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2024-04-24T06:42:45.418 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60133.log.gz 2024-04-24T06:42:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55919.log 2024-04-24T06:42:45.419 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55395.log.gz 2024-04-24T06:42:45.420 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80268.log 2024-04-24T06:42:45.420 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-04-24T06:42:45.422 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65598.log 2024-04-24T06:42:45.422 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55919.log.gz 2024-04-24T06:42:45.423 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80268.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.80268.log.gz 2024-04-24T06:42:45.423 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.43460.log 2024-04-24T06:42:45.424 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65598.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65598.log.gz 2024-04-24T06:42:45.425 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59304.log 2024-04-24T06:42:45.426 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-04-24T06:42:45.426 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-24T06:42:45.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-04-24T06:42:45.427 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59304.log.gz 2024-04-24T06:42:45.428 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43113.log 2024-04-24T06:42:45.429 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-04-24T06:42:45.430 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68376.log 2024-04-24T06:42:45.430 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-04-24T06:42:45.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2024-04-24T06:42:45.431 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43113.log.gz 2024-04-24T06:42:45.433 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67322.log 88.0% 2024-04-24T06:42:45.433 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68376.log.gz 2024-04-24T06:42:45.434 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-04-24T06:42:45.434 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2024-04-24T06:42:45.435 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61424.log 2024-04-24T06:42:45.435 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67322.log.gz 2024-04-24T06:42:45.436 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46961.log 2024-04-24T06:42:45.437 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61424.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-04-24T06:42:45.438 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54141.log 2024-04-24T06:42:45.438 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61424.log.gz 2024-04-24T06:42:45.439 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64873.log 2024-04-24T06:42:45.440 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46961.log.gz 2024-04-24T06:42:45.440 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74660.log 2024-04-24T06:42:45.440 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54141.log.gz 2024-04-24T06:42:45.441 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 2024-04-24T06:42:45.442 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74660.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64873.log.gz 2024-04-24T06:42:45.443 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68593.log 2024-04-24T06:42:45.443 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74660.log.gz 2024-04-24T06:42:45.444 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-04-24T06:42:45.444 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-04-24T06:42:45.445 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-04-24T06:42:45.445 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68593.log.gz 2024-04-24T06:42:45.446 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63590.log 2024-04-24T06:42:45.446 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-04-24T06:42:45.447 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39114.log 2024-04-24T06:42:45.448 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-04-24T06:42:45.448 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45600.log 2024-04-24T06:42:45.448 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63590.log.gz 2024-04-24T06:42:45.449 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69286.log 2024-04-24T06:42:45.449 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39114.log.gz 2024-04-24T06:42:45.450 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45600.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T06:42:45.451 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45600.log.gz 2024-04-24T06:42:45.452 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64056.log 2024-04-24T06:42:45.452 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69286.log.gz 2024-04-24T06:42:45.453 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-04-24T06:42:45.456 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53439.log 2024-04-24T06:42:45.456 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64056.log.gz 2024-04-24T06:42:45.456 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67448.log 2024-04-24T06:42:45.456 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T06:42:45.456 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-04-24T06:42:45.457 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68843.log 2024-04-24T06:42:45.457 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53439.log.gz 2024-04-24T06:42:45.458 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67448.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67448.log.gz 2024-04-24T06:42:45.458 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79490.log 2024-04-24T06:42:45.459 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-04-24T06:42:45.459 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68843.log.gz 2024-04-24T06:42:45.460 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.79490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76757.log 2024-04-24T06:42:45.461 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56487.log 2024-04-24T06:42:45.461 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-04-24T06:42:45.462 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-04-24T06:42:45.462 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76757.log.gz 2024-04-24T06:42:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65740.log 2024-04-24T06:42:45.463 INFO:teuthology.orchestra.run.smithi059.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.79490.log.gz 2024-04-24T06:42:45.464 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56487.log.gz 2024-04-24T06:42:45.464 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56284.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-04-24T06:42:45.465 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75339.log 2024-04-24T06:42:45.466 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53673.log 2024-04-24T06:42:45.466 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65740.log.gz 2024-04-24T06:42:45.467 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-04-24T06:42:45.467 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75339.log.gz 2024-04-24T06:42:45.468 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78032.log 2024-04-24T06:42:45.468 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53673.log.gz 2024-04-24T06:42:45.469 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62460.log 2024-04-24T06:42:45.469 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-04-24T06:42:45.470 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78032.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78032.log.gz 2024-04-24T06:42:45.470 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55158.log 2024-04-24T06:42:45.471 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-04-24T06:42:45.472 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62460.log.gz 2024-04-24T06:42:45.473 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76429.log 89.2% 2024-04-24T06:42:45.473 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55158.log.gz 2024-04-24T06:42:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73553.log 2024-04-24T06:42:45.474 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-04-24T06:42:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76429.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.76429.log.gz 2024-04-24T06:42:45.475 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62838.log 2024-04-24T06:42:45.476 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59472.log 2024-04-24T06:42:45.476 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73553.log.gz 2024-04-24T06:42:45.477 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-04-24T06:42:45.477 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62838.log.gz 2024-04-24T06:42:45.479 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59472.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59472.log.gz 2024-04-24T06:42:45.479 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69478.log 2024-04-24T06:42:45.479 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80142.log 2024-04-24T06:42:45.480 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69478.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-04-24T06:42:45.481 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T06:42:45.481 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69478.log.gz 2024-04-24T06:42:45.482 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59078.log 2024-04-24T06:42:45.482 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80142.log.gz 2024-04-24T06:42:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52559.log 2024-04-24T06:42:45.484 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74492.log 2024-04-24T06:42:45.484 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59078.log.gz 2024-04-24T06:42:45.485 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63954.log 2024-04-24T06:42:45.485 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52559.log.gz 2024-04-24T06:42:45.487 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78672.log 2024-04-24T06:42:45.487 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74492.log.gz 2024-04-24T06:42:45.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50428.log 2024-04-24T06:42:45.488 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63954.log.gz 2024-04-24T06:42:45.489 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-04-24T06:42:45.489 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78672.log.gz 2024-04-24T06:42:45.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38988.log 2024-04-24T06:42:45.490 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50428.log.gz 2024-04-24T06:42:45.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81145.log 2024-04-24T06:42:45.491 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-04-24T06:42:45.493 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38988.log.gz 2024-04-24T06:42:45.493 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58289.log 2024-04-24T06:42:45.494 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73820.log 2024-04-24T06:42:45.494 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81145.log.gz 2024-04-24T06:42:45.495 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73077.log 2024-04-24T06:42:45.495 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58289.log.gz 2024-04-24T06:42:45.496 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-04-24T06:42:45.496 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73820.log.gz 2024-04-24T06:42:45.497 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59204.log 2024-04-24T06:42:45.497 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73077.log.gz 2024-04-24T06:42:45.498 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74323.log 2024-04-24T06:42:45.499 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-04-24T06:42:45.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2024-04-24T06:42:45.500 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59204.log.gz 2024-04-24T06:42:45.501 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76489.log 2024-04-24T06:42:45.501 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74323.log.gz 2024-04-24T06:42:45.502 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52433.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2024-04-24T06:42:45.502 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67895.log 2024-04-24T06:42:45.503 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75877.log 2024-04-24T06:42:45.503 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76489.log.gz 2024-04-24T06:42:45.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62586.log 2024-04-24T06:42:45.504 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67895.log.gz 2024-04-24T06:42:45.505 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75877.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75877.log.gz 2024-04-24T06:42:45.506 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46182.log 2024-04-24T06:42:45.506 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-04-24T06:42:45.507 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62586.log.gz 2024-04-24T06:42:45.508 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-04-24T06:42:45.508 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2024-04-24T06:42:45.509 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2024-04-24T06:42:45.509 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-04-24T06:42:45.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43502.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-04-24T06:42:45.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40303.log 2024-04-24T06:42:45.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57960.log 88.8% 2024-04-24T06:42:45.512 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz 2024-04-24T06:42:45.513 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45768.log 2024-04-24T06:42:45.513 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40303.log.gz 2024-04-24T06:42:45.514 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71269.log 2024-04-24T06:42:45.514 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57960.log.gz 2024-04-24T06:42:45.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80478.log 2024-04-24T06:42:45.515 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45768.log.gz 2024-04-24T06:42:45.516 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50284.log 2024-04-24T06:42:45.516 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71269.log.gz 2024-04-24T06:42:45.517 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2024-04-24T06:42:45.517 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80478.log.gz 2024-04-24T06:42:45.518 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45432.log 2024-04-24T06:42:45.518 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50284.log.gz 2024-04-24T06:42:45.519 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73683.log 2024-04-24T06:42:45.520 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45432.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2024-04-24T06:42:45.520 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45432.log.gz 2024-04-24T06:42:45.521 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53607.log 2024-04-24T06:42:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-04-24T06:42:45.522 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73683.log.gz 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53607.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.53607.log.gz 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71329.log 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63834.log 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42185.log 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71329.log.gz 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mon.b.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71371.log 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63834.log.gz 2024-04-24T06:42:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.2.42185.log: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T06:42:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50536.log 2024-04-24T06:42:45.530 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.2.42185.log.gz 2024-04-24T06:42:45.531 INFO:teuthology.orchestra.run.smithi066.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-24T06:42:45.531 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56572.log 2024-04-24T06:42:45.532 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71371.log.gz 2024-04-24T06:42:45.532 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73119.log 2024-04-24T06:42:45.533 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50536.log.gz 2024-04-24T06:42:45.534 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-04-24T06:42:45.534 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56572.log.gz 2024-04-24T06:42:45.535 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55479.log 2024-04-24T06:42:45.535 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73119.log.gz 2024-04-24T06:42:45.536 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65432.log 2024-04-24T06:42:45.537 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-04-24T06:42:45.538 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76952.log 2024-04-24T06:42:45.538 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55479.log.gz 2024-04-24T06:42:45.539 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67135.log 2024-04-24T06:42:45.539 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65432.log.gz 2024-04-24T06:42:45.540 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58206.log 2024-04-24T06:42:45.540 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76952.log.gz 2024-04-24T06:42:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67135.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.80184.log 2024-04-24T06:42:45.542 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67135.log.gz 2024-04-24T06:42:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-04-24T06:42:45.543 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58206.log.gz 2024-04-24T06:42:45.544 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70469.log 2024-04-24T06:42:45.544 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80184.log.gz 2024-04-24T06:42:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log 2024-04-24T06:42:45.545 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-04-24T06:42:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70469.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70469.log.gz 2024-04-24T06:42:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72155.log 2024-04-24T06:42:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41232.log 2024-04-24T06:42:45.548 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2024-04-24T06:42:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-04-24T06:42:45.549 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72155.log.gz 2024-04-24T06:42:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-04-24T06:42:45.550 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41232.log.gz 2024-04-24T06:42:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80436.log 2024-04-24T06:42:45.552 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-04-24T06:42:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz 2024-04-24T06:42:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-04-24T06:42:45.554 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65024.log 2024-04-24T06:42:45.554 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80436.log.gz 2024-04-24T06:42:45.555 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61298.log 2024-04-24T06:42:45.555 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-04-24T06:42:45.556 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-04-24T06:42:45.557 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65024.log.gz 2024-04-24T06:42:45.558 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39953.log 2024-04-24T06:42:45.558 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61298.log.gz 2024-04-24T06:42:45.559 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52013.log 2024-04-24T06:42:45.559 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-04-24T06:42:45.560 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38816.log 2024-04-24T06:42:45.560 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39953.log.gz 2024-04-24T06:42:45.561 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61382.log 2024-04-24T06:42:45.562 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52013.log.gz 2024-04-24T06:42:45.563 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81733.log 89.1% 2024-04-24T06:42:45.563 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2024-04-24T06:42:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72478.log 2024-04-24T06:42:45.564 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61382.log.gz 2024-04-24T06:42:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72653.log 2024-04-24T06:42:45.565 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81733.log.gz 2024-04-24T06:42:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57040.log 2024-04-24T06:42:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72653.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72478.log.gz 2024-04-24T06:42:45.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72394.log 2024-04-24T06:42:45.568 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72653.log.gz 2024-04-24T06:42:45.569 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45684.log 2024-04-24T06:42:45.569 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57040.log.gz 2024-04-24T06:42:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81439.log 2024-04-24T06:42:45.571 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72394.log.gz 2024-04-24T06:42:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68250.log 2024-04-24T06:42:45.572 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45684.log.gz 2024-04-24T06:42:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-04-24T06:42:45.573 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81439.log.gz 2024-04-24T06:42:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-04-24T06:42:45.574 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68250.log.gz 2024-04-24T06:42:45.576 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66173.log 2024-04-24T06:42:45.576 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-04-24T06:42:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40878.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-04-24T06:42:45.577 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46393.log 2024-04-24T06:42:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49608.log 2024-04-24T06:42:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-04-24T06:42:45.579 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46393.log.gz 2024-04-24T06:42:45.580 INFO:teuthology.orchestra.run.smithi059.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.66173.log.gz 2024-04-24T06:42:45.581 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52055.log 2024-04-24T06:42:45.581 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49608.log.gz 2024-04-24T06:42:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49692.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-04-24T06:42:45.582 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-24T06:42:45.583 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39631.log 2024-04-24T06:42:45.583 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-04-24T06:42:45.584 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63750.log 2024-04-24T06:42:45.585 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-24T06:42:45.586 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69370.log 2024-04-24T06:42:45.586 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39631.log.gz 2024-04-24T06:42:45.587 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63750.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63750.log.gz 2024-04-24T06:42:45.587 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75507.log 2024-04-24T06:42:45.589 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44372.log 2024-04-24T06:42:45.589 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69370.log.gz 2024-04-24T06:42:45.590 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-04-24T06:42:45.590 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75507.log.gz 2024-04-24T06:42:45.591 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44372.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T06:42:45.592 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44372.log.gz 2024-04-24T06:42:45.593 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42777.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-04-24T06:42:45.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-04-24T06:42:45.594 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72211.log 2024-04-24T06:42:45.595 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76799.log 2024-04-24T06:42:45.595 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-04-24T06:42:45.596 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-04-24T06:42:45.597 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72211.log.gz 2024-04-24T06:42:45.598 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76799.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.76799.log.gz 2024-04-24T06:42:45.598 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49484.log 2024-04-24T06:42:45.599 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52776.log 2024-04-24T06:42:45.600 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-04-24T06:42:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2024-04-24T06:42:45.601 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49484.log.gz 2024-04-24T06:42:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80965.log 2024-04-24T06:42:45.602 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52776.log.gz 2024-04-24T06:42:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66234.log 2024-04-24T06:42:45.604 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2024-04-24T06:42:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65240.log 2024-04-24T06:42:45.605 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80965.log.gz 2024-04-24T06:42:45.606 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78258.log 2024-04-24T06:42:45.606 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66234.log.gz 2024-04-24T06:42:45.607 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42525.log 2024-04-24T06:42:45.608 INFO:teuthology.orchestra.run.smithi059.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.65240.log.gz 2024-04-24T06:42:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78258.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78258.log.gz 2024-04-24T06:42:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67937.log 2024-04-24T06:42:45.610 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81355.log 2024-04-24T06:42:45.610 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-04-24T06:42:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74239.log 2024-04-24T06:42:45.612 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67937.log.gz 2024-04-24T06:42:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78300.log 2024-04-24T06:42:45.613 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81355.log.gz 2024-04-24T06:42:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72566.log 2024-04-24T06:42:45.614 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74239.log.gz 2024-04-24T06:42:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74768.log 2024-04-24T06:42:45.615 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78300.log.gz 2024-04-24T06:42:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74151.log 2024-04-24T06:42:45.617 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72566.log.gz 2024-04-24T06:42:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80520.log 2024-04-24T06:42:45.619 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74768.log.gz 2024-04-24T06:42:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67685.log 2024-04-24T06:42:45.620 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74151.log.gz 2024-04-24T06:42:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80520.log.gz 2024-04-24T06:42:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69160.log 2024-04-24T06:42:45.623 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-04-24T06:42:45.623 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67685.log.gz 2024-04-24T06:42:45.624 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69160.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69160.log.gz 2024-04-24T06:42:45.624 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61695.log 2024-04-24T06:42:45.625 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-04-24T06:42:45.625 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-04-24T06:42:45.626 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74195.log 2024-04-24T06:42:45.626 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61695.log.gz 2024-04-24T06:42:45.627 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78606.log 2024-04-24T06:42:45.628 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-04-24T06:42:45.629 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81607.log 2024-04-24T06:42:45.629 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74195.log.gz 2024-04-24T06:42:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-04-24T06:42:45.631 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78606.log.gz 2024-04-24T06:42:45.631 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61172.log 2024-04-24T06:42:45.632 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81607.log.gz 2024-04-24T06:42:45.633 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40299.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz /var/log/ceph/ceph-client.admin.64265.log 2024-04-24T06:42:45.633 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.634 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51155.log 2024-04-24T06:42:45.634 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61172.log.gz 2024-04-24T06:42:45.635 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64265.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64265.log.gz 2024-04-24T06:42:45.635 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38512.log 2024-04-24T06:42:45.636 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53026.log 2024-04-24T06:42:45.637 INFO:teuthology.orchestra.run.smithi059.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51155.log.gz 2024-04-24T06:42:45.638 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51608.log 2024-04-24T06:42:45.638 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38512.log.gz 2024-04-24T06:42:45.639 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46224.log 2024-04-24T06:42:45.639 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53026.log.gz 2024-04-24T06:42:45.640 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2024-04-24T06:42:45.640 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51608.log.gz 2024-04-24T06:42:45.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39198.log 2024-04-24T06:42:45.641 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46224.log.gz 2024-04-24T06:42:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54929.log 2024-04-24T06:42:45.643 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2024-04-24T06:42:45.644 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-04-24T06:42:45.644 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39198.log.gz 2024-04-24T06:42:45.645 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-04-24T06:42:45.646 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54929.log.gz 2024-04-24T06:42:45.646 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74618.log 2024-04-24T06:42:45.646 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-04-24T06:42:45.647 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66502.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-04-24T06:42:45.648 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54521.log 2024-04-24T06:42:45.648 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38608.log 2024-04-24T06:42:45.649 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74618.log.gz 2024-04-24T06:42:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56702.log 2024-04-24T06:42:45.650 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54521.log.gz 2024-04-24T06:42:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-24T06:42:45.651 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38608.log.gz 2024-04-24T06:42:45.652 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-04-24T06:42:45.652 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56702.log.gz 2024-04-24T06:42:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54184.log 2024-04-24T06:42:45.653 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-24T06:42:45.655 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77990.log 2024-04-24T06:42:45.655 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-04-24T06:42:45.655 INFO:teuthology.orchestra.run.smithi066.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T06:42:45.656 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65908.log 2024-04-24T06:42:45.656 INFO:teuthology.orchestra.run.smithi059.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54184.log.gz 2024-04-24T06:42:45.657 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77990.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-24T06:42:45.657 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77990.log.gz 2024-04-24T06:42:45.658 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-04-24T06:42:45.659 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65908.log.gz 2024-04-24T06:42:45.659 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-04-24T06:42:45.662 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-04-24T06:42:45.662 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-04-24T06:42:45.662 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66048.log 2024-04-24T06:42:45.663 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz 2024-04-24T06:42:45.664 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77228.log 2024-04-24T06:42:45.664 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-04-24T06:42:45.666 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66436.log 2024-04-24T06:42:45.666 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66048.log.gz 2024-04-24T06:42:45.667 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66348.log 2024-04-24T06:42:45.667 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77228.log.gz 2024-04-24T06:42:45.668 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50032.log 2024-04-24T06:42:45.669 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66436.log.gz 2024-04-24T06:42:45.670 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42331.log 2024-04-24T06:42:45.670 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66348.log.gz 2024-04-24T06:42:45.671 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-04-24T06:42:45.671 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50032.log.gz 2024-04-24T06:42:45.672 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52984.log 2024-04-24T06:42:45.673 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42331.log.gz 2024-04-24T06:42:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53565.log 2024-04-24T06:42:45.674 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-04-24T06:42:45.675 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55242.log 2024-04-24T06:42:45.675 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52984.log.gz 2024-04-24T06:42:45.676 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40300.log 2024-04-24T06:42:45.677 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53565.log.gz 2024-04-24T06:42:45.678 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43770.log 2024-04-24T06:42:45.678 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55242.log.gz 2024-04-24T06:42:45.679 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-04-24T06:42:45.679 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40300.log.gz 2024-04-24T06:42:45.680 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60698.log 2024-04-24T06:42:45.681 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43770.log.gz 2024-04-24T06:42:45.682 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-04-24T06:42:45.682 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-04-24T06:42:45.683 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48679.log 2024-04-24T06:42:45.683 INFO:teuthology.orchestra.run.smithi059.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60698.log.gz 2024-04-24T06:42:45.684 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40836.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T06:42:45.685 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-04-24T06:42:45.686 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51415.log 2024-04-24T06:42:45.686 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48679.log.gz 2024-04-24T06:42:45.687 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57511.log 2024-04-24T06:42:45.688 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65066.log 2024-04-24T06:42:45.689 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51415.log.gz 2024-04-24T06:42:45.690 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.57511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53183.log 2024-04-24T06:42:45.691 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2024-04-24T06:42:45.692 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65066.log.gz 2024-04-24T06:42:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53183.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53183.log.gz 2024-04-24T06:42:45.693 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T06:42:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52899.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.57511.log.gz 2024-04-24T06:42:45.694 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2024-04-24T06:42:45.694 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79947.log 2024-04-24T06:42:45.696 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72071.log 2024-04-24T06:42:45.697 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2024-04-24T06:42:45.697 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79947.log.gz 2024-04-24T06:42:45.699 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68124.log 2024-04-24T06:42:45.699 INFO:teuthology.orchestra.run.smithi059.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.72071.log.gz 2024-04-24T06:42:45.700 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38689.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2024-04-24T06:42:45.700 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2024-04-24T06:42:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68124.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68124.log.gz 2024-04-24T06:42:45.702 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55689.log 2024-04-24T06:42:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43812.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2024-04-24T06:42:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75649.log 2024-04-24T06:42:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-04-24T06:42:45.705 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55689.log.gz 2024-04-24T06:42:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65324.log 2024-04-24T06:42:45.706 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75649.log.gz 2024-04-24T06:42:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53271.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-04-24T06:42:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.65324.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55116.log 2024-04-24T06:42:45.709 INFO:teuthology.orchestra.run.smithi059.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.65324.log.gz 2024-04-24T06:42:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-04-24T06:42:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55116.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T06:42:45.711 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55116.log.gz 2024-04-24T06:42:45.713 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-04-24T06:42:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67811.log 2024-04-24T06:42:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40304.log 2024-04-24T06:42:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76159.log 2024-04-24T06:42:45.718 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67811.log.gz 2024-04-24T06:42:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56045.log 2024-04-24T06:42:45.719 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40304.log.gz 2024-04-24T06:42:45.720 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-24T06:42:45.723 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.76159.log: 90.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-24T06:42:45.723 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43932.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-04-24T06:42:45.724 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48220.log 2024-04-24T06:42:45.724 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56045.log.gz 2024-04-24T06:42:45.725 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-04-24T06:42:45.726 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48220.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48220.log.gz 2024-04-24T06:42:45.727 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57166.log 2024-04-24T06:42:45.727 INFO:teuthology.orchestra.run.smithi059.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.76159.log.gz 2024-04-24T06:42:45.728 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-04-24T06:42:45.728 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-04-24T06:42:45.730 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79104.log 2024-04-24T06:42:45.731 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57166.log.gz 2024-04-24T06:42:45.731 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45012.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-04-24T06:42:45.732 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79104.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79104.log.gz 2024-04-24T06:42:45.733 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40291.log 2024-04-24T06:42:45.734 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61466.log 2024-04-24T06:42:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-04-24T06:42:45.736 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40291.log.gz 2024-04-24T06:42:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61466.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61466.log.gz 2024-04-24T06:42:45.739 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54737.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-04-24T06:42:45.739 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39630.log 2024-04-24T06:42:45.741 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57734.log 2024-04-24T06:42:45.742 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67243.log 2024-04-24T06:42:45.742 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39630.log.gz 2024-04-24T06:42:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57734.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57734.log.gz 2024-04-24T06:42:45.745 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67243.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67243.log.gz 2024-04-24T06:42:45.745 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77270.log 2024-04-24T06:42:45.746 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-04-24T06:42:45.750 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60283.log 2024-04-24T06:42:45.750 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77270.log.gz 2024-04-24T06:42:45.750 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38731.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-04-24T06:42:45.750 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60283.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60283.log.gz 2024-04-24T06:42:45.751 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-04-24T06:42:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47933.log 2024-04-24T06:42:45.754 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68208.log 2024-04-24T06:42:45.754 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-04-24T06:42:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47933.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47933.log.gz 2024-04-24T06:42:45.756 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68208.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68208.log.gz 2024-04-24T06:42:45.757 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45390.log 2024-04-24T06:42:45.758 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57233.log 2024-04-24T06:42:45.760 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81481.log 2024-04-24T06:42:45.760 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45390.log.gz 2024-04-24T06:42:45.761 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57233.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-04-24T06:42:45.762 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81481.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81481.log.gz 2024-04-24T06:42:45.763 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-04-24T06:42:45.764 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68081.log 2024-04-24T06:42:45.766 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63464.log 2024-04-24T06:42:45.766 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-04-24T06:42:45.767 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68081.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68081.log.gz 2024-04-24T06:42:45.768 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63464.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63464.log.gz 2024-04-24T06:42:45.769 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63548.log 2024-04-24T06:42:45.770 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65866.log 2024-04-24T06:42:45.772 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63876.log 2024-04-24T06:42:45.772 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63548.log.gz 2024-04-24T06:42:45.773 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65866.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65866.log.gz 2024-04-24T06:42:45.774 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63876.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63876.log.gz 2024-04-24T06:42:45.775 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81858.log 2024-04-24T06:42:45.776 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81818.log 2024-04-24T06:42:45.778 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45138.log 2024-04-24T06:42:45.778 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81858.log.gz 2024-04-24T06:42:45.779 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81818.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81818.log.gz 2024-04-24T06:42:45.780 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45138.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45138.log.gz 2024-04-24T06:42:45.781 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64958.log 2024-04-24T06:42:45.782 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70348.log 2024-04-24T06:42:45.784 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55563.log 2024-04-24T06:42:45.784 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64958.log.gz 2024-04-24T06:42:45.785 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70348.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70348.log.gz 2024-04-24T06:42:45.786 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55563.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55563.log.gz 2024-04-24T06:42:45.787 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59875.log 2024-04-24T06:42:45.788 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40794.log 2024-04-24T06:42:45.789 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-04-24T06:42:45.789 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59875.log.gz 2024-04-24T06:42:45.791 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40794.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40794.log.gz 2024-04-24T06:42:45.792 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51971.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-04-24T06:42:45.792 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60875.log 2024-04-24T06:42:45.794 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62754.log 2024-04-24T06:42:45.795 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78396.log 2024-04-24T06:42:45.795 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60875.log.gz 2024-04-24T06:42:45.796 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62754.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62754.log.gz 2024-04-24T06:42:45.798 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78396.log.gz 2024-04-24T06:42:45.798 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54237.log 2024-04-24T06:42:45.799 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62544.log 2024-04-24T06:42:45.801 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63690.log 2024-04-24T06:42:45.801 INFO:teuthology.orchestra.run.smithi059.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.54237.log.gz 2024-04-24T06:42:45.802 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62544.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62544.log.gz 2024-04-24T06:42:45.804 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63690.log.gz 2024-04-24T06:42:45.804 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log 2024-04-24T06:42:45.805 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38900.log 2024-04-24T06:42:45.807 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66394.log 2024-04-24T06:42:45.807 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2024-04-24T06:42:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38900.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38900.log.gz 2024-04-24T06:42:45.810 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66394.log.gz 2024-04-24T06:42:45.810 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55731.log 2024-04-24T06:42:45.811 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38858.log 2024-04-24T06:42:45.813 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76715.log 2024-04-24T06:42:45.813 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz 2024-04-24T06:42:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38858.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38858.log.gz 2024-04-24T06:42:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76715.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76715.log.gz 2024-04-24T06:42:45.816 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79863.log 2024-04-24T06:42:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68674.log 2024-04-24T06:42:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56364.log 2024-04-24T06:42:45.819 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79863.log.gz 2024-04-24T06:42:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68674.log.gz 2024-04-24T06:42:45.821 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56364.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56364.log.gz 2024-04-24T06:42:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80923.log 2024-04-24T06:42:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62502.log 2024-04-24T06:42:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52139.log 2024-04-24T06:42:45.825 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80923.log.gz 2024-04-24T06:42:45.826 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62502.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62502.log.gz 2024-04-24T06:42:45.827 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52139.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52139.log.gz 2024-04-24T06:42:45.827 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60241.log 2024-04-24T06:42:45.829 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-04-24T06:42:45.830 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52601.log 2024-04-24T06:42:45.831 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60241.log.gz 2024-04-24T06:42:45.832 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41088.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-04-24T06:42:45.832 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75691.log 2024-04-24T06:42:45.834 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74281.log 2024-04-24T06:42:45.834 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52601.log.gz 2024-04-24T06:42:45.835 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75691.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75691.log.gz 2024-04-24T06:42:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-04-24T06:42:45.837 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74281.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74281.log.gz 2024-04-24T06:42:45.837 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70590.log 2024-04-24T06:42:45.838 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42441.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-04-24T06:42:45.839 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54455.log 2024-04-24T06:42:45.840 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42567.log 2024-04-24T06:42:45.840 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70590.log.gz 2024-04-24T06:42:45.841 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-04-24T06:42:45.842 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42567.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54455.log.gz 2024-04-24T06:42:45.843 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42567.log.gz 2024-04-24T06:42:45.843 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55200.log 2024-04-24T06:42:45.844 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44118.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-04-24T06:42:45.845 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69244.log 2024-04-24T06:42:45.846 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61863.log 2024-04-24T06:42:45.846 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55200.log.gz 2024-04-24T06:42:45.847 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T06:42:45.847 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69244.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69244.log.gz 2024-04-24T06:42:45.849 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61863.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61863.log.gz 2024-04-24T06:42:45.849 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55521.log 2024-04-24T06:42:45.850 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54359.log 2024-04-24T06:42:45.852 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-04-24T06:42:45.852 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55521.log.gz 2024-04-24T06:42:45.854 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39962.log 2024-04-24T06:42:45.854 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54359.log.gz 2024-04-24T06:42:45.855 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52349.log 2024-04-24T06:42:45.855 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-04-24T06:42:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39962.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39962.log.gz 2024-04-24T06:42:45.856 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72993.log 2024-04-24T06:42:45.858 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52349.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52349.log.gz 2024-04-24T06:42:45.858 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-24T06:42:45.860 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51242.log 2024-04-24T06:42:45.860 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72993.log.gz 2024-04-24T06:42:45.861 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47305.log 2024-04-24T06:42:45.861 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-24T06:42:45.862 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2024-04-24T06:42:45.863 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51242.log.gz 2024-04-24T06:42:45.865 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.47305.log: /var/log/ceph/ceph-client.admin.49990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66071.log 2024-04-24T06:42:45.865 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2024-04-24T06:42:45.866 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T06:42:45.868 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66071.log: /var/log/ceph/ceph-mon.a.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.66071.log.gz 2024-04-24T06:42:45.869 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77948.log 2024-04-24T06:42:45.870 INFO:teuthology.orchestra.run.smithi059.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.47305.log.gz 2024-04-24T06:42:45.872 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54629.log 2024-04-24T06:42:45.872 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77948.log.gz 2024-04-24T06:42:45.873 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63996.log 2024-04-24T06:42:45.875 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54629.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54629.log.gz 2024-04-24T06:42:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73287.log 2024-04-24T06:42:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63996.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63996.log.gz 2024-04-24T06:42:45.877 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77614.log 2024-04-24T06:42:45.879 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73287.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73287.log.gz 2024-04-24T06:42:45.879 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose 91.5% -- /var/log/ceph/ceph-client.0.70872.log 2024-04-24T06:42:45.880 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T06:42:45.882 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.77614.log: /var/log/ceph/ceph-client.0.70872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59809.log 2024-04-24T06:42:45.884 INFO:teuthology.orchestra.run.smithi059.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.77614.log.gz 2024-04-24T06:42:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64019.log 2024-04-24T06:42:45.885 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59809.log.gz 2024-04-24T06:42:45.886 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log 2024-04-24T06:42:45.888 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64019.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64019.log.gz 2024-04-24T06:42:45.889 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-04-24T06:42:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56998.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2024-04-24T06:42:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2024-04-24T06:42:45.891 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47749.log: 94.3% -- replaced with /var/log/ceph/ceph-client.0.70872.log.gz 2024-04-24T06:42:45.892 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65824.log 2024-04-24T06:42:45.892 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-04-24T06:42:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40752.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2024-04-24T06:42:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-04-24T06:42:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61256.log 2024-04-24T06:42:45.895 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9%/var/log/ceph/ceph-client.admin.45971.log: -- replaced with /var/log/ceph/ceph-client.admin.65824.log.gz 2024-04-24T06:42:45.896 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-04-24T06:42:45.896 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56530.log 2024-04-24T06:42:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61256.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61256.log.gz 2024-04-24T06:42:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48721.log 2024-04-24T06:42:45.899 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81271.log 2024-04-24T06:42:45.899 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56530.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56530.log.gz 2024-04-24T06:42:45.900 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72908.log 2024-04-24T06:42:45.900 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48721.log.gz 2024-04-24T06:42:45.901 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81271.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81271.log.gz 2024-04-24T06:42:45.902 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48763.log 2024-04-24T06:42:45.903 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65640.log 2024-04-24T06:42:45.903 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72908.log.gz 2024-04-24T06:42:45.904 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73900.log 2024-04-24T06:42:45.904 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48763.log.gz 2024-04-24T06:42:45.905 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65640.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65640.log.gz 2024-04-24T06:42:45.905 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62330.log 2024-04-24T06:42:45.907 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-04-24T06:42:45.907 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73900.log.gz 2024-04-24T06:42:45.908 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51027.log 2024-04-24T06:42:45.909 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62330.log.gz 2024-04-24T06:42:45.909 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52097.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-04-24T06:42:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80310.log 2024-04-24T06:42:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58872.log 2024-04-24T06:42:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80310.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51027.log.gz 2024-04-24T06:42:45.912 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80310.log.gz 2024-04-24T06:42:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48137.log 2024-04-24T06:42:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.58872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62796.log 2024-04-24T06:42:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48137.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.58872.log.gz 2024-04-24T06:42:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2024-04-24T06:42:45.916 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48137.log.gz 2024-04-24T06:42:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62796.log.gz 2024-04-24T06:42:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73371.log 2024-04-24T06:42:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64807.log 2024-04-24T06:42:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51326.log: /var/log/ceph/ceph-client.admin.73371.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz 2024-04-24T06:42:45.920 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73371.log.gz 2024-04-24T06:42:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50344.log 2024-04-24T06:42:45.922 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64807.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64807.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-04-24T06:42:45.922 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.923 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50470.log 2024-04-24T06:42:45.923 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50344.log.gz 2024-04-24T06:42:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56445.log 2024-04-24T06:42:45.925 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-04-24T06:42:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40920.log 2024-04-24T06:42:45.927 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50470.log.gz/var/log/ceph/ceph-client.admin.56445.log: 2024-04-24T06:42:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56445.log.gz 2024-04-24T06:42:45.928 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.48331.log 2024-04-24T06:42:45.929 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2024-04-24T06:42:45.929 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40920.log.gz 2024-04-24T06:42:45.930 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60791.log 2024-04-24T06:42:45.930 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-04-24T06:42:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2024-04-24T06:42:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-04-24T06:42:45.933 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57902.log 2024-04-24T06:42:45.933 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60791.log.gz 2024-04-24T06:42:45.934 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-04-24T06:42:45.934 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.43644.log 2024-04-24T06:42:45.936 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57902.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57902.log.gz 2024-04-24T06:42:45.936 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78761.log 2024-04-24T06:42:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43644.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-04-24T06:42:45.937 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43197.log 2024-04-24T06:42:45.939 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33384.log 2024-04-24T06:42:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43197.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.78761.log.gz 2024-04-24T06:42:45.940 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43197.log.gz 2024-04-24T06:42:45.941 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-04-24T06:42:45.942 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78476.log 2024-04-24T06:42:45.942 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33384.log.gz 2024-04-24T06:42:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-04-24T06:42:45.943 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-04-24T06:42:45.945 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40626.log 2024-04-24T06:42:45.945 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78476.log.gz 2024-04-24T06:42:45.946 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59346.log 2024-04-24T06:42:45.946 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-04-24T06:42:45.947 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40626.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40626.log.gz 2024-04-24T06:42:45.947 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68716.log 2024-04-24T06:42:45.948 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-04-24T06:42:45.948 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59346.log.gz 2024-04-24T06:42:45.949 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58486.log 2024-04-24T06:42:45.950 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68716.log.gz 2024-04-24T06:42:45.951 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39030.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-04-24T06:42:45.951 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-04-24T06:42:45.952 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40542.log 2024-04-24T06:42:45.952 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58486.log.gz 2024-04-24T06:42:45.954 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51803.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-04-24T06:42:45.954 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40542.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67406.log 2024-04-24T06:42:45.955 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40542.log.gz 2024-04-24T06:42:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69762.log 2024-04-24T06:42:45.957 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51803.log.gz 2024-04-24T06:42:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57124.log 2024-04-24T06:42:45.958 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67406.log.gz 2024-04-24T06:42:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.69762.log: /var/log/ceph/ceph-client.admin.57124.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57124.log.gz 2024-04-24T06:42:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67177.log 2024-04-24T06:42:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55074.log 2024-04-24T06:42:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67177.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.69762.log.gz 2024-04-24T06:42:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70534.log 2024-04-24T06:42:45.964 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.67177.log.gz 2024-04-24T06:42:45.966 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48179.log 2024-04-24T06:42:45.966 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55074.log.gz 2024-04-24T06:42:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70534.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70534.log.gz 2024-04-24T06:42:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42651.log 2024-04-24T06:42:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69412.log 2024-04-24T06:42:45.969 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48179.log.gz 2024-04-24T06:42:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63792.log 2024-04-24T06:42:45.970 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42651.log.gz 2024-04-24T06:42:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52307.log 2024-04-24T06:42:45.972 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69412.log.gz 2024-04-24T06:42:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58264.log 2024-04-24T06:42:45.973 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63792.log.gz 2024-04-24T06:42:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52307.log.gz 2024-04-24T06:42:45.974 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:45.975 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58264.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58264.log.gz 2024-04-24T06:42:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72719.log 2024-04-24T06:42:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51392.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2024-04-24T06:42:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74109.log 2024-04-24T06:42:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2024-04-24T06:42:45.979 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72719.log.gz 2024-04-24T06:42:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74109.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74109.log.gz 2024-04-24T06:42:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55605.log 2024-04-24T06:42:45.981 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43323.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2024-04-24T06:42:45.981 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67727.log 2024-04-24T06:42:45.983 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.55605.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72048.log 2024-04-24T06:42:45.983 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55605.log.gz 2024-04-24T06:42:45.984 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67727.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67727.log.gz 2024-04-24T06:42:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-04-24T06:42:45.985 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72048.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72048.log.gz 2024-04-24T06:42:45.986 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53313.log 2024-04-24T06:42:45.987 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81187.log 2024-04-24T06:42:45.988 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48553.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-04-24T06:42:45.989 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53313.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53313.log.gz 2024-04-24T06:42:45.989 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78132.log 2024-04-24T06:42:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81187.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81187.log.gz 2024-04-24T06:42:45.990 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55877.log 2024-04-24T06:42:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-04-24T06:42:45.992 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78132.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78132.log.gz 2024-04-24T06:42:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55877.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55877.log.gz 2024-04-24T06:42:45.993 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62374.log 2024-04-24T06:42:45.994 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43155.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-04-24T06:42:45.995 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73329.log 2024-04-24T06:42:45.996 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62374.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62374.log.gz 2024-04-24T06:42:45.996 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80749.log 2024-04-24T06:42:45.997 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54563.log 2024-04-24T06:42:45.998 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73329.log.gz 2024-04-24T06:42:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80749.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80749.log.gz 2024-04-24T06:42:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52181.log 2024-04-24T06:42:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54563.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54563.log.gz 2024-04-24T06:42:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52857.log 2024-04-24T06:42:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52181.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52181.log.gz 2024-04-24T06:42:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-04-24T06:42:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.52857.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68801.log 2024-04-24T06:42:46.004 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52857.log.gz 2024-04-24T06:42:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49057.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-04-24T06:42:46.006 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38944.log 2024-04-24T06:42:46.007 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68801.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68801.log.gz 2024-04-24T06:42:46.007 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81918.log 2024-04-24T06:42:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-04-24T06:42:46.009 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38944.log.gz 2024-04-24T06:42:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81918.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81918.log.gz 2024-04-24T06:42:46.010 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-04-24T06:42:46.011 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56829.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-04-24T06:42:46.011 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55773.log 2024-04-24T06:42:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65132.log 2024-04-24T06:42:46.013 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49183.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-04-24T06:42:46.015 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55773.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55773.log.gz 2024-04-24T06:42:46.015 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58398.log 2024-04-24T06:42:46.015 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65132.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65132.log.gz 2024-04-24T06:42:46.016 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-04-24T06:42:46.017 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66288.log 2024-04-24T06:42:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58398.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58398.log.gz 2024-04-24T06:42:46.018 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39965.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-04-24T06:42:46.019 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2024-04-24T06:42:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66288.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66288.log.gz 2024-04-24T06:42:46.020 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65682.log 2024-04-24T06:42:46.021 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68292.log 2024-04-24T06:42:46.022 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48257.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2024-04-24T06:42:46.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65682.log.gz 2024-04-24T06:42:46.023 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39240.log 2024-04-24T06:42:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68292.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68292.log.gz 2024-04-24T06:42:46.024 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40500.log 2024-04-24T06:42:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76615.log 2024-04-24T06:42:46.026 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39240.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39240.log.gz 2024-04-24T06:42:46.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40500.log.gz 2024-04-24T06:42:46.028 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-04-24T06:42:46.028 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76615.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76615.log.gz 2024-04-24T06:42:46.029 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72306.log 2024-04-24T06:42:46.030 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50386.log 2024-04-24T06:42:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49267.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-04-24T06:42:46.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75046.log 2024-04-24T06:42:46.032 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72306.log.gz 2024-04-24T06:42:46.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50386.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50386.log.gz 2024-04-24T06:42:46.033 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74365.log 2024-04-24T06:42:46.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.75046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59246.log 2024-04-24T06:42:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74365.log.gz 2024-04-24T06:42:46.036 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-04-24T06:42:46.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59246.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59246.log.gz 2024-04-24T06:42:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78550.log 2024-04-24T06:42:46.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52475.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-04-24T06:42:46.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.78550.log: /var/log/ceph/ceph-client.admin.61653.log 2024-04-24T06:42:46.041 INFO:teuthology.orchestra.run.smithi059.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.75046.log.gz 2024-04-24T06:42:46.041 INFO:teuthology.orchestra.run.smithi059.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.78550.log.gz 2024-04-24T06:42:46.042 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62288.log 2024-04-24T06:42:46.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61653.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61653.log.gz 2024-04-24T06:42:46.043 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68166.log 2024-04-24T06:42:46.044 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.62288.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44297.log 2024-04-24T06:42:46.045 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62288.log.gz 2024-04-24T06:42:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68166.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68166.log.gz 2024-04-24T06:42:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75733.log 2024-04-24T06:42:46.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44297.log.gz 2024-04-24T06:42:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71982.log 2024-04-24T06:42:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.75733.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65390.log 2024-04-24T06:42:46.049 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75733.log.gz 2024-04-24T06:42:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45516.log 2024-04-24T06:42:46.051 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.71982.log.gz 2024-04-24T06:42:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65390.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65390.log.gz 2024-04-24T06:42:46.052 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-04-24T06:42:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60959.log 2024-04-24T06:42:46.053 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45516.log.gz 2024-04-24T06:42:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46098.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-04-24T06:42:46.055 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39629.log 2024-04-24T06:42:46.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60959.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60959.log.gz 2024-04-24T06:42:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78803.log 2024-04-24T06:42:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39629.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39629.log.gz 2024-04-24T06:42:46.058 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-04-24T06:42:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79821.log 2024-04-24T06:42:46.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49566.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-04-24T06:42:46.061 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81313.log 2024-04-24T06:42:46.061 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78803.log.gz 2024-04-24T06:42:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59983.log 2024-04-24T06:42:46.063 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79821.log.gz 2024-04-24T06:42:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81313.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81313.log.gz 2024-04-24T06:42:46.064 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62161.log 2024-04-24T06:42:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59917.log 2024-04-24T06:42:46.066 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62161.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59983.log.gz 2024-04-24T06:42:46.066 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62161.log.gz 2024-04-24T06:42:46.067 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78174.log 2024-04-24T06:42:46.068 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T06:42:46.069 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40710.log 2024-04-24T06:42:46.069 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59917.log.gz 2024-04-24T06:42:46.070 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-04-24T06:42:46.070 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78174.log.gz 2024-04-24T06:42:46.071 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40710.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40710.log.gz 2024-04-24T06:42:46.072 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53481.log 2024-04-24T06:42:46.072 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-04-24T06:42:46.073 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75565.log 2024-04-24T06:42:46.074 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64373.log 2024-04-24T06:42:46.075 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53481.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53481.log.gz 2024-04-24T06:42:46.076 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75565.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75565.log.gz 2024-04-24T06:42:46.076 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75465.log 2024-04-24T06:42:46.077 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64373.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64373.log.gz 2024-04-24T06:42:46.077 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67601.log 2024-04-24T06:42:46.079 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-04-24T06:42:46.079 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75465.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75465.log.gz 2024-04-24T06:42:46.080 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62203.log 2024-04-24T06:42:46.080 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67601.log.gz 2024-04-24T06:42:46.082 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44928.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-04-24T06:42:46.082 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62946.log 2024-04-24T06:42:46.083 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70264.log 2024-04-24T06:42:46.083 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62203.log.gz 2024-04-24T06:42:46.084 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62946.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62946.log.gz 2024-04-24T06:42:46.085 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-04-24T06:42:46.086 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70264.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70264.log.gz 2024-04-24T06:42:46.086 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-04-24T06:42:46.088 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78074.log 2024-04-24T06:42:46.088 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-04-24T06:42:46.089 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39072.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-04-24T06:42:46.089 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-04-24T06:42:46.090 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T06:42:46.090 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77087.log 2024-04-24T06:42:46.090 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78074.log.gz 2024-04-24T06:42:46.092 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76673.log 2024-04-24T06:42:46.092 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-04-24T06:42:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77087.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77087.log.gz 2024-04-24T06:42:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-04-24T06:42:46.094 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76673.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76673.log.gz 2024-04-24T06:42:46.095 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70222.log 2024-04-24T06:42:46.096 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69202.log 2024-04-24T06:42:46.097 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-04-24T06:42:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63224.log 2024-04-24T06:42:46.097 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70222.log.gz 2024-04-24T06:42:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69202.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69202.log.gz 2024-04-24T06:42:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-04-24T06:42:46.100 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.63224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63632.log 2024-04-24T06:42:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49015.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-04-24T06:42:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.63632.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57082.log 2024-04-24T06:42:46.103 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63632.log.gz 2024-04-24T06:42:46.104 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose 92.4% -- replaced with /var/log/ceph/ceph-client.0.63224.log.gz 2024-04-24T06:42:46.104 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.46667.log 2024-04-24T06:42:46.105 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57082.log.gz 2024-04-24T06:42:46.106 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39958.log 2024-04-24T06:42:46.106 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46667.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2024-04-24T06:42:46.107 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72352.log 2024-04-24T06:42:46.108 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64307.log 2024-04-24T06:42:46.109 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39958.log.gz 2024-04-24T06:42:46.109 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72352.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72352.log.gz 2024-04-24T06:42:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67769.log 2024-04-24T06:42:46.111 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64307.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64307.log.gz 2024-04-24T06:42:46.112 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T06:42:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53523.log 2024-04-24T06:42:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67769.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67769.log.gz 2024-04-24T06:42:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68513.log 2024-04-24T06:42:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5/var/log/ceph/ceph-client.admin.53523.log: --verbose -- /var/log/ceph/ceph-client.admin.41791.log 2024-04-24T06:42:46.115 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53523.log.gz 2024-04-24T06:42:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68513.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68513.log.gz /var/log/ceph/ceph-client.admin.62418.log 2024-04-24T06:42:46.116 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56914.log 2024-04-24T06:42:46.118 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41791.log.gz 2024-04-24T06:42:46.118 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62418.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62418.log.gz 2024-04-24T06:42:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50814.log 2024-04-24T06:42:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75423.log 2024-04-24T06:42:46.120 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56914.log.gz 2024-04-24T06:42:46.122 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.50814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-04-24T06:42:46.123 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75423.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75423.log.gz 2024-04-24T06:42:46.123 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55437.log 2024-04-24T06:42:46.125 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 93.0% -- replaced with /var/log/ceph/ceph-client.0.50814.log.gz 2024-04-24T06:42:46.125 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.125 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27369.log 2024-04-24T06:42:46.126 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55437.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55437.log.gz -5 2024-04-24T06:42:46.126 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61070.log 2024-04-24T06:42:46.128 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.tmp-client.admin.27369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58086.log 2024-04-24T06:42:46.128 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27369.log.gz 2024-04-24T06:42:46.129 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61070.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61070.log.gz 2024-04-24T06:42:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78439.log 2024-04-24T06:42:46.130 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58086.log.gz 2024-04-24T06:42:46.131 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46140.log 2024-04-24T06:42:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.78439.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72742.log 2024-04-24T06:42:46.133 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78439.log.gz 2024-04-24T06:42:46.133 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46140.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46140.log.gz 2024-04-24T06:42:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-04-24T06:42:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69328.log 2024-04-24T06:42:46.136 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.72742.log.gz 2024-04-24T06:42:46.137 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67853.log 2024-04-24T06:42:46.137 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-04-24T06:42:46.138 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69328.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69328.log.gz 2024-04-24T06:42:46.138 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58345.log 2024-04-24T06:42:46.140 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67853.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67853.log.gz 2024-04-24T06:42:46.140 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39945.log 2024-04-24T06:42:46.141 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61214.log 2024-04-24T06:42:46.142 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58345.log.gz 2024-04-24T06:42:46.142 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39945.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39945.log.gz 2024-04-24T06:42:46.143 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45810.log 2024-04-24T06:42:46.144 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61214.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61214.log.gz 2024-04-24T06:42:46.144 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65782.log 2024-04-24T06:42:46.145 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71639.log 2024-04-24T06:42:46.146 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45810.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45810.log.gz 2024-04-24T06:42:46.147 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73203.log 2024-04-24T06:42:46.147 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65782.log.gz 2024-04-24T06:42:46.148 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71639.log.gz 2024-04-24T06:42:46.148 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40584.log 2024-04-24T06:42:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61779.log 2024-04-24T06:42:46.150 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73203.log.gz 2024-04-24T06:42:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53225.log 2024-04-24T06:42:46.151 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40584.log.gz 2024-04-24T06:42:46.152 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61779.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61779.log.gz 2024-04-24T06:42:46.152 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58528.log 2024-04-24T06:42:46.153 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74067.log 2024-04-24T06:42:46.153 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53225.log.gz 2024-04-24T06:42:46.155 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59120.log 2024-04-24T06:42:46.155 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58528.log.gz 2024-04-24T06:42:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74067.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74067.log.gz 2024-04-24T06:42:46.156 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66919.log 2024-04-24T06:42:46.158 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-04-24T06:42:46.158 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59120.log.gz 2024-04-24T06:42:46.159 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65986.log 2024-04-24T06:42:46.159 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49906.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.66919.log.gz 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78216.log 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33650.log 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65986.log.gz 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78216.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78216.log.gz 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.68334.log 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33650.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33650.log.gz 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T06:42:46.166 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81397.log 2024-04-24T06:42:46.166 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68334.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68334.log.gz 2024-04-24T06:42:46.166 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-04-24T06:42:46.168 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70038.log 2024-04-24T06:42:46.168 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81397.log.gz 2024-04-24T06:42:46.169 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-04-24T06:42:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m0.873s 2024-04-24T06:42:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m3.964s 2024-04-24T06:42:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.407s 2024-04-24T06:42:46.169 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43586.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-04-24T06:42:46.170 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72479.log 2024-04-24T06:42:46.172 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2024-04-24T06:42:46.172 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70038.log.gz 2024-04-24T06:42:46.172 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72479.log.gz 2024-04-24T06:42:46.172 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.173 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47975.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2024-04-24T06:42:46.173 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76531.log 2024-04-24T06:42:46.178 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48637.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55353.log 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.76531.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.79212.log 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76531.log.gz 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55353.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55353.log.gz 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-04-24T06:42:46.179 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79212.log.gz 2024-04-24T06:42:46.180 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44640.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-04-24T06:42:46.181 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62670.log 2024-04-24T06:42:46.182 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-04-24T06:42:46.182 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-04-24T06:42:46.183 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57860.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62670.log.gz 2024-04-24T06:42:46.183 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.184 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48889.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-04-24T06:42:46.185 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72113.log 2024-04-24T06:42:46.186 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39408.log 2024-04-24T06:42:46.186 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57860.log.gz 2024-04-24T06:42:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72113.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.72113.log.gz 2024-04-24T06:42:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50158.log 2024-04-24T06:42:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74024.log 2024-04-24T06:42:46.189 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39408.log.gz 2024-04-24T06:42:46.191 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46437.log 2024-04-24T06:42:46.191 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50158.log.gz 2024-04-24T06:42:46.191 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74024.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74024.log.gz 2024-04-24T06:42:46.192 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51692.log 2024-04-24T06:42:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.46437.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68949.log 2024-04-24T06:42:46.194 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46437.log.gz 2024-04-24T06:42:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51692.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51692.log.gz 2024-04-24T06:42:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67364.log 2024-04-24T06:42:46.196 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68949.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68949.log.gz 2024-04-24T06:42:46.196 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62246.log 2024-04-24T06:42:46.197 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-04-24T06:42:46.198 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67364.log.gz 2024-04-24T06:42:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62246.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62246.log.gz 2024-04-24T06:42:46.199 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67979.log 2024-04-24T06:42:46.200 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43544.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-04-24T06:42:46.201 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62880.log 2024-04-24T06:42:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40287.log 2024-04-24T06:42:46.202 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67979.log: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T06:42:46.203 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67979.log.gz 2024-04-24T06:42:46.204 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62880.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62880.log.gz 2024-04-24T06:42:46.204 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-04-24T06:42:46.205 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40287.log.gz 2024-04-24T06:42:46.205 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64651.log 2024-04-24T06:42:46.206 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-04-24T06:42:46.206 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-04-24T06:42:46.207 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.64651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65174.log 2024-04-24T06:42:46.208 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48847.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-04-24T06:42:46.209 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-04-24T06:42:46.209 INFO:teuthology.orchestra.run.smithi059.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.64651.log.gz 2024-04-24T06:42:46.210 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73245.log 2024-04-24T06:42:46.210 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65174.log.gz 2024-04-24T06:42:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-04-24T06:42:46.211 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-04-24T06:42:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39450.log 2024-04-24T06:42:46.212 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73245.log.gz 2024-04-24T06:42:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71227.log 2024-04-24T06:42:46.213 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-04-24T06:42:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2024-04-24T06:42:46.214 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39450.log.gz 2024-04-24T06:42:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67490.log 2024-04-24T06:42:46.215 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71227.log.gz 2024-04-24T06:42:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55647.log 2024-04-24T06:42:46.217 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2024-04-24T06:42:46.218 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54995.log 2024-04-24T06:42:46.218 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67490.log.gz 2024-04-24T06:42:46.218 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45264.log 2024-04-24T06:42:46.219 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55647.log.gz 2024-04-24T06:42:46.220 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54995.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54995.log.gz 2024-04-24T06:42:46.220 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-24T06:42:46.221 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77906.log 2024-04-24T06:42:46.221 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45264.log.gz 2024-04-24T06:42:46.222 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81229.log 2024-04-24T06:42:46.222 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-24T06:42:46.223 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49404.log 2024-04-24T06:42:46.223 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77906.log.gz 2024-04-24T06:42:46.224 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68418.log 2024-04-24T06:42:46.224 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81229.log.gz 2024-04-24T06:42:46.225 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77140.log 2024-04-24T06:42:46.225 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49404.log.gz 2024-04-24T06:42:46.226 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2024-04-24T06:42:46.226 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68418.log.gz 2024-04-24T06:42:46.228 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80665.log 2024-04-24T06:42:46.228 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77140.log.gz 2024-04-24T06:42:46.228 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46625.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-04-24T06:42:46.228 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gz 2024-04-24T06:42:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66985.log 2024-04-24T06:42:46.230 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80665.log.gz 2024-04-24T06:42:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-04-24T06:42:46.230 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-04-24T06:42:46.231 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-04-24T06:42:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45180.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66985.log.gz 2024-04-24T06:42:46.233 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39968.log 2024-04-24T06:42:46.233 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2024-04-24T06:42:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70122.log 2024-04-24T06:42:46.234 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-04-24T06:42:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56189.log 2024-04-24T06:42:46.235 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39968.log.gz 2024-04-24T06:42:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70164.log 2024-04-24T06:42:46.236 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70122.log.gz 2024-04-24T06:42:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71555.log 2024-04-24T06:42:46.237 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56189.log.gz 2024-04-24T06:42:46.238 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2024-04-24T06:42:46.238 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70164.log.gz 2024-04-24T06:42:46.239 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46523.log 2024-04-24T06:42:46.240 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71555.log.gz 2024-04-24T06:42:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65514.log 2024-04-24T06:42:46.240 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2024-04-24T06:42:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46523.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-24T06:42:46.241 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46523.log.gz 2024-04-24T06:42:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68759.log 2024-04-24T06:42:46.243 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65514.log.gz 2024-04-24T06:42:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51887.log 2024-04-24T06:42:46.245 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73455.log 2024-04-24T06:42:46.245 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68759.log.gz 2024-04-24T06:42:46.246 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75381.log 2024-04-24T06:42:46.246 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51887.log.gz 2024-04-24T06:42:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74702.log 2024-04-24T06:42:46.248 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73455.log.gz 2024-04-24T06:42:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75381.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75381.log.gz 2024-04-24T06:42:46.249 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-04-24T06:42:46.250 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-04-24T06:42:46.251 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74702.log.gz 2024-04-24T06:42:46.251 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78695.log 2024-04-24T06:42:46.252 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-04-24T06:42:46.253 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58128.log 2024-04-24T06:42:46.253 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-04-24T06:42:46.254 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65556.log 2024-04-24T06:42:46.254 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78695.log.gz 2024-04-24T06:42:46.255 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-04-24T06:42:46.255 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58128.log.gz 2024-04-24T06:42:46.257 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50242.log 2024-04-24T06:42:46.257 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65556.log.gz 2024-04-24T06:42:46.258 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39617.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-04-24T06:42:46.258 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53397.log 2024-04-24T06:42:46.259 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68991.log 2024-04-24T06:42:46.260 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50242.log.gz 2024-04-24T06:42:46.261 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61568.log 88.7% 2024-04-24T06:42:46.261 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53397.log.gz 2024-04-24T06:42:46.262 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68991.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68991.log.gz 2024-04-24T06:42:46.262 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75607.log 2024-04-24T06:42:46.263 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56658.log 2024-04-24T06:42:46.264 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61568.log.gz 2024-04-24T06:42:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80226.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75607.log.gz 2024-04-24T06:42:46.265 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56658.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56658.log.gz 2024-04-24T06:42:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55833.log 2024-04-24T06:42:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80226.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80226.log.gz 2024-04-24T06:42:46.268 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78963.log 2024-04-24T06:42:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80352.log 2024-04-24T06:42:46.269 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55833.log.gz 2024-04-24T06:42:46.270 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59162.log 2024-04-24T06:42:46.270 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78963.log.gz 2024-04-24T06:42:46.272 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60917.log 2024-04-24T06:42:46.272 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80352.log.gz 2024-04-24T06:42:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59162.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59162.log.gz 2024-04-24T06:42:46.273 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80623.log 2024-04-24T06:42:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60917.log.gz 2024-04-24T06:42:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62080.log 2024-04-24T06:42:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64219.log 2024-04-24T06:42:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80623.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80623.log.gz 2024-04-24T06:42:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44414.log 2024-04-24T06:42:46.277 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62080.log.gz 2024-04-24T06:42:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79016.log 2024-04-24T06:42:46.278 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64219.log.gz 2024-04-24T06:42:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59767.log 2024-04-24T06:42:46.280 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44414.log.gz 2024-04-24T06:42:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70511.log 2024-04-24T06:42:46.281 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79016.log.gz 2024-04-24T06:42:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76841.log 2024-04-24T06:42:46.282 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59767.log.gz 2024-04-24T06:42:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70511.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70511.log.gz 2024-04-24T06:42:46.283 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2024-04-24T06:42:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45348.log 2024-04-24T06:42:46.285 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76841.log.gz 2024-04-24T06:42:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66131.log 2024-04-24T06:42:46.286 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2024-04-24T06:42:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45348.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45348.log.gz 2024-04-24T06:42:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61737.log 2024-04-24T06:42:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63506.log 2024-04-24T06:42:46.290 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61737.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61737.log.gz 2024-04-24T06:42:46.290 INFO:teuthology.orchestra.run.smithi059.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.66131.log.gz 2024-04-24T06:42:46.291 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.63506.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58044.log 2024-04-24T06:42:46.291 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63506.log.gz 2024-04-24T06:42:46.292 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-04-24T06:42:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-04-24T06:42:46.294 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58044.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58044.log.gz 2024-04-24T06:42:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49864.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2024-04-24T06:42:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54671.log 2024-04-24T06:42:46.295 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42483.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-04-24T06:42:46.296 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79905.log 2024-04-24T06:42:46.297 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74576.log 2024-04-24T06:42:46.299 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54671.log: /var/log/ceph/ceph-client.admin.79905.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54671.log.gz 2024-04-24T06:42:46.299 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79905.log.gz 2024-04-24T06:42:46.299 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39282.log 2024-04-24T06:42:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74576.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74576.log.gz 2024-04-24T06:42:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80394.log 2024-04-24T06:42:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51080.log 2024-04-24T06:42:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39282.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gz 2024-04-24T06:42:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80394.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80394.log.gz 2024-04-24T06:42:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67069.log 2024-04-24T06:42:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51080.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51080.log.gz 2024-04-24T06:42:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-04-24T06:42:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51524.log 2024-04-24T06:42:46.307 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44970.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.67069.log.gz 2024-04-24T06:42:46.307 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz 2024-04-24T06:42:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61611.log 2024-04-24T06:42:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51524.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51524.log.gz 2024-04-24T06:42:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71413.log 2024-04-24T06:42:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2024-04-24T06:42:46.310 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61611.log.gz 2024-04-24T06:42:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-04-24T06:42:46.312 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71413.log.gz 2024-04-24T06:42:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41004.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2024-04-24T06:42:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39627.log 2024-04-24T06:42:46.314 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44886.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-04-24T06:42:46.314 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59388.log 2024-04-24T06:42:46.315 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5/var/log/ceph/ceph-client.admin.39627.log: --verbose -- /var/log/ceph/ceph-client.admin.71717.log 2024-04-24T06:42:46.316 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39627.log.gz 2024-04-24T06:42:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59388.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59388.log.gz 2024-04-24T06:42:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73035.log 2024-04-24T06:42:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71717.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71717.log.gz 2024-04-24T06:42:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45726.log 2024-04-24T06:42:46.320 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.73035.log: -- /var/log/ceph/ceph-client.admin.64123.log 2024-04-24T06:42:46.320 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73035.log.gz 2024-04-24T06:42:46.321 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45726.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45726.log.gz 2024-04-24T06:42:46.321 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2024-04-24T06:42:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64123.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64123.log.gz 2024-04-24T06:42:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58440.log 2024-04-24T06:42:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.47833.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.81775.log 2024-04-24T06:42:46.324 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2024-04-24T06:42:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58440.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58440.log.gz 2024-04-24T06:42:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77182.log 2024-04-24T06:42:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81775.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81775.log.gz 2024-04-24T06:42:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43686.log 2024-04-24T06:42:46.328 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.77182.log: -5 --verbose -- /var/log/ceph/ceph-client.0.41687.log 2024-04-24T06:42:46.328 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77182.log.gz 2024-04-24T06:42:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43686.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2024-04-24T06:42:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49778.log 2024-04-24T06:42:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.41687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45096.log 2024-04-24T06:42:46.332 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.41687.log.gz 2024-04-24T06:42:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60561.log 2024-04-24T06:42:46.332 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49778.log.gz 2024-04-24T06:42:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45096.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45096.log.gz 2024-04-24T06:42:46.334 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-04-24T06:42:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.60561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74534.log 2024-04-24T06:42:46.336 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45558.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.60561.log.gz 2024-04-24T06:42:46.337 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-04-24T06:42:46.337 INFO:teuthology.orchestra.run.smithi059.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39964.log 2024-04-24T06:42:46.338 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44498.log 2024-04-24T06:42:46.338 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74534.log.gz 2024-04-24T06:42:46.339 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71811.log 88.1% 2024-04-24T06:42:46.340 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39964.log.gz 2024-04-24T06:42:46.341 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76573.log 2024-04-24T06:42:46.341 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44498.log.gz 2024-04-24T06:42:46.342 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71811.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.71811.log.gz 2024-04-24T06:42:46.342 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61130.log 2024-04-24T06:42:46.343 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76573.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76573.log.gz 2024-04-24T06:42:46.344 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46709.log 2024-04-24T06:42:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74450.log 2024-04-24T06:42:46.345 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61130.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61130.log.gz 2024-04-24T06:42:46.346 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56105.log 2024-04-24T06:42:46.346 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46709.log.gz 2024-04-24T06:42:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74450.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74450.log.gz 2024-04-24T06:42:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56872.log 2024-04-24T06:42:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53951.log 2024-04-24T06:42:46.349 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56105.log.gz 2024-04-24T06:42:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40668.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56872.log.gz 2024-04-24T06:42:46.350 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.351 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.53951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78845.log 2024-04-24T06:42:46.353 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40668.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40668.log.gz 2024-04-24T06:42:46.353 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69075.log 2024-04-24T06:42:46.354 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78845.log: 93.7% -- replaced with /var/log/ceph/ceph-client.0.53951.log.gz 2024-04-24T06:42:46.356 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46481.log 2024-04-24T06:42:46.356 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69075.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.78845.log.gz 2024-04-24T06:42:46.356 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69075.log.gz 2024-04-24T06:42:46.356 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80100.log 2024-04-24T06:42:46.358 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46481.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46481.log.gz 2024-04-24T06:42:46.358 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77336.log 2024-04-24T06:42:46.359 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-04-24T06:42:46.359 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80100.log.gz 2024-04-24T06:42:46.361 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77336.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77336.log.gz 2024-04-24T06:42:46.361 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-04-24T06:42:46.362 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44016.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-04-24T06:42:46.362 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78513.log 2024-04-24T06:42:46.363 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59550.log 2024-04-24T06:42:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44206.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-04-24T06:42:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78513.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78513.log.gz 2024-04-24T06:42:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52391.log 2024-04-24T06:42:46.366 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59550.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59550.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-04-24T06:42:46.366 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73641.log 2024-04-24T06:42:46.368 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52391.log.gz 2024-04-24T06:42:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-04-24T06:42:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72264.log 2024-04-24T06:42:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73641.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73641.log.gz 2024-04-24T06:42:46.371 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73725.log 2024-04-24T06:42:46.372 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72264.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72264.log.gz 2024-04-24T06:42:46.372 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81691.log 2024-04-24T06:42:46.373 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2024-04-24T06:42:46.374 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73725.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73725.log.gz 2024-04-24T06:42:46.375 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81691.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81691.log.gz 2024-04-24T06:42:46.375 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81565.log 2024-04-24T06:42:46.375 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-24T06:42:46.376 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49734.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2024-04-24T06:42:46.376 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51566.log 2024-04-24T06:42:46.377 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60175.log 2024-04-24T06:42:46.377 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81565.log.gz 2024-04-24T06:42:46.378 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62628.log 2024-04-24T06:42:46.378 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51566.log.gz 2024-04-24T06:42:46.379 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68887.log 2024-04-24T06:42:46.380 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60175.log.gz 2024-04-24T06:42:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81649.log 2024-04-24T06:42:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62628.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62628.log.gz 2024-04-24T06:42:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68887.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68887.log.gz 2024-04-24T06:42:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81649.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81649.log.gz 2024-04-24T06:42:46.496 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T06:42:46.564 INFO:teuthology.orchestra.run.smithi059.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T06:42:46.680 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T06:42:46.840 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T06:42:46.844 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-24T06:42:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m1.556s 2024-04-24T06:42:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m4.870s 2024-04-24T06:42:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m2.454s 2024-04-24T06:42:46.846 INFO:tasks.ceph:Archiving logs... 2024-04-24T06:42:46.846 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi059/log 2024-04-24T06:42:46.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T06:42:48.162 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi066/log 2024-04-24T06:42:48.163 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T06:42:48.795 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T06:42:48.805 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T06:42:48.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T06:42:48.813 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T06:42:48.957 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T06:42:48.967 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T06:42:48.967 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:42:48.975 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:42:49.027 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:42:49.035 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:42:49.038 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:42:49.038 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-24T06:42:49.038 INFO:teuthology.orchestra.run.smithi059.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +82us[ +82us] +/- 565ms 2024-04-24T06:42:49.038 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -561us[ -570us] +/- 68ms 2024-04-24T06:42:49.038 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 +744us[ +734us] +/- 89ms 2024-04-24T06:42:49.039 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:42:49.045 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:42:49.046 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-24T06:42:49.046 INFO:teuthology.orchestra.run.smithi066.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +120us[ +120us] +/- 565ms 2024-04-24T06:42:49.046 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 -546us[ -542us] +/- 68ms 2024-04-24T06:42:49.046 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +760us[ +763us] +/- 89ms 2024-04-24T06:42:49.046 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:42:49.047 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T06:42:49.057 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T06:42:49.058 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T06:42:49.067 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T06:42:49.310 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T06:42:49.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:42:49.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:42:49.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:42:49.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:42:49.710 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T06:42:49.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T06:42:49.729 INFO:teuthology.task.internal:Duration was 1363.486733 seconds 2024-04-24T06:42:49.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T06:42:49.739 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T06:42:49.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T06:42:49.747 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T06:42:49.889 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:42:49.908 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:42:50.228 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T06:42:50.228 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-04-24T06:42:50.228 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T06:42:50.299 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-04-24T06:42:50.300 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T06:42:50.379 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T06:42:50.379 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T06:42:50.387 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T06:42:50.474 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T06:42:50.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T06:42:50.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T06:42:50.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T06:42:50.691 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T06:42:50.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T06:42:50.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T06:42:50.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T06:42:50.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:42:50.892 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:42:50.983 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-04-24T06:42:50.995 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-04-24T06:42:51.039 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:42:51.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:42:51.091 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:42:51.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:42:51.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T06:42:51.161 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T06:42:51.161 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi059 2024-04-24T06:42:51.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T06:42:51.266 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164/remote/smithi066 2024-04-24T06:42:51.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T06:42:51.368 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T06:42:51.368 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T06:42:51.377 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T06:42:51.435 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T06:42:51.446 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T06:42:51.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T06:42:51.458 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T06:42:51.458 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T06:42:51.465 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T06:42:51.516 INFO:teuthology.orchestra.run.smithi059.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 06:42 /home/ubuntu/cephtest 2024-04-24T06:42:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 06:42 /home/ubuntu/cephtest 2024-04-24T06:42:51.527 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T06:42:51.539 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T06:42:51.567 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T06:42:51.596 DEBUG:teuthology.nuke:shortname: smithi059 2024-04-24T06:42:51.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:42:51.625 DEBUG:teuthology.nuke:shortname: smithi066 2024-04-24T06:42:51.625 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:42:51.628 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:09:55.922245', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCem8jKKgS1SP/G8/wY/oFvc/9wAUb1j7OEHKfIficKsbklLvdU+qnp/nZZiqAI3qiwxNqLNR18LL0xOOMGmgTA='} 2024-04-24T06:42:51.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:09:55.923985', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeJ6fIitlL1bfgWDiHLCLXODFveauWBlpHqh8YlI3t2I0rjmRWb1wrpdsPz2n6Dkv1JwMPEIugOab5eN+zmTkc='} 2024-04-24T06:42:51.677 INFO:teuthology.orchestra.console.smithi059:Power off 2024-04-24T06:42:51.677 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T06:42:51.700 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-04-24T06:42:51.700 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:51.718 INFO:teuthology.orchestra.console.smithi066:Power off 2024-04-24T06:42:51.719 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T06:42:51.728 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-24T06:42:51.742 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-04-24T06:42:51.742 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:51.767 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-24T06:42:55.730 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:55.769 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:55.840 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-24T06:42:55.883 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-24T06:42:59.843 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:59.884 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:42:59.955 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-24T06:42:59.994 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-04-24T06:43:00.094 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-04-24T06:43:03.958 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:43:04.067 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-04-24T06:43:04.168 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-04-24T06:43:04.273 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1363.4867329597473 failure_reason: '"2024-04-24T06:36:25.066126+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: false 2024-04-24T06:43:04.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T06:43:04.384 INFO:teuthology.run:FAIL